Yes, the word "Province" should have a capital letter when it is used as a proper noun to refer to a specific province. For example, "British Columbia is a province in Canada." However, when used generically to refer to a province in a general sense, it does not require a capital letter. For example, "I am from a small province in China."

Mary Janice Ziolkowski has written:

'The Felician Sisters of Livonia, Michigan' -- subject(s): Felician Sisters, Felician Sisters. Presentation of the Blessed Virgin Mary Province, History

Mered Atakhanov was born on January 1, 1914, in Godus, Mary province, Russia [now Turkmenistan].

Mary Clarence Friesenhahn has written:

'Catholic secondary education in the province of San Antonio' -- subject(s): Catholic Church, Education, History, Secondary Education