Yes, the word "Abbey" should be capitalized when referring to a specific abbey or using it as part of a proper noun, such as "Westminster Abbey" or "Abbey Road."
View page
You can say "Je m'appelle Abbey" in French, which translates to "My name is Abbey" in English.