Should you use your British accent in front of other people even if you're American-born?

Answer: YESS! Long Answer: If you have a family which has a British sounding accent and you have picked it up even though you were not born from the native country, you should be proud to have it.
It is a rare position to be born in a country and have an accent different to many other people. Try to think of it as a unique thing than "against the norm".