Would it be arrogant to wear a shirt that says I am a Brony deal with it?

Yes if you were "Brony" otherwise just odd.