Why does a B represent a flat symbol in music?

It isn't a letter "b," it's a flat symbol. Likewise, the sharp isn't a hashtag.