Why is the term brick building improper?

It isn't improper if it is a brick building. But a wood building with a brick veneer is not a brick building. Many people speak of a house as being a brick house, but it is probably a frame house with brick veneer. Perhaps that is what the question is about.