Why did the coat of arms originate?

Answer 1

Because one guy in a metal suit looks a lot like every other guy in a metal suit.

Coats of arms originated as, basically, jersey numbers for knights. It helps, in a battle of more than two people, to know which particular ones are your opponents and which are your allies. Painting your shield a distinctive color or combination of colors, or painting a design on it, helps your friends know who you are so that they can tell you're on their side and they shouldn't be trying to stick pointy bits of metal into you.

Remembering several hundred individual symbols could get complicated, so families tended to stick with the same design for generations. Kids would use dad's shield with a special additional design on it (called a mark of cadency) to show they were the Number 1 son, the Number 2 son, etc of the guy whose basic design they were using. When dad died, the number 1 son would get his design, and sons 2 through whatever would have to come up with their own design (maybe by using the same basic design with a palette swap), or by partitioning it with their father-in-law's design.