Why doesn't a soccer goalie have to wear a glove?

Why should a goalkeeper have to wear gloves in football? Gloves are only worn for padding and gripping reasons, so it is the choice of the keeper as to whether he prefers the grip of the gloves and whether he feels he needs padding. The only real argument for goalies to have to wear gloves is that if there is a packed penalty area, and the referee sees a hand touch the ball, it is easier in that split second to see if it was the goalkeeper or whether someone comitted hand ball.