Chess today is a recognized sport of the International Olympic Committee. The first official World Chess Champion, Wilhelm Steinitz, claimed his title in 1886; the current World Champion is Viswanathan Anand.
Yes. The IOC approves and look at curling! Is curling very physical? No, it's a skill game that uses the mind, just like chess. It has been said that curling is "chess on ice". Chess is a sport despite the many whom like to think otherwise.
Chess is a game contested between two combatants at a table similar to Arm Wrestling - where technique, strategy, celerity, strength, development, intellect, insight, resilience, intrepidity, courage, grit, fortitude, sturdiness and forcefulness, etc are at play - where the combat is defined in a physical manner for arm wrestling, but the combat is defined in a psychological manner for chess.
With sport being defined in terms of physical exertion, Arm Wrestling is uncompromisingly a representation of a sport; but with the physical exertion being unapproachable in comparison, Chess is a representation of a game.