The Ivy League schools (Yale, Princeton, Harvard, Brown, Rutgers, Dartmouth) were all playing some form of what they called football as early as the 1820s. A game between Rutgers and Princeton in 1869 is widely considered the first college football game. Rules were standardized in the 1870s, and the first professional teams sprang up in the 1890s.
The oldest team in the NFL is the Arizona Cardinals, who were founded in Chicago in 1898. The Cardinals and Chicago Bears both played in the NFL's inaugural 1920 season. The Colts didn't join the NFL until 1953.