First black person to go to college?

James Weber Smith admitted into the United States Military Academy in 1870

or

Richard Greener, who studied at Phillips Academy in 1865; however, became the first African-American graduate of Harvard - 1870