Has an incumbent president ever lost the nomination to run for a second term?
Four incumbent presidents have been denied a nomination to run by their own party. Franklin Pierce, Millard Fillmore, Andrew Johnson (sort of), and Chester A. Arthur; but only Pierce had actually been elected president. The rest were vice presidents who who ascended to the presidency after assassinations or deaths in office.
Pierce was our 14th President and served from 1853-1857. John Fremont defeated Pierce in his bid for renomination, making Pierce the only elected President (rather than a Vice President who succeeded to the position) to not gain his party's nomination for a second term.
None of the other three presidents had secured enough support in their own party to gain a nomination for a full term. Millard Fillmore, who took over when Zachary Taylor died in 1850, did not gain the Whig party nomination for a second term.
Andrew Johson, who took over after Lincoln's assassination, had a more complicated situation. Johnson had been elected as Lincoln's running mate as a "Union Democrat". Lincoln had hoped to unite the country with a southerner on the ticket during the Civil War in the election of 1864. But after Lincoln was assassinated and he ascended to the presidency, he faced a Republican congress that despised him. After being impeached -- and narrowly avoiding removal from office -- he attempted to win the Democratic nomination in 1868. Since several Confederate states hadn't yet rejoined the union to vote in that election, and he was not well-liked by northern Democrats either, he failed to win support from either major party. After he stepped down as president, he briefly served in the U.S. Senate again from Tennessee.
And finally, Chester A. Arthur, who took over upon Garfield's assassination, did not gain his party's nomination for a second term.
It's worth noting that all of these occasions occurred before the modern era of party primaries being decided by voters. These presidents had to win the approval of their own party officials, not the public at large. The closest scenario in the primary era was Lyndon Johnson in 1968, who ran for a second full term, but dropped out of the race after he only narrowly won the New Hampshire primary against Eugene McCarthy. This was considered a sign of a major weakness for an incumbent president. After Robert Kennedy joined the race, it became clear to Johnson that he would not win the nomination. On March 31, 1968, he announced he wouldn't run for another term.