Death or impeachment. They can resign, but I don't think this has ever happened.
Nixon resigned his office and Ford took over the presidency.
Chat with our AI personalities
The actual term of office as stated in the US Constitution, Article III, is that they shall serve "during good behavior." This means that they serve for life but may be impeached and removed from office if convicted of the same offenses that the President may be impeached and removed for.
Recall
John Kennedy and Franklin Roosevelt had well-documented love affairs while they were President. Woodrow Wilson had a very close female friend that he wrote to, but may have been faithful to his wife in the strict sense. Other presidents had affairs but I think they were all broken off by the time they were President.
The president, vice-president and federal judges can be removed from office for committing crimes. First the person must be officially charged with a crime or crimes, this is called impeachment, and then the person must stand trial in the Senate where a conviction requires a 2/3 vote. Members of Congress, however, are not subject to this process.
There have been two U.S. presidents who chose not to take a salary during their time in office. The first was George Washington, who declined a salary for his entire presidency. The second was Herbert Hoover, who also refused to accept a salary during his term in office. Both presidents were independently wealthy and felt that they did not need the salary for their service to the country.