Death or impeachment. They can resign, but I don't think this has ever happened.
Nixon resigned his office and Ford took over the presidency.
It could be from death as an example.
The actual term of office as stated in the US Constitution, Article III, is that they shall serve "during good behavior." This means that they serve for life but may be impeached and removed from office if convicted of the same offenses that the President may be impeached and removed for.
Recall
John Kennedy and Franklin Roosevelt had well-documented love affairs while they were President. Woodrow Wilson had a very close female friend that he wrote to, but may have been faithful to his wife in the strict sense. Other presidents had affairs but I think they were all broken off by the time they were President.
The president, vice-president and federal judges can be removed from office for committing crimes. First the person must be officially charged with a crime or crimes, this is called impeachment, and then the person must stand trial in the Senate where a conviction requires a 2/3 vote. Members of Congress, however, are not subject to this process.
Only three U.S. presidents have been formally impeached by Congress: Andrew Johnson, Bill Clinton and Donald Trump. So far, no U.S. president has ever been removed from office through impeachment. In addition to Johnson, Clinton and Trump, only one other U.S. president has faced formal impeachment inquiries in the House of Representatives: Richard Nixon.
One promise you make when you become president is to uphold the Constitution. Presidents who fail to do so may find themselves removed from office.
Impeachment
The actual term of office as stated in the US Constitution, Article III, is that they shall serve "during good behavior." This means that they serve for life but may be impeached and removed from office if convicted of the same offenses that the President may be impeached and removed for.
they are removed from office
As of May, 2012 there have been 11 U. S. Presidents since Harry Truman.
You may be thinking of Gray Davis, who was recalled from office by popular vote about a decade ago.
Recall
He could die, voluntarily resign or be removed from office via the impeachment process. He could temporarily be removed from office because of an inability to perform his duties.
The Constitution states that Justices "shall hold their Offices during good Behaviour." This means that the Justices hold office as long as they choose and can only be removed from office by impeachment. Source: The Supreme Court website.
crimes and misdemeanors
According to the Constitution, the President, Vice President, and all civil officers of the United States can be impeached and removed only for treason, bribery, or other high crimes and misdemeanors. Since the language is so broad, you could use many offenses to logically impeach elected officials. The term "impeach" does not in and of itself mean to remove someone from office. Impeach means to bring impeachment proceedings against them. For example, two presidents have been impeached: Andrew Johnson in 1868 for violating the Tenure of Office Act; and Bill Clinton in 1998 for lying to a Grand Jury (perjury). Both presidents were acquitted of the charges, and neither were removed from office. Contrary to popular belief, Richard Nixon was never impeached. He resigned the office of president before the House could impeach him. "The President, Vice President, and all civil officers of the United States" who may be impeached and removed only for "treason, bribery, or other high crimes and misdemeanors".[14] Several commentators have suggested that Congress alone may decide for itself what constitutes a "high crime or misdemeanor",
Andrew Johnson was impeached in 1868 for violating the Tenure of Office Act. Bill Clinton was impeached in 1998 for perjury and obstruction of justice. Note that Impeachment does not mean removal from office, it means that he is being investigated on suspicion of wrongdoing that may result in him being removed from office. In both of the above cases the President remained in office. Richard Nixon was not impeached, he resigned before the impeachment proceedings being considered agaisnt him could begin.