Who was the first US President to die in office but not in the White House?

Abraham Lincoln would be the first to die in office outside of the White House.

The first two who died in office, Harrison and Taylor , died at home in the White House. Lincoln was too badly hurt to move even to a hospital and so died in a rooming house bed across the street from the theatre where he was shot.