Where was the first presidential inauguration held?

The first presidential inauguration was in New York City, which was the first US capital.