It's just the name of the area the academy was built on.
The United States Military Academy was built at an existing U.S. Army base - West Point. That's where it got its name. West Point was the first of the American Army bases built after the Declaration of Independence was signed, and Benedict Arnold was its commander. Wikipedia has a post, and a link is provided.
The Hellcats which are a part of The West Point band which is comprised of Army enlisted musicians.
Both were regular army officers trained at West Point.
The Winkies.
The name of the US army military base in Swinefort Germany is called the Army Garrison Schweinfurt.
Fort Riley ???
Fort Sill.
Fort Rucker and the Redstone Arsenal.
Fort Lewis
base
Grant's real name was Hiram Ulysses Grant. However, when he was nominated for West Point, the Congressman who nominated him, wrote his name as Ulysses S. Grant, presumably thinking his middle name was Simpson, which was his mother's maiden name. He never corrected his name with West Point, was commissioned as Ulysses S.Grant and that became his official army name. Maybe he did not like is old initials of HUG.
The United States Military Academy at West Point (or simply the Academy or the Point) is located, much as the name might suggest, in the city of West Point, New York.