The Navy considers all vessels ships; however, submarines are historically referred to as boats due to the nature of the first submarines. A boat in Naval terminology is a vessel that is launched or tended from a larger ship. The earliest submarines required support vessels to maintain and launch them, hence they were termed boats.
The term "boat" goes back to the earliest days of submarine history, when submarines were literally launched from tenders (e.g., the Turtle). A 'boat" is launched from a ship - therefore, the earliest submarines were termed "boats" since they were launched and retrieved from ships.
Today, though only DSRV's and ROV's require a support vessel for operations, independent submarines are still referred to as boats not only due to the historical term, but the fact that they still require support from a Submarine Tender while in homeport for repairs and maintenance the crew cannot perform.
As Naval history and submarine technology progressed to the point where they became independent vessels unto themselves, the traditional name stuck, though the Navy itself doesn't make a distinction - a ship is ship even if it's a black sewer pipe with a nuclear reactor that travels underwater. It's not just the tradition though - all submariners are keen on distinguishing themselves from skimmers (surface ship types), and using "boat" rather than ship is one way of doing that.
Another part of that tradition is the business founded to create famed submarine builder John Holland's submarine USS Holland (SS-1), which still exists today, which has had a large part in creating submarine history as we know it - the Electric Boat Division of General Dynamics Corp., in Groton, CT.
Submariners refer to submarines as "boats" or simply as submarines.
As a former submariner, I do not refer to them as "ships" at all, only boats or submarines. There is an old saying in the submarine community:
"There are only two types of ships: submarines, and targets!"