Is a US military base abroad considered American soil?

No. The only places in other countries that are considered "American soil" are embassies and consulates. Military bases are generally on land leased by the host country.

Correct. Some people claim that it is American soil because children born there are American citizens. While this is true, this is not because the base is considered part of the US, rather, it's because the US's Immigration and Naturalization Act, long story short, says that children of American citizens are American citizens too. They will be issued a "certificate of birth abroad" by the US embassy/consulate near that particular base, which is the normal procedure in such a case.