Was it common in the Great Depression for women to leave their babies in public places?

No, not common, but it did happen occasionally. In fact, it still happens, but very rarely.