Can you only get pregnant when you're ovulating?

Technicly yes. However sperm can live in the vagina anywhere from 24 hours up to 7 days.