Can you get to Madagascar with a passport?

Yes. You will also need a visa, but they are supposed to be available after you arrive.