Who is the antagonist in Harry Potter and the Prisoner of Azkaban?

For most of the book, the antagonist is made out to be Sirius Black, who is believed to be a mass murderer. However, when Sirius is proved innocent and the true murderer is revealed (Peter Pettigrew) the antagonist switches to Peter.

The Dementors are also types of antagonists as they feed off of the happiness in the atmosphere and they don't care whether they eat away at the soul of an innocent person.