If a divorced woman remarries is her new husband automatically her children's stepfather?

Yes, but note that step-father does not imply any legal rights as far as guardianship, support or inheritance rights go.