Who can access a mail server?

Anyone who has the appropriate permissions.