How do you know that you are not the one who is crazy or PMS'ing and that he is really emotionally abusive?

I think that you answered your own question by saying that for a while you believed it. Abusers manipulate their partners into believing that they are the one at fault.
You are being abused, if:
(1) He repeats a certain behavior
(2) You asked him to stop (for whatever reason).
(3) He refuses and continues to behave the way he has.
You may well be abusing him - but that does not mean that he is not being abusive towards you. Both parties are sometimes abusive towards each other.
Please read the related link below.
people who are abusers rarely consider that they might be abusive. even if the stresses of the relationship lead into what might be considered reactive abuse, anyone who honestly tries to adjust to the other person's actual needs, actively listens to the other person, and makes every attempt to stop such behavior, probably is not an abuser. abusers do not take responsibility for their own actions, and in fact often blame the abused. when the abused person reacts to the abuse, the abuser calls that reaction abuse, and will use guilt to try to get the abused to feel responsible for the arguments or difficulties, as well as for the abuser's actions. this is one of the reasons getting away from an abuser is so important. everything clarifies then.