How can infidelity lead to emotional mistrust?

I think from my knowledge and reading a few books, the emotional situation is the one that happens first when a married man has this new friend that is there for him. An emotional affair is all about feelings and love, but infidelity is about sex. The emotional connection is the one that ruins the marriage of a couple.