No, the other way around. Higher testosterone levels will increase sperm.
A differing view is that testosterone is absorbed by some soft tissues in the body. it is a fact that women do get a boost in Testosterone from semen during sex, They also pick up testosterone from deep (french) kissing a man.
So if that is so & it is, Semen in the mouth will allow for the absorption of Testosterone. Swallowing, probably not, the acid in the stomach will break down the Testosterone.