Did peasants build castles?

Peasants were not responsible for building castles in Medieval times. Masons and other trained professionals were in charge of making sure castles were built properly.