The simplest answer is the following: the pilgrims were in fact Puritans. The Puritans (a religious sect which Oliver Cromwell belonged to) made their way to the New World as a way to escape religious persecution in England. The New World offered them the chance of establishing a colony where they could worship freely. The pilgrims were not the first Europeans to arrive in the New World. The Spanish settled St. Augustine much earlier and the Vikings likely were in the Newfoundland/Labrador area in the 11th Century.
Note: myths and legends abound about where the pilgrims first landed. Typically, it is assumed they landed at Plymouth Rock; however, this may in fact not be the case.