Were there more Loyalists or Patriots?

At first they were even but eventually the patriots grew larger