A fence is over your property line did you lose this property?

Maybe. In certain situations, if your state has the doctrine of mutual acquiescence, you could lose the land. You should contact a real estate attorney in your area for specific guidance on your state's laws and your particular situation.