Why did colonists settle near the water?

One reason would be because that is where their ship landed, and they didn't know what unknown evils may be lying in wait for them in the interior. Another big reason would be that food, in the form of fishing and digging for clams, etc., was known to exist on the shoreline, but they wouldn't necessarily know how to get food if they ventured further inland.