The year, make and model vehicle would help and there shouldn't be just water, it should be coolant.
That device is called the refill tube. This allows water to go down the overflow tube to refill the water closet bowl while the tank is filling.
To avoid any overflow
An overflow tank is a bottle that holds coolant that expands and leaves the radiator under driving conditions. The Coolant will go back into the radiator as needed.
An overflow is a box that hangs on the back of the tank (or is sometimes built in) and allows the water to "overflow" to a wet/dry filter (the tank sized filters people have underneath their tanks). The overflow has a U shaped tube that allows water to siphon from the main tank into the overflow box. The overflow box then drains the water down a tube to the wet/dry filter continuing the siphon by pulling the water as it drains. If you are going to purchase on, they should come with instructions. www.wetwebmedia.com is a good place to learn about all types of fish related things.
A gas tank can indeed overflow if it is not full. A gas tank can overflow if the pressure inside of it is too high.
if your water/antifreeze gets to hot it pushes out the over flow tube into the overflow container, as it cools down it will suck it out of the overflow and back into the radiator.
Make sure the water tank is low enough to take on new water or you might have an overflow problem.
If it is coming from the tank: NO. If it coming from the bowl: YES!
At the radiator or overflow tank. Do not add water, add coolant.
Replace the pressure relief valve.
the water overflow tank
This is probably an overflow pipe from the cold water tank in the loft.