Probably not, you might get sicker. Although cold weather has nothing to do with catching a "cold", they are caused by viruses which can live on door knobs, telephones, computers, etc. for up to three days. In the winter we are inside more and breathe the air of people who are coughing and sneezing, increasing our exposure to viral particles in the air. In the summer we are outside more and so we come into close contact with infected people less often. There is some evidence that getting wet and cold may lower our immunity slightly. If you want to swim in the winter, just make sure you dry off thoroughly before going out into the cold, and wash your hands frequently to avoid picking up wiruses form the world around you.