Do you need a wireless connection if you have a wireless router?

Not in most cases. Most wireless (or WiFi) routers also have an ethernet RJ45 port and you can connect that way with a CAT5 cable....even connect a switch for more ports.