Winter and Spring (July to September) is cold, very windy and wet. This means that it is a good time to visit the city if you are feeling particularly anti-social and do not want to see people on the beaches or on the mountain.
However, as a tourist, the best times are from October to May. Cape Town has a mild climate with a relatively dry summer and long beautiful summer days. The weather during the Cape summer is wonderful.
Note that tourist numbers peak in December and January, causing congestion and higher prices, so if possible avoid that time period.