This is a bit of a complicated answer because Carcassonne has
been occupied since the 6th century BCE by different groups who all
made contributions to building and rebuilding the castle and the
city surrounding it.
Carcassonne is in southwestern France first occupied by the
Celts. In the 1st century BCE, the Latin group, Volcae Tectosages,
fortified the fort and its building. In the 3rd and 4th centuries
CE, the Romans created more fortifications, including walls and the
beginnings of the castle. Ruins of these Roman defenses are still
visible today. Throughout the 5th through 11th centuries CE, it was
ruled by Visigoths and Arabs. It became the seat of a count, and
then a viscount in the 9th century. A proper castle was built
sometime in the 12th century and within the 13th and 14th
centuries, the town gained more citizens, becoming a normal
medieval fortress. The castle was improved upon and fortified until
the 18th century, when the architect Eugene-Emmanuel Viollet-de-Duc
was commissioned to restore the castle. The restoration began with
his commission in 1846 and was finished in 1910, some 30 years
after Viollet-de-Duc's death.