How did the Muslims govern their vast territories?

Answer 1

As the empire expanded from being only Arabia, the political organization of Islam changed from being one governed by tribal codes and ethics to one with an imperial capital, provinces, and ostentatious royalty. Democracy and meritocracy were replaced with monarchy. A bureaucracy was developed in Damascus using former Sassanian and Byzantine talents. This is rather unique because in most empires, a conquered population was kept out of the ruling apparatus. However, in the Arab Empires, there was a strong intent to integrate the conquered populations and many in the territories wished to become Muslim due either to conviction or to financial incentive. The Maghreb, Spain, and Afghanistan were militarily annexed and there were expeditions (military) sent as far as Xinjiang Province (East Turkestan) is China. The empire was held together by institutions such as the governates, roads, and police/soldiers, just like every other empire. It just happens that track record of violence and repression is better for the Arab Empire than contemporaneous states.

Answer 2

The Umayyads ruled by appointing local governors waliyinin each province or wilaya that would organize things locally and send the information to the Caliph in Damascus at reasonable intervals.