Are the first build castles or stone fortifications ever built by Africans?

The first stone castles or fortifications, were built in Medieval England while William the Conqueror ruled. They were not built by Africans but by English peasants.