Where does the expression cloth head come from?

I hate to say this at the rsk of sounding a bit racist but i think cloth head refers to middle eastern people i.e Indian Sikhs because they wear turbans