Does astigmatism cause the eyes to move side to side and if not what does?

Not normally. Most people have a small degree of astigmatism, and even quite large degrees of it need not be associated with the eyes wandering from side to side: "nystagmus"

Nystagmus in the young can occur with a wide range of eye disorders of childhood such as cataract, glaucoma, some disorders of the retina and albinism. It may also be found in children who have multiple disabilities such as Down's syndrome, though it is quite possible for no clear cause to be found.

Poor vision with very severe uncorrected astigmatism, could be a factor in some cases.

Nystagmus appearing later in life may be a symptom of another condition such as stroke, multiple sclerosis or even a blow to the head. There are many causes.