Why are eggs associated with the Easter bunny?
From pagan religion, both the egg and the bunny are symbols of fertility. In the pagan religion, Easter was the "spring festival", celebrating fertility and requesting that the Gods give good crops. It makes sense that the rabbit and the egg would be symbols of spring festival.
Easter is also a symbol of the Anglo-Saxon goddess, Eostre, as the holiday was originally named after. Easter is a relatively new adoption of spelling which was used, as said above, to help make the coexistence and/or transition of religions go much smoother.