If you were never married to the father of your child and there is not a court order for visitation would a judge remove the child from your custody if you move out of state?

No. If there is not a custodial order in place the law presumes that an unmarried mother has full custodial rights to her child and does not need permission from the biological father or the court in matters concerning the child. In addition, when a child is born out-of-wedlock the biological father must establish paternity before custodial, visitation and child support can be addressed.