Can the parent without primary physical residency take the child out of state on vacation without the custodial parents permission?

This depends upon the law in your state. In my state it is allowed unless the divorce decree prohibits it (rare). Otherwise, a noncustodial parent would be stuck vacationing in the same state for many years. I would notify the other parent in writing as early as possible prior to the vacation, if possible, and keep a copy of the email or letter you send.