If you and your friend both have full licenses and insurance at different company and were both set as casual drivers can you still drive his car?

Typically if you have permission to use someone else's car and that car is insured then, unless you have been previously excluded from the policy, it would be legal for you to drive and coverage would apply per the provisions of the insurance policy on your friends vehicle. Be careful though, some states allow for a reduction of the liability limits for an unlisted driver.