Shouldn't a driver with no insurance be at fault when they have an accident?

No, but they will be liable for a citation if the police are called.

Fault for the collision is based upon negligence. This can be founded upon carelessness (i.e. not yielding right of way), or violating a traffic regulation.

Not having insurance does not equate with fault for collision. However, if the law if the jurisdiction requires one or more types of auto insurance, which it likely does, a citation will probably issued for that.

California Vehicle Code Section3333.4 States:

If you have no liability insurance covering you for the operation of the vehicle that you were driving at the time of an accident, you get no money, zero compensation or your pain and suffering. Not a penny more than your reasonable medical bills. Period. End of story. (The only exception to this hard and fast rule is if the at-fault driver was convicted of driving under the influence pursuant to California Vehicle Code Section 23152 or 23153).

To add insult to injury, driving without insurance will subject you to having your car impounded, temporarily losing your driver's license and paying a hefty fine. See California Vehicle Code Section 16029 for further details.

Bottom line: If you're going to drive in California, get auto insurance.