Can a finance company cancel the loan if the signer doesn't drive the car?

No. The loan was made and the person who made it owes for the loan. To take this one step further if the car was wrecked the loan is still owed. So, it doesn't matter who drives the car.