Does your partial downpayment have to be returned to you if you never paid the balance and the dealer repossessed the car a few weeks after purchase?

NOT unless the contract stipulates that it will be. Otherwise, it is a contract in DEFAULT with the collateral in the lenders possession.