What can I do to a contractor who didn't finish his job and now I am in default with zoning?

If you didn't have a written contract with the contractor then there is nothing you can do because he is not entitled to finish the job if it's not in writing.