Are there other penalties for paying taxes late?

Yes, it starts out at 1/2 of one percent per month then increases to 1 per cent per month at some point. Interest is charged as well.