Why is throwing money away illegal?

Technically speaking, it does not belong to you. The ink and the paper belong to the federal government. It is a system that was set up by governments worldwide so that by possessing bills or coinage you could demonstrate that you have 'X-amount' of purchasing power, but the the bills and coins themselves (which have little or no intrinsic value) actually belong to the government.

Additional: Same thopry applies to your drivers license and vehicle registration - you do not 'own' them either. Those pieces of paper actually belong to the state and are issued to you only to indicate that the state officially authorizes you whatever usage they indicate.