What did the colonists refuse to pay for tea?

Taxes. That is the textbook answer. The colonists were angry that Britain had imposed a tax on tea.

The correct answer is more complex. The Boston Tea Party actually occurred when the British Parliament ended the tax on tea. Tea merchants and other businessmen were angry that the tax had ended, making tea they had smuggled in less valuable.