There is no clear origin to this phrase. In the 1700's, it was
just used as "burning your pocket," but by the 1800's it had become
"burning a hole in your pocket." The implication is that the coins
are so hot that you jerk them out of your pocket and throw them
away.