Why say it's toast?

When we say something is toast, we are implying that it is ruined, like burned toast.