How do you know when you should end a relationship?

When the voice in your head tells you to leave. This is usually acurate. If you are asking the question, that alone should tell you there is a problem.