That doesn't have to do with any idom, people can agree on
anything they want to.
Things that people can agree on means things we all think are
true, such as scientific facts. You also hear this used when the
speaker is hoping that people will agree with him, as in "I think
we can all agree on this!"