that's how the ball bounces

Also, that's the way the ball bounces or the cookie crumbles. That is the way matters have worked out and nothing can be done about it. For example, I'm sorry you got fired but that's how the ball bounces, or They wanted a baby girl but got a third boy--that's the way the cookie crumbles. These phrases allude to an odd bounce or a crumbled cookie that cannot be put back together. [Colloquial; mid-1900s]