When you are in what you think is a casual relationship - What does he mean when he says you would fall for me or you've fallen for me?
He is suggesting that you would or have fallen in love with him. If this is just a casual relationship you should reinforce that with him so he knows and is not expecting or thinking more. (unless of course you want more)
If your girlfriend does not want a serious relationship at the moment, then you need to decide what you want. If you are okay with a casual relationship then you should stay with her. But if it will only hurt you to not be in a more serious relationship, then perhaps it would be best for you to move on and find someone that wants the same things out of a relationship that you do.
What would a narcissist say to one if you tell them that you only want a casual relationship with them?
Most likely they wouldn't want a relationship with you if they couldn't benefit from it whether casual or not. If they feel they can benefit and manipulate you out of something, then the game playing begins. Be aware, they will immediately manipulate ways to get whatever they want and then will disappear after receiving it. Again, narcissist cannot be casual in any setting, they have to be the winners or they want nothing at all…