Is he too old for me if I am 18 and he's 24?

No, not if you feel it's right and that you both feel good about it there is no reason why (only) 6 years should affect that. Age doesn't matter. Hope it goes well.