Is it wrong to not be attracted to people of your own race?

Of course not. Love doesn't pick people for their color.