I don't think it is as OK as you think, however anti-racist
people have many friends that are racist so sometimes rather than
argue / fight with friends they keep their anti-racist opinions to
themselves. I think you may find that there are also a lot of
anti-racist people out there, but unfortunately the racist "voice"
seems to be the loudest.