It's anecdotal for sure (x race is scared/uncomfortable of y race). I've heard it from different people from different races. It's not unique to white people.