Do you think that it is right I take some people's word about a place and never visit there?

I think there is a lot of truth in what you say, but even racists (at least in Europe) do not generally express racial hatred to people who are obviously tourists. It is more about not wanting other races to become integrated into their society, which is very ignorant and uncivilized of them. But I am sure you would not have a problem in any of the major tourist places in Italy.