So your theory is that, to avoid inbreeding, we developed a "KILL KILL KILL" response to differences and diversity. Somehow, I don't think that that makes sense, unless you're thinking of it as a way to get societies to reject people born as a result of inbreeding. Neanderthal predation is a much better explanation, at least in my opinion: a intrinsic fear of things that look very like us but different because, not so long ago, things very similar to us were trying to kill and rape[1] us.

[1] Ellyagg mentions a study that showed that humans have some male neanderthal DNA, but not female neanderthal DNA. This, to me, suggests rape. http://news.ycombinator.com/item?id=2688389