Brexit was absolutely about Economic conditions. Anti immigration sentiment was one big factor that everyone talks about, but in reality the sentiment itself was a symptom on display for the underlying condition of people having their living standards drop. The second big factor people kept bringing up was that EU was making decisions for UK, which people thought needed to be reversed. Austerity measures, which EU adopted, were the single biggest reason why people felt they needed more control. They also slowed down the post 2008 recovery, which meant people weren’t doing as well anymore. Anti immigration sentiment also rose once people were unhappy and needed someone to take the blame.
When my boomer dad complains about the Pakistani family he went to sell something to having 2 BMWs it's definitely racist.
But the racism is inflamed by poor economic conditions, and the rationale is "how come these people that aren't even from here have a better life than me?"
Sadly, people seeking power know this and use the racism to get power and don't fix the underlying cause, which was never immigration — it was wealth inequality.
If they were of the same race as your dad, the question "how come these people that aren't even from here have a better life than me?" would still be valid.
If he can tell they're not from there, then he'll be mad. If he thinks they are fellow Englishmen he'll be less mad. That's the point, how prejudice often has a ground in economic conditions