Firefox's extension system was powerful but there was next to no chance that XUL based UI was ever going to be adopted in the other major browsers. I'm sure that tapping in to the Chrome extension ecosystem was a big factor in the decision but I would argue that the Chrome extension API was also a far better suited candidate for cross browser standardization.
Firefox's extension system was arguably too powerful. Back when I ran Firefox, I regularly ran into extensions that conflicted because they monkeypatched the same parts of the browser UI.
