There might be confusion about causality here. Generally women find more success in a field that's not jammed full of highly competitive men who push anyone weaker out of the way. Where they move on to some other target, the playing field becomes more fair.

Your theory sounds more plausible to me. I'm male and find an all-male environment incredibly competitive, toxic and tiring to myself. It's very hard to get reasonable discussions and almost always degenerates into competition.

So I can imagine that you just bail if you aren't conditioned to put up with that kind of crap.

In the short term that may be true, but over a timespan of decades or generations, social attitudes about the roles of women more generally are more important. It is only very recently that the kind of constraints you mention are influential—previous generations would simply see women (or men) excluded from various domains.