That's all true but I don't think it explains the effect GP is talking about.
Security aside, I think it just comes down to what the business incentives are for different approaches to software development. To develop good software you need small teams of experts with ___domain knowledge and very strong programming skills. But good software isn't the only kind of profitable software.
A lot of big businesses are working with multipliers on the benefit side that eclipse the factors on the cost side. If you don't have a particular piece of software 'X', where X is something that would generate revenue multiplied across a really large customer base, then almost any strategy that produces X is going to be a strong one. From a business perspective it's not going to make any difference if you hire a 5-man black team to pump out the perfect bit of software or 150 cheap consultants to hack together a giant pile of shit. The former might make you $150M instead of $120M, but nobody is going to have stats on the path not taken to compare to in retrospect, so there's roughly zero market pressure to develop software properly.
Then there's two factors that explain why businesses don't take the better approach. The first is that every big business I've worked for has had layers of management and departments, where budgets are allocated artificially and aren't perfectly tied to results like they would in a pure software business with flat management that's reinvesting profits into their flagship product. That creates an environment where the objectives for middle management aren't perfectly tied to the objectives for the business (e.g. increasing their team's head count at the expense of their software).
The second is that having a larger team mitigates risk. If you put together a small team of crack developers to build a perfect product there's somewhat of a chance that they fail. Things like turnover and personal circumstances affect you a lot more. The swarm of average contractors doesn't have as big of an upside, but they're much more likely to get you to 80% of the way there, so if that's indistinguishable to a perfect solution (as above) then there's this constant natural pressure to overhire and reduce the quality of the team to the lowest common denominator. Big businesses love treating their devs as cogs, even if all the literature on the engineering side suggests that that approach doesn't work.
Basically, I don't think it's just a case of 'managers are idiots and don't know how to build good software', businesses build crap software because it works and it's profitable. But it also creates constant opportunities for startups to eat their lunch, and the worse the economy gets, the better your software has to be (because the cost side of the equation starts to matter more).
Security aside, I think it just comes down to what the business incentives are for different approaches to software development. To develop good software you need small teams of experts with ___domain knowledge and very strong programming skills. But good software isn't the only kind of profitable software.
A lot of big businesses are working with multipliers on the benefit side that eclipse the factors on the cost side. If you don't have a particular piece of software 'X', where X is something that would generate revenue multiplied across a really large customer base, then almost any strategy that produces X is going to be a strong one. From a business perspective it's not going to make any difference if you hire a 5-man black team to pump out the perfect bit of software or 150 cheap consultants to hack together a giant pile of shit. The former might make you $150M instead of $120M, but nobody is going to have stats on the path not taken to compare to in retrospect, so there's roughly zero market pressure to develop software properly.
Then there's two factors that explain why businesses don't take the better approach. The first is that every big business I've worked for has had layers of management and departments, where budgets are allocated artificially and aren't perfectly tied to results like they would in a pure software business with flat management that's reinvesting profits into their flagship product. That creates an environment where the objectives for middle management aren't perfectly tied to the objectives for the business (e.g. increasing their team's head count at the expense of their software).
The second is that having a larger team mitigates risk. If you put together a small team of crack developers to build a perfect product there's somewhat of a chance that they fail. Things like turnover and personal circumstances affect you a lot more. The swarm of average contractors doesn't have as big of an upside, but they're much more likely to get you to 80% of the way there, so if that's indistinguishable to a perfect solution (as above) then there's this constant natural pressure to overhire and reduce the quality of the team to the lowest common denominator. Big businesses love treating their devs as cogs, even if all the literature on the engineering side suggests that that approach doesn't work.
Basically, I don't think it's just a case of 'managers are idiots and don't know how to build good software', businesses build crap software because it works and it's profitable. But it also creates constant opportunities for startups to eat their lunch, and the worse the economy gets, the better your software has to be (because the cost side of the equation starts to matter more).