I think we agree. But the car is a much bigger part of it than you're letting on.
The reason we zone this way is because people are unable to imagine their lives without being able to drive around. The very first thing people say in my town when an apartment tower goes up is "where am I going to park? Do they have a plan for moving the parking spots that this tower displaced? Traffic on tower St. is going to get so much worse!"
If the expectation is that everyone owns a car, upzoning is a huge problem -- it actually is, because everything that scares a car brain ("I can't park!" "It's going to be loud!" "This will increase traffic!") is actually true. Rather than limiting cars, of course, we choose to limit zoning. Which is actually quite a logical thing to do if everyone is to own a car.
If you get rid of cars you get rid of collectors, arterials, onramps and offramps, turn lanes, parking spots... Those are things that are fundamentally incompatible with a major, dense, vibrant city. You can easily fit 50,000 people, their workplaces, senior care homes, schools, and restaurants into the land area of a 4 way cloverleaf that is designed to service 50,000 round trip commutes by car.
It's not even the car that is the problem. It's the lack of viable alternatives to the car. In my city in North America there are almost 1.5 million people, I live about 4 kms from the city center and it's extremely uncomfortable without a car, even though I live on multiple bus and one LRT line within ten minutes of walking. I purposefully bought and built where I am because of all this connectivity, maybe 5 % of the whole city has it as good as I do, the rest have it worse to much worse. I basically live in the best situation in the city for alternatives and it's worse than the crappiest parts of the city for alternative sin places like the Netherlands.
To make alternatives you can look at what the Netherlands does. They have much more relaxed lot coverage, height limits, parking requirements, etc. so they get the same single family square footage in way less geographic area and they intermix that with much higher density so their cities proper are 3-4 times denser than where I live while not feeling cramped. They didn't get rid of the car, what they did do was ensure people had great walking, biking and transit options to fulfill the normal trips that make up their lives. That is vs where I live where I am fine getting downtown with transit, but to my kids schools, groceries, or any other workplace outside of the core, I am screwed. The extra density makes providing these transit options possible (along with reasonable regulation on other areas leading to reasonable density)
But in the US, at least, the underlying argument is about class, and race. If you haven't already, it might be worth reading about redlining, the historical practice of outlining which neighborhoods were "safe" to issue mortgage loans in because they were white and not at risk of non-whites moving in. This was about race, primarily.
Redlining became illegal starting in the '70s in many places, and its replacement was exclusionary zoning---a practice that continues to this day that relies instead on race's strong correlate, wealth. Apartments and other similar higher-density arrangements are much more affordable--and as a result, accessible to racial minorities (and the crime, etc. that racial minorities were "associated" with).
All the modern arguments around traffic jams, neighborhood character, transit, etc.---these are all modern-day variants of the same old arguments, because again, who can afford a 1:1 ratio of cars to people-in-household to get around the low-density suburbs?
So when you hear people say "I can't park! It'll be loud!" consider if they're really saying "I don't want those other people who rely on transit in my neighborhood", but using the words "there won't be room for me to park!"
The idea that if you remove cars from 90% of the population that you won't sometimes need to drive something into these areas is laughable. Even Tokyo has roads.
Tokyo has roads, but household car ownership is 41% prefecture wide; that's well under 1car:1person. In inner city, it is probably close or above 90%. That inner city traffic is pretty much taxis and commercial vehicles. The same is true of any place which is properly built-up with options to get around. Manhattan is another good example: 20% car ownership. In a 2 person household, that's your 90% number!
Building subways is impossible for no good reason nowadays. But it's quite easy to put light rail, bike lanes, and BRT in. People find a way to get around.
You don't need to take cars by force; you just need to let market forces do it. That starts with getting rid of parking, setbacks, and exclusionary zoning requirements for new development in areas of town that are underdeveloped (it's not hard to find these areas in even medium sized American cities). If people really do want cars in those built-up areas, they will rent a monthly space. It's a funny thing though, people seem to re-evaluate how valuable a car is once they have to pay $400/mo for a parking space.
If what you say is true, and everyone truly must have a car, rents will fall in those areas to compensate (developers, not having to deal with onerous parking and setback requirements, still turn a profit). In practice, this does not happen, because places without cars are nice places to live. They appreciate and attract investment, which puts money in the city coffer to improve transit options.
It does not take much land to do this. An area the size of 10,000 single family homes -- often a single neighborhood. In San Diego that's Midway, in SF it's Berkeley and the Sunset, in NYC it's Staten Island. These places all have developers who will break ground in months on collectively hundreds of thousands of new housing units if they are allowed to do it and earn a profit.
It is not fundamentally hard to do this, at all, if you can picture in your mind a human being not owning a car.
The reason we zone this way is because people are unable to imagine their lives without being able to drive around. The very first thing people say in my town when an apartment tower goes up is "where am I going to park? Do they have a plan for moving the parking spots that this tower displaced? Traffic on tower St. is going to get so much worse!"
If the expectation is that everyone owns a car, upzoning is a huge problem -- it actually is, because everything that scares a car brain ("I can't park!" "It's going to be loud!" "This will increase traffic!") is actually true. Rather than limiting cars, of course, we choose to limit zoning. Which is actually quite a logical thing to do if everyone is to own a car.
If you get rid of cars you get rid of collectors, arterials, onramps and offramps, turn lanes, parking spots... Those are things that are fundamentally incompatible with a major, dense, vibrant city. You can easily fit 50,000 people, their workplaces, senior care homes, schools, and restaurants into the land area of a 4 way cloverleaf that is designed to service 50,000 round trip commutes by car.