> Hmm maybe the problem is actually interacting with humans naturally
Almost every driver has a rythym (or pattern, if you will) to the way they drive; and we pick up on this, and interact with them accordingly on the road. Even most bad drivers have a pattern to the way they drive. The more experienced you are, the easier it is to "read" fellow motorists, and safely interact with them on the road. One of the most difficult scenarios is when you come upon someone who drives with No rythym or pattern whatsoever. You have no clue what that person is Likely (Emphasis on likely) to do next.
I have no experience sharing the road with autonomous vehicles, but I suspect they will Have No rythym to the way they drive, or rather none that is discernible to a human. IMO this will be The major obstacle to overcome if humans and bots are to share the Same road at the same time.
What's the more interesting question is not whether there will be corner cases where self-driving fails, but when the average performance is so much better than humans that AI saves 10s if not 100s of lives every day, but that's still a reality where AI driving software could be actively out there involved in the deaths of a single digit number of different people a day or week.
For sure deploy if the numbers are right. It's the edge cases that are interesting.
Cameras are fine in the day, but when I struggle to see at night, in the rain, with spray from cars, glare from on coming traffic... that's where I would LOVE to have a lidar assisted heads up display. How Tesla think they can do better with just cameras compared to another car with cameras AND lidar just baffles the mind.
Not better in absolute terms, but better in terms of the best car for a certain price. Sure a $200k car with redundant LIDAR and horribly inefficient aerodynamics because a giant spheres on top (like weymo) might be better than a $50k with just cameras.
But the hard part is the software, and it's unclear that the added complexity of lidar + camera will allow the software to be better than just cameras.
As an example a recent study at Cornell shows that a stereo pair of cameras mounted high up behind the windshield provided similar quality 3d data to LIDAR. Search for "Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving" or the more consumer friendly "Elon Musk Was Right: Cheap Cameras Could Replace Lidar on Self-Driving Cars, Researchers Find".
Seems much easier to work on a stereo video stream that includes 2D, color, and extracted 3D features than trying to achieve sensor fusion with lidar + video. Especially if you want to make full use of color for things like brake lights, traffic signals, color/markings of an ambulance, school bus, lane markings, and other important details that are invisible to LIDAR.
Especially consider that if weather is terrible and the camera vision is so bad that only 5mph is safe. If LIDAR can see significantly better, do you allow it to drive at 50 mph because it can see 200 feet? For just social reasons it seems like driving as much like a human as possible is going to be best until all cars are autonmous.
Cars and humans inhabit mutual space to such an extent that perhaps level-5 self-driving wouldn't be enough. What we need might be a general AI that can comprehend its surroundings and connect them to a symbolic hierarchy of intentions well enough to obey Asimov's 3 laws.
This is why I think the solution will be something like "SDV lanes" -- separate lanes for robots on the highway, like bus lanes. Current roads are designed for humans communicating with each other, and if we wait for general AI to implement self-driving cars, we'll be waiting a long time.
