Have you ever used Lyft. In my experience (SF), it literally takes minutes for a car to show up. More often than not, I am still looking for may keys/wallet/jacket or brushing my teeth when the car gets here. I imagine an autonomous car service would be even faster, as car availability could be highly tuned to demand (Lyft and Uber are using comparatively crude methods of demand/availability management, surge pricing).
As others have mentioned, the car-sharing model works best in urban areas, and no one is suggesting that we ban individuals from owning cars. But I will also add that we have spent the last 100 years building infrastructure and public spaces to accommodate for a car-centric society, but this has had a negative impact on people who prefer other forms of transportation. Public transportation is severely underfunded and inadequate in most places, pedestrians and bikers are getting mowed down by reckless drivers, tons of public land is being poorly utilized as parking space, and our air is getting polluted to unprecedented levels. Your "peace of mind" comes at a high cost to the rest of society. Autonomous cars are not a silver bullet for all of these problems by any means, but they can go a long way in getting us to a better place with most of these issues.
