Your report matches many other real world reports I've read. I'm pretty good at day dreaming or thinking while driving, so having to keep my hands ready to take over while being completely alert that FSD might error would be a big downgrade in my driving experience. I'd rather drive myself where my subconscious muscle memory does the driving so my conscious mind can think about other things. Having to pay attention to what FSD was doing would be a drag and prevent me from relaxing.