Yes you've identified the key difference which is one of the tests used in UK courts to determine if someone is really a contractor or actually a disguised employee and ability to set price matters.

If Uber did let drivers set price then it wouldn't be facing these issues and would just be an arbitrage marketplace. Of course if they did that their prices wouldn't be low and they wouldn't be as popular since currently they subsidise pricing.