Be careful about A/B tests, from what I understand Amazon relies on the extensively and they have helped it produce its Amazon website.
Enough said.
If you have two choices one bad and another less bad you will chose the less bad, when instead you may just want to step back and ask is either choice the correct one.
This is the strangest warning against A/B tests I've heard. "Be careful because you might end up producing something that is #1 internet retailer". The Amazon home page might not be as sexy as the latest whizz-bang startup, but success doesn't lie.
Thank you. I can't count the times I've tried to explain this to people. The purpose of amazon's site is not that visitors think it's pretty, or even easy to use. It's to get you to buy as much as possible. And there's convincing evidence that it's quite successful in fulfilling that purpose.
Why do supermarkets put the most frequently sold items at the back, when it would be so much easier for customers if they were right by the entrance?
I don't particularly like Amazon's site either, I think it's cluttered, but it has improved a bit. It's still not the prettiest thing, but what it does right, though, is it makes it very, very easy to buy things. And that's all it needs to do.
But you're right about not always choosing the "less bad" when there are other options you may not be considering.
The problem is though that this observation is often used as an excuse to ignore data or avoid doing small iterative experiments. In most cases, this is a mistake. In other words, you could do a lot worse than reproducing Amazon.com.
Enough said.
If you have two choices one bad and another less bad you will chose the less bad, when instead you may just want to step back and ask is either choice the correct one.