> It is popular to think that A/B testing leaves you in local maxima, but what about alternatives?
The alternative involves creativity to create a vision to achieve. A/B testing are pretty good tools to evaluate differing opinions on some vision, as the only deciding tool for what is worth to keep or not it's pretty short-sighted.
Is it really that much better? I lived through the rise of A/B testing, professionally I've been responsible for implementing platforms for experimentation in at least 5 different companies and over the years I saw it transition from a tool to a crutch. Everything now is "data-based" which means incessant A/B testing, it still depends on your average PM deciding the parameters of the experiment, leading to the current state of affairs: a bunch of uninspired features that are solely judged on the data produced, no soul, no vision behind most of it.
And it feels exactly like that, I can feel how tech products just became that, an incessant stream of features being tested, implemented or tossed.
I'd much rather have data-supported intuition than this statistical machine in motion.
The alternative involves creativity to create a vision to achieve. A/B testing are pretty good tools to evaluate differing opinions on some vision, as the only deciding tool for what is worth to keep or not it's pretty short-sighted.
Is it really that much better? I lived through the rise of A/B testing, professionally I've been responsible for implementing platforms for experimentation in at least 5 different companies and over the years I saw it transition from a tool to a crutch. Everything now is "data-based" which means incessant A/B testing, it still depends on your average PM deciding the parameters of the experiment, leading to the current state of affairs: a bunch of uninspired features that are solely judged on the data produced, no soul, no vision behind most of it.
And it feels exactly like that, I can feel how tech products just became that, an incessant stream of features being tested, implemented or tossed.
I'd much rather have data-supported intuition than this statistical machine in motion.