I think experimentation is hard-wired into the brain (its partly how we're able to learn so much so quickly as children), but rigor isn't. So is Mythbusters' approach to experimentation without rigor a net win? I spend probably a third of each episode wondering about that. Its very frustrating, but then something blows up, so generally it works out ok.
Mythbusters is close to correct, which is good enough.
They actively involve their community, which goes the rest of the way. If there is a serious lack of rigor, their audience will point it out, and an experiment will be rerun.
I sometimes think the lack of rigor is serious: most Mythbusters' viewers could probably create similarly rigorous experiments if they had the inclination, what is lacking isn't so much holding beliefs to experiment, but interpreting the validity and applicability of experiments.
Some of Mythbusters' experiments are reasonable enough, but a lot of others are a little too sloppy for my taste, teaching a lazy habit of confirming beliefs by constructing plausible narratives.
Key words "if they had the inclination." Imprinting minds with the idea that ideas should be tested by experiment is of tremendous value. Rigor can follow once the inclination is there.
Agreed, absolutely. You have to motivate rigor. I actually think the show does a fine job of that.
Sometimes they'll do a half-assed experiment, bust some myth, and then get viewer mail criticizing their technique. Then they'll revisit the myth, with better controls! This is pretty much exactly how real science works. Nobody does rigorous science because they want to spend enormous amounts of time and money doing the same boring experiment over and over, gradually imposing tighter and tighter controls on everything. They do it because otherwise their half-assed papers will get rejected by their extremely critical peers.
Sure, but convincing yourself of the validity of your own theories is always easier than it should be. That's human nature.
Among other things, rigor costs money and time. People's self-criticism tends to taper off as the deadline looms. This is a particularly important factor in Mythbusters -- my impression is that the Mythbusters folks are exactly as self-critical as their budget allows them to be.
I wouldn't be too convinced it's that close to correct... Usually the set up a handful of experiments, get as close to the original situation as they can (which is often not that close - sometimes they're basing their experiments on movie events and guessing metrics), and give it a go a few times. But that said, the show is about entertainment rather than hard-core scientific vigour, so I'll continue to watch and enjoy :-)
Sometimes their basic science has been lacking. (Compressed air jet-boat was supposed to work by "pushing against the water"?) Though they have been improving, I've noticed.
