Hacker News new | past | comments | ask | show | jobs | submit login

Not in academia, but what I hear is that very few results are ever attempted to be reproduced.

So if you publish an unreproducible paper, you can probably have a full career without anyone noticing.




Papers that can't be reproduced sound like they're not very useful, either.

I know it's not as simple as that, and "useful" can simply mean "cited" (a sadly overrated metric). But surely it's easier to get hired if your work actually results in something somebody uses.


Papers are reproducible in exactly the same way that github projects are buildable, and in both cases anything that comes fully assembled for you is already a product.

If your academic research results in immediately useful output all of the people waiting for that to happen step in and you no longer worry about employment.


The reality is a bit different.

The "better" journals are listed in JCR. Nearly 40% of them have impact factor less than 1, it means that on average papers in them are cited less than 1 times.

Conclusion: even in better journals, the average paper is rarely cited at all, which means that definitely the public has rarely heard of it or found it useful.


> Papers that can't be reproduced sound like they're not very useful, either.

They’re not useful at all. Reproduction of results isn’t sexy, nobody does it. Almost feels like science is built on a web on funding trying to buy the desired results.


Reproduction is boring, but it would often happen incidentally to building off someone else's results.

You tell me that this reaction creates X, and I need X to make Y. If I can't make my Y, sooner or later it's going to occur to me that X is the cause.

Like I said, I know it's never that easy. Bench work is hard and there are a million reasons why your idea failed, and you may not take the time to figure out why. You won't report such failures. And complicated results, like in sociology, are rarely attributable to anything.


That's true for some kinds of research but a lot of academic output isn't as firm as "X creates Y".

Replicability is overrated anyway. Loads of bad papers will replicate just fine if you try. They're still making false claims.

https://blog.plan99.net/replication-studies-cant-fix-science...


I've had this idea that reproduction studies in one's C.V should become a sort of virtue signal, akin to philanthropy among the rich. This way, some percentage of one's work would need to be reproduction work or otherwise they would be looked down upon, and this would create the right incentive to do go.


Reproduction is rarely done because it is not "new science". Everyone is funding only "new science".


Depends on the field.

Psycho* is rife with that.


> Depends on the field.

Yeah...It's more on the less Pure domains...And mostly overseas?... :-) https://xkcd.com/435/

"A 2016 survey by Nature on 1,576 researchers who took a brief online questionnaire on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers, 67% of medical researchers, 64% of earth and environmental scientists, and 62% of all others), and more than half have failed to reproduce their own experiments."

https://en.wikipedia.org/wiki/Replication_crisis


> And mostly overseas

Where is the data that supports that?

And by overseas what do you mean, or are we talking about USA "defaultism" here?




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: