Let's think about academia for a second and the metrics we use. You must publish often and your work most be "novel." The more novel, the more prestige, the more frequent the faster you rise.
This is a system that's problematic for science, but crazy in humanities. One reason in science is it discourages replication. But there is a parallel here in something like history, reinvestigation. Something like the Roman Empire is well studied and there's lower novelty to be gained. But do we want to discourage such research? Discourage modern people for maintaining such records?
Under the current system, we can encourage these people to continue studying popular areas but do we encourage maintaining historical truth and integrity? I conjecture that the answer is no. It encourages making things up, especially to fit a modern narrative. It's a system that makes people not know they're lying because they are true believers and no credible experts push back.
This isn't a problem unique to the study of history, but you can find issues like this is all research in academia. It is very hard to measure the quality of research. Citations, h-index, and so on do not mean a lot. We've seen in the past few months very popular researchers found to be frauds, with high positions in prestigious institution like Stanford. Even in practical areas that should be treatable: psychology, medicine, machine learning, and more. We want to believe, but do we have strong evidence and strong understanding? We make easy metrics at first to be "good enough" or because we lack better ones. But we often forget the limitations of those metrics as time goes by because we get caught up in the rat race. You can't just ask yourself if the metric makes sense, but instead need to ask if there's an easier way to score on the metric that isn't the intended object of measurement. We can see here that citations can be hacked easily by fun stories, surprising results, and we'll never uncover them because the only people who attempt to replicate are those in junior positions like undergrads and grad students, who are more likely to question their results than the original work. Those failures never get written about.
Science is really about progressing knowledge. Publications are really about communication. Both these are very vague and can be accomplished in many different ways.
I say instead of making narrow easy to define metrics that will never be good enough we embrace the chaos and noise. Let the researchers be free and let ̶G̶o̶d̶ time sort it out. We don't know what is good research until 5-20 years down the line. We don't know who's a good teacher until the same. By removing the metrics you force nuance to not be ignored. Most of the time it's a "know it when you see it" definition and if that's true we have to be careful about levels of confidence. The thing that makes us human is the ability to handle nuance. It is what differentiates us from animals and LLMs. If the downside is that in a field where people are required to think hard and carefully have to... Think hard and carefully, then that's okay. ML is very successful with the arxiv model because despite much hype, noise, and even many bad actors, us researchers can sniff it out. We have to do it anyways, as that's our job.
I fear if we don't fix things we will continue to just degrade our foundation. We've seen this experiment fail over the last century. It's time to just get rid of the bureaucracy. Let academics be academics. Micromanagement isn't helping. Research can take decades, but people concerned with the next quarter will never allow that. Academics is about risks, challenging the status quo, and nuance. It is not to be run like a business. I am happy for this sector to not be profitable because I know the profits are indirect and benefit everyone. But returns on investments may take centuries, and that's okay.
I often think the same, in fact in my todo list is (in the background) think of more ways to reduce the bureaucracy I face on a daily basis, which takes many forms - from arguing with reviewer #2 over whether or not my finding is notable, to trying to convince a panel that my idea is worth funding.
From a system design perspective, alas resources are limited and have to be allocated somehow, and in the absence of any metrics at all as you suggest, it would ultimately come back to who can tell the better story about their work. This is still at best only weakly correlated with the actual quality of the work.
Exactly. But I think if we at least recognized that metrics are guides rather than targets we could get reviewer 2 to stop wasting everyone's time or get an AC to tell them to shut up. It wastes a lot of tax payers money.
Similarly I think we need to recognize academic funding as that of long term investments. You can be highly risky and should be. In reality our total investment into this sector is very low, we aren't even investing a penny from our dollar.
And I could go on about the scam of "peer reviewer" which just seems like a way to extract government money, using another players workforce, and providing at best a trivial amount of utility
This is such an academic take! There's tons of fraud, bias and false claims in research, therefore the fix is to get rid of whatever trivial checks and balances we already have and let researchers do or say whatever they want.
The "let the researchers be free" is the system we already have and it's not working. Researchers need much less freedom than they have today. They need to be micromanaged far more aggressively, by people who aren't impressed by them. Society needs to treat them as potential adversaries, regulate the hell out of them and toss them in prison when they step out of line, no different to company CEOs or corporate scientists (see Theranos). Also reducing their number by 20x or so would be a good start.
Whatever happens, they definitely need to lose the freedom to publish false claims. Also universities are run by people who will never do this. Academic fraud happens all the time and there's never any follow up. Therefore universities need to lose their freedom too.
There are really only two solutions to these sorts of frauds:
1. Defund it all, let the private sector seek the right level of management
2. Try to fix it with more micromanagement and regulation in the public sector.
"Let academics be academics" is exactly how large sections of the population have ended up holding academia in outright contempt.
> There's tons of fraud, bias and false claims in research, therefore the fix is to get rid of whatever trivial checks and balances we already have and let researchers do or say whatever they want.
You've completely mischaracterized the argument. The problem is that the metrics don't work and that the metrics themselves incentive fraud more than they incentive quality research. Call it the cobra effect or goodhart's law, but it's the same thing.
The reason I present this solution is because I'm saying to get rid of the carrot that leads us towards fraud. You may also be able to infer that I'm suggesting to take the money out of academics too, which is a different carrot pointed in the same direction. The point is to make the system only enticing for those that want to research and learn, not to those that want to get rich.
Unfortunately they'll always be able to publish false claims. That's because claims are almost never immediately verifiable. It can take decades many times. We also do not want to cause people to fear being wrong so much that they never attempt to make any meaningful progress. The process of researching is being wrong over and over until you find something that doesn't look wrong. Then you communicate it to others. And like I said, one thing that is discouraged by the system that I want to encourage is that of replication. This is your metric that you seek. Unfortunately it still doesn't completely been verified results but it does build high confidence. Replication allows us to make the mistakes and learn from them. It teases out the charlitians as their work will always fail to be reproduced. Just because you do not think there are guides to my system does not mean there aren't.
> They need to be micromanaged far more aggressively, by people who aren't impressed by them.
This is such a business person take! To overly simplify metrics, to believe everything can be codified by numbers, and any complexity is non-existent to the system. I say this as a mathematician. Metrics are guides and always limited. You assume naively that everyone is chasing money when most academics aren't. Pursuing my PhD is pretty similar to turning down hundreds of thousands of dollars, and profit maximization is at the masters level.
The problem you have is that you're looking at the goals wrong. You're making this system profit oriented. But the goal instead is to educate and to expand human knowledge. You may think that these are aligned. I'm sure you've created a good explanation for why these are aligned. But the common mistake far too many make is that you have to challenge your metrics and ask yourself how else they can be achieved besides your intended route. You must constantly ask yourself this and try to find easier paths than the intended one, especially as time goes on. But you're just one person and there's a reason all code eventually gets hacked. There's a certain irony that on a forum called hacker news that we pretend that mathematics/metrics can't be hacked but all code can, despite these being just different representations.
All metrics can be hacked because they aren't perfect rulers with all the nuances built in. It is why all rules are meant to be broken, because the environment changes and a simple set of words does not completely codify the intent and spirit of those words. You are always required to be human, not automata, because the incompleteness is fundamental to the system, to the world.
> You've completely mischaracterized the argument ... The reason I present this solution
You haven't presented a solution. What you're making here, though I think you don't recognize it, is a classical Marxist argument. It goes, people are only bad because of the system of incentives within which they work. If we abolish that system then everyone will become good.
If you want to have full time professionals doing research then there must be a definition of what "professional" means and that must be measured and enforced by outsiders. That means management and will often mean micro-management, because as we've seen researchers will otherwise drift towards a local minima of making false claims as a group without any way to self-correct.
Now, if you want to argue for the mass defunding of academia then please do so! But that isn't what you argued for so far. And it wouldn't directly address the false claims problem except by reducing the overall volume of claims.
Replication isn't the right place to start with reform. The system is so broken it's routinely allowing through bad claims that don't require some fancy replication effort to detect, you just need someone with some time to read them and do basic sanity checks. Just paying professional peer reviewers would be a good start (require paper submissions to fund this process even if they get rejected to reduce journal CoI). This type of trivial reform could be done tomorrow but nobody seems to be doing it. But even if they tried, it would be immediately subverted by academic institutions. The only reforms to science that stick are ones imposed by outsiders who are the ones who want results.
> To overly simplify metrics, to believe everything can be codified by numbers, and any complexity is non-existent to the system.
Nobody in business believes that. Well run businesses don't overly rely on metrics to define productivity and frequently rely on intuitive things like the judgement of managers, which is in turn judged by the quality of their results by customers and so on. Part of what makes capitalist competition work well is that it ensures a constant search for the right balance between metric and management defined career success.
> You assume naively that everyone is chasing money when most academics aren't
They are all chasing money, that's why they demand a salary for their work and fill out the grant forms, instead of being a hobbyist in a shed. Academic denial of this is how we end up with repeated cases of blatant fraud that can be detected by anyone who simply reads the paper and its references, with no reform or even admission that the system is flawed.
Arguments like the one you've presented here are honestly the best sort of evidence for the abolition of academia. These are institutions overrun by people with an understanding of human nature so warped they learned nothing from 20th century history and probably never will. Self reform seems impossible with such a culture. We need politicians who will grab the bull by its horns and sweep it aside.
This is absolutely hilarious considering why communist countries fall. Because planned economies are too inflexible to adapt to rapidly changing environments. Communists love metrics. They love bureaucracy. They love management. They love simplicity. To boil things down into bit size components that are easily digestible.
You have things exactly backwards. I am preaching to embrace chaos, not order. It is the claim that big brother not only doesn't know best, but that no brother can. That truth has a lower bound to complexity but deciept has the capacity to be infinitely simple. That is why so many can fool us, because we abhor complexity. Even the most complex conspiracy theories boil down to nothing distinguishable from "wizards did it." They replace all knowing gods with all knowing men, because in the end it is better than evil rule than chaos. Because at least when evil rule there is order, there is someone in charge. My claim is that there is no one in charge and everyone is just as lost as you, pretending to know their way.
If you read Marx and Engels you will find that they absolutely love meritocracy. Something I'm literally calling a pipe dream and unobtainable. Something I'm claiming that pursuing leads you in the opposite direction of intent. You requested micromanagement, and to the extreme. I cannot see how you disagree with those who's most fundamental belief is "from each according to his ability." To be all that you can be, or be a failure to the state. The people who want to so intemently know you're life. Communist Russia, China, and North Korea are not known for their freedom and chaos. They are known for heavy handed authoritarians who want to micromanage, to subjugate, because they know best.
Talk about calling the kettle black. You impose a position on academics because it is one you want to believe but not one you've cared to validate. To see the stark divide between those in the humanities and those in stem. I have actually read the "Communist Manifesto" and other works like Stalin's "Dialectical and Historical Materialism"[0]. You know why I don't believe them and do not preach their words? Because unlike many of those who believe as well as those who hate, I actually read them before I made up my mind. I know their claims not because others have told me what they say but because I heard it from the horse's mouth. Maybe you should too.
And before you talk about Smith's Invisible Hand, maybe you should read "The Theory of Moral Sentiments" and "The Wealth of Nations." Maybe you should also read the work of his best friend.
You are so quick to call out others for being "sheeple" yet you demonstrate that you have not read the source materials. Are you afraid? If they convert you then clearly your beliefs were not strong. But if they are wrong then you can actual counter the arguments being made rather than strawmen which may not even exist. I'll tell you from personal experience, nothing has turned me off communism more then the Communist Manifesto itself. You speak of merit, but how can you claim it if you do not read the source material. If you do not give it a true fair shot, a good faith read. There's is no magic spells and reading shit will not cause you to believe shit.
You've shown all throughout this that that you have an abundance of confidence but a dearth of expertise. You have had ample opportunities to explain your arguments, to act in good faith, but at every turn you say no more than "trust me." Even the Russians followed that with "but verify" (and Reagan popularized in the west). I'm not willing to trust those who do not provide the means to verify. Nor do I trust those who do not care to respond to arguments being made. You have a lot more in common with Stalin than you may like to believe, for you too -- like in [0] -- happily create not just strawmen, but conjure arguments never made so that you can easily topple them.
You peach answers while I peach uncertainty. How dare you compare me to men who seek to be gods.
I've read the Communist Manifesto too. It is long on condemnation of the existing system and short on how their proposed alternative would work (because there wasn't one).
So what exactly is your proposed plan because it doesn't seem to be spelled out explicitly? You want to defund all of academia and let the market drive research efforts? I'm all for that. But the net effect would be that researchers have managers who hold them to account for their effort, same as any employees.
What I'm criticizing here is the frequent academic response that all the problems will be solved if the (very small) amount of supervision and bureaucracy they're put under is swept away. This critique is much like that Manifesto. It doesn't explain how anything will be fixed by embracing the chaos, as you put it. It just sort of asserts that it will.
You libertarians are no different from the communists you so protest. One just worships the invisible hand of the free market -- something Smith would be devastated to see -- while the other worships the omniscient hand of big brother. Both seek appeal to a higher power that knows all. You worship a market and your claim is that one size fits all. For you yourself said that all strive richest and wealth. So nothing I can say because you won't believe me that I do not seek wealth or richest but just wish to live my days reading and doing math. Your model is to strict to believe I exist, or my partner, or my best friend. I know many who seek wealth, especially in my field of ML, but I promise you that there are many like me who just want to learn.
I have said my due, but my due is not simple. I cannot spoon feed you something which I claim cannot be spoon fed. I'll admit my ideas are likely a pipe dream, because they require one to think long and hard, to constantly re-evaluate to a changing ecosystem. To iterate and improve. For there is a bound to the simplicity of reality, but complexity goes against our deepest desires. There is no spoon feeding but you must read my words carefully. For your priors are getting in the way. Stop being a true believer and seek to understand what people are trying to say, not what you know they mean. For the latter is impossible for nothing you can say will convince me that you're an omniscient god.
I don't think there's much to respond to here, but for some reason both your posts had been flag killed. I don't think this argument is much good and frankly your posts are a little bit offensive, but that shouldn't result in being cancelled so I vouched for them.
This is a system that's problematic for science, but crazy in humanities. One reason in science is it discourages replication. But there is a parallel here in something like history, reinvestigation. Something like the Roman Empire is well studied and there's lower novelty to be gained. But do we want to discourage such research? Discourage modern people for maintaining such records?
Under the current system, we can encourage these people to continue studying popular areas but do we encourage maintaining historical truth and integrity? I conjecture that the answer is no. It encourages making things up, especially to fit a modern narrative. It's a system that makes people not know they're lying because they are true believers and no credible experts push back.
This isn't a problem unique to the study of history, but you can find issues like this is all research in academia. It is very hard to measure the quality of research. Citations, h-index, and so on do not mean a lot. We've seen in the past few months very popular researchers found to be frauds, with high positions in prestigious institution like Stanford. Even in practical areas that should be treatable: psychology, medicine, machine learning, and more. We want to believe, but do we have strong evidence and strong understanding? We make easy metrics at first to be "good enough" or because we lack better ones. But we often forget the limitations of those metrics as time goes by because we get caught up in the rat race. You can't just ask yourself if the metric makes sense, but instead need to ask if there's an easier way to score on the metric that isn't the intended object of measurement. We can see here that citations can be hacked easily by fun stories, surprising results, and we'll never uncover them because the only people who attempt to replicate are those in junior positions like undergrads and grad students, who are more likely to question their results than the original work. Those failures never get written about.
Science is really about progressing knowledge. Publications are really about communication. Both these are very vague and can be accomplished in many different ways.
I say instead of making narrow easy to define metrics that will never be good enough we embrace the chaos and noise. Let the researchers be free and let ̶G̶o̶d̶ time sort it out. We don't know what is good research until 5-20 years down the line. We don't know who's a good teacher until the same. By removing the metrics you force nuance to not be ignored. Most of the time it's a "know it when you see it" definition and if that's true we have to be careful about levels of confidence. The thing that makes us human is the ability to handle nuance. It is what differentiates us from animals and LLMs. If the downside is that in a field where people are required to think hard and carefully have to... Think hard and carefully, then that's okay. ML is very successful with the arxiv model because despite much hype, noise, and even many bad actors, us researchers can sniff it out. We have to do it anyways, as that's our job.
I fear if we don't fix things we will continue to just degrade our foundation. We've seen this experiment fail over the last century. It's time to just get rid of the bureaucracy. Let academics be academics. Micromanagement isn't helping. Research can take decades, but people concerned with the next quarter will never allow that. Academics is about risks, challenging the status quo, and nuance. It is not to be run like a business. I am happy for this sector to not be profitable because I know the profits are indirect and benefit everyone. But returns on investments may take centuries, and that's okay.