When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
I've heard this before, but why would it be true? Serious question.
I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories - but ChatGPT can! I've seen Penrose argue that AI is impossible, and yet I think that ChatGPT and AlphaZero prove him wrong. I know about Linus Pauling and quasicrystals. Is this a general rule, or are people sometimes wrong regardless of their age?
There's also a danger that it's ageist. Such things shouldn't be said unless there's good backing.
>I've seen Chomsky argue that LLMs can't regurgitate his linguistical theories
When has he said this? For the most part I feel Chomsky has been misunderstood when it comes to LLMs. As best as I can tell what Chomsky has said is that LLMs do not provide any insight into how language works, it's not really a scientific advancement so much as it's an engineering breakthrough.
The fact that LLMs exist and can mimic natural language does not in anyway give us insight into how humans construct language. People have been able to construct objects that can produce natural language for close to 100,000 years, but that doesn't mean that those people understood the nature of that language.
Chomsky said that LLMs are statistical regurgitators which means LLMs can never actually reason and explain which language understanding requires. That they are a wrong model of computation by definition.
It's an interesting position and I'm sympathetic toward it, he could be partly right in the end.
Regurgitators can't have internal representations? Sometimes the best way to regurgitate is to learn an internal representation. That doesn't mean it suddenly stopped being a statistical model.
Ok I actually thought about this a fair bit a few days ago and I think I have a good answer for this.
You’ve probably heard of the cheap bar trick that goes something like: “And what does a cow drink? Milk!”.
Irrespective of intelligence, humans tend to make silly cognitive errors like this because we are fundamentally pattern marchers.
In order to become a forerunner in a field, you necessarily have to be good at abstract pattern matching.
What happens as you age is that you no longer have the need to question assumptions because you know what’s real and what’s not. There’s also the decrease of white matter and an increase of grey matter which doesn’t help this.
As time goes on, certain assumptions change, essentially deprecating certain chunks of your crystallized learnings.
Some chunks of your thinking are still valid, so when you think something can be done, it most likely can be done.
However, if something falls outside your crystallized learning, you get a strong sense it’s wrong, when it might be because of your outdated assumptions.
You can try to hotswap the assumptions you have, but it becomes like Jenga the more years of experience you have in your field.
You either have to start from scratch and rebuild your lifetimes worth of learnings from the ground up or be super careful in reassessing everything you know
A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it ...
An important scientific innovation rarely makes its way by gradually winning over and converting its opponents: it rarely happens that Saul becomes Paul. What does happen is that its opponents gradually die out, and that the growing generation is familiarized with the ideas from the beginning: another instance of the fact that the future lies with the youth.
It was written down by Arthur C Clarke who was an author. It's just a rule of thumb really. I haven't looked into data on it but it seems like a common enough thing that there's something to it. As to why? I have no idea. Something lik: Older scientists are more conservative, therefore if they say something is impossible, they might just be out of touch with new developments. But if they say something is possible take it seriously because they don't use that word lightly.
- the scientists people know about are generally older
- older people are often thought of as wiser, or may indeed be so
- when a famous scientist - who is already likely to be older, and who has a history of getting things right - gets something wrong, then it's more jarring and noticeable
My theory then is that it isn't true, but we notice such cases more.
Also, examples of a theory being true doesn't prove the theory right. Bayes' theorem seems instructive here.
And Chomsky is in touch with other colleagues who agree with him, it's not as if his disagreement stems from being an old, isolated hermit. At the least you'd have to argue his colleagues are also mistaken.
The usual explanation is that they will call impossible something which goes against their life's work because in their mind it nullifies it, while a youngster has less or zero "sunken cost".
A related saying: "science advances a funeral at a time", meaning the old-guard blocks new theories for the same reason, they go against their life's work.
This is true, but misses the important part that they (the older set) are often correct. For every new idea that really changes everything there are a huge number that die on the vine or just become a ho-hum tool in a big toolbox.
Most new ideas are less interesting and impactful than they seem when you are in the middle of their creation. You never really get to see what's happening until much much later.
A variant of all this is that you should trust the old guard when they tell you something can be done, but not when they tell you it can't. There is a good quote about that I've forgotten.
The corollary is that you shouldn't really trust the young turks on anything, but you should support their efforts and test the results.
It's very human to see yourself as Planck in the early 1900s
not Wolfram in the early 2000s.
In this case, however, the elderly scientist is stating things are possible, so Clarke's law doesn't apply. What he is saying is possible, is very bad.
When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.