The biggest point I want to make us that that cult narrative isn't just people mistaking what they read about LLMs. That narrative started with how LLM authors write about LLMs.
An LLM is not a language model, it is a text model. An LLM does not lie, it provides whatever continuation happens from modeling the prompt into the training corpus; with no notion of truth or fact. An LLM does not invent novel behaviors to solve patterns, it finds patterns that were already in the text that humans consider novel.
Practically everything I have just said is in conflict with how LLMs are presented by their authors. That's where the cult narrative is rooted.
An LLM is not a language model, it is a text model. An LLM does not lie, it provides whatever continuation happens from modeling the prompt into the training corpus; with no notion of truth or fact. An LLM does not invent novel behaviors to solve patterns, it finds patterns that were already in the text that humans consider novel.
Practically everything I have just said is in conflict with how LLMs are presented by their authors. That's where the cult narrative is rooted.