Unlike Google, you can download Wikipedia and use it offline. I hope to see the same thing happening for a useful LLM. Of course that’s not feasible right now, but hopefully those costs will come down and we do actually get to run a self-hosted version of this.
The tricky thing about data is that the world constantly changes. A downloaded Wikipedia has a lot of value, but it does grow stale. And it has the advantage of being a repository of relatively static facts in a way that, say, a search engine is not.
Search engines (and I suspect a ChatGPT-style engine, if one wants to talk about it about current events, things currently available, or other topics of the day) have to be continuously refreshed to be relevant. So many things that those engines are used for frequently (including the keyword "ChatGPT" itself) had no definition months ago, let alone an inaccurate definition.
Most data isn't static like code; it must be continuously re-invested in to stay relevant.
> Search engines (and I suspect a ChatGPT-style engine, if one wants to talk about it about current events, things currently available, or other topics of the day) have to be continuously refreshed to be relevant.
Maybe. It depends on what you're searching for. I'd say that 80% of the searches I engage in don't need a particularly fresh database to satisfy.
