For a certain definition of "software": when only doing one training run costs an 8 digits sum (requiring hardware one order of magnitude more expensive than that to run) I kinda dispute the "all they do is software".
It's definitely not "all software": a big part of their advantage compared to actually free and open models is the insane hardware they have access to.
The free and open LLMs are doing very well compared to OpenAI once you take into account that the cost to train them is 1/100th to 1/1000th what it costs to train the OpenAI models.
This can be seen with StableDiffusion: once the money is poured in training the models and then the model made free, suddenly the edge of proprietary solutions is tiny (if it even exists at all).
I'd like to see the actually open and free models trained on the hardware used to train OpenAI: then we'd see how much of a "software edge" OpenAI has.
And my guess is it'd be way less impressive than you make it out to be.
They are using hardware, yes, but they are not creating (which is what I mean by "doing") the hardware. Anyone else with funding could have access to the same hardware for running their software, and other people did do that, and do do that (now, of course, in a drastically tighter supply/demand situation).
I do not wanna be flippant here: Obviously having easy access to money and a good standing with the right people is making things A LOT simpler, but other people could have reasonably convinced someone to give them money to built the same software. That's what VCs do, after all.
Regarding the rest: Feels very much like a different topic. I'll pass.
> I'd like to see the actually open and free models trained on the hardware used to train OpenAI: then we'd see how much of a "software edge" OpenAI has.
It would seem like you're talking about what "software edge" OpenAI has in the future, when others have caught up, while parent is talking about the existing "software edge" OpenAI has today, which you seem to implicitly agree with, as you're talking about OpenAI maybe not having any edge in the future.
For a certain definition of "software": when only doing one training run costs an 8 digits sum (requiring hardware one order of magnitude more expensive than that to run) I kinda dispute the "all they do is software".
It's definitely not "all software": a big part of their advantage compared to actually free and open models is the insane hardware they have access to.
The free and open LLMs are doing very well compared to OpenAI once you take into account that the cost to train them is 1/100th to 1/1000th what it costs to train the OpenAI models.
This can be seen with StableDiffusion: once the money is poured in training the models and then the model made free, suddenly the edge of proprietary solutions is tiny (if it even exists at all).
I'd like to see the actually open and free models trained on the hardware used to train OpenAI: then we'd see how much of a "software edge" OpenAI has.
And my guess is it'd be way less impressive than you make it out to be.