I feel like, given my experience lately with all the API models currently available, that this is only a fact if the models google is using internally are SIGNIFICANTLY better than what is available publicly even on closed models.
Claude 3.5-sonnet (latest) is barely able to stay coherent on 500 LOC files, and easily gets tripped up when there are several files in the same directory.
I have tried similarly with o1-preview and 4o, and gemini pro...
If google is using a 5M token context window LLM with 100k+ token-output trained on all the code that is not public... then I can believe this claim.
This just goes to show how critical of an issue this is that these models are behind closed doors.
> This just goes to show how critical of an issue this is that these models are behind closed doors.
How is competitive advantage, using in-house developed/funded tools, a critical issue? Every company has tools that only they have, that they pay significantly for to develop, and use extensively. It's can often be the primary thing that really differentiates companies who are all doing similar things.
Claude 3.5-sonnet (latest) is barely able to stay coherent on 500 LOC files, and easily gets tripped up when there are several files in the same directory.
I have tried similarly with o1-preview and 4o, and gemini pro...
If google is using a 5M token context window LLM with 100k+ token-output trained on all the code that is not public... then I can believe this claim.
This just goes to show how critical of an issue this is that these models are behind closed doors.