That's assuming that large language models or transformers are the key to AGI, which is looking increasingly unlikely:

https://arxiv.org/abs/2311.00871