Think about aluminum smelting. At some point in the past, only a few researchers could smelt aluminum, and while it used a ton of energy, it was just a few research projects. Then, people realized that aluminum was lighter than steel and could replace it... so suddenly everybody was smelting aluminum. The method to do this involves massive amounts of electricity... but it was fine, because the value of the product (to society) was more than high enough to justify it. Eventually, smelters moved to places where there were natural sources of energy... for example, the Columbia Gorge dam was used to power a massive smelter. Guess where Google put their west coast data center? Right there, because aluminum smelting led to a superfund site and we exported those to growing countries for pollution reasons. So there is lots of "free, carbon-neutral" power from hydro plants.
The interesting details are: the companies with large GPU/TPU fleets are already running them in fairly efficient setups, with high utilization (so you're not blowing carbon emissions on idle machines), and can scale those setups if demand increases. This is not irresonsible. And, the scaleup will only happen if the systems are actually useful.
Basically there are 100 other things I'd focus on trimming environment impact for before LLMs.
The interesting details are: the companies with large GPU/TPU fleets are already running them in fairly efficient setups, with high utilization (so you're not blowing carbon emissions on idle machines), and can scale those setups if demand increases. This is not irresonsible. And, the scaleup will only happen if the systems are actually useful.
Basically there are 100 other things I'd focus on trimming environment impact for before LLMs.