Welcome. (I had no idea that MIT had a paywall… apparently uBlock just silently bypasses it? Or I just don’t read enough every month.)
PS, if you don’t want to read all that, the NotebookLM video overview (a powerpoint with a narrator) is actually pretty good at summarizing their report:
It’s not that long of an article (10 pages) albeit some of it is dense going for my knowledge base.
IF I understand it correctly these are figures for the serving AI, specifically Google’s Gemini apps, on a per prompt basis, not the energy of training it. The marginal costs of each prompt. I may be mistaken but the fixed costs involved in the initial training mode are not part of this analysis.
Those marginal costs are 0.03g CO2e, 0.24 Wh energy, and 0.26mL water per prompt.
Hence “the impact of a single prompt is low compared to many daily activities…”, lower than previous estimates, and they explain why with some self congratulations (mostly improved software efficiencies and clean energy procurement for their centers).
What is missing is some estimate of how many prompts are occurring each day either on a per user or aggregate basis. And the even more useful number would be a comparison to how those numbers compare for a worker producing the same amount of “work” by previous means, or a person using AI in recreation time, as a companion, or as entertainment, would use doing the activity they had done previously before AI.
I suspect it would come out looking not too bad. Even amortizing the energy cost of the training to a per prompt basis and adding it in. But I can’t conclude it from this information. Unless I am missing something. Definitely possible.
Interesting. My car gets 205 Wh/mile. Pretending character.ai is the same energy cost as Gemini, my kid on their phone in the backseat would have to put in 9-10 prompts per second to match the energy use of the car. That’s obviously ridiculous, but hundreds of prompts total per night is not. So quite possible that a night of chatting with Optimus Prime on character.ai would use enough energy to drive a mile.
Today on Marketplace was an interview with a reporter from the New York Times, who claims that currently 4% of all power generation in the US goes towards data centers, and within a few years it will be 12% of all power generation.
Perhaps the full article goes into it, but the radio piece did not, that non-AI datacenter loads are far less power intensive than AI. Hosting a message board, or streaming Netflix content is going to be a small fraction of the energy used by a single 700 watt Nvidia GPU.
Paywalled link; I’m not a subscriber.
But if you’re streaming to your big TV, you’re using ~100 W continuously, as compared to the 1 W of your phone, or maybe 10 W of a tablet/laptop and a small share of that 700 W GPU (the few seconds it takes to generate the response compared to the time it takes to read/listen to it and input a new prompt).
Gifting it.
Doesn’t seem to preview but hopefully links.
One of their sources is this:
Traditional utility planning assumes predictable 1%-2% annual demand growth over decades, but data centers are driving regional growth rates of 20%-30% annually. This mismatch between conventional planning timelines and demand growth has exposed limitations in capacity planning practices and increased short-run electricity generation costs, with some markets heavily utilizing older and more costly fossil-fuel generators in the short run … Regional cost surge: Central and Northern Virginia face projected 2030 electricity cost increases exceeding 25%, the highest regional increase in the model.
Coal gets lifeline: More than 25 GW of aging coal plants otherwise scheduled for retirement would continue operating primarily to serve data center demand.
Emissions spike: Power sector emissions could increase 30% compared to scenarios without data center growth, reaching 275 million metric tonnes of CO2 annually by 2030. That matches the entire annual carbon output of France.
Carbon leakage: Virginia’s data center growth drives increased fossil fuel use in nearby states like Ohio, Pennsylvania, and West Virginia, potentially undermining state and regional climate goals.
Good catch.
Thanks.
This from the International Energy Agency
Our Base Case finds that global electricity consumption for data centres is projected to double to reach around 945 TWh by 2030 in the Base Case, representing just under 3% of total global electricity consumption in 2030. …
…China and the United States are the most significant regions for data centre electricity consumption growth, accounting for nearly 80% of global growth to 2030. Consumption increases by around 240 TWh (up 130%) in the United States, compared to the 2024 level. In China it increases by around 175 TWh (up 170%). In Europe it grows by more than 45 TWh (up 70%). Japan increases by around 15 TWh (up 80%).
Comparing data centre electricity consumption normalised per capita can give a sense of the importance of this sector in different economies. Africa has the lowest consumption at less than 1 kWh of data centre electricity consumption per capita in 2024, rising to slightly less than 2 kWh per capita by the end of the decade. However, there are strong differences within the region, with South Africa showing strong growth and per-capita consumption more than 15 times larger than the continental average in 2030, with an intensity higher than 25 kWh per capita. By contrast, the United States has the highest per-capita data centre consumption, at around 540 kWh in 2024. This is projected to grow to over 1 200 kWh per capita by the end of the decade, which is roughly as much as 10% of the annual electricity consumption of an American household. This intensity is also one order of magnitude higher than any other region in the world.
Next linked page continues with estimates based on expected energy production changes …
CO2 emissions from electricity generation for data centres peak at around 320 Mt CO2 by 2030, before entering a shallow decline to around 300 Mt CO2 by 2035. Despite rapid growth, data centres remain a relatively small part of the overall power system, rising from about 1% of global electricity generation today to 3% in 2030, accounting for less than 1% of total global CO2emissions.
And IF AI delivers on substantially greater productivity there may be significant decreases in CO2 per unit of production implied by that.
Here’s the IEA’s track record of photovoltaic installation predictions:
Somehow, every single year they predict the share of PV will flatline or even decrease from here on out. And every year they become more wrong at an exponential rate.
China has already started on a decline for CO2 emissions:
This will become more dramatic as time goes on, despite increasing energy use, mainly due to PV buildout (with some nuclear). Hopefully the US can follow, despite certain obstacles.
The llama2 paper includes a section “2.2.1 Training Hardware & Carbon Footprint” that estimates the carbon footprint of llama2 pre-training (the main training process).
The summary is:
A cumulative of 3.3M GPU hours of computation was performed on hardware of type A100-80GB (TDP of 400W or 350W). We estimate the total emissions for training to be 539 tCO2eq, of which 100% were directly offset by Meta’s sustainability program.∗∗
Note tCO2eq is ‘tonne of carbon dioxide equivalent’. That’s ~33M times higher (if I did the math correctly).
They didn’t include similar sections in the papers for llama3 or 4, but assume it is substantially higher. For example llama3 pre-trained for 37M+ GPU hours on H100’s with TDP of 350-700W – worst case is 20x llama2.
Link to the llama2 paper:
Higher than the marginal cost? Absolutely. So then you need to ask how many times a model will be queried, once it’s trained, so you can amortize that cost over the number of queries.
Right. To be fair this is an apples to watermelons comparison. The numbers are from different sized solutions so 33M is a pretty rough estimate. What I thought was interesting is that the training doesn’t dwarf the queries. It is likely the reverse.
Is it?
Is there any estimate for how many queries each iteration of any model has supported to figure out how to either amortize it or alternatively come up with a grand total?
How did you reach that conclusion? (Just wanting to understand the math better)
I didn’t find good, unbiased estimates. This TechCrunch article (which sources from a paywalled Axios article) claims ChatGPT processes 2.5B prompts per day. And claims Google Search processes ~14B searches per day (although it is not clear how many of them trigger Google AI Search – should be a fraction), but there are other Google apps that use their LLMs.
Augmenting my earlier comment: The model card for llama3 405B claims 11,390 tCO2eq for training. They don’t give numbers for llama4 Behemoth, but scaling up from the llama4 Scout/Maverick numbers it might be ~14ktCO2eq.
As an aside they also claim:
Training Greenhouse Gas Emissions: Estimated total location-based greenhouse gas emissions were 1,999 tons CO2eq for training. Since 2020, Meta has maintained net zero greenhouse gas emissions in its global operations and matched 100% of its electricity use with clean and renewable energy; therefore, the total market-based greenhouse gas emissions for training were 0 tons CO2eq.
If we assume the current GPT produced about as much CO2eq, then at 0.03 gCO2eq, 2.5B prompts / day produces 75 tCO2eq /day and would take ~200 days to get to 15 ktCO2eq.
Incorrectly apparently – very incorrectly TBH. At 0.03 gCO2eq, 18B prompts produces 540 tCO2eq which is about equal to the training cost of llama2.