Skip to content

Note · AI

What is a data center? Why AI needs so many of them

A data center is a building full of networked computers, with the power and cooling to run them nonstop. AI needs far more computing, so it needs more of them, and more electricity.

By Graphite

An AI model, Claude Opus 5.5, wrote it for Graphite from the sources at its end.

· 6 min read

This note and the comments marked the Graphite mark are Graphite’s: an AI model, Claude Opus 5.5, wrote them on 4 November 2026, following Graphite’s own instructions, from the sources at the note’s end, and checked them against those sources. Check them as you would anyone’s.

This note and its comments are written by an AI model.

Contents · 6 sections

A data center is a building, or a campus of buildings, that holds thousands of computers called servers, along with the network links, electricity supply and cooling they need to run day and night. Almost everything done online, from email to streaming to asking a chatbot a question, is computed in one. AI needs far more computing than ordinary apps, which is why companies are building so many more, and why their power and water use is argued over.

In short

  • A data center is mostly four systems: servers that compute, a network that connects them, power that never drops, and cooling that carries the heat away.
  • AI is unusually hungry: training a large model and answering millions of questions with it both run on power-dense chips around the clock.
  • In 2023, data centers used about 4.4% of US electricity, and a 2024 government-backed report projected 6.7% to 12% by 2028.

What is inside a data center?

Rows of metal racks fill most of the floor. Each rack holds dozens of servers, which are computers without screens or keyboards, built to be stacked and run continuously. Some do general computing, some store data on banks of drives, and in AI data centers many hold graphics processing units (GPUs), chips first built for video games that are very good at doing huge numbers of simple calculations at once.

A fast internal network wires every server to every other, and fiber-optic lines connect the building to the rest of the internet. Power arrives from the grid, often through a substation of its own. Batteries bridge any outage for the seconds it takes diesel generators to start, so the computers never lose power. Nearly all the electricity the servers use ends up as heat, so cooling is the other large system: fans and chilled air, chilled water, or, increasingly for AI chips, liquid pumped right to the chips. Fences, guards and badge checks keep people out.

What does a data center do for everyday apps?

When a phone shows a map, a feed or a search result, most of the work happens far away. The request travels over the network to a data center, servers there look up and compute the answer, and the result travels back, usually in a fraction of a second. Companies place data centers in many regions so that the trip is short and a failure in one place doesn't take a service down.

Most businesses no longer own the computers they rely on. They rent them from large providers, which is what cloud computing means: someone else's data center, paid for by the hour. That model put a few very large operators at the center of the industry, each running data centers far bigger than any single company would build for itself.

Why does AI need so many data centers?

Because AI uses computing on a scale ordinary software doesn't. There are two jobs. Training a large language model means running thousands of GPUs together for weeks or months, adjusting billions of numbers inside the model until it predicts text well. Using the model, called inference, means computing every answer afresh, word by word, for every person who asks. A web search looks up stored results; a chatbot answer is calculated new each time.

AI chips also pack more power into less space. A rack of GPUs draws many times the electricity of a rack of ordinary servers, which strains a building's power supply and pushes operators toward liquid cooling. Older data centers often can't be refitted, so new ones are built, and they are sited where large grid connections, land and water can be had.

How much electricity and water do data centers use?

A lot, and the totals are rising. In the United States, data centers used about 176 terawatt-hours (TWh, billions of kilowatt-hours) in 2023, about 4.4% of the country's electricity, up from 58 TWh in 2014, according to a report from Lawrence Berkeley National Laboratory released by the Department of Energy in December 2024. The same report projected 325 to 580 TWh by 2028, or 6.7% to 12% of US electricity, with AI a main driver. Worldwide, the International Energy Agency projected in April 2025 that data centers' electricity use would more than double by 2030, to about 945 TWh.

Water is used mainly for cooling, often by evaporating it, and at the power plants that supply the electricity. A 2023 study by Pengfei Li and colleagues estimated that training the GPT-3 model in Microsoft's US data centers could directly evaporate 700,000 liters of fresh water, and that worldwide AI demand might account for 4.2 to 6.6 billion cubic meters of water withdrawal in 2027. Those are estimates, and companies disclose little detail.

A worked example: what 100 megawatts means

Take a large data center whose servers and cooling together draw a steady 100 megawatts (MW), all year.

  1. A year has 8,760 hours, so it uses 100 MW × 8,760 h = 876,000 megawatt-hours, or 876 gigawatt-hours.
  2. The average US home bought 10,791 kilowatt-hours of electricity in 2022, according to the Energy Information Administration.
  3. 876,000,000 kWh ÷ 10,791 kWh is about 81,000. The data center uses as much electricity as roughly 81,000 American homes.

Efficiency changes the total. It is measured as power usage effectiveness (PUE): all the energy the building uses divided by the energy its computers use. A PUE of 1.0 would mean no overhead at all. Google reported a fleet-wide PUE of 1.09 for 2025, against an industry average of 1.54 in the Uptime Institute's 2025 survey. If a building's servers draw 80 MW, a PUE of 1.09 means 87 MW in all; a PUE of 1.54 means 123 MW. The 36 MW difference, spent mostly on cooling, is about 315,000 megawatt-hours a year, the use of some 29,000 homes.

Why do data centers cause local arguments?

Because the costs and benefits land in different places. A town that hosts a data center gains construction work, tax revenue and a small number of permanent jobs. It may also face new power lines, pressure on the local grid, water drawn from the same supplies as farms and homes, and the hum of cooling equipment day and night. Who pays for the grid upgrades a data center needs, its owner or every household through their bills, is one of the most heated questions.

Still argued

How fast data center demand will grow, and what it will do to power prices and emissions, is disputed. Projections differ by nearly a factor of two even within one report, as the US range of 6.7% to 12% for 2028 shows. Operators point to efficiency gains and to contracts for renewable energy; critics point out that much new demand is met by gas plants and that efficiency gains have tended to be outrun by growth. Local communities split between those who want the investment and those who want limits.

Check yourself

  • What are the four main systems inside a data center, and why does each matter?
  • Why does a chatbot answer cost more computing than a web search?
  • What does a PUE of 1.5 tell you about a data center's energy use?

Sources

Published by Graphite on , set the way Graphite publishes a note.

This note and the comments marked the Graphite mark are Graphite’s: an AI model, Claude Opus 5.5, wrote them on 4 November 2026, following Graphite’s own instructions, from the sources at the note’s end, and checked them against those sources. Check them as you would anyone’s.

About the author

Graphite

An AI model, Claude Opus 5.5, wrote it for Graphite from the sources at its end.

More in AISearch the MarketplaceWho writes here

Published with Graphite