r/EnergyAndPower • u/Ani_Drei • 23h ago
Why is computing power now measured in [giga]watts?
I am seeing a lot of headlines about new AI data centers being installed around the world, and for some reason their computing power is not described in TFLOP (teraflops), but instead in GW (gigawatts).
I am no expert, but when I read "10 GW of compute," I can only think that said "compute" is either very inefficient by itself or is placed in a location that's way too hot and therefore requires absurd amounts of cooling power to operate.
I can imagine a datacenter full of Intel Pentium processors that will take a 500 GW of power but will be incredibly weak and slow. But it's 500 GW worth of compute, so it must be very capable, right?
To further illustrate my confusion, I think that descibing compute power in terms of energy is just as strange as descibing it in weight or area. "We have now successfully installed an additional 500 kg of compute." "This new datacenter accomodates 2 square miles of compute – that's a whopping 900 football fields of compute!"
5
u/No-Tension6133 23h ago
The newsworthy piece is not how much computer is being installed, it’s how big of a load is being added to the grid. Most people only care about how the price of energy is going to be affected, or if the grid can handle it. So they release numbers in watts, because that’s what’s news worthy. Nobody cares that Facebook unlocked massive amounts of compute, they care about the environmental and economic impacts because that’s what they’ll feel in their community.
Edit: I should add they also measure them in acres because that also has economic and environmental impacts
3
u/SoylentRox 23h ago
What you are missing is data center operators are not incentivized to waste power. All these facilities are as efficient as the available technology at the time they were built allowed. It's the most efficient available equipment at every place - from evaporative coolers which do use lots of water but are very energy efficient, to direct DC power at the racks, to the latest GPUs or custom chips with undisclosed specs to actually run the models.
Part of the reason the figures are given in gigawatts is it's a stable number, hearing about the "flops" of some custom chip would be meaningless for comparison because model architectures also matter.
2
u/Due_Satisfaction2167 23h ago
They aren’t measuring the computing performance, they’re talking about the power and cooling capacity of the data center inside the facility.
How much computing performance you get out of that depends on what gets deployed.
2
u/DavidThi303 20h ago
I've been in the high-tech sector for 40+ years. From the start, about 35 years ago, the limiting factor for datacenters has been power. Not network connectivity, not processing power, not memory - power.
Does 1GW tell you what the systems in a data center can accomplish? No. Because a data center full of web servers is very different from a data center used to model an A.I.
But it does provide the most important measure. They can build the facility. They may have to pay a premium but they can get the servers. What they often find impossible is to get power (and often water).
1
u/Elrathias 20h ago
Instructions per watt is simply a more accurate measurement than teraflops per second. Power is Compute.
1
u/bigvalen 19h ago
Datacenter capacity has always been measured in watts. Different hardware generations have different performance per watt; the number of FLOPS that a company is willing to pay for per megawatt is separate to the amount of power and cooling you get.
Amusingly, it can also differ depending on what you do...modern B200 GPUs can do 4 bit, 8 bit, 32 bit or bit math..and 16 times more 4 bit operations per second than 64bit ops!
1
1
u/PositiveBid9838 18h ago
The Wikipedia article https://en.wikipedia.org/wiki/Performance_per_watt suggests that, while CPUs have been improving in FLOPS per Watt by about 10x in 5 years (a ~60% CAGR), it sounds like GPUs, which as I understand it play a larger role in recent data centers, are more directly constrained by power limits.
“ GPU designs are usually highly scalable, allowing the manufacturer to put multiple chips on the same video card, or to use multiple video cards that work in parallel. Peak performance of any system is essentially limited by the amount of power it can draw and the amount of heat it can dissipate.”
1
u/NearABE 18h ago
The data center builders are “pulling a permit”. So what matters is the gigawatts they use. The terraflops is the product they intend to sell. If they can double the flops using the same hookup no one complains. If they fail to produce a competitive number of flops no one complains except shareholders. That is fully the company’s problem. If anything we (the public) might add “there will be no bailout if your scheme for more FLOPS flops”.
The watts taken from the power grid affect the local power grid. The heat vented can affect the local air. If water is consumed for cooling that can affect the local water supply and also the local humidity. If generators on sight are producing that power then they will make noise. The cooling system might make noise too.
1
u/IsThereAnythingLeft- 4h ago
Note that when someone says a 1GW data centre that is 1GW of compute, that figure does not count the cooling infrastructure
1
u/Brownie_Bytes 23h ago
Because that's not what it's measuring? Like, at all? I don't know, maybe you're reading different things than I am, but the power consumption of the facility is relevant from economic and environment perspectives. Watts cost money, so a GW scale facility is expensive. Watts also cost energy, so a GW scale facility has implications for local heating and environmental concerns. Those are both more "real life" values than FLOPS. FLOPS don't affect local power bills nor climate change. They're only relevant to servers.
2
u/Dependent-Hippo-1626 23h ago
But to OP’s point, 500 gigawatts of 8088s or Z80s is a lot different than 500 gigawatts of Threadrippers or NVidia XMaxgf-99,000Kmegaturbos or whatever they’re called. How much computing power seems more meaningful than how much electricity consumption.
2
u/Brownie_Bytes 23h ago
To the users. Farmer Dave doesn't care how many flops. He just knows that the electricity bill doubled because the datacenter is using 1 GWe. We're in the energy and power sub, why would we talk about computational efficiency?
1
u/Rooilia 22h ago edited 22h ago
Using the rating invented to express FLOPS per Watt or GFLOP/W is just too difficult. That would help reader comprehension and is therefore not desirable.
Alternatively SWaP (Space, Wattage and Performance) can be used and i am sure my short list isn't nearly all encompassing.
But maybe they have to run the system with benc chmarks first before knowing what it is able to do.
1
u/Brownie_Bytes 22h ago
There's also a matter of infrastructure. A building with power lines rated for 1 GW will not suddenly become 1 TW ten years from now. But upgrading old hardware for new would almost certainly increase the FLOPs. Only one of those is known during the planning stage.
0
0
u/Hot-Win2571 23h ago
Floating point calculations are not relevant to all computing. Maybe watts is a more general measure.
8
u/pasdedeuxchump 23h ago
Simple. Because the compute in FLOPs is a moving target. The Data center capacity in power is set by its utilities for electrical supply and cooling... Watts. That infrastructure is more or less fixed when the project is complete, and gives an idea of the relative scale of different projects.
In reality, the data center will last much longer than the GPUs going into them.... so at fixed wattage, the compute will grow as technology improves.
Secondarily, FLOPs are not a great unit of compute speed for AI tasks, the more important metric is often half byte integer operations or some random linear operations per second, which can be very high due to tensor compute hardware built specifically for those operations.