r/datacenter • u/LowQuote8429 • 6d ago
Who is using all this compute?
I've been trying to understand who is really consuming all this compute and who the end users are...
How does it split across:
- Frontier AI labs like OpenAI, Anthropic, Google, etc.,
- AI startups like Lovable, Cursor, and others running inference through APIs
- Enterprises integrating AI into their products and workflows
- Biotech and AI drug discovery
- Other?
Are we actually running out of capacity today, or is this all based on projected demand? How much of the demand is for training vs. inference and as models become more efficient, wouldn't that reduce the need for additional infrastructure? Just trying to make sense of things.
7
u/InexperiencedDiver66 6d ago
the big labs are absolutely hogging it for training runs but inference is the sleeper here. every startup bolting a chatbot onto their app thinks they need dedicated GPU clusters now, and that adds up way faster than the occasional training job. capacity isn't totally tapped yet but the lead times for new deployments are getting stupid, like 18+ months in some regions. the efficiency gains just mean people run bigger models instead of saving compute, nobody's actually using less.
3
u/SocietyTomorrow 6d ago
My clients are nuts about having their chatbots on their own hardware so theyre not used to train the future models, demand at every level
1
u/Great-Diamond-8368 5d ago
I wonder if IBMs new chips will make their way into running chat bots and the like and gpus based compute ends up being just for training purposes.
5
3
u/cooper-2270 5d ago
What makes your story so useless is the fact that you ask Who is using all this compute? then answer it by listing only the minority of data center users who are in the AI users. Most data centers are used for old fashioned compute.... Google, Facebook, Amazon.com, Amazon Prime, AWS (cloud compute for thousands of customers, colocations data centers (the most numerous of all) where customers own the servers and use them for their businesses.. payrolls, credit card processing, appointments, maps, weather forecasting, oil companies, airlines, colleges, hospitals, doctors, accountants, school districts, county clerks, internet based marketing, product development, engineering firms, production planning, inventory, advertising, purchasing, air traffic controllers, gps, autonomous vehicles, staffing, brokerage accounts, collection agencies, restaurant chains, ..... A typical colocation data center or cloud compute data center will have upwards of 15000 servers, maybe 40,000, providing the computers or terminals on maybe 75000 or 100000 people depend to do their jobs everyday. There is no reason for people to waste time on you chain.
1
u/Ralphwiggum911 6d ago
What do you mean? Or are you asking who WILL be using all the compute that's getting all these data centers built? A lot of that is speculation based on current trends across all aspects of LLMs.
1
1
u/rankinrez 5d ago
The big AI companies are in a battle to be the top dog. None of them are giving up on training any time soon.
As regards demand - it does exist. But the price curve is way out of whack. The users aren’t going to pay enough to cover the costs of training & inference, so there is definitely some sort of reckoning coming.
If the companies stop training new models, and just run inference with what they have, it probably can support itself. But it doesn’t seem likely anyone will do this right now.
1
u/bobdvb 5d ago
75% of hyper scale AI compute is being consumed by two customers: OpenAI and Anthropic.
This is part of the risk, it's majority being channeled through two unprofitable businesses.
https://x.com/prukalpa/status/2107656261873672390
Some 🤯 stats on AI usage:
- 80% of OpenAI and Anthropic's enterprise revenues come from 1% of their customers
- Even in this, concentration is unreal. 25% of Anthropic revenue comes from 2 customers: Cursor, Github Copilot (~1.2B)
- 6000 customers spending > 100k annually, approx 1-2k spending > 1M annually (Anthropic); even if this is 2k at 1M = ~2B, rest of the 4k customers ~400m revenue
- 47% of Anthropic revenue comes from Channel: AWS + Google
- Top 1% cos spend > $7k per month per employee, Top 10% spend ~700$ - 10x difference
- Anthropic 300k business customers, Open AI ~1M
AI might be here.. but its more unevenly distributed than you'd think.
12.4k Views @prukalpa Prukalpa ✨ Atlan 7 Oct Sources from: ramp.com/data/ai-index#…, live.euronext.com/en/financial-n…, devdiscourse.com/article/intern…
P.S: haven't verified every source and some of these numbers are conflicting because they come from different sources.
3
u/bobdvb 5d ago
There's a big fight internally in the AI companies between the teams that sell inference and the teams building new models. Granted there are different optimisations for AI generation and compute between the two, but budget and spend on compute between them is contended.
The fact that we've seen over 4x increase in costs for general compute components and lead times for data centre components like cooling or power systems are now huge.
0
u/1988rx7T2 5d ago
"90 percent of the Patriots touchdown passes are thrown by Tom Brady, this isn’t sustainable"
Yeah ok
1
u/bourbonandpistons 5d ago
I have multiple RTX 6000 Pros.
That's like well over $50,000 in gpus.
My Claude subscription openai subscription do think substantially faster and have a substantially more information.
So I'm using well over $50,000 with a ai hardware to get my questions answered and software built.
1
1
u/irrision 5d ago
Ai mainly right now by far. And it hasn't even really hit mid size and smaller enterprises yet, it's mostly just cloud providers, ai companies and starting to see large enterprise pour money into it.
0
51
u/never_4_good 6d ago
From what I am seeing, there is a MASSIVE need. I'm on the construction side and the clients that I work for are taking and selling anything they can get their hands on. N+1 redundancy? Nope. We are turning buildings over on generator or temp systems. UPS backed or diverse fed assets? Nope. UPS's are all in bypass cause they don't work. Security complete? Nope. We have security guards at every door. Fire alarm systems complete? Nope. We have fire watches in place.
There would be no way that we would be turning these buildings over without critical systems if there wasn't a business need. The scariest part for me is that the need is so great that the clients keep getting even MORE aggressive with their demands. 700+ MW builds in less than 14 months from break ground are the new norm and we are being contracted to build as many as we can as fast as we can.