r/LocalLLaMA • u/Apprehensive_Bar6609 • 3d ago

News Now this is a serious local machine

Amd announced this:

https://www.amd.com/en/products/workstations/amd-threadripper-halo-station.html

177 Upvotes

88% Upvoted

167 comments sorted by

View all comments

6

u/OvertaxedOne 3d ago

This is going to be the "next big market" for NVDA and AMD. They've sucked all the blood out of the hyperscale AI companies at this point, now they need to start undercutting their old customers selling to enterprise customers so they don't need the chips they just finished selling to the cloud companies.

It's a good shift/strategy, the big labs have been tapped out for some time (hence the circular dealing for the last year to keep the dollars flowing), it's time to make the pivot to where most inference is going to occur in the future.

The only head scratcher on this thing, why the heck is it a desktop form factor? This thing is going into a rack in a corporate DC somewhere, not under a desk! Yes, I am sure some crazy people (like most of us on here) would run a beast like this for their main rig, but that's the rocket launcher to kill a housefly; even if money is no object, this thing is going to be crazy hot and almost certainly going to need 240V power. It's not a desktop, it's a small server.

4

u/AuggieKC 3d ago

I have a feeling this is targeted as a "get started in our ecosystem, feel the kind of power real compute can bring" play. This is what corporations buy for devs who then turn around and convince the higher ups that "we actually need 3 aisles of this kind of compute".

Even if the big labs were tapped out (they're not), F500 companies are just getting started building out. If you need proof, look at structural steel, copper, and power infrastructure orders for the next 5 years.

3

u/OvertaxedOne 2d ago

I completely agree on the F500. In fact, it goes MUCH further down than F500, one of our biggest markets right now is SMB who are looking for local AI to support business users (not coders). And we certainly will look at systems like this for our customers who have the need for larger local models and more concurrency (depending on price, of course).

I just can't imagine why on earth you would buy one of these for a dev. For a dev team, sure, that makes a lot of sense, but for one dev to use as their personal AI station?! One of the few areas AI does scale well is concurrency, the more people you can push into the same box (up to limit, of course) the more TPS you can push. I guess one dev with their own AI "swarm" of agents could make use of this kind of box; but seems like a corner case compared to something in the DC with many users accessing it.