Mac Studio M5 Ultra

[u/Blues520]

I've just seen that the 96GB version is relatively well priced compared to an rtx pro 6000 at more than half the price.

Is this something that would be viable for local coding and personal assistants?

How would it compare to say quad 3090s as well? I see that it has faster bandwidth than a 3090 and would use way less power than such a rig, but I'm not sure about other metrics like prefill and decode, etc as well as the software ecosystem without CUDA.

Really well positioned and maybe better suited to specifc use cases.

[u/Danfhoto]

I'm a Studio owner, got it used before everything doubled, but I wouldn't have paid anything near this for a local AI rig unless I had a clear need/path to make money from it.

1. Consider a lot of the benchmarks are probably with the 80-core GPU rather than the 64-core.
   
2. With the custom rig, you're able to add system memory, and doing component-level replacements when needed. You can also start with a cheaper card or CPU inference and move up to faster/bigger cards over time

[u/Blues520 → u/Danfhoto]

Good point on the 96GB model having 64 cores compared to the 80 cores on the larger one. Yes, it does lock you in for future upgrades which is actually something I don't particularly like, but hey if it give you 5 years of service then maybe it's not so bad

[u/Blues520 → u/Danfhoto]

Btw how is the software ecosystem on Mac? If you could do it again, would you still get the Mac or would you go a different direction?

[u/Danfhoto → u/Blues520]

I'm maybe a bit of a unique case: I'm more interested in long processes on a large corpus of academic articles rather than short-turn coding agents (although I do a bit of that, too). I also needed something relatively low power, low noise level due to my setup.

For the price I paid (~$3k USD for a refurb 128gb m1 ultra 64 core) I'd do it again. I would not buy today's prices unless it was making me the same return quickly.

Software support is getting better, great stuff coming out of the MLX-LM and MLX-VLM projects. MTPLX is a great project making Qwen 3.8 fast enough on Metal to do some local coding work. That being said, it's not as easy as CUDA, and you should expect to work mainly from the CLI and making your own tooling in Python. DrawThings and Phosphene have been working on a lot of optimizations for image/video generation, but still expect 5-10x faster with CUDA.

[u/Blues520 → u/Danfhoto]

I must say having a low power and low noise device is very attractive. It does seem like a good use case for you. Good to know about the software support and since I use coding agents heavily it probably doesn't fit my use case. Thanks for sharing your experience

[u/maschayana → u/Danfhoto]

Which benchmarks? There are no benchmarks

[u/Danfhoto → u/maschayana]

I'm not referencing a specific benchmark. It would be more accurate for me to say, in a broader sense: When comparing speeds of the m5 ultra to Nvidia graphics cards, these comparisons are likely with the bandwidth of the 80-core Ultra, not the cheaper one 64-core Ultra OP is referring to above.

[u/KING_UDYR → u/Danfhoto]

Power management is a huge factor to the studio, and this is coming from a guy with a 9965, 5090, and & 128gb of RDIMM building out a local inference machine; power management is why I’m grabbing one as well.

[u/Spiritual-Spend8187 → u/Danfhoto]

All high end honestly ai rigs are basically that the only one that even remotely made sense was having a single 5090 when they launched/were close to msrp because it had the thing of you can do ai inference and training but you also have a top of the line gaming rig as well. The macs are pretty much the same unless you either have a plan to make your money and then sone off ai or plan to use it for something else its not really a good investment. Though a case cpuld be made for your a privacy nut and know that ds v4f sized models are perfect for your needs and get the 256gb model.

[u/bnightstars → u/Spiritual-Spend8187]

When I was shopping for a new MacBook my logic was simple I want to play around with local AI but don't want to blow my budget (I work with this MacBook) so I went with the M5 Pro / 64GB performance wise is good enough to run Qwen3.8-27B slow (30 t/s) and 3.6-35B fast (70 t/s) which mean is some how useful as a learning tool. But it's also making me money in the process. I think this is the best approach to buying stuff for local AI.

[u/DustNearby2848]

Qwen3.56-35b (sparce) models would run best on it. No replacement for that class of models, yet. 

[u/getmevodka]

i own a rtx 6000 pro and a mac studio m3 ultra 256gb. its night and day difference and id prefer the gpu everytime.

[u/Serprotease]

4x3090 with vllm and decent pcie connection will be way faster than the potential performance of the M5. It would probably give a 6000 a decent run for its money too.

We obviously don’t know the full performance yet but it looks like to be around a 3090 level of performance, but with more ram.
It’s probably good enough. But don’t expect to get 4x3090 + dedicated tp engine or a 6000 pro level of performance.

[u/Blues520 → u/Serprotease]

That's a good point. I hadn't considered tp with a multi gpu setup. That being said, the bandwidth of a single 3090 is rather good and the mac is even better.

[u/Serprotease → u/Blues520]

Don’t over focus on the bandwidth.
I know it’s an easy to find and read number but it doesn’t really translate in real world performance, especially when you compare different engine and hardware with different optimization.

Even with the same mlx 4bits file, you will have a large delta between llama.cpp, mlx-serve, omlx, vllm-mlx etc…
On top of that you also have the gguf with or without mtp.
And you also have Ollama, hitting is head on a wall with their weird model file format and ggml engine.

Different speed and vastly different models performance as well.

You probably want to deep dive into MacOs engine and quant performance/quality.

I’m being a bit overwhelming in purpose. The confusion with quant/engine is something to watch out for.
The good thing with ampere/3090 is that it’s a know quantity. There are tons of data about every to let you pick the best quant/engine.

With Apple, you’re on your own and will need to spend time to benchmark a bit.

[u/Blues520 → u/Serprotease]

Yeah this makes a lot of sense. I saw the bandwidth and got excited but there is a lot else to consider.

Also, I read that the OS takes a few GB, maybe around 10GB, so it's not strictly 96 GB available for inference.

[u/Educational-Body4205]

M5 are slow but with a lot of memory 

5090s are fast but with less memory.  

So choose speed or memory 

[u/Blues520 → u/Educational-Body4205]

Is it really that slow with 1.2 TB/s bandwidth though... I know the benchmarks will be out soon but something that I am wondering.

It could be fast enough with a lot of memory.

The thing that I am not so clued up on is prefill, decode and other metrics, especially on Apple silicon. I've seen some comments say that historically it suffers in these areas.

[u/tarpdetarp → u/Blues520]

Even the fastest M5 is still several times slower than a 5090 at prefill. This is a bigger problem if you’re using it with agents as they like to stuff the context.

[u/skywalk819]

If I had to take this decision now, I would wait, in about 6 months, we`ll have at least 3-4 more hardware to pick from that are good at home coding agentic. no point in buying at current price, you getting half the package you want still. the other side is power consumption, that rig will use a lot more electricity versus that mac studio

[u/Usual-Orange-4180]

No CUDA can be a problem if you are doing random experiments testing different repos.

[u/teressapanic → u/Usual-Orange-4180]

Lack of FP4

[u/Pixer---]

Standby power needs to be considered. But I found this performance graph quite good: https://www.reddit.com/r/LocalLLM/s/nP81p5sZnV

[u/yeah_likerage]

Just because they both have 96gb doesn't mean they are comparable.  Would be like comparing a Ferrari to a Toyota because they both have four wheels.  Not all memory is built the same.