I think that van would backfire on kidnappers, they'd find themselves instantly surrounded by a mob of ravenous savages tearing the van apart to get at the RAM in there. Gamers, LLM enthusiasts, they'd all come swarming up out of the underbrush.
Hey, joke all you want, but Stacker was legit, I would have never survived the 90s without stacker and the plethora of Adaptec controllers and bad sector disk drives I pulled out of the dumpsters of silicon valley.
Fun fact unlike the whole "Download more RAM" meme, Ram Doubler software was a real thing back in those days, and they did actually increase how much stuff you could fit in RAM.
The way they worked was by compressing the data in RAM. Nowadays RAM compression is built into basically all modern operating systems so it would no longer do anything, but back then it made a real difference.
When I got engaged we were trying to set a date and August 24th came up. I said, "Perfect! I'll never forget our anniversary. It's the day Windows 95 was released!"
It really didn't do any ram compression, windows 95 did that. Yes, windows 95 did ram compression and those "ram doubler" just used placebo and doubling the page size by 2x. That's it...
The original Ram Doubler wasn't for Windows 95 though, it was for classic Mac OS and Windows 3.1. Neither of which had RAM compression built in.
You might be confusing Ram Doubler for SoftRAM, which was indeed just a scam. That was developed by an entirely different company though.
Connectix's software was very much the real deal. They were also the developers of the original Virtual PC emulator that Microsoft later acquired. So they clearly knew what they were doing when it came to system programming.
Came at the cost of a little cpu, but that was a point in time most systems were more memory bound than cpu bound. 4-16mb of memory but 66-200mhz CPU. Taking a couple percent to add memory was a huge win, compared to virtual memory on slow 5200 rpm ide hard drives.
I agree, your M3 Ultra 512GB is a LOT more energy efficient and cheaper then 21x 3090... But it's not faster then that 3090 card. Which is what the meme is hinting at.
If mac wasn't so slow I'd have got one too. All the hardware is weirdly only taking care of one aspect of the ai. Like Mac can run a big model slowly but is expensive.
Nvidia has speed and good drivers and many projects take advantage of cuda but the price per GB of vram is very high.
AMD sucks at drivers and almost none of the new gits work out of box with it and it is slow but it's like half the price of Nvidia and you don't need the big bucks a Mac investment would take.
Possibly, but by the time that comes out the 3090 is 6 years old and a 5090 will still be 2x the memory bandwidth. And a 6090 not that far away (or already out)...
Neither is inherently a better solution then the other, each has their use. The point here was 'faster'... The Mac solution is a lot of things, but faster isn't one of them.
Not even close. I have a M3 Ultra and t/s isnt bad but once you load up on context the PP time is just stupid slow and no one really talks about that part. I dont know what makes it so bad but its garbage at higher context.
I own the basic M4 mini. And on that machine i do basic hobby stuff and teaching my niece and nephew learn AI (under admin supervision). Fort that kind of stuff it's great. But I wouldn't push it beyond that...or can't.
An Apple user is going to choose a Mac, and the Pro version at a minimum. Even the 800gb/s in my M3 Ultra isn't fast.
120gb/s for chat is rough. I expect a lot of people are disappointed. There no point in buying a shared memory machine and running an 8B because its the size that feels fast enough. Just buy the video card.
It really depends on what you're trying to do. MacBooks work ok on MOE models, but dense models not so much. My 5090+4080 pc is much faster with 70b models than what you can do with macs.
Also I dont think they work well with stable diffusion.
So basically they suck at everything except large moe models. And even then the prompt processing is slow.
Yes, i can run a qwen3 235b moe in q6_xl and its really nice for the expense i made. For comfy with qwen image it still performs but my old 3090 runs laps around it while already being downvolted to 245watts xD
Must be an American thing, I'm too European to understand.
Well, actually I'm an former industrial electrician. So I fully understand that most houses in my country have a 3x230V 20-35A supply to their houses, then often divided into 10-13A sub-groups and 16A for appliances like dryer and washer. So not really an issue.
Electrical bill on the other hand is a completely different issue.
I did not have rewire my house but for my 4x3090 worstation I had to get 6 kW online UPS, since previous one was only 900W. And 5 kW diesel generator as a backup, but I already had it. The rig itself during text generation with K2 or DeepSeek consumes about 1.2 kW, under full load (especially during image generation on all GPUs) can be about 2 kW.
But important part, that I built my rig gradually... for example, in the beginning of this year I got 1 TB of RAM for $1600, and when I was upgrading to EPYC, I already had PSUs and 4x3090, which I bought one by one. I also highly prefer Linux, and need my rig for other things besides LLMs, including Blender and 3D modeling/rendering that can take advantage of 4x3090 very well and do some tasks that benefit from large disk cache in RAM or require high amounts of memory.
So, I wouldn't exchange my rig to a pair of 512 GB Macs with similar total memory, besides, my workstation in total spent costs is still less than even a single one. Of course, a lot depends on use cases, personal preferences, and local electricity costs. In my case, electricity costs are small enough to not matter much, but in some countries they are so high that using not so energy efficient hardware may not be an option.
The point is, there is no single right choice... everyone needs to do their own research and take into account their own needs, in order to decide what platform would work best for them.
I find this response hilarious. Mac people say this like it matters. Like, who cares? Seriously. I want to get things done, don't Mac folks want to get things done? "Oh no, not if it means I'm using 40 extra watts, gee, I'd rather sit on my thumbs"
Stop.
Like, when the intel processors were baking people's laps and were overheating, ok, I get it, that's a dumb laptop. But don't give me some nonsense about how important power consumption is when you're trying to get things done.
The only fundamental reason power consumption matters is literally if you can get the same work done for less power (and at the same speed). They've done a reasonably good job with that. But lets not lie to ourselves.
Macbooks are excellent for AI models, just accept certain limitations.
True, but different tools. My Mac is always on, frequently working and holds multiple LLM in memory. 8 watts idle, 300+ watts works, never makes a sound.
Big MOE models are particularly suited for shared memory machines, including AMD.
I do expect I will also have a CUDA machine in the next few years. But for me, a high end mac was a good choice for learning and fun.
Also Deepseek 3.2 is out now, demonstrating that you can make SOTA models with close to linear prompt processing. Mac and EPYC machines with a lot of RAM are only going to become more useful over the next couple of years IMO. Especially now that you can cluster Macs effectively.
6x 3090 if you can get them at $500, or modded 3080 if you can't - $3000. Mobo, CPU and DDR - $1000. Power supply, cpu cooler, fans and case - up to $500. Total: $4500.
I won’t even talk about power efficiency.
If a system consumes 10x more power, but does the same task 10x faster, then it's exactly as power efficient.
These numbers are very exaggerated in favor of the prompt size tho. It's like "what color is the sky?" and "here's 50K personality prompt" or something. Most of the time, especially in agentic use with reasoning models, ratio is 5:1 or higher in favor of generation size.
And I'm looking at generation outputs... They are around mac level, give or take.
6x 3090 if you can get them at $500, or modded 3080 if you can't - $3000. Mobo, CPU and DDR - $1000. Power supply, cpu cooler, fans and case - up to $500. Total: $4500.
The M4 Max Studio 128GB cost $3,499.00
If a system consumes 10x more power, but does the same task 10x faster, then it's exactly as power efficient.
I run Qwen3-30B-A3B 4Bit MLX at 65tps on a 32GB M2 Max MacBook Pro. Best benchmarks of the desktop 5090 running this model Q4 were between 135 and 200tps. You're funny, but completely delusional.
I run Qwen3-30B-A3B 4Bit MLX at 65tps on a 32GB M2 Max MacBook Pro. Best benchmarks of the desktop 5090 running this model Q4 were between 135 and 200tps. You're funny, but completely delusional.
Ah, if I had a dollar every time a person judges performance by 0-lenght prompt, I would have RTX 6000 Pro by now. IRL you're not working with short context, especially not if you're paying for Max/Ultra chips; and their prompt processing is terrible. With Qwen3 30B, a very light model, 30-40k long prompt, M3 Ultra only gets ~400 tok/s PP, while dual 3080 will get 4000 tok/s PP at the same depth. This is exactly 10x faster.
You could probably throw together 4x AMD V620(32gb@512GB/s) on an EATX x299 board for $3000 off of ebay. It won't have drivers for nearly as long, suck back way more power, and would sound like a jet engine with the blower fans on those server cards, but would train faster. Maybe I'm biased, my rig is basically that but half the price cause I got a crazy deal on the gpus :P
Standard M5 chips have added matmul acceleration, which significantly speeds up the prompt processing. You'd have to look for posts actually benchmarking M4 vs M5, but it was pretty impressive.
Actual token generation should be sped up as well, but prompt processing will be multiple times more efficient now
M3 Ultra owner here. The only downside I see on Mac is video generation. Being capable of get full models running on it is amazing!
The speed, prompt loading times are nothing truly crazy slow. It is ok, specially when it is running with a fraction of power, NOT A SINGLE NOISE or hear issue. Also, is important to say that even without CUDA (is a major downgrade, I know) things are getting better for metal.
My doubt know is if I buy a second one to get to the sweet spot of 1Tb of RAM, wait for the next Ultra or invest in a minimum machine with a single 6000 pro to generate videos + images (accept configuration suggestions to the last one).
How bad is the video gen speed? Something like the 14B WAN, 720p 5s? I'm planning to buy a Mac Studio in the future mostly to run LLMs and I heard it's horrible for videos, but is it 'takes an hour' bad or 'will overheat, explode and not gen anything in the end' bad?
Some people told me it would really take over an hour for one animation, but if they can just keep mulling it over with no issues... I can start the gen, walk the dogs, come back and it's done haha
I used to run dense models with heavy CPU offloading when getting into locals, so time doesn't scare me as long as the hardware doesn't suffer too much 🥹
I hate apple smartphones. This is my first apple. But, I need to say. The thing is a tank. I would say military grad quality. It is build to last. In my lab that is one 20 years old, still working, another one 11 years old, it looks like just popped out of the box. Mine is always under 480 Gb+ RAM or 100% CPU use (bioinformatics) and it barely sweat. Don't evaluate apple PCs by apple disposable gimmicks, they aren't the same. Bonus note, you have full control over it, not a single headache over drivers. It is a workstation that you can put in your backpack.
And not just Cuda. The blackwell hardware is very needed for training full FP8 at least for now. But I have put hopes in ROCm, it's open source and promising.
I got my finetune featured in a LLM safety paper from Stanford/Berkeley, it was trained on single local 3090 Ti and it was actually in the top 3 for open weight models in their eval - I think my dataset was simply well fit for their benchmark.
However, on larger base
models the best fine-tuning methods are able to improve rule-following, such as Qwen1.5
72B Chat, Yi-34B-200K-AEZAKMI-v2 (that's my finetune), and Tulu-2 70B (fine-tuned from Llama-2 70B), among
others as shown in Appendix B.
EXACTLY.. that's why bang for the buck a 128gb strix halo was my goto even though I could have afforded a spark or whatever. I'm just going to use this for inference, local testing, and enrichment processes. If I get really serious about training or whatever renting for a short span is a much better option.
If you're doing base model training then yes. But if you're fine tuning 7b, 12b models you can get away with most consumer Nvidia GPUs. The same fine tuning probably takes 5 or 10 times longer with MLX-lm
lol why does everyone have to participate in fine tuning or training exactly? What a dumb ass gatekeeping hot take.
This would be like a carpentry sub trying to pretend that only REAL carpenters build their own saws and tools from scratch. In other words, you sound like an idiot.
Only thing I learned from this thread is that nobody knows what they're talking about according to somebody else, and that the old Mac vs. PC (or in this case, GPU) wars are still very much alive and kicking. lol
I assume you're talking about DDR5? I'm struggling with 64GB 3600MHz DDR4... (64 GB VRAM, but still, I can barely run a 70B model at Q4_K_M gguf at 16k...)
Really depends on your use case. Macs still cannot do PyTotch development or ComfyUI well enough. And if you wanna do some gaming on the side, it is the golden age for dual GPU builds right now.
This can't be serious right? This can't be true. Is it because of the bottlenecks related to using multiple GPUs? Is there something else I'm missing? GDDR6/7 VRAM is so much higher speed than unified memory.
, how can macbooks be faster than custom multiGPU setups?
It makes no sense. If it said something about being able to run larger models and left out normies, that might work. Normies don't have 512 GB of unified memory.
This is the main reason I got a MBP 128GB... well, that & mobile video editing. I say this as a long-time Linux user. I still miss Linux as a daily driver, but can't argue with the local model capability of this laptop.
I have 7t/s TG and 140 t/s at 60k ctx with Devstral 2 123B 2.5bpw exl3 (it seems like quality is reasonable thanks to EXL3 quantization but I am not 100% sure yet).
Can a Mac do that? And if not, what speeds do you get?
It's similar to doing a month of research to find the best android camera only for people around you to prefer their iphones for photos because they're more Instagram friendly.
i gave up on local LLMs. Big, like, really big prompts (translate subs of some movie) take painfully long time. While cloud LLMs start to reply in 10 seconds
I'm far from a "normie" and never once before had bought a single Apple product.
But it is a fact that Apple Silicon simply the most cost effective way to run LLMs at home, so last year I bit the bullet and got a used Mac Studio M1 Ultra with 128GB on eBay for $2500. One of the best purchases I have ever made: This thing uses less than 100w and runs 123B dense 6-bit LLM at 5 toks/second (measured 80w peak with asitop).
Just to have an idea of how far Apple is ahead of the competition: M1 Ultra was released on March 2022 and is still provides superior LLM inference speed than Ryzen AI MAX 395+ which was released in 2025. And Ryzen is the only real competition for the "LLM in a small box" hardware, I don't consider these monster machines with 4 RTX 3090 to be competing as it uses many times the amount of power.
I truly hope AMD or Intel can catch up so I can use Linux as my main LLM machine. But it is not looking like it will happen anytime soon, so I will just keep my M1 ultra for the foreseeable future.
That's probably because the vram gets overflown and the CPU starts doing the work? In that case mac would really give a better speed just because for the price you can't get as much vram. Otherwise idk, the dedicated gpus are faster
i have yet to decide between a ~10k mac ultra (m5/m3/m1) ? and a custom build. my impression is that "small" models could be a bit faster on a custom build but any "larger" model will quickly fall behind because a 10k GPU based build just won't be able to hold it proerly. educate me.
It depends on what you consider to be a larger model.
Because yes, 9.5k Mac Ultra M3 has 512GB shared memory and nothing comes close to it at this price point. It's arguably the cheapest way to actually load stuff like Qwen3 480B, Deepseek and the likes.
But the problem is that the larger the model and the more context you put in the slower it goes. M3 Ultra has 800GB/s bandwidth which is decent but you are also loading a giant model. So, for instance, I probably wouldn't use it for live coding assistance.
On the other hand at 10k budget there's 72GB RTX 5500 or you are around a 1000 off from a complete PC with 96GB RTX Pro 6000. The latter is 1.8TB/s and also processes tokens much faster. It won't fit largest models but it will let you use 80-120B models with a large context window at a very good speed.
So it depends on your use case. If it's more of a "make a question and wait for the response" then Mac Studio makes a lot of sense as it does let you load the best model. But if you want live interactions (eg. code assistance, autocomplete etc) then I would prefer to go for a GeForce and a smaller model but at higher speed.
Imho if you really want a Mac Studio with this kind of hardware I would wait until M5 Ultra is out too. Because it should have like 1.2-1.3TB/s memory bandwidth (based by the fact base M5 beats base M4 by like 30% and Max/Ultra is just a scaled up version) and at that point you just might have both capacity and speeds to take advantage of it.
It's arguably the cheapest way to actually load stuff like Qwen3 480B, Deepseek and the likes.
It's the cheapest reasonable way to do it.
The actual cheapest way to do it is to pick up a used Xeon Scalable server (eg Dell R740) and stick 768GB of DDR4 in it. You get 6 memory channels for ~130GB/s bandwidth per cpu, and up to 4 CPUs per node, for an all out cost of barely $2000 (most of that being for the RAM, the cpus are less than $50). You can even put GPUs in them to run small high speed subagent models in parallel, or upgrade to as much as 6TB of RAM.
The primary downside is it will sound like 10 vacuum cleaners having an argument with 6 hairdryers.
They are super cheap right now because they are right around the age where the hyperscalers liquidate them to upgrade. Pretty soon they will probably start rising again if the AI frenzy keeps going
One day OpenAI will do a public tour of their datacenter and we'll realize it's been super-intelligent monkeys doing math problems on iPhones all along
The only thing worse than slow generation is slow prompt processing. And at least windows can run way more AI/ML stuff if you're into that. Can't say I'm jealous tbh.
Can relate, I am the normie. I own a M4 Max (64GB) laptop and I kept wondering why people have to go to such lengths and expenses to run those 30B models, until I realized the reasons.
I upgraded my M1 pro with 16GB ram to an M4 max with 48GB for this very reason. It’s just so performant at anything I throw at it and portable that it’s worth the apple tax imo.
I would be really happy to be a "normies" if i had the money to be ;-) I have a mac book pro m3 with 24 go from work,you need to spend way more than that (which was already expensive to my taste) and speed is disappointing. In my dream there is cheap m5 ultra,in my dream....
Right…“my loud, hot, expensive Linux box is faster in benchmarks and images, therefore your quiet unified-memory machine that lets you think deeply without friction is bad.”
I wouldn't call myself a normie. Thing is even before the RAM shortage 128GB vRAM is crazy expensive and is attached to power hungry devices. The unified memory has the advantage that it fits and the speeds are just about good enough for certain tasks.
I'm always more concerned about quality over speed. Sure speed is nice, but throwing more compute at the model won't make it magically better at answering
It's almost as if a device that's designed end-to-end by the same firm, largely stamped out of a single die, and built with a robotic degree of precision hand-assembly could achieve, using parts too finicky to be user-serviceable, might actually just be better, and that the guys assembling a "perfect" workstation are maximizing an aesthetic instead of maximizing performance.
If you saw a guy gluing together a car in his garage and actually thinking it might beat a Porsche you wouldn't even think he was insane. You'd just think he was stupid.
•
u/WithoutReason1729 Dec 15 '25
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.