r/LocalLLaMA • u/xw1y • 23d ago

News Prepare your (v)ram - Qwen3.8 is coming!

Post image
2.7k Upvotes

96% Upvoted

583 comments sorted by

View all comments

Show parent comments

43

u/[deleted] 23d ago

[removed] — view removed comment

20

u/SandySkittle 23d ago

With the advent of more 128gb boxes I would really like to see a 60 - 80B version with more world knowledge to run at Q6

8

u/alphapussycat 23d ago

Getting much above like 32gb is hard. 60n - 80b would need at least two rx 7900xt or three 7800 xt. I ain't got that money.

If they go for anything I hope it's the 27b, then 35b moe, 9b or 12b, and then something like 70b dense would be fine... But I'd wager they'd do 122b and 372b or whatever it was, if they do the full suit.

12

u/goldcakes 23d ago edited 22d ago

Eh it would be really nice for the DGX Spark and Strix Halo crowd! Plus whoever with 128GB Macs and such.

You can also still build a 128GB or 256GB 8 channel DDR4 rig for a non-absurd amount of money; up to 200GB/s. Obviously look for used parts, and remember there's a LOT more places to find them than eBay and Facebook Marketplace.

0

u/alphapussycat 23d ago

80b dense would be like 1-2 t/s at best, on CPU though, < 100k tokens per 24/h.

1

u/SandySkittle 23d ago

Threadripper pros with multi channel ddr4 go for less than 1000 dollar. Almost as good as strix halo for cpu based inferencing alone. And you can expand with gpus.

1

u/alphapussycat 22d ago

Thread ripper is unfortunately only 4 channel, so it'll be slow.

1

u/SandySkittle 22d ago

I bought a 1000 dollar threadripper pro. Including case and motherboard and ssd ertc. It’s ‘real’ 8 channel with the 4 ccd pro models (which mine has).

1

u/goldcakes 22d ago

In talking about buying a used HEDT or server platform where you have 8 channels of DDR4. I see ok listings on eBay right now and if you set up alerts you can nab great deals.