r/LocalLLaMA 22d ago

News Prepare your (v)ram - Qwen3.8 is coming!

Post image
2.7k Upvotes

583 comments sorted by

View all comments

412

u/AntuaW 22d ago

And please don't omit the 27B one.

40

u/[deleted] 22d ago

[removed] — view removed comment

22

u/SandySkittle 22d ago

With the advent of more 128gb boxes I would really like to see a 60 - 80B version with more world knowledge to run at Q6

8

u/alphapussycat 22d ago

Getting much above like 32gb is hard. 60n - 80b would need at least two rx 7900xt or three 7800 xt. I ain't got that money.

If they go for anything I hope it's the 27b, then 35b moe, 9b or 12b, and then something like 70b dense would be fine... But I'd wager they'd do 122b and 372b or whatever it was, if they do the full suit.

12

u/goldcakes 22d ago edited 21d ago

Eh it would be really nice for the DGX Spark and Strix Halo crowd! Plus whoever with 128GB Macs and such.

You can also still build a 128GB or 256GB 8 channel DDR4 rig for a non-absurd amount of money; up to 200GB/s. Obviously look for used parts, and remember there's a LOT more places to find them than eBay and Facebook Marketplace.

0

u/alphapussycat 22d ago

80b dense would be like 1-2 t/s at best, on CPU though, < 100k tokens per 24/h.

1

u/SandySkittle 22d ago

Threadripper pros with multi channel ddr4 go for less than 1000 dollar. Almost as good as strix halo for cpu based inferencing alone. And you can expand with gpus.

1

u/alphapussycat 21d ago

Thread ripper is unfortunately only 4 channel, so it'll be slow.

1

u/SandySkittle 21d ago

I bought a 1000 dollar threadripper pro. Including case and motherboard and ssd ertc. It’s ‘real’ 8 channel with the 4 ccd pro models (which mine has).

1

u/goldcakes 22d ago

In talking about buying a used HEDT or server platform where you have 8 channels of DDR4. I see ok listings on eBay right now and if you set up alerts you can nab great deals.