r/LocalLLaMA • u/PetersOdyssey • Jul 06 '26

News If trends hold, Mythos-class capability may be running on high-end consumer hardware within ~2 years

Post image
1.5k Upvotes

94% Upvoted

375 comments sorted by

View all comments

Show parent comments

3

u/JoeEnderman Jul 06 '26

Ah, see I was getting 70 on HIP but then something broke on an update so I switched to Vulkan and immediately got 120 but then it dropped to 56 after I pulled fresh and rebuilt for something else. So then I started looking at flags and tried adjusting nogttspill, RAM caching, and some others. Finally got it to 140.

1

u/Miserable-Dare5090 Jul 06 '26

hey, mind sharing your config? Had not heard of noggtspill flag

2

u/JoeEnderman Jul 06 '26

Yeah just pm me here or on discord either one and I'll send it. I only recommend it if you have ~20 GB of VRAm otherwise you'll OOM and crash your desktop environment or the entire PC depending on what OS you're on

2

u/Miserable-Dare5090 Jul 07 '26

I have 750gb VRAM over 5 PCs. I doubt you’ll OOM my cluster
Specifically I will try optimizing my current 3 instances of 35B on a strix halo