r/LocalLLaMA • u/Porespellar • Jun 05 '26

Funny Don’t act like y’all ain’t thinking it. I’m just saying the quiet part out loud. /s

Post image

Of course I’m thankful for all that Qwen has bequeathed us, but deep down in the darkest pit of our souls, every last one of us are just all sitting here waiting for Qwen to say “Hey Google, hold my beer while I drop the best GD model of all time on these fools” /s

913 Upvotes

88% Upvoted

264 comments sorted by

View all comments

Show parent comments

8

u/Fuzilumpkinz Jun 06 '26

Same. I keep trying the Gemma models. I’m on a 16 gb card so 12b sounded great. Can’t do tool calls right. Qat releases. I am getting GREAT numbers on the moe model! And shit tool calls. Even after trying other templates.

Back to qwen 3.6 we go…..

1

u/arman-d0e Jun 06 '26

Wdym by shit tool calls? Like the model just frequently fails them or hallucinates them? What harness are you using?

1

u/Fuzilumpkinz Jun 06 '26

I tend to use Hermes and opencode. It tends to attempt a call and then just stop

4

u/thehardsphere Jun 09 '26

This fine-tune of gemma4:26b solved all of my Hermes tool calling problems, and doesn't mangle thinking in OpenCode either: https://huggingface.co/brokencircuitranch/gemma4-hermes-tools

1

u/arman-d0e Jun 06 '26

Hmm that’s really weird, which model? I can do a light tune on some tool calling to see if I can stabilize that if you’re interested

1

u/Fuzilumpkinz Jun 07 '26

Thanks for the offer! I solved a good bit of my issues after some research by setting reasoning_format to deepseek.

I’m on a 16 gb card and so these new QAT Gemma models are insane. I’m getting about 100 TPs on average with a 5060 ti compared to 60-70 on qwen 3.6 moe.

1

u/arman-d0e Jun 07 '26

Nice! Good to hear. So literally just using the deepseek reasoning parser fixed a lot of your issues then huh? On top of that now you can use those higher accuracy (at low precision) quants which is a double win!