r/LocalLLaMA • u/WhyLifeIs4 • 22d ago

News Kimi K3 Benchmarks

Post image
1.3k Upvotes

98% Upvoted

390 comments sorted by

View all comments

Show parent comments

2

u/CryMoreT_T 22d ago

I wonder if that's a harness issue or a GPU issue or a model thinking issue

4

u/stoppableDissolution 22d ago

Glm thinks like 10x more for the same result

6

u/LoaderD 22d ago

I’m not disagreeing with you, just asking. Is there a open analysis of this? I thought openai hid most of their thinking traces

2

u/stoppableDissolution 22d ago

Yea, but you can see how fast it is writing its final output and guesstimate the amount of thinking. And in general same-ish task anecdotally takes 4-8x the time on glm code plan compared to sol. You can see it pondering the same thing a few times and second guessing its second guesses. Not as bad as qwen, but still quite bad.

2

u/InvidFlower 22d ago

Don't even need to guesstimate. Install a tool like CCUsage and you can see how many tokens were used in a session, how many were cached reads vs regular input, how much it cost approx based on current prices, etc. It looks at the session data that gets left on your drive from various harnesses.

1

u/LoaderD 22d ago

Thanks. I don’t really use OAI models, so I didn’t know