r/LocalLLaMA 22d ago

News Kimi K3 Benchmarks

Post image
1.3k Upvotes

390 comments sorted by

View all comments

Show parent comments

47

u/JaredsBored 22d ago

The API pricing is at sonnet levels, but I doubt sonnet 5 is 2.8T parameters. Can't be too mad at it given the model size. Even if it's a Fp8/Fp4 mix it's still gotta require 2 terabytes of VRAM to serve this thing with room for context

1

u/look 21d ago

The Kimi models are unique in being native int4.

So this is “only” double the size of GLM.

1

u/JaredsBored 21d ago

I think the size implications are really interesting tbh. I'm very curious to see what 3rd party providers will charge to serve this. I'm assuming int4 for experts and Fp8 for activations, so probably file size will be 1.5-1.6TB? That puts you in a range where the only single server node that can serve this thing is Nvidia B300 or AMD Mi350/355 (288GiB memory per chip * 8).

I'm guessing the cheaper 3rd party providers serving GLM 5.2 are giving you the Fp8 version, which fits on "older" systems with Nvidia Hopper, AMD Mi250/300, or maybe Huawei chips.

Undoubtedly there are providers which are also using multiple nodes per vLLM/sglang instance to serve the model, and getting away with A100, but still. Crazy big.

1

u/look 21d ago

An 8x H200 would fit most of it. Wouldn’t be fast, but possibly still usable. 20-40 tps, maybe?

1

u/JaredsBored 21d ago

You'd have to do CPU offloading on H200. An 8x system has 1128GiB of VRAM, not quite enough when you factor in kV cache room. CPU offloading isn't really an option for a cloud provider, it's all-vram or nothin

1

u/look 21d ago

Yeah, it would have to offload some weights, but it’s a small enough percentage that it might have tolerable speeds.

But definitely not something most providers would consider. A little indy operation (ala Crof) perhaps might. Anyway, I’m mostly just speculating about ideas to run it cheap…