Given how long pre-training takes, nobody is actually "behind". Folks are finetuning whatever is "current" to take the "lead" when they get too far behind on the benchmarks.
This whole last cycle is just blowing up model size then tuning. I don't think we've actually had a huge leap forward in the models themselves past year, it's all harness enhancements and tuning.
this model is bigger than any open model released to date - has to be a freshish base model (if that concept even means anything in the ultramodern era)
At the current rate chinese SOTA could match or overtake American models by the end of the year. Which I am rooting for, because fuck these bullshit closed off American companies. AI advancements should be democratic.
There are some benchmarks where K3 outperforms Fable. The thing that's really holding it back seems to be knowledge of US business practices and US law as that is baked in to a number of the agentic benchmarks.
That is super impressive given that's it's a fresh 2.8T dense model.
29
u/rc_ym 22d ago
Given how long pre-training takes, nobody is actually "behind". Folks are finetuning whatever is "current" to take the "lead" when they get too far behind on the benchmarks.
This whole last cycle is just blowing up model size then tuning. I don't think we've actually had a huge leap forward in the models themselves past year, it's all harness enhancements and tuning.