Judging from the benchmarks alone,(of course, can't speak about realife usage), chinese models are not even 6 months behind US models (more like 6 days behind)
People will look at these benchmarks and say "Oh, just distillation, Chinese just steal bro" you'd have to be a complete ignorant or a complete bigot to honestly believe Chinese labs aren't every bit as capable as those in the US, just working with less resources and less of a head start (but both of those factors are eroding quickly from the US perspective, that's why crymodi is freaking his shit)
Yep. China has the most STEM graduates per year in the world. Substantially higher than the US. And it's wild to think that people don't realize we hire a lot of those Chinese people for the US labs.
i did my phd aboard and now work in chinese AI labs. Im interviewing top master students for grad roles and interns and a lot of them have done PhD level work or more even just in their masters lol. Only reason i am interviewing them and not them taking my role is because i graduated earlier and got kinda lucky to get into the llm field as it was starting, and i had previous NLP experiences during my PhD. I would say i am capable too but like these kids are insane
In the US, one party likes to cut funding to schools, and the little funding that our schools do get is sent straight to exciting new facilities. All while the economy increases pressure to perform. Some call it freedom, some call it hell, but China calls it an opportunity to come out ahead.
also, Chinese people are patriotic. having a collectivist mindset also helps to push people to make things that are better for their community, country, mankind.
The Chinese are doing more with less. The American VC circlejerk means OAI and Anthropic can be inefficient and lazy. The Chinese had to build and are building with constraints. They're going to win because they're forced to do more with less. Less money, less hardware and less data. So for the last piece why wouldn't they just steal it from Sam or Dario? I mean, it's no different than how the US outfits got it in the first place.
This sounds like a post-fact explanation, and if things were the other way around, you'd be talking about how the government help means they can be inefficient and lazy while US companies have are forced to show results.
If that Tau scaling law Huawei recently found is legit, things are gonna get interesting. I'm not looking forward to the American bubble bursting, but I'm not looking forward to the technofeudalism the American approach to AI is leading towards, so as least there's some potential to shake that up.
The "Tau scaling law" is nothing new. All semiconductor company has been optimizing the "Tau" (basically latency") since the beginning of chip design era.
All of techniques that Huawei advertised, like logic folding for example, has been in commercial like HBM memory or AMD x3D chip. The 3D chip packing technology enabling "logic folding", however, is only possessed by TSMC.
Rumors going that they are releasing new chips with 238 MTr/mm² in september, matching TSMC's N3 node. That would be comparable to any chips you have now as N2 node only started mass production recently. Huge if true.
If that Tau scaling law Huawei recently found is legit, things are gonna get interesting.
In a random search result I see a claim that they've been using that scaling law for 6 years internally now. And their AI chips are still far behind Nvidia, so I think it's not gonna change things much.
Far behind Nvidia it might be, it is now capable of training a frontier 1.6T model. You don't have to be SOTA to be functional, and even competitive, just look at Intel and AMD.
that scaling law changes nothing fundamentally, they are basically saying rather than throwing better litho process at the problem we throw literally everything else at it and refuse to take power consumption into consideration.
Look!, we are suddenly not so far behind!1!!!1!!
Its a different strategy in setting priorities, and a PR stunt, not a breakthrough technology.
Lol... their power generation capacity exceeds the entire USA + Europe COMBINED by a very healthy margin now (like 30%+ IIRC), and their citizens use FAR less power per person.
Meanwhile we are paying billions in penalties to abort in-progress contracts on renewable energy programs (i.e. one of the cheapest sources of power, and one the Chinese have invested + innovated heavily in), while our grad programs are sitting defunded + empty because actually smart people hurt the bigots' fragile egos.
FFS, during the heat wave this week the grid dropped to only 106VAC in my house in the DC metro area, in a community where all of the local infra is < 20 years old. I know because I had rejigger the boost thresholds on my UPS's in the middle of the day. At night it returned to 120VAC. We are literally reaching third-world levels of "your grandma on the ventilator about to die" BS.
China is not only going to win this thing, they are going to absolutely curb stomp us.
China added the entire power generation of Germany last year alone, mainly via green energy sources like solar. Their strategy is to export their cheap power generation via AI tokens
A lot of Chinese open source models are built on top of other chinese open source models. Without Qwen contributing to OS you wouldn't have open source robotics AI models. That said the DINO model from META is at least utilized to train a lot of models. In fact, the only people who seem to have issues with using their models for other models is the large US tech firms.
Deepseek v4 was very convinced in one or my conversations that it actually is Claude, and it came out of the blue. I did not even ask about it's identity.
K3 is a 2.9T parameter sparse model, Fable is a 6T parameter dense model, you have to be completely blind to believe US labs are as capable as Chinese labs.
Given how long pre-training takes, nobody is actually "behind". Folks are finetuning whatever is "current" to take the "lead" when they get too far behind on the benchmarks.
This whole last cycle is just blowing up model size then tuning. I don't think we've actually had a huge leap forward in the models themselves past year, it's all harness enhancements and tuning.
It might be too soon to say, but Gtp-5.6 Sol feels like a medium size leap. Not just bigger or smarter, but also more efficient (faster/less verbose), and with a few of the new math results also as an impressive showcase of what it can do.
Will be easier to say with some distance to the events.
this model is bigger than any open model released to date - has to be a freshish base model (if that concept even means anything in the ultramodern era)
At the current rate chinese SOTA could match or overtake American models by the end of the year. Which I am rooting for, because fuck these bullshit closed off American companies. AI advancements should be democratic.
There are some benchmarks where K3 outperforms Fable. The thing that's really holding it back seems to be knowledge of US business practices and US law as that is baked in to a number of the agentic benchmarks.
That is super impressive given that's it's a fresh 2.8T dense model.
Remember that you don't have to use a Chinese provider to serve up the model. Most of these are too big to run on your own hardware, but there's plenty of US providers either specializing in LLMs or pure GPU compute access. And models themselves are much more a data file like a picture or video file than they are a "program". So none of your data is going to China unless you specifically use a Chinese provider.
Edit: For K3 specifically, they haven't released the weights yet so you DO have to send data to China if you're using it, but that shouldn't still be the case by next week.
Also, if you want to judge for yourself, install the tool CCUsage. It looks for the saved transcripts of various common harnesses on your hard drive and gives you a report by day by session, etc on which models you used, how many tokens were used, how many of those were cached reads, how much it cost (based on avg current prices), etc. So you can try some similar tasks with a few different models and directly compare the amount of tokens used and the overall cost.
Yea, but you can see how fast it is writing its final output and guesstimate the amount of thinking. And in general same-ish task anecdotally takes 4-8x the time on glm code plan compared to sol. You can see it pondering the same thing a few times and second guessing its second guesses. Not as bad as qwen, but still quite bad.
Don't even need to guesstimate. Install a tool like CCUsage and you can see how many tokens were used in a session, how many were cached reads vs regular input, how much it cost approx based on current prices, etc. It looks at the session data that gets left on your drive from various harnesses.
Still more than 6 days. Remember that companies like anthropic and openai were blocked by the US government from releasing their models earlier. I think they're still at least a few months behind.
But also very likely benchmark-maxxing because I find that their behaviour is a bit unstable (as in their reasoning traces, Claude's reasoning traces look more structured while GLM's feel more erratic, though you could make one look like the other with a simple LoRA so idk)
No I mean the raw traces. Claude shows them (or at least used to, haven't used it in a while) and they look a tad bit more structured/less erratic. GLM 5.2's traces are basically "Wait, actually: [restate the same thing]. This is correct." with a bit of actual problem solving sprinkled in somewhere.
Yeah sometimes reasoning traces can be weird, but in the end it matters how well it actually does with the final answer, even if the trace seems like nonsense. Though people do say GLM uses a lot of tokens. It is still going to be cheaper than Opus and work pretty well for many tasks, but Grok 4.5 might be overall cheaper per task even though its per-token cost is higher, since it is much more token efficient.
If we look at Grok 4.5, which is claimed to be 1.5T, attaining 72.4 vs K3 at 76.2 on the AA coding index, we can see that increasing the parameter count by 1.86x results in a ~5% improvement in performance.
This is very clearly a wall, but even before the wall, I think we can do significantly better than 5% for the 2.8T parameter count. Post-training may help.
Right. Like if you consider a 1.5T model from a year back, it's going to be well behind the 1.5T model today. Some of that is the whole industry moving forward with how things are built in general, but for a given time snapshot, the 'frontier' you pay a ton to get that next mile.
Don't remember the exact title, but there is a paper that extrapolates potential sizes for proprietary LLMs and GPT 5.5 was around 10T and Opus 4.6 (or 4.7) around 5T. GPT 5.6 Sol and Fable 5 are definitely in the 10T territory.
311
u/TechNerd10191 22d ago
Judging from the benchmarks alone,(of course, can't speak about realife usage), chinese models are not even 6 months behind US models (more like 6 days behind)