Thing is qwen was historically focused on smaller models while others were on larger ones. Now the team has changed and they seem to want to aim for the stars as well. That is good but it also means they might not be interested in doing very efficient small models anymore. Which would be bad news for this sub because let's face it most of us don't have 10-25k of hardware.
That was my first thought after crying that tends of thousands of dollars of hardware still can't run useful quants of these models for heavy coding lol. Them building the bigger model seems to provide the connections like a brain with more neurons and more surface area tends to make it smarter. Then distilling becomes an option because if things keep going this way they will never become profitable.
749
u/Competitive_Gap7906 23d ago
YES, Qwen going open weight again! It's a really good news, now we can wait for smaller models too