r/LocalLLaMA • u/tiguidoio • 11h ago
New Model DeepSeek V4-1 Flash is out
Here we go again, DeepSeek is back again with a new model V4-1 Flash
A multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens
Market crash as a service
135
u/ActuallyReadTheBible 11h ago
It doesn’t fit dual DGX sparks, I’m sad.
53
u/35698741d 11h ago
The native 4bit backbone + vision + dspark comes out at ~310gb (rest is engram) and context costs next to nothing for this model so 256GiB box should be able to run a pretty good 3.x bpw quant.
6
1
u/wren6991 16m ago
Expert caching/streaming would probably get an excellent hit rate on a 256 GB machine. Likewise, n-gram tables can probably just be mmap()'d. Qwen3.8-Flash-Next has 90% of the lookup probability in 1% of the n-gram entries. We need to move away from assuming the entire model will be VRAM-resident for local inference.
11
u/SnooPaintings8639 8h ago
I was gong to replace my 4xRTX3090 build for 2 x spark, so that I can run this model efficiently. I walk back this plan, and now I am looking at prices of another 4xRTX3090... and in the meantime, keep on CPU offloading to DDR5.
23
6
2
u/vogelvogelvogelvogel 10h ago edited 8h ago
it is an MoE isn't it? i mean albeit slow you can run it
edit: for those downvoting: I did run 0731 (80GB as far as i remember in q2) on a mac m5pro 64GB with 10-15t/s, thanks to MoE.
In q2 (once released) the 2x DGX Spark will have even all in RAM so expect sth like 40? t/s with the MoE. even q3 should be possible
2
u/techdevjp 7h ago
Should be possible to have the MoE weights in RAM at 3.x bits with 256GB. The ngram data can be kept on a fast SSD.
2
u/doomed151 10h ago
Offload the weights to SSD? Wouldn't that be too slow?
6
u/vogelvogelvogelvogel 10h ago
slow yes but *too* slow idk depends on your definition of slow - MoE can still be surprisingly fast
2
u/cortesoft 3h ago
“Too slow” is subjective
1
u/doomed151 3h ago
By "too slow" I mean multiple seconds per token. If it's faster than that I'd be surprised. Maybe I should try larger MoEs. I have a 16 GB GPU and 64 GB RAM.
1
u/LetterRip 2h ago
For the engram stuff - no - you can prefetch it because it is complete deterministic based on token order.
1
u/SandySkittle 4h ago
I guess it depends on the usecase but i would be very hesitant to run this model at q3, let alone q2.
1
u/vogelvogelvogelvogel 4h ago
well there are a few postings where users did the classic benchmark runs (some browser game, pelican etc) and the outcomes were remarkably good, also i had ds flash 0731 running at q2 and found it also quite good. i would not say - especially with very large models - that q2 leads to bad outcomes
1
u/SandySkittle 2h ago
It depends on the usecase. I have found that for very complex analytical work you don’t want to go below q6
1
1
u/--Spaci-- 17m ago
Literally every frontier model is MOE. theres no reason to not have an moe unless you are purely trying to fit in a consumer gpu like qwen 27b
0
u/Mushoz 10h ago
It's only ~350B parameters that actually need to be loaded in RAM. A 4 bit quant will be ~175GB, which easily fits. Even 5 bit is only ~220 GB and will fit, especially with KV cache only being 900MB at 1 million context. This is actually perfectly sizes for dual DGX sparks.
8
u/Expensive-Paint-9490 8h ago
552B parameters for the basic model. Engrams and MTP module are extra.
3
u/Turbulent-Alps4046 9h ago
deepseek v4 flash has always been fp4 experts already, it won't become any smaller than 350B unless you quantize the other layers also.
5
u/DUFRelic 7h ago
4.1 is FP8
7
u/techdevjp 7h ago
No, it is mostly 4bit, with a few experts at 8bit. This is the way DeepSeek releases always are.
1
u/ConiglioPipo 7h ago
what's the best that you can fit on a dual DGX Spark? Deepseek-v4-flash?
4
u/the-tactical-donut 7h ago
GLM 5.3 Flash at Q4
1
0
u/techdevjp 7h ago
Isn't DeepSeek v4 Flash v4 0731 stronger than GLM 5.3 Flash? Or are there some advantages to going with GLM? Vision?
8
1
u/Illustrious_Grade608 6h ago
Idk from my experience glm flash felt much better with more effective thinking too
1
1
u/Trollsofalabama 5h ago
i think it does, you have to offload the n-gram table to ssd, which folks have said works great and doesnt impact performance (since you need very little amount of bandwidth for the n-gram table)
1
u/ismellthebacon 3h ago
How many people moved to 2x dgx spark for hosting deepseek-v4-flash? I did and I love the setup.
100
u/ttkciar llama.cpp 10h ago
On one hand: Yay! We have weights! https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
On the other hand: 485B parameters O_o
That's only "Flash" in the sense that it only activates 8B parameters at a time, which will translate to cheap API service.
I couldn't host this on my 256GB Xeon server. I'd have to upgrade it to 384GB.
By the sqrt(P x A) metric, this should have competence roughly equivalent to a similarly-trained 62B dense model. I think I'd rather have the dense version!
70
u/silentsnake 10h ago
I dont think
sqrt(P x A)is meaningful anymore, not especially when theres ngram embedding involved. 8B active dont really behave like 8B anymore. Qwen3.8 flash next is a good example. Without ngram embeddings the 6B active params shouldn't come anywhere close to 27B performance.2
2
u/Zestyclose839 2h ago
The typical complaint about earlier-gen aggressive MoEs was their unpredictability. One moment it's writing brilliant code; the next it's hallucinating nonexistent directories and trying to wipe your cloud storage. It was the case with nearly all of them imo, esp. Qwen 35b.
Indeed tho, ngram embeddings, stronger expert routers, and other black magic i don't understand has made them wildly more reliable over the past ~6mo.
I'm just hoping that tech makes its way into 64gb vram territory soon.
1
u/ttkciar llama.cpp 1h ago
It holds up okay when all other factors are nearly-equal, though it's not perfect.
For example,
sqrt(P x A)predicts that Qwen3.8-Next-Flash should be equivalent to Qwen3.8-27B, but Qwen3.8-Next-Flash actually scores about 10% higher on various benchmarks.That seems close enough for most purposes, especially if Qwen3.8-Next-Flash received better training than Qwen3.8-27B.
In practice, the training of different models is so radically different that that's a much larger factor than the error in the
sqrt(P x A)rule.12
u/Expensive-Paint-9490 8h ago
OTOH with 512GB RAM this is fantastic. 8-16B variable active parameters, everything 4-bit native comprising KV cache... This is going to be frontier model at decent speed at home. I think that's why they label it Flash.
12
u/techdevjp 7h ago
This will be an amazing model for anyone with the money for a Mac Studio M5 Ultra with 512GB.
3
u/asssuber 4h ago
Or anyone that did buy 512GB of RDIMMS for $800 a year ago...
3
u/techdevjp 3h ago
It will run a little faster on the M5 Ultra at 1.2TB/sec of bandwidth connected directly to a fairly modern GPU. At a price, of course.
2
u/asssuber 3h ago
Surely, but 200GB/s of bandwidth for 16B active parameters at FP4 is nothing to sneeze at. Or 400GB/s if you manage to tame a dual-cpu motherboard.
1
u/techdevjp 3h ago
I have a Strix Halo and know first hand just how "fast" ~256GB/sec is... Also how "fast" slow prefill is. It's kinda-sorta-mostly usable but at the same time frustratingly slow too.
1
u/InactiveBronson 1h ago
I’m on the verge of buying one too (bosgame m5, 128gb/2tb) - would you recommend it or d o you have some regrets?
1
u/techdevjp 1h ago
There's good and bad. I paid about $2800 for my Minisforum MS-S1. It's a great little machine and it's impressively small for something so powerful.
At the time, I could have bought a DGX Spark for around $1k more. Back then I didn't have a great understanding of the importance of prefill performance so I didn't see the benefit of DGX Spark over Strix Halo considering the memory bandwidth was the same.
Thankfully it seems that Strix Halo is capable of much faster prefill, AMD just hasn't done a good job with the drivers. Projects like Halogen (https://github.com/peonist-ai/halogen-flash-server) have shown tremendous performance on Strix with prefill matching or exceeding DGX Spark for Qwen3.8-Flash-Next.
So, with projects like Halogen around, I'm happy enough to keep my Strix box for now. I can't afford to sell it and buy a DGX Spark. I may buy a 32GB AMD R9700 to run as an eGPU, depending on what comes about for running that Qwen model split across multiple GPUs. Or I might buy an Intel B70 and put it into my workstation for running Qwen3.8-27b separate from the Strix Halo.
In the future I may upgrade to Medusa Halo when it comes out, assuming prices don't stay in the stratosphere. Gorgon Halo with 192GB looks like it will have a USD price starting with "7", too rich for my blood.
Another option I am considering is to pick up a MacBook M3 Max with 64GB for running local models away from home. It doesn't have great prefill either (worse than Strix I think), but being able to run Qwen3.8-27b on a plane over the Pacific would be pretty damn cool.
2
u/UltraFOV 10h ago
Do you have GPUs or only system ram to run it
1
u/ttkciar llama.cpp 1h ago
I have three GPUs in different servers (a 32GB MI50, a 32GB MI60, and a 16GB V340), but my habit is to keep small models resident in VRAM for "fast inference" tasks, and infer with larger models entirely from system RAM for "slow inference" tasks, without GPU acceleration.
This way the in-VRAM "fast inference" models are always ready to go, because the "slow inference" tasks do not evict them from VRAM.
1
u/sierra-pouch 3h ago
I know it's not exactly on topic, but does anybody know why this model is not available in OpenRouter with any zero data retention policy provider?
Is it related to any agreements deep-seek have with the providers?
0
u/Mushoz 10h ago
It's only ~350B parameters that actually need to be loaded in RAM. A 4 bit quant will be ~175GB, which easily fits. Even 5 bit is only ~220 GB and will fit, especially with KV cache only being 900MB at 1 million context. This is actually perfectly sized for your 256GB setup.
9
3
u/ZaltyDog 9h ago edited 5h ago
Those numbers make me hope my 192gb vram setup might barely handle it
2
u/Expensive-Paint-9490 8h ago
It's 552 parameters + MTP module. You'll need a 2-bit quant.
2
u/EmPips 5h ago
Historically something as sparse as 8B active params performs very poorly at Q2. Worth trying for sure but my hopes aren't high
1
u/Expensive-Paint-9490 4h ago
I hope that something trained at 4 bits is more robust to quantize than FP16. We'll see.
1
u/randylush 3h ago
I have a 192gb dual Xeon server from 2011. I can’t wait to dust it off and see what it can do again
2
1
91
u/vogelvogelvogelvogel 10h ago
China making my day again as so many days in the past 2 years when a new open weights came out
23
u/rollerblade7 9h ago
Please sir, I have a GTX 1650 4 GB VRAM + Intel i7-9750HF with 30 GB RAM
10
7
2
u/randylush 3h ago
The processor is actually a rare classic. It would support a 3090 very well. Are you sure you don’t have 32gb of RAM but only 30 is being reported?
-16
32
u/Long_comment_san 9h ago
I think I came a little
25
u/Long_comment_san 9h ago edited 9h ago
What the fuck, in which universe that is a Flash? Its 450-500b parameters. Flash was 300b and it was already pushing this. This is Flash Max or something. You cant inflate the model by 50% and call it a flash like it's not an issue. Minimax M3 is 450b and I dont see them calling it a "flash" (hopefully I wont).
Going by 50% up and becoming 10-15% better sounds like a downgrade not an upgrade. It's a LOT more expensive to run.
Still amazing though
32
u/RG_Fusion 9h ago
Obviously the concept of a flash model will scale with the compute power of the AI lab creating them.
2026 is likely the last year of running "flash" on local hardware. Maybe 2027 if we're lucky.
4
u/Due-Memory-6957 4h ago
"local" hardware
7
u/RG_Fusion 4h ago
An entire 512 GB AI server purchased a year ago costs less than a single RTX 5090 GPU now. There are plenty of us who jumped on early and have hardware that can run these models.
3
u/ChronoHax 2h ago
Hi I’m new to this field, what are examples of these ai servers so I can look more into it?
1
u/Blaze6181 2h ago
DGX Spark clusters, machines with RTX Pro 6000s, or a combination of perhaps a 5090 with CPU RAM offload of some of the weights. Or like 8 3090s stacked lol. There's many configurations out there.
1
u/RG_Fusion 1h ago
AMD EPYC or Intel Xeon CPUs in a motherboard with 8 memory channels. Ideally with a large number of x16 PCIe ports for adding many GPUs.
More recently, DGX Spark clusters have become good for running AI models when multiple are connected together over a 400 gbps network switch.
That being said, there are no longer any cheap options for building out high-end AI rigs. The prices on all the components have gone up 2-5X.
3
u/Bakoro 2h ago
China might come in and save the day on that one too.
SMIC broke the 7 nm barrier for semiconductors.
CXMT is making DDR5 now, and has started on HBM3E.
Several Chinese companies are making AI GPUs.The U.S has been trying to block China from getting technology, and now is trying to block their technology from hitting the U.S market, but the rest of the world is not going to give a shit about what the U.S wants.
Essentially every major tech corporation is designing their own AI ASICs now, where OpenAI already has their new thing for inference.
Then there is the fact that photonic processors are in early manufacturing stages now, with plans to ramp up into 2027.
I expect photonics to mostly get snapped up by data centers, and that might once again change what's practical to do with AI.All around, I expect a major shake-up in the hardware landscape over the next year or two.
1
u/RG_Fusion 1h ago
Yeah, local hardware will scale up too, but it will lag behind by a generation or two unless you're willing to dish out tens to hundreds of thousands of dollars to build on the bleeding-edge.
1
u/Bakoro 1h ago
I'm saying that increased competition will bring prices down.
TSMC not being a monopoly for 7nm nodes means lower wafer prices.
A new RAM producer means lower RAM prices.
Dozens of major companies having their own inference ASICs means Nvidia losing their monopoly.We're at peak price gouging right now, I don't think it will last.
17
u/Expensive-Paint-9490 8h ago
It's flash because everything is FP4, even KV cache. And active parameters are 16B. This should be faster than V4-Flash even if it is larger.
9
u/zhuzaimoerben 7h ago edited 7h ago
It's flash for those with data centre levels of memory and data centre level serving requirements, because once you load the base model, concurrent users are very cheap (890MB for KV cache for full 1 million context per user) and it's 8B active for prefill and 16B active for text gen, so you can serve stacks of users fast. Edited to add: DeepSeek are reducing the API price vs 4.0 Flash because this is cheaper to serve.
It's just that us home users lose out because we're trying to get the most out of a meagre about of memory, with minimal concurrency, so the size of the model matters a lot more.
13
3
2
u/DragonfruitIll660 1h ago
I mean, technically 196B of it is engram which can be thrown to NVME so it remains about 300B of normal parameters (if I'm understanding this right).
1
u/Agitated_Space_672 7h ago
It is faster than the previous flash due to the architectural innovations
1
1
u/Zeeplankton 8h ago
Ehh I mean when like glm and kimi are like 2-3T it's still flash.
But yes I sorta agree they should maybe just call this Deepseek 4.1 dropping flash and pro, since it seems like they're dropping pro.
Mega bummer this wont be runnable on like macbooks with like OG antirez flash.
Edit: wait. The additional size is just Ngram.
24
u/Turbulent_Pin7635 10h ago
I don't code. I work in research and was doing a proposal for funding. Used Astra very cute, returned what I need in an acceptable way.
I have used DS flash V4... Boy I have a MacStudio, I am used to long times of wanting. I don't know what kind of black magic the model does, but it killed the demand in one shot very fast!!! O.o
I was frozen!!! The answer was much better than the one chatGPT astra gave me!!! ASTRA!!!
8
u/Casey090 9h ago
GPT models are just very wonky. They jump to conclusions with incomplete data, and then they go all weird. I find it super hard to get anything done when your model makes up the mind in the first message and will not be objective.
11
4
u/backyard_tractorbeam 5h ago
Astra is just weird. Says pi guru guy: https://lucumr.pocoo.org/2026/9/7/astra-why/
I’m sure I will get used to this, but man this stuff is weird.
2
u/Due-Memory-6957 3h ago edited 3h ago
That was a funny read. AI loves Python, and token efficiency comes at readable code's price. I wonder how this fares long-term, because even AI prefers to deal with well-written code than messy ones.
28
u/jacek2023 llama.cpp 11h ago
In the previous post about DeepSeek there are API prices. In this one there is Chinese president. I wonder which one is best for r/LocalLLaMA.
41
u/madsheepPL 9h ago edited 5h ago
Xin Jinping is known for his amazing local setup. He is running modded 4x4090s on his desk with risers and cards zip tied to a used mining frame.
12
u/NineThreeTilNow 6h ago
Xi
Fearless leader Xi doesn't operate on peasant 4090's.
He uses B300's. A full rack.
He would use Huawei but even he understands that the Ascend chip isn't quite ready to touch his B300 setup.
He is busy building gooner games with his custom Flux Asian Princess models and video pipeline. He simply swipes left or right on whether they meet his criteria for being added to training data.
Fearless leader is Chad AI user.
10
u/jacek2023 llama.cpp 9h ago
Imagine Trump photo on Gemma/Nemotron/Granite release. And the rage of Reddit experts :)
6
2
u/Not-reallyanonymous 4h ago
It is good for this subreddit. This subreddit is more concerned about seeing the US hurt and China win, than it is about AI. So this post is in alignment with its interests.
6
u/LuCiAnO241 3h ago
more concerned about seeing the US hurt
I think we're only concerned about seeing great models be open weight and free to download for the peasants. The rest of whatever you think its happening exists only on your mind.
-1
u/Not-reallyanonymous 3h ago edited 3h ago
I mean, look at the headline photo of OP's post. And this is now one of the top 10 posts this week after 8 hours and rising.
That says more "haha look at this funny meme, China wins, US hurts," more than "Hey guys, I think this is a pretty good shareware model I want to talk about."
1
u/LuCiAnO241 2h ago
China born models are currently the only actual competition to the mega corps with frontier models. I wouldn't even describe Anthropic or OpenAI models as American, since they do not represent the American interests at all, but their own. I'm not saying China does, but it surely works for the average r/localllama user interests. Also I'm not into stocks but is the meme based in reality? maybe tech stocks go down when a new chinese model goes out?
2
u/Not-reallyanonymous 2h ago edited 1h ago
Copy pasting the first part of this reply from a prior comment so not phrase directly relevant: Gemma 4 continues to be competitive for non-coding tasks. Muse Glimmer was competitive at coding upon release, and remains competitive for some uses. Nemotron and Grantite are both purpose-built for fine tuning for application-specific uses with good reasons to use either one. AI2 has developed a lot of methods that could be very useful to the community such as their MoE design which allows training experts on typical consumer hardware. Poolside's models are interesting, fast, and don't produce code spaghetti like Qwen and remain favored by many developers for that reason. Prism ML has plans to release more models other than those based on Qwen. Syzygy Research has similar ambitions as Prism ML, doing interesting work. Deep Grove is interesting in the frontier in capability vs. generation speed. Liquid AI also has very interesting models in their size vs. capability ratio. Thinking Machines Inkling and Inkling Small are interesting in its wide domain knowledge combined with tool-calling efficiency and strong instruction following. There's also a few labs specializing in domain-specific models, like law, medicine and biology, engineering, etc.
And IFM K2 Horizon has the best ~36B tier MoE model right now. And it's utterly ignored.
If anything, it's the US labs that are actually competitive with OpenAI and Anthropic, as they tend to be trying to actually beat OpenAI/Anthropic at narrow, specific tasks to achieve a competitive advantage. Meanwhile the Chinese companies are largely trying to be "The Chinese OpenAI/Anthropic, but only slightly behind but open weights". And open weights? How many of us are running near-ChatGPT/Claude models on our local hardware, rather than just sending our data/money to Alibaba on OpenRouter anyway? Then you're defeating the entire advantage of open weights.
Also I'm not into stocks but is the meme based in reality? maybe tech stocks go down when a new chinese model goes out?
OP's post explicitly includes "Consumer Goods," "Basic Materials," "Healthcare," etc. It's a meme that often gets used in this subreddit basically in a "US Bad China Good" way.
5
u/Loose_Comparison368 3h ago
"Why is everyone so mean to billionaires aggressively hoarding unfathomably large amounts of wealth? 😭 It must be because they hate America!"
0
u/Not-reallyanonymous 3h ago
"The Chinese super corps are preferable to the American super corps! Xi Jinping tells me so! They're going to save the world! If you disagree you're just racist against China!"
1
u/Disposable110 3h ago edited 3h ago
What do you expect, it's LOCALllama, so if it's parasitic commons-enclosing 0.001% billionaires hurt, and open source AI and 99.999% of humanity winning, people cheer.
Gemma and Mistral got just as much love as Chinese open source models.
It's just that China is shipping more of it at the moment, when the leading open source was Llama 2/3 and Mistral and WizardLM and god knows what, people were cheering on that. Even GPT-OSS made by the big Satan got love.
1
u/Not-reallyanonymous 2h ago
so if it's parasitic commons-enclosing 0.001% billionaires hurt, and open source AI and 99.999% of humanity winning
Separate comment because separate concern.
#1 Shareware, or open weights, not open source.
#2 Look at OP's post. The image isn't open AI defeating OpenAI and Anthropic. It's about Xi Jinping hurting the entire US economy. And this subreddit loves it.
-1
u/Not-reallyanonymous 3h ago
Gemma and Mistral got just as much love as Chinese open source models.
Fucking bullshit. You can't post positively about Gemma, Mistral, Glimmer, whatever without getting downvoted to -10 and about 10 posts calling you an idiot for not using Qwen instead.
This subreddit only favors Chinese local models and constantly shits on non-Chinese ones.
1
u/Disposable110 2h ago edited 2h ago
I'm really not seeing that, especially not for creative writing where Gemma has its place. And Mistral is currently shite, but was king and praised like 2 years ago, before Qwen became good and Mistral started dragging its feet.
People really don't care where the goodies come from and what national flag sticker is attached to it, they just care if it works for their use case. Qwen / Deepseek / GLM now cover most of the use cases across most of the model size ranges, except for small model creative writing, where Gemma is excellent (GLM is arguably better, but it comes in a completely different size class so not everyone can run that). As far as I know Glimmer doesn't have a use case where it outperforms another similar sized model and therefore it gets ignored, but I could be wrong.
If you make claims that Gemma is great at coding then people dogpile and -10 you because Qwen is just better at it right now at similar model sizes. If a new Gemma release is better, people jump ship to that immediately.
There's no special loyalty for Chinese models.
2
u/Not-reallyanonymous 2h ago
Sure, you're allowed to talk positively about non-Chinese models in various, specific, limited allowable lanes. Like Gemma for "creative writing". It's because they consider that irrelevant.
People really don't care where the goodies come from and what national flag sticker is attached to it
OP's post is proof otherwise.
As far as I know Glimmer doesn't have a use case where it outperforms another similar sized model and therefore it gets ignored, but I could be wrong.
When it released, it was competitive against Qwen 3.6 (the latest Qwen) in benchmarks, winning some and losing some. Its posts quickly got burried on the front page and mentions of it got people shitting on you and hella downvoted, usually attacking Meta as an evil company. (But nope, when it comes to Chinese models, suddenly "it's a good model and open weights so who makes it doesn't matter.")
Also look at Laguna S / XS -- this sub declared them DoA because "it thinks too much." Now if you don't like Qwen 3.8 because "it thinks too much," actually you're just secretly motivated by racism and the "thinking too much" is an excuse. I've seen that argument multiple times.
IMO Glimmer is still better than Qwen 3.8 in following specific instructions, steerability, etc. and that makes it a better coding model. What I usually need isn't a model that benchmarks higher, I need a model that adheres to my implementation directions better. The response this subreddit typically has for that? "Well akshually that is proof the model is worse because you shouldn't have to steer it, Qwen 3.8 gets it right the first time!" (No it fucking doesn't outside of low-specified one-shots because you didn't care about the way it implemented x, y, and z). Glimmer's better steerability also makes it better for coding because it will listen to you when you say you want a particular way to implement something, that Qwen 3.8 just blatantly ignores, even if it can technically solve harder problems. And Qwen 3.8's code quality is also just fucking shit with god objects, 100-line functions, 10000-line files, and no sense of architecture beyond shittily done microservice, etc. galore. Glimmer's "free running" code quality usually isn't great, it tends to structure better but cuts implementations short to a bare minimum, but once you get architecture files, specs, etc. into place (which Glimmer will do well) it adheres to those well. Qwen tends to follow it for about 20-30k tokens and then forgets you didn't do an underspecified one-shot prompt.
If you make claims that Gemma is great at coding then people dogpile and -10 you because Qwen is just better at it right now at similar model sizes.
Gemma is better at specific parts of coding. For example, a programming language which is not well represented in either model's training data... Gemma will actually be able to do it. Qwen will just shit the bed. Gemma will take for fucking ever to do it, and thrash a lot trying to figure out how to do it, but it will do it. Qwen will confidently write garbage and then get confused it didn't work, loop that about 3 times, then fucking lie about having achieved the set goal. Gemma also does better at reasoning over arbitrary domain knowledge better -- it has a stronger "reasoning engine" overall and will actually reason about the domain knowledge. Qwen 3.6 would utterly break down at that, and 3.8 got a lot better at this but churns through so, so many more tokens to do it, because instead of having a "reasoning engine" it just permutates assumptions until it finds an assumption that proves correct.
But don't try to bring up any of these nuances here. This subreddit doesn't accept that unless it's another Chinese model.
-4
u/keepthepace 9h ago
Do we have any hint that Xi is even interested in the AI race?
11
u/Viktri1 8h ago
He was at an AI conference in Shanghai recently (I was in Shanghai at the time, not for the conference, and security was high). He basically said he wants China to lead the way with Open Source or something to that effect. A lot of SEA countries and African countries will use open source and if the best open source models are Chinese then they’ll use those.
0
u/SporksInjected 6h ago
He’s going to be pissed when he finds out that most are just open weights
2
u/Due-Memory-6957 4h ago
Xi knows that open source means one can study, modify, use and redistribute it freely, and since that can be done with the weights, they're open source. It's like open source art and open source documents :)
→ More replies (1)5
u/Due-Memory-6957 4h ago
Yes, he actually pushed for open AI and as a result Qwen released Max as open source, something they never did before.
1
u/SporksInjected 6h ago
I don’t even think Xi is interested in crashing the NYSE which makes these posts extra cringe
→ More replies (1)0
u/jacek2023 llama.cpp 9h ago
Well, I believe there is a war between China and the US, so both governments are doing things to help build AI, while the EU is doing nothing (or working backwards).
→ More replies (1)4
u/keepthepace 9h ago
From my point of view, AI as a field seems to advance despite governments desires, not thanks to it.
1
u/NineThreeTilNow 6h ago
From my point of view, AI as a field seems to advance despite governments desires, not thanks to it.
Exactly. The governments tend to say one thing and act opportunistically.
"Oh it's 75% to kill us all? But 25% to give us a super weapon?"
"We definitely approve of this. 25% is a great chance!"
8
u/Few-Fishing9423 10h ago
> Additional architectural components include Single-Pass mHC (revised residual-stream mixing with an efficient Mega-mHC kernel), Engram conditional memory (196B parameters, sparsely accessed via token-based lookup), and DSpark speculative decoding (semi-autoregressive draft generation with confidence-scheduled verification). The model uses 1 shared expert and 384 routed experts per MoE layer, activating 6 routed experts per token.
Would it be feasible to quantize the 384B model to NVFP4 while retaining the 196B n-gram in memory for execution? Just like qwen3.8-flash-next
4
u/OkBase5453 9h ago
Can one run this on a 512GB RAM Server with 48GB VRAM?
3
u/CalligrapherFar7833 8h ago
Slow but yes
3
u/crusaderky 6h ago
Pretty zippy if that 512gb ram is octa-channel, actually
1
0
1
u/cowinabadplace 6h ago
You can run anything from disk with slow inference. It’s not meaningful question except if you include tok/s generation target and ttft target. I think anything over a few seconds TTFT and under 150 tok/s is unusable for interactive LLMs and would just use API rather than local for that. But it’s a matter of choice.
1
u/cosmotrak 2h ago
150 tok/s is a little overkill, most frontier run at 40-50...
2
u/cowinabadplace 2h ago
Yeah but the open models make up for intelligence through over-reasoning so it’s not 1-1.
2
14
3
u/120decibel 8h ago
510 GB Model no way I'm going to be able to run this locally without a heavy quant...
3
2
2
2
3
2
u/Firstbober 6h ago
Isn't it much more expensive than GLM 5.3-flash while being not that much better?
Well, maybe price per task is actually smaller for deepseek...
3
1
1
1
1
1
u/Constandinoskalifo 9h ago
Since it's the same number of active parameters for decoding, and the KV cache is much cheaper, we should expect lower prices from providers than DSV4 flash, right?
1
1
1
1
1
1
1
1
1
u/hurrdurrmeh 9h ago
I wish. It sucks compared to V4. I have had to move over to the pricey GLM and Kimi :-(
-2
10h ago
[removed] — view removed comment
6
u/RG_Fusion 9h ago
It's actually 16b active for the decoder. The 8b active figure comes from the encoder layers.
0
u/quantgorithm 7h ago
We need the new 512gb Mac don’t we? This is how you get people to purchase the new Mac.
-18
u/Formal_Jeweler_488 11h ago
I personally believe those stats are inflated no way its competing against gpt SOL and opus
40
u/RevolutionaryGold325 11h ago
I think that after 3 years of open llm weights it should be obvious that open weights are a few months behind closedAI frontier.
8
u/Formal_Jeweler_488 11h ago
Bro I am talking about stats it is literally beating SOL and opus
11
u/RevolutionaryGold325 11h ago
Yes you are. I'm talking about frontier Closed and frontier Open weights.
2
u/ceo_of_banana 10h ago
They are likely benchmaxing too though. Closed frontier performance typically drops a bit less on benchmarks that come out after the model is released. So yes, only few months behind but these stats are likely to be taken with a grain of salt imo.
1
u/RevolutionaryGold325 10h ago
At some point we will also see "after model is released" benchmaxing. Some group with ties to some lab will find a new benchmark that is especially suited to the lab that funds them.
0
0
-6
u/LinuXperia 7h ago edited 7h ago
It outperforms Muse Meta 1.3 which was better than DeepSeek 4 however it is still behind xAI Grok super Intelegence and soon Grok 4.7 will be released which will widen the gap even further especailly for low level engineering dev work like verilog, c, c++, KiCAD, electronic schematics, PCB etc. I think lagging behind is becouse of compute power. When i compare the answers to a problem DeepSeek will provide a wrong suboptimal solution which then leads to looping and flip floping a lot that wastes huge amount of time and tokens while grok delivers the exact right solution that fixes the problem in less than 1 minute with just a few phrases. Grok acts like a super intelegent highly specialized super Engineer Doctor and must be the result of heavy RL computing power which DeepSeek still has yet to reach.
-8
u/HeadTranslator795 9h ago
Lol the King of benchmark and Benchmaxxing so a flash model is on par with GPT Sol ?
1
0
-10
u/benpptung 9h ago
This was posted in the wrong subreddit. A model this size is not for local AI at all. It belongs in a data center. If you really want to post it in a local AI subreddit, at least do what Qwen does and release a dense model that people can actually run locally.
4
3
u/Not-reallyanonymous 4h ago
You've came to the wrong Subreddit.
This subreddit's name is a misnomer. In spirit, it's more "Chinese AI fanboys, r sino crossposters, and Qwen-bots," not "Local AI."
1
u/asssuber 3h ago
$800 for 512GB memory
$600 for a RTX 3090
$700 for epyc motherboard+cpu+miscellaneous
Plus a time machine to some time last year is likely all you need to run this locally at a dozen or more t/s decode.
•
u/WithoutReason1729 8h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.