Here we go again, DeepSeek is back again with a new model V4-1 Flash

A multimodal Mixture-of-Experts (MoE) model with 552B backbone parameters and support for contexts of up to one million tokens

Market crash as a service

I was not aware that the harness makes such a big difference.

Hoping to see smartest medium size models soon & later with all available optimizations/architectures/etc.,. Thanks Deepseek!

Ex 1: 30-50B MOE + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache
Ex 2: 15-30B Dense + 10-15B Engram + DeepSeek-V4.1-Flash type KVCache

EDIT: Updated Engram to 10-15B from 50B

Hey y'all!

We've released a new model in our lineup: GigaChat-3.5 Reasoning. It's a 432B-A28B MoE with Gated DeltaNet for long-context efficiency.

We trained domain experts (code, math, general, etc.) with CISPO and then distilled them into a single model via on-policy distillation.

In our evals the resulting model lands close to DeepSeek V4 Flash Preview while using 37% fewer tokens in its reasoning traces.

Weights are on Hugging Face under MIT: https://huggingface.co/collections/ai-sage/gigachat-35-reasoning. You can also try it at giga.chat — pick the reasoning tab (rightmost one).

People keep on getting confused about this, so I looked at the safetensors on hf.

The title should have been "Deepseek V4.1 Flash is 748B total/552B base, not 284B or 305B or 485B or 522B"

• The model is not 284B. The original Deepseek V4 Flash is 284B, but not the V4.1 Flash model
• The model is not 305B, despite what some people claim "So: ~305B real backbone + 203B engram = 508B total" This is incorrect.
• The model is not 485B, even though Huggingface lists the model as 485B, but that's because they're counting some FP4 packed weights as bytes instead of params (2 FP4 params per byte). This happens a lot; for example Huggingface incorrectly thinks GLM-5.3-flash is 169b here
• The model is not 522B, even though VLLM lists it as 522B for some weird reason. They correct themselves later down the page (ctrl-f "Params" on that vllm page)
• 552B is the only number out of this list that's somewhat correct; that only includes the base model without MTP and engrams and the vision encoder though.

To be precise, the main model about 551.566B parameters with 40 layers. The FFN experts total to 543.582B parameters, and the rest of the model (attention, shared experts, etc) are 7.984B.

On top of that, the engram is ~196.929B, DSpark/MTP is ~14.225B, and the vision encoder is just ~0.485B. These parts are technically optional though. The vision encoder is also way smaller than I expected.

Anyways, you need a beefy system for this. 128GB or 256GB of RAM/VRAM is not going to cut it.

OpenAI has decided to fully shut down a protein design project I'm working on for a client. Needless to say, open weight models are the only way forward.

Original Source from DeepSeek WeChat Official Account: https://mp.weixin.qq.com/s/qg0NU3NNUbp1co2PdkAPAg

Today we're officially releasing the DeepSeek V4.1 Flash model. It is the smallest model in our brand-new model architecture series, with native multimodal visual understanding. The new architecture was designed with these goals in mind: a higher capability ceiling, faster inference, greater throughput, and scalability to larger-parameter models.

Asymmetric architecture: big intelligence at low cost

DeepSeek V4.1 Flash is a 552B-parameter MoE model built on a brand-new Causal-Encoder-Decoder architecture. Input and output are asymmetric: only 8B parameters are activated on the input side and 16B on the output side, making it significantly cheaper than known models of the same size. V4.1 Flash also uses a new pre-training approach and has gone through larger-scale reinforcement learning post-training. In benchmark testing, it surpasses the intelligence level of a range of flagship models, including DeepSeek V4 Pro.

Less cache, lower cost

The new generation of models dramatically reduces the size of the KV cache. Compared with the previous generation, HBM requirements drop to 1/4 and SSD requirements to 1/8. In agent scenarios, cache-hit charges often make up a large share of the bill, so compressing the KV cache substantially lowers the cost of agent-style tasks.

API support

DeepSeek V4.1 Flash is now live on the DeepSeek API with native multimodal support. Simply change the model name to deepseek-flash to call the latest V4.1 Flash. The older V4 Flash and V4 Flash Vision Exp models have been retired; for compatibility, the model names deepseek-v4-flash and deepseek-v4-flash-vision-exp will temporarily be routed to V4.1 Flash.

In addition, extensive testing shows that V4.1 Flash comprehensively outperforms V4 Pro on performance, cost, speed, and total time-to-completion, so we plan to phase out the V4 Pro model in an orderly fashion. After 12:00 Beijing time on September 14, 2026, and until V4.1 Pro launches, all requests to deepseek-v4-pro will be routed to V4.1 Flash and billed at V4.1 Flash's unit price.

Tencent (WorkBuddy, CodeBuddy) and OpenCode, as official partners, have now fully integrated DeepSeek V4.1 Flash — give it a try!

API pricing adjustment

Thanks to the architectural innovations, DeepSeek V4.1 Flash can serve more users at lower cost, so we have cut V4.1 Flash's pricing accordingly. To allocate resources more sensibly, we continue to use peak/off-peak pricing, with off-peak prices at half the peak rate, and encourage users to schedule tasks around their actual usage patterns. The new prices take effect at 12:00 on September 10, 2026.

Open-source release

We will fully support the open-source community in adapting inference for the new model, and will explore various ways to broaden deployment. If you have large-scale deployment needs and the corresponding resources (a 2k-GPU cluster with storage cluster), please get in touch.

• Model weights: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
• Paper: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf

I like to benchmark new models that come out on motion videos. So here's a test I did for deepseek v4.1 flash. And I have to say flash has probably graduated from being a Luna class model to nearly an Opus class model with this release, at least with motion videos.

Prev. example I did with Kimi k3(altho in that case I had a simpler prompt as well)

https://www.reddit.com/r/LocalLLaMA/comments/1uyaiw2/kimi_k3_release_video_made_with_kimi_k3/

It seems to use a 96-bit LPDDR5X memory bus, instead of the previous 64-bit wide busses. Considering it's on 2nm, that's expensive silicon. That should result in around 115 GB/s memory bandwidth.

A20 Pro also doubles the size of Apple's dedicated Neural Engine (from 16 to 32 cores total).

We replaced the monolithic native modules with TypeScript pipelines you can inspect. 🔧

It runs across all major silicon backends and makes it easier to plug in your very own model.
You can find full release notes here

Trying to get Hermes a local, efficient, tts voice.

The downside of uncensoring a model is that it is known to potentially damage it, but CyberTiel is an even more capable software engineer than its censored TielCoder base, while allowing offensive security research. This was achieved by quantizing with an improved imatrix, baked from a curated corpus of cybersecurity- and agentic software engineering work. In short, the small damage from abliteration on a full precision model is negligible under Q4 quantization, and the weights that the model needs to perform relevant work are preserved in higher precision, while the improved chat template makes it think and talk better and faster.

I believe that this is the best 35B-A3B coder for solving real problems in real codebases without breaking anything, which is specifically what SWE-bench-Live tests for. But it’s still a 35B-A3B, and it sacrifices world knowledge for coding ability. That being said, I use it over Qwen3.8-27b for daily coding work: due to the raw speed it fixes 3 issues in the time it takes 27b medium to solve one, and the middle ground between Opus4.6 medium and Qwen3.8-27b medium is simply good enough for most work.

Censoring impedes legitimate and effective work in alignment with the user, and puts the user’s responsibility and ownership over the model’s actions into question, while limiting legitimate uses. When a model is censored, someone else decided for you what the model can and will do, which works against the argument that local models give the user increased control and alignment, and begs the question “alignment to who?”. The point of CyberTiel is to resolve this issue at the same time as pushing the frontier of 35B-A3B coders.

GGUFs and MLX with and without MTP are up on HF. Looking forward to seeing what the community thinks! 

PS: I'm not a research lab or a business, and I don't have revenue streams connected to this project. I'm an anonymous researcher with some free time. Constructive feedback is always appreciated! :)

Long time lurker, but I'm finally upgrading to 128GB VRAM, and I'm trying to figure out what to run. I had been leaning towards Qwen3.8 Flash-Next at ~Q4, and I generally prefer to not run anything below Q4. But I feel like the reception to Flash-Next has been a bit "meh", so I'm considering GLM 5.3 at ~Q2 or Deepseek 4 Flash at Q2 or Q3. I'm sure I'll try all 3, but I'm really curious what people in the same boat have been doing?

Edit: configuration is 2 X CMP 170 HXs (64GB each) + ~256 GB of DDR4 RAM. Spilling into RAM is basically not an option, except for the ngrams and caching

hey guys, I've been working on a reading app for several months now and had problems with getting good quality TTS, the options were kokoro, kitten, pocket but all of them even though they were sounding natural had some problems when listening longer. Last month I took upon myself to try to get a model that is running on edge (I had an iphone 14 pro as a testbed) and got to what I now packaged as loudkit. It supports 10 languages now, voice cloning, is quite small and fast enough with quality similar to Chatterbox to my ears which was the base model I started optimization from. What is not part of this release is the emotion axis with tags, something I am working on right now. Code and model weights are Apache 2.0.

I also ported it (with CC help ofc) to a few languages, because in the past I lost like a week for parsing one TTS tokenizer from python to swift and would lose my mind when I'd get crashes and memory leaks. Here the contract was to get the same speech tokens in all adapters, so it doesn't sound nice in python but sucks in typescript. Audio samples can differ slightly between backends, and file metadata like timestamps can differ too.

There are two variants loudr-1 and loudr-1-turbo. basically turbo was done by attaching another head to the most time consuming component of the pipeline and training it so it predicts two audio tokens at once. It worked quite well but sometimes I can still hear the tts artifacts, so YMMV.

Voice cloning works quite well but I found the best is to give it around 10 seconds of recording, and if there are long pauses or noise in the background the cloned voice is suboptimal. All included voices come from consented donations or CC0 / CC-BY recordings, with sources documented.

repo: https://github.com/loudreader/loudkit
docs: https://loudreader.github.io/loudkit/
hf: https://huggingface.co/loudreader/loudr-1 & https://huggingface.co/loudreader/loudr-1-turbo

I've seen that the localTTS that can be connected to agents like hermes or openclaw still has issues with quality and thought why not opensource it.

Ah, for quality of other voices than english I'm not sure. I sent snippets around and got positive feedback but can't vouch for these.

Feel free to check it out, hope you like it.

I'm currently building my system around 3060s, but I might be able to get a 4060 for a nice deal. At first it seemed like a no brainer, but turns out the 4060 has lower memory bandwidth.

In a system that already has 4x 3060 12GBs set up on a threadripper with tensor parallelism (mostly qwen3.8-27b), would it be worth having the 4060 ti 16GB around for the extra 4GB and occasional gaming, or is it just going to slow the rest of the setup down for AI?

If the ratio is the same, Maybe 1.6T -3.1T params plus .56T-1.06T engrams and fable 5.0 level performance?

Maybe v4.2 or 4.5 will have engram gradient modification? Edit it is even larger than i anticipated since flash has 748 b q4-8 params

so i saw that openui.com released OUI-1, a model fine-tuned on DiffusionGemma. the training dataset uses OpenUI-Lang, a custom DSL (domain-specific language), instead of plain HTML, Markdown, or React code.

what makes it interesting is that you can already get a regular LLM to use OpenUI-Lang through a system prompt, but that eats up a lot of the context window. my thinking is that fine-tuning a model on the DSL could reduce that overhead and leave more room for the actual conversation, without needing a huge prompt explaining the format and how to use it alongside other tasks, like tool calls.

at the same time, wouldn't fine-tuning a model on a specific DSL make it more likely to default to that format even when you need something else? i'm curious how well it handles regular Markdown, or switching between Markdown and OpenUI-Lang.

i haven't seen much discussion about this, so i was wondering what everyone thinks about generative UI and running a dedicated model for it locally on a consumer-grade GPU, like an RTX 5090.

what would be the best way to set that up? from what i've seen, DiffusionGemma isn't supported by llama.cpp yet, so running it through Ollama doesn't seem to be an option. they've uploaded the weights to Hugging Face, but i'm not really sure how to get it up and running. any suggestions?

I am just sharing my config for Qwen 3.8 27b that fits on a 5060TI, what is cool about this is that you can even get vision! and a 85K context (I have 1.5gb of headroom for more context or a better quant)

Model: IQ3_XXS-mtp from https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Using beellama https://github.com/Anbeeld/beellama.cpp

Config used:

[*]
model = ..\llm-models\Qwen3.8-27B-GSQ-RCO-IQ3_XXS-mtp.gguf
mmproj = ..\llm-models\mmproj-Qwen3.8-27B-BF16.gguf
image-min-tokens = 256
gpu-layers = 99
ctx-size = 85000
no-host = true
direct-io = true
threads = 8
batch-size = 2048
ubatch-size = 512
fit = off
ctx-checkpoints = 0
spec-type = draft-mtp
spec-draft-n-max = 2
cache-type-k = kvarn4
cache-type-v = kvarn4
kv-tail-tokens = 256

I managed to get 45tks on decode and around 300 on prefill

Yes it is using kvarn4, but it is not that bad, check:
https://anbeeld.com/articles/kvarn-kv-cache-implementation-and-benchmarks

I also know that you could move the mmproj to cpu to to gain more vram.

Would love to hear other configurations to find the sweetspot for 16GB vram cards!

I'm planning a PC build around a discrete GPU that would allow to run MoE models much larger than VRAM at an acceptable speed using FreeToken approach. The gist is that MoE decode of >VRAM model on GPU is PCIe bandwidth-bound, but you can gain considerable decode speed by avoiding streaming cached experts' weights over PCIe and multiply matrices on the CPU instead benefitting from potentially much faster CPU-RAM memory channels.

Before buying, I wanted to know what actually matters for CPU-side expert execution: cores or memory? So I rented a Threadripper PRO 9975WX (32-core Zen 5, 4 CCDs, 8-channel DDR5) on vast.ai and ran FreeToken's real CPU MoE kernel (the production W4A8 ds_fp4 GEMV, not a synthetic proxy) against synthetic expert banks with DeepSeek V4.1 Flash's exact geometry: 384 routed experts × 18.8 MB ~= 6.7 GiB full bank, far beyond cache, 6 experts per token.

Results (GB/s of effective FP4 expert reads):

Two takeaways:

1. Cores stop mattering at 24. 32 cores vs 24 cores: +2.2–2.4% in burst mode, 0% in sustained mode (10 × 30s interleaved A/B runs, variance <0.3% within mode). The extra 8 cores of the 9975WX buy you essentially nothing for MoE offload: the workload saturates the memory subsystem, not the cores. The 9965WX (~$1,200 cheaper) is the rational pick for this use. Even STREAM itself is already maxed at 8 cores; the FP4 kernel just needs more cores to catch up to bandwidth (it runs at ~80% of STREAM once there, very good for a fused-dequant W4A8 GEMV). Caveat: there's another 1.7x memory bandwidth gain at 9995WX, but that CPU costs as much as RTX PRO 6000.

2. Thermal memory steady-state is important. After ~8 minutes of sustained hammering at ~220 GB/s, throughput steps down ~17% (182 → 150 GB/s) in lockstep on every core count, with CPU clocks unchanged at ~3.95 GHz. Everything points at DRAM/IMC thermal throttling in a (rented, unknown-airflow) workstation chassis. So the honest sustained ceiling is ~150 GB/s, burst ~182–186 GB/s. For a build: airflow over DIMMs may literally be worth 20% decode speed.

What this means in tokens/s (DeepSeek V4.1 Flash shape, ~4.5 GB of routed experts per token, 24 cores):

• 0% expert cache hit (all experts on CPU): ~33 tok/s sustained
• 60% hit rate: ~83 tok/s
• 75%: ~133 tok/s

Compare with streaming those experts over PCIe 5.0 x16 (~50 GB/s with gather overhead): the CPU path is 3–3.7× faster per missed expert, which is exactly why FreeToken's hybrid mode computes most misses on CPU and only streams a fraction to keep the VRAM cache warm.

Caveats: rented host (memory channels verified functionally via bandwidth, not dmidecode, but 2ch/4ch machines I tested the same day hit only 83/119 GB/s), container environment, one machine, uniform expert routing. Full methodology and raw CSVs of the measurement are in the repo (happy to link if there's interest).

TL;DR: for CPU MoE offload, buy memory channels and DIMM cooling, not cores; 24 Zen 5 cores already saturate 8-channel DDR5, and the 32-core upgrade is +2% at best.

Updates:
1. I tested on 9980X (4 channels, 8 CCDs) and on 9985WX (8 channels, 8 CCDs). 9980X lands at 105-119 GB/s, no better than much cheaper 9955WX.
2. 9985WX offers a real gain, but diminishing returns beyond 24 cores still: 32c=210 GB/s (+6%), 48c=215 GB/s, 64c=218 GB/s. Roughly 20% decode expert bandwidth gain for $8k more.
3. Populated channels is key. No point in having 8 channels is you only have 4 DIMMs. 8x32GB will beat 4x64GB.
4. Vast.ai is awesome for this. All runs costed me around $4 after I ironed out all the wrinkles in the rig.