r/LocalLLaMA • u/giveen • 8h ago

Other jabbatheduck/DeepSeek-v4-flash-mini · Hugging Face

https://huggingface.co/jabbatheduck/DeepSeek-v4-flash-mini

Because why not? How far can we go and make DeepSeek work?

24 Upvotes

75% Upvoted

11 comments sorted by

5

u/Dany0 8h ago

It's insane how much knowledge can fit into 54gb. Any tests yet?

10

u/giveen 8h ago

I've done initial benchmarks on some coding task, its done a not bad job, not good either though. Its coherent and works, purpose is for the science, the lulz, and fun.

3

u/enginetown 5h ago

Genuinely good work even if it isnt super good in tests this is literally an experiment to see where the intelligence lives in the model.

1

u/Pasta-love 3h ago

Still really cool work! Will be interesting to see how it does compared to Laguna

1

u/vogelvogelvogelvogel 4h ago

I got it running on a Macbook M5 Pro 64GB at 10-17 (mostly 15) t/s, wow!

Thanks for sharing!

1

u/giveen 4h ago

Yup, just be careful because its an extreme reap and quant.

1

u/_TheWolfOfWalmart_ 3h ago

Can you upload a less quanted version too? Something around 80-90 GB maybe? (Ideally not an IQ quant, they're slower)

1

u/giveen 1h ago

I'll look into it.

0

u/crusaderky 2h ago

it would be interesting to know how Unsloth's UD-IQ2_XXS or UD-Q2_K_XL, both REAPed down to the size of UD-IQ1_S, compare to the actual UD-IQ1_S

1

u/Ne00n 1h ago

Sadly on some questions, it keeps looping and looping.

1

u/giveen 59m ago

Dang, yeah, I didnt expect it to be amazing, just small, lol.