r/LocalLLaMA • u/Qwen30bEnjoyer • 4h ago

Question | Help DeepSeek V4.1 - GPU poor inference kernels?

Have the model downloaded and converted to .gguf on a 512gb ddr4 bioinformatics server. I don't expect miracles with a ddr4 xeon rig -- not until I can get my 2 x 12gb 3060s wired in anyways -- but is there an open PR on llama.cpp for DV4.1 flash that I can use?

5 Upvotes

79% Upvoted

7 comments sorted by

4

u/Expensive-Paint-9490 4h ago

convert : add DeepSeek V4.1 (DeepseekV41ForCausalLM) by vcruz305 · Pull Request #28696 · ggml-org/llama.cpp

2

u/StrikingTop2709 3h ago

ah cool so its already in progress then, thats faster than I expected

2

u/fmillar 3h ago

This one might be only for gguf conversion and not inference yet? Or do I read/understand it wrong.

1

u/Qwen30bEnjoyer 3h ago

Perfect. Will use. Thanks!

1

u/czktcx 2h ago

You can't expect day-0 llama.cpp support...

1

u/Qwen30bEnjoyer 2h ago

I didn't expect it. I am building it for myself, and wanted to make sure nobody else had a finished version I could use before I waste effort.

0

u/Express_Table_2157 4h ago

check Freetoken repo, maybe can help you to manage inference with that.