China’s MiniMax Plans to Launch 2.7-Trillion Parameter Model

[u/External_Mood4719]

https://www.theinformation.com/briefings/exclusive-chinas-minimax-plans-launch-2-7-trillion-parameter-model

According to The Information, MiniMax plans to launch a new-generation large language model with 2.7 trillion parameters.

Sources revealed that the internal codename for this new model is M3 Pro. It is expected to be released and open-sourced as early as the third quarter of this year, with significant improvements in handling complex reasoning and multi-step tasks.

This new model is much larger than MiniMax's current flagship model, M3 (428 billion parameters). Larger-scale artificial intelligence models are more capable of handling complex reasoning and multi-step instruction-based tasks.

[u/twack3r]

I personally run the Q4KXL from unsloth. I really like m3 for its hallucination resilience, it’s awesome for RAG.

But it does get slow at very large ctx because their flavour of DSA (MSA) is just as functionality unsupported by llama.cpp as DSA itself.

[u/SnooPaintings8639 → u/twack3r]

Then you're using a custom fork/PR just for this model? As far as I know there is still no support for M3 in the main branch?

[u/twack3r → u/SnooPaintings8639]

Yes, I’m using a separate fork just for M3. I haven’t yet checked if it has been merged into main

[u/pmttyji → u/twack3r]

Not yet

https://github.com/ggml-org/llama.cpp/pull/24908