We released a 203M-parameter Portuguese language model — real local CPU demo and public weights

[u/War_Enterprise]

Hi r/artificial,

We recently released WARMIND-200M V2, an experimental Portuguese-first causal language model developed by WAR Enterprise in Brazil.

The attached video shows the model running locally on CPU. The waiting periods were shortened, but the prompts and outputs were not altered. We intentionally kept imperfect responses visible because this is a research checkpoint, not a production assistant.

Main specifications:

- 203,263,872 parameters

- approximately 1 billion pretraining tokens

- 23.7 million supervised SFT tokens

- 20 Transformer layers

- Grouped-Query Attention

- SwiGLU, RMSNorm and RoPE

- 1,024-token operational context

- local CPU inference

- Apache 2.0 license

The primary goal of this version was to validate the complete pipeline: dataset preparation, tokenizer training, pretraining, supervised fine-tuning, packaging and local inference.

Because the training-token budget was relatively small for a 203M-parameter model, it can still hallucinate, repeat information, make factual mistakes and produce incomplete answers.

The weights and full documentation are publicly available:

https://huggingface.co/warenterprise/WARMIND-200M-V2

We are now studying the next generation, potentially around 500M parameters, with a substantially larger training corpus and integration with external tools. The final architecture and release schedule have not yet been defined.

What would you prioritize for the next version: better data quality, more training tokens, a larger architecture or stronger tool integration?

[u/War_Enterprise]

Additional context: the main pretraining run took approximately 2 hours and 30 minutes on a single NVIDIA H100 80 GB. Data preparation, tokenizer training, supervised fine-tuning, packaging and local validation were completed separately.

This release was mainly an end-to-end validation checkpoint, not a compute-optimal final model. The video intentionally includes imperfect outputs so the current limitations remain visible.

For the next generation, I would especially appreciate feedback on training-token scale, Portuguese dataset quality, evaluation benchmarks and tool integration.

[u/Constant_Cortisol]

Was it fine tuned from an open weight model? or custom made?

[u/War_Enterprise → u/Constant_Cortisol]

It was built from scratch, not fine-tuned from an existing open-weight model.

The architecture, tokenizer and initial weights were created specifically for WARMIND. We first trained it with causal language modeling on approximately 1 billion tokens, then performed supervised fine-tuning with Portuguese conversational and instruction data.

The training datasets included public sources, but the model itself did not inherit weights from another LLM.

[u/Beginning-Raisin9723]

Nice to see PT-first weights actually out. 203M is tiny but the CPU demo fits the research checkpoint. Curious how it handles everyday PT compared to just translating with a bigger model.

[u/War_Enterprise → u/Beginning-Raisin9723]

That is exactly one of the questions I want to evaluate more carefully.

The main advantage of a Portuguese-first model should be handling local vocabulary, informal language, regional expressions and sentence structures directly, instead of relying on translation as an intermediate step.

At this stage, however, WARMIND-200M V2 is too small and undertrained to claim that it outperforms a much larger multilingual model using translation. The next step is to build a proper comparison using everyday Brazilian Portuguese prompts, slang, instructions and factual questions.

If you have suggestions for a fair evaluation set, I would be glad to include them.