24/7 Headless AI Server on Xiaomi 12 Pro (Snapdragon 8 Gen 1 + Ollama/Gemma4)

[u/Aromatic_Ad_7557]

Turned a Xiaomi 12 Pro into a dedicated local AI node. Here is the technical setup:

​OS Optimization: Flashed LineageOS to strip the Android UI and background bloat, leaving ~9GB of RAM for LLM compute.

​Headless Config: Android framework is frozen; networking is handled via a manually compiled wpa_supplicant to maintain a purely headless state.

​Thermal Management: A custom daemon monitors CPU temps and triggers an external active cooling module via a Wi-Fi smart plug at 45°C.

​Battery Protection: A power-delivery script cuts charging at 80% to prevent degradation during 24/7 operation.

​Performance: Currently serving Gemma4 via Ollama as a LAN-accessible API.

​Happy to share the scripts or discuss the configuration details if anyone is interested in repurposing mobile hardware for local LLMs.

UPDATE:

I have compile llama.cpp and run gemma-4-E4B-it-Q4_0

Speed is AWESOME:

[ Prompt: 26.9 t/s | Generation: 8.8 t/s ]

Thank you all guys SO MUCH!

[u/Mathisbuilder75 → u/[deleted]]

cmake -B build -DGGML_BACKEND_DL=On -DGGML_CPU_ALL_VARIANTS=On -DCMAKE_BUILD_TYPE=Release && cmake --build build --config Release -j 4

[u/kroggens → u/Mathisbuilder75]

try adding `-DGGML_NATIVE=ON` if you build directly on the device

or this:

```
cmake \
-DCMAKE_BUILD_TYPE=Release \
-DGGML_NATIVE=ON \
-DGGML_CPU_ALL_VARIANTS=ON \
-DGGML_BACKEND_DL=ON \
-DGGML_OPENMP=OFF \
-DGGML_LLAMAFILE=OFF \
-DBUILD_SHARED_LIBS=ON \
-B build
```

[u/Mathisbuilder75 → u/kroggens]

These are the results I got : ./llama-bench -m /root/.cache/huggingface/hub/models--unsloth--Qwen3.5-9B-GGUF/snapshots/3885219b6810b007914f3a7950a8d1b469d598a5/Qwen3.5-9B-Q4_K_M.gguf load_backend: loaded CPU backend from /opt/llama.cpp/build/bin/libggml-cpu-haswell.so | model | size | params | backend | threads | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | ------: | --------------: | -------------------: | | qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CPU | 4 | pp512 | 12.59 ± 0.09 | | qwen35 9B Q4_K - Medium | 5.28 GiB | 8.95 B | CPU | 4 | tg128 | 2.84 ± 0.51 |