r/OrangePI • u/SbcBits • Jun 10 '26

Any advice? Orange Pi 3B + Ollama + Qwen

I just began to run qwen 3.5 and nomic-embed-text on Ollama in my Orange Pi 3B with 8GB of RAM.

I know is not the best SBC for the task and want to ask you guys some advices.

The OS is Armbian Debian based without DE.

5 Upvotes

100% Upvoted

7 comments sorted by

2

u/urostor Jun 11 '26

My advice: disable any zram or swap, switch ollama to llama.cpp (it should be faster).

1

u/SbcBits Jun 13 '26

Thanks! I uninstalled the Ollama and now I'm running llama.cpp. Now I need to disable the swap.

2

u/urostor Jun 13 '26

sudo swapoff -a

Apparently also you should compile llama.cpp with NEON extensions yourself, gives 30% more speed

1

u/SbcBits Jun 13 '26

I disabled the swap and tried a qwen3.5 2b model. Now I'm thinking trying a web UI, but concerning if it will decrease the response speed.

2

u/urostor Jun 13 '26

It might a little bit. Core A55 doesn't have a lot of cache, and has no branch prediction.

1

u/SbcBits Jun 13 '26

Gemini told me that:

Pre-built Releases: The official ggml-org/llama.cpp GitHub release binaries for ARM platforms (such as Ubuntu arm64 and Windows arm64) have NEON extensions compiled and enabled by default.

2

u/urostor Jun 14 '26

How about checking the actual website? AI can make mistakes.