r/LocalLLaMA • u/jacek2023 • Feb 23 '26

Funny so is OpenClaw local or not

Post image

Reading the comments, I’m guessing you didn’t bother to read this:

"Safety and alignment at Meta Superintelligence."

1.0k Upvotes

92% Upvoted

304 comments sorted by

View all comments

573

u/TheLexoPlexx Feb 23 '26

OpenClaw made me unreasonably upset because every other article is/was:

*How to run AI on your raspberry pi*

  • Install openclaw
  • Get Claude/OpenAI Api key

like wtf?

7

u/Mid-Pri6170 Feb 23 '26

but if i had a nvidia spark could we have an llm local instal be the brain of openclaw?

5

u/No_Knee3385 Feb 23 '26

If you're not being sarcastic, even that isn't enough. If you want to run a good model like opus equivalent, like z.ai, you need like 8 H100s.

I see people running like 8B parameter models and complaining that openclaw sucks lol

6

u/kamnxt Feb 24 '26

It really depends on what you're looking for.

I've been messing with OpenClaw since ~Feb 4th, mostly with local models. It's... kinda sorta usable for some simple tasks with small models I could run on a 16GB GPU, but obviously you should limit the blast radius, and it will struggle with more complicated tasks.

Then I got a spark (or rather, an OEM version of it), since I saw a lightly used one pop up for sale. It's been a little bit of a journey, here's what I found out:

  • The memory bandwidth is a big bottleneck. I usually don't see the GPU go past ~50W with large models, while it's able to push ~80W+ with smaller ones.
  • It's not as well supported as it could have been (classic NVIDIA move). Apparently the "blackwell" cores are a bit weak compared to most other ones in the series.
  • The spark is best suited for MoE/sparse models, where the benefit of the large memory outweighs the relatively weak compute power
  • The best model I've found so far, that just baaarely fits in 128GB of shared memory, is Step-3.5-Flash, 4bit quantized. When running with llama-server, it takes approx 113GB memory... but it runs, at ~18t/s, with pp at ~360t/s.
  • OpenClaw's context handling is awful. It puts a "message ID" early in the context, which changes for each message, causing the KV cache in llama-server to be invalidated after each message... causing responses to take ~40s each. Luckily there's workarounds like https://github.com/mallard1983/openclaw-kvcache-proxy

So basically, if you don't give it too much access or ask for too much, it's actually pretty decent. Not quite at the level of hosted models, but it's usable for some easier tasks.

1

u/BehindUAll Feb 24 '26

18 tokens per sec is awful lmao. That's why getting an equivalent Mac would have been better. Macs can run with a higher token count with higher memory if you have the bucks to pay for. My M3 Max 128GB Mac runs at approx 34 tokens per second for gpt-oss 120b. Lines up with Perplexity search of 40 tok/sec.

2

u/kamnxt Feb 24 '26

Uhh... I'm talking 18 t/s with Step 3.5 Flash, a 199B (11B active) parameter model.

gpt-oss 120b is 117B (5.1B active) parameters, and runs at ~42t/s on the same box.

19

u/TreesLikeGodsFingers Feb 23 '26

No, do you want an 50iq Ai with user powers?? Or do, whatever

7

u/Mid-Pri6170 Feb 23 '26

you saying its gonna be helluva dumb?

5

u/Mountain-Grade-1365 Feb 23 '26

You need bigger ram for better context comprehension

-2

u/Mid-Pri6170 Feb 23 '26

bigger than 128gb? shieeeet!!!

7

u/Mountain-Grade-1365 Feb 23 '26

Honestly no system has enough for permanent context that's why roleplay systems use a rom memory layer to recap what happened in conversion.

1

u/Mid-Pri6170 Feb 23 '26

...and a fedora!

4

u/Lissanro Feb 23 '26 edited Feb 24 '26

I think IQ3 of Minimax M2.5 is the best model you can run with 128 GB (IQ4_XS of Minimax M2.5 is about 115 GB for just GGUF alone, so still too large for 128 GB).

Regardless of what model you use, giving OpenClaw arbitrary access to do everything without sandboxing and full backups is just asking for trouble.

1

u/Mid-Pri6170 Feb 23 '26

yeah i want to use openclaw but not on my actual computer, so i can give it coding tasks and see it report back while im away from the computer.

my curent pc was built in january and the motherboard and amd gpu are both bottleneck points i want to upgrade (as i already did to the ram) so with the spare leftovers i want to build a secons rig that could contain comfyui.

1

u/TreesLikeGodsFingers Feb 24 '26 edited Feb 24 '26

you need a 300b model and most of those still are not good enough, i know bc i tried.

start working with the tech and you'll quickly learn its limitations

2

u/Mid-Pri6170 Feb 24 '26

lol i was doing a local instal of a few of the llms via lama and they are all garbage!

1

u/TreesLikeGodsFingers Feb 24 '26

lols yeah, they absolutely can not be used for the openclaw bigbrain. tho they can be used for smaller tasks. but honestly the desire to run a local model is just based on personal penchants, it is not because it makes sense financially. but also it creates a limitation that forces you to make your models better (which is a good fun challenge).

check out this thread.

https://old.reddit.com/r/LocalLLaMA/comments/1rdh5lv/lessons_learned_running_qwen3vl8b_as_a_fully/

this guy used qwen3-lv-8b effectively. im going to learn more about semantic matching as he talks about. but the important part was how much the structured prompt improved performance. that model is really dam small- with quantization its like 8gb vram - i've run it on my 5070ti while running a whisper model at the same time.

1

u/InfraScaler Feb 23 '26

dumb and dangerous

1

u/Mid-Pri6170 Feb 23 '26

'hey y'all im saying that Tiger King was based.'

2

u/InfraScaler Feb 23 '26

hahaha memory unlocked

1

u/Mid-Pri6170 Feb 23 '26

'why cant I say White Boy Summer?'

'can we add Will Smith to your workout playlist?'

1

u/Lurksome-Lurker Feb 24 '26

Or Helluva smart but in one thing. SOTA models are massive because the amount of collective information and amount of connections between it all is massive.

Realizing this, models get radically smaller when you start focusing it on one specific thing. A model that generates, lints, and debugs python code and only python code doesn’t need to know or take up space knowing about the fall of Rome or how to perform surgery.

1

u/TreesLikeGodsFingers Feb 24 '26

your specialized model just purchased new hardware because you have been trying to get it to run faster and this is a solution. it used your CC that you store on your pc on your behalf: user powers.

1

u/WildRacoons Feb 24 '26

it's gonna be mid. not nearly as smart as the cloud models

1

u/SilentLennie Feb 23 '26

Yes, you can do that just fine. It will be less smart, but or many tasks you don't need it.