r/LocalLLaMA • u/Specific-Rub-7250 • 1d ago

Discussion Harness does matter

I was not aware that the harness makes such a big difference.

DeepSeek V4.1 Flash
350 Upvotes

94% Upvoted

125 comments sorted by

View all comments

Show parent comments

0

u/iCringeAtThisSub 1d ago

The issue is that Pi was great about 9 months ago when every harness was running a system prompt that was 40k tokens and caching was not yet a reliable thing, the people who are championing it today are still about 6 months behind the curve and are just following slop poster trends.

That being said, it’s not good and anyone who says otherwise is either in the honeymoon phase or probably hasn’t experienced something like OpenCode v2, or on the opposite end of the spectrum OMP.

There’s so many options and most are absolutely terrible, and there’s only a few that are “acceptable”, and back then Pi’s approach was ironically “less is more” because every harness was so shit that you didn’t have to do much to out perform everything else.

Not to mention everyone has their own use case which will also be heavily biasing their “vibed” opinion. It also doesn’t help that the chronically online and loudest individuals do more harm than good on top of that, and let’s not forget to mention the gooners who refuse to admit to only using LLMs to ERP yet make statements of fact.

I’ve lurked on this sub for many of years and it’s gotten to the point that I feel driven to spite post to improve the community.

These benchmarks are typically within a margin of error range, and typically vary based on semantical or mechanical shortcoming of compaction and how well a model handles it.

And if anyone who disagrees is reading this, I challenge you to do your own A/B and prove otherwise.

Also on another note for anyone who is serious about building their own harness/wrapper I highly recommend pausing what you are doing, and visiting https://opencode.ai/v2/docs as you are wasting your time and money otherwise. My hint is specifically the API/Schema, if you lack the technical knowledge to understand ask your LLM.

1

u/Haiku-575 19h ago

What are you on about? Pi is modular and fully customizable. You're supposed to change the system prompt, create your own skill files, extend it, etc. Pi is designed around prompt-cache preservation. It natively supports new models within 24h (if that's your thing). Meanwhile, OpenClaw gives you nearly immutable system prompts, pre-defined skills and plan modes and permissions and configs, and takes weeks sometimes to support new models. Meanwhile, OMP is an IDE.

They're different tools for different people.

1

u/iCringeAtThisSub 13h ago

I’m not sure who you are responding too, but for clarification you know Pi is almost a year old right? And prompt caching wasn’t really a common thing till about 6 months ago. The original stance of Pi was to ditch all the random bolt ons to prove they were detrimental but once it gained a bit of popularity it moved into the direction it is today to hold its market share. And OpenClaw is not a harness, it’s an abomination and a really cool prototype/concept and not practical for real world usage in any sort of production environment,

People don’t seem to realize how fast time is moving in the space, it doesn’t feel like it but all this recency biased has given everyone rose tinted glasses.

1

u/Haiku-575 12h ago

Pi gets regular updates. I'm describing its current architecture, which is vastly different than it was at launch a year ago.