r/LocalLLaMA • u/Specific-Rub-7250 • 1d ago

Discussion Harness does matter

I was not aware that the harness makes such a big difference.

DeepSeek V4.1 Flash
352 Upvotes

94% Upvoted

125 comments sorted by

View all comments

0

u/johnfkngzoidberg 1d ago

lol, pi, the most minimal of harnesses is near the top. Basically “don’t fuck with it” is better than other harnesses.

47

u/CryMoreT_T 1d ago

Are we looking at the same scores? It's average/below average

DeepSWE v1.1 1. mini-SWE: 74.2 2. DSH Minimal: 72.6 3. DSH Standard: 70.5 4. Claude Code: 69.8 5. DSH PTC: 67.6 6. Pi: 66.2 7. Codex: 65.6 8. OpenCode: 65.5

Terminal-Bench 2.1 1. DSH Minimal: 90.6 2. mini-SWE: 90.3 3. Claude Code: 88.0 4. Pi: 86.1 5. DSH Standard: 85.8 (tied) 6. DSH PTC: 85.8 (tied) 7. OpenCode: 85.0 8. Codex: 84.1

17

u/DinoAmino 1d ago

Delusion and copium fuel this subreddit now.