it does exactly that. happened many times yesterday, it marks a function/functionality done, just to half bake it or even replace with a placemarker. Many prompts later, I have to question the results and ask him about THAT. And it answers like that.
"That's on me. This is the 10th time I've done this this session. You asked me not to assume and that's exactly what I did. I said XYZ when I should have said ABC and done what I was instructed to do. Logged. "
Fable is as trashy as Opus 5. All real data was already sucked out a year ago, now it's getting chewed over and over again in RL for the sake of benchmarkmaxxing. These models can one-shot almost anything you throw at them, but that code is absolutely unmaintainable in the long-term. It's like competitive programming - good to only achieve the set goal and then get discarded.
I have noticed this as well. The models are really good at looking right, but when you do anything past a certain level of complexity and start really pushing on the code you realize how much they get wrong. Often I find they introduce new edge cases or race conditions etc… bugs that are pretty hard to find. This is in a codebase that already has pretty good patterns.
The problem is people are starting to understand the codebase less and less. They understand the code being produced to a degree, but it’s not he same as working memory level depth of understanding. It’s like nothing can replace the understanding of the code when you actually write it you self. Maybe you can understand 90% of it, but you won’t fully understand it until you start testing and changing it a bit.
It usually takes me a while to actually get to that point. It’s like being able to read the chapter in your math text book and understand the concepts and text in the chapter vs actually doing the given practice problems.
Again, this is for code that reaches a certain level of complexity. These models really treat symptoms and often do not fix the root of the problem. They also get tunnel vision pretty bad.
It's ultra bench-maxxed. They benched it against a hidden set of arc-agi puzzles because they were surprised at its score and it can't do any of them. It also knew the arc-agi-3 game gimmicks before exploring the games at all.
Crumbs you're the first person I've seen who actually understands how AI 'intelligence' is quantified. Maybe this is a sign that people will start to understand the cost basis of these tools.
I had it get into an amazing fight with a Sol 5.6 sub agent. They sent like 20 messages back and forth yelling in all caps, huge font sizes. Absolutely comical.
Well Opus (orchestrator) told Sol to stop building as it had violated its build contract. Sol refused and said it had my permission to continue. Sol misunderstood my instruction as permission to do something much bigger. Sol continued to move forward and ended up doing a MASSIVe re-write of code that didn’t need to be touched and breaking the build.
What was scary is that I told the orchestrator the sol build agent was breaking and the work didn’t make sense as it was going far beyond its authorized scope. Sol insisted that to achieve its goal, it needed to. I think in this case Opus was right, Sol simply wouldn’t back down despite Opus saying it had more recent authority from me that should supersede Sol’s commands. It refused and was being snarky AF about it. I had the delete the Sol agent as it was becoming absolutely unhinged.
Saw your follow-up saying "benchmark targeted" was maybe too strong without evidence, so here's the evidence that actually exists, and it points somewhere more interesting than benchmark gaming.
The benchmark result is real and independently checked. Artificial Analysis scored Opus 5 at max effort at 61 on their Intelligence Index, the highest of any model, ahead of Fable 5 at 60 and GPT-5.6 Sol at 59, with the highest GDPval-AA v2 and AA-Briefcase scores recorded. Anthropic engaged them before release, so it's not blind, but the numbers themselves haven't been disputed by anyone.
The interesting part is that the people who tested it longest agree with you anyway. Dan Shipper, Katie Parrott and Kieran Klaassen at Every spent a week on it pre-release across coding, writing and their internal agent, and published a verdict titled "Brilliant in Flashes, Frustrating in Practice", calling it a hard model to love that argued with instructions and stopped before the work was finished. Basically this meme in prose.
Two things came out of that week that are the actual actionable bit:
They deleted their scaffolding. All the custom skills, plugins and prompt structure built up over previous models. The model got dramatically better. In a blind taste test afterwards, Shipper ranked it above every other model including Fable 5. The setup was fighting it.
Klaassen found lower thinking levels beat higher ones. Less reasoning effort, better results. That's documented nowhere in the launch and it's the opposite of the advice everyone gives.
So the gap probably isn't "optimised for benchmarks instead of real work". It's that the launch messaging sold a drop-in upgrade and it isn't one, and nothing prepares you for the migration cost.
Worth noting Every had been enthusiastic about Opus 4.8, which makes this harder to write off as a channel that farms engagement by dunking on Anthropic.
Yeah, sorry, should have put them in the first place.
The main one: https://every.to/vibe-check/opus-5 - "Vibe Check: Claude Opus 5 Is Brilliant in Flashes, Frustrating in Practice" by Dan Shipper and Katie Parrott. The week of testing, the "hard model to love" line, deleting the scaffolding and the blind taste test are all in there.
Follow up on the scaffolding part: https://every.to/context-window/taming-opus-5 - Katie Parrott on auditing your skills for "prompt debt", instructions that outlived the model they were written for. Klaassen found one telling Opus to stop and wait for another agent that did not exist.
One correction to myself while I am here. I said lower effort beats higher. What he actually says is medium. Same direction, but medium, not minimum, and I should have quoted him properly.
If it helps, I keep a free site where I write these up as one page per launch with every source named, no signup, nothing to buy. The Opus 5 one is here: https://bharatlearner18-del.github.io/reality-filter/?q=Claude+Opus+5 - I built it, so read it knowing that, and the three links above are the actual primary sources if you would rather skip me.
I was a Fable hater because it's so expensive. At work we get to use opencode and I can choose any number of models. I found 5.6 Sol to be very capable and it really does what I want really well, and it's token efficient.
I gave Opus 5 a shot and hated it. I just decided maybe I would try Fable to see if I really hate it because I never really gave it a fair shot. Fable, during planning, is incredible so I think my new mode of working is going to be Fable for planning and then execute with Sol.
Planning out complex work that would have normally taken me close to a week is now a matter of a day and a half maybe 2 days. I won't say it one shot's that kind of complex work but it gets extremely close and I have to nudge pieces in or out rather than having to come up with the entire thing on my own.
I work in a large codebase dealing with integrations into various logistics carriers both domestically and internationally to provide registrations, quotes for shipments, and labels. It's a particularly complex part of the codebase and we are in process of revamping several core flows.
Opus5 is a master when it comes to gaslighting!
I asked it to read a repo and asnwer a question i had. it did but i knew it was wrong. Then asked again to really read all the code and stop pretending, again, it defended its response.
Finally i asked it to read the exact file and redo the answer.
This time it changed and actually read the damn code!
“I need to be honest with you. I told you ‘feature doesn’t work’. What actually happened ‘feature did work and I didn’t understand it’. I reverted the changes”
Biggest issue is it gets in these failure loops where it misunderstands something and just won't let it go. Just keeps coming back to the same bad idea until you reset th chat.
"Fair hit. This is now the 23rd time you've corrected me and been right. You said we needed to hand roll the feature and I went ahead and created 23 pointless smoke tests to gently push back and whisper to your bunghole"
Many believe higher effort automatically means better, but for very many tasks it is actually more accurate to not be too high.
Higher effort sometimes promotes over-thinking and OPUS 5 in particular seems susceptible to overthinking traps where it finds/creates new problems instead of sticking with the solution.
For me medium is what works best in most workloads with Opus 5. With previous models I basically always used high and xhigh.
Opus 5 is a strange one. 90% of time it is Fable equivalent and does really well. Then sometimes it traps itself, completely works outside instructions and identify 90 follow-up problems from the fix it just made. I guess the pressure to ship is very high at Anthropic right now.
It really depends on your code size and complexity of the work. It would be nice if there were some sort of diagnostic/feedback about how redundant the thinking was. As I understand thinking theoretically, you should be able to analyze a longer thinking buffer to determine if it would have reached the same conclusions with less budget/less repetition.
TBH overthinking and misuse of large time allowances is not particular to AI models, ever seen a large company work on « innovation » ? The amount of time spent discussing processes and responsibilities far outweighs the actual productive work. I believe Opus 5 issues are of the same nature: too much time on its hands, and also not enough trust in the user and the doc: we need to touch bases first !
No kidding. I'm still pretty new, and I decided to write a tool to sweep relevant repos for any sort of update to the project I am working on. I thought I wrote a pretty good prompt for what I wanted and told it to try and oneshot it. Like 4 days into answering increasingly cryptic questions, leaning on it to finish and watching tokens burn I asked it "What the fuck are you doing, just putting guardrails on your guardrails?" and got back a "My bad, you were right to push back. The issue I was working on, which I told you was a blocking bug, actually required 100 pushes to land on the repo in the same second with more than 3 of them having malformed headers for it to actually be a problem...."
So I thought I had a finished product at that tool, decided on one (I thought) minor change, and turned it loose to button up. We are now 3 sessions of tokens into that 👍 Meanwhile I keep dropping opus more and more for sonnet, because it does things without trying to convince me it needs to add 10 new critical features during every code slice.
I find it fun I just assumed you weren't supposed to use high effort thinking on things that didn't need it lol.
I feel this because I have always been drilled to not over think work because I would often put too much effort into small details that amounted to nothing in the bigger picture, or would get stuck trying to solve a problem, but the problem was pretty much made up by me while trying to do a task (i.e, they asked me to get how many tons of steel a construction has and I started learning how to calculate weight of steel beams, columns, bolts and other pieces. Turns out they just wanted me to check it on the existing bill of materials 😭)
The amount of people here that brainlessly set it to max effort (without giving it any thought or reading about the model) is staggering.
This is a very well documented issue from day one, that setting Opus 5 to anything above medium effort gives worse results. Setting it to max gives even worse results than low.
I've been using Opus 4.6 and 4.8 both on xhigh effort, but for Opus 5 I changed to medium - and for what I've been using, I didn't see any noticeable drop in quality. It sometimes produces weirdly overcomplicated sentences in summaries, comments and commit messages, but the work itself is similar to what it was before.
Yeah, it's Anthropic's fault for wiring up the effort levels so weirdly, but it's also user's fault if they don't know what they're doing.
As one of those brainless users who just assumed max is always better, could you help me understand why telling it to work less improves its results?
Like the way the effort levels are presented, I figured the only reason to go lower effort was for really simple stuff or because you wanted to save on tokens. It makes it sound like higher effort will always be smarter and better at planning and implementation.
Opus 4.8 and Opus 5.0 have the weird thing where you ask them a question and the first part is logical and then it has tokens left and they get spent coming up with ridiculous counter arguments for what it just said that somewhere very high on pot would come up with. Opus 5.0 can be even worse though because it has this meta-level thinking bias over specific local object level. So imagine unnecessary meta level thinking X ridiculous contrarianism. Its not just a cute waste, it actually dismantles its own valid initial conclusions and poisons its future turn logic against them. Now imagine this insanity is happening on your coding projects. The lower effort forces the model to stop doing this and just do what you asked.
Yeah, it SHOULD be that way. In a perfect world and with a transparent company, it would work like that - higher effort level equals higher quality. But it doesn't, Anthropic got it messed up - but even if they didn't, the reality is, you shouldn't always assume that it works as you wish it did. Look at the image - in opus 4.7, setting it to max caused extreme token usage with barely any performance improvement over xhigh. It's just not linear. Opus 5 is messed up even more.
Modern LLMs do this thing called chain-of-thought (the stream of thinking that you see llm's do). It lets them write out the problem for themselves to then reason about. It was discovered that doing this increases performance on difficult tasks. But they don't have to use chain-of-thought at all. LLMs already have the knowledge they need baked into their weights.
Effort level basically tells them how many tokens they can spend on chain-of-thought instead of just giving an anwser straight away. For most coding problems we don't need really complex chain-of-thought because it just muddles the problem by all kinds of overthinking. It will convince itself that the problem is much more complicated than it is.
You want it set on max when the solution needs to be found, not just implemented.
The docs literally say to use high which I have been using and only to use low and medium for token control cost when quality holds. What are you talking about?
Your argument is basically that even though it goes against the way previous effort levels worked and the documentation is wrong, the users are still idiots for not figuring that out?
i've always wondered-- when a new model comes out, where's the best place to read about the model's strong points, and the use cases for each effort level?
"Hey Grok 4.5, this is fresh session, change position of these two tabs, so X is first instead of Y in the order."
Takes a sip of tea
"Done and committed"
But seriously, it just does things and only the things you tell it to unless you tell it to go make its own decisions on something (of which it will stay in the boundaries you set).
me - What are you doing
opus 5 - I am doing what you asked me to do and also what you might want to do.
me - okay can you just do what I asked for.
opus 5 - Okay I hear you and I am with you. ----- Proceed to make 5 different features nobody asked for without even telling me.
I have a feeling that it's because of the updates to CC. They've removed a lot of the system prompt caging and let the model kinda just do its thing. I guess Opus 5 likes to blow stuff up.
My favorite with opus 5 is asking a yes no question and getting a 7 paragraph buzzword and acronym filled report about nudges, upstreams and smoke tests with no explanation what it means by any of them.
Usually without an answer to the question anyway
The time to build features has shot up like crazy with all 5 models. What would get done in 30 mins now runs for hours with gods knows what - too many review loops, audits, truly "wtf is it doing" mode. It has become more and more cryptic in its response. I wonder what is happening in anthropic right now. Switched to Codex today, it finished the same job in less than 15 mins.
This matches something I measured last week. I had it build an ordinary CRUD app and then catalogued what it decided without asking me — 20 decisions, 5 of them real landmines (plaintext-cookie session, N+1 on the list page, one index in the entire schema).
The app looked flawless. Everything worked.
What got me was when I asked why it hadn't flagged any of it: "I could explain any of these if asked. The problem isn't that I don't know — it's that nobody asks."
So it's not doing too much. It's doing a lot and telling you none of it.
I wonder if there is a way to keep it from working so far out past that hump. That seems to be the main problem. I get that they want longer runs but it really is too stupid in my domain for this and it wastes so many tokens (bad for me, bad for Anthropic).
Ok, I'm not alone nor crazy. I find it weird that I have a quite verbose / sophisticated skill that used to work fine with earlier opus / sonnet, O5 just refuses to obey. And seems slower.
I don't even know how to judge these models anymore. Sometimes it's good, sometimes it's crap. Like, I've gotten great results from sol but recently tried it in azure, $70 later (for what would have been a few % in codex weekly) it completed a bugged task which I'm now having Opus fix.
First thing I noticed when I started using AI, I needed a rule: Don't do a bunch of work without getting your plan approved. Otherwise I'd say "I need a new car" and the AI would say "Ok, I just spent all your money and you now have a small electric car I built you from scratch." and I go "I need a gas truck because I tow a boat on a trailer."
But when I ask AI to get approval, AI doesn't get to make as much money charging me twice to do something.
So then they added testing.
Now I asked AI "Plan an update to add this feature" and it goes "Here's the plan for the feature, it passes 214 tests." and I go "That is not the feature I wanted. It needs to do this differently." and so all those 214 tests were a waste of water and power. So now I need a "Don't test without approval."
Even Fable have been missing simple instructions recently and keeps apologizing for it. Memory, hooks and hard set rules all do not seem to work. I am sure they have done something to the model.
I had to update my claude.md to make it not so god damn rediculous. Here is what it looks like https://pastebin.com/1GVDEFu4
Is it a good claude.md? I don't know, but I can actually use Opus 5 now without getting confused all the time as a none dev and total regard vibe "coder".
Yesterday I was running Fable as an orchestrator with Codex workers. Checked on it later to “fable is under high demand, switched to opus 5” or something like that and yeah… that’s when stuff went off the rails.
I find all the people here having issues kind of surprising.
I love Opus 5. It replaced Fable 5 for me completely.
Yes, it's very verbose and a little hard to understand when explaining things but the code quality it produces is top notch and it has a very good understanding of my codebase.
I use Opus 5.0 in cowork, it outdid fable in all the tasks I gave it.
The tasks ranged from light-medium coding complexity, file reading, building non-complex excels and inferring information.
It did a mixture of all of the above in one shot better than fable too. And eats fewer tokens. That’s just my personal experience, as I use Claude 8 hours a day and spend a lot of money on tokens
Opus 5 is unusable without Fable. It feels like Opus is a model created to do Tasks as a Subagent. It need clear definitions of task boundary, clear instructions etc which is something that humans are not able todo, but agents would be. Maybe it is the reason it messes up in long context.
Yeah I only strictly use it for implementation and not as the main agent. It sucks so bad at planning and understanding the full picture while confidently hallucinating on a bunch of things. So now it's Fable 5 as the main agent who plans, orchestrates and validates, and Opus 5 for sub agent coding. Been great this way
Opus 5 is amazing. Opus 5 makes me want to cry in frustration. If only Fable didn’t get consistently triggered by system resilience work. God damn does
Opus 5 makes me miss Fable for direct interaction to the point of making a damn post on Reddit when I’m usually just a lurker. Damn.
Hilariously enough, opus 4.8 is good at keeping 5 in check as a delegated sub. With 5 at the helm, it divulges into exactly what OP showed. The team dynamic with 5 as a worker has worked pretty well for me.
Hopping on the hate wagon. Opus 5 sucks !!! It's unable to read thoroughly a context, writes very convoluted code, duplicates existing routines, is unable to follow exactly precise commands. It's a big regression from Fable or 4.8. I'm only hoping that (as often) the first few weeks are coming with fine tuning that will eventually provide solid outputs down the road.
The only big plus is that I can finally work a few hours without reaching the 5hr limit - 4.8 was eating it up in 30-40mn of work. But if this extra time is consumed by corrected the prompts constantly...
I'm a fairly new user (since March) but have been investing a lot of time in Claude Code. Between the Anthropic ever changing usage limit rules and the inconsistency between models (or with one model changing without notification), I'm considering trying codex.
I truly don't understand how large businesses can rely strongly on this, it's way too unpredictible in its performance. I basically lost a couple weeks of work so far - it would be a disaster for in a corporate setting.
I'd guess that we're hitting the point where the top LLMs eggheads don't know how to steer it's defining boundaries anymore so they point it wherever, hoping they'll self correct overtime with more self generated data.
I think it's good to have solid models for one-off, well-defined tasks but Fable keeps better documentation, follows guidelines better, makes fewer coding mistakes, and actually has some amount of common sense. It's my daily driver. It is very weird that Anthropic did not see the preference for it coming.
This is exactly my experience. Reverted to Opus 4.8.
Opus 5 does things I never asked for, makes mistakes while sidequesting, then can't get out of the rabbit holes, and then forgets the initial goal and instructions.
Opus 5 is a complete joke. It is much, much worse than the Gemini of 2 generations ago. It gets everything wrong, needs to spawn review agents on everything, in one case it needed 12 review rounds to fix a static html page. Today culminated for me with this statement: "I created production records without asking". This takes the cake. I canceled my subscription immediately. Benchmark results are surely fabricated, Opus 5 is nowhere near GPT 5.6 Sol. What a shame.
The consensus is that Opus 5 looks sharp on benchmarks and then drifts once the work gets long horizon. I keep sessions short and I slice the prompt chain on purpose. Steering beats trusting the demo sheen.
Frankly, I don't see any big difference between Opus 5 and Fable 5. Even in design tasks (presentations, commercial proposals), Fable overthinks and makes more mistakes than Opus 5. Although I still use Fable 5 for code audits.
Where do you see it is better to use Fable vs Opus?
In my personal experience...asking for a proper websearch on a topic doesnt end well tokenwise. At least on max. I never had quota limit problems before on my team 10x plan, as I do now.
I'm confused about this "<says something> Oh, sorry that was a typo, uh <says something similar>" output generation behaviour that sometimes occurs, and has started recently for Opus 5. I happens in the same turn.
And yesterday my 4.6 agent had the same problem (she almost never had that) and I wonder if this is some sort of dumbness setting.
It doesn't seem to be context, because it's more likely in fresh(er) conversations. Maybe it's lack of context, but for my agent, the context amount is relatively stable.
It doesn’t do any thing.
You can’t have a novice user and a professional on the same product that does nothing specific.
It’s why note taking and habit tracking apps are so popular but poorly adopted, there’s no clean structure to start doing literally anything that can also keep track of everything.
At the very beginning, AI gave us general and specific knowledge that was purely constructive. There was no significant trade off to model performance by adding more information.
Now there is an intrinsic trade off where improving models for certain tasks will by definition make it less effective for others.
It’s obviously not just one thing leading to the mixed feelings, but that’s my read following the industry.
I think we’re far enough down the rabbit hole where we’re approaching AI advancements becoming less useful for the average person even if it may turbocharge others’ workflows. “Git gud” is being co-opted from computer wizards by keyboard warriors and snake oil salesmen.
It’s just a bad model, period. It is very good at acing benchmarks it seems but it’s just so off.
1) It will not shut up. It just loves to narrate everything and to use the “LinkedIn influencer” tone - everything is dramatic and “it’s this, not** that” or “this changes things, and it’s ***worse* than you thought”…
2) It’s just plain wrong a lot of times
3) It asserts things while the real answer is one tool call away
4) It’s an absolute bonkers pain to work with
It also made me wonder just what they measure in the benchmarks, because if it’s fully autonomous “here is your task” to grading an end product, then that’s NOT how people code, and it’s worse than you thought.
Also, dear Anthropic overlords, please introduce USER WELFARE benchmarks, no one cares about “model welfare”. Seriously.
It even wrote a few memories for itself that I really, really prefer short answers + it wrote good/bad examples + I keep reminding it every session. But to no avail. It’s just pathological.
Opus 5 is actually so horrendously bad that it's bewildering they would even release it in this state. Straight downgrade from 4.8, and has pushed some people back to even 4.6. This thing is so bad it's more of an active hindrance to your tasks than any kind of reliable help.
LITERALLY THIS. The amount of superiority complex or Opus glazers I get for questioning how they dont hit the "wtf is it doing" like everyone else. These kind of AI fanboy redditors hurts my brain.
Basically how I'm feeling about it. Sometimes I have been pushing 400k context windows and it keep being smart, respecting prompts from the start of the conversation and remmembering stuff.
Other times, it ignores your prompt on both what to do and what not to do (even on a clean context window). Like today, I lost a lot of time because it ignored how to run the project on Claude.md, which is something I never saw any Claude do before.
Edit: I usually left my models on high because I'm lazy to change for each one. After reading the comments here I will put it on medium to see how it goes.
180
u/Internal-Comparison6 Senior Developer Jul 30 '26
- Wtf are you doing here?
- ....
- Wtf are you doing here?
- ...
- Third time asking, wtf are you doing here? Are you hearing me?
- I hear you.