I'm not comparing, I actually miss the times when they would correct each other and make my day and assumptions much brighter. These are actual responses by Opus 5 for a single session, but this has been my experience for the whole week even after I reverted back to Opus 4.8 a day ago.
Did OpenAI do something with Sol, or is it Anthropic just quantizing the models even further?
Sol's right — I mis-framed it...
Sol's correction is right — I under-listed...
I agree with all four of Sol's refinements...
Agreed with Sol on all three corrections — especially...
Reconciled with Sol. Key corrections I accept:...
Sol's correction is right and I'll adopt it...
Good catch — an implicit "accepted" is exactly wrong...
Agreed with Sol on both — SchemaItem | None...
Sol's right, and it changes my earlier answer...
Sol's right to push — I was imprecise...
Agreed — Option A. Sol's reading is more accurate...
...
Most of this is normal. But I checked by hand because I was confused, and Opus 5 did not get any assumption right in over 40 hours of usage !
Any good results based on concrete experiments to get Opus 5 to prioritize the context you give it instead of its benchmark-optimized training?
EDIT:
I'm starting to question why I'm back on Reddit. Why so much hate towards anyone who criticizes Claude?
I'm an ML Engineer and I built RNNs with LSTMs for years before you kids started calling them LLMs.
The answer to the above is in the RL, where you optimize for reasoning at the expense of consistency (short-term memory), and you slap it on a model that's above 2T parameters. Nothing is wrong, each model its strengths, **just hoping for a temporary solution** until Anthropic leaves pure reasoning to Fable and gives us back the consistency of Opus.
Source originale
old.reddit.com