points by n4r9 4 days ago | 0 comments
Sonnet 5.5 is within a percentage point or two of Opus 5.5 on Terminal Bench.
[–] user43928 4 days ago
[–] n4r9 4 days ago
My workplace disabled Opus access in June, saying that it is basically the same performance for most coding tasks at a lower cost. That might have changed now. But the main point I'd like to make is that even with Sonnet 5.5 I have a similar experience to OP - code that makes odd performance choices, is over-engineered, and arbitrarily ignores rules we put in the context. For small self-contained apps doing basic stuff it's okay. For 100k+ line codebases running fairly complex algorithmic routines it needs heavy supervision.
[–] user43928 4 days ago
My codebase has 500k lines and some 3D work that needed optimization on mobile. At no point was it needed to supervise the code in the implementation.
Optimizing performance is one of the tasks best suited to AI, since the outcome is measurable.
I can't comment on complex algorithms specifically, but it's not at all the case that this works only for basic apps or cannot handle complex, performance critical work.
When I looked at a benchmark, it suggested Sonnet 5.5 wasn't much cheaper than Opus 5.5 at Low to Medium effort for similar performance. That might not be representative of real world use though.