Spec was good, all tests green, app worked. If I didn't actually go in and try to understand the code I would have shipped it like this.
So here is my question to people saying it's solved : do you actually know that it is or does it simply appear so because all the tests are green?
[–] user43928 4 days ago
[–] n4r9 4 days ago
[–] user43928 4 days ago
When I looked at a benchmark, it suggested Sonnet 5.5 wasn't much cheaper than Opus 5.5 at Low to Medium effort for similar performance. That might not be representative of real world use though.
[–] n4r9 4 days ago
[–] user43928 4 days ago
Optimizing performance is one of the tasks best suited to AI, since the outcome is measurable.
I can't comment on complex algorithms specifically, but it's not at all the case that this works only for basic apps or cannot handle complex, performance critical work.
[–] prmph 4 days ago
First, you question the model. I have reams and reams of conversations with Opus 5.5 that shows it makes architectural messes all the time
And then you are saying you needed to have performance tests (which many devs don't even bother with) to catch bad architecture?? lol
So what if the architectural problem is not one that would be caught with a performance test? It seems you lack imagination to think of all the ton of issues that that would be very complicated to catch with tests.
[–] user43928 4 days ago
I don't think I commented on performance tests at all.
Maybe I lack imagination here, yes, but generally I go by experience rather than talking about imagined issues. That is precisely what many here seem to do when it comes to explaining why AI couldn't work for them, and why reviewing the code remains necessary.
[–] prmph 4 days ago
Me: Why this:
> I'm adding a lookup to persistEphemeralVFolder so that a repeated search reuses the user's root vFolder with the same query instead of hitting the unique index.
Claude Opus 5.5: A search stores its query as an ephemeral root vFolder. The new unique index on (user, parent query, query) now rejects a second root vFolder with the same query. As written, persistEphemeralVFolder always inserts, so repeating a search would fail its insert, the retry with a fresh short code would fail too, and the search would return an error. The Ephemeral VFolder paragraph we rewrote in the SPECS also says a search reuses the existing vFolder with its query, so the user gets the same short code back.
The lookup I proposed would also copy the query-then-create logic already in the PUT /vfolders handler. A cleaner option is one shared engine function that finds or creates a vFolder by query, called by both the handler and persistEphemeralVFolder. Shall I do that, or keep the local lookup?
Churned for 6s · done 1:10 PM
Me: I mean, is it not cleaner to fix this: "persistEphemeralVFolder always insert"
And generally, we want the db and its indexes to do their work, not duplicating that in application code with pre-lookups and stuff
Claude Opus 5.5: You're right. Both persistEphemeralVFolder and the lookup I put in the PUT /vfolders handler should be replaced by a save with conflict keys (user, parent query, query), so the unique index decides whether a row is created or updated.
[–] user43928 3 days ago
[–] Arkhaine_kupo 4 days ago
if the cost of writing code collapses, what are the barriers to including these kind of tests?
It seems like the flow and work when using LLMs has to change, spend longer on the plan, spend more time on defensive testing, have more comprehensive tests (input validation, cybersecurity, performance, data hygine) and less time on functionality
if you want to write code the old way where the architecture is human readable and the tests are an after thought, then thats on you if you then ignore the part where the human has to read it.
TLDR: either go full agent, or actually learn the code being produced, but not both.
First of all, it sounds unlikely to me that such an implementation was made with Opus 5.5 while it never mentioned the workaround.
Secondly, if the performance was indeed inadequate, that's something I would have caught during QA testing and simply have had the agent fix it.