Hacker News

points by user43928 4 days ago | 0 comments

It's obviously getting better, by the month, and the degrading of training data leading to a collapse has been a fantasy from the start.

"Perceived productivity gain" is a baseless conclusion. At this point it's abundantly clear that the productivity gain is real and massive.

Where I see the incoherence is in that people see too much value in code review and optimizations.

I believe many of the optimizations are premature. If you did not spot the problem in your QA testing, chances are it was not that important in the first place.

If the architecture wasn't perfect, it is now extremely cheap to change it later, as you realize real deficiencies. The agent will do it for you in a matter of hours, and largely on its own.

You will have to budget a lot of QA effort for verifying the significant changes before going to production, but I don't think it's a deal breaker.

[–] Timwi 4 days ago

> At this point it's abundantly clear that the productivity gain is real and massive.

Oh? Where is it? Why am I not seeing it as a user if it's abundantly clear? In none of the software I've used frequently in the last year have I seen any of those “real and massive” improvements. Not saying they don't improve at all, but the rate hasn't increased since the advent of LLMs.

I'm still waiting for that shiny new era of software productivity to impact any actual users.

[–] rolisz 4 days ago

I wrote several tools for myself that have a huge impact on my personal life (a calorie/food tracker, a very custom Todo app that works by combining several strategies that I learned over the years, a personal knowledge base that surfaces information better for me). But it's a personal app, I'm not gonna make it public

[–] dimitrios1 4 days ago

Congrats? I used to write one of these every so often in a couple hours when I was learning a new programming language.

Don’t take this as directed at you because I don’t know your background, but I’m starting to wonder if many who called themselves engineers here that have similar takes to this ever knew how to actually program to begin with.

[–] rolisz 4 days ago

I mean in total it's about 300k lines of code (across all the tools I built for myself). It's a poor metric, but it's not something you put together in a couple of hours.

Oh, and specifically the food logging part: no, you didn't do that in a couple of hours years ago. Taking pictures of food and having estimates grounded in your personal "recipe" library was a major research project until vLLMs came out.

[–] spence-s 4 days ago

There is still like 20 free versions of these on the market (pre-ai) I am sure yours is barely different. You just wasted time and tokens and pat yourself on the back for it and somehow feel this can be applied generally as "productivity" and "massive imporvements in software"

[–] rozularen 4 days ago

sure dimitrios, you were producing a fully fledged UI + backend in a couple of hours while also learning a new language.

you also were doing another 2 or 3 tasks in parallel while implementing them - just like with AI

sure, dimitrios :)

[–] dimitrios1 4 days ago

No not fully fledged and feature complete. But a baseline POC with the core functionality? Isn’t every react tutorial these days a Todo app?

Anyways my point wasn’t to try and make myself out to be some John Henry (I could have done without the snarky "congrats", apologies). It’s that drowned out in the noise of AI based development, we forget that there are amazing software engineers out there (I’m not placing myself in this category) that were incredibly productive in a short amount of time. Quite some time ago, I watched, in awe, a live stream of two prominent Go developers completely rewrite the HTTP package, for example. And this was over the course of a night.

We need a new metric to gauge AI usefulness other than “I built this thing quickly”

[–] gamblor956 3 days ago

It used to be that most programmers were good enough to reimplement a basic database app in a few hours as a way of learning a new language or framework.

The pool of programmers today is many times larger and less trained and so correspondingly most of today's programmers simply aren't very good and are astounded by feats that were once basic learning tasks.

[–] kragen 4 days ago

I did that when I learned React, but it took me all night, not a couple of hours.

[–] podocarp 2 days ago

Yes, now you just produce the app and not learn anything. That's how you do it in 4 hours and parallel.

[–] hobo123 4 days ago

Maybe because a year ago the productivity wasn't yet massive, and adoption was still niche.

Both of those things have changed a lot, and I think a year from now you will definitely see changes (maybe not massive, but still).

[–] user43928 4 days ago

Just as one reference point, for my mobile app, it took me 6 months of approximately full time workload to get it ready to submit the initial release.

In the beginning I was using GPT-5.4 in April.

I can't speak to what other devs are building, and there's certainly no lack of people pushing out slop made in a weekend.

Companies and teams with code review are not adapting at the same speed.

I figure it will take a few additional months for the benefits to become more noticeable to end users.

As someone working on the frontier, so to say, I believe I have a better picture.

[–] kragen 4 days ago

> I believe many of the optimizations are premature. If you did not spot the problem in your QA testing, chances are it was not that important in the first place.

QA testing can't find missed optimization opportunities unless the performance is absolutely abysmal. Even then, it can't distinguish abysmal performance due to missing easy optimization opportunities from abysmal performance that's going to require you to sacrifice functionality.

If an app performs well because it caches the result of a slow function, it's going to pass QA just fine. If optimizing that slow function would make it fast, you can drop the cache and have a simpler and more robust app — but QA can never tell you that.

[–] user43928 4 days ago

So in that scenario it passed QA just fine, because the cache made it fast anyway.

Is having a simpler and more robust implementation worth the time to review, understand, and improve it? I'd put that at easily 5-10x the effort.

If in the future performance was found inadequate, you could still have the agent revisit the slow function.

That's what I mean with the optimization being premature.

[–] kragen 3 days ago

Having a more complex and less robust implementation adds a multiplier to all of your future costs. When a system is 200k lines of code instead of 10k lines of code, every new feature you add and bug you fix takes 5–10× the effort.

[–] user43928 3 days ago

I had similar concerns when I was working with GPT-5.5 earlier this year. At the time my project was perhaps ~150k LOC.

After struggling with some more complex features even after many iterations, I started to doubt, and questioned whether I'm writing myself into a dead-end.

I did not. I still made progress and things worked out just fine. Then Fable and later GPT-5.6 Sol came out, boosting productivity and complexity of features I could implement successfully.

The effort only ever went down, and my app was released in the AppStore last week, at about ~500k LOC.

I understand why people might believe that using AI and not manually cleaning up suboptimal or overly defensive implementation would eventually result in an unmaintainable mess.

Based on my personal experience and 500+ hours put into my project, I can confidently say that this is not the case. Increased effort to maintain is negligible.

[–] kragen 2 days ago

Can you sketch out a little bit of what your velocity looks like? What's a feature you implemented recently, and some bugs you fixed, and how much effort they took? Have you had an AI-driven project get bogged down with decreasing velocity, and what did that look like? Maybe that's what you mean by "struggling with some more complex features"?

It's probably true that keeping things simple is a little bit of a religious issue for me, but maybe my experience with humans writing software doesn't apply in the same way to AI coding. Maybe a lot of my religious beliefs are wrong!

[–] user43928 2 days ago

In the last two weeks, 51 MRs, +50k -7k LOC.

I find it hard to talk about any major feature without doxxing myself, since I'm particularly proud of the features that differentiate my app from competitors.

Trying to keep it generic, 3 MRs were for optimizing the app for the iPhone Duo. My app ships 7 AI models that run on-device.

Back then with GPT-5.5, I wouldn't say that the problem was decreasing velocity because of complexity adding up.

It was more like that the open problems were difficult. Matching animations to the gesture, manually handling scroll position in a browser during rotation. When you're already 10+ rounds into instrumenting and layering fixes on top of each other, it can get frustrating.

However, things worked out eventually. I suspect with today's Opus 5.5, many of these debugging rounds wouldn't have been necessary, since it can work for 2h+ on its own with the simulator, which GPT-5.5 at the time did not.

Yes, it's an interesting topic, and at work I've often felt accomplishment from simple, efficient solutions.

Even today Opus 5.5 suggested a plan that I'd call ridiculous. Instead of using a simple package manager feature, it proposed manually handling the issue and ensuring consistency via a new check in the CI. It was idiotic, really.

At the end of the day, it probably wouldn't have made a difference if I went with it, since the proposed ridiculous solution would also have worked. If my CI would ever be bogged down by 50 jobs, I could always start a conversation and probably resolve it in half a day.

And that's with today's Opus 5.5, while the capabilities are now improving every other month.

This ended up a pretty long comment, but I guess what I want to say is that in my experience AI is working well also in large projects, and that many of the fears around maintenance and technical debt have not manifested for me in practice.

[–] kragen 2 days ago

Thanks, this is super informative! It sounds like maybe you weren't running multiple concurrent agents in the GPT-5.5 days, but now you are? Maybe you're on one of those US$200-a-month plans?

[–] podocarp 2 days ago

Just reflect on the fact that systemd has 1.6m lines of code, and you're shipping a third of that in a mobile app.

[–] kragen 2 days ago

I'm running systemd on my cellphone right now, and I also have installed, for example, Firefox, LLVM, the Whisper speech recognizer, and PARI/GP. It sounds like you're implying that the amount of code is unwarranted by the functionality, but he hasn't told you anything about the functionality, only what platform it runs on, which is mostly irrelevant to the functionality.

[–] hbrn 4 days ago

> I believe many of the optimizations are premature

Does it work both ways? How hard should we fight premature optimizations that LLMs are introducing? Or the crazy amount useless tests?

[–] user43928 3 days ago

I think it's usually not worth it.

If the test suite takes too long, it can easily be cleaned up later rather than spending extra time judging test cases in every new MR.

Maintenance of test cases has also become trivial, even with thousands of them. When the pipeline goes red on master I click the new chat button and run /check-pipeline, where I put some concise instructions, and the failure gets investigated and resolved in short order.

[–] hbrn 3 days ago

It sounds like the problem isn't really that engineer's optimizations are premature. The problem is that engineers have an opinion about the code that doesn't really matter.

What I think you're saying is that opinions that LLMs produce are good enough and malleable, so why even have an opinion?

Which is debatable - oftentimes their opinions are not good enough, and not malleable at all.

We're all still learning how to distinguish those. But in order to learn this, you have to read and understand the code they produce. LLMs can migrate a codebase to Rust, but they still suggest walking to a car wash. Every day they make decisions that humans would be fired for.

LLMs are definitely not yet capable enough to blindly trust them, and there's a pretty good chance they never will be.