Hacker News

points by marcus_holmes 3 days ago | 0 comments

I cannot read a 50-page T&C document. I mean, obviously, I can physically do it, but I don't have the patience, and I'd lose track of what I'd read. Getting an LLM to summarise a 50-page T&C document is simple, it doesn't lose track, and it spits out a summary in seconds.

I cannot read a 50K LOC code base at once. I mean, obviously, I can physically do it, but I don't have the patience, etc. I need mental shortcuts like abstractions layers to hold even parts of the code base in my head. LLMs don't. They can read that in seconds and hold the whole thing in context.

[–] gadsdengrappler 3 days ago

This doesn’t really support the argument that KISS et al don’t apply to LLMs. They may not ‘need’ these principles for the same reasons as humans, but will still benefit from the context having higher information density.

[–] keybrd-intrrpt 2 days ago

> but will still benefit from the context having higher information density.

That's an assumption.

And is that what we really care about anymore? Maybe we are ok accepting some "inefficiencies" in the code for the LLM but we get better CPU utilization or cache/RAM usage

[–] gadsdengrappler 2 days ago

> That's an assumption.

So is that ;) But yeah, the link between context length and performance reduction is well researched. Automatic context management has been getting better, but it’s not a panacea, especially in very large codebases and long sessions.

> Maybe we are ok accepting some "inefficiencies" in the code for the LLM but we get better CPU utilization or cache/RAM usage

If by ‘inefficiencies’ you mean choosing hardware-friendly architectures over those that correspond to a human-friendly world model, then I would agree with you (this was already required in my domain pre-LLMs). Even still that does not stop one from adhering to DRY, KISS, etc as the abstractions required are often either free or close to free at runtime.