points by gadsdengrappler 3 days ago | 0 comments
This doesn’t really support the argument that KISS et al don’t apply to LLMs. They may not ‘need’ these principles for the same reasons as humans, but will still benefit from the context having higher information density.
[–] keybrd-intrrpt 2 days ago
[–] gadsdengrappler 2 days ago
> That's an assumption.
So is that ;) But yeah, the link between context length and performance reduction is well researched. Automatic context management has been getting better, but it’s not a panacea, especially in very large codebases and long sessions.
> Maybe we are ok accepting some "inefficiencies" in the code for the LLM but we get better CPU utilization or cache/RAM usage
If by ‘inefficiencies’ you mean choosing hardware-friendly architectures over those that correspond to a human-friendly world model, then I would agree with you (this was already required in my domain pre-LLMs). Even still that does not stop one from adhering to DRY, KISS, etc as the abstractions required are often either free or close to free at runtime.
That's an assumption.
And is that what we really care about anymore? Maybe we are ok accepting some "inefficiencies" in the code for the LLM but we get better CPU utilization or cache/RAM usage