points by Fordec 1 day ago | 0 comments
This sort of stuff has me believing that decision model controlled/prioritized context is the future of context management beyond the raw AGENTS.md file.
Why give all the context to an agent, when the context can also be curated, updated and made concise automatically for each new problem.
[–] vkgcr 13 hours ago
[–] yfontana 1 day ago
I too feel like context management has a lot of room for improvement, but the problem right now is that dynamic context management will most likely tank your cache hit rate. We need caching that's not just based on prefix-matching for this to make sense.
[–] jeef_berky 1 day ago
My bandaid fix for this is having a large number of sessions or "agents" and being ok with them being inactive at times, but they maintain their context. This has to be in some kind of system with appropriately sized "departments" to justify that agent's existence, or to justify it splitting into two, or creating sub-agents, etc. Yes, there is a cost when you cold start an agent after a week, but for now I'd rather pull that whole 400k+ context back without layers and layers of /compact. And then hopefully you keep that agent going for long enough to benefit from any cache-hit discounts.
Not really a good analogy, but it feels like having an image you like but every time you view it, it runs it through another loop of jpg compression. Use it when it's most effective, otherwise let it preserve it's details.
[–] bigcat12345678 1 day ago
Yes × 1000...
Jev is the classic type of scientific discovery, any human can see it's effectiveness without actually trying it. We were only deprived the discovery because of the incredible noises created by the AGI/ASI ferver....
[–] yieldcrv 1 day ago
most people only need if statements
wait till they found out about that nanosecond latency “conditional model”
turns out I’ve been an “RLIF” training expert my whole career
[–] oblio 23 hours ago
Amusingly the biggest revolution of this tech wave will probably be the automatic if writing.
It's honestly surprising how little work gets put into building harnesses which carefully steer the LLM through sophisticated context curation. Probably due to the belief that we shouldn't care much because AGI is just around the corner.
Related (but shameless) plug : I've actually been hacking a project over the past two weeks to address this very issue. I'd like to create an SDK which embeds the evaluation of LLMs into code seamlessly, and does "behind the curtains" context curation, to really enable "hybrid" code/LLM agentic applications. (if you're curious => https://github.com/VKCR/semiotique)