The types of systems from the paper get more interesting when you’re dealing with really tight voice-based conversation loops. If the workload isn’t latency sensitive and you can just have a harness execute the types of searches you mentioned it’s rarely worth it to explore these types of optimizations or memory structures (or using embeddings at all). However, if I ask a model for something we talked about yesterday I don’t want to have to wait for it to reason and then make a tool call then read it and reason again. Even the, “let me check on that” gets tired quickly. They mentioned a 27x speed up putting the retrieval at 0.79 seconds vs 19s which is very promising for voice-based personal assistants.

This is true, but consider that we have agentic token injection (what you are referring, prompting), embedding injection (memorilla) and continual learning.

So the first will update _what_ information the model reasons/infers about, the last will update _how_ the model reasons/ infers about.

These are very different.

As far as I understand memorilla sits in the middle. And it embeds not tokens but embedding vectors with all their complexity. The compression (semantic) is significant and in my mind is super interesting.

I think semantic embeddings are only a part of the picture. A hybrid system is definitely needed but only text based approaches are miserable. My agents keep on repeating the same mistakes across sessions. We need to first get rid of this notion of a session.