Hacker News

points by aszen 16 hours ago | 0 comments

Interesting article to read but by defining memory as a prediction about future needs they are side stepping the real problem that these models cannot learn continuously i.e. update their weights. Opus 5.5s real memory , experience and knowledge is set in stone no amount of markdown tricks will change that.

This is a better research direction I feel

https://research.google/blog/the-sleep-framework-a-new-way-f...

[–] ffsm8 15 hours ago

Continuously updating weights will never be viable for a company like anthropic that's providing cloud inference.

So if you're hoping for that you should probably just look at the models that can be ran locally... And then you prolly realize that that's still several years into the future.

[–] DriverDaily 12 hours ago

Without updating weights, AI will always be the main character in Memento with a hyper-optimized filing cabinet.

AI that remembers through continuous experience instead of documents would be significant and lucrative enough to change the business model.

[–] tombot 11 hours ago

Could you say more about why it’s not viable for cloud inference? I’d love to understand this

[–] ford 15 hours ago

I think the end state of any productive LLM includes both: - a real-time knowledge/context graph that keeps track of state - continuously updating weights

Long term I think it's probably more interesting to explore continuously updating weights. But (and this may be yet another opinion that falls victim to the bitter lesson) even if it's _theoretically_ possible to encode everything in weights alone, I'm not sure it's not the most optimal. Today it's too expensive & finnicky to get better on your own tasks without overfitting (ie [0])

Practically (like next few months) getting the right instructions & context would be a huge improvement in capabilities.

[0] https://thinkingmachines.ai/blog/on-policy-distillation/#tra...

[–] zbyforgotp 10 hours ago

Why real memory is weights? Why the learning entity must be the model and not a composite system?

[–] ghoblin 6 hours ago

Because of generalization. Updating weights allows the model to internalize patterns and apply what it learned to novel situations rather than just retrieving and reusing stored information

[–] nullc 5 hours ago

Linear attention like Qwen's GDN is essentially equivalent to dynamic weight update.

I think for many case 'memory' not tied to immediate context is problematic, particularly if its illegible. It's common for agents to make an error and then fixate on them-- e.g. has a wrong conclusion about a root cause, states it emphatically, and then it keeps coming back to haunt you.

With memory consisting of files on disk you can hunt out and destroy the source of such confusion. If it gets buried in weights you may be SOL.