Linear attention like Qwen's GDN is essentially equivalent to dynamic weight update.

I think for many case 'memory' not tied to immediate context is problematic, particularly if its illegible. It's common for agents to make an error and then fixate on them-- e.g. has a wrong conclusion about a root cause, states it emphatically, and then it keeps coming back to haunt you.

With memory consisting of files on disk you can hunt out and destroy the source of such confusion. If it gets buried in weights you may be SOL.