That's assuming the current models are able to find all the bugs, and that we won't have a better model out every few months that will be able to find even more obscure bugs.

We would also have to burn a trillion tokens a day verifying all the C code the hard way forever going forward. It would be cheaper to rewrite the kernel in Rust, subsystem and driver by driver at a time.

That's a fair question, and actually one I ask myself too.

AI doesn't give a strong guarantee that things work though, while Rust gives a much stronger (not absolute) guarantee.

But maybe the way of the future is to auto-generate proofs that the code is memory safe and then just leave things where they are, instead of doing rewrites that require maintainer buy ins. The advantage of C plus proofs would be that the AI generated proofs could live separately so it wouldn't touch workflows of the maintainers.

The only required output they have to process is evidence of vulnerabilities in changes proposed to them. They could outsource even that to contributors.

You could in principle use LLM as a formatter, linter and a borrow checker, but an actual linter, formatter and rust compiler are still faster and cheaper to run on a million lines of code. They are more reliable too because of context size limits and attention thingy (read those thinking traces when it approaches 1M limit an fucks up tool calls -- you will get the sleep deprived person vibes from it).

So first you would run the three above to pick the low hanging fruit and then ask the shaitan in a box to find more bugs. Looking from another angle -- rust is another a tool AI can use to find memory bugs, so why not have rust.

Then again, I asked the thing to write some C (more or less from scratch) few weeks ago and the next time I looked it was chasing segfaults and wondering where they came from.