But if the LLM can use decompilers in the process? It already uses many tools (grep, python, sed, etc...); so why not a decompiler and then take it from there?

I've seen this approach already in practice: LLM+decompiler give better code than the original code that the binary was compiled from.

I think you're right, but rather surprisingly I have some experimental data on this question that is inconclusive at best. My hobby project is symbolasm, which is intended to close exactly the gaps you describe: by naming variables and introducing types, it re-adds much of the context that assembly language erases. I ran a quick'n'dirty experiment to see if it would make the LLM more efficient at writing code than raw assembler.

The task was rewriting an MP3 decoder from Rust into ARM assembly language, Cortex-M33 flavor specifically, optimizing for both speed and code size. The output is approx 3k instructions. The tl;dr is that Opus 5.5 nailed the task, whether writing assembly or symbolasm. It took approximately 2x tokens for the latter, which I think is largely explained by the fact that it was dealing with a novel language nowhere represented in its training set.

The quality of the result was comparable; the raw asm was slightly faster and slightly less dense, which I consider insignificant because I think you can get the same effect just by tuning the prompt. In particular, if I were re-running the experiment, I might set a strict code size budget and then ask it to optimize for speed within that budget, allowing comparison on a single scalar.

The experiment was compelling in another way: the code was 3x denser and 1.5x faster than -O3 output from Rust. So for certain applications, going to assembler will be worth it, especially as the cost of generating the code goes to zero.

[1]: https://github.com/raphlinus/symbolasm