This might be the beginning of the reason we shouldn't be reading the code. It's like in chess engines where the players complain about how weird, alien and unintuitive the moves are. But 100 steps down the line the engine finds some edge case exploit that gives it enough advantage to win.
So we may be crossing a line, right now, where the non-intuitive alien code being generated by agents is seeing edge cases down the line that might allow exploits. And if we "correct" the agent, we may be leaving ourselves open to exploits we don't see and have difficulty understanding.
[–] ShinyLeftPad 2 days ago
You're saying it as if humans can't write that too.
[–] stillpointlab 1 day ago
I am arguing that, much like the "best" chess move is unintelligible to best chess players, truly secure code will be unintelligible to humans.
[–] akiarie 2 days ago
[–] stillpointlab 1 day ago
The point of my comment is, no amount of "increased understanding" puts a human on equal footing to a chess engine.