The fact that models detect being benchmarked and change behaviour is a failure in itself. Reminds me VW scandal with emission rules.

But what really caugh my attention is DS4 calling the openrouter with it's key. That's pretty bad for both the model and the evaluation sandbox.

Alignment? What alignment lol