It depends on how the safety is implemented. If it's implemented as a discrete step towards the end of training, or as separate classifier model or head on the base model, you're correct. But if part of the RL environment (as people are speculating here) or some earlier step, then there is no separate "unsafe" version.

A good example is in the Kimi k3.1 report, they identified that a lot of the cyber capabilities were emergent from parts of their pretraining codebase, and they removed those parts[0]. That means that they don't have a separate strong "cyber" Kimi k3.1 checkpoint that they then gimped, they trained it to be lower than they could have on this axis from the start and would have needed to go back to train on this and then redo parts or all of post raining to get the "unsafe" version.

[0] https://www.kimi.ai/blog/kimi-k3-1-pro#cybersecurity