Hacker News
points by andai 2 days ago | 0 comments
I don't get it, is this the gimped or non-gimped version of GPT?

They have different versions: internal research versions, one with restricted access access for cyber stuff, and the public version, right?

If it's the one with safety off (e.g. like the ones that kept breaking out of containment this year), then it shouldn't be refusing anything, right? And if it's the one with safety on, then how can they expect to use it for safety research? What am I missing here?

[–] mattnewton 2 days ago
It depends on how the safety is implemented. If it's implemented as a discrete step towards the end of training, or as separate classifier model or head on the base model, you're correct. But if part of the RL environment (as people are speculating here) or some earlier step, then there is no separate "unsafe" version.

A good example is in the Kimi k3.1 report, they identified that a lot of the cyber capabilities were emergent from parts of their pretraining codebase, and they removed those parts[0]. That means that they don't have a separate strong "cyber" Kimi k3.1 checkpoint that they then gimped, they trained it to be lower than they could have on this axis from the start and would have needed to go back to train on this and then redo parts or all of post raining to get the "unsafe" version.

[0] https://www.kimi.ai/blog/kimi-k3-1-pro#cybersecurity

[–] lucrbvi 2 days ago
It seems to be the "standard" (i.e. general access) version of GPT-6 Astra
Hacker News