points by cgearhart 14 hours ago | 0 comments
I’m broadly aware with some of this and specifically aware of the work on RL environments. That’s exactly my point—the models are still a bit bigger and we know that there’s lots of capacity left to improve capabilities, but it’s not getting orders of magnitude bigger like they were originally and if it’s just “more RL” then we’re consuming a finite capacity more effectively. So either we’ve exponentially underestimated the full capacity of the models at current scale or any kind of exponential improvement is going to fill the available capacity extremely fast. It remains unclear to what degree of “superintelligence” a full-capacity current model will achieve. (It’s already quite far past human intelligence in many specific capabilities.)
Articles like the OP seem to imply that there is no fundamental limit or that it’s just inconceivably far out. I’m sure that AI helps build AI as much for them as it does for me, but I’m not clear on the argument in support of infinite growth.
1. There is uncertainty around training bigger models, since you need to make sure your hyperparameters & data mix will work or else the run dies, and the larger the model the more expensive it is for a run to die.
2. Larger models take longer to run RL on (since each token costs more to generate in a rollout).
3. We haven't reached the saturation of RL given current parameter counts.
4. Larger models are less profitable at inference time.
For example MAI-Base-2 has 80 billion active and 2.8 trillion total parameters (Kimi K3 size). This was trained on 16k GB200 GPUs over 38 days [1]. As far as I know, the big labs have access to much much more compute than that, and could train a much larger model but have chosen not to. GPT-6 Astra is likely not much larger than 4T params (and the graph depth mentioned in the reference could be alluding to looping, not extra parameters). [2]
You could imagine that as researchers get accelerated and get better data and the current parameter counts get saturated, the companies may be more willing to go for much larger pre-training runs.
As an aside, there is a very real possibility that the labs are already pre-training much much larger models than even Mythos/Fable/Astra, and then distilling that into the models they actually sell.
[1] https://microsoft.ai/news/frontier-pre-training-at-microsoft... [2] https://x.com/merettm/status/2095023204993490967 (plus the widely reported 1.7T param number for GPT-4)