I’m kinda surprised that we expect the models to have infinite capacity to improve. It seems clear that we haven’t hit any fundamental limit yet, but that doesn’t mean no limit exists. If it does start improving exponentially, then either we’ve dramatically underestimated their capacity or they’ll hit that limit quite quickly, right?
[–] namr2000 17 hours ago
[–] cgearhart 14 hours ago
Articles like the OP seem to imply that there is no fundamental limit or that it’s just inconceivably far out. I’m sure that AI helps build AI as much for them as it does for me, but I’m not clear on the argument in support of infinite growth.
[–] namr2000 8 hours ago
1. There is uncertainty around training bigger models, since you need to make sure your hyperparameters & data mix will work or else the run dies, and the larger the model the more expensive it is for a run to die.
2. Larger models take longer to run RL on (since each token costs more to generate in a rollout).
3. We haven't reached the saturation of RL given current parameter counts.
4. Larger models are less profitable at inference time.
For example MAI-Base-2 has 80 billion active and 2.8 trillion total parameters (Kimi K3 size). This was trained on 16k GB200 GPUs over 38 days [1]. As far as I know, the big labs have access to much much more compute than that, and could train a much larger model but have chosen not to. GPT-6 Astra is likely not much larger than 4T params (and the graph depth mentioned in the reference could be alluding to looping, not extra parameters). [2]
You could imagine that as researchers get accelerated and get better data and the current parameter counts get saturated, the companies may be more willing to go for much larger pre-training runs.
As an aside, there is a very real possibility that the labs are already pre-training much much larger models than even Mythos/Fable/Astra, and then distilling that into the models they actually sell.
[1] https://microsoft.ai/news/frontier-pre-training-at-microsoft... [2] https://x.com/merettm/status/2095023204993490967 (plus the widely reported 1.7T param number for GPT-4)
* Better RL environments that themselves are generated by agents
* Architecture improvements that increase effective capability per parameter or FLOP
If you're curious about the latter I'd look into manifold constrained hyper connections, engram, deepseek compressed attention, kimi delta attention etc. At first these might seem like just speedup tricks, but several of these allow the overall capacity of the model to increase per training FLOP or parameter. This then lets pretraining & the RL environments grow & be more effective.