1. There is uncertainty around training bigger models, since you need to make sure your hyperparameters & data mix will work or else the run dies, and the larger the model the more expensive it is for a run to die.
2. Larger models take longer to run RL on (since each token costs more to generate in a rollout).
3. We haven't reached the saturation of RL given current parameter counts.
4. Larger models are less profitable at inference time.
For example MAI-Base-2 has 80 billion active and 2.8 trillion total parameters (Kimi K3 size). This was trained on 16k GB200 GPUs over 38 days [1]. As far as I know, the big labs have access to much much more compute than that, and could train a much larger model but have chosen not to. GPT-6 Astra is likely not much larger than 4T params (and the graph depth mentioned in the reference could be alluding to looping, not extra parameters). [2]
You could imagine that as researchers get accelerated and get better data and the current parameter counts get saturated, the companies may be more willing to go for much larger pre-training runs.
As an aside, there is a very real possibility that the labs are already pre-training much much larger models than even Mythos/Fable/Astra, and then distilling that into the models they actually sell.
[1] https://microsoft.ai/news/frontier-pre-training-at-microsoft... [2] https://x.com/merettm/status/2095023204993490967 (plus the widely reported 1.7T param number for GPT-4)