While the distillation argument is brought up again and again, I find it very dubious. This is not to say that it doesn't happen, but I have my doubts that it is a limitation. DeepSeek V4.1 Flash, for example, can reason itself through some pretty complex mathematical problems. Definitely not millenium-prize level problems and frontier models are superior for research-level mathematics, but it is still surprisingly close even for some of that [1, 2].

Where it differs from the larger models is that it has less knowledge; respectably as it performs for its size, 552B parameters are still a lot less than the multiple trillions that frontier models have. Sol/Astra/Opus/Fable may know the solution to the problem directly or to a very adjacent problem, while DeepSeek has to work its way through it from first principles and then sometimes fails (because LLM reasoning is really a heuristic-guided search). But this is also a failure mode that I have seen with Luna.

[1] As evidenced e.g. by ErdosBench. https://www.ulam.ai/erdosbench

[2] Using this as an example over software engineering tasks here, because it tests reasoning capability more than knowledge retrieval.