In their last technical report (for the original Inkling model), they mentioned 'To bootstrap post-training, we ran an initial SFT on synthetic data generated by open-weights models including Kimi K2.5. The bootstrap accounts for a small fraction of compute, with the majority being employed for large-scale RL on synthetic and human-created environments.' and in their most recent one they said 'The H3 and H4 mixtures consisted mostly of trajectories sampled from earlier models including Kimi K3 and our own models, filtered for correctness and alignment.' So it does seem like traces from larger models do contribute. I think Microsoft is one of the few companies that has actively made an effort to avoid distillation with their MAI series.

Everyone is building on everything that comes before.

Houses are not built from scratch either. The know-how, power, water, roads, materials…