points by wisty 2 days ago | 0 comments
The article notes that this is mainly because the cheap models tend to be distillations of the frontier closed source models, and the developers of the original closed source frontier models have an edge in training their cheaper versions (since they have the source). I'd add that they also have funding, and want to blitzscale. OpenAI / Google can offer a cheap model to get more training data (plan dependent), or have a lot of capacity due to training requirements.
It is also evidence of commodicisation / brutal competition. I wonder if they cut costs because there is stiff competition from open weight models (and arguably we are in the blitzscaling phase where they try to corner the market, not the enshitification stage).
Where it differs from the larger models is that it has less knowledge; respectably as it performs for its size, 552B parameters are still a lot less than the multiple trillions that frontier models have. Sol/Astra/Opus/Fable may know the solution to the problem directly or to a very adjacent problem, while DeepSeek has to work its way through it from first principles and then sometimes fails (because LLM reasoning is really a heuristic-guided search). But this is also a failure mode that I have seen with Luna.
[1] As evidenced e.g. by ErdosBench. https://www.ulam.ai/erdosbench
[2] Using this as an example over software engineering tasks here, because it tests reasoning capability more than knowledge retrieval.