points by YetAnotherNick 11 hours ago | 0 comments
DeepSWE was released on May and Deepseek v4 flash was released on April. By definition it couldn't be benchmaxxed on this particular benchmark. My guess is the mutation is not like real world and it is tripping small models, not benchmaxed model.
[–] viraptor 10 hours ago
[–] jameson 7 hours ago
DS flash also has V4.1 update and supposedly achieves 77.0 on deepswe v1.1
[–] formerly_proven 7 hours ago
That's a totally new model.
[–] YetAnotherNick 10 hours ago
That makes sense. I wish they run it on some model released before deepswe release like gemini-3.5-flash.
Edit: I think that's the answer. This page lists the score as 53.3% which is close to the 54.4% reported here https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 for the 0731 update. The original from April was called preview, doesn't list the score, but in the comparison with the update it scores 7.3%.