points by embedding-shape 11 hours ago | 0 comments
> There is always a huge discrepancy between benchmarks and real world capabilities.
I mean of course, the real world is messy, the models are only good for what they've been trained on, and post-training frequently takes benchmark scores into account too, leading to this sort of "Great score, sucks in the real world" syndrome that seemingly most models are affected by.
Aún no hay comentarios.