points by criley2 16 hours ago | 0 comments
For me, qwen 3.8 27b on my m5 max is so slow and dumb that I can't think of a single use for it. Haiku, Luna, or data center scale open models are ten times faster and ten times smarter (and inference so cheap that it's essentially free). It was a nifty toy to brag to friends about but why would I waste my time goofing with something so slow and dumb?
[–] jrm4 16 hours ago
For roughly the same reason you might slow down and read a book on how to do something as opposed to speedrunning 20 youtube videos?
[–] criley2 13 hours ago
Are you insinuating that your agents are more effective at 20 tok/s than 200 tok/s?
I don't even understand your analogy. I'm not watching the youtube video or reading the book. The model is. And a big computer can watch 20 youtube videos faster than a small computer can watch 1, and do a far better job understanding all of them (bigger model at data center scale is both smarter and faster than smaller model at local scale).
[–] LeBit 14 hours ago
On my Mac Mini M4 Pro with 48G I find Qwen 3.8 27B perfectly usable, useful and fast enough.
Are you trying to one shot create the Linux kernel?
[–] liuliu 15 hours ago