Hacker News

points by UncleOxidant 4 days ago | 0 comments

> Qwen-3.8-27B - the best thing that happened in AI so far.

Qwen3.8-Flash_Next should be in the running for that title as well. It's astonishingly good. I use both of them. If I need more memory on the machine for other things I run 27B, If I can get by with much less RAM I run QFN (takes about 102GB on my Strix Halo box with 262,144 context). QFN runs quite a bit faster on my hardware - about 50t/s. 27B with MTP starts out around 22t/s but degrades over time as the context fills up. Both are great models, but I'd give QFN a bit of an edge (at least for the coding projects I've worked on). But both feel on par with Claude Opus 4.6 - 4.8.

[–] simonw 4 days ago

Apparently Qwen 4 might land this week, and it uses the same architecture as 3.8-Flash-Next (so various LLM serving tools should be able to handle it already). I'm hoping Qwen 4 includes some smaller models - the last 4B and 9B from Qwen were for 3.5.

[–] boomskats 4 days ago

I remember seeing somewhere credible that it was going to be Thursday this week (tomorrow?), but don't remember where.

Assume you've seen the Apsara conf announcement / the planned 'tiers'[0]. Do you have a planned use case for a 4b / 9b? Hopefully the overthinking will get better on the smaller models.

[0]: https://www.yottalabs.ai/post/qwen-4-max-vs-plus-vs-flash-vs...

[–] ch_sm 3 days ago

I’m really wishing for a 35b-a3b version. It’s just such a great speed-capability trade off.

[–] fastball 4 days ago

If they actually release the weights.

[–] sigmoid10 4 days ago

The head of Qwen just confirmed that 4 and 5 will remain open.

[–] NewJazz 3 days ago

Open or "community license"

[–] tancop 3 days ago

Doesn't matter for local use anyway.

[–] t0mas88 4 days ago

Agree, I run QFN on my AI Pro R9700 (32GB) using https://github.com/Niko1221/Strata It's even faster than on a Strix Halo. Typically I get around 90 tokens/sec with a small context, down to 70 tokens/sec when it's near the maximum context usage.

[–] mrheosuper 4 days ago

I run QFN I2 XS on my 3060ti + 48GB RAM for it. Get around 2x token with 200k Context window. Good enough for me.

[–] pxmpxm 4 days ago

Err strata is just a fork of llama cpp with fancier uis, right?

[–] MadameMinty 4 days ago

Strata manages MoE expert offloading completely differently, which for QFN means ludicrous speed.

[–] MadameMinty 4 days ago

QFN is obscene. It runs faster than 27B on all my setups, and it always outdoes it in my uses (from coding to translation to vision-related): from somewhat to a lot.

[–] j_bum 4 days ago

What do you mean “outdoes” — over complicates?

[–] MadameMinty 4 days ago

Surpasses. At least in my case, 27B tends to waste more time in the reasoning stage, too, so 27B overcomplicates. QFN just does a better job. Sometimes I eyeball, sometimes it's measurable like translation chrF++ 59.3 vs 67.5.

[–] ako 4 days ago

Google: Outdo is a verb meaning to do something better than someone else, or to surpass someone or something in quality, quantity, or performance.

[–] bityard 4 days ago

Which quant of QFN are you running on your Strix Halo?

HF is telling me that IQ4-XS is the largest one likely to fit without much contortion.

[–] UncleOxidant 4 days ago

That's the one I'm running using a fork of the gufo inference engine as gufo upstream doesn't support that quant.

[–] Zetaphor 4 days ago

I'm running a regular Q4, what about that quant format justifies the fork?

[–] xnzakg 4 days ago

I switched back to glm-5.3-flash from qwen3.8-flash-next.

qwen was faster in t/s, but feels like glm spends less tokens thinking and seems better at finding issues, to the point where it ends up being faster.

[–] pxmpxm 4 days ago

Same experience, the 4bit quantized xlarge flash next runs at around 70t/s across 3 radeon v620s. Still have to retry the MTP version with full resolution.

[–] justinclift 4 days ago

> Qwen3.8-Flash_Next should be in the running for that title as well.

Unfortunately it still hallucinates, which limits it's usefulness for real world tasks. :(

[–] bityard 4 days ago

Every model hallucinates, if not given the right context and tools.

[–] justinclift 4 days ago

I find they hallucinate even with the right context and tools.

So, while useful for some people, for the things I care about they're presently pretty useless. :(

[–] komali2 4 days ago

How do you tailor your context and what tools do you use?

[–] hasperdi 4 days ago

give it information it needs to complete it's task, and not more so it doesn't get confused.