What frontier models are you using? I'm literally sitting with Qwen 3.8 27B locally and trying my damnest to get it to work at least 80% of the quality Astra et al have, and can't get close to it. It keeps being so verbose, regardless of inference parameters, system prompts, reasoning effort, and keeps going on tangents. I'm running the BF16 weights via vLLM, can't imagine how much worse the experience must be with the quantized weights people run with the average consumer GPU.
Even with tight guiding, as I guess you could say I do more "AI pair-programming" than vibecoding, it just loses the plot to often and waste time, enough that it's clearly not an alternative to today's frontier models.
[–] pizza234 4 days ago
[–] zkmon 4 days ago
For comparison, I also used GLM-5.3, Muse-Spark etc, with no significant improvement for ordinary/BAU coding tasks.
[–] embedding-shape 4 days ago
99% of the time, I'm not. But if it cannot reasonably write a basic CLI for transforming data without properly following a ~1K token long system prompt to the letter, then waste my time on it?
"Ah yes, I forgot about part X" or "I misunderstood the intent of Y" are so frequent compared to SOTA models, that it's hard to justify relying on it, even though I'm able to run it pretty much 24/7 at home, if it just was intelligent enough to actually make sense to do so.
[–] badsectoracula 4 days ago
I did had it forget some stuff over long runs that went through several context compactions and i'd expect a larger model to be able to handle more vague instructions or have wider breadth of knowledge (if you're trying to make Qwen 3.8 27B do stuff outside the mainstream you need to provide it with some reference material to use) but i never had a problem with it not following instructions.
The harness might matter though but i'm using plain Pi (which is a very barebones harness with a very small system message) without any addons.
[–] mycall 4 days ago
Would something like LMCache help here?
Local models can't get even close to frontier ones; it's just bragging. There's a reason why models like Astra have 2 orders of magnitude more parameters (and run on specialized clusters).
Compared to frontier models, local ones (like Qwen Flash) are much slower, they have much less room for context, and most importantly (people forget to mention) they produce much buggier code. Ah, Qwen also goes into loops sometimes. I've used them on powerful (consumer) hardware.
Having said, no doubt that Qwen-Flash-Next is considerably better than Qwen-3.8-27b - but on the other hand, it's several times bigger. Deepseek Flash also keeps getting better but bigger.