Hacker News

points by TechTechTech 4 days ago | 0 comments

We configured 6 machines each having 2x5090 for local Qwen3.8 in the past months. Prices are very high.

The performance of local AI for coding, given correct setup & harness, is very good and on par for 95% of use cases compared to closed models.

Local usage for end-users is unmetered and token speeds are very fast and keep improving every few weeks due to optimizations being done in the community.

Data remains local which makes EU compliance checkboxes way easier since data never touches anything related to the CLOUD act.

[–] user43928 4 days ago

> The performance of local AI for coding, given correct setup & harness, is very good and on par for 95% of use cases compared to closed models

While I don't know your specific use cases, when interpreted generally, this seems trivially disproven by putting eg. websites made by Opus 5.5 and the local model side by side.

[–] thombles 4 days ago

I have merely a laptop 5090 (24 GB) and I can run Qwen3.8 Flash IQ3_XXS at 110-140 tok/s on Strata, 128K context. If Artificial Analysis' benchmarks mean anything, it sits somewhere around GPT 5.6 Terra/Sol. I use both that and Codex/Sol 6.1 most days of the week, for different projects. I don't feel like I'm particularly compromised when working locally - I very rarely think "ah man I wish I could just point Codex at this". The biggest problem is speed - it needs more reasoning time and I can only work single-threaded.

[–] lp92 2 days ago

Those benchmarks are easily gamed in my opinion. I take them woth a huge grain of salt.

I am currently building an integrated CAD + Aero + CAM software package for rapid UAV design iteration, and the open models are all not very good at it. They make a ton of mistakes and I have to end up redoing a lot. And that's using very detailed specs and TDDs provided as context to the models. Opus/Astra/Argon/Gemini Flash 3.8/Sol does a significantly better job in my experience.

[–] yread 3 days ago

Have you tried basalt? I understand that it allows multiple concurrent requests

[–] iLoveOncall 4 days ago

Imagine thinking that the look of a website is the right metric to judge the performance of AI models.

[–] salviati 4 days ago

If your job is to produce websites that look good it's an excellent way to judge performance of AI models _for your workflow_.

What's the problem?

[–] totetsu 3 days ago

Most of the look of the thing is about the built in tools and skills and harness settings, not the quality of the code. those things can be replicated locally as well.

[–] literalAardvark 4 days ago

The problem is you can change the prompt a little and get correct results on a local AI too.

Frontend has been solved for years.

[–] user43928 4 days ago

I don't think changing the prompt alone is going to do it, but iteratively providing feedback can get you there.

With GPT-5.4 or GPT-5.5, or of course also by writing the code by hand if we want to go all the way, it was well possible to build nice frontends to our liking.

I still don't want to go back when I can do it better and faster with Opus 5.5.

[–] rolisz 4 days ago

It is one metric to judge them. And if you do frontend stuff, it's probably a fairly relevant one.

[–] Gareth321 4 days ago

While I don't doubt your efficacy, the cost of those machines today, especially considering the high cost of electricity in Europe, would be uneconomical versus buying subscriptions. The reason to do this setup is data sovereignty, and there are plenty of cases where this is required.

[–] lisplist 3 days ago

Why 2 5090s instead of just 1 6000? That's 64GB VRAM vs 96GB VRAM. Perfomance is theoretically worse on the 6000 vs 2x 5090s, but given the overhead of tensor parallelism and how slow PCIE is, I imagine it's pretty close for less than half the power draw and heat.

[–] pritambarhate 4 days ago

>> given correct setup & harness

I am geninuely interested what stack you are using here. Could you please elaborate?

[–] TechTechTech 4 days ago

We are continuously improving the workflow and things improve almost on a weekly basis. Current setup is custom Gitlab project structure + OpenCode + Mattpocock/skills followed by a well defined align/plan/build/review/learn engineering process.

It feels like things changed in the past few months. We are now truly able to do what we need to do without Claude etc, where this was not feasible earlier this year.

[–] latentsea 3 days ago

I use the Qwen models a lot in my private life and they're not quite on par with the SOTA models, but they've definitely gotten to the point where they're usable. Particularly if you invest a lot in harness engineering, they become very usable. It'll be even better when Qwen4 comes out, and less harness engineering will be needed for the same results, and an equivalent amount of harness engineering tuned for the updated models weaknesses will produce results much closer to current SOTA. I was also pretty stunned to see Unsloth's IQ1_M quants of Deepseek v4.1 Flash 1017 held up too. Even that has become runnable locally. It's truly a new era.

https://unsloth.ai/docs/models/deepseek-v4.1-flash

[–] gaws 3 days ago

> We configured 6 machines each having 2x5090 for local Qwen3.8 in the past months. Prices are very high.

Who is "we"?