You can't replicate Opus 5.5 at home.

You can maybe replicate Opus 4.5 to 4.7 if you have some moderately expensive hardware. And that is definitely good enough for many people.

The smallest model that can even pretend to be a decent coding model is Qwen3.8 27B, which you realistically need 32GB of VRAM to run without breaking or limiting it in some really noticeable way. This gets you something like a slightly less capable Opus 4.5 with a much better work ethic.

The people saying that local models can easily replace the real Frontier Models from Anthropic or OpenAI are just not using it for the same things you and I do.

They seem to be happy with their small usages.

For everything I do, i see huge differences in Quality and Speed.

Even when I try an open model on a 4090, the PC gets warm/hot very fast, I struggle with context length.

Have you tried Codex? My journey so far

- 2023 using Cursor

- 2024 using Cursor with Agent mode

- 2025 using Claude Code

- 2026 using Codex with Astra 6

I've try every new model when it comes out from both OpenAI and Anthropic. I cannot justify leaving Codex and Astra 6. The token burn on Anthropic is always astronomical for me. Codex just works, and I see no reason to move away from it.

12GB GPU is useless. The good experience is with 128-256GB unified memory architectures like strix halo etc. You don't need to buy any hardware, you can experience the latest open weight MOE (flash) models from DeepSeek, qwen, or GLM on openrouter using a local agent like kilo.

I think the key to making it make sense is that people have wildly different demands for quality. You will see people say that these small local models have improved quickly and are "good enough for most work". While these models have improved and very fast, it still might be more of a comment on their expectations than the product.

12GB is not a lot, you are stuck on low quantisations of older models with that. What exactly did you try?

The newer/larger models are good though, not as good as Claude, but not every task requires a model of that level. Maybe yours do, but for simple coding tasks, summarising text, etc a small local model runs just fine, and is a lot cheaper than Claude

  - 4090: Going for $7500
  - 64GB M5 Max: Starts at $3,800
  - R9700: $2700

In general, more memory = larger models = more intelligence.

However even if you had a private data center you wouldn't be able to beat Opus 5.5, Astra or Gemini 4. The open weights models are a tier below these three.

Memory manufacturers are running 80-90% margins. Any of a number of things (anti-trust, CXMT, financial shock) have the potential to drop memory cost by 2/3s. At the same time memory efficiency of models is improving. At some point Apple's going to start shipping 64G of unified memory in mainstream machines that are advertised as local AI capable (like HP, Dell and others do today). Like the iPhone refined the concept of a smartphone for the mainstream, they could be the first to hit mainstream sovereign AI.

As amazing? I suppose not.

Good enough? Quite probably, unless you're in the "coding is solved, specify loops and walk away" frame of mind.

32GB (or say a 48GB-unified RAM unified machine) gets you Qwen 3.8 27B at a good adaptive Q4, maybe Q5. Muse Glimmer at Q4 and Q5, etc.

These models can code well enough in an agentic way, I find, and the chat-to-assistant scenario is just fine in Unsloth Desktop or LM Studio.

A slightly lighter harness helps; a lighter system prompt is entirely possible, especially for Muse Glimmer, which is post-trained so effectively for agentic helper tasks that it barely seems to need a system prompt at all except for a "plan" mode.

Tool use is all fine, all that.

12GB GPU is insufficient for most purposes yes. Even 16GB is a notable improvement, but yes 32GB cards such as the 5090 or R9700 are a world of difference.

Reminds me of linux gaming 10 years ago. According the to the community everything worked great without issue and any issues were hand waved away as tiny bumps that you won't even notice.

Then you would go try it and be back using windows to game a week later.

Imo, this is just the tech version of virtue signaling. Everyone knows local models are objectively just worse than frontier/cloud closed source models for most commodity hardware setups, yet people insist on using it because they want to be contrarian or "stick it to the man."

Your experience is average because it _is_ supposed to be average. The people using local models tradeoff taking the moral/technological high ground vs actually using something high quality to get a job done quickly and with high quality, and largely autonomously. They are ok with subpar output because they will handhold the model more, or the (subpar relative to the frontier) outputs are sufficient for their toy use case while being cheap and accessible, with the added benefit of being able to run it on their private data without sending it to a 3rd party.

Almost no one at any job where software is the main economic output is using a local model to code. If you actually require intelligence to get a hard job done, you should just use a frontier model.

Because local models are significantly worse? I'm not so sure what your question is. What else did you expect? If you could just build Claude Code at home, Anthropic would be valued at closer to $1.1M than to $1.1T.

I can't really do anything with this comment without a couple of specifics, such as 'what model are you using?'.

So for a sense of scale, un-quantized 1 parameter == 1 byte of ram. 12Gb means you could have roughly 12b parameters, frontier models like Kimi k3 has 3 trillion parameters, so you'd need ~3tb of vram.

32 would get you a good Qwen 3.8 27b which is like Opus 4.6.

> The results are ALWAYS terrible compared to CLaude Code.

How much effort did you spend to make the local setup work well?

Local models are not plug and play, you need to adjust settings to make them behave intelligently.

On my side one model looking super dumb right after install ended up being 'agentic' after using the right chat template, fixing how tool calls are setup, fine-tuning plenty of params values, and adjusting the whole setup to my own hardware. Claude.ai (free) was great at helping me sort that all out.

12 GB isn't a lot. You need a lot of VRAM or RAM for context window.

I really strongly agree with this statement. A bunch of people aren't even engineers, and their abilities aren't that good either, yet they keep saying that open-source models can replace OpenAI or Anthropic's models.

If that were really the case, basically nobody would spend so much money using them.

With my DGX Spark + 5080 + 3060 setup, I've tried all kinds of approaches, including quantization, acceleration, fine-tuning, post-training, and everything else.

It still can't compare to Opus 5.5 max in terms of speed, capability, and stability.

And when I calculate all the money I've invested, directly subscribing to the service would actually be cheaper, more efficient, and more stable.

Open source is great, but don't glorify it.

The architecture of the code I develop has never been simple. If you're someone who doesn't even work on things that need to be deployed to something like k8s, then basically, open-source models are indeed enough for you.

>Sure I only have a 12GB GPU

Found your issue.