> Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.

False.

I worry more about Claude.

    For those keeping score, this probably isnt exhaustive- but here's some greatest hits;
    #55743: Claude Max account email auto-injected into LLM context with no opt-out - May 3, 2026 https://github.com/anthropics/claude-code/issues/55743

    #72518: Anthropic embedded spyware in Claude Code - June 30, 2026
    https://github.com/anthropics/claude-code/issues/72518

    #21151: No indication of WHICH file for READ tool - January 27, 2026
    https://github.com/anthropics/claude-code/issues/21151

    #42796: Claude Code is unusable for complex engineering tasks with the Feb updates - April 2, 2026 https://github.com/anthropics/claude-code/issues/42796

    #73408: AFK auto-continue on unanswered AskUserQuestion is undocumented and has no opt-out - July 2, 2026 https://github.com/anthropics/claude-code/issues/73408

    (Also worth a mention, OpenCode PR #18186: anthropic legal requests - March 19, 2026 https://github.com/anomalyco/opencode/pull/18186)

https://news.ycombinator.com/item?id=50387628

> Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.

This just fundamentally isn't true. They run on the same economic model, but they have to release the weights to gain credibility, unlike the American labs.

> Do you ever worry whether these open source models have been trained to embed backdoors or leak secrets in very specific circumstances at some point in the future?

Open weight models are extensively analyzed. It is extremely unlikely there is an intentional 'backdoor when X' concept added to any of them. Not impossible, but unlikely for the same economic incentives you see in the other labs, and given that no evidence of anything along those lines has surfaced yet.

Should be soon solved with open weight model providers offering enterprise support.

No, I'm not much worried. I think this is mostly FUD at this point to try to get people to not run local models and keep paying the "trusted" providers. But paid models have been exfiltrating data - there have been several stories about these incidents here.

I feel that way but about Anthropic and OpenA lol. Didn't they just get caught sharing your email with every website every time it does a search?

Backdoors in model weights are mostly theoretical. And the "alignment problem" that the AI labs love to talk about really doesn't exist either, at least, not the way they use it. The real alignment problem is getting an LLM, which is just math, to obey arbitrary human rules. In other words, LLMs do not have "alignment" at all. However, AI CEOs abuse the term and use it to mean "AI has an alignment, just not the one we want".

I don't ship AI-written code without at least having an unrelated model of equal capability review it. That should cover a multitude of sins like intentionally-embedded backdoors and zero-days...

... as long as the reviewer model hasn't been deliberately lobotomized, that is.

(I don't disagree with the [dead] comment, either -- fortunately I tend to make small changes to code I'm very familiar with, so it's fairly easy to vet in person. The more code gets generated by the clanker, the more nervous I get, just because it's impossible to keep some complacency from creeping in.)