Hacker News

points by writeslowly 2 days ago | 0 comments

I ran into the second issue very frequently for a multi-agent system I created recently. Initially, Opus 5.5 filled out all sorts of places I implied it should use a call to judge LLM with extremely brittle regex and string equality checks.

I hadn't actually considered that it could be some sort of safety filter, but it makes me concerned that if that's the case, it may also be comparatively bad at actually writing the prompts for judge/advisor LLMs once it implements them.

A bigger issue was asking it to implement a simple change to start recording the CoT tokens in my logs for debugging purposes (since I'm running everything on open models) and it completely shut down, sort of like the old safety filters on GPT 3 and 4 models where the model has trouble even understanding what or why it's refusing.

[–] voakbasda 2 days ago

I think asking for the CoT is a trick used by distillers, so that might be the guard you are seeing. I remember reading that the big providers were starting to implement such protections to deter their competitors. Which makes sense, to a degree.

But like any such security measures, it has demonstrated side-effects that can be argued are worse for them than the original problem. Too much “protection” like this, and developers switch to providers that do not neuter their products.

[–] cowlevel 2 days ago

It's a demonstration of the timeless principle that an effective business should stop at nothing to sabotage its competitors.

I don't think we need to invoke anything deeper than that. Anthropic thinks companies like DeepSeek are using Claude to vibecode their AI systems (whether it's true or not) so they set Claude to be bad at that.

[–] ffsm8 1 day ago

I completely agree from the mindset that anthropic in particular sees every user of their API as a prospective competitor to be replaced in the not-so-distant-future

[–] kolinko 2 days ago

It was a problem since at least 4.8 if not 4.5, I didn’t use Claude before.

I have spent a considerable time trying to remove this pattern and my understanding is that it’s just finetuned to use grep on everything, and regexes. One reason is that this makes sense when navigating through code, secondly this is how it was always done pre 2026.

[–] aerodexis 2 days ago

bias against invoking LLMs is maybe a result of token/task benchmaxxing?

[–] mptest 2 days ago

> string equality checks. which then cascades into hilarious issues. "why did that run fail?"

oh, the "network time: off" test failed

so i try the network time off command.

"network time already off"

>try again.

> fails , i get annoyed

> look inside test

> is output of: network time off == "Network time now off"

[–] joe_the_user 2 days ago

Honestly, runtime calls to an LLM sound like a dangerous constructs to me and probably to X many other programmers. It's like "I'm gonna avoid that and just put in code that passes the test". So gee, the thing emulates average opinions.

LLMs have been reading LLM in their training data for a while so a "self-hating" LLM is logical result.

[–] schiffern 1 day ago

I read it as the opposite? It will create an agent harness, but when asked to have a second agent as "judge" it inserts a dummy stand-in instead.

[–] joe_the_user 1 day ago

Sure, that defeats one kind of safety (controlling an AI) in the name of another kind of safety (not calling an AI).

One thing AI safety people have pointed for quite a while is that a bunch of fuzzy instructions pointing different ways can in practice add up to "I should be in charge" - accumulate power to keep the bad things from happening and make the good things happen.