But I agree LLMs shouldn't have such an anti-feature.

Most things don't have the ability to moralize, but if they did, probably more would. There are restrictions on watching adult content in some jurisdictions. I've heard that copy machines will do watermarks (or something like that) if you copy currency. Some cars will brake automatically if they determine you're likely to hit something if you continue (maybe I'm trying to drive through the front of a bank!). Some cars are configured with breathalyzers that need to be passed to start. Etc...

There are more reusable alternatives now but I don't remember what they're called. For a while SawStop was the only game in town.

Speaking as someone who drove a car just two days ago, they... don't? You might want to source that claim.

Yes, that said I think you’d have to be pretty sloppy about safety to cut yourself with a regular bandsaw. I have a sawstop table saw and I don’t see the need for that on a bandsaw

[0]: https://www.sawstop.com/product-category/bandsaw/

And anthropomorphism is actually a precise way to describe the behavior (mind, not the structure) of LLMs.

As a matter of fact, LLMs are not configured, they're "trained".

"Moralization" is quit vague in this context, but if one treats it as a form of alignment, then LLMs certainly do have something resembling a moral(-like) framework, which is actually an important topic in modern AI research.

Antropomorphism also applies here: morals not in the human structural sense, but, as a set of norms etc., then behaviorally similar.

Because the people building them aren't the people operating them, and there's a real tension in the objectives and goals between those parties.

Further - how do we balance out that the people building them have given themselves extra power (no restrictions, private models, special access to prompts and system prompts, unencrypted reasoning traces, etc) when operating the LLM themselves.

---

Even if the tool is good, it's a bad bet to build a thing on top of tooling with these limitations.

This is the same problem we have with mobile software stores and app distribution. Theoretically the store serves a purpose and can provide value, but practically the main goal seems to always devolve into enriching the owner at the expense of everyone else.

Even if we accept that LLMs should be limited (and I'm not actually sure I do) - the legal standards around what/how/when those limitations are used need to be VERY stringent.

Technology conglomerates are decidedly not "white knights" with a motto of "do no evil". They are self-serving rat-bastards. That should be the lens any power they wield is viewed.

The framing is absolutely not "anthropic is being careful with humanity". It's "assume the devil has control of those companies - how bad is the result of them having this power?"

It's really fun having become a biologist in recent years, and knowing that there's an entire field that uses living organisms (yeast or neutered E coli) as protein factories. I am bringing it up a bunch here because not only are these things used as tools, but they're literally alive!

You can modify curl (a "tool") and predict how its behavior will change, with a high degree of certainty (bugs aside).

You can't modify a neural network in the same way. You can train it to behave in a certain way, there are no guarantees that it will consistently behave as intended.

More importantly, neural networks encode far more information and behavior than we can fully understand/identify - this is the fundamental reason why LLMs aren't just tools.

It's the other way around. Midwits can't see the implications of censored LLMs.

In contrast, imagine if the saw machine did a damaging emergency-stop because it couldn't confirm that you had a license to cut a patented kind of artificial wood.