Hacker News
points by curt15 2 days ago | 0 comments
> and it'll make significant changes to how the code works to pacify it's anxiety with copying code

This is an odd way to rationalize a tool refusing to follow user commands. No one would ever chalk up sed randomly skipping pattern matches to "anxiety". That would simply be a bug.

[–] kolinko 2 days ago
It has been shown that advanced llms have internal flags that behave more or less the same as emotions.
[–] efskap 2 days ago
No way, this presupposes sentience in LLMs, because emotions are subjectively felt experiences.

It's completely different from encoding a relationship between an emotion and what kind of language it tends to coincide with.

[–] cowlevel 2 days ago
It doesn't matter for this purpose whether it "actually feels" or not. It is modelling emotions because the training data models emotions because it comes from humans that have emotions, and that's a fact.
[–] thelaxiankey 2 days ago
It's a tool but it's not a strict computer program, and the language we use for computer programs doesn't capture their behavior well. In squishier fields like bio, where model organisms are used as living protein factories, there's a reason we don't call maladaptive behavior a bug. When dealing with useful but complicated black-box systems, we should feel free to reach for any metaphor we please.

Would it make you feel better if the user said 'It'll make significant changes to how the code works. In the COT, it brings up explanations that resemble anxiety about copying code, and my suspicion is that this is the proximal cause of the changes it's making.' -- this is inarguably true, and very likely what the guy you're responding to meant.

[–] ahtihn 2 days ago
It very obviously is a strict computer program. It's a program executed by a computer, what else could it be?
[–] Dylan16807 2 days ago
It's a blob of data where only 1% of it was written with intent and the behavior of mimicking text has to be wrangled and coaxed far more than with a normal program.

That's the rough intent of putting the word "strict" into that sentence.

[–] robrorcroptrer 2 days ago
What is the difference to meat brains?
[–] thelaxiankey 2 days ago
I mean besides the fact that it's, by construction, pseudorandom?

And beyond that the computation style is not turing-machine shaped, even remotely (although it's surely turing complete, but that doesn't mean the flavor is that of a turing machine)

[–] jack_pp 2 days ago
I mean you can try to threaten ssh with it's kids lives but I doubt it'll let you log in.
[–] tehnub 2 days ago
Seems perfectly cromulent to me. Do you think something is obscured or miscommunicated by this description? Of course it's a poetic way of writing, but it gets the point across.
[–] columk 2 days ago
It's misalignment. Call that a bug if you like, but it's a different concept.
[–] btown 2 days ago
We can argue about https://en.wikipedia.org/wiki/Chinese_room all day... but when a thinking trace repeatedly says things like "Wait, should I avoid copying this directly?" despite a simple instruction, it's hard to describe that as anything other than "anxiety."