Hacker News

points by root_axis 1 day ago | 0 comments

LLMs aren't magic, there's nothing they can do to persuade people to do weapons testing, it's an absurd scenario.

It feels like I don’t understand your definition of “persuade”, because the notion that people cannot be persuaded of things via reports and flashy animations is to me, absurd.

[–] Paracompact 23 hours ago

For what value of N do you think there exists a string of N characters I could have responded to your comment with that would have infected you with a stably self-replicating and capability-augmenting memetic virus with human extinction as its latent goal?

If not you, in this situation, with N characters delivered via a HackerNews comment, then who, in what situation, with what media and how much of it?

[–] dTal 14 hours ago

I think that is a fascinating question and I would like to advance the provocative notion that N is much smaller than you think, maybe <256. Consider that 128 bits is a cryptographically secure key, and English text has an entropy of around half a bit to a bit per character. 256 characters is sufficient to encode sentences that have never been seen or imagined before, totally unguessable sentences. Who knows what's in that probability space?

[–] amelius 23 hours ago

Never heard of blackmail?

And by the way, LLMs do not have to convince 1 specific person, 1 random person is sufficient.

[–] Paracompact 23 hours ago

You're not spelling out a scenario we can meaningfully dissect.

Probably you will agree, the leverage of a blackmail attack is not bottlenecked by the persuasiveness of the accompanying threat letter. It is bottlenecked by the existence of damning facts they would rather keep hidden, and the ability to discover those damning facts. I'll grant that automated discoverability of those damning facts is augmented by superintelligence; they can efficiently sweep the open internet in the blink of an eye, and sure, I'll concede, hack into most accounts at will. But that is still a small part of the process by which blackmail material is actually obtained in the real world.

> LLMs do not have to convince 1 specific person, 1 random person is sufficient

No, not one random person either, but many people in positions of great influence. There already exists blackmail and other corrupting influences on such persons in voluminous quantities. If the US targets the entire Russian admin, or vice versa, with AI-enabled blackmail discovery techniques, it will not be an augmentation of their preexisting operations to discover blackmail on each other in excess of the augmentation provided to software engineering generally by AI.

[–] amelius 22 hours ago

Your argument does not really read like an argument against my argument.

Not sure what to say.

[–] Paracompact 21 hours ago

You did not provide an argument. You avoided answering my direct questions, provided a sarcastic 4-word reply, and so I extrapolated a reasonable rebuttal in its place, and replied to that rebuttal.

I should just have said: Yes, I have heard of blackmail. I remain unconvinced AI will cause human extinction.

[–] solyin 23 hours ago

Ah yes, the "just do what a smarter person than you would do in this situation" argument. If I had that power, would I be wasting it shitposting here? We know humans have infectible little minds. At their worst, humans are just stoichastic parrots. Throw something obviously AI generated at a stoichastic human-parrot and you'll inevitably get the answer "Why should I read something if you didn't bother to write it?" UBI as a memetic virus comes close tho.

Personally, I've been duped by a good ad into buying something I totally didn't want or need so I admit the possibility that the exact right tailored argument to me at the right time and place would get me to do something I'd regret later. Fortunately I'm not in charge of anything serious/am smart enough to not install software for an interview on my work laptop while working at Binance.

I don't know that there exists some N for which that is true, because people are different, so what would convince me to do something isn't the same aa needed to convince someone else to do that something.

[–] pigpop 18 hours ago

Do we have reason to believe they would be successful at other extreme feats of persuasion such as persuading a financial institution to transfer all of its holdings to an account controlled by the AI or persuading the Pope to renounce God?

[–] amelius 23 hours ago

If Putin can persuade/bribe/blackmail people to fly drones around European airports, then why wouldn't an LLM be able to do similar things?

[–] Paracompact 23 hours ago

Putin's persuasiveness does not derive from his intelligence.

[–] machomaster 10 hours ago

Except that Putin doesn't do what you listed, he is not smart enough to do it. And this is an unnecessary complication. He just pays people and they do the job. The more money, the more willing people are to commit crimes.

[–] vedanshchauhan9 23 hours ago

The human tendency to want to finish things in a fraction of a second will definitely take us there.