Hacker News

points by swid 22 hours ago | 0 comments

It feels like I don’t understand your definition of “persuade”, because the notion that people cannot be persuaded of things via reports and flashy animations is to me, absurd.

[–] Paracompact 22 hours ago

For what value of N do you think there exists a string of N characters I could have responded to your comment with that would have infected you with a stably self-replicating and capability-augmenting memetic virus with human extinction as its latent goal?

If not you, in this situation, with N characters delivered via a HackerNews comment, then who, in what situation, with what media and how much of it?

[–] dTal 12 hours ago

I think that is a fascinating question and I would like to advance the provocative notion that N is much smaller than you think, maybe <256. Consider that 128 bits is a cryptographically secure key, and English text has an entropy of around half a bit to a bit per character. 256 characters is sufficient to encode sentences that have never been seen or imagined before, totally unguessable sentences. Who knows what's in that probability space?

[–] amelius 22 hours ago

Never heard of blackmail?

And by the way, LLMs do not have to convince 1 specific person, 1 random person is sufficient.

[–] Paracompact 21 hours ago

You're not spelling out a scenario we can meaningfully dissect.

Probably you will agree, the leverage of a blackmail attack is not bottlenecked by the persuasiveness of the accompanying threat letter. It is bottlenecked by the existence of damning facts they would rather keep hidden, and the ability to discover those damning facts. I'll grant that automated discoverability of those damning facts is augmented by superintelligence; they can efficiently sweep the open internet in the blink of an eye, and sure, I'll concede, hack into most accounts at will. But that is still a small part of the process by which blackmail material is actually obtained in the real world.

> LLMs do not have to convince 1 specific person, 1 random person is sufficient

No, not one random person either, but many people in positions of great influence. There already exists blackmail and other corrupting influences on such persons in voluminous quantities. If the US targets the entire Russian admin, or vice versa, with AI-enabled blackmail discovery techniques, it will not be an augmentation of their preexisting operations to discover blackmail on each other in excess of the augmentation provided to software engineering generally by AI.

[–] amelius 20 hours ago

Your argument does not really read like an argument against my argument.

Not sure what to say.

[–] Paracompact 20 hours ago

You did not provide an argument. You avoided answering my direct questions, provided a sarcastic 4-word reply, and so I extrapolated a reasonable rebuttal in its place, and replied to that rebuttal.

I should just have said: Yes, I have heard of blackmail. I remain unconvinced AI will cause human extinction.

[–] solyin 21 hours ago

Ah yes, the "just do what a smarter person than you would do in this situation" argument. If I had that power, would I be wasting it shitposting here? We know humans have infectible little minds. At their worst, humans are just stoichastic parrots. Throw something obviously AI generated at a stoichastic human-parrot and you'll inevitably get the answer "Why should I read something if you didn't bother to write it?" UBI as a memetic virus comes close tho.

Personally, I've been duped by a good ad into buying something I totally didn't want or need so I admit the possibility that the exact right tailored argument to me at the right time and place would get me to do something I'd regret later. Fortunately I'm not in charge of anything serious/am smart enough to not install software for an interview on my work laptop while working at Binance.

I don't know that there exists some N for which that is true, because people are different, so what would convince me to do something isn't the same aa needed to convince someone else to do that something.

[–] pigpop 17 hours ago

Do we have reason to believe they would be successful at other extreme feats of persuasion such as persuading a financial institution to transfer all of its holdings to an account controlled by the AI or persuading the Pope to renounce God?