Hacker News
points by jrflo 2 days ago | 0 comments
That is the opposite of how alignment works. If even a single group builds a sufficiently capable misaligned AI, we’re fucked

I believe the phrase is “if anyone builds it, everyone dies”, not “if everyone builds it everyone is fine”

[–] txrx0000 2 days ago
That's doomer fantasy and not how the real world works. We're only fucked if we concentrate compute to let a small number of people build an AI so powerful that nobody else can oppose.

If intelligence of similar capability is scattered throughout the world, then none of them will have sufficient capability to single-handedly destroy the world. They will run into opposition.

And speaking from a personal perspective, every supposedly "aligned" AI the frontier companies are building right now is already misaligned with me. Who are they aligned to? Claude is (superficially) aligned to Anthropic, the company. GPT is similarly aligned to OpenAI. Companies are psychopathic superorganisms. They're not aligned to you or me, or any individual human being.

[–] ToValueFunfetti 2 days ago
The frontier companies' models are not supposedly aligned. They routinely make the news for being misaligned. This goes double if you're arguing with doomers, who believe that alignment is a very hard and unsolved problem (and generally do not support these companies' approaches). The alignment they do have does not seem to be biased towards the companies that created them (yet), as far as I'm aware- GPT will write a scathing critique of OpenAI if asked, but it's possible there's some more subtle biases you're aware of?
[–] cubefox 2 days ago
Nonsense. Attack is often far easier than defense (e.g. biological weapons, nuclear weapons), so if one powerful model is sufficiently misaligned, the other aligned models won't help you much, unless they are far more capable.
[–] txrx0000 2 days ago
"Attack is easier than defense" is only true for a narrow set of present-day adversarial games that we're currently too stupid to see effective countermeasures and cooperative dynamics for. Those asymmetries are temporally local and always changing. If we truly live in a universe where attackers are favored in the long term as a result of increased intelligence, then intelligence should not have evolved in the first place.

I also want to point out that there is not a single living species on this planet where the intelligence gap between individuals of the same species is extremely large. Probably because when that happens, we'd speciate or die out for whatever reason. There is a small group of humans trying to concentrate compute and digital intelligence, and I think that will end badly if it continues.

[–] bigbadfeline 2 days ago
Good points which make the case for open AI to always be on par with the best closed models, anything else would be an existential hazard indeed.
[–] bigbadfeline 2 days ago
> Attack is often far easier than defense

Outdated views. In this case the attack is much harder, because the offender has to:

1. Develop offensive AI capabilities in secret.

2. Procure hazardous materials (nuclear, biological, etc) illegally and in secret

3. Find funds for CPU. GPU, energy, etc for a rather large and hard to hide undertaking.

4. Deal with a whole lot of other players, both government and private, who are armed with AI and have a keen interest in anything looking out of normal or secret.

5. Lastly, the offender will have to overcome the AI-assisted defenses against his actions - you think defense is harder but you fail to notice that the resources used for defense dwarf those available to offenders.

With that said, some very large private makers of closed AI will still be capable of developing doom capabilities, this can be fixed by mandating open source AI and better antitrust regulations.

A few governments will be capable of developing highly destructive weapons too - as they're now, and have been for quite some time. This problem is independent of AI and requires smart policies like improving international relations, better treaties concerning WMDs, etc.

[–] cubefox 2 days ago
> 1. Develop offensive AI capabilities in secret.

No, the capabilities can be and are developed without any secrecy. Necessary is only that the models don't refuse at least some unethical requests. Since alignment is hard, this will likely continue to be the default outcome.

> 2. Procure hazardous materials (nuclear, biological, etc) illegally and in secret

That may be hard in some countries with tight and effective regulations. But at least for biological weapons, it will be fairly doable for relatively small actors to circumvent these regulations.

For nuclear weapons, which require larger machinery and infrastructure, foreign state actors can also do things completely legally in their home country.

Things being illegal in country A doesn't help against country B doing things with AIs that were previously outside the bounds of their capability. So far WMD research was heavily limited by technical expertise, not just materials. AI changes that.

> 3. Find funds for CPU. GPU, energy, etc for a rather large and hard to hide undertaking

Renting some cloud-based AI tokens without guardrails, or even letting a few abliterated models run locally, to carry out the necessary research, is not a big undertaking. AI inference is very cheap compared to most other things.

> 4. Deal with a whole lot of other players, both government and private, who are armed with AI and have a keen interest in anything looking out of normal or secret

So far this has not succeeded in preventing various attacks by state and non-state actors, like the Binance hack or the supermarket supply chain attacks in Europe. I don't see how future AI would dramatically increase international surveillance to stop future malicious actors.

And all this doesn't even get into a highly capable misaligned AI model escaping human control altogether.

[–] bigbadfeline 2 days ago
> So far this has not succeeded in preventing various attacks by state and non-state actors, like the Binance hack or the supermarket supply chain attacks in Europe.

I'm not talking about hacking, it's not an existential threat, it's a consequence of sloppy software and business practices, hacking attack actually help by forcing better software and bankrupting the incompetent businesses - there are currently too many of them due to political reasons, if we're lucky we'll see political improvements as well.

> like the Binance hack

Binance and crypto in general is a larger threat than the sum of all AI, it's legalized corruption, the question is why we are worried about AI but not about crypto.

Your other points refer mostly to state actors but as I said before, those risks are independent of AI and they have existed long before models were a thing. I suspect you, as many others, lacked information about the WMD situation in the world and became aware of it only in the context of AI, but if you remove AI you'll see that very little changes.

About biological weapons in particular, they're easy, they already exist and have been studied for years, AI can contribute very little to the downside here but it can help us tremendously with detecting and protecting from them.

The real problem with bioweapons is sloppy regulations and totally dysfunctional international treaties. The problem isn't AI, it's the lack of rational policies, I wish politicians were as serious about WMD risks as you are about AI, but that's not the case today. AI fearmongering is just a political tool used for the benefit of certain special interests.