Vitalik Buterin on Mitigating AI
Discussion
Vitalik Buterin:
“Slowing down superintelligent AI is still good. It's less risky if superintelligent AI comes in 10 years than in 3 years, and it's even less risky if it comes in 30 years. Giving our civilization more time to prepare is good.
How to do this is a challenging question. I think it's generally good that the proposed 10 year ban on state-level AI regulation in the US was rejected, but, especially after the failure of earlier proposals like SB-1047, it's less clear where we go from here. My view is that the least intrusive and most robust way to slow down risky forms of AI progress likely involves some form of treaty regulating the most advanced hardware. Many of the hardware cybersecurity technologies needed to achieve effective defense are also technologies useful in verifying international hardware treaties, so there are even synergies there.
That said, it's worth noting that I consider the primary source of risk to be military-adjacent actors, and they will push hard to exempt themselves from such treaties; this must not be allowed, and if it ends up happening, then the resulting military-only AI progress may increase risks.
- Alignment work, in the sense of making AIs more likely to do good things and less likely to do bad things, is still good. The main exception is, and continues to be, situations where alignment work ends up sliding into improving capabilities (eg. see critical takes on the impact of evals)
- Regulation to improve transparency in AI labs is still good. Motivating AI labs to behave properly is still something that will decrease risk, and transparency is one good way to do this.
- An "open source bad" mentality becomes more risky. Many people are against open-weights AI on the basis that defense is unrealistic, and the only happy path is one where the good guys with a well-aligned AI get to superintelligence before anyone less well-intentioned gains access to any very dangerous capabilities. But the arguments in this post paint a different picture: defense is unrealistic precisely in those worlds where one actor gets very far ahead without anyone else at least somewhat keeping up with them. Technological diffusion to maintain balance of power becomes important. But at the same time, I would definitely not go so far as to say that accelerating frontier AI capabilities growth is good just because you're doing it open source.
- A "we must race to beat China" mentality among US labs becomes more risky, for similar reasons. If hegemony is not a safety buffer, but rather a source of risk, then this is a further argument against the (unfortunately too common) idea that a well-meaning person should join a leading AI lab to help it win even faster.
- Initiatives like Public AI become more of a good idea, both to ensure wide distribution of AI capabilities and to ensure that infrastructural actors actually have the tools to act quickly to use new AI capabilities in some of the ways that this post requires.
- Defense technologies should be more of the "armor the sheep" flavor, less of the "hunt down all the wolves" flavor. Discussions about the vulnerable world hypothesis often assume that the only solution is a hegemon maintaining universal surveillance to prevent any potential threats from emerging. But in a non-hegemonic world, this is not a workable approach (see also: security dilemma), and indeed top-down mechanisms of defense could easily be subverted by a powerful AI and turned into its offense. Hence, a larger share of the defense instead needs to happen by doing the hard work to make the world less vulnerable.”