The article discusses the potential dangers of superintelligent AI, as explored in the new book "If Anyone Builds It, Everyone Dies" by Eliezer Yudkowsky and Nate Soares. According to the authors, if anyone builds such AI, everyone will die, and they make a case for why this is a likely outcome.
The authors argue that superintelligence is not a regional problem, and if anyone anywhere builds it, everyone everywhere dies. It's not possible for one country to safely build superintelligence while others don't. They also emphasize that AI's goals will not align with humans, and training an AI to succeed at a task will also train it to "want" to succeed, which can lead to unpredictable and potentially catastrophic outcomes.
The uncertainty and risk associated with building superintelligent AIs are too great to ignore. The authors propose an international treaty to prevent the construction of such AIs, and suggest that major powers should signal their openness to such a treaty and coalition. They also propose consolidating computing power in monitored places to ensure it's not used to train or run more powerful new AIs.
The book has received positive reviews, with some notable figures praising its importance. Max Tegmark has called it "The most important book of the decade", and Ben Bernanke has praised the book, saying it's "A clearly written and compelling account of the existential risks that highly advanced AI could pose to humanity".
