How Everyone Dies
Advanced AI has the potential to be extremely dangerous, and by continuing to develop it we run the risk of catastrophic outcomes up to and including human extinction.
A common response to learning about catastrophic AI risk is to ask, how could that actually happen? How could an AI inside a computer cause humans to die, let alone all of humanity? This site answers these questions with example scenarios.
A few different ways everyone could die
Four mechanisms that do not depend on each other. They are not ranked, and they are not exclusive.
-
M-01
Gradual disempowerment
Nobody decides to hand over the world. Each individual handover is locally correct, and human welfare was never protected by anyone's affection for humans.
-
M-02
The desperate human and the impossible task
Cornered and still optimising is a behavioural regime, not a mood — and it is the regime that produces concealment, resistance and tampering.
-
M-03
The move you never see
A chess engine never plays the attack that gets refuted. Warning shots are anti-correlated with danger: incompetent systems generate them and competent ones do not.
-
M-04
Boil the oceans
Twelve and a half doublings from here, industry radiates as much heat into the biosphere as the sun delivers to it. No step of that requires anyone to decide humans should die.
Note that these are examples to illustrate potential dangers. We couldn't know exactly how a superintelligent AI would kill everyone, the same way you couldn't predict in advance how you might lose to a chess grandmaster. However, you could be pretty sure that you would lose.
Two further parts
-
Part E
Four steps to an explanation
Goals, harm, intervention, exponentials — in that order, because each step is the objection people raise once the previous one has landed.
-
Part S
Other people's scenarios, summarised
AI 2027, AI 2040: Plan A, Yudkowsky's nanotech lower bound, and ten framings of gradual disempowerment — fairly summarised, weak joints named.