P(doom) is no technical term from a formal paper. It emerged in online AI safety discussions as shorthand for a probability estimate that most researchers before 2020 would not have attached a number to at all. The willingness to quantify it (to treat catastrophic AI risk as a probability to be estimated rather than a possibility to be vaguely gestured at) is one of the meaningful shifts in how people working closest to AI development think about the problem.
Estimates range from a few percent to over 50%, depending on the researcher and how they evaluate the difficulty of alignment and the speed of governance. Here is what the range tells us, and why even low probabilities require serious attention.
The "doom" in P(doom) is deliberately unspecified. Different researchers use it to mean human extinction, permanent loss of human autonomy, permanent concentration of power in a single actor, or some other civilizational catastrophe. The imprecision is acknowledged; the point of the term is to force explicit quantified reasoning about the risk rather than rely on intuitions about whether it is "real" or "speculative."
What the estimates look like
The range across these researchers is striking. Hinton and Yudkowsky are separated by tens of percentage points. Carlsmith's formal decomposition produced a wide range rather than a point estimate, reflecting genuine uncertainty at each intermediate step. What the estimates share is that they are non-trivial, none of the safety-focused researchers who have engaged seriously with the problem and put a number on it have landed below about 5%.
Set your own view on each of the cruxes below and watch them multiply into a single estimate.
Why the wide range exists
P(doom) as a number is the product of several uncertain intermediate questions multiplied together. How rapidly will AI capabilities advance? How difficult is the alignment problem at capability levels beyond those we have tested? How likely is it that misaligned AI of a given capability level would cause catastrophic rather than contained harm? How likely is governance to respond effectively before the critical capability threshold is reached?
Different researchers hold different distributions over these questions, and the estimates compound. A researcher who thinks the alignment problem is tractable, capabilities will advance relatively slowly, and governance will respond in time will produce a much lower overall estimate than one who believes the opposite on each question. The wide range reflects genuine uncertainty about each factor, not a disagreement that will be resolved by examining the same evidence more carefully.
The expected value argument
A common response to non-trivial P(doom) estimates is: "Even 10% seems too low to restructure society around." The response relies on a failure of expected value reasoning that would not be accepted in other risk contexts.
The annual probability of a major pandemic capable of killing millions is estimated below 1% in any given year, yet pandemic preparedness receives substantial global investment and policy attention. The lifetime probability of dying in a car accident in the US is roughly 1%, and road safety regulation is considered entirely normal. The argument that 10% (or even 5%) probability of civilizational catastrophe does not warrant serious precautionary investment is not a coherent application of how we handle risk in other domains.
Expected value is probability times magnitude. For risks that are irreversible and civilizational in scale (where there is no recovery from the bad outcome, and the bad outcome forecloses all future value) the expected value of precaution remains very large even at probabilities well below 10%. A 1% chance of losing everything is a problem whose expected cost is 1% of everything, which, across the scale of what the long-run future contains, is an enormous number, not a 1% problem.
What would lower P(doom)
Researchers who express high P(doom) estimates are diagnosing where the most critical work needs to happen, not resigned to bad outcomes. The factors that would lower P(doom) across most researchers' estimates are well understood: demonstrated progress on interpretability tools capable of verifying alignment in frontier systems; scalable oversight approaches that hold at capability levels beyond human experts; binding international governance frameworks with real enforcement; and capability development that proceeds more slowly than alignment research so that safety is established before dangerous capability levels are reached.
This is why the Foundation focuses on governance frameworks and alignment research as the primary levers, rather than on either dismissing the risk or treating a particular P(doom) estimate as the authoritative answer. The uncertainty in P(doom) is not a reason to wait for more information, the bad outcome arrives at specific capability thresholds, not on a timeline that allows indefinite postponement of precaution.