Before a commercial airliner type is certified to carry passengers, regulators demand a structured safety argument: claims, reasoning, and evidence thick enough to reject. In the United Kingdom, nuclear site licensing still runs through a safety case the Office for Nuclear Regulation must accept before operations. Many medical devices follow the same burden. The operator does not get to run first and explain later. Permission follows a case that survived hostile review.
An AI safety case borrows that shape. It is a structured, evidence-backed argument that a particular system is safe enough to train or deploy in a particular setting. Applied to frontier models, it forces an affirmative case from the builder. Others no longer have to prove danger before anyone hits the brakes. The builder has to show safety, or stop.
What a real AI safety case would contain
A serious case is more than a checklist. It has a shape.
- A claim: this model, used in this way, does not pose an unacceptable risk of specified harms.
- An argument: the reasoning that connects evidence to the claim, including how identified risks are handled and why the safeguards are adequate.
- Evidence: results from capability evaluations, red-teaming, security measures, and analysis the argument depends on.
It also has to state assumptions and failure points. A good case is falsifiable. It tells a reviewer what would need to be true for the conclusion to hold, so the reviewer can check whether those conditions actually hold. Without that, theater.
Why the discipline is valuable
Writing the argument down surfaces gaps a confident release note can hide. A risk is harder to wave away when you must construct explicit reasoning that addresses it. The method matches how other high-hazard fields got safer: by reasoning about hazards before operation, not only by counting wrecks afterward. It also slots into the threshold logic of responsible scaling policies, where a crossed capability line should trigger a case strong enough to justify the next step.
The folk objection, named
The pushback is practical: give the science a few more years of evaluations and interpretability, then safety cases will write themselves; requiring them now only slows useful systems. On this view, the paperwork should track the evidence, not lead it.
In aviation the case can lean on mature science, known failure rates, understood physics, and decades of fleet data. For a frontier model, an honest case runs into how little we can currently prove. We cannot yet demonstrate that a capable model is not deceptively aligned. We cannot rule out capabilities nobody thought to test. We cannot show that behavior in evaluation will hold in deployment, especially if the system is sandbagging.
A rigorous case for a sufficiently advanced model would need assurances current science cannot supply. So an honest attempt often produces, as its real output, a clear statement of why the system cannot yet be shown to be safe. The paperwork did its job when it refuses to rubber-stamp a gap.
The value of a safety case is the permission to refuse.
Why the Foundation supports them
A governance regime built on safety cases refuses to treat inability to prove danger as a license to proceed. Burden sits on the builder. If the builder cannot meet it, the answer is not to train the next jump and hope. Safety cases should be mandatory, independently reviewed rather than self-graded, and required before the largest training runs, not after a product launch.
Made binding in that form, they become one of the stronger tools available. That is why they feature in the wider design of our plan.