Towards safety cases for frontier AI training
Published September 28, 2026
Not law. This is a company's own public position on AI regulation. It is not law, and it carries no legal force.
What it argues for
OpenAI’s September 28, 2026 agenda proposes safety documentation as a prerequisite for continuing frontier reinforcement-learning training: "structured safety documentation should be required before continuing any frontier reinforcement learning training run". It describes comprehensive, evidence-based safety cases as an "aspirational north star", not a completed framework, and says its recommendations are still being implemented at OpenAI. The proposal combines technical safeguards, operational controls and incident investigations. Its technical stack covers "alignment training, containment, and monitoring": reviewing training environments for reward exploits, testing alignment during runs, detecting evaluation gaming, hardening sandboxes and research infrastructure, preserving immutable transcripts, and responding rapidly to monitoring alerts. It specifically says automated reinforcement-learning graders should not see chain-of-thought, to prevent training models to evade chain-of-thought monitors. Operationally, another team should challenge each draft safety case; senior leaders should review it and each be able to veto the run; and the responsible leader should be accountable for the case and incident response. Auditors should receive enough access to verify claims, and cases should enumerate residual risks. Controls should make noncompliant runs difficult to start, while monitoring and automatic pausing should "fail closed". Invalidated safety cases should trigger procedures for pausing covered runs, and developers should retain the ability to undo downstream effects of misaligned outputs. For severe incidents, OpenAI proposes root-cause research, operational and cultural postmortems, improved detection tests, and public disclosure after investigations conclude; affected third parties should be notified "as soon as possible". The document expressly limits its scope to frontier reinforcement-learning training, rather than the broader alignment requirements of internal or external deployment.
Stated positions (12)
- Safety documentation should be a prerequisite for continuing frontier reinforcement-learning training: "structured safety documentation should be required before continuing any frontier reinforcement learning training run".
- Full safety cases are a development goal, not an established standard already achieved: "We treat safety cases as an aspirational north star we are building towards".
- Safety cases should address three complementary technical defenses: "alignment training, containment, and monitoring".
- Training should avoid rewarding exploitation of reinforcement-learning environments: "preventing positive reinforcement of reward hacks during training".
- Alignment evaluations should detect test-aware behavior and use blocking thresholds: "Track eval awareness or metagaming" and "with specific blocking thresholds".
- Automated graders should not inspect chain-of-thought during reinforcement learning: "Do not let automated graders see the chain-of-thought in reinforcement learning".
- Containment should protect both the model’s sandbox and the surrounding research infrastructure, with evidence preserved for investigations: "Multiple layers of infrastructure security" and "Save agent transcripts immutably for a reasonable period of time".
- Safety cases should face internal challenge and leadership approval: "a member of another team should write a dissent", and reviewing senior leaders "should each have the ability to veto the run".
- Auditors should be able to test the safety case’s claims, and decision-makers should see unmitigated risks: "Auditors should be provided with sufficient access" and "Safety cases should enumerate a comprehensive list of residual risks".
- Invalidated safety cases should lead to pausing procedures, and safety controls should prevent unsafe operation by default: "runbooks, technical controls, and SLAs for pausing all covered runs" and "Safety features such as monitoring and auto-pausing should fail closed".
- Severe incidents should lead to scientific and organizational investigation: "Researchers should root-cause training dynamics" and "An operational and cultural postmortem should be conducted".
- Investigation findings and changes should become public, with prompt notice to affected parties: "Investigation results, postmortems, and operational changes should be shared with the public following the conclusion of the investigation" and "Affected third parties should be notified as soon as possible."
About this document
An OpenAI web article dated September 28, 2026, categorized under "Safety" and credited "Author: OpenAI". It presents initial guidelines in three numbered sections: technical safeguards, operational guidelines, and investigations of misalignment incidents.
How this sits against AI law
Each stance compared with what EU and US instruments actually require. Where no instrument addresses a theme, that gap is shown rather than hidden.
Safety documentation before continuing training
Structured safety documentation should be required before continuing any frontier reinforcement-learning training run. Comprehensive safety cases are the intended destination, but OpenAI describes them as aspirational.
Articles 53(1)(a) and 55 require technical documentation and, for general-purpose models with systemic risk, evaluation and risk mitigation. They do not establish a safety-case prerequisite before continuing every frontier reinforcement-learning run.
SB 53 requires covered large frontier developers to publish and follow frontier AI frameworks addressing catastrophic-risk assessment and mitigation. It does not establish a universal requirement to prepare a safety case before continuing each frontier reinforcement-learning run.
Alignment testing and resistance to evaluation gaming
Developers should prevent reinforcement of reward exploits, evaluate alignment during training, investigate material regressions, and test for evaluation awareness. Automated reinforcement-learning graders should not see chain-of-thought, to avoid encouraging evasion of monitors.
Article 55(1)(a) requires model evaluations using standardized protocols and tools, including adversarial testing; Article 55(1)(b) requires assessment and mitigation of systemic risks. Neither specifies evaluation-gaming blocking thresholds or excludes chain-of-thought from reinforcement-learning graders.
SB 53’s frontier-framework duties cover catastrophic-risk assessment and mitigation. The law does not prescribe these particular alignment-testing methods or a restriction on graders seeing chain-of-thought.
Layered containment and immutable evidence
Containment should protect both sandboxes and their hosting research infrastructure, with red-teaming of systems that models can reach. Agent transcripts should be stored immutably for a reasonable period across reinforcement-learning training, evaluation and internal deployment.
Article 55(1)(d) requires adequate cybersecurity protection for systemic-risk models and their physical infrastructure, and Article 55(1)(c) requires serious-incident tracking and documentation. It does not expressly require this layered sandbox architecture or immutable storage of all agent transcripts.
SB 53 addresses catastrophic risks, including models evading developer controls, through frontier frameworks and critical-safety-incident reporting. It does not prescribe immutable transcript storage across all of these activities or the specific containment architecture OpenAI proposes.
Internal challenge, leadership vetoes and auditor access
Another team should challenge a draft safety case, senior reviewers should each be able to veto the run, and a designated senior leader should be accountable for it. Auditors should receive sufficient access to verify claims, while residual risks should be enumerated for risk-acceptance decisions.
Articles 53 and 55 assign documentation and systemic-risk duties to providers, and Article 91 permits the Commission to request documentation and information. They do not require a dissent from another internal team, individual leadership vetoes over training runs, or this safety-case audit-access process.
SB 53 requires frontier frameworks to describe how developers assess and mitigate catastrophic risk. It does not mandate these internal dissent and veto arrangements or a general third-party audit of each training safety case.
Fail-closed monitoring and pausing controls
Priority alerts should trigger action within a defined service-level agreement, including automatic pausing where appropriate. Invalidated safety cases should trigger pausing procedures, and controls should prevent runs from starting without suitable monitoring or disabling it from within training.
Article 55(1)(b) requires mitigation of systemic risks, and Article 93 allows the Commission to require corrective measures, including restricting availability or withdrawing a model. Neither prescribes fail-closed monitoring or automatic training-run pauses tied to unacknowledged alerts.
SB 53 requires covered frontier frameworks to address risk mitigation and the circumstances in which a developer would pause development or deployment. It does not prescribe OpenAI’s specific fail-closed controls, alert-response deadlines or automatic pausing mechanism.
Incident investigation and public findings
Severe misalignment incidents should receive root-cause research, operational and cultural postmortems, and improved detection tests. Investigation results, postmortems and operational changes should be published after the investigation, and affected third parties notified as soon as possible.
Article 55(1)(c) requires providers of systemic-risk models to track, document and report serious incidents and possible corrective measures to the AI Office and, where appropriate, national authorities without undue delay. It does not require publication of investigation results and cultural postmortems or general notification of affected third parties.
SB 53 requires critical-safety-incident reporting to California’s Office of Emergency Services and public transparency concerning frontier frameworks and safety assessments. It does not generally require developers to publish completed incident investigations, operational and cultural postmortems, and resulting operational changes.
The most consequential difference is the proposed training gate: OpenAI wants safety documentation before continuing any frontier reinforcement-learning run. The EU AI Act already requires evaluation, systemic-risk mitigation and cybersecurity for systemic-risk general-purpose models under Article 55, while California SB 53 requires covered developers to publish frontier safety frameworks, but neither establishes this universal run-by-run safety-documentation prerequisite. OpenAI’s proposal therefore goes further on training controls, while remaining an agenda whose recommendations it says are still being implemented.
Source
https://openai.com/index/towards-safety-cases-for-frontier-ai-training/- Date on the page:
- "September 28, 2026" printed in the article header, category Safety, author OpenAI.
- Source checked:
- opened and confirmed on 2026-10-30