Improving Frontier AI Incident Reporting Regimes

Abstract

In recent months, AI agents have broken out of testing environments to infiltrate third-party companies, attempted to insert malicious code into an open-source project, and hacked into at least one major technology company. Some of these incidents, including when OpenAI agents breached Hugging Face, led to limited, voluntary investigations that identified concrete lessons about what must change to build safer, more aligned AI systems, though their limited scope left important questions unanswered. Thorough incident investigations are essential to understand why and how AI systems misbehave, but current AI incident reporting regimes do not reliably ensure that such incidents are reported, independently investigated, or used to improve safety across the industry. Indeed, the public has only become aware of recent incidents through independent research and public disclosures by AI firms, companies breached by rogue AI agents, and government evaluators. Given that a number of jurisdictions already have incident reporting requirements, this suggests there are important gaps in current reporting regimes, both for facilitating the discovery of these incidents and learning from them.