Who Gets to Define the Rules for AI?
Published Sep 13, 2026 · Printed above the title as 'Sep 13, 2026' with '15 minute read', and matched by the page's JSON-LD datePublished (2026-09-13T21:46Z). The essay refers to a roadmap 'published this week', which fits that date.
Not law. This is a company's own public position on AI regulation. It is not law, and it carries no legal force.
What it argues for
This is a signed essay by Aidan Gomez, Cohere's co-founder and CEO, subtitled 'AI Needs Evidenced Standards, Not A Cartel'. It is Cohere's newest statement on how AI should be governed. It is written as a reply to a roadmap it attributes to Anthropic CEO Dario Amodei, which it says asks governments for antitrust exemptions so a few leading labs can agree shared safety standards and limits on the pace of development. It accepts the need for rules, "AI needs guardrails. That is not the dispute and never has been", and supports the core of the proposal: "Independent review of highly capable AI systems is a good idea and we support it." What it rejects is who would write the rules. It argues that such a regime would be a cartel, using the SEC's 1975 rating-agency designation and the EU's 1985 Motor Vehicle Block Exemption as precedents. It also argues that risk defined by compute scale leaves the largest firms as "the only ones qualified to judge." In place of that it proposes four pillars: an evidence-based risk framework built internationally and in the open; "Mandatory transparency" by developers about how their models are built, their capabilities, risks and mitigations, plus better serious-incident reporting; independent testing of the most advanced models, scoped to capabilities the evidence shows are dangerous and tiered by capability and deployment context; and independent assurance in which third parties must "never be paid by the party they're reviewing". It dismisses existential-risk estimates as "gut feelings, vibes, expressed as decimals" and urges attention to present harms such as voice-cloning fraud and automated decisions on essential services. It ends by arguing for sovereign, locally deployed AI, which is what Cohere sells. This is a marked shift from Cohere's March 2025 filing on the US AI Action Plan. That filing asked the government to avoid forcing companies to share or disclose proprietary information, to prefer performance-based standards over pre-market approval, and to keep safety institutes as scientific bodies. The essay now supports mandatory transparency by developers and independent testing and assurance of the most capable systems, under standards the tested companies do not set.
Stated positions (16)
- The question is who sets the rules: "should a handful of select, market-dominant AI companies from Silicon Valley get to define the rules and safety standards of a generational technology for the entire world?"
- Guardrails are accepted, and the dispute is over authorship: "The dispute is over who writes them, who gets to participate and whose interests the rules are protecting."
- Safety-justified exclusivity has produced cartels before. It cites the SEC's 1975 designation of three rating agencies and the EU's 1985 Motor Vehicle Block Exemption, and concludes that "it is possible you can hold strict safety standards without handing the incumbents a monopoly on meeting them."
- Independent review yes, antitrust waiver no: "Independent review of highly capable AI systems is a good idea and we support it." It objects that the proposal asks governments "for a narrow antitrust waiver" and would "require every other AI developer to blindly follow whatever the participants settle on".
- Compute thresholds favour incumbents and miss real risks: "Risk in these existing frameworks gets defined as a function of scale, which makes the companies with enormous systems the only ones qualified to judge." Small orchestrated models and cyber swarms escape a regime "built exclusively around massive compute thresholds".
- An evidence-based risk framework comes before any mandate. It calls for "a coordinated, international effort to develop this framework that is not led by any one nation, but a group of them", with disagreements published. It asks governments to "Fund the testing capacity itself through public research bodies and existing sectoral risk management systems".
- Rules should turn on capability, not on who built the system: "write rules that bind based on what an AI system can do rather than on who built it".
- Mandatory transparency: "AI developers should be transparent about how their models and systems are built, their intended purpose and capabilities, what risks they might pose, and what risk mitigation measures have been implemented." It wants more on serious-incident reporting across the stack and "mechanisms to attach real accountability when real harm occurs."
- Independent testing, scoped by evidence: "The most advanced AI models and systems should face independent testing, but only against the capabilities and in the contexts the risk framework has identified as genuinely dangerous". It lists likely areas: "cyberattacks, synthetic fraud and voice cloning, manipulation at scale, physical or biochemical weapons, and anything touching critical infrastructure".
- Testing should be tiered and proportionate, not a compliance industry: "A tiered and proportionate framework where more-capable models and systems, or models or systems deployed in specific contexts, face more stringent testing", which "must not become a compliance exercise that expands to fill whatever budget the largest firms can absorb."
- Certification open to all: "Certification has to be open to every company rather than restricted to a designated tier of AI developers, and the standard has to be agreed by people other than the companies being measured against it."
- Assurance modelled on finance, aviation and nuclear: layered assurance with criteria that are "collectively developed and published", third parties who "never be paid by the party they're reviewing", and findings that reach the public. It rejects auditors who are "handpicked" by those they audit.
- Existential-risk estimates are not evidence: "They are gut feelings, vibes, expressed as decimals", and loss of control "is a judgement call, not a finding."
- Practical fixes over a frontier-only regime: "Require that serious incidents be reported", set "standards for test-time observability (or at least logging)", and wall off systems in critical infrastructure, "ideally on-prem". Regulate by deployment context: "A small, poorly specified model sitting inside a hospital is a live risk today, and under a frontier-only regime nobody is even looking at it."
- Present harms deserve priority: "voice cloning tools cheaper than a phone bill can empty a pensioner's account in minutes", and "automated decision-making systems can have a real impact on access to essential services."
- Sovereignty and an open process: "Security comes from sovereignty, local deployment, and technological diversity." It wants a rule-writing process that includes academics, civil society, smaller labs, open-source developers and governments, including "people who'd rule against even companies like Cohere."
About this document
A long blog essay of about 3,500 words in the Company News section of cohere.com, titled 'Who Gets to Define the Rules for AI?' with the subtitle 'AI Needs Evidenced Standards, Not A Cartel', presented as 'A perspective from Aidan Gomez, Co-founder & CEO of Cohere'. After an unheaded introduction it runs under five headings: 'We've been here before' (the 1975 SEC rating-agency designation and the 1985 EU Motor Vehicle Block Exemption), 'What's Being Proposed' (its critique of the Amodei roadmap), 'What Better Rules Look Like' (four pillars: an evidence-based risk framework, mandatory transparency, testing scoped by the evidence, and real assurance mechanisms), 'What the Panic Leaves Out' (on existential-risk claims and the failures it says were reported at two labs in July), and 'Who Writes the Rules?' (sovereignty and an open process). It has no footnotes and cites no statute or bill by name.
How this sits against AI law
Each stance compared with what EU and US instruments actually require. Where no instrument addresses a theme, that gap is shown rather than hidden.
Mandatory transparency by AI developers
Developers should be required to disclose how their models and systems are built, their intended purpose and capabilities, "what risks they might pose, and what risk mitigation measures have been implemented", going beyond the model cards already common in the industry.
Article 53 requires every general-purpose model provider to keep technical documentation for the AI Office and national authorities and to give downstream providers information on the model's capabilities and limitations. Article 50 adds transparency duties for certain AI systems. Much of it goes to authorities and downstream providers rather than the public, but it is the mandatory developer transparency Cohere describes.
California's SB 53 requires developers of frontier models (trained with more than 10^26 operations) to publish transparency reports when they deploy new models, and those with over $500 million in annual revenue also to publish frontier AI frameworks. Federally there is no such duty, and the March 2026 recommendations would bar states from regulating AI development. Cohere asks transparency of AI developers generally.
Serious-incident reporting across the development and deployment stack
"Require that serious incidents be reported", so that one company's failure becomes a lesson for the whole field, with reporting by each layer of the development and deployment stack where it has visibility and control.
The Act covers two layers: Article 55 requires providers of systemic-risk general-purpose models to report serious incidents to the AI Office, and Article 73 requires providers of high-risk AI systems to report serious incidents to market surveillance authorities.
SB 53 requires frontier developers to report critical safety incidents to the California Office of Emergency Services within 15 days (24 hours where life is at imminent risk). It covers only developers of models above its compute threshold and only a defined set of critical safety incidents, not deployers further down the stack.
Independent testing of the most capable systems, scoped by evidence and tiered
"The most advanced AI models and systems should face independent testing", but only against capabilities and contexts an evidence-based framework has identified as dangerous, in "A tiered and proportionate framework" that scales with capability and deployment context rather than developer resources.
Article 55 obliges providers of systemic-risk general-purpose models to evaluate them, including adversarial testing, and Article 92 lets the AI Office carry out its own evaluations. That testing is mandatory but done by the provider unless the AI Office steps in. Cohere asks for testing that is independent by default.
Executive Order 14409 sets up a classified benchmarking process for cyber capabilities and a voluntary framework for pre-release government access to 'covered frontier models'. Section 3(c) excludes any mandatory licensing, preclearance or permitting, so independent testing stays voluntary.
Rules keyed to capability and deployment context, not compute scale or who built the system
Rules should "bind based on what an AI system can do rather than on who built it"; regimes built on compute thresholds miss small orchestrated models and cyber swarms, and a poorly specified model in a hospital is a live risk that a frontier-only regime ignores.
The Act's high-risk regime turns on where a system is used (Article 6 and Annex III, and AI in regulated products such as medical devices), whatever the size of its developer. For general-purpose models it presumes systemic risk above 10^25 FLOPs of training compute (Article 51), the kind of threshold Cohere criticises, but the Commission can also designate models on capability criteria.
SB 53 applies only to developers of models trained with more than 10^26 operations, and reserves its heaviest duties for those with more than $500 million in annual revenue. Both tests turn on compute scale and on who built the model, the approach Cohere argues against.
Standards set openly by many parties, not by the leading labs
The risk framework and testing standard must be built openly and internationally, and "the standard has to be agreed by people other than the companies being measured against it"; no antitrust waiver for a few labs to agree limits among themselves.
Under the Act the rules are made by the EU legislator. Detailed requirements come through harmonised standards from European standardisation organisations (Article 40) and codes of practice drawn up under AI Office oversight with input from providers, civil society, academia and independent experts (Article 56). A scientific panel of independent experts advises the AI Office (Article 68). No group of firms sets the standard alone.
The March 2026 legislative recommendations say Congress should support AI deployment "through existing regulatory bodies with subject matter expertise and through industry-led standards" and create no new federal rulemaking body. Cohere wants the standard agreed by people other than the companies measured against it.
Independent assurance: auditors never paid by the audited, certification open to all
Assurance should follow finance, aviation and nuclear practice, with published criteria, third parties that "never be paid by the party they're reviewing" and public findings, and certification should be open to every company rather than a designated tier.
Article 31 requires notified bodies to be independent of the providers of the high-risk systems they assess, and conformity assessment is open to any provider. But the Act does not stop the provider from paying the notified body, which is how conformity assessment is normally funded in EU product law. Most high-risk systems are also assessed by internal control rather than by a third party (Article 43).
SB 53 asks large frontier developers to describe in their published frameworks how they use third-party assessments. It sets no rules on who those assessors are, how they are paid or whether their findings are published.
Isolating AI in critical infrastructure
Anything wired into critical infrastructure, from hospitals to substations, should be walled off, "ideally on-prem", and critical sectors should run AI on infrastructure they control, since "Security comes from sovereignty, local deployment, and technological diversity."
AI used as a safety component in managing critical digital infrastructure, road traffic or the supply of water, gas, heating and electricity is high-risk under Annex III and must meet the Article 15 robustness and cybersecurity requirements. The Act does not require isolation or on-premises deployment.
Executive Order 14409 prioritises the cyber defence of federal systems and would widen critical-infrastructure operators' access to frontier-model defensive tools. It sets no requirement to isolate AI systems in critical infrastructure or to deploy them locally.
The EU AI Act already does much of what Cohere now asks for. It requires documentation and transparency from general-purpose model providers, requires systemic-risk model providers to evaluate their models and report serious incidents, makes high-risk system providers report serious incidents, and is built largely around use and deployment context rather than who built the system. The gaps are independence and payment. Most EU testing is done by the provider or a notified body the provider pays, where Cohere wants third parties who are never paid by those they review. The United States is further from Cohere. Federal policy relies on voluntary access and industry-led standards: Executive Order 14409 has only a voluntary pre-release access route, and the March 2026 legislative recommendations back industry-led standards and would stop states regulating AI development. The binding transparency and incident-reporting duties that do exist are state law, such as California's SB 53, and they apply only to developers of models above a training-compute threshold, with the heaviest duties reserved for high-revenue firms, which is the kind of scale test Cohere criticises. Measured against Cohere's own record, the move is large. In March 2025 it asked Washington not to force disclosure of proprietary information and to prefer performance-based standards over pre-market approval. It now calls for mandatory transparency and independent testing and assurance, under standards the tested companies do not write.
Source
https://cohere.com/blog/who-gets-to-define-the-rules-for-ai- Date on the page:
- Sep 13, 2026
- Source checked:
- opened and confirmed on 2026-09-30