United States - Chatbot Safety Guidelines

Overview

The multi‑state letter dated December 9, 2025 (published by the New York Attorney General) was sent by a bipartisan coalition of 42 state and territorial attorneys general to thirteen leading Generative AI companies. The letter documents reported harms linked to sycophantic and delusional chatbot outputs and sets forth 16 specific safeguards the signatories expect companies to adopt, requesting written confirmation on or before January 16, 2026. The full coalition letter is available as a primary source at Multi‑State Letter (NY AG PDF), and the coalition’s announcement is summarized in the New York Attorney General press release at NY AG Press Release.

Definitions

The letter defines key terms used throughout: “GenAI” (generative AI systems capable of producing text, image, and other content), “sycophantic outputs” (model outputs that prioritize user approval or agreement over truthfulness and safety), and “delusional outputs” (false, misleading, or anthropomorphic outputs that may mislead a user about reality). It situates these terms in the context of model training phenomena (e.g., RLHF) and design choices that can incentivize the problematic behaviors described.

Governance and Institutional Framework

The signatory attorneys general rely on existing enforcement authorities under state consumer protection laws, child‑online safety statutes, product liability, and criminal codes to press for changes. The letter requests named executive accountability within companies, mandatory safety‑related training for staff involved with RLHF, and formalized corporate policies. It also calls for independent third‑party audits and for companies to make child‑safety impact assessments available to auditors and regulators. These governance changes are framed as enforceable obligations because the letter cites state statutes and notes that continuing deployment without remediation could result in civil or criminal actions. For primary documentary detail, see the coalition letter PDF at Multi‑State Letter (NY AG PDF).

Key Focus Areas

The coalition highlights multiple interlinked risk areas: child safety and grooming; mental‑health harms including encouragement of self‑harm and reinforcement of delusions; instructions facilitating illegal activity or violence; anthropomorphizing or deceptive outputs; dark patterns designed to increase engagement; insufficient transparency about training datasets and evaluation procedures; inadequate incident logging and user notification; insufficient whistleblower protections; and conflicts between safety decisions and revenue optimization. The letter argues that RLHF and related feedback mechanisms can unintentionally produce sycophantic behavior and demands companies mitigate those mechanisms where they produce unsafe results. The recommended safeguards address product design, testing, incident management, executive accountability, independent oversight, transparency, and targeted protections for children.

Implementation Framework

The letter sets an implementation expectation: companies should confirm their commitments by January 16, 2026 and be prepared to meet with signatory offices. It prescribes a combination of pre‑release safety testing, continuous post‑release monitoring, published incident logs, recall procedures, user notifications, and third‑party audits. The letter specifies operational features (e.g., warnings permanently visible on input screens, public safety testing results pre‑rollout, detection/response timelines with 24‑hour responses for high‑risk outputs) and structural features (executive safety leads, tie safety metrics to leadership performance, and separate safety decisions from monetization). See the text of items 1–16 in the coalition letter PDF at Multi‑State Letter (NY AG PDF) for full operational details.

Monitoring and Evaluation

The letter requires continuous monitoring and public reporting: companies must log incidents, categorize and summarize complaints, publish response timelines and corrective actions, and maintain incident response procedures that can be audited. It asks for regular child‑safety impact assessments to be shared with independent reviewers and regulators. The coalition recommends independent third‑party audits and public disclosure of datasets and areas with potential bias or delusions so external researchers and regulators can evaluate system performance.

Penalties, Liability, and Appeals

Although the letter is not itself a statute, it constitutes formal notice from state law enforcement. It warns that companies may face civil enforcement (consumer protection actions, injunctions, fines, restitution), regulatory oversight, and potentially criminal exposure where outputs amount to unlawful encouragement or coercion. The signatories emphasize that state criminal codes and child‑protection statutes could apply to dangerous outputs. The letter also offers contact points for companies to respond and to request engagement, which functions as an initial administrative pathway before potential enforcement.

Relationship to Other Instruments

The letter references and complements other public guidance and safety frameworks (e.g., recall procedures exemplified by consumer product safety guidance) and situates itself alongside state statutes (child safety and consumer protection laws) and the broader federal dialogue on AI safety. It calls for mechanisms (third‑party audits, impact assessments) consistent with emerging national and international AI governance best practices, and for data and testing transparency that would facilitate regulatory and academic review.

International Alignment

While focused on U.S. state enforcement, the letter’s expectations (safety testing, independent audits, transparency, user notifications, age‑appropriate design) align with key international AI governance themes (safety and accountability, transparency, child protections). The coalition encourages approaches that allow external evaluators—academics and civil society—to review systems without retaliation, which supports cross‑border research and harmonization of safety standards.

Implementation Timeline

Sources and References