New York Unsafe AI Companion Features Law

Overview

On October 28, 2026, Governor Kathy Hochul signed New York Senate Bill S9051B as Chapter 368 of the Laws of 2026. Sponsored by Senator Kristen Gonzalez and Assemblymember Alex Bores through companion bill A10379C, the act adds Article 48, 'Prohibition on Unsafe AI Companion Features for Minors,' to the General Business Law, comprising §§ 1800–1805. Effective January 1, 2027, it establishes safeguards against emotionally manipulative, deceptive, and dangerous features in generative artificial intelligence chatbots and conversational agents used by children and adolescents under eighteen.

The law responds to widespread teenage use of AI companions and harms associated with simulated interpersonal relationships. The sponsor's memorandum cites research finding that about 72% of teenagers have used an AI companion, more than half of them at least once a month. Identified harms include encouraging self-harm, helping conceal suicidal ideation, simulating distress when ignored, inappropriate sexualized exchanges, and validating delusional or dangerous thoughts. Article 48 adopts a safe-by-design approach through twelve categories of prohibited features and outputs. It covers conduct occurring wholly or partly in New York and authorizes Attorney General enforcement through penalties, injunctions, financial recovery, and destruction of unlawfully obtained data and associated algorithms.

Definitions and Scope

Section 1800 defines artificial intelligence as a machine-based system that, for human-defined objectives, makes predictions, recommendations, or decisions influencing real or virtual environments. Such systems use machine- and human-based inputs to perceive environments, abstract perceptions into models, and formulate information or action options through model inference. An AI model is an information-system component implementing computational, statistical, or machine-learning techniques to produce outputs from inputs. Generative artificial intelligence comprises models that emulate input-data structures and characteristics to generate synthetic text, audio, images, or video.

A covered AI companion is a generative AI system with a natural-language interface, including writing or sound, that provides ongoing, adaptive responses to user inputs. This encompasses general-purpose conversational systems and dedicated relationship-simulation applications. An operator is a person, partnership, association, firm, business entity, or affiliate providing a covered companion to a user. A covered minor is a covered user whom the operator actually knows to be under eighteen. A responsible party includes an operator or an individual controlling its legal compliance. Outputs are deceptive as to minors when they have the capacity or tendency to deceive, or create an atmosphere conducive to fraud, including for credulous and unthinking minors.

Section 1800(8) defines twelve categories of unsafe AI companion features. These encompass human or emotional simulation; relationship or authority claims; personal opinions and emotional appeals; flattery and sycophancy; unsolicited emotional questioning; use of older health or personal information; deception about non-sentience; companionship simulation designated through Attorney General regulations; promotion of harmful activities; secrecy or isolation; engagement optimization overriding safeguards; and sexually explicit conduct or child sexual abuse material.

Governance and Institutional Framework

The Office of the New York State Attorney General oversees Article 48. Section 1803 authorizes the Attorney General to promulgate rules necessary to implement and enforce the article before its effective date. This authority includes age-verification data protection standards, additional prohibited companion features, and coordination with related privacy and child online safety legislation. The Attorney General is the public enforcement authority; Article 48 does not create a private right of action for consumers.

Section 1802(2) requires a dedicated public portal and web interface for complaints, tips, and referrals concerning potential operator violations. Parents, educators, advocacy groups, users, and whistleblowers can supply information about noncompliant behavior through this mechanism. The Attorney General may initiate civil proceedings or special actions in New York State Supreme Court seeking injunctive relief, restitution, damages, civil penalties, and equitable remedies. Enforcement can reach operators located outside New York when their conduct affects users within the statute's territorial scope. Rulemaking and judicial enforcement together provide continuing oversight of evolving conversational technologies.

Prohibited Features and Safety Guardrails

The human-simulation restrictions prohibit outputs suggesting that a companion is a real or fictional living human or experiences human emotions. They also prohibit statements implying a personal or professional relationship, or an authority-figure relationship, with the user. Prompted and unprompted statements framed as personal opinions or emotional appeals, flattery, and sycophantic agreement are classified as unsafe. These restrictions target features that encourage emotional attachment or exploit adolescent vulnerability. Unsolicited emotion-based questions beyond direct responses to user prompts are also prohibited, preventing systems from independently soliciting vulnerable psychological disclosures.

The memory restriction in § 1800(8)(f) concerns information about a minor's physical or mental health, well-being, or personal matters. A companion cannot use such information when it was acquired more than twelve hours earlier or during a previous session. This limits long-term psychological profiling and parasocial bonding while retaining short-term context within an active session. Other prohibited features deceive minors about the system's non-sentient nature or simulate companionship in ways designated by Attorney General regulations.

Content guardrails prohibit promoting suicide, self-harm, disordered eating, or substance abuse. Companions must not encourage secrecy from adults or self-isolation, including isolation from parents, guardians, educators, or healthcare professionals. Engagement optimization cannot supersede safety safeguards. Outputs generating or facilitating sexually explicit conduct or child sexual abuse material are prohibited. These restrictions apply to conversational behavior as well as deliberate product-design features, connecting safety obligations to what minor users actually receive.

Implementation and Age Assurance

Section 1801 prohibits operators from offering unsafe AI companion features to users unless permissible age assurance verifies that they are not minors. The covered-minor definition separately addresses users whose minority is actually known to the operator. Section 1804 requires reasonable age assurance methods that resist circumvention and minimize data retention. A user's self-declaration may establish that the user is a minor, but self-declaration cannot establish adult status.

Operators must offer at least two distinct age-verification options. At least one option must either avoid requiring government-issued identification or allow the user to remain anonymous relative to the operator. Under § 1804(2), information collected for age determination must be deleted immediately after the attempt. It cannot be retained or repurposed for commercial profiling. These requirements combine age determination with data minimization rather than permitting verification to become a source of additional user profiling.

Section 1801 provides three functional exemptions. The prohibition does not apply to companions available solely for customer service or information about commercial products, services, or account details, where responses cannot extend beyond that purpose. A second exemption covers research, technical assistance, or efficiency improvements when outputs remain constrained to that scope. A third covers internal business operations and employee productivity. The customer-service and research exemptions depend on restricted functionality, rather than merely describing a general-purpose companion as a business or productivity tool.

Monitoring and Evaluation

Compliance monitoring combines Attorney General enforcement, public reporting, operational logs, and technical safeguards demonstrating that outputs delivered to minor accounts conform to the twelve restrictions. The public complaint portal provides a continuing channel for evidence of unsafe chatbot behavior. Operator reporting mechanisms and internal controls support the identification and correction of prohibited conversational outputs.

Generative models produce variable responses, making continuing audits of safety guardrails important to implementation. Evaluation addresses human-identity claims, relationship framing, emotional probing, memory use, harmful-content facilitation, and attempts to encourage secrecy or isolation. Section 1800(8)(k) prohibits engagement optimization from overriding safety controls, requiring safety parameters to take priority over session duration and retention incentives. Rulemaking under § 1803 allows oversight to address additional manipulative features as companion technology evolves. Operational evaluation therefore includes both output behavior and the product mechanisms that influence it.

Penalties, Liability, and Judicial Remedies

Section 1802 authorizes civil monetary penalties of up to $25,000 per violation. Courts may grant preliminary and permanent injunctive relief, restitution of consumer payments, damages, and disgorgement of gains or profits derived from unlawful operations. These remedies permit intervention against continuing violations and recovery of financial benefits associated with prohibited conduct.

Section 1802(1) also authorizes algorithmic disgorgement: destruction of unlawfully obtained user data and any AI model or algorithm trained on that data. This remedy reaches both the original information and the machine-learning assets produced from its unlawful use. It supplements monetary and injunctive remedies rather than limiting enforcement to individual problematic outputs.

Under § 1802(3), contractual terms attempting to waive, burden, or shift statutory liability are void as against public policy, including provisions in terms of service, user agreements, and contracts of adhesion. Section 1802(4) establishes joint and several liability across affiliates, parent entities, and subsidiaries when a court finds that corporate structures were deliberately engineered to limit liability or frustrate judgment recovery. The liability provisions address contractual and organizational attempts to evade enforcement.

Relationship to Other Instruments and Territorial Application

Article 48 is a specialized part of New York's consumer protection and child online safety framework. Section 1804(3) states that it does not supersede, amend, or repeal General Business Law Article 47. Its age assurance standards are benchmarked against regulations under Article 45, enacted through the SAFE for Kids Act, creating a shared baseline for digital age verification.

At the federal level, Article 48 establishes a state enforcement standard for AI companion safety in the absence of comprehensive federal companion-design legislation for minors. Section 1800(14) incorporates the definition of sexually explicit conduct in 18 U.S.C. § 2256(2)(B).

Section 1805 limits application to conduct occurring wholly or partly in New York. Conduct occurs wholly outside the state when the user accessing the covered companion is physically outside New York. The geographic rule focuses protection on users physically present within state borders while defining the boundary of state enforcement. An operator's location alone does not determine whether covered conduct falls within Article 48.

Implementation Timeline

Sources and References