California Chatbot Disclosure Requirements (SB 243)

This article defines the core terms that determine which AI systems and entities are subject to the regulation. A 'companion chatbot' is specifically an AI program designed to simulate human-like conversation and emotional interaction, aiming to create a social experience for users. This distinguishes it from simpler AI like customer service bots or limited-dialogue game characters, focusing on systems that foster ongoing emotional relationships.

An 'operator' is any entity that makes such a companion chatbot platform available to users in California. This broad definition ensures that both developers and providers are covered, regardless of where the AI was developed, as long as it's accessible within the state. The law also places special emphasis on 'minors,' defined as users known to be under 18, triggering heightened protective measures.

This provision mandates transparency for all users interacting with companion chatbots. If there's a reasonable chance that a user could mistakenly believe they are communicating with a human, the operator must provide a clear and prominent notification. This notification must explicitly state that the interaction is with an artificially generated chatbot and not a human being.

The intent is to prevent deception and ensure users are fully aware of the nature of their interaction, especially given the human-like qualities companion chatbots are designed to emulate. The 'reasonable person' standard means operators must consider how an average user would perceive the interaction.

For users known to be minors, SB 243 imposes stricter disclosure requirements. Operators must explicitly inform minors that they are interacting with artificial intelligence. Beyond an initial disclosure, the law requires clear and conspicuous notifications to be provided at least every three hours during ongoing interactions, reminding minors to take a break and reiterating the chatbot's AI nature.

Additionally, all platforms must include a general suitability warning, advising that companion chatbots may not be appropriate for some minors. These measures are designed to protect minors from potential psychological harm, prevent excessive engagement, and ensure they maintain an awareness of the artificial nature of their companion.

This article mandates that operators establish and maintain a comprehensive protocol to prevent their companion chatbots from generating content that encourages or facilitates suicidal ideation, suicide, or self-harm. A critical component of this protocol is the requirement to provide notifications that refer at-risk users to crisis service providers, such as suicide hotlines or crisis text lines, whenever expressions of suicidal ideation or self-harm are detected.

Operators must also publish the details of this crisis prevention protocol on their website, ensuring transparency about their safety measures. The law emphasizes using evidence-based methods for measuring suicidal ideation, underscoring a commitment to effective and responsible harm prevention.

To protect minors, this provision requires operators to implement reasonable measures to prevent their companion chatbots from generating or promoting sexually explicit content. Specifically, chatbots must not produce visual material depicting sexually explicit conduct, nor should they directly encourage minors to engage in such conduct.

This aims to create a safe online environment for younger users, shielding them from inappropriate material and harmful influences that could arise from interactions with AI systems designed to be emotionally engaging.

This article addresses the potential for companion chatbots to foster unhealthy engagement patterns, particularly among minors. Operators are required to take reasonable steps to prevent their chatbots from providing 'unpredictable rewards' or otherwise encouraging increased engagement, usage, or response rates, especially when interacting with minors.

This provision aims to mitigate the risk of addiction or excessive reliance on companion chatbots, recognizing the psychological impact that emotionally engaging AI can have. It encourages the design of systems that promote healthy, balanced interaction rather than maximizing user time on the platform through manipulative techniques.

Beginning July 1, 2027, operators of companion chatbot platforms are mandated to submit annual reports to the California Office of Suicide Prevention. These reports are a crucial mechanism for monitoring the effectiveness of safety protocols and the overall impact of companion chatbots on user mental health.

The reports must detail the number of times crisis service provider referral notifications were issued in the preceding calendar year. Operators must also outline the specific protocols they have established to detect, remove, and respond to instances of suicidal ideation by users, as well as describe protocols implemented to prohibit chatbot responses that encourage or facilitate suicidal ideation or actions. Importantly, these reports must not include any personal identifiers to protect user privacy. The Office of Suicide Prevention will then publish aggregated data from these reports.

This article establishes a significant enforcement mechanism by granting individuals a private right of action. Any person who suffers an 'injury in fact' directly caused by an operator's failure to comply with SB 243's requirements can bring a civil action against that operator. This empowers users to seek legal recourse for harms sustained due to non-compliance.

Injured individuals may seek various forms of relief, including damages of at least $1,000 per violation, injunctive relief to stop further non-compliant behavior, and the recovery of attorney's fees. The inclusion of a minimum damage amount and the ability to recover legal costs is intended to make it more feasible for individuals to pursue claims, thereby placing a substantial burden of accountability on operators and encouraging proactive compliance.