Use-case guide
AI in Content Moderation & Online Platforms
Anyone running a platform with user-generated content, or shipping a generative-AI product the public uses, sits at the intersection of three accelerating regimes: the EU AI Act's transparency rules for generative AI (Article 50), the EU Digital Services Act's content-moderation obligations, and a wave of national rules on AI-generated political content, intimate-image deepfakes, and child sexual abuse material. China's Generative AI Measures add a fourth regime if your service reaches mainland users. The compliance asks aren't just 'don't generate bad stuff' — they're about labelling, audit logs, takedown SLAs, and risk assessments.
For: Trust-and-safety teams, platform policy leads, generative-AI product teams, marketplaces with user-generated content
What's at stake
Generative AI has explicit transparency duties
EU AI Act Article 50 requires providers to: (1) mark AI-generated output as machine-readable AI-generated, (2) tell users when they're interacting with AI (chatbots), and (3) label deepfakes prominently. Providers of GPAI models face their own transparency duties (Article 53).
Platforms have their own content-moderation regime in the EU
The Digital Services Act (DSA) requires platforms to act on illegal content, run annual risk assessments, give users notice of moderation decisions, and submit to audits. Very Large Online Platforms (45M+ EU users) have additional systemic-risk duties around generative AI.
Deepfakes are illegal in specific scenarios
Most jurisdictions now prohibit AI-generated intimate images without consent, AI-generated election content without disclosure, and AI-generated child sexual abuse material outright (even synthetic). Penalties range from civil damages to criminal liability for executives.
China requires pre-launch security assessment for public-facing services
Under the Generative AI Measures, any generative-AI service offered to the public in mainland China must complete a CAC security assessment and algorithm filing before launch. Content moderation against prohibited categories is enforced via Cyberspace Administration takedown orders.
Regulations that apply
EU AI Act (Article 50 + Chapter V GPAI)
LawTransparency obligations for chatbots, generative content labelling, deepfake marking. GPAI providers face training-data transparency and systemic-risk duties.
Where in the text: Articles 50, 52, 53, 55.
China Generative AI Measures
LawPre-launch security assessment, algorithm filing, real-name user verification, labelling of AI-generated images and video, content-moderation against prohibited categories.
Where in the text: Articles 7, 9, 14, 17.
EU Digital Services Act
LawNotice-and-action procedures, statement of reasons for moderation decisions, annual risk assessments for VLOPs, audit obligations, transparency reporting.
Where in the text: Articles 16, 17, 26, 34, 37 (DSA).
US state deepfake laws
LawMost US states now have laws covering intimate-image deepfakes, election deepfakes, or both. Civil and/or criminal liability for non-consensual synthetic content.
Where in the text: California AB 2655 / SB 926; New York Chap. 56/2024; Texas SB 751; many others.
Do
- ✓Watermark or otherwise mark AI-generated content at point-of-generation in a machine-readable way (C2PA, SynthID, or equivalent).
- ✓Implement a tiered review pipeline: keyword + model + human, with audit logs at every step.
- ✓Provide users with a 'why was this moderated?' explanation that meets DSA Article 17 standards even if you're outside the EU — it's quickly becoming the global baseline.
- ✓Get user consent and run age verification before generative-AI features that can produce real-people likenesses.
- ✓Maintain an internal taxonomy of prohibited categories that maps to each market you operate in — and audit moderation outcomes against it monthly.
Don't
- ✗Don't ship an AI feature that can generate realistic images of real people without consent or age controls. The legal exposure is now significant in EU, US, and China.
- ✗Don't ignore the Article 50 chatbot disclosure even if the user 'obviously knows' it's AI. The disclosure must be unambiguous.
- ✗Don't deploy a generative-AI service publicly in mainland China without completing the CAC security assessment and algorithm filing first.
- ✗Don't rely solely on the AI's own moderation classifier — a self-marking AI is, predictably, biased to under-report its own failures.
- ✗Don't conflate 'free speech' arguments with EU DSA compliance. The DSA doesn't ask you to remove lawful speech; it asks you to be transparent about moderation. Conflating the two creates avoidable legal risk.
Also worth knowing
If you're a Very Large Online Platform under the DSA: your generative-AI features sit inside the systemic-risk assessment (Article 34) and are subject to the independent audit (Article 37). If you're a small platform: the asymmetric obligations of the DSA mean most of the heaviest duties don't apply yet, but the transparency-of-AI duties under the EU AI Act do, regardless of platform size.
Want a tailored answer?
The wizard takes your jurisdiction, AI use case, and data types and gives you the top-3 regulations to focus on — in 60 seconds.
Start the wizard →Educational guide. Not legal advice. For specific compliance decisions, consult qualified counsel in the relevant jurisdiction.
Note: this guide was drafted with AI assistance — Anthropic Claude.