Cybersecurity Requirements

Official/legal meaning. Under the EU Artificial Intelligence Act (Regulation (EU) 2024/1689), high‑risk AI systems must be designed and developed to achieve an appropriate level of accuracy, robustness and cybersecurity and to remain resilient against attempts by unauthorised third parties to alter their use, outputs or performance. The Act requires technical solutions proportionate to the risks and explicitly lists measures addressing AI‑specific threats such as data poisoning, model poisoning, adversarial examples, confidentiality attacks and model extraction. (EU AI Act, Article 15; Recitals 76, 77, 115).

Context and scope. The phrase cybersecurity requirements in AI regulation refers to the set of technical and organisational controls that apply especially to high‑risk AI systems but may also inform best practice for other AI. For AI covered by the AI Act, these requirements are lifecycle‑wide (design, development, deployment, operation, maintenance and decommissioning) and must be appropriate to the system’s intended purpose, its risk profile, and the relevant ICT infrastructure. Where an AI system is also within the scope of a separate horizontal EU regulation on cybersecurity of products with digital elements, fulfilling the essential cybersecurity requirements under that horizontal regime may be accepted as demonstrating conformity with the AI Act’s cybersecurity requirements. (EU AI Act, Article 15; Recital 77).

Practical implications for businesses and providers. For providers and deployers of high‑risk AI systems, cybersecurity requirements translate into concrete obligations to assess AI‑specific attack surfaces, integrate security into design and QA processes, and document measures in technical documentation and declarations of conformity. Organisations should adopt an outcomes‑based approach that aligns with established security frameworks and standards (for example NIST cybersecurity guidance, the NIST AI RMF, ISO family standards and applicable EU cybersecurity law) while tailoring controls to AI‑specific threats. The EU Act highlights that technical solutions should, where appropriate, include measures to prevent, detect, respond to and mitigate attacks on training data, pre‑trained components, inputs and model confidentiality. (EU AI Act, Article 15; Recitals 76, 115; NIST AI RMF, Section 3.3).

Key elements / typical criteria. Cybersecurity requirements for AI systems commonly include the following categories (illustrative, non‑exhaustive):

• Threat and risk assessment: identification of AI‑specific threats (data/model poisoning, adversarial inputs, model extraction, membership inference) and mapping them to impacts on safety, privacy and fundamental rights. (EU AI Act, Article 15; Recital 76).
• Preventive controls: access controls, encryption of datasets and model weights, secure CI/CD and supply‑chain controls, integrity checks, and measures to harden models against adversarial manipulation. (EU AI Act, Recital 115).
• Detection and monitoring: logging, anomaly detection, model‑behaviour monitoring, and incident detection tuned to AI failure modes (e.g., sudden drift, atypical queries). (EU AI Act, Article 15; NIST AI RMF).
• Response and recovery: incident response plans, fail‑safe or graceful degradation mechanisms, rollback/retrain procedures and forensic capability for attacks on AI assets. (EU AI Act, Article 15).
• Documentation & assurance: technical documentation, conformity evidence, declarations of conformity, and post‑market monitoring demonstrating implemented cybersecurity measures. (EU AI Act, Articles 11, 42; Recital 77).

Examples and cross‑references to other frameworks. Common real‑world scenarios covered by AI cybersecurity requirements include: poisoning a training dataset to bias an outcome (data poisoning); crafting adversarial inputs that force incorrect outputs (model evasion); querying models to reconstruct proprietary model parameters (model extraction); and exfiltrating confidential training records via membership inference. Practitioners should map EU Act obligations to complementary guidance such as the NIST AI RMF’s secure and resilient characteristic (NIST AI RMF 1.0, Section 3.3) and the OECD principle on Robustness, security and safety (OECD AI Principles). ISO/IEC 22989 and related ISO/IEC AI standards provide terminology and trustworthiness concepts (security, resilience) that can be used to align organisational policies with regulatory requirements (ISO/IEC 22989:2022; NIST AI RMF; OECD AI Principles).

Where to look in the law and standards. The core legal text is the EU AI Act: see Article 15 (Accuracy, robustness and cybersecurity) and related recitals (Recitals 76, 77, 115) for the Act’s explicit requirements and interaction with horizontal cybersecurity regulation. Sectoral or national cybersecurity laws (e.g., NIS2, product cybersecurity rules) and recognised frameworks (NIST Cybersecurity Framework, NIST AI RMF, ISO/IEC AI and cybersecurity standards) provide practical implementation guidance. Businesses should ensure that technical documentation and declarations of conformity record how cybersecurity measures meet the AI Act’s requirements. (EU AI Act, Article 15; Recital 77; NIST AI RMF, Section 3.3; OECD AI Principles; ISO/IEC 22989:2022).