From transparency to accountability and back: A discussion of access and evidence in ai auditing

Abstract

This paper by Cen and Alur (2024) addresses the critical need for effective AI auditing in light of increasing AI regulation. It explores the balance between transparency and accountability, focusing on the types and extent of access required for meaningful AI audits. The authors examine various contexts in which AI audits arise, including regulatory requirements like the EU AI Act, GDPR, and DSA, emphasizing the importance of independent verification of developer claims. The paper discusses the challenges of providing sufficient access to AI systems for auditing purposes while protecting intellectual property and managing resource constraints. It considers different auditing approaches, such as pre-deployment risk assessments, ongoing monitoring, and compliance testing, and highlights the need for a healthy AI auditing ecosystem with clear standards and processes. Key findings revolve around the necessity of defining minimal access requirements for effective and efficient AI audits. The work suggests that data disclosure audits can encourage good company practices and prevent downstream harms. The discussion contributes to the development of AI governance frameworks by emphasizing the need for robust auditing tools and standardized metrics to promote ethical and responsible AI deployment. The paper underscores the importance of establishing clear audit trails and documentation to ensure AI systems are scalable, defensible, and trustworthy, ultimately fostering greater trust and safer AI adoption.