Canada - Algorithmic Impact Assessment (2019)

Overview

The Algorithmic Impact Assessment (AIA) is a standardized, nation-wide tool maintained by the Treasury Board of Canada Secretariat (TBS) to operationalize the Directive on Automated Decision-Making and ensure federal institutions assess, mitigate and publish risks from automated decision systems (ADS). Introduced in 2019 and distributed openly (including via a public GitHub repository), the AIA consists of risk and mitigation questionnaires that yield an impact score and corresponding level (I–IV). The tool is used by departments to guide governance, testing and transparency measures before a system reaches production and to support ongoing monitoring and public disclosure obligations.

Definitions

Key terms used by the AIA include "automated decision system" (any technology used to make or support administrative decisions, including rule-based systems, machine learning, natural language processing and generative AI), "impact level" (a four-tier scale from Level I to Level IV that reflects potential harm to rights, health, economic interest and ecosystems), "raw impact score" and "mitigation score" (numerical outputs from the questionnaire used to determine final impact), and "client" (an individual or entity affected by the decision). The Directive’s and AIA’s scope, guidance and examples of "administrative decision" are set out on the TBS site and supporting guidance documents.

Governance and Institutional Framework

The AIA is embedded in the Government of Canada’s governance architecture for responsible AI. The Treasury Board of Canada Secretariat, through its Office of the Chief Information Officer (OCIO), maintains the tool, issues interpretation guidance and oversees departmental compliance. Departments listed under the Financial Administration Act (Schedules I, I.1 and II) are subject to the Directive; some entities (for example the Canada Revenue Agency) may be subject to separate arrangements. The AIA requires departmental senior officials and designated approval authorities to review, sign-off and publish completed assessments to the Open Government Portal. Departments must also consult legal services, privacy officers (where personal information is used) and other specialists as part of the assessment and mitigation planning.

Key Focus Areas

The AIA questionnaire and supporting guidance concentrate on multiple risk domains: project governance (roles, approval authorities, program objectives), system architecture and security (design, data flows and cybersecurity classification), algorithmic design (model type, explainability and training data provenance), decision context (what decisions are made and to whom they apply), impact analysis (risks to rights, health, dignity, economic interests and intersectional groups), data governance (personal information, retention, provenance and accuracy), consultation (stakeholder and community engagement, including GBA+ analysis), and mitigation measures (human oversight, testing regimes, peer review and monitoring). For high-impact systems (Levels III–IV), the Directive and AIA require additional steps such as external peer review, strengthened human intervention, and more rigorous testing and documentation.

Implementation Framework

Departments are expected to complete an AIA at the start of the design phase and again before production. The online AIA tool runs as a questionnaire (the official web instance and an open-source implementation are available) that produces a results package exportable as JSON or PDF for departmental records and publication. Appendix C of the Directive maps mitigation requirements according to impact level; departments must integrate AIA results into project governance artifacts (project charters, risk registers, procurement materials and security assessments). The AIA scoring logic (raw impact vs mitigation score and the 80% mitigation adjustment rule) drives whether additional controls are required before deployment.

Monitoring and Evaluation

Once deployed, automated decision systems must be monitored in production: the Directive and AIA require regular reviews and re-assessments whenever system functionality or scope changes, and on a scheduled basis aligned with monitoring plans. Departments must track outcomes, fairness and accuracy metrics, complaint and recourse data, and maintain records sufficient to support audits and investigations. The OCIO provides central support and may request updates, while departmental internal audit and TBS oversight processes assess compliance with the directive and AIA publication requirements.

Penalties, Liability, and Appeals

The Directive (and the AIA which operationalizes it) is a mandatory policy instrument for federal institutions; non-compliance can trigger administrative enforcement by TBS including corrective action demands, requirement to halt or remediate deployments, internal accountability measures and escalation to deputy heads or the Treasury Board. The AIA itself does not create criminal penalties; affected individuals retain rights under the Privacy Act, Access to Information Act and other applicable statutes to seek remedies. Departments are also required to provide recourse and review mechanisms for individuals affected by decisions as part of mitigation measures tied to higher impact levels.

Relationship to Other Instruments

The AIA is closely linked to the Directive on Automated Decision-Making and complements other federal instruments including the Privacy Act, Policy on Service and Digital, guidance on generative AI, and the Open Government publication requirements. The AIA’s open-source code is maintained in the TBS GitHub repository, enabling public reuse and international adaptation. Completed AIAs are published on the Open Government Portal alongside departmental transparency materials.

International Alignment

The Canadian AIA has been referenced internationally as a practical public-sector model for algorithmic impact assessment. Its open approach aligns with OECD and international best practice on AI governance and is used as an exemplar by international observatories such as the OECD Observatory of Public Sector Innovation. The AIA’s risk-tiering, open-source implementation and publication requirements facilitate cross-jurisdictional comparisons and inform transnational policy dialogue on responsible public-sector AI.

Implementation Timeline

Sources and References