Inference
Technical processes and infrastructure for running AI models.
How the laws define this
Both sources agree that inference involves operating trained artificial intelligence models to serve active functions or generate outputs. The crucial point of divergence centers on whether the concept defines a procedural action or the underlying technical environment. Under ISO/IEC 23053:2022 - Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML), inference is defined strictly as a process: applying a trained model to new data to produce outputs or predictions. In contrast, the North Rhine-Westphalia AI Strategy Paper 2.0 (KI-Strategiepapier NRW 2.0) defines inference as the technical execution and infrastructure required to operate AI models, particularly large language models. Rather than focusing solely on data inputs and outputs, this strategy emphasizes model hosting, compute resources, and serving mechanisms, specifically discussing the feasibility of providing inference services on state-owned servers in cooperation with HPC.nrw to support academia and administration. Compliance scope therefore depends on whether a rule targets algorithmic data processing or hosting infrastructure.
Synthesised from the 2 statutory definitions below. Each one is quoted in full, with its source.
How jurisdictions differ
What is distinctive in each definition, binding law in force first.
| Jurisdiction | Instrument | Status | What is distinctive |
|---|---|---|---|
| ISO | ISO/IEC 23053:2022 - Framework for Artificial Intelligence (AI) Systems Using Machine Learning (ML) | In Force | This definition narrows inference to the core process of applying a trained model to new data for outputs or predictions, omitting any mention of infrastructure or execution mechanisms. |
| North Rhine-Westphalia, Germany | North Rhine-Westphalia AI Strategy 2.0 | In Force | This definition widens inference to encompass the full technical execution and infrastructure for operating AI models, including hosting, compute, and serving on state-owned servers. |
Definitions (2)
The technical execution and infrastructure required to operate AI models (particularly large language models), including model hosting, compute resources, and serving mechanisms; the strategy discusses feasibility of providing inference services on state-owned servers (in cooperation with HPC.nrw) to support academia and administration.
The process of applying a trained model to new data to produce outputs or predictions.
Not to be confused with
Infrastructure refers specifically to compute and storage resources like HPC and cloud services, while inference encompasses the broader processes and use of such resources to run AI models.
Questions and answers
What does inference mean in the ISO AI standard?
In ISO/IEC 23053:2022, inference is the process of applying a trained model to new data to produce outputs or predictions. This fits within the standard's framework for machine learning systems across the AI lifecycle.
How does North Rhine-Westphalia define AI inference infrastructure?
The North Rhine-Westphalia AI Strategy 2.0 defines inference as the technical execution and infrastructure needed to operate AI models, such as model hosting, compute resources, and serving mechanisms. It discusses providing these services on state-owned servers for academia and administration.
How do definitions of inference differ between standards and strategies?
The ISO standard limits inference to the model application process, while the North Rhine-Westphalia strategy expands it to include operational infrastructure like hosting and compute.
Related Terms
Inference (Inferenz)
The runtime process of executing AI models to produce outputs....
High-performance computing (HPC)
Advanced computational infrastructure supporting AI research and deployment....
Infrastructure
Compute and storage resources like HPC and cloud services....
Infrastructure (AI layered model)
Compute, data centers and national platforms supporting AI....
Digital Sovereignty (Digitale Souveränität)
Autonomy and control over data, infrastructure, and AI tools....