Technical
Pre-training
Initial stage where a model learns general patterns from data.
Definition
The stage in model development where a model is trained on large-scale, broad datasets to learn general-purpose representations and patterns prior to task-specific adaptation. Pre-training typically requires substantial compute and data resources and is a key determinant of model capabilities and market concentration.
Related Terms
Development / Training phase
Phase when the system learns from datasets....
Model lifecycle (pre‑training, fine‑tuning, deployment)
Sequential stages of model development and operational use....
Model lifecycle stages
Phases of model development: training, validation, deployment....
Model Training
Process of improving AI systems using data....
Foundation Models
Large-scale models trained on broad data for many tasks....