Foundation model for comprehensive transcriptional regulation analysis

Abstract

Transcription regulation is a complex biological process that researchers have long sought to decode. With recent ad- vancements in artificial intelligence (AI), foundation models, particularly those based on transformer architectures, are emerging as promising tools to address this challenge. These models are first pre-trained to capture intricate patterns within input genomic data, such as DNA sequences and epigenomic or transcrip- tion factor binding information, mainly through self-supervised learning. This foundational training allows them to then be fine-tuned for specific biological contexts, predicting and interpreting how genomic features influence transcription regulation. This perspective reviews the development and applications of founda- tion models in transcription regulation, with a focus on their training methods, architectures, and the biological insights they provide into gene regulation. We also discuss the potential of multi-modal models, which integrate diverse data types to enhance predictive power and interpretability. With ongoing progress in model design and data integration, AI holds the potential to transform our understanding of gene expression and ad- vance transcription regulation research......