Foundation Model‐Enabled Multimodal Deep Learning for Prognostic Prediction in Colorectal Cancer with Incomplete Modalities: A Multi‐Institutional Retrospective Study

作者：Linhao Qu, Chengsheng Zhang, Yingyong Hou, Feng Tang, Weiqi Sheng, Donghui Huang, Zhijian Song · 发表于：Advanced Science · 年份：2026 · DOI：10.1002/advs.202510931 · 被引用次数：1 · 研究领域：Radiomics and Machine Learning in Medical Imaging、Artificial Intelligence in Healthcare and Education、Colorectal Cancer Screening and Detection

Accurate prognostic prediction for colorectal cancer is essential for optimizing personalized treatment strategies and improving patient outcomes. Current unimodal approaches encounter significant limitations in effectively leveraging multimodal data and confront challenges with the issue of missing modalities. A novel multimodal deep learning framework named FLARE, which integrates pathological images, radiological imaging, and clinical text reports, is introduced to provide accurate risk assessments for colorectal cancer survival and progression. FLARE employs foundation models to achieve efficient feature extraction, utilizes an attention-based multi-branch framework to enhance synergy and distinctiveness across modalities, and incorporates a diversity-promoting loss function. To address the issue of incomplete data, FLARE integrates modality and missing-aware prompts, pseudo embeddings, and a modality-level augmentation strategy, thereby effectively mitigating potential performance degradation. The performance of FLARE is retrospectively assessed using a dataset of 1679 colorectal cancer patients from four independent clinical centers. Its superior prognostic capability is demonstrated through Kaplan-Meier analysis and the concordance index. FLARE effectively stratified patients into high- and low-risk groups. It achieved the highest concordance index across all validation cohorts, significantly outperforming traditional clinical models and existing multimodal methods, th...