A novel virtual patient approach for cross-patient multimodal fusion in enhanced breast cancer detection

作者：Younes Akbari, Faseela Abdullakutty, Somaya Al-Máadeed, Rafif Al-Saady, Ahmed Bouridane, Rifat Hamoudi · 发表于：Computerized Medical Imaging and Graphics · 年份：2025 · DOI：10.1016/j.compmedimag.2025.102687 · 被引用次数：4 · 研究领域：AI in cancer detection、Advanced Neural Network Applications、COVID-19 diagnosis using AI

Multimodal medical imaging combining conventional imaging modalities such as mammography, ultrasound, and histopathology has shown significant promise for improving breast cancer detection accuracy. However, clinical implementation faces substantial challenges due to incomplete patient-matched multimodal datasets and resource constraints. Traditional approaches require complete imaging workups from individual patients, limiting their practical applicability. This study investigates whether cross-patient multimodal fusion combining imaging modalities from different patients, can provide additional diagnostic information beyond single-modality approaches. We hypothesize that leveraging complementary information from heterogeneous patient populations enhances cancer detection performance, even when modalities originate from separate individuals. We developed a novel virtual patient framework that systematically combines imaging modalities across different patients based on quality-driven selection strategies. Two training paradigms were evaluated: Fixed scenario with 1:1:1 cross-patient combinations (∼250 virtual patients), and Combinatorial scenario with systematic companion selection (∼20,000 virtual patients). Multiple fusion architectures (concatenation, attention, and averaging) were assessed, and we designed a novel co-attention mechanism that enables sophisticated cross-modal interaction through learned attention weights. These fusion networks were evaluated using histopa...