Multi-view contrastive learning and symptom extraction insights for medical report generation

作者：Qi Bai, Xiaodi Zou, Ahmad Alhaskawi, Yanzhao Dong, Haiying Zhou, Sohaib Hasan Abdullah Ezzi, Vishnu Goutham Kota, Mohamed Hasan Hasan AbdullaAbdulla, Sahar Ahmed Abdalbary, Xianliang Hu, Hui Lü · 发表于：Scientific Reports · 年份：2025 · DOI：10.1038/s41598-025-00570-w · 被引用次数：3 · 研究领域：Topic Modeling、Biomedical Text Mining and Ontologies、Machine Learning in Healthcare

The task of generating medical reports automatically is of paramount importance in modern healthcare, offering a substantial reduction in the workload of radiologists and accelerating the processes of clinical diagnosis and treatment. Current challenges include handling limited sample sizes and interpreting intricate multi-modal and multi-view medical data. In order to improve the accuracy and efficiency for radiologists, we conducted this investigation. This study aims to present a novel methodology for medical report generation that leverages Multi-View Contrastive Learning (MVCL) applied to MRI data, combined with a Symptom Consultant (SC) for extracting medical insights, to improve the quality and efficiency of automated medical report generation. We introduce an advanced MVCL framework that maximizes the potential of multi-view MRI data to enhance visual feature extraction. Alongside, the SC component is employed to distill critical medical insights from symptom descriptions. These components are integrated within a transformer decoder architecture, which is then applied to the Deep Wrist dataset for model training and evaluation. Our experimental analysis on the Deep Wrist dataset reveals that our proposed integration of MVCL and SC significantly outperforms the baseline model in terms of accuracy and relevance of the generated medical reports. The results indicate that our approach is particularly effective in capturing and utilizing the complex information inherent in...