Predicting glioblastoma molecular subtypes and prognosis with a multimodal model integrating convolutional neural network, radiomics, and semantics

作者：Sheng Wei Zhong, Jiaxin Ren, Zepeng Yu, Yida Peng, Cheng‐Wei Yu, Davy Deng, Yangyiran Xie, Zhenqiang He, Hao Bo Duan, Bo Wu, Hui Li, Wenzhuo Yang, Yang Bai, Ke Sai, Yinsheng Chen, Chengcheng Guo, De‐Pei Li, Ye Cheng, Xiangheng Zhang, Yonggao Mou · 发表于：Journal of neurosurgery · 年份：2022 · DOI：10.3171/2022.10.jns22801 · 被引用次数：27 · 研究领域：Glioma Diagnosis and Treatment、Brain Tumor Detection and Classification、Radiomics and Machine Learning in Medical Imaging

OBJECTIVE: The aim of this study was to build a convolutional neural network (CNN)-based prediction model of glioblastoma (GBM) molecular subtype diagnosis and prognosis with multimodal features. METHODS: In total, 222 GBM patients were included in the training set from Sun Yat-sen University Cancer Center (SYSUCC) and 107 GBM patients were included in the validation set from SYSUCC, Xuanwu Hospital Capital Medical University, and the First Hospital of Jilin University. The multimodal model was trained with MR images (pre- and postcontrast T1-weighted images and T2-weighted images), corresponding MRI impression, and clinical patient information. First, the original images were segmented using the Multimodal Brain Tumor Image Segmentation Benchmark toolkit. Convolutional features were extracted using 3D residual deep neural network (ResNet50) and convolutional 3D (C3D). Radiomic features were extracted using pyradiomics. Report texts were converted to word embedding using word2vec. These three types of features were then integrated to train neural networks. Accuracy, precision, recall, and F1-score were used to evaluate the model performance. RESULTS: The C3D-based model yielded the highest accuracy of 91.11% in the prediction of IDH1 mutation status. Importantly, the addition of semantics improved precision by 11.21% and recall in MGMT promoter methylation status prediction by 14.28%. The areas under the receiver operating characteristic curves of the C3D-based model in the I...