Multimodal Deep Learning For Treatment Response Prediction In Head And Neck Squamous Cell Carcinoma Tissue Slides

作者：Yuqing Zhang, Ranran Zhang, Jiaxin Hou, Jing CAI, Na Zhang, Zhanli Hu, Wanming Hu, Wenjian Qin · 年份：2026 · DOI：10.1109/nnice68970.2026.11465462 · 研究领域：Head and Neck Cancer Studies、Voice and Speech Disorders、AI in cancer detection

Accurate prediction of pathological complete response (pCR) to neoadjuvant therapy (NAT) is crucial for personalizing treatment in head and neck squamous cell carcinoma (HNSCC). We developed a multi-modal fusion model that leverages routine pre-treatment biopsy specimens to predict pCR. Our approach integrates H&E whole-slide images (WSIs) with computationally generated Ki-67 virtual-staining images, which serve as computational counterparts to standard Ki-67 IHC index. These imaging features are combined with clinical metadata in a model that dynamically weights each modality. The proposed model achieved robust performance in five-fold cross-validation (ACC: 0.77 ± 0.04, F1: 0.80 ± 0.03, AUC: 0.78 ± 0.08), significantly outperforming single-modality benchmarks. t-SNE visualization confirmed that the features enhanced by multi-modal integration from virtual Ki-67 images exhibited high discriminatory power. This method demonstrates the potential of computational pathology to enhance treatment response prediction.