Large‐language‐model empowered 3D dose prediction for intensity‐modulated radiotherapy

作者：Zehao Dong, Yixin Chen, Hiram A. Gay, Yao Hao, Geoffrey D. Hugo, Pamela Samson, Tianyu Zhao · 发表于：Medical Physics · 年份：2024 · DOI：10.1002/mp.17416 · 被引用次数：15 · 研究领域：Advanced Radiotherapy Techniques、Advances in Oncology and Radiotherapy、Advanced Neural Network Applications

Abstract Background Treatment planning is currently a patient specific, time‐consuming, and resource demanding task in radiotherapy. Dose‐volume histogram (DVH) prediction plays a critical role in automating this process. The geometric relationship between DVHs in radiotherapy plans and organs‐at‐risk (OAR) and planning target volume (PTV) has been well established. This study explores the potential of deep learning models for predicting DVHs using images and subsequent human intervention facilitated by a large‐language model (LLM) to enhance the planning quality. Method We propose a pipeline to convert unstructured images to a structured graph consisting of image‐patch nodes and dose nodes. A novel Dose Graph Neural Network (DoseGNN) model is developed for predicting DVHs from the structured graph. The proposed DoseGNN is enhanced with the LLM to encode massive knowledge from prescriptions and interactive instructions from clinicians. In this study, we introduced an online human‐AI collaboration (OHAC) system as a practical implementation of the concept proposed for the automation of intensity‐modulated radiotherapy (IMRT) planning. Results The proposed DoseGNN model was compared to widely employed DL models used in radiotherapy, including Swin Transformer, 3D U‐Net CNN, and vanilla MLP. For PTV, DoseGNN achieved the mean absolute error (MAE) of , , , and between true plans and predicted plans that were 64%, 53%, 64%, 61% of the best baseline model. For the worst case among ...