Generative pretrained transformer 4: an innovative approach to facilitate value-based healthcare

作者：Han Lyu, Zhixiang Wang, Zhixiang Wang, Jia Li, Jing Sun, Xinghao Wang, Pengling Ren, Linkun Cai, Zhenchang Wang, Zhenchang Wang, Max Wintermark · 发表于：Intelligent Medicine · 年份：2023 · DOI：10.1016/j.imed.2023.09.001 · 被引用次数：5 · 研究领域：Artificial Intelligence in Healthcare and Education、Radiomics and Machine Learning in Medical Imaging、Radiology practices and education

: Appropriate medical imaging is important for value-based care. We aim to evaluate the performance of Generative Pre-trained Transformer 4 (GPT-4), a innovative natural language processing model, in providing appropriate medical imaging automatically in different clinical scenarios. : IRB approval was not required due to the usage of non-identifiable data. Instead, we used 112 questions from the American College of Radiology (ACR) Radiology-TEACHES Program as prompts, which is a open-sourced question and answer program to guide appropriate medical imaging. We included 69 free-text case vignettes and 43 simplified cases. For the performance evaluation of GPT-4 and GPT-3.5, we considered the recommendations of ACR guidelines as the gold standard, and then three radiologists analyzed the consistency of the responses from the GPT models with those of the ACR. We set a five score criterion for the evaluation of the consistency. A paired t test was applied to assess the statistical significance of the findings. : For the performance of the GPT models in free-text case vignettes, the accuracy of GPT-4 was 92.9%, whereas the accuracy of GPT-3.5 was just 78.3%. GPT-4 can provide appropriate suggestions to reduce overutilization of medical imaging than GPT-3.5 (t=3.429, P=0.001). For the performance of the GPT models in simplified scenarios, the accuracy of GPT-4 and GPT-3.5 was 66.5% and 60.0%, respectively. The differences was not statistically significant (t=1.858, P=0.070). GPT-4 ...