Evaluation of Deep Learning-Based Automated Detection of Primary Spine Tumors on MRI Using the Turing Test

作者：Hanqiang Ouyang, Fanyu Meng, Jianfang Liu, Xinhang Song, Yuan Li, Yuan Yuan, Chunjie Wang, Ning Lang, Shuai Tian, Meiyi Yao, Xiaoguang Liu, Huishu Yuan, Shuqiang Jiang, Liang Jiang · 发表于：Frontiers in Oncology · 年份：2022 · DOI：10.3389/fonc.2022.814667 · 被引用次数：21 · 研究领域：Medical Imaging and Analysis、Management of metastatic bone disease、Brain Tumor Detection and Classification

Background: Recently, the Turing test has been used to investigate whether machines have intelligence similar to humans. Our study aimed to assess the ability of an artificial intelligence (AI) system for spine tumor detection using the Turing test. Methods: Our retrospective study data included 12179 images from 321 patients for developing AI detection systems and 6635 images from 187 patients for the Turing test. We utilized a deep learning-based tumor detection system with Faster R-CNN architecture, which generates region proposals by Region Proposal Network in the first stage and corrects the position and the size of the bounding box of the lesion area in the second stage. Each choice question featured four bounding boxes enclosing an identical tumor. Three were detected by the proposed deep learning model, whereas the other was annotated by a doctor; the results were shown to six doctors as respondents. If the respondent did not correctly identify the image annotated by a human, his answer was considered a misclassification. If all misclassification rates were >30%, the respondents were considered unable to distinguish the AI-detected tumor from the human-annotated one, which indicated that the AI system passed the Turing test. Results: The average misclassification rates in the Turing test were 51.2% (95% CI: 45.7%-57.5%) in the axial view (maximum of 62%, minimum of 44%) and 44.5% (95% CI: 38.2%-51.8%) in the sagittal view (maximum of 59%, minimum of 36%). The misclass...