Exploring the capacities of ChatGPT : A comprehensive evaluation of its accuracy and repeatability in addressing helicobacter pylori ‐related queries

作者：Yongkang Lai, Foqiang Liao, Jiulong Zhao, Chunping Zhu, Yi Hu, Zhaoshen Li · 发表于：Helicobacter · 年份：2024 · DOI：10.1111/hel.13078 · 被引用次数：26 · 研究领域：Artificial Intelligence in Healthcare and Education、Clinical Reasoning and Diagnostic Skills、Meta-analysis and systematic reviews

BACKGROUND: Educational initiatives on Helicobacter pylori (H. pylori) constitute a highly effective approach for preventing its infection and establishing standardized protocols for its eradication. ChatGPT, a large language model, is a potentially patient-friendly online tool capable of providing health-related knowledge. This study aims to assess the accuracy and repeatability of ChatGPT in responding to questions related to H. pylori. MATERIALS AND METHODS: Twenty-one common questions about H. pylori were collected and categorized into four domains: basic knowledge, diagnosis, treatment, and prevention. ChatGPT was utilized to individually answer the aforementioned 21 questions. Its responses were independently assessed by two experts on H. pylori. Questions with divergent ratings were resolved by a third reviewer. Cohen's kappa coefficient was calculated to assess the consistency between the scores of the two reviewers. RESULTS: The responses of ChatGPT on H. pylori-related questions were generally satisfactory, with 61.9% marked as "completely correct" and 33.33% as "correct but inadequate." The repeatability of the responses of ChatGPT to H. pylori-related questions was 95.23%. Among the responses, those related to prevention (comprehensive: 75%) had the best response, followed by those on treatment (comprehensive: 66.7%), basic knowledge (comprehensive: 60%), and diagnosis (comprehensive: 50%). In the "treatment" domain, 16.6% of the ChatGPT responses were categorized...