Integrating AI into clinical education: evaluating general practice trainees’ proficiency in distinguishing AI-generated hallucinations and impacting factors

作者：Jiacheng Zhou, Jintao Zhang, Rongrong Wan, Xiaochuan Cui, Q. Liu, Hua Guo, Xiaofen Shi, Bingbing Fu, Jia Meng, Bo Yue, Yunyun Zhang, Zhiyong Zhang · 发表于：BMC Medical Education · 年份：2025 · DOI：10.1186/s12909-025-06916-2 · 被引用次数：32 · 研究领域：Artificial Intelligence in Healthcare and Education、Simulation-Based Education in Healthcare、Clinical Reasoning and Diagnostic Skills

OBJECTIVE: To assess the ability of General Practice (GP) Trainees to detect AI-generated hallucinations in simulated clinical practice, ChatGPT-4o was utilized. The hallucinations were categorized into three types based on the accuracy of the answers and explanations: (1) correct answers with incorrect or flawed explanations, (2) incorrect answers with explanations that contradict factual evidence, and (3) incorrect answers with correct explanations. METHODS: This multi-center, cross-sectional survey study involved 142 GP Trainees, all of whom were undergoing General Practice Specialist Training and volunteered to participate. The study evaluated the accuracy and consistency of ChatGPT-4o, as well as the Trainees' response time, accuracy, sensitivity (d'), and response tendencies (β). Binary regression analysis was used to explore factors affecting the Trainees' ability to identify errors generated by ChatGPT-4o. RESULTS: A total of 137 participants were included, with a mean age of 25.93 years. Half of the participants were unfamiliar with AI, and 35.0% had never used it. ChatGPT-4o's overall accuracy was 80.8%, which slightly decreased to 80.1% after human verification. However, the accuracy for professional practice (Subject 4) was only 57.0%, and after human verification, it dropped further to 44.2%. A total of 87 AI-generated hallucinations were identified, primarily occurring at the application and evaluation levels. The mean accuracy of detecting these hallucinations ...