How accurate are ChatGPT-4 responses in chronic urticaria? A critical analysis with information quality metrics

作者：Iván Chérrez-Ojeda, Marco Faytong‐Haro, Patricio Álvarez-Muñoz, José Ignacio Larco, Erika De Arruda Chaves, Isabel Rojo, Carol Vivian Moncayo, German D. Ramón, Gabriela Rodas‐Valero, Emek Kocatürk, Giselle Mosnaim, Karla Robles‐Velasco · 发表于：World Allergy Organization Journal · 年份：2025 · DOI：10.1016/j.waojou.2025.101071 · 被引用次数：7 · 研究领域：Urticaria and Related Conditions、Asthma and respiratory diseases、Drug-Induced Adverse Reactions

Background: The increasing use of artificial intelligence (AI) in healthcare, especially in delivering medical information, prompts concerns over the reliability and accuracy of AI-generated responses. This study evaluates the quality, reliability, and readability of ChatGPT-4 responses for chronic urticaria (CU) care, considering the potential implications of inaccurate medical information. Objective: The goal of the study was to assess the quality, reliability, and readability of ChatGPT-4 responses to inquiries on CU management in accordance with international guidelines, utilizing validated metrics to evaluate the effectiveness of ChatGPT-4 as a resource for medical information acquisition. Methods: LEN/EuroGuiDerm/APAAACI recommendations and utilized as prompts for ChatGPT-4 to obtain responses in individual chats for each question. The inquiries were categorized into 3 groups: A.) Classification and Diagnosis, B.) Assessment and Monitoring, and C.) Treatment and Management Recommendations. The responses were separately evaluated by allergy specialists utilizing the DISCERN instrument for quality assessment, Journal of the American Medical Association (JAMA) benchmark criteria for reliability evaluation, and Flesch scores for readability analysis. The scores were further examined by median calculations and Intraclass Correlation Coefficient assessments. Results: Categories A and C exhibited insufficient reliability according to JAMA, with median scores of 1 and 0, respec...