Trustworthy AI for Medicine: Continuous Hallucination Detection and Elimination with CHECK

作者：Carlos Garcia-Fernandez, Luis Felipe, Monique Shotande, M. Zitu, A. Tripathi, Ghulam Rasool, I. E. El Naqa, Vivek Rudrapatna, Gilmer Valdes · 发表于：arXiv.org · 年份：2025 · DOI：10.48550/arxiv.2506.11129 · 被引用次数：9 · 研究领域：Computer Science

Large language models (LLMs) show promise in healthcare, but hallucinations remain a major barrier to clinical use. We present CHECK, a continuous-learning framework that integrates structured clinical databases with a classifier grounded in information theory to detect both factual and reasoning-based hallucinations. Evaluated on 1500 questions from 100 pivotal clinical trials, CHECK reduced LLama3.3-70B-Instruct hallucination rates from 31% to 0.3% - making an open source model state of the art. Its classifier generalized across medical benchmarks, achieving AUCs of 0.95-0.96, including on the MedQA (USMLE) benchmark and HealthBench realistic multi-turn medical questioning. By leveraging hallucination probabilities to guide GPT-4o's refinement and judiciously escalate compute, CHECK boosted its USMLE passing rate by 5 percentage points, achieving a state-of-the-art 92.1%. By suppressing hallucinations below accepted clinical error thresholds, CHECK offers a scalable foundation for safe LLM deployment in medicine and other high-stakes domains.