Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support

作者：Mahmud Omar, Vera Sorin, Jeremy D. Collins, David Emil Reich, Robert Freeman, Nicholas P. Gavin, Alexander W. Charney, Lisa S. Stump, Nicola Luigi Bragazzi, Girish N. Nadkarni, Eyal Klang · 发表于：Communications Medicine · 年份：2025 · DOI：10.1038/s43856-025-01021-3 · 被引用次数：178 · 研究领域：Adversarial Robustness in Machine Learning、Artificial Intelligence in Healthcare and Education、Machine Learning in Healthcare

Large language models (LLMs) show promise in clinical contexts but can generate false facts (often referred to as “hallucinations”). One subset of these errors arises from adversarial attacks, in which fabricated details embedded in prompts lead the model to produce or elaborate on the false information. We embedded fabricated content in clinical prompts to elicit adversarial hallucination attacks in multiple large language models. We quantified how often they elaborated on false details and tested whether a specialized mitigation prompt or altered temperature settings reduced errors. We created 300 physician-validated simulated vignettes, each containing one fabricated detail (a laboratory test, a physical or radiological sign, or a medical condition). Each vignette was presented in short and long versions—differing only in word count but identical in medical content. We tested six LLMs under three conditions: default (standard settings), mitigating prompt (designed to reduce hallucinations), and temperature 0 (deterministic output with maximum response certainty), generating 5,400 outputs. If a model elaborated on the fabricated detail, the case was classified as a “hallucination”. Hallucination rates range from 50 % to 82 % across models and prompting methods. Prompt-based mitigation lowers the overall hallucination rate (mean across all models) from 66 % to 44 % (p < 0.001). For the best-performing model, GPT-4o, rates decline from 53 % to 23 % (p < 0.001). Temperature ad...