Large language models management of complex medication regimens: a case-based evaluation

作者：Aaron Chase, Amoreena Most, Shaochen Xu, Erin Barreto, Brian Murray, Kelli Henry, Susan Smith, Tanner Hedrick, Xianyan Chen, Sheng Li, Tianming Liu, Andrea Sikora · 发表于：Frontiers in Pharmacology · 年份：2025 · DOI：10.3389/fphar.2025.1514445 · 被引用次数：7 · 研究领域：Topic Modeling、Natural Language Processing Techniques、Machine Learning in Healthcare

Background: Large language models (LLMs) have shown the ability to diagnose complex medical cases, but only limited studies have evaluated the performance of LLMs in the development of evidence-based treatment plans. The purpose of this evaluation was to test four LLMs on their ability to develop safe and efficacious treatment plans on complex patients managed in the intensive care unit (ICU). Methods: Eight high-fidelity patient cases focusing on medication management were developed by critical care clinicians including history of present illness, laboratory values, vital signs, home medications, and current medications. Four LLMs [ChatGPT (GPT-3.5), ChatGPT (GPT-4), Claude-2, and Llama-2-70b] were prompted to develop an optimized medication regimen for each case. LLM generated medication regimens were then reviewed by a panel of seven critical care clinicians to assess safety and efficacy, as defined by medication errors identified and appropriate treatment for the clinical conditions. Appropriate treatment was measured by the average rate of clinician agreement to continue each medication in the regimen and compared using analysis of variance (ANOVA). Results: Clinicians identified a median of 4.1-6.9 medication errors per recommended regimen, and life-threatening medication recommendations were present in 16.3%-57.1% of the regimens, depending on LLM. Clinicians continued LLM-recommended medications at a rate of 54.6%-67.3%, with GPT-4 having the highest rate of medicatio...