Accuracy of Large Language Models to Identify Stroke Subtypes Within Unstructured Electronic Health Record Data

作者：Dylan Owens, Danh Q. Nguyen, Michael Dohopolski, Justin F. Rousseau, Eric D. Peterson, Ann Marie Návar · 发表于：Stroke · 年份：2025 · DOI：10.1161/strokeaha.125.051993 · 被引用次数：15 · 研究领域：Acute Ischemic Stroke Management、Machine Learning in Healthcare、Artificial Intelligence in Healthcare and Education

BACKGROUND: While International Classification of Diseases, Tenth Revision codes suffice for identifying stroke events in surveillance, accurately classifying stroke types and subtypes using electronic health records remains challenging due to limitations in structured data. This often necessitates manual review of clinical documentation. This study evaluated whether a large language model, Generative Pre-Trained Transformer 4 Omni (GPT-4o), can accurately identify stroke types and subtypes from unstructured clinical notes. METHODS: We implemented a retrieval-augmented generation framework with GPT-4o to classify stroke types (ischemic versus hemorrhagic) and ischemic stroke subtypes using electronic health records data. The American Heart Association Get With The Guidelines–Stroke registry served as the gold standard. Model development used a 20% subset of Get With The Guidelines–Stroke–linked data from UT Southwestern Medical Center (UTSW), with the remaining 80% reserved for testing. External validation used data from the Parkland Health and Hospital System (PHHS). A total of 4123 stroke hospitalizations from January 2019 to August 2023 were included (UTSW: n=2047; PHHS: n=2076). Three prompting strategies—zero-shot chain-of-thought, expert-guided, and instruction-based—were evaluated. Predictions of GPT-4os were compared with classifications made by trained abstractors contributing to the Get With The Guidelines–Stroke registry. RESULTS: In the external validation set, 79...