ChatGPT augmented clinical trial screening

作者：Jacob Beattie, Dylan Owens, Ann Marie Návar, Luiza Giuliani Schmitt, Kimberly Taing, Sarah Neufeld, Daniel X. Yang, Christian Chukwuma, Ahmed Gul, Dong Soo Lee, Neil Desai, Dominic H. Moon, Jing Wang, Steve Jiang, Michael Dohopolski · 发表于：Machine Learning Health · 年份：2025 · DOI：10.1088/3049-477x/adbd47 · 被引用次数：2 · 研究领域：Artificial Intelligence in Healthcare and Education、Ethics in Clinical Research、Biomedical and Engineering Education

Manual clinical trial screening is costly and inefficient. Structured EHR data often fail to adequately capture trial criteria, but large language models (LLMs) like GPT-3.5 and GPT-4 offer a potential solution by analyzing unstructured EHR text. Optimal deployment strategies remain unclear. We evaluated GPT-3.5 and GPT-4 in screening 74 patients (35 eligible, 39 ineligible) for a phase 2 head and neck cancer trial. EHR data included progress notes, pathology, and imaging reports. Fourteen trial criteria (e.g. stage, histology, prior treatments) were manually annotated as the ground truth. Three prompting methods were tested: structured output (SO), chain of thought (CoT), and self-discover (SD), with further optimization by expert (EG) or LLM guidance (LLM-G). Prompts were refined on 20 patients and validated on 54. Performance metrics included accuracy, sensitivity, specificity, and micro F 1 score. Strict eligibility required all criteria to be met, while proportional eligibility flagged patients based on criteria proportions. Screening time and cost were also analyzed. GPT-3.5 achieved a median accuracy of 0.761, with SO + EG performing best. GPT-4 had higher accuracy (0.838), with SD achieving the highest Youden index (0.729). For strict eligibility, GPT-4’s CoT + EG approach had the highest accuracy (0.65). Proportional eligibility performed better overall, with GPT-4’s CoT + LLM-G achieving the highest AUC (0.82). Screening time per patient ranged from 1.4–12.4 min, an...