AI-driven evidence synthesis: data extraction of randomized controlled trials with large language models

作者：Jiayi Liu, Honghao Lai, Weilong Zhao, J. Huang, Danni Xia, Hui Liu, Xufei Luo, Bingyi Wang, Bei Pan, Liangying Hou, Yaolong Chen, Long Ge · 发表于：International Journal of Surgery · 年份：2025 · DOI：10.1097/js9.0000000000002215 · 被引用次数：12 · 研究领域：Artificial Intelligence in Healthcare and Education、Topic Modeling、Meta-analysis and systematic reviews

The advancement of large language models (LLMs) presents promising opportunities to enhance evidence synthesis efficiency, particularly in data extraction processes, yet existing prompts for data extraction remain limited, focusing primarily on commonly used items without accommodating diverse extraction needs. This research letter developed structured prompts for LLMs and evaluated their feasibility in extracting data from randomized controlled trials (RCTs). Using Claude (Claude-2) as the platform, we designed comprehensive structured prompts comprising 58 items across six Cochrane Handbook domains and tested them on 10 randomly selected RCTs from published Cochrane reviews. The results demonstrated high accuracy with an overall correct rate of 94.77% (95% CI: 93.66% to 95.73%), with domain-specific performance ranging from 77.97% to 100%. The extraction process proved efficient, requiring only 88 seconds per RCT. These findings substantiate the feasibility and potential value of LLMs in evidence synthesis when guided by structured prompts, marking a significant advancement in systematic review methodology.