Facilitating critical appraisal of evidence using automation tools

作者：Alexandra Bannach‐Brown, Nils Hijlkema, Niels van Beuningen, Wynand Alkema, Carlijn R. Hooijmans, Kimberley E. Wever, Malcolm Macleod, Olena Maksym, Tomáš Novotný, Lorenzo Niccolai, Vieri Emiliani · 发表于：EFSA Supporting Publications · 年份：2025 · DOI：10.2903/sp.efsa.2025.en-9515 · 研究领域：Explainable Artificial Intelligence (XAI)、Artificial Intelligence in Healthcare and Education、Meta-analysis and systematic reviews

Abstract The European Food Safety Authority (EFSA) initiated the “Roadmap for actions on Artificial Intelligence for evidence management in risk assessment” project in May 2021, aiming to optimize risk assessment processes by integrating Artificial Intelligence (AI) methodologies. A key focus is the automation of the Risk of Bias (RoB) assessment, a critical and time‐consuming step in systematic reviews that evaluates the methodological quality of studies. Implemented under Framework Contract OC/EFSA/AMU/2021/03 ‐ Specific Contract 3, this project built and piloted a prototype automation tool to support the risk of bias assessment process. This report details the steps taken to define current EFSA processes, select appropriate critical appraisal tools and use cases, and develop and validate a prototype automation tool based on a GPT model with zero‐shot learning (AutoCAT). The implementation phase includes integrating AutoCAT with existing systematic review platforms, and deployment within the EFSA environment. The assessment phase involved evaluating the tool with development and system test datasets, and user testing with EFSA scientific experts. The project's outcomes are discussed, highlighting the limitation of the pilot prototype and proposing future work and potential applications in other contexts.