REFLECTIVE-TIAB: cost-effective prompt optimization for large language model-based title and abstract screening in literature reviews

作者：Á. Jóźwiak, Attila Imre, Judit Hagymásy, Judit Tittmann, A. Nagy, Sándor Kovács, Przemysław Kardas, Job F. M. van Boven, I. Mommers, T. Ágh · 发表于：Expert review of pharmacoeconomics & outcomes research · 年份：2026 · DOI：10.1080/14737167.2026.2691995 · 研究领域：Medicine

ABSTRACT Background Title and abstract screening is a labor-intensive stage of systematic reviews. Large language models (LLMs) can automate this process, but performance depends heavily on prompt design and model selection, which is typically manual and time-consuming. Our objective was to evaluate whether automated, reflection-driven prompt optimization improves LLM performance during title and abstract screening. Research design and methods REFLECTIVE-TIAB uses the GEPA reflective prompt optimizer to improve prompts under an asymmetric loss penalizing false negatives. Nine LLMs screened 8,520 de-duplicated records from a COPD exacerbation predictor search. A 100-abstract gold standard was constructed from inter-model disagreements and was expert-labeled. The prompt was optimized on Llama 3.3 70B via DSPy/GEPA and evaluated across all nine models. Results Optimization improved recall across all LLMs (+3.7% to +37.1%). Gemini 3 Flash Preview achieved the highest performance (91% accuracy, F1 81.6%) while costing 25-fold less per abstract than GPT-5.2, which ranked among the lowest-performing models. A prompt optimized on a single open-source model is generalized to all nine without retraining. Total optimization cost was $6.36. Conclusions REFLECTIVE-TIAB provides automated, model-transferable prompt optimization for literature screening at negligible cost. Model price did not predict screening performance. The framework could substantially reduce screening workload while pr...