ForTIFAI: fending off recursive training induced failure for AI model collapse

作者：Soheil Zibakhsh Shabgahi, Pedram Aghazadeh, Azalia Mirhoseini, Farinaz Koushanfar · 发表于：npj Artificial Intelligence · 年份：2026 · DOI：10.1038/s44387-026-00127-w · 研究领域：Generative Adversarial Networks and Image Synthesis、Machine Learning in Healthcare、Topic Modeling

Abstract The increasing reliance on generative AI models is rapidly increasing the volume of synthetic data, with projections suggesting that most available new data for training could be machine-generated by 2030 1 . This shift to a mainly synthetic data presents a critical challenge: repeated training in synthetic data leads to a phenomenon known as model collapse, where model performance degrades over generations of training, eventually rendering the models ineffective. While the causes of model collapse are increasingly understood, effective mitigation strategies remain scarce. We address this challenge by leveraging a key insight: auto-regressive models tend to generate text sequences to which they assign high confidence. Based on this observation, we introduce Confidence-Aware Loss (CAL) functions, proposing Truncated Cross-Entropy (TCE) as a key instance. CAL mitigates model collapse by down-weighing high-confidence tokens during training, effectively filtering out likely machine-generated artifacts from the learning process. Our experiments demonstrate that models trained with CAL not only learn effectively, but also exhibit significantly increased resilience, tolerating over 2.3 × more synthetic data before the onset of collapse. In addition, we provide an open-source benchmark for collapse dynamics in mixed-data settings. Our results demonstrate that confidence-aware training objectives can substantially delay collapse onset, offering a practical tool for model robu...