Comparative analysis of algorithmic approaches in ensemble learning: bagging vs. boosting

作者：Hongke Zhao, Wenhui Liu, Yaxian Wang, Likang Wu · 发表于：Scientific Reports · 年份：2025 · DOI：10.1038/s41598-025-15971-0 · 被引用次数：5 · 研究领域：Data Mining Algorithms and Applications、Machine Learning and Data Classification、Imbalanced Data Classification Techniques

Ensemble learning is widely applied in various real-world settings, with Bagging and Boosting being two core algorithms. Although these techniques have been extensively investigated through experimental comparisons of their performance in various scenarios, few studies have analyzed and quantified their benefits, costs, and complexities to support algorithm-aware decision making. In this study, we develop a theoretical model to compare Bagging and Boosting in terms of performance, computational costs, and ensemble complexity, and validate it through experiments on four datasets (MNIST, CIFAR-10, CIFAR-100, IMDB) with varying data complexity and computational environments. The results show that, for MNIST, as ensemble complexity increases (e.g., from 20 to 200), Bagging's performance improves from 0.932 to 0.933 before plateauing, while Boosting improves from 0.930 to 0.961 before showing signs of overfitting. At the same ensemble complexity, such as 200 base learners, Boosting requires approximately 14 times more computational time than Bagging, indicating substantially higher computational costs. Similar patterns are observed across the other three datasets, confirming the generality of our findings and revealing consistent trade-offs between performance and computational costs. Taken together, these results confirm the robustness of our theoretical predictions and provide a foundation for practical guidance. Specifically, decision-makers prioritizing cost-efficiency may pre...