Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model

作者：Alexandra Sasha Luccioni, Sylvain Viguier, Anne‐Laure Ligozat · 发表于：arXiv (Cornell University) · 年份：2022 · DOI：10.48550/arxiv.2211.02001 · 被引用次数：195 · 研究领域：Green IT and Sustainability、Machine Learning in Materials Science

Progress in machine learning (ML) comes with a cost to the environment, given that training ML models requires significant computational resources, energy and materials. In the present article, we aim to quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes of~\carboneq~if we consider only the dynamic power consumption, and 50.5 tonnes if we account for all processes ranging from equipment manufacturing to energy-based operational consumption. We also study the energy requirements and carbon emissions of its deployment for inference via an API endpoint receiving user queries in real-time. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of ML models and future research directions that can contribute towards improving carbon emissions reporting.