FedBRB: A Solution to the Small-to-Large Scenario in Device-Heterogeneity Federated Learning

作者：Tianchi Liao, Ziyue Xu, Qing Hu, Hong‐Ning Dai, Huawei Huang, Zibin Zheng, Chuan Chen · 发表于：IEEE Transactions on Mobile Computing · 年份：2025 · DOI：10.1109/tmc.2025.3610985 · 被引用次数：1 · 研究领域：Privacy-Preserving Technologies in Data、Stochastic Gradient Optimization Techniques、Brain Tumor Detection and Classification

Recently, the success of large models has demonstrated the importance of scaling up model sizes. However, it is difficult to directly train large models locally on multiple mobile devices due to their intrinsic computational constraints. To address this challenge, it becomes a crucial need to train larger global models by training small local models on devices. As a distributed learning approach, federated learning (FL) allows multiple devices to train models locally and aggregate them to form the global model by sharing the updated parameters with the server, thus enabling the co-training of models. This promising feature has spurred an increasing interest in exploring the collaborative training of large models. Despite the advent of existing device-heterogeneity FL approaches, they still have limitations in fully covering the parameter space of the global model. To fill this gap, we propose a novel approach calledFedBRB(Block- wiseRolling and weightedBroadcast). The core idea of FedBRB is to utilize local models of small devices to train all modules of a large global model and broadcast the trained parameters to the entire space, thereby enabling faster information sharing. This approach not only improves training efficiency but also fully utilizes limited computational resources. Experiments demonstrate that FedBRB can produce significant performance gains, achieving state-of-the-art results. Additionally, this paper provides theoretical and experimental analyses of FedBRB...