Split Learning Over Wireless Networks: Parallel Design and Resource Management

作者：Wen Wu, Mushu Li, Kaige Qu, Conghao Zhou, Xuemin Shen, Weihua Zhuang, Xu Li, Weisen Shi · 发表于：IEEE Journal on Selected Areas in Communications · 年份：2023 · DOI：10.1109/jsac.2023.3242704 · 被引用次数：278 · 研究领域：Privacy-Preserving Technologies in Data、Advanced MIMO Systems Optimization、Indoor and Outdoor Localization Technologies

Split learning (SL) is a collaborative learning framework, which can train an artificial intelligence (AI) model between a device and an edge server by splitting the AI model into a device-side model and a server-side model at a cut layer. The existing SL approach conducts the training process sequentially across devices, which incurs significant training latency especially when the number of devices is large. In this paper, we design a novel SL scheme to reduce the training latency, namedCluster-basedParallelSL(CPSL) which conducts model training in a “first-parallel-then-sequential” manner. Specifically, the CPSL is to partition devices into several clusters, parallelly train device-side models in each cluster and aggregate them, and then sequentially train the whole AI model across clusters, thereby parallelizing the training process and reducing training latency. Furthermore, we propose a resource management algorithm to minimize the training latency of CPSL considering device heterogeneity and network dynamics in wireless networks. This is achieved by stochastically optimizing the cut layer selection, device clustering, and radio spectrum allocation. The proposed two-timescale algorithm can jointly make the cut layer selection decision in a large timescale and device clustering and radio spectrum allocation decisions in a small timescale. Extensive simulation results on non-independent and identically distributed data demonstrate that the proposed solution can greatly re...