U-SplitDoRA: an improved privacy-preserved U-shaped split parameter-efficient fine-tuning framework through weight decomposition for large language models

作者：Samar Singh, Brindha Subburaj, R. Alagewaran, Sherly Alphonse · 发表于：Frontiers in Artificial Intelligence · 年份：2026 · DOI：10.3389/frai.2026.1807960 · 研究领域：Medicine

As large language models (LLMs) are getting bigger with respect to the parameter count, ranging from a few million to billions, methods like parameter-efficient fine-tuning (PEFT) have emerged as a crucial approach for adapting these LLMs, such as GPT, Llama, and DeepSeek, to resource-constrained and privacy-sensitive environments. The robustness of large language models (LLMs) while operating on complex tasks and with large datasets makes them feasible for various application domains. This also demands the availability of more public datasets to train LLMs in the future. The federated learning (FL) technique, where several entities collaboratively train a machine learning model without sharing their data, is a widely adopted decentralized training framework. This is followed by a central server, which aggregates the models to create a global model. FL LLM fine-tuning has gained attention recently to overcome the aforementioned training data scarcity issue. LLMs are collaboratively fine-tuned by several data owners without disclosing their private data. The large number of trainable parameters has a direct effect on training such complex models on the client side. The split learning technique, through model partitioning, solves the training overhead by offloading certain training tasks to the server side. Previous research based on the split learning approach for FL LLM fine-tuning, namely SplitLoRA and HSpliLoRA, sets the foundation for further research in this direction. Fr...