AdaSplitLoRA: Adaptive Split Federated Learning for Efficient LLM Fine-Tuning in Wireless Networks

초록

This letter proposes Adaptive Split LoRA (AdaSplitLoRA), a framework that jointly optimizes adaptive low rank adaptor (LoRA) rank allocation and dynamic bandwidth allocation for communication-efficient split federated learning (SFL)-based large language model (LLM) fine-tuning. We formulate a joint per-round latency minimization problem over server-side LoRA ranks and uplink bandwidth, and decompose it into two independent subproblems. For server-side rank adaptation, we employ a gradient-based importance heuristic to address the inherent intractability of the discrete rank optimization. For uplink bandwidth allocation, we design a min-max straggler latency problem, prove its convexity, and obtain the global optimum efficiently using interior-point methods. Experimental results show that AdaSplitLoRA outperforms or remains competitive with fixed/adaptive-rank baselines while using fewer server-side adapter parameters and achieving favorable accuracy-latency tradeoffs in heterogeneous wireless SFL settings.

키워드