Semi-decentralized federated learning with client pairing for efficient mutual knowledge transfer

초록

In Decentralized Federated Learning (DFL), Deep Mutual Learning (DML) improves global accuracy under non-independent and identically distributed (non-IID) data by enabling knowledge exchange over clients, but introduces extra training overhead and delays convergence. To solve this issue, we propose DKT-CP (Coordinator-assisted Decentralized Federated Learning with Client Pairing for Efficient Mutual Knowledge Transfer), a novel DFL framework that dynamically pairs clients with the most divergent data distributions to enhance the effectiveness of DML. A lightweight coordinator calculates a Kullback-Leibler divergence (KLD) matrix in the first round using client data distribution information, reducing computational overhead. Accordingly, to enable dynamic client pairing, DKT-CP adopts a two-step strategy: for each selected local update client, the coordinator first identifies a subset of the most dissimilar clients based on the KLD matrix, then randomly selects one from this set as the DML partner. This approach ensures that clients are matched with highly dissimilar peers, maximizing knowledge transfer, while also encouraging exploration and ensuring fairness by preventing the same dissimilar pairs from being selected repeatedly across rounds. Experimental results demonstrate that the proposed algorithm outperforms existing DML-based and averaging-based algorithms, achieving on average 29% higher global accuracy and 40% higher F1-score compared to baseline methods in highly Non-IID environments.

키워드