Client-Lightweight Soft Clustering Federated Learning

Abstract

Clustered federated learning is a highly effective approach to addressing the non-independent and identically distributed (Non-IID) problem in conventional federated learning. Soft clustering federated learning allows clients to belong to multiple clusters, which not only captures the common characteristics of clients within the same cluster but also fully adapts to the unique properties of each client’s local data. This significantly improves model personalization and generalization ability, and greatly enhances model performance and training efficiency in Non-IID scenarios. Recent advanced soft clustering federated learning methods no longer rely on client model parameters for clustering. Instead, they generate asymmetric similarities via privacy-preserving set intersection operations on clients’ local datasets, which are then sent to the server for clustering. This avoids the privacy risks associated with uploading model parameters from clients. In this work, we optimize this clustering paradigm by adopting locality-sensitive hashing and an improved privacy-preserving set intersection scheme. Our method substantially reduces the communication and computational overhead among clients while maintaining competitive model performance.