Towards Fast-Convergent Federated Learning with non-IID Data

In order to maintain privacy-sensitive data and to facilitate collaborative machine learning (ML) among distributed nodes, Federated Learning (FL) has emerged as an attractive paradigm, where local nodes collaboratively train a task model under the orchestration of a central server without accessing end-user data. However, the non-independentand-identically-distributed (non-IID) data samples across participating nodes slow model training and impose additional

communication rounds for FL to converge. In this talk, I will present our recent efforts in addressing this issue, aiming to accelerate model convergence under the presence of nodes with non-IID datasets. Firstly, we propose an adaptive weighting strategy that assigns weight proportional to node contribution instead of according to the size of local datasets. It can reinforce positive (suppress negative) node contribution dynamically, leading to a significant communication round reduction. Secondly, we design a probabilistic node selection scheme that can preferentially select nodes to boost model convergence of FL with non-IID datasets. The proposed scheme adjusts the probability for each node to be selected in each round based on measuring the relationship between the local gradient and the global gradient from participating nodes. The superiority of the proposed approaches over the commonly adopted Federated Averaging (FedAvg) algorithm has been verified by extensive experimental results.