| 1. |
- Ekblom, Ebba, et al.
(författare)
-
EFFGAN: Ensembles of fine-tuned federated GANs
- 2022
-
Konferensbidrag (refereegranskat)abstract
- Decentralized machine learning tackles the problemof learning useful models when data is distributed amongseveral clients. The most prevalent decentralized setting todayis federated learning (FL), where a central server orchestratesthe learning among clients. In this work, we contribute to therelatively understudied sub-field of generative modelling in theFL framework.We study the task of how to train generative adversarial net-works (GANs) when training data is heterogeneously distributed(non-iid) over clients and cannot be shared. Our objective isto train a generator that is able to sample from the collectivedata distribution centrally, while the client data never leaves theclients and user privacy is respected. We show using standardbenchmark image datasets that existing approaches fail in thissetting, experiencing so-called client drift when the local numberof epochs becomes to large and local parameters drift too faraway in parameter space. To tackle this challenge, we proposea novel approach namedEFFGAN: Ensembles of fine-tunedfederated GANs. Being an ensemble of local expert generators, EFFGAN is able to learn the data distribution over all clientsand mitigate client drift. It is able to train with a large numberof local epochs, making it more communication efficient thanprevious works
|
|
| 2. |
- Listo Zec, Edvin, et al.
(författare)
-
Decentralized Adaptive Clustering of Deep Nets is Beneficial for Client Collaboration
- 2023
-
Ingår i: FL 2022. - Cham : Springer Nature. ; , s. 59-71
-
Konferensbidrag (refereegranskat)abstract
- We study the problem of training personalized deep learning models in a decentralized peer-to-peer setting, focusing on the setting where data distributions differ between the clients and where different clients have different local learning tasks. We study both covariate and label shift, and our contribution is an algorithm which for each client finds beneficial collaborations based on a similarity estimate for the local task. Our method does not rely on hyperparameters which are hard to estimate, such as the number of client clusters, but rather continuously adapts to the network topology using soft cluster assignment based on a novel adaptive gossip algorithm. We test the proposed method in various settings where data is not independent and identically distributed among the clients. The experimental evaluation shows that the proposed method performs better than previous state-of-the-art algorithms for this problem setting, and handles situations well where previous methods fail.
|
|
| 3. |
- Zec, Edvin Listo, et al.
(författare)
-
Decentralized adaptive clustering of deep nets is beneficial for client collaboration
- 2022
-
Konferensbidrag (refereegranskat)abstract
- We study the problem of training personalized deep learning models in a decentralized peer-to-peer setting, focusing on the setting where data distributions differ between the clients and where different clients have different local learning tasks. We study both covariate and label shift, and our contribution is an algorithm which for each client finds beneficial collaborations based on a similarity estimate for the local task. Our method does not rely on hyperparameters which are hard to estimate, such as the number of client clusters, but rather continuously adapts to the network topology using soft cluster assignment based on a novel adaptive gossip algorithm. We test the proposed method in various settings where data is not independent and identically distributed among the clients. The experimental evaluation shows that the proposed method performs better than previous state-of-the-art algorithms for this problem setting, and handles situations well where previous methods fail
|
|
