Adaptive Expert Models for Personalization in Federated Learning

Isaksson, MartinKTH,Programvaruteknik och datorsystem, SCS,Ericsson Research,GALE (author)

QC 20220628
Federated Learning (FL) is a promising framework for distributed learning whendata is private and sensitive. However, the state-of-the-art solutions in thisframework are not optimal when data is heterogeneous and non-Independent andIdentically Distributed (non-IID). We propose a practical and robust approachto personalization in FL that adjusts to heterogeneous and non-IID data bybalancing exploration and exploitation of several global models. To achieve ouraim of personalization, we use a Mixture of Experts (MoE) that learns to groupclients that are similar to each other, while using the global models moreefficiently. We show that our approach achieves an accuracy up to 29.78 % andup to 4.38 % better compared to a local model in a pathological non-IIDsetting, even though we tune our approach in the IID setting.

Listo Zec, EdvinKTH,Programvaruteknik och datorsystem, SCS,RISE,GALE(Swepub:kth)u1dmmynf (author)
Cöster, RickardEricsson AB (author)
Daniel, Gillblad5Chalmers AI Research Center, Chalmers University of Technology, Göteborg, Sweden; AI Sweden (author)
Girdzijauskas, SarunasKTH,Programvaruteknik och datorsystem, SCS(Swepub:kth)u1k70r02 (author)
KTHProgramvaruteknik och datorsystem, SCS (creator_code:org_t)

In:International Workshop on Trustworthy Federated Learningin Conjunction with IJCAI 2022 (FL-IJCAI'22)

By the author/editor: Isaksson, Martin; Listo Zec, Edvin; Cöster, Rickard; Daniel, Gillblad; Girdzijauskas, S ...

About the subject

ENGINEERING AND TECHNOLOGY: ENGINEERING AND ...; and Electrical Engin ...; and Computer Systems

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.