Private Stochastic Dual Averaging for Decentralized Empirical Risk Minimization

QC 20220929
In this work, we study the decentralized empirical risk minimization problem under the constraint of differential privacy (DP). Based on the algorithmic framework of dual averaging, we develop a novel decentralized stochastic optimization algorithm to solve the problem. The proposed algorithm features the following: i) it perturbs the stochastic subgradient evaluated over individual data samples, with which the information about the dataset can be released in a differentially private manner; ii) it employs hyperparameters that are more aggressive than conventional decentralized dual averaging algorithms to speed up convergence. The upper bound for the utility loss of the proposed algorithm is proven to be smaller than that of existing methods to achieve the same level of DP. As a by-product, when removing the perturbation, the non-private version of the proposed algorithm attains the optimal O(1/t) convergence rate for smooth stochastic optimization. Finally, experimental results are presented to demonstrate the effectiveness of the algorithm.

Johansson, Karl H.,1967-KTH,Reglerteknik,Digital Futures(Swepub:kth)u1h0d9k2 (författare)
Shi, YangUniv Victoria, Dept Mech Engn, Victoria, BC V8W 3P6, Canada. (författare)
KTHReglerteknik (creator_code:org_t)

Ingår i:9th IFAC Conference on Networked Systems NECSYS 2022Zürich, Switzerland, 5–7 July 2022: Elsevier BV, s. 43-48

Om ämnet

TEKNIK OCH TEKNOLOGIER: TEKNIK OCH TEKNO ...; och Annan teknik; och Övrig annan tekn ...

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.