Sökning: onr:"swepub:oai:DiVA.org:kth-325693" >
Adaptive Stochastic...
Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge IoT
-
- Lei, Wanlu (författare)
- KTH,Teknisk informationsvetenskap,Interconnection Design in Baseband and Interconnect Department, Ericsson AB, Stockholm, Sweden
-
- Ye, Yu (författare)
- KTH,Teknisk informationsvetenskap
-
- Xiao, Ming, 1975- (författare)
- KTH,Teknisk informationsvetenskap
-
visa fler...
-
- Skoglund, Mikael, 1969- (författare)
- KTH,Teknisk informationsvetenskap
-
Han, Z. (författare)
-
visa färre...
-
(creator_code:org_t)
- Institute of Electrical and Electronics Engineers (IEEE), 2022
- 2022
- Engelska.
-
Ingår i: IEEE Internet of Things Journal. - : Institute of Electrical and Electronics Engineers (IEEE). - 2327-4662. ; 9:22, s. 22958-22971
- Relaterad länk:
-
https://urn.kb.se/re...
-
visa fler...
-
https://doi.org/10.1...
-
visa färre...
Abstract
Ämnesord
Stäng
- Edge computing provides a promising paradigm to support the implementation of Internet of Things (IoT) by offloading tasks to nearby edge nodes. Meanwhile, the increasing network size makes it impractical for centralized data processing due to limited bandwidth, and consequently a decentralized learning scheme is preferable. Reinforcement learning (RL) has been widely investigated and shown to be a promising solution for decision-making and optimal control processes. For RL in a decentralized setup, edge nodes (agents) connected through a communication network aim to work collaboratively to find a policy to optimize the global reward as the sum of local rewards. However, communication costs, scalability, and adaptation in complex environments with heterogeneous agents may significantly limit the performance of decentralized RL. Alternating direction method of multipliers (ADMM) has a structure that allows for decentralized implementation and has shown faster convergence than gradient descent-based methods. Therefore, we propose an adaptive stochastic incremental ADMM (asI-ADMM) algorithm and apply the asI-ADMM to decentralized RL with edge-computing-empowered IoT networks. We provide convergence properties for the proposed algorithms by designing a Lyapunov function and prove that the asI-ADMM has O(1/k) + O(1/M) convergence rate, where k and M are the number of iterations and batch samples, respectively. Then, we test our algorithm with two supervised learning problems. For performance evaluation, we simulate two applications in decentralized RL settings with homogeneous and heterogeneous agents. The experimental results show that our proposed algorithms outperform the state of the art in terms of communication costs and scalability and can well adapt to complex IoT environments.
Ämnesord
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Reglerteknik (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Control Engineering (hsv//eng)
Nyckelord
- Communication efficiency
- decentralized edge computing
- reinforcement learning (RL)
- stochastic alternating direction method of multiplier (ADMM)
- Complex networks
- Data handling
- Decision making
- Edge computing
- Gradient methods
- Internet of things
- Job analysis
- Lyapunov functions
- Random processes
- Reinforcement learning
- Scalability
- Stochastic systems
- Alternating directions method of multipliers
- Convergence
- Decentralised
- Optimisations
- Reinforcement learnings
- Stochastic alternating direction method of multiplier
- Stochastics
- Task analysis
- Optimization
Publikations- och innehållstyp
- ref (ämneskategori)
- art (ämneskategori)
Hitta via bibliotek
Till lärosätets databas