| 1. |
- Zhang, Mengfan, et al.
(författare)
-
Data Driven Decentralized Control of Inverter based Renewable Energy Sources using Safe Guaranteed Multi-Agent Deep Reinforcement Learning
- 2024
-
Ingår i: IEEE Transactions on Sustainable Energy. - 1949-3029 .- 1949-3037. ; 15:2, s. 1288-1299
-
Tidskriftsartikel (refereegranskat)abstract
- The wide integration of inverter based renewable energy sources (RESs) in modern grids may cause severe voltage violation issues due to high stochastic fluctuations of RESs. Existing centralized approaches can achieve optimal results for voltage regulation, but they have high communication burdens; existing decentralized methods only require local information, but they cannot achieve optimal results. Deep reinforcement learning (DRL) based methods are effective to deal with uncertainties, but it is difficult to guarantee secure constraints in existing DRL training. To address the above challenges, this paper proposes a projection embedded multi-agent DRL algorithm to achieve decentralized optimal control of distribution grids with guaranteed 100% safety. The safety of the DRL training is guaranteed via an embedded safe policy projection, which could smoothly and effectively restrict the DRL agent action space, and avoid any violation of physical constraints in distribution grid operations. The multi-agent implementation of the proposedalgorithm enables the optimal solution achieved in a decentralized manner that does not require real-time communication for practical deployment. The proposed method is tested in modified IEEE 33-bus distribution and compared with existing methods; the results validate the effectiveness of the proposed method in achieving decentralized optimal control with guaranteed 100% safety and without the requirement of real-time communications
|
|
| 2. |
- Zhang, Mengfan, et al.
(författare)
-
Data Driven Decentralized Control of Inverter Based Renewable Energy Sources Using Safe Guaranteed Multi-Agent Deep Reinforcement Learning
- 2024
-
Ingår i: IEEE Transactions on Sustainable Energy. - : Institute of Electrical and Electronics Engineers (IEEE). - 1949-3029 .- 1949-3037. ; 15:2, s. 1288-1299
-
Tidskriftsartikel (refereegranskat)abstract
- The wide integration of inverter based renewable energy sources (RESs) in modern grids may cause severe voltage violation issues due to high stochastic fluctuations of RESs. Existing centralized approaches can achieve optimal results for voltage regulation, but they have high communication burdens; existing decentralized methods only require local information, but they cannot achieve optimal results. Deep reinforcement learning (DRL) based methods are effective to deal with uncertainties, but it is difficult to guarantee secure constraints in existing DRL training. To address the above challenges, this paper proposes a projection embedded multi-agent DRL algorithm to achieve decentralized optimal control of distribution grids with guaranteed 100% safety. The safety of the DRL training is guaranteed via an embedded safe policy projection, which could smoothly and effectively restrict the DRL agent action space, and avoid any violation of physical constraints in distribution grid operations. The multi-agent implementation of the proposed algorithm enables the optimal solution achieved in a decentralized manner that does not require real-time communication for practical deployment. The proposed method is tested in modified IEEE 33-bus distribution and compared with existing methods; the results validate the effectiveness of the proposed method in achieving decentralized optimal control with guaranteed 100% safety and without the requirement of real-time communications.
|
|
| 3. |
- Zhang, Mengfan, et al.
(författare)
-
DNN Assisted Projection based Deep Reinforcement Learning for Safe Control of Distribution Grids
- 2024
-
Ingår i: IEEE Transactions on Power Systems. - : Institute of Electrical and Electronics Engineers (IEEE). - 0885-8950 .- 1558-0679. ; 39:4, s. 5687-5698
-
Tidskriftsartikel (refereegranskat)abstract
- Deep reinforcement learning (DRL) is a promising solution for voltage control of distribution grids with high penetration of inverter-based renewable energy sources (RESs). Yet, when adopting the DRL-based control method, the safe and optimal operation of the system cannot be guaranteed at the same time, as the conventional DRL agent is not designed to solve the hard constraint problem. To address this challenge, this paper proposes a deep neural network (DNN) assisted projection based DRL method for safe control of distribution grids. First, a finite iteration projection algorithm is proposed to guarantee hard constraints by converting a non-convex optimization problem into a finite iteration problem. Next, a DNN assisted projection method is proposed to accelerate the calculation of projection and achieve the practical implementation of hard constraints in DRL problem. Finally, a DNN Projection embedded twin-delayed deep deterministic policy gradient (DPe-TD3) method is proposed to achieve optimal operation of distribution grids with guaranteed 100% safety of the distribution grid. The safety of the DRL training is guaranteed via the embedded Projection DNN in TD3 with participation in gradient return process, which could smoothly and effectively project the DRL agent actions into the feasible area, thus guaranteeing the safety of data driven control and the optimal operation at the same time. The case studies and comparisons are conducted in the IEEE 33 bus system to show the effectiveness of the proposed method.
|
|
