| 1. |
- Aslani, Mohammad, et al.
(författare)
-
Developing adaptive traffic signal control by actor-critic and direct exploration methods
- 2019
-
Ingår i: Proceedings of the Institution of Civil Engineers. - : Thomas Telford. - 0965-092X .- 1751-7710. ; 172:5, s. 289-298
-
Tidskriftsartikel (refereegranskat)abstract
- Designing efficient traffic signal controllers has always been an important concern in traffic engineering. This is owing to the complex and uncertain nature of traffic environments. Within such a context, reinforcement learning has been one of the most successful methods owing to its adaptability and its online learning ability. Reinforcement learning provides traffic signals with the ability automatically to determine the ideal behaviour for achieving their objective (alleviating traffic congestion). In fact, traffic signals based on reinforcement learning are able to learn and react flexibly to different traffic situations without the need of a predefined model of the environment. In this research, the actor-critic method is used for adaptive traffic signal control (ATSC-AC). Actor-critic has the advantages of both actor-only and critic-only methods. One of the most important issues in reinforcement learning is the trade-off between exploration of the traffic environment and exploitation of the knowledge already obtained. In order to tackle this challenge, two direct exploration methods are adapted to traffic signal control and compared with two indirect exploration methods. The results reveal that ATSC-ACs based on direct exploration methods have the best performance and they consistently outperform a fixed-time controller, reducing average travel time by 21%.
|
|
| 2. |
- Aslani, Mohammad, et al.
(författare)
-
Traffic signal optimization through discrete and continuous reinforcement learning with robustness analysis in downtown Tehran
- 2018
-
Ingår i: Advanced Engineering Informatics. - : Elsevier BV. - 1474-0346 .- 1873-5320. ; 38, s. 639-655
-
Tidskriftsartikel (refereegranskat)abstract
- Traffic signal control plays a pivotal role in reducing traffic congestion. Traffic signals cannot be adequately controlled with conventional methods due to the high variations and complexity in traffic environments. In recent years, reinforcement learning (RL) has shown great potential for traffic signal control because of its high adaptability, flexibility, and scalability. However, designing RL-embedded traffic signal controllers (RLTSCs) for traffic systems with a high degree of realism is faced with several challenges, among others system disturbances and large state-action spaces are considered in this research.The contribution of the present work is founded on three features: (a) evaluating the robustness of different RLTSCs against system disturbances including incidents, jaywalking, and sensor noise, (b) handling a high-dimensional state-action space by both employing different continuous state RL algorithms and reducing the state-action space in order to improve the performance and learning speed of the system, and (c) presenting a detailed empirical study of traffic signals control of downtown Tehran through seven RL algorithms: discrete state Q-learning(λ" role="presentation">), SARSA(λ" role="presentation">), actor-critic(λ" role="presentation">), continuous state Q-learning(λ" role="presentation">), SARSA(λ" role="presentation">), actor-critic(λ" role="presentation">), and residual actor-critic(λ" role="presentation">).In this research, first a real-world microscopic traffic simulation of downtown Tehran is carried out, then four experiments are performed in order to find the best RLTSC with convincing robustness and strong performance. The results reveal that the RLTSC based on continuous state actor-critic(λ" role="presentation">) has the best performance. In addition, it is found that the best RLTSC leads to saving average travel time by 22% (at the presence of high system disturbances) when it is compared with an optimized fixed-time controller.
|
|
