| 1. |
- Aslani, Mohammad, et al.
(författare)
-
Continuous residual reinforcement learning for traffic signal control optimization
- 2018
-
Ingår i: Canadian journal of civil engineering (Print). - : NRC Research Press. - 0315-1468 .- 1208-6029. ; 45:8, s. 690-702
-
Tidskriftsartikel (refereegranskat)abstract
- Traffic signal control can be naturally regarded as a reinforcement learning problem. Unfortunately, it is one of the most difficult classes of reinforcement learning problems owing to its large state space. A straightforward approach to address this challenge is to control traffic signals based on continuous reinforcement learning. Although they have been successful in traffic signal control, they may become unstable and fail to converge to near-optimal solutions. We develop adaptive traffic signal controllers based on continuous residual reinforcement learning (CRL-TSC) that is more stable. The effect of three feature functions is empirically investigated in a microscopic traffic simulation. Furthermore, the effects of departing streets, more actions, and the use of the spatial distribution of the vehicles on the performance of CRL-TSCs are assessed. The results show that the best setup of the CRL-TSC leads to saving average travel time by 15% in comparison to an optimized fixed-time controller.
|
|
| 2. |
- Aslani, Mohammad, et al.
(författare)
-
Developing adaptive traffic signal control by actor-critic and direct exploration methods
- 2019
-
Ingår i: Proceedings of the Institution of Civil Engineers. - : Thomas Telford. - 0965-092X .- 1751-7710. ; 172:5, s. 289-298
-
Tidskriftsartikel (refereegranskat)abstract
- Designing efficient traffic signal controllers has always been an important concern in traffic engineering. This is owing to the complex and uncertain nature of traffic environments. Within such a context, reinforcement learning has been one of the most successful methods owing to its adaptability and its online learning ability. Reinforcement learning provides traffic signals with the ability automatically to determine the ideal behaviour for achieving their objective (alleviating traffic congestion). In fact, traffic signals based on reinforcement learning are able to learn and react flexibly to different traffic situations without the need of a predefined model of the environment. In this research, the actor-critic method is used for adaptive traffic signal control (ATSC-AC). Actor-critic has the advantages of both actor-only and critic-only methods. One of the most important issues in reinforcement learning is the trade-off between exploration of the traffic environment and exploitation of the knowledge already obtained. In order to tackle this challenge, two direct exploration methods are adapted to traffic signal control and compared with two indirect exploration methods. The results reveal that ATSC-ACs based on direct exploration methods have the best performance and they consistently outperform a fixed-time controller, reducing average travel time by 21%.
|
|
| 3. |
- Aslani, Mohammad, et al.
(författare)
-
Traffic signal optimization through discrete and continuous reinforcement learning with robustness analysis in downtown Tehran
- 2018
-
Ingår i: Advanced Engineering Informatics. - : Elsevier BV. - 1474-0346 .- 1873-5320. ; 38, s. 639-655
-
Tidskriftsartikel (refereegranskat)abstract
- Traffic signal control plays a pivotal role in reducing traffic congestion. Traffic signals cannot be adequately controlled with conventional methods due to the high variations and complexity in traffic environments. In recent years, reinforcement learning (RL) has shown great potential for traffic signal control because of its high adaptability, flexibility, and scalability. However, designing RL-embedded traffic signal controllers (RLTSCs) for traffic systems with a high degree of realism is faced with several challenges, among others system disturbances and large state-action spaces are considered in this research.The contribution of the present work is founded on three features: (a) evaluating the robustness of different RLTSCs against system disturbances including incidents, jaywalking, and sensor noise, (b) handling a high-dimensional state-action space by both employing different continuous state RL algorithms and reducing the state-action space in order to improve the performance and learning speed of the system, and (c) presenting a detailed empirical study of traffic signals control of downtown Tehran through seven RL algorithms: discrete state Q-learning(λ" role="presentation">), SARSA(λ" role="presentation">), actor-critic(λ" role="presentation">), continuous state Q-learning(λ" role="presentation">), SARSA(λ" role="presentation">), actor-critic(λ" role="presentation">), and residual actor-critic(λ" role="presentation">).In this research, first a real-world microscopic traffic simulation of downtown Tehran is carried out, then four experiments are performed in order to find the best RLTSC with convincing robustness and strong performance. The results reveal that the RLTSC based on continuous state actor-critic(λ" role="presentation">) has the best performance. In addition, it is found that the best RLTSC leads to saving average travel time by 22% (at the presence of high system disturbances) when it is compared with an optimized fixed-time controller.
|
|
| 4. |
- Forte, Castela, et al.
(författare)
-
Deep Learning for Identification of Acute Illness and Facial Cues of Illness
- 2021
-
Ingår i: Frontiers in Medicine. - : Frontiers Media SA. - 2296-858X. ; 8
-
Tidskriftsartikel (refereegranskat)abstract
- Background: The inclusion of facial and bodily cues (clinical gestalt) in machine learning (ML) models improves the assessment of patients' health status, as shown in genetic syndromes and acute coronary syndrome. It is unknown if the inclusion of clinical gestalt improves ML-based classification of acutely ill patients. As in previous research in ML analysis of medical images, simulated or augmented data may be used to assess the usability of clinical gestalt.Objective: To assess whether a deep learning algorithm trained on a dataset of simulated and augmented facial photographs reflecting acutely ill patients can distinguish between healthy and LPS-infused, acutely ill individuals.Methods: Photographs from twenty-six volunteers whose facial features were manipulated to resemble a state of acute illness were used to extract features of illness and generate a synthetic dataset of acutely ill photographs, using a neural transfer convolutional neural network (NT-CNN) for data augmentation. Then, four distinct CNNs were trained on different parts of the facial photographs and concatenated into one final, stacked CNN which classified individuals as healthy or acutely ill. Finally, the stacked CNN was validated in an external dataset of volunteers injected with lipopolysaccharide (LPS).Results: In the external validation set, the four individual feature models distinguished acutely ill patients with sensitivities ranging from 10.5% (95% CI, 1.3-33.1% for the skin model) to 89.4% (66.9-98.7%, for the nose model). Specificity ranged from 42.1% (20.3-66.5%) for the nose model and 94.7% (73.9-99.9%) for skin. The stacked model combining all four facial features achieved an area under the receiver characteristic operating curve (AUROC) of 0.67 (0.62-0.71) and distinguished acutely ill patients with a sensitivity of 100% (82.35-100.00%) and specificity of 42.11% (20.25-66.50%).Conclusion: A deep learning algorithm trained on a synthetic, augmented dataset of facial photographs distinguished between healthy and simulated acutely ill individuals, demonstrating that synthetically generated data can be used to develop algorithms for health conditions in which large datasets are difficult to obtain. These results support the potential of facial feature analysis algorithms to support the diagnosis of acute illness.
|
|
