Search: hsv:(NATURAL SCIENCES) hsv:(Computer and Information Sciences) >
A runtime monitorin...
A runtime monitoring framework to enforce invariants on reinforcement learning agents exploring complex environments
-
- Mallozzi, Piergiuseppe, 1990 (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU),Chalmers tekniska högskola,Chalmers University of Technology
-
- Castellano, Ezequiel (author)
- The Graduate University for Advanced Studies (SOKENDAI)
-
- Pelliccione, Patrizio (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU),Universita degli Studi dell'Aquila,University of L'Aquila,Chalmers tekniska högskola,Chalmers University of Technology
-
show more...
-
- Schneider, Gerardo, 1967 (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU),Chalmers tekniska högskola,Chalmers University of Technology
-
- Tei, Kenji (author)
- Waseda University
-
show less...
-
(creator_code:org_t)
- IEEE, 2019
- 2019
- English.
-
In: RoSE 2019, IEEE/ACM 2nd International Workshop on Robotics Software Engineering, p.5-12. - : IEEE. - 9781728122496
- Related links:
-
https://gup.ub.gu.se...
-
show more...
-
https://doi.org/10.1...
-
https://research.cha...
-
show less...
Abstract
Subject headings
Close
- © 2019 IEEE. Without prior knowledge of the environment, a software agent can learn to achieve a goal using machine learning. Model-free Reinforcement Learning (RL) can be used to make the agent explore the environment and learn to achieve its goal by trial and error. Discovering effective policies to achieve the goal in a complex environment is a major challenge for RL. Furthermore, in safety-critical applications, such as robotics, an unsafe action may cause catastrophic consequences in the agent or in the environment. In this paper, we present an approach that uses runtime monitoring to prevent the reinforcement learning agent to perform 'wrong' actions and to exploit prior knowledge to smartly explore the environment. Each monitor is de?ned by a property that we want to enforce to the agent and a context. The monitors are orchestrated by a meta-monitor that activates and deactivates them dynamically according to the context in which the agent is learning. We have evaluated our approach by training the agent in randomly generated learning environments. Our results show that our approach blocks the agent from performing dangerous and safety-critical actions in all the generated environments. Besides, our approach helps the agent to achieve its goal faster by providing feedback and shaping its reward during learning.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Software Engineering (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Engineering (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
- SAMHÄLLSVETENSKAP -- Utbildningsvetenskap -- Lärande (hsv//swe)
- SOCIAL SCIENCES -- Educational Sciences -- Learning (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Robotteknik och automation (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Robotics (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Keyword
- LTL invariants
- Reinforcement learning
- Reward shaping
- Runtime monitoring
- Reinforcement learning
Publication and Content Type
- ref (subject category)
- kon (subject category)
Find in a library
To the university's database