| 1. |
- Källström, Johan, 1976-, et al.
(författare)
-
Multi-Agent Multi-Objective Deep Reinforcement Learning for Efficient and Effective Pilot Training
- 2019
-
Ingår i: Proceedings of the 10th Aerospace Technology Congress (FT). - : Linköping University Electronic Press. - 9789175190068 ; , s. 101-111
-
Konferensbidrag (refereegranskat)abstract
- The tactical systems and operational environment of modern fighter aircraft are becoming increasingly complex. Creating a realistic and relevant environment for pilot training using only live aircraft is difficult, impractical and highly expensive. The Live, Virtual and Constructive (LVC) simulation paradigm aims to address this challenge. LVC simulation means linking real aircraft, ground-based systems and soldiers (Live), manned simulators (Virtual) and computer controlled synthetic entities (Constructive). Constructive simulation enables realization of complex scenarios with a large number of autonomous friendly, hostile and neutral entities, which interact with each other as well as manned simulators and real systems. This reduces the need for personnel to act as role-players through operation of e.g. live or virtual aircraft, thus lowering the cost of training. Constructive simulation also makes it possible to improve the availability of training by embedding simulation capabilities in live aircraft, making it possible to train anywhere, anytime. In this paper we discuss how machine learning techniques can be used to automate the process of constructing advanced, adaptive behavior models for constructive simulations, to improve the autonomy of future training systems. We conduct a number of initial experiments, and show that reinforcement learning, in particular multi-agent and multi-objective deep reinforcement learning, allows synthetic pilots to learn to cooperate and prioritize among conflicting objectives in air combat scenarios. Though the results are promising, we conclude that further algorithm development is necessary to fully master the complex domain of air combat simulation.
|
|
| 2. |
- Källström, Johan, 1976-, et al.
(författare)
-
Reinforcement Learning for Computer Generated Forces using Open-Source Software
- 2019
-
Ingår i: Proceedings of the 2019 Interservice/Industry Training, Simulation, and Education Conference (IITSEC). ; , s. 1-11
-
Konferensbidrag (refereegranskat)abstract
- The creation of behavior models for computer generated forces (CGF) is a challenging and time-consuming task, which often requires expertise in programming of complex artificial intelligence algorithms. This makes it difficult for a subject matter expert with knowledge about the application domain and the training goals to build relevant scenarios and keep the training system in pace with training needs. In recent years, machine learning has shown promise as a method for building advanced decision-making models for synthetic agents. Such agents have been able to beat human champions in complex games such as poker, Go and StarCraft. There is reason to believe that similar achievements are possible in the domain of military simulation. However, in order to efficiently apply these techniques, it is important to have access to the right tools, as well as knowledge about the capabilities and limitations of algorithms. This paper discusses efficient applications of deep reinforcement learning, a machine learning technique that allows synthetic agents to learn how to achieve their goals by interacting with their environment. We begin by giving an overview of available open-source frameworks for deep reinforcement learning, as well as libraries with reference implementations of state-of-the art algorithms. We then present an example of how these resources were used to build a reinforcement learning environment for a CGF software intended to support training of fighter pilots. Finally, based on our exploratory experiments in the presented environment, we discuss opportunities and challenges related to the application of reinforcement learning techniques in the domain of air combat training systems, with the aim to efficiently construct high quality behavior models for computer generated forces.
|
|
| 3. |
- Källström, Johan, 1976-, et al.
(författare)
-
Tunable Dynamics in Agent-Based Simulation using Multi-Objective Reinforcement Learning
- 2019
-
Ingår i: Proceedings of the 2019 Adaptive and Learning Agents Workshop (ALA), 2019. ; , s. 1-7
-
Konferensbidrag (refereegranskat)abstract
- Agent-based simulation is a powerful tool for studying complex systems of interacting agents. To achieve good results, the behavior models used for the agents must be of high quality. Traditionally these models have been handcrafted by domain experts. This is a difficult, expensive and time consuming process. In contrast, reinforcement learning allows agents to learn how to achieve their goals by interacting with the environment. However, after training the behavior of such agents is often static, i.e. it can no longer be affected by a human. This makes it difficult to adapt agent behavior to specific user needs, which may vary among different runs of the simulation. In this paper we address this problem by studying how multi-objective reinforcement learning can be used as a framework for building tunable agents, whose characteristics can be adjusted at runtime to promote adaptiveness and diversity in agent-based simulation. We propose an agent architecture that allows us to adapt popular deep reinforcement learning algorithms to multi-objective environments. We empirically show that our method allows us to train tunable agents that can approximate the policies of multiple species of agents.
|
|
