| 1. |
- Weichwald, Sebastian, et al.
(författare)
-
Learning by Doing : Controlling a Dynamical System using Causality, Control, and Reinforcement Learning
- 2022
-
Ingår i: Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track. - : PMLR. ; 176, s. 246-258
-
Konferensbidrag (refereegranskat)abstract
- Questions in causality, control, and reinforcement learning go beyond the classical machine learning task of prediction under i.i.d. observations. Instead, these fields consider the problem of learning how to actively perturb a system to achieve a certain effect on a response variable. Arguably, they have complementary views on the problem: In control, one usually aims to first identify the system by excitation strategies to then apply model-based design techniques to control the system. In (non-model-based) reinforcement learning, one directly optimizes a reward. In causality, one focus is on identifiability of causal structure. We believe that combining the different views might create synergies and this competition is meant as a first step toward such synergies. The participants had access to observational and (offline) interventional data generated by dynamical systems. Track CHEM considers an open-loop problem in which a single impulse at the beginning of the dynamics can be set, while Track ROBO considers a closed-loop problem in which control variables can be set at each time step. The goal in both tracks is to infer controls that drive the system to a desired state. Code is open-sourced ( https://github.com/LearningByDoingCompetition/learningbydoing-comp ) to reproduce the winning solutions of the competition and to facilitate trying out new methods on the competition tasks.
|
|
| 2. |
- Baumann, Dominik
(författare)
-
Fast and Resource-Efficient Control of Wireless Cyber-Physical Systems
- 2019
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cyber-physical systems (CPSs) tightly integrate physical processes with computing and communication to autonomously interact with the surrounding environment.This enables emerging applications such as autonomous driving, coordinated flightof swarms of drones, or smart factories. However, current technology does notprovide the reliability and flexibility to realize those applications. Challenges arisefrom wireless communication between the agents and from the complexity of thesystem dynamics. In this thesis, we take on these challenges and present three maincontributions.We first consider imperfections inherent in wireless networks, such as communication delays and message losses, through a tight co-design. We tame the imperfectionsto the extent possible and address the remaining uncertainties with a suitable controldesign. That way, we can guarantee stability of the overall system and demonstratefeedback control over a wireless multi-hop network at update rates of 20-50 ms.If multiple agents use the same wireless network in a wireless CPS, limitedbandwidth is a particular challenge. In our second contribution, we present aframework that allows agents to predict their future communication needs. Thisallows the network to schedule resources to agents that are in need of communication.In this way, the limited resource communication can be used in an efficient manner.As a third contribution, to increase the flexibility of designs, we introduce machinelearning techniques. We present two different approaches. In the first approach,we enable systems to automatically learn their system dynamics in case the truedynamics diverge from the available model. Thus, we get rid of the assumption ofhaving an accurate system model available for all agents. In the second approach, wepropose a framework to directly learn actuation strategies that respect bandwidthconstraints. Such approaches are completely independent of a system model andstraightforwardly extend to nonlinear settings. Therefore, they are also suitable forapplications with complex system dynamics.
|
|
| 3. |
|
|
| 4. |
- Baumann, Dominik
(författare)
-
Learning and Control Strategies for Cyber-physical Systems: From Wireless Control over Deep Reinforcement Learning to Causal Identification
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cyber-physical systems (CPS) integrate physical processes with computing and communication to autonomously interact with the environment. This enables emerging applications such as autonomous driving or smart factories. However, current technology does not provide the dependability and adaptability to realize those applications. CPS are systems with complex dynamics that need to be adaptive, communicate with each other over wireless channels, and provide theoretical guarantees on proper functioning. In this thesis, we take on the challenges imposed by wireless CPS by developing appropriate learning and control strategies.In the first part of the thesis, we present a holistic approach that enables provably stable feedback control over wireless networks. At design time (i.e., prior to execution), we tame typical imperfections inherent in wireless networks, such as communication delays and message loss. The remaining imperfections are then accounted for through feedback control. At run time (i.e., during execution), we let systems reason about communication demands and allocate communication resources accordingly. We provide theoretical stability guarantees and evaluate the approach on a cyber-physical testbed, featuring a multi-hop wireless network supporting multiple cart-pole systems.In the second part, we enhance the flexibility of our designs through learning. We first propose a framework based on deep reinforcement learning to jointly learn control and communication strategies for wireless CPS by integrating both objectives, control performance and saving communication resources, in the reward function. This enables learning of resource-aware controllers for nonlinear and high-dimensional systems. Second, we propose an approach for evaluating the performance of models of wireless CPS through online statistical analysis. We trigger learning in case performance drops, that way limiting the number of learning experiments and reducing computational complexity. Third, we propose an algorithm for identifying the causal structure of control systems. We provide theoretical guarantees on learning the true causal structure and demonstrate enhanced generalization capabilities inherited through causal structure identification on a real robotic system.
|
|
| 5. |
- Gräfe, Alexander, et al.
(författare)
-
Towards remote fault detection by analyzing communication priorities
- 2022
-
Ingår i: 2022 IEEE 61st Conference on Decision and Control (CDC). - : Institute of Electrical and Electronics Engineers (IEEE). - 9781665467612 - 9781665467605 - 9781665467629 ; , s. 1758-1763
-
Konferensbidrag (refereegranskat)abstract
- The ability to detect faults is an important safety feature for event-based multi-agent systems. In most existing algorithms, each agent tries to detect faults by checking its own behavior. But what if one agent becomes unable to recognize misbehavior, for example due to failure in its onboard fault detection? To improve resilience and avoid propagation of individual errors to the multi-agent system, agents should check each other remotely for malfunction or misbehavior. In this paper, we build upon a recently proposed predictive triggering architecture that involves communication priorities shared throughout the network to manage limited bandwidth. We propose a fault detection method that uses these priorities to detect errors in other agents. The resulting algorithms is not only able to detect faults, but can also run on a low-power microcontroller in real-time, as we demonstrate in hardware experiments.
|
|
| 6. |
- Sukhija, Bhavya, et al.
(författare)
-
GOSAFEOPT : Scalable safe exploration for global optimization of dynamical systems
- 2023
-
Ingår i: Artificial Intelligence. - : Elsevier BV. - 0004-3702 .- 1872-7921. ; 320
-
Tidskriftsartikel (refereegranskat)abstract
- Learning optimal control policies directly on physical systems is challenging. Even a single failure can lead to costly hardware damage. Most existing model-free learning methods that guarantee safety, i.e., no failures, during exploration are limited to local optima. This work proposes GOSAFEOPT as the first provably safe and optimal algorithm that can safely discover globally optimal policies for systems with high-dimensional state space. We demonstrate the superiority of GOSAFEOPT over competing model-free safe learning methods in simulation and hardware experiments on a robot arm.(c) 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons .org /licenses /by /4 .0/).
|
|
