| 1. |
- Huck, Tom P., et al.
(författare)
-
Hazard Analysis of Collaborative Automation Systems: A Two-layer Approach based on Supervisory Control and Simulation
- 2023
-
Ingår i: Proceedings - IEEE International Conference on Robotics and Automation. - 1050-4729. ; 2023-May, s. 10560-10566
-
Konferensbidrag (refereegranskat)abstract
- Safety critical systems are typically subjected to hazard analysis before commissioning to identify and analyse potentially hazardous system states that may arise during operation. Currently, hazard analysis is mainly based on human reasoning, past experiences, and simple tools such as checklists and spreadsheets. Increasing system complexity makes such approaches decreasingly suitable. Furthermore, testing-based hazard analysis is often not suitable due to high costs or dangers of physical faults. A remedy for this are model-based hazard analysis methods, which either rely on formal models or on simulation models, each with their own benefits and drawbacks. This paper proposes a two-layer approach that combines the benefits of exhaustive analysis using formal methods with detailed analysis using simulation. Unsafe behaviours that lead to unsafe states are first synthesised from a formal model of the system using Supervisory Control Theory. The result is then input to the simulation where detailed analyses using domain-specific risk metrics are performed. Though the presented approach is generally applicable, this paper demonstrates the benefits of the approach on an industrial human-robot collaboration system.
|
|
| 2. |
- Jorge, Emilio, 1992, et al.
(författare)
-
Reinforcement learning in real-time geometry assurance
- 2018
-
Ingår i: Procedia CIRP. - : Elsevier BV. - 2212-8271. ; 72, s. 1073-1078
-
Konferensbidrag (refereegranskat)abstract
- To improve the assembly quality during production, expert systems are often used. These experts typically use a system model as a basis for identifying improvements. However, since a model uses approximate dynamics or imperfect parameters, the expert advice is bound to be biased. This paper presents a reinforcement learning agent that can identify and limit systematic errors of an expert systems used for geometry assurance. By observing the resulting assembly quality over time, and understanding how different decisions affect the quality, the agent learns when and how to override the biased advice from the expert software.
|
|
| 3. |
- Sjöberg, Anders, et al.
(författare)
-
Online geometry assurance in individualized production by feedback control and model calibration of digital twins
- 2023
-
Ingår i: Journal of Manufacturing Systems. - : Elsevier BV. - 0278-6125. ; 66, s. 71-81
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper, we consider online calibration of a Digital Twin and its use for control and optimization in the assembly process of sheet metal parts. This calibration is done based on a feedback signal received by calculating the quality of the simulated assemblies as compared to the prediction made by the Digital Twin. We develop a Kalman filter-based approach for online calibration of the Digital Twin, which in turn is used by a one-step look-ahead optimizer to define an online control scheme. This control scheme balances directly predicted quality gains against reduced uncertainty whose purpose is to enable long-term quality gains. The usage of a calibrated model in a one-step look-ahead optimizer as a controller allows to incorporate the benefits of the usage of Digital Twins for individualized control, where the control parameters of a production cell are optimized in a Digital Twin based on measured properties of the production inputs, over nominal control, where control parameters are set with respect to some reference production inputs, in an approach which is able to use measured final production quality for feedback control. The proposed approach is evaluated by computer simulations of two industrial product assembly use cases. In the first case, it demonstrates significant gains in quality of the produced assemblies, while in the second case it shows negligible to small improvements. The second case is, however, rather insensitive to miscalibration, which enables only small gains.
|
|
| 4. |
- Cronrath, Constantin Christian Justin, 1990, et al.
(författare)
-
Energy reduction in paint shops through energy-sensitive on-off control
- 2016
-
Ingår i: IEEE International Conference on Automation Science and Engineering. - 2161-8070 .- 2161-8089. - 9781509024094 ; , s. 1282-1288
-
Konferensbidrag (refereegranskat)abstract
- Energy efficiency is the key for a sustainable manufacturing. However, energy efficiency measures often struggle with barriers, which inhibit their implementation. Environmental discrete event simulation is an approach increasingly discussed in research to overcome such barriers. Paint shops, which are responsible for 50-70% of an assembly plant's energy utilization in vehicular manufacturing, are seldom considered in this research, though. Therefore, the specific requirements on an energy consumption model for painting systems were investigated and implemented in simulation software. In addition, an energy-sensitive algorithm is proposed to reveal energy efficiency potentials based on an on-off control strategy. The result is a comprehensive simulation concept, that contributes to the removal of energy efficiency barriers by enabling a detailed evaluation of improvement measures and by indicating worthwhile saving potentials. A case study for a typical paint shop shows that the total energy saving potential is 7% in total, corresponding to more than 300 MWh of annual savings. © 2016 IEEE.
|
|
| 5. |
- Cronrath, Constantin, 1990, et al.
(författare)
-
Enhancing digital twins through reinforcement learning
- 2019
-
Ingår i: IEEE International Conference on Automation Science and Engineering. - 2161-8070 .- 2161-8089. ; 2019-August, s. 293-298
-
Konferensbidrag (refereegranskat)abstract
- Digital Twins are core enablers of smart and autonomous manufacturing systems. Although they strive to represent their physical counterpart as accurately as possible, slight model or data errors will remain. We present an algorithm to compensate for those residual errors through Reinforcement Learning (RL) and data fed back from the manufacturing system. When learning, the Digital Twin acts as teacher and safety policy to ensure minimal performance. We test the algorithm in a sheet metal assembly context, in which locators of the fixture are optimally adjusted for individual assemblies. Our results show a fast adaption and improved performance of the autonomous system.
|
|
| 6. |
- Cronrath, Constantin, 1990, et al.
(författare)
-
Formal Properties of the Digital Twin-Implications for Learning, Optimization, and Control
- 2020
-
Ingår i: IEEE International Conference on Automation Science and Engineering. - 2161-8070 .- 2161-8089. ; 2020-August, s. 679-684
-
Konferensbidrag (refereegranskat)abstract
- Digital twins are regarded as enablers of smart and autonomous manufacturing systems. The digital twin concept essentially refers to a ultra-realistic digital model of a products or system, coupled by a bidirectional automated data exchange, used for simulation, optimization, and control. Although the concept has gained significant attention, its conceptual basis is still weak. We review common definitions and descriptions of digital twins and refine the concept in system theoretic terms. With this sharpened perspective on digital twins, we sketch out three basic problems that need to be solved to turn a digital model into a digital twin. To that end, we call attention to challenges that need to be researched into.
|
|
| 7. |
- Cronrath, Constantin, 1990, et al.
(författare)
-
How Useful is Learning in Mitigating Mismatch Between Digital Twins and Physical Systems?
- 2024
-
Ingår i: IEEE Transactions on Automation Science and Engineering. - 1558-3783 .- 1545-5955. ; 21:1, s. 758-770
-
Tidskriftsartikel (refereegranskat)abstract
- In the control of complex systems, we observe two diametrical trends: model-based control derived from digital twins, and model-free control through AI. There are also attempts to bridge the gap between the two by incorporating learning-based AI algorithms into digital twins to mitigate mismatches between the digital twin model and the physical system. One of the most straightforward approaches to this is direct input adaptation. In this paper, we ask whether it is useful to employ a generic learning algorithm in such a setting, and our conclusion is "not very". We denote an algorithm to be more useful than another algorithm based on three aspects: 1) it requires fewer data samples to reach a desired minimal performance, 2) it achieves better performance for a reasonable number of data samples, and 3) it accumulates less regret. In our evaluation, we randomly sample problems from an industrially relevant geometry assurance context and measure the aforementioned performance indicators of 16 different algorithms. Our conclusion is that blackbox optimization algorithms, designed to leverage specific properties of the problem, generally perform better than generic learning algorithms, once again finding that "there is no free lunch".
|
|
| 8. |
- Cronrath, Constantin, 1990
(författare)
-
On Reinforcement Learning and Digital Twins for Intelligent Automation
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Current trends, such as the fourth industrial revolution and sustainable manufacturing, enable and necessitate manufacturing automation to become more intelligent to meet ever new design requirements in terms of flexibility, speed, quality, and cost. Two distinct research streams towards intelligent manufacturing exist in the scientific literature: the model-based digital twin approach and the data-driven learning approach. Research that incorporates advantages of the one into the other approach is frequently called for. Accordingly, this thesis investigates how machine learning can be used to mitigate the model-system mismatch in digital twins and how prior model-based knowledge can be introduced in reinforcement learning in the context of intelligent automation. In terms of mitigating mismatches in digital twins, research presented in this thesis suggests that learning is of limited usefulness when employed naively in static and systemic mismatch scenarios. In such settings, blackbox optimization algorithms, that leverage properties of the problem, are more useful in terms of sample-efficiency, performance within a given budget, and regret (i.e. when compared to an optimal controller). Learning seems to be of some merit, however, in individualized production control and when used for adapting parameters within a digital twin. An additional research outcome presented in this thesis is a principled method for incorporating prior knowledge in form of automata specifications into reinforcement learning. Furthermore, the benefits of introducing rich prior model-based knowledge in form of economic non-linear model predictive controllers as model class for function approximation in reinforcement learning is demonstrated in the context of energy optimization. Lastly, this thesis highlights that adaptive economic non-linear model predictive control may be understood as a unifying framework for both research streams towards intelligent automation.
|
|
| 9. |
- Cronrath, Constantin, 1990, et al.
(författare)
-
Relevant Safety Falsification by Automata Constrained Reinforcement Learning
- 2022
-
Ingår i: IEEE International Conference on Automation Science and Engineering. - 2161-8070 .- 2161-8089. ; 2022-August, s. 2273-2280
-
Konferensbidrag (refereegranskat)abstract
- Complex safety-critical cyber-physical systems, such as autonomous cars or collaborative robots, are becoming increasingly common. Simulation-based falsification is a testing method for uncovering safety hazards of such systems already in the design phase. Conventionally, the falsification method takes the form of a static optimization. Recently, dynamic optimization methods such as reinforcement learning have gained interest for their ability to uncover harder-to-find safety hazards. However, these methods may converge to risk-maximising, but irrelevant behaviors. This paper proposes a principled formulation and solution of the falsification problem by automata constrained reinforcement learning, in which rewards for relevant behavior are tuned via Lagrangian relaxation. The challenges and proposed methods are demonstrated in a use-case example from the domain of industrial human-robot collaboration, where falsification is used to identify hazardous human worker behaviors that result in human-robot collisions. Compared to random sampling and conventional approximate Q-learning, we show that the proposed method generates equally hazardous, but at the same time more relevant testing conditions that expose safety flaws.
|
|
