| 1. |
- Aaltonen, Harri, et al.
(författare)
-
A simulation environment for training a reinforcement learning agent trading a battery storage
- 2021
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 14:17
-
Tidskriftsartikel (refereegranskat)abstract
- Battery storages are an essential element of the emerging smart grid. Compared to other distributed intelligent energy resources, batteries have the advantage of being able to rapidly react to events such as renewable generation fluctuations or grid disturbances. There is a lack of research on ways to profitably exploit this ability. Any solution needs to consider rapid electrical phenomena as well as the much slower dynamics of relevant electricity markets. Reinforcement learning is a branch of artificial intelligence that has shown promise in optimizing complex problems involving uncertainty. This article applies reinforcement learning to the problem of trading batteries. The problem involves two timescales, both of which are important for profitability. Firstly, trading the battery capacity must occur on the timescale of the chosen electricity markets. Secondly, the real-time operation of the battery must ensure that no financial penalties are incurred from failing to meet the technical specification. The trading-related decisions must be done under uncertainties, such as unknown future market prices and unpredictable power grid disturbances. In this article, a simulation model of a battery system is proposed as the environment to train a reinforcement learning agent to make such decisions. The system is demonstrated with an application of the battery to Finnish primary frequency reserve markets.
|
|
| 2. |
- Aaltonen, Harri, et al.
(författare)
-
Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach
- 2022
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 15:14
-
Tidskriftsartikel (refereegranskat)abstract
- Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multiâ€objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physicsâ€based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs.
|
|
| 3. |
|
|
| 4. |
- Atmojo, Udayanto Dwi, et al.
(författare)
-
A Service-Oriented Programming Approach for Dynamic Distributed Manufacturing Systems
- 2020
-
Ingår i: IEEE Transactions on Industrial Informatics. - : IEEE. - 1551-3203 .- 1941-0050. ; 16:1, s. 151-160
-
Tidskriftsartikel (refereegranskat)abstract
- Dynamic reconfigurability and adaptability are crucial features of the future manufacturing systems that must be supported by adequate software technologies. Currently, they are typically achieved as add-ons to existing software tools and run-time systems, which are not based on any formal foundation such as formal model of computation (MoC). This paper presents the new programming paradigm of Service Oriented SystemJ (SOSJ), which targets dynamic distributed software systems suited for future manufacturing applications. SOSJ is built on a merger and the synergies of two programming concepts of (1) Service Oriented Architecture (SOA), to support dynamic software system composition, and (2) SystemJ programming language based on a formal MoC, which targets correct by construction design of static distributed software systems. The resulting programming paradigm allows the design and implementation of dynamic distributed software systems.
|
|
| 5. |
- Azangoo, Mohammad, et al.
(författare)
-
A methodology for generating a digital twin for process industry: a case study of a fiber processing pilot plant
- 2022
-
Ingår i: IEEE Access. - : IEEE. - 2169-3536. ; 10, s. 58787-58810
-
Tidskriftsartikel (refereegranskat)abstract
- Digital twins are now one of the top trends in Industry 4.0, and many companies are using them to increase their level of digitalization, and, as a result, their productivity and reliability. However, the development of digital twins is difficult, expensive, and time consuming. This article proposes a semi-automated methodology to generate digital twins for process plants by extracting process data from engineering documents using text and image processing techniques. The extracted information is used to build an intermediate graph model, which serves as a starting point for generating a model in a simulation software. The translation of a graph-based model into a simulation software environment necessitates the use of simulator-specific mapping rules. This paper describes an approach for generating a digital twin based on a steady state simulation model, using a Piping and Instrumentation Diagram (P&ID) as the main source of information. The steady state modeling paradigm is especially suitable for use cases involving retrofits for an operational process plant, also known as a brownfield plant. A methodology and toolchain is proposed, consisting of manual, semi-automated and fully automated steps. A pilot scale brownfield fiber processing plant was used as a case study to demonstrate our proposed methodology and toolchain, and to identify and address issues that may not occur in laboratory scale case studies. The article concludes with an evaluation of unresolved concerns and future research topics for the automated development of a digital twin for a brownfield process system.
|
|
| 6. |
- Azangoo, Mohammad, et al.
(författare)
-
Digital Twin-Assisted Controlling of AGVs in Flexible Manufacturing Environments
- 2021
-
Ingår i: 2021 IEEE 30th International Symposium on Industrial Electronics (ISIE). - : IEEE.
-
Konferensbidrag (refereegranskat)abstract
- Digital Twins are increasingly being introduced for smart manufacturing systems to improve the efficiency of the main disciplines of such systems. Formal techniques, such as graphs, are a common way of describing Digital Twin models, allowing broad types of tools to provide Digital Twin based services such as fault detection in production lines. Obtaining correct and complete formal Digital Twins of physical systems can be a complicated and time consuming process, particularly for manufacturing systems with plenty of physical objects and the associated manufacturing processes. Automatic generation of Digital Twins is an emerging research field and can reduce time and costs. In this paper, we focus on the generation of Digital Twins for flexible manufacturing systems with Automated Guided Vehicles (AGVs) on the factory floor. In particular, we propose an architectural framework and the associated design choices and software development tools that facilitate automatic generation of Digital Twins for AGVs. Specifically, the scope of the generated digital twins is controlling AGVs in the factory floor. To this end, we focus on different control levels of AGVs and utilize graph theory to generate the graph-based Digital Twin of the factory floor.
|
|
| 7. |
- Azangoo, Mohammad, et al.
(författare)
-
Hybrid Digital Twin for process industry using Apros simulation environment
- 2021
-
Ingår i: 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). - : IEEE.
-
Konferensbidrag (refereegranskat)abstract
- Making an updated and as-built model plays an important role in the life-cycle of a process plant. In particular, Digital Twin models must be precise to guarantee the efficiency and reliability of the systems. Data-driven models can simulate the latest behavior of the sub-systems by considering uncertainties and life-cycle related changes. This paper presents a step-by-step concept for hybrid Digital Twin models of process plants using an early implemented prototype as an example. It will detail the steps for updating the first-principles model and Digital Twin of a brownfield process system using data-driven models of the process equipment. The challenges for generation of an as-built hybrid Digital Twin will also be discussed. With the help of process history data to teach Machine Learning models, the implemented Digital Twin can be continually improved over time and this work in progress can be further optimized.
|
|
| 8. |
- Azangoo, Mohammad, et al.
(författare)
-
Towards a 3D Scanning/VR-based Product Inspection Station
- 2020
-
Ingår i: Proceedings. - : IEEE. ; , s. 1263-1266
-
Konferensbidrag (refereegranskat)abstract
- Quality control of products plays an important role in various stages of the manufacturing process. In particular the final control of the quality of a product before being shipped to a customer is crucial for maintaining customer satisfaction and avoiding costly recalls. Automating quality inspection and integrating it into a seamless Industry 4.0 setting is therefore an important topic in factory automation.We present early work towards an automated product inspection station. Our inspection station features a 3D scanner as well as a Virtual Reality headset for remote human inspection. In addition, our concept provides for automated analysis of scans in the cloud. We present an architectural concept as well as an early prototype.
|
|
| 9. |
- Bejarano, Ronal, et al.
(författare)
-
Assessing Long Distance Communication Alternatives for the Remote Control of AGVs
- 2020
-
Ingår i: Proceedings. - : IEEE. ; , s. 69-76
-
Konferensbidrag (refereegranskat)abstract
- Remote monitoring and control of factory equipment promises a more streamlined and therefore less expensive system operation and maintenance. The geographical distance between a factory and its control center, however, may influence the Quality of Service parameters of the network connections which might stymie the overall control process. To get a better understanding of these potential issues and their impact, we conducted a series of measurements over varying distances for the remote control, operation and simulation of Automated Guided Vehicles (AGVs) that are often used in modern factory environments. To achieve these tests, we defined three communication patterns reflecting local and remote connections as well as the usage of cloud-based services. Applying these patterns, we connected the Factory of the Future at the Aalto University in Finland with the VxLab at the RMIT University in Australia and the Microsoft Azure cloud in the Netherlands. This allowed us to measure important Quality of Service networking parameters for the communication over short, medium, and very long distances. In this paper, we present first empirical results and discuss their impact on the remote control of AGVs.
|
|
| 10. |
- Bejarano, Ronal, et al.
(författare)
-
Towards enhanced live visualization based on communication delay prediction for remote AGV operation
- 2021
-
Ingår i: 2021 26th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). - : IEEE. ; , s. 01-04
-
Konferensbidrag (refereegranskat)abstract
- This paper presents the development progress of a methodology to enhance the dynamic visualization of remote Automated Guided Vehicles (AGVs) using open source frameworks as Gazebo and ROS. The approach is based on a deterministic path pre-calculated by edge computing, accessible for a 3D web visualization cloud platform. The trajectory displayed is verified by a live position streaming from the AGV and a predicted communication delay value. By succeeding on the methodology proposed, it is expected to enhance the fidelity of the 3D live representation for every trajectory executed dynamically and autonomously by AGVs in the shop floor, leaving behind an initial scenario with low visualization fidelity due non-deterministic behavior of long-distance communication channels (including wireless networks essential for AGVs). This work aims to have an impact on improving the user experience of remote webbased interfacing tools to visualize the behavior of cyber-physical systems in agile manufacturing.
|
|
