| 1. |
- Aaltonen, Harri, et al.
(författare)
-
A simulation environment for training a reinforcement learning agent trading a battery storage
- 2021
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 14:17
-
Tidskriftsartikel (refereegranskat)abstract
- Battery storages are an essential element of the emerging smart grid. Compared to other distributed intelligent energy resources, batteries have the advantage of being able to rapidly react to events such as renewable generation fluctuations or grid disturbances. There is a lack of research on ways to profitably exploit this ability. Any solution needs to consider rapid electrical phenomena as well as the much slower dynamics of relevant electricity markets. Reinforcement learning is a branch of artificial intelligence that has shown promise in optimizing complex problems involving uncertainty. This article applies reinforcement learning to the problem of trading batteries. The problem involves two timescales, both of which are important for profitability. Firstly, trading the battery capacity must occur on the timescale of the chosen electricity markets. Secondly, the real-time operation of the battery must ensure that no financial penalties are incurred from failing to meet the technical specification. The trading-related decisions must be done under uncertainties, such as unknown future market prices and unpredictable power grid disturbances. In this article, a simulation model of a battery system is proposed as the environment to train a reinforcement learning agent to make such decisions. The system is demonstrated with an application of the battery to Finnish primary frequency reserve markets.
|
|
| 2. |
- Aaltonen, Harri, et al.
(författare)
-
Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach
- 2022
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 15:14
-
Tidskriftsartikel (refereegranskat)abstract
- Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multiâ€objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physicsâ€based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs.
|
|
| 3. |
- Abrishambaf, Reza, et al.
(författare)
-
Distributed home automation system based on IEC61499 function blocks and wireless sensor networks
- 2017
-
Ingår i: Proceedings of the IEEE International Conference on Industrial Technology. - Piscataway, NJ : Institute of Electrical and Electronics Engineers (IEEE). - 9781509053209 ; , s. 1354-1359
-
Konferensbidrag (refereegranskat)abstract
- In this paper, a distributed home automation system will be demonstrated. Traditional systems are based on a central controller where all the decisions are made. The proposed control architecture is a solution to overcome the problems such as the lack of flexibility and re-configurability that most of the conventional systems have. This has been achieved by employing a method based on the new IEC 61499 function block standard, which is proposed for distributed control systems. This paper also proposes a wireless sensor network as the system infrastructure in addition to the function blocks in order to implement the Internet-of-Things technology into the area of home automation as a solution for distributed monitoring and control. The proposed system has been implemented in both Cyber (nxtControl) and Physical (Contiki-OS) level to show the applicability of the solution
|
|
| 4. |
|
|
| 5. |
- Al-Safi, Yazen, et al.
(författare)
-
An ontology-based reconfiguration agent for intelligent mechatronic systems
- 2007
-
Ingår i: Holonic and multi-agent systems for manufacturing. - Berlin : Encyclopedia of Global Archaeology/Springer Verlag. - 9783540744788 ; , s. 114-126
-
Konferensbidrag (refereegranskat)abstract
- This paper discusses an ontology-based reconfiguration agent that uses ontological knowledge of the manufacturing environment for the purpose of reconfiguration without human intervention. The current mass customization era requires increased flexibility and agility in the manufacturing systems to adapt changes in manufacturing requirements and environments. Our configuration agent minimizes the overheads of the current reconfiguration process by automating it. It infers facts about the manufacturing environment from the ontological knowledge model and then decides whemer the current environment can support the given manufacturing requirements
|
|
| 6. |
- Alsafi, Yazen, et al.
(författare)
-
A deployment of an ontology-based reconfiguration agent for intelligent mechatronic systems
- 2008
-
Ingår i: ISIE08. - Piscataway, NJ : IEEE Communications Society. - 9781424416653 ; , s. 1780-1785
-
Konferensbidrag (refereegranskat)abstract
- This paper discusses the extension of an ontology-based reconfiguration agent that uses ontological knowledge of the manufacturing environment for the purpose of reconfiguration without human intervention. The current mass customization era requires increased flexibility and agility in the manufacturing systems to adapt changes in manufacturing requirements and environments. Our configuration agent minimises the overheads of the current reconfiguration process by automating it. It infers facts about the manufacturing environment from the ontological knowledge model and then decides whether the current environment can support the given manufacturing requirements. This paper proposes an extension of the agent architecture enabling the integration between the high level planning with the distributed low level control compliant with the upcoming IEC 61499 function blocks standard
|
|
| 7. |
- Alsafi, Yazen, et al.
(författare)
-
Ontology-based reconfiguration agent for intelligent mechatronic systems in flexible manufacturing
- 2010
-
Ingår i: Robotics and Computer-Integrated Manufacturing. - : Elsevier BV. - 0736-5845 .- 1879-2537. ; 26:4, s. 381-391
-
Tidskriftsartikel (refereegranskat)abstract
- This paper proposes a novel approach to achieving fast reconfiguration of modular manufacturing systems, based on an ontology-based reconfiguration agent. The agent uses ontological knowledge of the manufacturing environment for the purpose of reconfiguration without human intervention. The current mass customization era requires increased flexibility and agility in the manufacturing systems to adapt changes in manufacturing requirements and environments. Our configuration agent minimises the overheads of the current reconfiguration process by automating it. It infers facts about the manufacturing environment from the ontological knowledge model and then decides whether the current environment can support the given manufacturing requirements. This paper proposes the agent architecture enabling the integration between the high level planning with the distributed low level control compliant with the upcoming IEC 61499 function blocks standard
|
|
| 8. |
- Altmojo, Udayanto Dwi, et al.
(författare)
-
On Achieving Reliable Communication in IEC 61499
- 2018
-
Ingår i: 2018 IEEE 23RD INTERNATIONAL CONFERENCE ON EMERGING TECHNOLOGIES AND FACTORY AUTOMATION (ETFA). - Piscataway, NJ : IEEE. - 9781538671085 ; , s. 147-154
-
Konferensbidrag (refereegranskat)abstract
- This paper proposes a novel extension for communication in the IEC 61499 standard. Inspired by the features found in the formal programming language SystemJ, the extension supports reliable and guaranteed communication in distributed execution of function block application(s)/program(s). The extension utilizes mechanisms agnostic on underlying network protocols and are based on formal semantics that guarantee data delivery. The use of proposed extension, called channel, is demonstrated in an industrial automation-type example.
|
|
| 9. |
- Atmojo, Udayanto Dwi, et al.
(författare)
-
A Design Pattern for Systems Composed from Intelligent Mechatronic Modules with Wireless Communication
- 2019
-
Ingår i: Proceedings. - : IEEE. ; , s. 41-48
-
Konferensbidrag (refereegranskat)abstract
- This paper presents a software design pattern for systems composed from distributed intelligent mechatronic modules which communicate with each other via wireless interface. The design pattern is formed by integrating the reliable communication mechanism called channel, which comes from a system-level programming language SystemJ. The paper demonstrates how the design pattern can be applied in an industrial automation example.
|
|
| 10. |
- Atmojo, Udayanto Dwi, et al.
(författare)
-
A Service-Oriented Programming Approach for Dynamic Distributed Manufacturing Systems
- 2020
-
Ingår i: IEEE Transactions on Industrial Informatics. - : IEEE. - 1551-3203 .- 1941-0050. ; 16:1, s. 150-160
-
Tidskriftsartikel (refereegranskat)abstract
- Dynamic reconfigurability and adaptability are crucial features of the future manufacturing systems that must be supported by adequate software technologies. Currently, they are typically achieved as add-ons to existing software tools and run-time systems, which are not based on any formal foundation such as formal model of computation (MoC). This paper presents the new programming paradigm of Service Oriented SystemJ (SOSJ), which targets dynamic distributed software systems suited for future manufacturing applications. SOSJ is built on a merger and the synergies of two programming concepts of (1) Service Oriented Architecture (SOA), to support dynamic software system composition, and (2) SystemJ programming language based on a formal MoC, which targets correct by construction design of static distributed software systems. The resulting programming paradigm allows the design and implementation of dynamic distributed software systems.
|
|
