| 1. |
- Aaltonen, Harri, et al.
(författare)
-
A simulation environment for training a reinforcement learning agent trading a battery storage
- 2021
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 14:17
-
Tidskriftsartikel (refereegranskat)abstract
- Battery storages are an essential element of the emerging smart grid. Compared to other distributed intelligent energy resources, batteries have the advantage of being able to rapidly react to events such as renewable generation fluctuations or grid disturbances. There is a lack of research on ways to profitably exploit this ability. Any solution needs to consider rapid electrical phenomena as well as the much slower dynamics of relevant electricity markets. Reinforcement learning is a branch of artificial intelligence that has shown promise in optimizing complex problems involving uncertainty. This article applies reinforcement learning to the problem of trading batteries. The problem involves two timescales, both of which are important for profitability. Firstly, trading the battery capacity must occur on the timescale of the chosen electricity markets. Secondly, the real-time operation of the battery must ensure that no financial penalties are incurred from failing to meet the technical specification. The trading-related decisions must be done under uncertainties, such as unknown future market prices and unpredictable power grid disturbances. In this article, a simulation model of a battery system is proposed as the environment to train a reinforcement learning agent to make such decisions. The system is demonstrated with an application of the battery to Finnish primary frequency reserve markets.
|
|
| 2. |
- Giovanelli, Christian, et al.
(författare)
-
Exploiting Artificial Neural Networks for the Prediction of Ancillary Energy Market Prices
- 2018
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 115:7
-
Tidskriftsartikel (refereegranskat)abstract
- The increase of distributed energy resources in the smart grid calls for new ways to profitably exploit these resources, which can participate in day-ahead ancillary energy markets by providing flexibility. Higher profits are available for resource owners that are able to anticipate price peaks and hours of low prices or zero prices, as well as to control the resource in such a way that exploits the price fluctuations. Thus, this study presents a solution in which artificial neural networks are exploited to predict the day-ahead ancillary energy market prices. The study employs the frequency containment reserve for the normal operations market as a case study and presents the methodology utilized for the prediction of the case study ancillary market prices. The relevant data sources for predicting the market prices are identified, then the frequency containment reserve market prices are analyzed and compared with the spot market prices. In addition, the methodology describes the choices behind the definition of the model validation method and the performance evaluation coefficient utilized in the study. Moreover, the empirical processes for designing an artificial neural network model are presented. The performance of the artificial neural network model is evaluated in detail by means of several experiments, showing robustness and adaptiveness to the fast-changing price behaviors. Finally, the developed artificial neural network model is shown to have better performance than two state of the art models, support vector regression and ARIMA, respectively
|
|
| 3. |
- Kahawala, Sachin, et al.
(författare)
-
Robust Multi-Step Predictor for Electricity Markets with Real-Time Pricing
- 2021
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 14:14
-
Tidskriftsartikel (refereegranskat)abstract
- Real-time electricity pricing mechanisms are emerging as a key component of the smart grid. However, prior work has not fully addressed the challenges of multi-step prediction (Predicting multiple time steps into the future) that is accurate, robust and real-time. This paper proposes a novel Artificial Intelligence-based approach, Robust Intelligent Price Prediction in Real-time (RIPPR), that overcomes these challenges. RIPPR utilizes Variational Mode Decomposition (VMD) to transform the spot price data stream into sub-series that are optimized for robustness using the particle swarm optimization (PSO) algorithm. These sub-series are inputted to a Random Vector Functional Link neural network algorithm for real-time multi-step prediction. A mirror extension removal of VMD, including continuous and discrete spaces in the PSO, is a further novel contribution that improves the effectiveness of RIPPR. The superiority of the proposed RIPPR is demonstrated using three empirical studies of multi-step price prediction of the Australian electricity market.
|
|
| 4. |
- Karhula, Niko, et al.
(författare)
-
Validating the Real-Time Performance of Distributed Energy Resources Participating on Primary Frequency Reserves
- 2021
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 14:21
-
Tidskriftsartikel (refereegranskat)abstract
- A significant body of research has emerged for adapting diverse intelligent distributed energy resources to provide primary frequency reserves (PFR). However, such works are usually vague about the technical specifications for PFR. Industrial practitioners designing systems for PFR markets must pre-qualify their PFR resources against the specifications of the market operator, which is usually a transmission system operator (TSO) or independent system operator (ISO). TSO and ISO requirements for PFR have been underspecified with respect to real-time performance, but as fossil-fuel based PFR is being replaced by various distributed energy resources, these requirements are being tightened. The TSOs of Denmark, Finland, Norway, and Sweden have recently released a joint pilot phase specification with novel requirements on the dynamic performance of PFR resources. This paper presents an automated procedure for performing the pre-qualification procedure against this specification. The procedure is generic and has been demonstrated with a testbed of light emitting diode (LED) lights. The implications of low bandwidth Internet of Things communications, as well as the need to avoid abrupt control actions that irritate human users, have been investigated in the automated procedure.
|
|
| 5. |
- Nefedov, Evgeny, et al.
(författare)
-
Internet of Energy Approach for Sustainable Use of Electric Vehicles as Energy Storage of Prosumer Buildings
- 2018
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 11:8
-
Tidskriftsartikel (refereegranskat)abstract
- Vehicle-to-building (V2B) technology permits bypassing the power grid in order to supply power to a building from electric vehicle (EV) batteries in the parking lot. This paper investigates the hypothesis stating that the increasing number of EVs on our roads can be also beneficial for making buildings sustainably greener on account of using V2B technology in conjunction with local photovoltaic (PV) generation. It is assumed that there is no local battery storage other than EVs and that the EV batteries are fully available for driving, so that the EVs batteries must be at the intended state of charge at the departure time announced by the EV driver. Our goal is to exploit the potential of the EV batteries capacity as much as possible in order to permit a large area of solar panels, so that even on sunny days all PV power can be used to supply the building needs or the EV charging at the parking lot. A system architecture and collaboration protocols that account for uncertainties in EV behaviour are proposed. The proposed approach is proven in simulation covering one year period for three locations in different climatic regions of the US, resulting in the electricity bill reductions of 15.8%, 9.1% and 4.9% for California, New Jersey and Alaska, respectively. These results are compared to state-of-the-art research in combining V2B with PV or wind power generation. It is concluded that the achieved electricity bill reductions are superior to the state-of-the-art, because previous work is based on problem formulations that exploit only a part of the potential EV battery capacity
|
|
| 6. |
- Subramanya, Rakshith, et al.
(författare)
-
A Virtual Power Plant Solution for Aggregating Photovoltaic Systems and Other Distributed Energy Resources for Northern European Primary Frequency Reserves
- 2021
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 14:5
-
Tidskriftsartikel (refereegranskat)abstract
- Primary frequency reserves in Northern Europe have traditionally been provided with hydro plants and fossil fuel-burning spinning reserves. Recently, smart distributed energy resources have been equipped with functionality needed to participate on frequency reserves. Key categories of such resources include photovoltaic systems, batteries, and smart loads. Most of these resources are small and cannot provide the minimum controllable power required to participate on frequency reserves. Thus, virtual power plants have been used to aggregate the resources and trade them on the frequency reserves markets. The information technology aspects of virtual power plants are proprietary and many of the details have not been made public. The first contribution of this article is to propose a generic data model and application programming interface for a virtual power plant with the above-mentioned capabilities. The second contribution is to use the application programming interface to cope with the unpredictability of the frequency reserve capacity that the photovoltaic systems and other distributed energy resources are able to provide to the frequency reserves markets in the upcoming bidding period. The contributions are demonstrated with an operational virtual power plant installation at a Northern European shopping center, aggregating photovoltaic Primary Frequency Reserves resources.
|
|
| 7. |
- Aaltonen, Harri, et al.
(författare)
-
Bidding a Battery on Electricity Markets and Minimizing Battery Aging Costs: A Reinforcement Learning Approach
- 2022
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 15:14
-
Tidskriftsartikel (refereegranskat)abstract
- Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multiâ€objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physicsâ€based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs.
|
|
| 8. |
- Sierla, Seppo, et al.
(författare)
-
A Review of Reinforcement Learning Applications to Control of Heating, Ventilation and Air Conditioning Systems
- 2022
-
Ingår i: Energies. - : MDPI. - 1996-1073. ; 15:10
-
Forskningsöversikt (refereegranskat)abstract
- Reinforcement learning has emerged as a potentially disruptive technology for control and optimization of HVAC systems. A reinforcement learning agent takes actions, which can be direct HVAC actuator commands or setpoints for control loops in building automation systems. The actions are taken to optimize one or more targets, such as indoor air quality, energy consumption and energy cost. The agent receives feedback from the HVAC systems to quantify how well these targets have been achieved. The feedback is captured by a reward function designed by the developer of the reinforcement learning agent. A few reviews have focused on the reward aspect of reinforcement learning applications for HVAC. However, there is a lack of reviews that assess how the actions of the reinforcement learning agent have been formulated, and how this impacts the possibilities to achieve various optimization targets in single zone or multi-zone buildings. The aim of this review is to identify the action formulations in the literature and to assess how the choice of formulation impacts the level of abstraction at which the HVAC systems are considered. Our methodology involves a search string in the Web of Science database and a list of selection criteria applied to each article in the search results. For each selected article, a three-tier categorization of the selected articles has been performed. Firstly, the applicability of the approach to buildings with one or more zones is considered. Secondly, the articles are categorized by the type of action taken by the agent, such as a binary, discrete or continuous action. Thirdly, the articles are categorized by the aspects of the indoor environment being controlled, namely temperature, humidity or air quality. The main result of the review is this three-tier categorization that reveals the community’s emphasis on specific HVAC applications, as well as the readiness to interface the reinforcement learning solutions to HVAC systems. The article concludes with a discussion of trends in the field as well as challenges that require further research.
|
|
