| 1. |
- Björk, Joakim, 1989-
(författare)
-
Fundamental Control Performance Limitations for Interarea Oscillation Damping and Frequency Stability
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- With the transition towards renewable energy and the deregulation of the electricity markets, the power system is changing. Growing electricity demand and more intermittent power production increase the need for transfer capacity. Lower inertia levels due to a higher share of renewables increase the need for fast frequency reserves (FFR). In this thesis, we study fundamental control limitations for improving the damping of interarea oscillations and frequency stability.The first part of the thesis considers the damping of oscillatory interarea modes. These system-wide modes involve power oscillating between groups of generators and are sometimes hard to control due to their scale and complexity. We consider limitations of decentralized control schemes based on local measurements, as well as centralized control schemes with limitations associated to actuator dynamics and network topology. It is shown that the stability of asynchronous grids can be improved by modulating the active power of a single interconnecting high-voltage direct current (HVDC) link. One challenge with modulating HVDC active power is that the interaction between interarea modes of the two grids may have a negative impact on system stability. By studying the controllability Gramian, we show that it is possible to improve the damping in both grids as long as the frequencies of their interarea modes are not too close. It is demonstrated how the controllability, and therefore the achievable damping, deteriorates as the frequency difference becomes small. With a modal frequency difference of 5%, the damping can be improved by around 2 percentage points whereas a modal frequency difference of 20% allows for around 8 percentage points damping improvement. The results are validated by simulating two HVDC-interconnected 32-bus power system models. We also consider the coordinated control of two and more HVDC links. For some network configurations, it is shown that the interaction between troublesome interarea modes can be avoided. The second part considers the coordination of frequency containment reserves (FCR) in low-inertia power systems. A case study is performed in a 5-machine model of the Nordic synchronous grid. We consider a low-inertia test case where FCR are provided by hydro power. The non-minimum phase characteristic of the waterways limits the achievable bandwidth of the FCR control. It is shown that a consequence of this is that hydro-FCR fails at keeping the frequency nadir above the 49.0 Hz safety limit following the loss of a HVDC link that imports 1400 MW. To improve the dynamic frequency stability, FFR from wind power is considered. For this, a new wind turbine model is developed. The turbine is controlled at variable-speed, enabling FFR by temporarily borrowing energy from the rotating turbine. The nonlinear wind turbine dynamics are linearized to facilitate a control design that coordinate FFR from the wind with slow FCR from hydropower. Complementary wind resources with a total rating of 2000 MW, operating at 70–90% rated wind speeds, is shown to be more than enough to fulfill the frequency stability requirements. The nadir is kept above 49.0 Hz without the need to install battery storage or to waste wind energy by curtailing the wind turbines.
|
|
| 2. |
- Johansson, Alexander
(författare)
-
Coordination of cross-carrier truck platooning
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The need for sustainable transportation solutions is urgent as the demand for mobility of goods and people is expected to multiply in the upcoming decades. One promising solution is truck platooning, which shows great potential in reducing the energy consumption and operational costs of trucks. To utilize the benefits of truck platooning to the fullest, trucks with different schedules and routes in a road network need coordination to form platoons. This thesis addresses platoon coordination when trucks can wait at hubs to form platoons. We assume there is a reward for driving in a platoon and a cost for waiting at a hub, and the objective is to maximize the overall profit. We focus on coordinating trucks from different carriers, which is important considering that many platoon opportunities are lost if only trucks from the same carrier form platoons.In the first contributions of the thesis, we propose coordination solutions where carriers aim to maximize their own profits through cross-carrier platoon cooperation. We propose an architecture of a platoon-hailing service that stores reported platooning plans of carriers and, based on these, informs carriers about the platoons their trucks can join when they make platooning decisions. A realistic simulation study shows that the cross-carrier platooning system can achieve energy savings of 3.0% and 5.4% when 20% and 100% of the trucks are coordinated, respectively. A non-cooperative game is then formulated to model the strategic interaction among trucks with individual objectives when they coordinate for platooning and make decisions at the beginning of their journeys. The existence of at least one Nash equilibrium is shown. In the case of stochastic travel times, feedback-based solutions are developed wherein trucks repeatedly update their equilibrium decisions. A simulation study with stochastic travel times shows that the feedback-based solutions achieve platooning rates only $5\%$ lower than a solution where the travel times are known. We also explore Pareto-improving coordination guaranteeing each carrier is better off coopering with others, and models for distributing the profit within platoons.In the last contributions of the thesis, we study the problem of optimally releasing trucks at hubs when arriving according to a stochastic process, and a priori information about truck arrivals is inaccessible; this may be sensitive information to share with others. First, we study the release problem at hubs in a hub-corridor where the objective is to maximize the profit over time. The optimality of threshold-based release policies is shown under the assumption that arrivals are independent or that arrivals are dependent due to the releasing behavior at the preceding hub in the corridor. Then, we study the release problem at a single hub where the aim is to maximize the profit of trucks currently at the hub. This is realistic if trucks are only willing to wait at the hub if they can increase their own profits. Stopping time theory is used to show the optimality of a threshold-based release policy when arrivals are independent and identically distributed. These contributions show that simple coordination approaches can achieve high profits from platooning, even under limited information.
|
|
| 3. |
- Alisic, Rijad
(författare)
-
Defense of Cyber-Physical Systems Against Learning-based Attackers
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cyberattacks against critical infrastructures pose a serious threat to society, as they can have devastating consequences on the economy, security, or public health. These infrastructures rely on a large network of cyber components, such as sensors, controllers, computers, and communication devices, to monitor and control their physical processes. An adversary can exploit the vulnerabilities in these cyber components to gain access to the system and manipulate its behavior or functionality.This thesis proposes methods that can be employed as a first line of defense against such attacks for Cyber-Physical Systems. In the first part of the thesis, we consider how uninformed attackers can learn to attack a Cyber-Physical System by eavesdropping through the cyber component. By learning to manipulate the plant, the attacker could figure out how to destroy the physical system before it is too late or completely take it over without raising any alarms. Stopping the attacker at the learning stage would force the attacker to act obliviously, increasing the chances of detecting them.We analyze how homomorphic encryption, a technique that allows computation on encrypted data, hinders an attacker's learning process and reduces its capabilities to attack the system. Specifically, we show that an attacker must solve challenging lattice problems to find attacks that are difficult to detect. Additionally, we show how the detection probability is affected by the attacker's solution to the problems and what parameters of the encryption scheme can be tweaked to increase the detection probability. We also develop a novel method that enables anomaly detection over homomorphically encrypted data without revealing the actual signals to the detector, thereby discouraging attackers from launching attacks on the detector. The detection can be performed using a hypothesis test. However, special care must be taken to ensure that fresh samples are used to detect changes from nominal behavior. We also explore how the adversary can try to evade detection using the same test and how the system can be designed to make detection easier for the defender and more challenging for the attacker.In the second part of the thesis, we study how information leakage about changes in the system depends on the system's dynamics. We use a mathematical tool called the Hammersley-Chapman-Robbins lower bound to measure how much information is leaked and how to minimize it. Specifically, we study how structured input sequences, which we call events, can be obtained through the output of a dynamical system and how this information can be hidden by adding noise or changing the inputs. The system’s speed and sensor locations affect how much information is leaked. We also consider balancing the system’s performance and privacy when using optimal control. Finally, we show how to estimate when the adversary’s knowledge of the event becomes accurate enough to launch an attack and how to change the system before that happens. These results are then used to aid the operator in detecting privacy vulnerabilities when designing a Cyber-Physical System, which increases the overall security when removed.
|
|
| 4. |
- Li, Yuchao
(författare)
-
Approximate Methods of Optimal Control via Dynamic Programming Models
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Optimal control theory has a long history and broad applications. Motivated by the goal of obtaining insights through unification and taking advantage of the abundant capability to generate data and perform online simulation, this thesis studies the discrete-time infinite horizon optimal control problems and introduces some approximate solution methods via abstract dynamic programming (DP) models. The proposed methods involve approximation in value space through the use of data and simulator, apply to a broad class of problems, and strike a good balance between satisfactory performance and computational expenditure.First, we consider deterministic problems with nonnegative stage costs. We derive sufficient conditions under which a local controllability condition holds for the constrained nonlinear systems, and apply the results to establish the convergence of the classical algorithms, including value iteration, policy iteration (PI), and optimistic PI. These results provide a starting point for the design of suboptimal schemes. Then we propose algorithms that take advantage of system trajectory or the presence of parallel computing units to approximate the optimal costs. These algorithms can be viewed as variants of model predictive control (MPC) or rollout, and can be applied to deterministic problems with arbitrary state and control spaces, and arbitrary dynamics. It admits extensions to problems with trajectory constraints, and a multiagent structure. Via the viewpoint provided by the abstract DP models, we also derive the performance bounds of MPC applied to unconstrained and constrained linear quadratic problems, as well as their nonlinear counterparts. These insights suggest new designs of MPC, which likely lead to larger feasible regions of the scheme while costing hardly any loss of performance measured by the costs accumulated over infinite stages. Moreover, we derive algorithms to address problems with a fixed discount factor on future costs. We apply abstract DP models to analyze $\lambda$-PI with randomization algorithms for problems with infinite policies. We show that a contraction property induced by the discount factor is sufficient for the well-posedness of the algorithm. Moreover, we identify the conditions under which the algorithm is convergent with probability one. Guided by the analysis, we exemplify a data-driven approximate implementation of the algorithm for the approximation of the optimal costs of constrained linear and nonlinear control problems. The obtained optimal cost approximations are applied in a related suboptimal scheme. Then we consider discounted problems with discrete state and control spaces and a multiagent structure. When applying rollout to address the problem, the main challenge is to perform minimization over a large control space. To this end, we propose a rollout variant that involves reshuffling the order of the agents. The approximation of the costs of base policies is through the use of on-line simulation. The proposed approach is applied to address multiagent path planning problems within a warehouse context, where through on-line replanning, the robots can adapt to a changing environment while avoiding collision with each other.
|
|
| 5. |
- Liu, Hanxiao, 1995-
(författare)
-
Analysis, Detection, and Mitigation of Attacks in Cyber-physical Systems
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cyber-Physical Systems (CPS) offer close integration among computational elements, communication networks, and physical processes. Such systems play an increasingly important role in a large variety of fields, such as manufacturing, health care, environment, transportation, defence, and so on. Due to the wide applications and critical functions of CPS, increasing importance has been attached to their security. In this thesis, we focus on the security of CPS by investigating vulnerability under cyber-attacks, providing detection mechanisms, and developing feasible countermeasures against cyber-attacks.The first contribution of this thesis is to analyze the performance of remote state estimation under linear attacks. A linear time-invariant system equipped with a smart sensor is studied. The adversary aims to maximize the state estimation error covariance while staying stealthy. The maximal performance degradation that an adversary can achieve with any linear first-order false data injection attack under strict stealthiness for vector systems and $\epsilon$-stealthiness for scalar systems is characterized. We also provide an explicit attack strategy that achieves this bound and compare it with strategies previously proposed in the literature. The second problem of this thesis is about the detection of replay attacks. We aim to design physical watermark signals and corresponding detector to protect a control system against replay attacks. For a scenario where the system parameters are available to the operator, a physical watermarking scheme to detect the replay attack is introduced. The optimal watermark signal design problem is formulated as an optimization problem, and the optimal watermark signal and detector are derived. Subsequently, for systems with unknown parameters, we provide an on-line learning mechanism to asymptotically derive the optimal watermarking signal and corresponding detector.The third problem under investigation is about the detection of false-data injection attacks when the attacker injects malicious data to flip the distribution of the manipulated sensor measurements. The detector decides to continue taking observations or to stop based on the received signals, and the goal is to have the flip attack detected as fast as possible while trying to avoid terminating the measurements when no attack is present. The detection problem is modeled as a partially observable Markov decision process (POMDP) by assuming an attack probability, with the dynamics of the hidden states of the POMDP characterized by a stochastic shortest path (SSP) problem. The optimal policy of the SSP solely depends on the transition costs and is independent of the assumed attack probability. By using a fixed-length window and suitable feature function of the measurements, a Markov decision process (MDP) is used to approximate the POMDP. The optimal solution of the MDP is obtained by reinforcement learning. The fourth contribution of this thesis is to develop a sensor scheduler for remote state estimation under integrity attacks. We seek a trade-off between the energy consumption of communications and accuracy of state estimation when the acknowledgment (ACK) information, sent by the remote estimator to the local sensor, is compromised. The sensor scheduling problem is formulated as an infinite horizon discounted optimal control problem with infinite states. We first analyze the underlying MDP and show that the optimal schedule without ACK attack is of threshold type. Thus, we can simplify the problem by replacing the original state space with a finite state space. For the simplified MDP, when ACK is under attack, the problem is modelled as a POMDP. We analyze the induced MDP that uses a belief vector as its state for the POMDP. The properties of the exact optimal solution are studied via contractive models and it is shown that the threshold solution for the POMDP cannot be readily obtained. A suboptimal solution is provided instead via a rollout approach based on reinforcement learning. We present two variants of rollout and provide corresponding performance bounds.
|
|
| 6. |
- Olguín Muñoz, Manuel Osvaldo Jesús, 1992-
(författare)
-
An Emulation-Based Performance Evaluation Methodology for Edge Computing and Latency Sensitive Applications
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cloud Computing, with its globally-accessible nature and virtually unlimited scalability, has revolutionized our daily lives since its widespread adoption in the early 2000s. It allows us to access our documents anywhere, keep in touch with friends, back up photos, and even remotely control our appliances. Despite this, Cloud Computing has limitations when it comes to novel appli- cations requiring real-time processing or low-latencies. Applications such as Cyber-Physical Systems (CPSs) or mobile eXtended Reality (XR), which in turn also have great transformative potential, are unable to run on the Cloud. Edge Computing is emerging as a potential solution to these limitations by bringing computation closer to the edge of the network, thereby reducing latency and enabling real-time decision making. The combination of Edge Computing and modern mobile network technologies such as 5G offers potential for massive deployments of latency-sensitive applications. However, scaling and understanding these deployments poses important challenges such the optimization of latency through multiple processing steps and trade-offs in wireless system choice, protocols, hardware, and algorithms. Existing approaches have so far been unsuccessful in capturing the complex effects arising from the interplay between network and compute in these systems. This dissertation addresses the challenge of performance evaluation of Edge Computing and the applications enabled by this paradigm with two key contributions to literature. First, a methodological approach to experimentally studying the trade-offs between system responsiveness and resource consumption in latency-sensitive applications such as CPSs and XR is introduced. These applications and systems feature characteristics, such as tight interaction with the physical world and the involvement of humans, that make them challenging to study through simulated approaches or analytical modeling. The approach presented in this thesis involves therefore the emulation of the client-side workload while maintaining the real server-side process and network stack to retain the realism of network and compute effects. Next, an exploration of the requirements for accuracy in the emulation is presented. This work discusses the extent to which accuracy in the emulation can open new avenues for optimization of these systems. To this end, the first-ever realistic model of human timings for a particular class of Mobile Augmented Reality (MAR) applications is provided. The model is combined with a mathematical framework to study the potential for optimization in Edge Computing applications. Results indicate that the methodology introduced in this work offers advantages over existing methods by improving efficiency, repeatability, and replicability. By fully integrating workload components into the emulated software domain, this methodology reduces complexity while still capturing complex effects of network and compute factors that are challenging to model. This approach represents thus an important contribution to literature, as it consists of a comprehensive method for the performance evaluation of Edge environments, encompassing both the application and the infrastructure. Furthermore, results from the exploration into the implications of realism in the emulation suggest that incorporating enhanced realism in client-side emulation can enable the implementation of optimization approaches that would otherwise be infeasible. In particular, this work highlights the significance of considering human behavior and reactions in addition to system-related metrics and performance optimizations in the context of MAR.
|
|
| 7. |
- Stefansson, Elis
(författare)
-
Complexity-aware Decision-making with Applications to Large-scale and Human-in-the-loop Systems
- 2023
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis considers control systems governed by autonomous decision-makers and humans. We formalise and compute low-complex control policies with applications to large-scale systems, and propose human interaction models for controllers to compute interaction-aware decisions.In the first part of the thesis, we consider complexity-aware decision-making, formalising the complexity of control policies and constructing algorithms that compute low-complexity control policies. More precisely, first, we consider large-scale control systems given by hierarchical finite state machines (HFSMs) and present a planning algorithm for such systems that exploits the hierarchy to compute optimal policies efficiently. The algorithm can also handle changes in the system with ease. We prove these properties and conduct simulations on HFSMs with up to 2 million states, including a robot application, where our algorithm outperforms both Dijkstra's algorithm and Contraction Hierarchies. Second, we present a planning objective for control systems modelled as finite state machines yielding an explicit trade-off between a policy's performance and complexity. We consider Kolmogorov complexity since it captures the ultimate compression of an object on a universal Turing machine. We prove that this trade-off is hard to optimise in the sense that dynamic programming is infeasible. Nonetheless, we present two heuristic algorithms obtaining low-complexity policies and evaluate the algorithms on a simple navigation task for a mobile robot, where we obtain low-complexity policies that concur with intuition. In the second part of the thesis, we consider human-in-the-loop systems and predict human decision-making in such systems. First, we look at how the interaction between a robot and a human in a control system can be predicted using game theory, focusing on an autonomous truck platoon interacting with a human-driven car. The interaction is modelled as a hierarchical dynamic game, where the hierarchical decomposition is temporal with a high-fidelity tactical horizon predicting immediate interactions and a low-fidelity strategic horizon estimating long-term behaviour. The game enables feasible computations validated through simulations yielding situation-aware behaviour with natural and safe interactions. Second, we seek models to explain human decision-making, focusing on driver overtaking scenarios. The overtaking problem is formalised as a decision problem with perceptual uncertainty. We propose and numerically analyse risk-agnostic and risk-aware decision models, judging if an overtaking is desirable. We show how a driver's decision time and confidence level can be characterised through two model parameters, which collectively represent human risk-taking behaviour. We detail an experimental testbed for evaluating the decision-making process in the overtaking scenario and present some preliminary experimental results from two human drivers.
|
|
| 8. |
- Alisic, Rijad, 1994-
(författare)
-
Privacy of Sudden Events in Cyber-Physical Systems
- 2021
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cyberattacks against critical infrastructures has been a growing problem for the past couple of years. These infrastructures are a particularly desirable target for adversaries, due to their vital importance in society. For instance, a stop in the operation of a critical infrastructure could result in a crippling effect on a nation's economy, security or public health. The reason behind this increase is that critical infrastructures have become more complex, often being integrated with a large network of various cyber components. It is through these cyber components that an adversary is able to access the system and conduct their attacks.In this thesis, we consider methods which can be used as a first line of defence against such attacks for Cyber-Physical Systems (CPS). Specifically, we start by studying how information leaks about a system's dynamics helps an adversary to generate attacks that are difficult to detect. In many cases, such attacks can be detrimental to a CPS since they can drive the system to a breaking point without being detected by the operator that is tasked to secure the system. We show that an adversary can use small amounts of data procured from information leaks to generate these undetectable attacks. In particular, we provide the minimal amount of information that is needed in order to keep the attack hidden even if the operator tries to probe the system for attacks. We design defence mechanisms against such information leaks using the Hammersley-Chapman-Robbins lower bound. With it, we study how information leakage could be mitigated through corruption of the data by injection of measurement noise. Specifically, we investigate how information about structured input sequences, which we call events, can be obtained through the output of a dynamical system and how this leakage depends on the system dynamics. For example, it is shown that a system with fast dynamical modes tends to disclose more information about an event compared to a system with slower modes. However, a slower system leaks information over a longer time horizon, which means that an adversary who starts to collect information long after the event has occured might still be able to estimate it. Additionally, we show how sensor placements can affect the information leak. These results are then used to aid the operator to detect privacy vulnerabilities in the design of a CPS.Based on the Hammersley-Chapman-Robbins lower bound, we provide additional defensive mechanisms that can be deployed by an operator online to minimize information leakage. For instance, we propose a method to modify the structured inputs in order to maximize the usage of the existing noise in the system. This mechanism allows us to explicitly deal with the privacy-utility trade-off, which is of interest when optimal control problems are considered. Finally, we show how the adversary's certainty of the event increases as a function of the number of samples they collect. For instance, we provide sufficient conditions for when their estimation variance starts to converge to its final value. This information can be used by an operator to estimate when possible attacks from an adversary could occur, and change the CPS before that, rendering the adversary's collected information useless.
|
|
| 9. |
- Baumann, Dominik
(författare)
-
Learning and Control Strategies for Cyber-physical Systems: From Wireless Control over Deep Reinforcement Learning to Causal Identification
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Cyber-physical systems (CPS) integrate physical processes with computing and communication to autonomously interact with the environment. This enables emerging applications such as autonomous driving or smart factories. However, current technology does not provide the dependability and adaptability to realize those applications. CPS are systems with complex dynamics that need to be adaptive, communicate with each other over wireless channels, and provide theoretical guarantees on proper functioning. In this thesis, we take on the challenges imposed by wireless CPS by developing appropriate learning and control strategies.In the first part of the thesis, we present a holistic approach that enables provably stable feedback control over wireless networks. At design time (i.e., prior to execution), we tame typical imperfections inherent in wireless networks, such as communication delays and message loss. The remaining imperfections are then accounted for through feedback control. At run time (i.e., during execution), we let systems reason about communication demands and allocate communication resources accordingly. We provide theoretical stability guarantees and evaluate the approach on a cyber-physical testbed, featuring a multi-hop wireless network supporting multiple cart-pole systems.In the second part, we enhance the flexibility of our designs through learning. We first propose a framework based on deep reinforcement learning to jointly learn control and communication strategies for wireless CPS by integrating both objectives, control performance and saving communication resources, in the reward function. This enables learning of resource-aware controllers for nonlinear and high-dimensional systems. Second, we propose an approach for evaluating the performance of models of wireless CPS through online statistical analysis. We trigger learning in case performance drops, that way limiting the number of learning experiments and reducing computational complexity. Third, we propose an algorithm for identifying the causal structure of control systems. We provide theoretical guarantees on learning the true causal structure and demonstrate enhanced generalization capabilities inherited through causal structure identification on a real robotic system.
|
|
| 10. |
- Björk, Joakim, 1989-
(författare)
-
Performance Quantification of Interarea Oscillation Damping Using HVDC
- 2019
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- With the transition towards renewable energy, and the deregulation of the electricity market, generation patterns and grid topology are changing. These changes increase the need for transfer capacity. One limiting factor, which sometimes leads to underutilization of the transmission grid, is interarea oscillations. These system-wide modes involve groups of generators oscillating relative to each other and are sometimes hard to control due to their scale and complexity. In this thesis we investigate how high-voltage direct current (HVDC) transmission can be used to attenuate interarea oscillations. The thesis has two main contributions.In the first contribution we show how the stability of two asynchronous grids can be improved by modulating the active power of a single interconnecting HVDC link. One concern with modulating HVDC active power is that the interaction between interarea modes of the two grids may have a negative impact on system stability. By studying the controllability Gramian, we show that it is always possible to improve the damping in both grids as long as the frequencies of their interarea modes are not too close. For simplified models, it is explicitly shown how the controllability, and therefore the achievable damping improvements, deteriorates as the frequency difference becomes small.The second contribution of the thesis is to show how coordinated control of two (or more) links can be used to avoid interaction between troublesome interarea modes. We investigate the performance of some multivariable control designs. In particular we look at input usage as well as robustness to measurement, communication, and actuator failures. Suitable controllers are thereby characterized.
|
|
