| 1. |
- Abdalmoaty, Mohamed, 1986-
(författare)
-
Learning Stochastic Nonlinear Dynamical Systems Using Non-stationary Linear Predictors
- 2017
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The estimation problem of stochastic nonlinear parametric models is recognized to be very challenging due to the intractability of the likelihood function. Recently, several methods have been developed to approximate the maximum likelihood estimator and the optimal mean-square error predictor using Monte Carlo methods. Albeit asymptotically optimal, these methods come with several computational challenges and fundamental limitations.The contributions of this thesis can be divided into two main parts. In the first part, approximate solutions to the maximum likelihood problem are explored. Both analytical and numerical approaches, based on the expectation-maximization algorithm and the quasi-Newton algorithm, are considered. While analytic approximations are difficult to analyze, asymptotic guarantees can be established for methods based on Monte Carlo approximations. Yet, Monte Carlo methods come with their own computational difficulties; sampling in high-dimensional spaces requires an efficient proposal distribution to reduce the number of required samples to a reasonable value.In the second part, relatively simple prediction error method estimators are proposed. They are based on non-stationary one-step ahead predictors which are linear in the observed outputs, but are nonlinear in the (assumed known) input. These predictors rely only on the first two moments of the model and the computation of the likelihood function is not required. Consequently, the resulting estimators are defined via analytically tractable objective functions in several relevant cases. It is shown that, under mild assumptions, the estimators are consistent and asymptotically normal. In cases where the first two moments are analytically intractable due to the complexity of the model, it is possible to resort to vanilla Monte Carlo approximations. Several numerical examples demonstrate a good performance of the suggested estimators in several cases that are usually considered challenging.
|
|
| 2. |
- Abdalmoaty, Mohamed, 1986-, et al.
(författare)
-
Simulated Pseudo Maximum Likelihood Identification of Nonlinear Models
- 2017
-
Ingår i: The 20th IFAC World Congress. - : Elsevier. ; 50:1, s. 14058-14063
-
Konferensbidrag (refereegranskat)abstract
- Nonlinear stochastic parametric models are widely used in various fields. However, for these models, the problem of maximum likelihood identification is very challenging due to the intractability of the likelihood function. Recently, several methods have been developed to approximate the analytically intractable likelihood function and compute either the maximum likelihood or a Bayesian estimator. These methods, albeit asymptotically optimal, are computationally expensive. In this contribution, we present a simulation-based pseudo likelihood estimator for nonlinear stochastic models. It relies only on the first two moments of the model, which are easy to approximate using Monte-Carlo simulations on the model. The resulting estimator is consistent and asymptotically normal. We show that the pseudo maximum likelihood estimator, based on a multivariate normal family, solves a prediction error minimization problem using a parameterized norm and an implicit linear predictor. In the light of this interpretation, we compare with the predictor defined by an ensemble Kalman filter. Although not identical, simulations indicate a close relationship. The performance of the simulated pseudo maximum likelihood method is illustrated in three examples. They include a challenging state-space model of dimension 100 with one output and 2 unknown parameters, as well as an application-motivated model with 5 states, 2 outputs and 5 unknown parameters.
|
|
| 3. |
|
|
| 4. |
- Bottegal, Giulio, et al.
(författare)
-
A new kernel-based approach to system identification with quantized output data
- 2017
-
Ingår i: Automatica. - : Elsevier BV. - 0005-1098 .- 1873-2836. ; 85, s. 145-152
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we introduce a novel method for linear system identification with quantized output data. We model the impulse response as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential stability. This serves as a starting point to cast our system identification problem into a Bayesian framework. We employ Markov Chain Monte Carlo methods to provide an estimate of the system. In particular, we design two methods based on the so-called Gibbs sampler that allow also to estimate the kernel hyperparameters by marginal likelihood maximization via the expectation-maximization method. Numerical simulations show the effectiveness of the proposed scheme, as compared to the state-of-the-art kernel-based methods when these are employed in system identification with quantized data. (C) 2017 Elsevier Ltd. All rights reserved.
|
|
| 5. |
- Bottegal, G., et al.
(författare)
-
On maximum likelihood identification of errors-in-variables models
- 2017
-
Ingår i: IFAC-PapersOnLine. - : Elsevier. - 2405-8963. ; 50:1, s. 2824-2829
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper, we revisit maximum likelihood methods for identification of errors-in-variables systems. We assume that the system admits a parametric description, and that the input is a stochastic ARMA process. The cost function associated with the maximum likelihood criterion is minimized by introducing a new iterative solution scheme based on the expectation-maximization method, which proves fast and easily implementable. Numerical simulations show the effectiveness of the proposed method.
|
|
| 6. |
- Ebadat, Afrooz, 1986-
(författare)
-
Experiment Design for Closed-loop System Identification with Applications in Model Predictive Control and Occupancy Estimation
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The objective of this thesis is to develop algorithms for application-oriented input design. This procedure takes the model application into account when designing experiments for system identification.This thesis is divided into two parts. The first part considers the theory of application-oriented input design, with special attention to Model Predictive Control (MPC). We start by studying how to find a convex approximation of the set of models that result in acceptable control performance using analytical methods when controllers with no closed-form control law, for e.g., MPC are employed. The application-oriented input design is formulated in time domain to enable handling of signals constraints. The framework is extended to closed-loop systems where two cases are considered i.e., when the plant is controlled by a general but known controller and for the case of MPC. To this end, an external stationary signal is designed via graph theory. Different sources of uncertainty in application-oriented input design are investigated and a robust application-oriented input design framework is proposed.The second part of this thesis is devoted to the problem of estimating the number of occupants based on the information available to HVAC systems in buildings. The occupancy estimation is first formulated as a two-tier problem. In the first tier, the room dynamic is identified using temporary measurements of occupancy. In the second tier, the identified model is employed to formulate the problem as a fused-lasso problem. The proposed method is further developed to be used as a multi-room estimator using a physics-based model. However, since it is not always possible to collect measurements of occupancy, we proceed by proposing a blind identification algorithm which estimates the room dynamic and occupancy, simultaneously. Finally, the application-oriented input design framework is employed to collect data that is informative enough for occupancy estimation purposes.
|
|
| 7. |
- Eckhard, Diego, et al.
(författare)
-
Cost function shaping of the output error criterion
- 2017
-
Ingår i: Automatica. - : Elsevier. - 0005-1098 .- 1873-2836. ; 76, s. 53-60
-
Tidskriftsartikel (refereegranskat)abstract
- Identification of an output error model using the prediction error method leads to an optimization problem built on input/output data collected from the system to be identified. It is often hard to find the global solution of this optimization problem because in most cases both the corresponding objective function and the search space are nonconvex. The difficulty in solving the optimization problem depends mainly on the experimental conditions, more specifically on the spectra of the input/output data collected from the system. It is therefore possible to improve the convergence of the algorithms by properly choosing the data prefilters; in this paper we show how to perform this choice. We present the application of the proposed approach to case studies where the standard algorithms tend to fail to converge to the global minimum.
|
|
| 8. |
- Everitt, Niklas, 1987-
(författare)
-
Module identification in dynamic networks: parametric and empirical Bayes methods
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The purpose of system identification is to construct mathematical models of dynamical system from experimental data. With the current trend of dynamical systems encountered in engineering growing ever more complex, an important task is to efficiently build models of these systems. Modelling the complete dynamics of these systems is in general not possible or even desired. However, often, these systems can be modelled as simpler linear systems interconnected in a dynamic network. Then, the task of estimating the whole network or a subset of the network can be broken down into subproblems of estimating one simple system, called module, embedded within the dynamic network.The prediction error method (PEM) is a benchmark in parametric system identification. The main advantage with PEM is that for Gaussian noise, it corresponds to the so called maximum likelihood (ML) estimator and is asymptotically efficient. One drawback is that the cost function is in general nonconvex and a gradient based search over the parameters has to be carried out, rendering a good starting point crucial. Therefore, other methods such as subspace or instrumental variable methods are required to initialize the search. In this thesis, an alternative method, called model order reduction Steiglitz-McBride (MORSM) is proposed. As MORSM is also motivated by ML arguments, it may also be used on its own and will in some cases provide asymptotically efficient estimates. The method is computationally attractive since it is composed of a sequence of least squares steps. It also treats the part of the network of no direct interest nonparametrically, simplifying model order selection for the user.A different approach is taken in the second proposed method to identify a module embedded in a dynamic network. Here, the impulse response of the part of the network of no direct interest is modelled as a realization of a Gaussian process. The mean and covariance of the Gaussian process is parameterized by a set of parameters called hyperparameters that needs to be estimated together with the parameters of the module of interest. Using an empirical Bayes approach, all parameters are estimated by maximizing the marginal likelihood of the data. The maximization is carried out by using an iterative expectation/conditional-maximization scheme, which alternates so called expectation steps with a series of conditional-maximization steps. When only the module input and output sensors are used, the expectation step admits an analytical expression. The conditional-maximization steps reduces to solving smaller optimization problems, which either admit a closed form solution, or can be efficiently solved by using gradient descent strategies. Therefore, the overall optimization turns out computationally efficient. Using markov chain monte carlo techniques, the method is extended to incorporate additional sensors.Apart from the choice of identification method, the set of chosen signals to use in the identification will determine the covariance of the estimated modules. To chose these signals, well known expressions for the covariance matrix could, together with signal constraints, be formulated as an optimization problem and solved. However, this approach does neither tell us why a certain choice of signals is optimal nor what will happen if some properties change. The expressions developed in this part of the thesis have a different flavor in that they aim to reformulate the covariance expressions into a form amenable for interpretation. These expressions illustrate how different properties of the identification problem affects the achievable accuracy. In particular, how the power of the input and noise signals, as well as model structure, affect the covariance.
|
|
| 9. |
- Everitt, Niklas, et al.
(författare)
-
Variance analysis of linear SIMO models with spatially correlated noise
- 2017
-
Ingår i: Automatica. - : Elsevier. - 0005-1098. ; 77, s. 68-81
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we address the identification of linear time-invariant single-input multi-output (SIMO) systems. In particular, we assess the performance of the prediction error method by quantifying the variance of the parameter estimates. Using an orthonormal representation for the modules composing the SIMO structure, we show that the parameter estimate of a module depends on the model structure of the other modules, and on the correlation structure of the output disturbances. We provide novel results which quantify the variance-error of the parameter estimates for finite model orders, where the effects of noise correlation structure, model structure and input spectrum are visible. In particular, we show that a sensor does not increase the accuracy of a module if common dynamics have to be estimated. When a module is identified using less parameters than the other modules, we derive the noise correlation structure that gives the minimum total variance. The implications of our results are illustrated through numerical examples and simulations.
|
|
| 10. |
|
|
