| 1. |
- Abdalmoaty, Mohamed, 1986-
(författare)
-
Identification of Stochastic Nonlinear Dynamical Models Using Estimating Functions
- 2019
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Data-driven modeling of stochastic nonlinear systems is recognized as a very challenging problem, even when reduced to a parameter estimation problem. A main difficulty is the intractability of the likelihood function, which renders favored estimation methods, such as the maximum likelihood method, analytically intractable. During the last decade, several numerical methods have been developed to approximately solve the maximum likelihood problem. A class of algorithms that attracted considerable attention is based on sequential Monte Carlo algorithms (also known as particle filters/smoothers) and particle Markov chain Monte Carlo algorithms. These algorithms were able to obtain impressive results on several challenging benchmark problems; however, their application is so far limited to cases where fundamental limitations, such as the sample impoverishment and path degeneracy problems, can be avoided.This thesis introduces relatively simple alternative parameter estimation methods that may be used for fairly general stochastic nonlinear dynamical models. They are based on one-step-ahead predictors that are linear in the observed outputs and do not require the computations of the likelihood function. Therefore, the resulting estimators are relatively easy to compute and may be highly competitive in this regard: they are in fact defined by analytically tractable objective functions in several relevant cases. In cases where the predictors are analytically intractable due to the complexity of the model, it is possible to resort to {plain} Monte Carlo approximations. Under certain assumptions on the data and some conditions on the model, the convergence and consistency of the estimators can be established. Several numerical simulation examples and a recent real-data benchmark problem demonstrate a good performance of the proposed method, in several cases that are considered challenging, with a considerable reduction in computational time in comparison with state-of-the-art sequential Monte Carlo implementations of the ML estimator.Moreover, we provide some insight into the asymptotic properties of the proposed methods. We show that the accuracy of the estimators depends on the model parameterization and the shape of the unknown distribution of the outputs (via the third and fourth moments). In particular, it is shown that when the model is non-Gaussian, a prediction error method based on the Gaussian assumption is not necessarily more accurate than one based on an optimally weighted parameter-independent quadratic norm. Therefore, it is generally not obvious which method should be used. This result comes in contrast to a current belief in some of the literature on the subject. Furthermore, we introduce the estimating functions approach, which was mainly developed in the statistics literature, as a generalization of the maximum likelihood and prediction error methods. We show how it may be used to systematically define optimal estimators, within a predefined class, using only a partial specification of the probabilistic model. Unless the model is Gaussian, this leads to estimators that are asymptotically uniformly more accurate than linear prediction error methods when quadratic criteria are used. Convergence and consistency are established under standard regularity and identifiability assumptions akin to those of prediction error methods.Finally, we consider the problem of closed-loop identification when the system is stochastic and nonlinear. A couple of scenarios given by the assumptions on the disturbances, the measurement noise and the knowledge of the feedback mechanism are considered. They include a challenging case where the feedback mechanism is completely unknown to the user. Our methods can be regarded as generalizations of some classical closed-loop identification approaches for the linear time-invariant case. We provide an asymptotic analysis of the methods, and demonstrate their properties in a simulation example.
|
|
| 2. |
- Abdalmoaty, Mohamed, 1986-
(författare)
-
Learning Stochastic Nonlinear Dynamical Systems Using Non-stationary Linear Predictors
- 2017
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The estimation problem of stochastic nonlinear parametric models is recognized to be very challenging due to the intractability of the likelihood function. Recently, several methods have been developed to approximate the maximum likelihood estimator and the optimal mean-square error predictor using Monte Carlo methods. Albeit asymptotically optimal, these methods come with several computational challenges and fundamental limitations.The contributions of this thesis can be divided into two main parts. In the first part, approximate solutions to the maximum likelihood problem are explored. Both analytical and numerical approaches, based on the expectation-maximization algorithm and the quasi-Newton algorithm, are considered. While analytic approximations are difficult to analyze, asymptotic guarantees can be established for methods based on Monte Carlo approximations. Yet, Monte Carlo methods come with their own computational difficulties; sampling in high-dimensional spaces requires an efficient proposal distribution to reduce the number of required samples to a reasonable value.In the second part, relatively simple prediction error method estimators are proposed. They are based on non-stationary one-step ahead predictors which are linear in the observed outputs, but are nonlinear in the (assumed known) input. These predictors rely only on the first two moments of the model and the computation of the likelihood function is not required. Consequently, the resulting estimators are defined via analytically tractable objective functions in several relevant cases. It is shown that, under mild assumptions, the estimators are consistent and asymptotically normal. In cases where the first two moments are analytically intractable due to the complexity of the model, it is possible to resort to vanilla Monte Carlo approximations. Several numerical examples demonstrate a good performance of the suggested estimators in several cases that are usually considered challenging.
|
|
| 3. |
- Abdalmoaty, Mohamed, 1986-, et al.
(författare)
-
Linear Prediction Error Methods for Stochastic Nonlinear Models
- 2019
-
Ingår i: Automatica. - : Elsevier. - 0005-1098 .- 1873-2836. ; 105, s. 49-63
-
Tidskriftsartikel (refereegranskat)abstract
- The estimation problem for stochastic parametric nonlinear dynamical models is recognized to be challenging. The main difficulty is the intractability of the likelihood function and the optimal one-step ahead predictor. In this paper, we present relatively simple prediction error methods based on non-stationary predictors that are linear in the outputs. They can be seen as extensions of the linear identification methods for the case where the hypothesized model is stochastic and nonlinear. The resulting estimators are defined by analytically tractable objective functions in several common cases. It is shown that, under certain identifiability and standard regularity conditions, the estimators are consistent and asymptotically normal. We discuss the relationship between the suggested estimators and those based on second-order equivalent models as well as the maximum likelihood method. The paper is concluded with a numerical simulation example as well as a real-data benchmark problem.
|
|
| 4. |
- Abdalmoaty, Mohamed, 1986-, et al.
(författare)
-
On Re-Weighting, Regularization Selection, and Transient in Nuclear Norm Based Identification
- 2015
-
Ingår i: IFAC-PapersOnLine. - : Elsevier. - 2405-8963. ; 48:28, s. 92-97
-
Tidskriftsartikel (refereegranskat)abstract
- In this contribution, we consider the classical problem of estimating an Output Error model given a set of input-output measurements. First, we develop a regularization method based on the re-weighted nuclear norm heuristic. We show that the re-weighting improves the estimate in terms of better fit. Second, we suggest an implementation method that helps in eliminating the regularization parameters from the problem by introducing a constant based on a validation criterion. Finally, we develop a method for considering the effect of the transient when the initial conditions are unknown. A simple numerical example is used to demonstrate the proposed method in comparison to classical and another recent method based on the nuclear norm heuristic.
|
|
| 5. |
- Abdalmoaty, Mohamed Rasheed, 1986-, et al.
(författare)
-
A Simulated Maximum Likelihood Method for Estimation of Stochastic Wiener Systems
- 2016
-
Ingår i: 2016 IEEE 55th Conference on Decision and Control (CDC). - : IEEE. - 9781509018376 - 9781509018444 - 9781509018383 ; , s. 3060-3065
-
Konferensbidrag (refereegranskat)abstract
- This paper introduces a simulation-based method for maximum likelihood estimation of stochastic Wienersystems. It is well known that the likelihood function ofthe observed outputs for the general class of stochasticWiener systems is analytically intractable. However, when the distributions of the process disturbance and the measurement noise are available, the likelihood can be approximated byrunning a Monte-Carlo simulation on the model. We suggest the use of Laplace importance sampling techniques for the likelihood approximation. The algorithm is tested on a simple first order linear example which is excited only by the process disturbance. Further, we demonstrate the algorithm on an FIR system with cubic nonlinearity. The performance of the algorithm is compared to the maximum likelihood method and other recent techniques.
|
|
| 6. |
- Abdalmoaty, Mohamed R., 1986-, et al.
(författare)
-
Application of a Linear PEM Estimator to a Stochastic Wiener-Hammerstein Benchmark Problem⁎
- 2018
-
Ingår i: IFAC-PapersOnLine. - : Elsevier B.V.. - 2405-8963. ; 51:15, s. 784-789
-
Tidskriftsartikel (refereegranskat)abstract
- The estimation problem of stochastic Wiener-Hammerstein models is recognized to be challenging, mainly due to the analytical intractability of the likelihood function. In this contribution, we apply a computationally attractive prediction error method estimator to a real-data stochastic Wiener-Hammerstein benchmark problem. The estimator is defined using a deterministic predictor that is nonlinear in the input. The prediction error method results in tractable expressions, and Monte Carlo approximations are not necessary. This allows us to tackle several issues considered challenging from the perspective of the current mainstream approach. Under mild conditions, the estimator can be shown to be consistent and asymptotically normal. The results of the method applied to the benchmark data are presented and discussed.
|
|
| 7. |
- Abdalmoaty, Mohamed Rasheed, 1986-, et al.
(författare)
-
Consistent Estimators of Stochastic MIMO Wiener Models based on Suboptimal Predictors
- 2018
-
Ingår i: 2018 IEEE Conference on Decision and Control (CDC). - : IEEE. - 9781538613955 - 9781538613948 - 9781538613962 ; , s. 3842-3847
-
Konferensbidrag (refereegranskat)abstract
- We consider a parameter estimation problem in a general class of stochastic multiple-inputs multiple-outputs Wiener models, where the likelihood function is, in general, analytically intractable. When the output signal is a scalar independent stochastic process, the likelihood function of the parameters is given by a product of scalar integrals. In this case, numerical integration may be efficiently used to approximately solve the maximum likelihood problem. Otherwise, the likelihood function is given by a challenging multidimensional integral. In this contribution, we argue that by ignoring the temporal and spatial dependence of the stochastic disturbances, a computationally attractive estimator based on a suboptimal predictor can be constructed by evaluating scalar integrals regardless of the number of outputs. Under some conditions, the convergence of the resulting estimators can be established and consistency is achieved under certain identifiability hypothesis. We highlight the relationship between the resulting estimators and a recently proposed prediction error method estimator. We also remark that the method can be used for a wider class of stochastic nonlinear models. The performance of the method is demonstrated by a numerical simulation example using a 2-inputs 2-outputs model with 9 parameters.
|
|
| 8. |
- Abdalmoaty, Mohamed R., 1986-, et al.
(författare)
-
Identification of a Class of Nonlinear Dynamical Networks⁎
- 2018
-
Ingår i: IFAC-PapersOnLine. - : Elsevier B.V.. - 2405-8963. ; 51:15, s. 868-873
-
Tidskriftsartikel (refereegranskat)abstract
- Identification of dynamic networks has attracted considerable interest recently. So far the main focus has been on linear time-invariant networks. Meanwhile, most real-life systems exhibit nonlinear behaviors; consider, for example, two stochastic linear time-invariant systems connected in series, each of which has a nonlinearity at its output. The estimation problem in this case is recognized to be challenging, due to the analytical intractability of both the likelihood function and the optimal one-step ahead predictors of the measured nodes. In this contribution, we introduce a relatively simple prediction error method that may be used for the estimation of nonlinear dynamical networks. The estimator is defined using a deterministic predictor that is nonlinear in the known signals. The estimation problem can be defined using closed-form analytical expressions in several non-trivial cases, and Monte Carlo approximations are not necessarily required. We show, that this is the case for some block-oriented networks with no feedback loops and where all the nonlinear modules are polynomials. Consequently, the proposed method can be applied in situations considered challenging by current approaches. The performance of the estimation method is illustrated on a numerical simulation example.
|
|
| 9. |
- Abdalmoaty, Mohamed, 1986-, et al.
(författare)
-
Simulated Pseudo Maximum Likelihood Identification of Nonlinear Models
- 2017
-
Ingår i: The 20th IFAC World Congress. - : Elsevier. ; 50:1, s. 14058-14063
-
Konferensbidrag (refereegranskat)abstract
- Nonlinear stochastic parametric models are widely used in various fields. However, for these models, the problem of maximum likelihood identification is very challenging due to the intractability of the likelihood function. Recently, several methods have been developed to approximate the analytically intractable likelihood function and compute either the maximum likelihood or a Bayesian estimator. These methods, albeit asymptotically optimal, are computationally expensive. In this contribution, we present a simulation-based pseudo likelihood estimator for nonlinear stochastic models. It relies only on the first two moments of the model, which are easy to approximate using Monte-Carlo simulations on the model. The resulting estimator is consistent and asymptotically normal. We show that the pseudo maximum likelihood estimator, based on a multivariate normal family, solves a prediction error minimization problem using a parameterized norm and an implicit linear predictor. In the light of this interpretation, we compare with the predictor defined by an ensemble Kalman filter. Although not identical, simulations indicate a close relationship. The performance of the simulated pseudo maximum likelihood method is illustrated in three examples. They include a challenging state-space model of dimension 100 with one output and 2 unknown parameters, as well as an application-motivated model with 5 states, 2 outputs and 5 unknown parameters.
|
|
| 10. |
- Annergren, Mariette, 1982-
(författare)
-
Application-Oriented Input Design and Optimization Methods Involving ADMM
- 2016
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis is divided into two main parts. The first part considers application-oriented input design, specifically for model predictive control (MPC). The second part considers alternating direction method of multipliers (ADMM) for ℓ1 regularized optimization problems and primal-dual interior-point methods.The theory of system identification provides methods for estimating models of dynamical systems from experimental data. This thesis is focused on identifying models used for control, with special attention to MPC. The objective is to minimize the cost of the identification experiment while guaranteeing, with high probability, that the obtained model gives an acceptable control performance. We use application-oriented input design to find such a model. We present a general procedure of implementing application-oriented input design to unknown, possibly nonlinear, systems controlled using MPC. The practical aspects of application-oriented input design are addressed and the method is tested in an experimental study.In addition, a MATLAB-based toolbox for solving application-oriented input design problems is presented. The purpose of the toolbox is threefold: it is used in research; it facilitates communication of research results; it helps an engineer to use application-oriented input design.Several important problems in science can be formulated as convex optimization problems. As such, there exist very efficient algorithms for finding the solutions. We are interested in methods that can handle optimization problems with a very large number of variables. ADMM is a method capable of handling such problems. We derive a scalable and efficient algorithm based on ADMM for two ℓ1 regularized optimization problems: ℓ1 mean and covariance filtering, and ℓ1 regularized MPC. The former occurs in signal processing and the latter is a specific type of model based control.We are also interested in optimization problems with certain structural limitations. These limitations inhibit the use of a central computational unit to solve the problems. We derive a distributed method for solving them instead. The method is a primal-dual interior-point method that uses ADMM to distribute all the calculations necessary to solve the optimization problem at hand.
|
|
