| 1. |
- Abdalmoaty, Mohamed, 1986-
(författare)
-
Learning Stochastic Nonlinear Dynamical Systems Using Non-stationary Linear Predictors
- 2017
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The estimation problem of stochastic nonlinear parametric models is recognized to be very challenging due to the intractability of the likelihood function. Recently, several methods have been developed to approximate the maximum likelihood estimator and the optimal mean-square error predictor using Monte Carlo methods. Albeit asymptotically optimal, these methods come with several computational challenges and fundamental limitations.The contributions of this thesis can be divided into two main parts. In the first part, approximate solutions to the maximum likelihood problem are explored. Both analytical and numerical approaches, based on the expectation-maximization algorithm and the quasi-Newton algorithm, are considered. While analytic approximations are difficult to analyze, asymptotic guarantees can be established for methods based on Monte Carlo approximations. Yet, Monte Carlo methods come with their own computational difficulties; sampling in high-dimensional spaces requires an efficient proposal distribution to reduce the number of required samples to a reasonable value.In the second part, relatively simple prediction error method estimators are proposed. They are based on non-stationary one-step ahead predictors which are linear in the observed outputs, but are nonlinear in the (assumed known) input. These predictors rely only on the first two moments of the model and the computation of the likelihood function is not required. Consequently, the resulting estimators are defined via analytically tractable objective functions in several relevant cases. It is shown that, under mild assumptions, the estimators are consistent and asymptotically normal. In cases where the first two moments are analytically intractable due to the complexity of the model, it is possible to resort to vanilla Monte Carlo approximations. Several numerical examples demonstrate a good performance of the suggested estimators in several cases that are usually considered challenging.
|
|
| 2. |
- Anubhab, Ghosh, et al.
(författare)
-
DeepBayes -- an estimator for parameter estimation in stochastic nonlinear dynamical models
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Stochastic nonlinear dynamical systems are ubiquitous in modern, real-world applications. Yet, estimating the unknown parameters of stochastic, nonlinear dynamical models remains a challenging problem. The majority of existing methods employ maximum likelihood or Bayesian estimation. However, these methods suffer from some limitations, most notably the substantial computational time for inference coupled with limited flexibility in application. In this work, we propose DeepBayes estimators that leverage the power of deep recurrent neural networks in learning an estimator. The method consists of first training a recurrent neural network to minimize the mean-squared estimation error over a set of synthetically generated data using models drawn from the model set of interest. The a priori trained estimator can then be used directly for inference by evaluating the network with the estimation data. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We experiment with two popular recurrent neural networks -- long short term memory network (LSTM) and gated recurrent unit (GRU). We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches. We also provide a study on a real-world nonlinear benchmark problem. The experimental evaluations show that the proposed approach is asymptotically as good as the Bayes estimator.
|
|
| 3. |
- Ghosh, Anubhab, et al.
(författare)
-
DeepBayes—An estimator for parameter estimation in stochastic nonlinear dynamical models
- 2024
-
Ingår i: Automatica. - : Elsevier Ltd. - 0005-1098 .- 1873-2836. ; 159
-
Tidskriftsartikel (refereegranskat)abstract
- Stochastic nonlinear dynamical systems are ubiquitous in modern, real-world applications. Yet, estimating the unknown parameters of stochastic, nonlinear dynamical models remains a challenging problem. The majority of existing methods employ maximum likelihood or Bayesian estimation. However, these methods suffer from some limitations, most notably the substantial computational time for inference coupled with limited flexibility in application. In this work, we propose DeepBayes estimators that leverage the power of deep recurrent neural networks. The method consists of first training a recurrent neural network to minimize the mean-squared estimation error over a set of synthetically generated data using models drawn from the model set of interest. The a priori trained estimator can then be used directly for inference by evaluating the network with the estimation data. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We experiment with two popular recurrent neural networks — long short term memory network (LSTM) and gated recurrent unit (GRU). We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches. We also provide a study on a real-world nonlinear benchmark problem. The experimental evaluations show that the proposed approach is asymptotically as good as the Bayes estimator.
|
|
| 4. |
- Sjöberg, Jonas, 1964, et al.
(författare)
-
A System, Signals and Identification Toolbox in {M}athematica with Symbolic Capabilities
- 2009
-
Ingår i: 15th IFAC Symposium on System Identification, SYSID 2009. - 1474-6670. - 9783902661470 ; , s. 747-751
-
Konferensbidrag (refereegranskat)abstract
- In this contribution we describe a new signals, systems and identification toolbox for the symbolic and numerical computation system Mathematica. The toolbox provides functionality for computation of properties of systems and signals ranging from frequency responses, zeros and poles to signal spectra and spectral factorizations. It also includes a wide range of identification algorithms ranging from spectral analysis to subspace and prediction error identification of models for non-linear systems. The symbolic capabilities of Mathematica are used to allow the user to construct very general model structures, and for pre-processing, such as gradient calculations, when optimizing the parameters in such structures.
|
|
| 5. |
- Ramazi, Pouria, et al.
(författare)
-
Variance analysis of identified linear MISO models having spatially correlated inputs, with application to parallel Hammerstein models
- 2014
-
Ingår i: Automatica. - : Elsevier BV. - 0005-1098 .- 1873-2836. ; 50:6, s. 1675-1683
-
Tidskriftsartikel (refereegranskat)abstract
- This contribution concerns variance analysis of linear multi-input single-output models when the inputs are temporally white but where different inputs may be correlated. An expression is provided for the variance of a linearly parametrized estimate of the frequency response function from one block, i.e. from one input to the output. In particular, this expression reveals that the variance increases in one block when the number of estimated parameters in another block is increased, but levels off when the number of parameters in the other block reaches the number of parameters in the block in question. It also quantifies exactly how correlation between inputs affects the resulting accuracy and a graphical representation is provided for this purpose. The results are applicable to parallel MISO Hammerstein models when the nonlinearities are known and generalize an existing variance expression for this type of model.
|
|
| 6. |
- Risuleo, Riccardo Sven, 1986-, et al.
(författare)
-
Semi-parametric kernel-based identification of Wiener systems
- 2018
-
Ingår i: 2018 IEEE Conference on Decision and Control (CDC). - Piscataway, NJ : Institute of Electrical and Electronics Engineers (IEEE). - 9781538613955 ; , s. 3874-3879
-
Konferensbidrag (refereegranskat)abstract
- We present a technique for kernel-based identification of Wiener systems. We model the impulse response of the linear block with a Gaussian process. The static nonlinearity is modeled with a combination of basis functions. The coefficients of the static nonlinearity are estimated, together with the hyperparameters of the covariance function of the Gaussian process model, using an iterative algorithm based on the expectation-maximization method combined with elliptical slice sampling to sample from the posterior distribution of the impulse response given the data. The same sampling method is then used to find the posterior-mean estimate of the impulse response. We test the proposed algorithm on a benchmark of randomly-generated Wiener systems.
|
|
| 7. |
- Rojas, Cristian R., et al.
(författare)
-
Sparse Estimation of Polynomial and Rational Dynamical Models
- 2014
-
Ingår i: IEEE Transactions on Automatic Control. - 0018-9286 .- 1558-2523. ; 59:11, s. 2962-2977
-
Tidskriftsartikel (refereegranskat)abstract
- In many practical situations, it is highly desirable to estimate an accurate mathematical model of a real system using as few parameters as possible. At the same time, the need for an accurate description of the system behavior without knowing its complete dynamical structure often leads to model parameterizations describing a rich set of possible hypotheses; an unavoidable choice, which suggests sparsity of the desired parameter estimate. An elegant way to impose this expectation of sparsity is to estimate the parameters by penalizing the criterion with the l(0) "norm" of the parameters. Due to the non-convex nature of the l(0)-norm, this penalization is often implemented as solving an optimization program based on a convex relaxation (e. g., l(1)/LASSO, nuclear norm, ...). Two difficulties arise when trying to apply these methods: (1) the need to use cross-validation or some related technique for choosing the values of regularization parameters associated with the l(1) penalty; and (2) the requirement that the (unpenalized) cost function must be convex. To address the first issue, we propose a new technique for sparse linear regression called SPARSEVA, with close ties with the LASSO (least absolute shrinkage and selection operator), which provides an automatic tuning of the amount of regularization. The second difficulty, which imposes a severe constraint on the types of model structures or estimation methods on which the l(1) relaxation can be applied, is addressed by combining SPARSEVA and the Steiglitz-McBride method. To demonstrate the advantages of the proposed approach, a solid theoretical analysis and an extensive simulation study are provided.
|
|
| 8. |
- Abdalmoaty, Mohamed R., 1986-, et al.
(författare)
-
Application of a Linear PEM Estimator to a Stochastic Wiener-Hammerstein Benchmark Problem⁎
- 2018
-
Ingår i: IFAC-PapersOnLine. - : Elsevier B.V.. - 2405-8963. ; 51:15, s. 784-789
-
Tidskriftsartikel (refereegranskat)abstract
- The estimation problem of stochastic Wiener-Hammerstein models is recognized to be challenging, mainly due to the analytical intractability of the likelihood function. In this contribution, we apply a computationally attractive prediction error method estimator to a real-data stochastic Wiener-Hammerstein benchmark problem. The estimator is defined using a deterministic predictor that is nonlinear in the input. The prediction error method results in tractable expressions, and Monte Carlo approximations are not necessary. This allows us to tackle several issues considered challenging from the perspective of the current mainstream approach. Under mild conditions, the estimator can be shown to be consistent and asymptotically normal. The results of the method applied to the benchmark data are presented and discussed.
|
|
| 9. |
- Abdalmoaty, Mohamed Rasheed, 1986-, et al.
(författare)
-
Consistent Estimators of Stochastic MIMO Wiener Models based on Suboptimal Predictors
- 2018
-
Ingår i: 2018 IEEE Conference on Decision and Control (CDC). - : IEEE. - 9781538613955 - 9781538613948 - 9781538613962 ; , s. 3842-3847
-
Konferensbidrag (refereegranskat)abstract
- We consider a parameter estimation problem in a general class of stochastic multiple-inputs multiple-outputs Wiener models, where the likelihood function is, in general, analytically intractable. When the output signal is a scalar independent stochastic process, the likelihood function of the parameters is given by a product of scalar integrals. In this case, numerical integration may be efficiently used to approximately solve the maximum likelihood problem. Otherwise, the likelihood function is given by a challenging multidimensional integral. In this contribution, we argue that by ignoring the temporal and spatial dependence of the stochastic disturbances, a computationally attractive estimator based on a suboptimal predictor can be constructed by evaluating scalar integrals regardless of the number of outputs. Under some conditions, the convergence of the resulting estimators can be established and consistency is achieved under certain identifiability hypothesis. We highlight the relationship between the resulting estimators and a recently proposed prediction error method estimator. We also remark that the method can be used for a wider class of stochastic nonlinear models. The performance of the method is demonstrated by a numerical simulation example using a 2-inputs 2-outputs model with 9 parameters.
|
|
| 10. |
- Abdalmoaty, Mohamed R., 1986-, et al.
(författare)
-
Identification of a Class of Nonlinear Dynamical Networks⁎
- 2018
-
Ingår i: IFAC-PapersOnLine. - : Elsevier B.V.. - 2405-8963. ; 51:15, s. 868-873
-
Tidskriftsartikel (refereegranskat)abstract
- Identification of dynamic networks has attracted considerable interest recently. So far the main focus has been on linear time-invariant networks. Meanwhile, most real-life systems exhibit nonlinear behaviors; consider, for example, two stochastic linear time-invariant systems connected in series, each of which has a nonlinearity at its output. The estimation problem in this case is recognized to be challenging, due to the analytical intractability of both the likelihood function and the optimal one-step ahead predictors of the measured nodes. In this contribution, we introduce a relatively simple prediction error method that may be used for the estimation of nonlinear dynamical networks. The estimator is defined using a deterministic predictor that is nonlinear in the known signals. The estimation problem can be defined using closed-form analytical expressions in several non-trivial cases, and Monte Carlo approximations are not necessarily required. We show, that this is the case for some block-oriented networks with no feedback loops and where all the nonlinear modules are polynomials. Consequently, the proposed method can be applied in situations considered challenging by current approaches. The performance of the estimation method is illustrated on a numerical simulation example.
|
|
