| 1. |
- Alhashimi, Anas, 1978-, et al.
(författare)
-
Bayesian strategies for calibrating heteroskedastic static sensors with unknown model structures
- 2018
-
Ingår i: 2018 European Control Conference (ECC). - Piscataway, NJ : IEEE. - 9783952426982 - 9781538653036 ; , s. 2447-2453
-
Konferensbidrag (refereegranskat)abstract
- This paper investigates the problem of calibrating sensors affected by (i) heteroskedastic measurement noise and (ii) a polynomial bias, describing a systematic distortion of the measured quantity. First, a set of increasingly complex statistical models for the measurement process was proposed. Then, for each model the authors design a Bayesian parameters estimation method handling heteroskedasticity and capable to exploit prior information about the model parameters. The Bayesian problem is solved using MCMC methods and reconstructing the unknown parameters posterior in sampled form. The authors then test the proposed techniques on a practically relevant case study, the calibration of Light Detection and Ranging (Lidar) sensor, and evaluate the different proposed procedures using both artificial and field data.
|
|
| 2. |
- Aravkin, Aleksandr, et al.
(författare)
-
Generalized Kalman smoothing: Modeling and algorithms
- 2017
-
Ingår i: Automatica. - : PERGAMON-ELSEVIER SCIENCE LTD. - 0005-1098 .- 1873-2836. ; 86, s. 63-86
-
Tidskriftsartikel (refereegranskat)abstract
- State-space smoothing has found many applications in science and engineering. Under linear and Gaussian assumptions, smoothed estimates can be obtained using efficient recursions, for example Rauch Tung Striebel and Mayne Fraser algorithms. Such schemes are equivalent to linear algebraic techniques that minimize a convex quadratic objective function with structure induced by the dynamic model. These classical formulations fall short in many important circumstances. For instance, smoothers obtained using quadratic penalties can fail when outliers are present in the data, and cannot track impulsive inputs and abrupt state changes. Motivated by these shortcomings, generalized Kalman smoothing formulations have been proposed in the last few years, replacing quadratic models with more suitable, often nonsmooth, convex functions. In contrast to classical models, these general estimators require use of iterated algorithms, and these have received increased attention from control, signal processing, machine learning, and optimization communities. In this survey we show that the optimization viewpoint provides the control and signal processing community great freedom in the development of novel modeling and inference frameworks for dynamical systems. We discuss general statistical models for dynamic systems, making full use of nonsmooth convex penalties and constraints, and providing links to important models in signal processing and machine learning. We also survey optimization techniques for these formulations, paying close attention to dynamic problem structure. Modeling concepts and algorithms are illustrated with numerical examples. (C) 2017 Elsevier Ltd. All rights reserved.
|
|
| 3. |
- Bottegal, Giulio, et al.
(författare)
-
A new kernel-based approach to system identification with quantized output data
- 2017
-
Ingår i: Automatica. - : Elsevier BV. - 0005-1098 .- 1873-2836. ; 85, s. 145-152
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper we introduce a novel method for linear system identification with quantized output data. We model the impulse response as a zero-mean Gaussian process whose covariance (kernel) is given by the recently proposed stable spline kernel, which encodes information on regularity and exponential stability. This serves as a starting point to cast our system identification problem into a Bayesian framework. We employ Markov Chain Monte Carlo methods to provide an estimate of the system. In particular, we design two methods based on the so-called Gibbs sampler that allow also to estimate the kernel hyperparameters by marginal likelihood maximization via the expectation-maximization method. Numerical simulations show the effectiveness of the proposed scheme, as compared to the state-of-the-art kernel-based methods when these are employed in system identification with quantized data. (C) 2017 Elsevier Ltd. All rights reserved.
|
|
| 4. |
- Bottegal, Giulio, et al.
(författare)
-
Robust EM kernel-based methods for linear system identification
- 2016
-
Ingår i: Automatica. - : Elsevier BV. - 0005-1098 .- 1873-2836. ; 67, s. 114-126
-
Tidskriftsartikel (refereegranskat)abstract
- Recent developments in system identification have brought attention to regularized kernel-based methods. This type of approach has been proven to compare favorably with classic parametric methods. However, current formulations are not robust with respect to outliers. In this paper, we introduce a novel method to robustify kernel-based system identification methods. To this end, we model the output measurement noise using random variables with heavy-tailed probability density functions (pdfs), focusing on the Laplacian and the Student's t distributions. Exploiting the representation of these pdfs as scale mixtures of Gaussians, we cast our system identification problem into a Gaussian process regression framework, which requires estimating a number of hyperparameters of the data size order. To overcome this difficulty, we design a new maximum a posteriori (MAP) estimator of the hyperparameters, and solve the related optimization problem with a novel iterative scheme based on the Expectation-Maximization (EM) method. In the presence of outliers, tests on simulated data and on a real system show a substantial performance improvement compared to currently used kernel-based methods for linear system identification. (C) 2016 Elsevier Ltd. All rights reserved.
|
|
| 5. |
- Chen, Tianshi, et al.
(författare)
-
Continuous-time DC kernel - a stable generalized first order spline kernel
- 2016
-
Ingår i: 2016 IEEE 55TH CONFERENCE ON DECISION AND CONTROL (CDC). - : IEEE. - 9781509018376 ; , s. 4647-4652
-
Konferensbidrag (refereegranskat)abstract
- The stable spline kernel and the diagonal correlated kernel are two kernels that have been tested extensively in kernel-based regularization methods for LTI system identification. As shown in our recent works, although these two kernels are introduced in different ways, they share some common features, e.g., they all belong to the class of exponentially convex locally stationary kernels, and state-space model induced kernels. In this work, we further show that similar to the derivation of the stable spline kernel, the continuous-time diagonal correlated kernel can be derived by applying the same "stable" coordinate change to a "generalized" first order spline kernel, and thus can be interpreted as a stable generalized first order spline kernel. This interpretation provides new facets to understand the properties of the diagonal correlated kernel. Due to this interpretation, new eigendecompositions, explicit expression of the norm, and new maximum entropy interpretation of the diagonal correlated kernel are derived accordingly.
|
|
| 6. |
- Chen, Tianshi, et al.
(författare)
-
Maximum entropy properties of discrete-time first-order stable spline kernel
- 2016
-
Ingår i: Automatica. - : Elsevier BV. - 0005-1098 .- 1873-2836. ; 66, s. 34-38
-
Tidskriftsartikel (refereegranskat)abstract
- The first order stable spline (SS-1) kernel (also known as the tunedcorrelated kernel) is used extensively in regularized system identification, where the impulse response is modeled as a zero-mean Gaussian process whose covariance function is given by well designed and tuned kernels. In this paper, we discuss the maximum entropy properties of this kernel. In particular, we formulate the exact maximum entropy problem solved by the SS-1 kernel without Gaussian and uniform sampling assumptions. Under general sampling assumption, we also derive the special structure of the SS-1 kernel (e.g. its tridiagonal inverse and factorization have closed form expression), also giving to it a maximum entropy covariance completion interpretation.
|
|
| 7. |
- Chen, Tianshi, et al.
(författare)
-
Sparse multiple kernels for impulse response estimation with majorization minimization algorithms
- 2012
-
Ingår i: Decision and Control (CDC), 2012. - : IEEE. - 9781467320658 - 9781467320641 ; , s. 1500-1505
-
Konferensbidrag (refereegranskat)abstract
- This contribution aims to enrich the recently introduced kernel-based regularization method for linear system identification. Instead of a single kernel, we use multiple kernels, which can be instances of any existing kernels for the impulse response estimation of linear systems. We also introduce a new class of kernels constructed based on output error (OE) model estimates. In this way, a more flexible and richer representation of the kernel is obtained. Due to this representation the associated hyper-parameter estimation problem has two good features. First, it is a difference of convex functions programming (DCP) problem. While it is still nonconvex, it can be transformed into a sequence of convex optimization problems with majorization minimization (MM) algorithms and a local minima can thus be found iteratively. Second, it leads to sparse hyper-parameters and thus sparse multiple kernels. This feature shows the kernel-based regularization method with multiple kernels has the potential to tackle various problems of finding sparse solutions in linear system identification.
|
|
| 8. |
- Chen, Tianshi, et al.
(författare)
-
Spectral analysis of the DC kernel for regularized system identification
- 2015
-
Ingår i: 2015 54TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC). - : IEEE. - 9781479978861 ; , s. 4017-4022
-
Konferensbidrag (refereegranskat)abstract
- System identification with regularization methods has attracted increasing attention recently and is a complement to the current standard maximum likelihood/ prediction error method. In this paper, we focus on the kernel-based regularization method and give a spectral analysis of the so-called diagonal correlated (DC) kernel, one family of kernel structures that has been proven useful for linear time-invariant system identification. In particular, using the theory of Bessel functions, we derive the eigenvalues and corresponding eigenfunctions of the DC kernel. Accordingly, we derive the Karhunen-Loeve expansion of the stochastic process whose covariance function is the DC kernel.
|
|
| 9. |
- Chen, Tianshi, et al.
(författare)
-
System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques
- 2014
-
Ingår i: IEEE Transactions on Automatic Control. - : Institute of Electrical and Electronics Engineers (IEEE). - 0018-9286 .- 1558-2523. ; 59:11, s. 2933-2945
-
Tidskriftsartikel (refereegranskat)abstract
- Model estimation and structure detection with short data records are two issues that receive increasing interests in System Identification. In this paper, a multiple kernel-based regularization method is proposed to handle those issues. Multiple kernels are conic combinations of fixed kernels suitable for impulse response estimation, and equip the kernel-based regularization method with three features. First, multiple kernels can better capture complicated dynamics than single kernels. Second, the estimation of their weights by maximizing the marginal likelihood favors sparse optimal weights, which enables this method to tackle various structure detection problems, e. g., the sparse dynamic network identification and the segmentation of linear systems. Third, the marginal likelihood maximization problem is a difference of convex programming problem. It is thus possible to find a locally optimal solution efficiently by using a majorization minimization algorithm and an interior point method where the cost of a single interior-point iteration grows linearly in the number of fixed kernels. Monte Carlo simulations show that the locally optimal solutions lead to good performance for randomly generated starting points.
|
|
| 10. |
- Pillonetto, Gianluigi, et al.
(författare)
-
Distributed multi-agent Gaussian regression via finite-dimensional approximations
- 2019
-
Ingår i: IEEE Transactions on Pattern Analysis and Machine Intelligence. - : IEEE. - 0162-8828 .- 1939-3539. ; 41:9, s. 2098-2111
-
Tidskriftsartikel (refereegranskat)abstract
- We consider the problem of distributedly estimating Gaussian processes in multi-agent frameworks. Each agent collects few measurements and aims to collaboratively reconstruct a common estimate based on all data. Agents are assumed with limited computational and communication capabilities and to gather M noisy measurements in total on input locations independently drawn from a known common probability density. The optimal solution would require agents to exchange all the M input locations and measurements and then invert an M×M matrix, a non-scalable task. Differently, we propose two suboptimal approaches using the first E orthonormal eigenfunctions obtained from the Karhunen-Loève (KL) expansion of the chosen kernel, where typically E≪M. The benefits are that the computation and communication complexities scale with E and not with M, and computing the required statistics can be performed via standard average consensus algorithms. We obtain probabilistic non-asymptotic bounds that determine a priori the desired level of estimation accuracy, and new distributed strategies relying on Stein's unbiased risk estimate (SURE) paradigms for tuning the regularization parameters and applicable to generic basis functions (thus not necessarily kernel eigenfunctions) and that can again be implemented via average consensus. The proposed estimators and bounds are finally tested on both synthetic and real field data.
|
|
