| 1. |
- Gustafsson, Fredrik, et al.
(författare)
-
Asymptotic Properties of Just-in-Time Models
- 1997
-
Ingår i: Proceedings of the 11th IFAC Symposium on System Identification. - Linköping : Linköping University Electronic Press. - 0080425925 ; , s. 1249-1254
-
Konferensbidrag (refereegranskat)abstract
- The concept of Just-in-Time models has been introduced for models that are not estimated until they are really needed. The prediction is taken as a weighted average of neighboring points in the regressor space, such that an optimal bias/variance trade-off is achieved. The asymptotic properties of the method are investigated, and are compared to the corresponding properties of related statistical non-parametric kernel methods. It is shown that the rate of convergence for Just-in-Time models at least is in the same order as traditional kernel estimators, and that better rates probably can be achieved.
|
|
| 2. |
- Iouditski, Anatoli, et al.
(författare)
-
Adaptive DWO Estimator of a Regression Function
- 2007
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- We address a problem of non-parametric estimation of an unknown regression function f : [-1/2, 1/2] → R at a fixed point x0 € (-1/2, 1/2) on the basis of observations (xi, yi), i = 1,..,n such that yi = f(xi) + ei, where ei ~ N(0, σ2) is unobservable, Gaussian i.i.d. random noise and xi € [-1/2, 1/2] are given design points. Recently, the Direct Weight Optimization (DWO) method has been proposed to solve a problem of such kind. The properties of the method have been studied for the case when the unknown function f is continuously differentiable with Lipschitz constant L. The minimax optimality and adaptivity with respect to the design have been established for the resulting estimator. However, in order to implement the approach, both L and σ are to be known. The subject of the submission is the study of an adaptive version of DWO estimator which uses a data-driven choice of the method parameter L.
|
|
| 3. |
|
|
| 4. |
- Nazin, Alexander, et al.
(författare)
-
Direct Weight Optimization for Approximately Linear Functions : Optimality and Design
- 2007
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- The Direct Weight Optimization (DWO) approach to estimating a regression function is studied here for the class of approximately linear functions, i.e., functions whose deviation from an affine function is bounded by a known constant. Upper and lower bounds for the asymptotic maximum MSE are given, some of which also hold in the non-asymptotic case and for an arbitrary fixed design. Their coincidence is then studied. Particularly, under mild conditions, it can be shown that there is always an interval in which the DWO-optimal estimator is optimal among all estimators. Experiment design issues are also studied.
|
|
| 5. |
- Nazin, Alexander, et al.
(författare)
-
Direct Weight Optimization in Nonlinear Function Estimation and System Identification
- 2007
-
Ingår i: Proceedings of the 6th International Conference on System Identification and Control Problems (SICPRO '07). - Linköping : Linköping University Electronic Press.
-
Konferensbidrag (refereegranskat)abstract
- The Direct Weight Optimization (DWO) approach to estimating a regression function and its application to nonlinear system identification has been proposed and developed during the last few years by the authors. Computationally, the approach is typically reduced to a quadratic or conic programming and can be effectively realized. The obtained estimates demonstrate optimality or sub-optimality in a minimax sense w.r.t. the mean-square error criterion under weak design conditions. Here we describe the main ideas of the approach and represent an overview of the obtained results.
|
|
| 6. |
- Nazin, Alexander, et al.
(författare)
-
Direct Weight Optimization in Statistical Estimation and System Identification
- 2007
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- The Direct Weight Optimization (DWO) approach to statistical estimation and the application to nonlinear system identification has been proposed and developed during the last few years. Computationally, the approachis typically reduced to a convex (e.g., quadratic or conic) program, whichcan be solved efficiently. The optimality or sub-optimality of the obtained estimates, in a minimax sense w.r.t. the estimation error criterion, can be analyzed under weak a priori conditions. The main ideas of the approach are discussed here and an overview of the obtained results is presented.
|
|
| 7. |
- Nazin, Alexander V., et al.
(författare)
-
Asymptotically Optimal Smoothing of Stochastic Approximation Estimates for Regression Parameter Tracking
- 2001
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- The sequence of estimates formed by the LMS algorithm for a standard linear regression estimation problem are considered. It is known since earlier that smoothing these estimates by simple averaging will lead to, asymptotically, the recursive least squares algorithm. In this paper it is first shown that smoothing the LMS estimates using amatrix updating will lead to smoothed estimates with optimal tracking properties, also in the case the true parameters are changing as a random walk. The choice of smoothing matrix should be tailored to the properties of the random walk. Second, it is shown that the same accuracy can be obtained also for a simplified algorithm, SLAMS, which is based on averages and requires much less computations.
|
|
| 8. |
- Roll, Jacob, 1974-, et al.
(författare)
-
A General Direct Weight Optimization Framework for Nonlinear System Identification
- 2005
-
Ingår i: Proceedings of the 16th IFAC World Congress. - Linköping : Linköping University Electronic Press. - 9783902661753 ; , s. 29-29
-
Konferensbidrag (refereegranskat)abstract
- The direct weight optimization (DWO) approach is a method for finding optimal function estimates via convex optimization, applicable to nonlinear system identification. In this paper, an extended version of the DWO approach is introduced. A general function class description --- which includes several important special cases --- is presented, and different examples are given. A general theorem about the principal shape of the weights is also proven.
|
|
| 9. |
- Roll, Jacob, et al.
(författare)
-
A Non-Asymptotic Approach to Local Modelling
- 2003
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Local models and methods construct function estimates or predictions from observations in a local neighborhood of the point of interest. The bandwidth, i.e., how large the local neighborhood should be, is often determined based on asymptotic analysis. In this paper, an alternative, non-asymptotic approach that minimizes a uniform upper bound on the mean square error for a linear estimate is proposed. It is shown, for the scalar case, that the solution is obtained from a quadratic program, and that it maintains many of the key features of the asymptotic approaches. Moreover, examples show that the proposed approach in some cases is superior to an asymptotically based local linear estimator.
|
|
| 10. |
- Roll, Jacob, et al.
(författare)
-
Local Modelling of Nonlinear Dynamic Systems Using Direct Weight Optimization
- 2003
-
Ingår i: Proceedings of the 13th IFAC Symposium on System Identification. - Linköping : Linköping University Electronic Press. - 0080437095 ; , s. 1554-1559
-
Konferensbidrag (refereegranskat)abstract
- Local models and methods construct function estimates or predictions from observations in a local neighborhood of the point of interest. The bandwidth, i.e., how large the local neighborhood should be, is often determined based on asymptotic analysis. In this paper, an alternative, non-asymptotic approach that minimizes a uniform upper bound on the mean square error for a linear estimate is used. It is shown that the estimator is obtained from a quadratic program, that an automatic bandwidth selection is obtained, and that the approach can be seen as a local version of fitting affine models to data. Finally, the approach is applied to two benchmark systems.
|
|
