| 1. |
- Adib Yaghmaie, Farnaz, et al.
(författare)
-
Linear Quadratic Control Using Model-Free Reinforcement Learning
- 2023
-
Ingår i: IEEE Transactions on Automatic Control. - : IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC. - 0018-9286 .- 1558-2523. ; 68:2, s. 737-752
-
Tidskriftsartikel (refereegranskat)abstract
- In this article, we consider linear quadratic (LQ) control problem with process and measurement noises. We analyze the LQ problem in terms of the average cost and the structure of the value function. We assume that the dynamics of the linear system is unknown and only noisy measurements of the state variable are available. Using noisy measurements of the state variable, we propose two model-free iterative algorithms to solve the LQ problem. The proposed algorithms are variants of policy iteration routine where the policy is greedy with respect to the average of all previous iterations. We rigorously analyze the properties of the proposed algorithms, including stability of the generated controllers and convergence. We analyze the effect of measurement noise on the performance of the proposed algorithms, the classical off-policy, and the classical Q-learning routines. We also investigate a model-building approach, inspired by adaptive control, where a model of the dynamical system is estimated and the optimal control problem is solved assuming that the estimated model is the true model. We use a benchmark to evaluate and compare our proposed algorithms with the classical off-policy, the classical Q-learning, and the policy gradient. We show that our model-building approach performs nearly identical to the analytical solution and our proposed policy iteration based algorithms outperform the classical off-policy and the classical Q-learning algorithms on this benchmark but do not outperform the model-building approach.
|
|
| 2. |
- Adib Yaghmaie, Farnaz, et al.
(författare)
-
Using Reinforcement Learning for Model-free Linear Quadratic Control with Process and Measurement Noises
- 2019
-
Ingår i: 2019 IEEE 58TH CONFERENCE ON DECISION AND CONTROL (CDC). - : IEEE. - 9781728113982 - 9781728113999 ; , s. 6510-6517
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we analyze a Linear Quadratic (LQ) control problem in terms of the average cost and the structure of the value function. We develop a completely model-free reinforcement learning algorithm to solve the LQ problem. Our algorithm is an off-policy routine where each policy is greedy with respect to all previous value functions. We prove that the algorithm produces stable policies given that the estimation errors remain small. Empirically, our algorithm outperforms the classical Q and off-policy learning routines.
|
|
| 3. |
- Alickovic, Emina, et al.
(författare)
-
A System Identification Approach to Determining Listening Attention from EEG Signals
- 2016
-
Ingår i: 2016 24TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO). - : IEEE. - 9780992862657 - 9781509018918 ; , s. 31-35
-
Konferensbidrag (refereegranskat)abstract
- We still have very little knowledge about how ourbrains decouple different sound sources, which is known assolving the cocktail party problem. Several approaches; includingERP, time-frequency analysis and, more recently, regression andstimulus reconstruction approaches; have been suggested forsolving this problem. In this work, we study the problem ofcorrelating of EEG signals to different sets of sound sources withthe goal of identifying the single source to which the listener isattending. Here, we propose a method for finding the number ofparameters needed in a regression model to avoid overlearning,which is necessary for determining the attended sound sourcewith high confidence in order to solve the cocktail party problem.
|
|
| 4. |
- Alickovic, Emina, et al.
(författare)
-
A Tutorial on Auditory Attention Identification Methods
- 2019
-
Ingår i: Frontiers in Neuroscience. - : FRONTIERS MEDIA SA. - 1662-4548 .- 1662-453X. ; 13
-
Tidskriftsartikel (refereegranskat)abstract
- Auditory attention identification methods attempt to identify the sound source of a listeners interest by analyzing measurements of electrophysiological data. We present a tutorial on the numerous techniques that have been developed in recent decades, and we present an overview of current trends in multivariate correlation-based and model-based learning frameworks. The focus is on the use of linear relations between electrophysiological and audio data. The way in which these relations are computed differs. For example, canonical correlation analysis (CCA) finds a linear subset of electrophysiological data that best correlates to audio data and a similar subset of audio data that best correlates to electrophysiological data. Model-based (encoding and decoding) approaches focus on either of these two sets. We investigate the similarities and differences between these linear model philosophies. We focus on (1) correlation-based approaches (CCA), (2) encoding/decoding models based on dense estimation, and (3) (adaptive) encoding/decoding models based on sparse estimation. The specific focus is on sparsity-driven adaptive encoding models and comparing the methodology in state-of-the-art models found in the auditory literature. Furthermore, we outline the main signal processing pipeline for how to identify the attended sound source in a cocktail party environment from the raw electrophysiological data with all the necessary steps, complemented with the necessary MATLAB code and the relevant references for each step. Our main aim is to compare the methodology of the available methods, and provide numerical illustrations to some of them to get a feeling for their potential. A thorough performance comparison is outside the scope of this tutorial.
|
|
| 5. |
- Ancuti, Codruta O., et al.
(författare)
-
NTIRE 2023 HR NonHomogeneous Dehazing Challenge Report
- 2023
-
Ingår i: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). - Vancover : Institute of Electrical and Electronics Engineers (IEEE).
-
Konferensbidrag (refereegranskat)abstract
- This study assesses the outcomes of the NTIRE 2023 Challenge on Non-Homogeneous Dehazing, wherein novel techniques were proposed and evaluated on new image dataset called HD-NH-HAZE. The HD-NH-HAZE dataset contains 50 high resolution pairs of real-life outdoor images featuring nonhomogeneous hazy images and corresponding haze-free images of the same scene. The nonhomogeneous haze was simulated using a professional setup that replicated real-world conditions of hazy scenarios. The competition had 246 participants and 17 teams that competed in the final testing phase, and the proposed solutions demonstrated the cutting-edge in image dehazing technology.
|
|
| 6. |
|
|
| 7. |
- Andersson, Maria, et al.
(författare)
-
Recognition of Anomalous Motion Patterns in Urban Surveillance
- 2013
-
Ingår i: IEEE Journal on Selected Topics in Signal Processing. - : IEEE Signal Processing Society. - 1932-4553 .- 1941-0484. ; 7:1, s. 102-110
-
Tidskriftsartikel (refereegranskat)abstract
- We investigate the unsupervised K-means clustering and the semi-supervised hidden Markov model (HMM) to automatically detect anomalous motion patterns in groups of people (crowds). Anomalous motion patterns are typically people merging into a dense group, followed by disturbances or threatening situations within the group. The application of K-means clustering and HMM are illustrated with datasets from four surveillance scenarios. The results indicate that by investigating the group of people in a systematic way with different K values, analyze cluster density, cluster quality and changes in cluster shape we can automatically detect anomalous motion patterns. The results correspond well with the events in the datasets. The results also indicate that very accurate detections of the people in the dense group would not be necessary. The clustering and HMM results will be very much the same also with some increased uncertainty in the detections.
|
|
| 8. |
- Ardeshiri, Tohid
(författare)
-
Analytical Approximations for Bayesian Inference
- 2015
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Bayesian inference is a statistical inference technique in which Bayes’ theorem is used to update the probability distribution of a random variable using observations. Except for few simple cases, expression of such probability distributions using compact analytical expressions is infeasible. Approximation methods are required to express the a priori knowledge about a random variable in form of prior distributions. Further approximations are needed to compute posterior distributions of the random variables using the observations. When the computational complexity of representation of such posteriors increases over time as in mixture models, approximations are required to reduce the complexity of such representations.This thesis further extends existing approximation methods for Bayesian inference, and generalizes the existing approximation methods in three aspects namely; prior selection, posterior evaluation given the observations and maintenance of computation complexity.Particularly, the maximum entropy properties of the first-order stable spline kernel for identification of linear time-invariant stable and causal systems are shown. Analytical approximations are used to express the prior knowledge about the properties of the impulse response of a linear time-invariant stable and causal system.Variational Bayes (VB) method is used to compute an approximate posterior in two inference problems. In the first problem, an approximate posterior for the state smoothing problem for linear statespace models with unknown and time-varying noise covariances is proposed. In the second problem, the VB method is used for approximate inference in state-space models with skewed measurement noise.Moreover, a novel approximation method for Bayesian inference is proposed. The proposed Bayesian inference technique is based on Taylor series approximation of the logarithm of the likelihood function. The proposed approximation is devised for the case where the prior distribution belongs to the exponential family of distributions.Finally, two contributions are dedicated to the mixture reduction (MR) problem. The first contribution, generalize the existing MR algorithms for Gaussian mixtures to the exponential family of distributions and compares them in an extended target tracking scenario. The second contribution, proposes a new Gaussian mixture reduction algorithm which minimizes the reverse Kullback-Leibler divergence and has specific peak preserving properties.
|
|
| 9. |
- Ardeshiri, Tohid, et al.
(författare)
-
Approximate Bayesian Smoothing with Unknown Process and Measurement Noise Covariances
- 2015
-
Ingår i: IEEE Signal Processing Letters. - : Institute of Electrical and Electronics Engineers (IEEE). - 1070-9908 .- 1558-2361. ; 22:12, s. 2450-2454
-
Tidskriftsartikel (refereegranskat)abstract
- We present an adaptive smoother for linear state-space models with unknown process and measurement noise covariances. The proposed method utilizes the variational Bayes technique to perform approximate inference. The resulting smoother is computationally efficient, easy to implement, and can be applied to high dimensional linear systems. The performance of the algorithm is illustrated on a target tracking example.
|
|
| 10. |
- Ardeshiri, Tohid, et al.
(författare)
-
Bayesian Inference via Approximation of Log-likelihood for Priors in Exponential Family
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- In this paper, a Bayesian inference technique based on Taylor series approximation of the logarithm of the likelihood function is presented. The proposed approximation is devised for the case where the prior distribution belongs to the exponential family of distributions. The logarithm of the likelihood function is linearized with respect to the sufficient statistic of the prior distribution in exponential family such that the posterior obtains the same exponential family form as the prior. Similarities between the proposed method and the extended Kalman filter for nonlinear filtering are illustrated. Further, an extended target measurement update for target models where the target extent is represented by a random matrix having an inverse Wishart distribution is derived. The approximate update covers the important case where the spread of measurement is due to the target extent as well as the measurement noise in the sensor.
|
|
