| 1. |
- Anubhab, Ghosh, et al.
(författare)
-
DeepBayes -- an estimator for parameter estimation in stochastic nonlinear dynamical models
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- Stochastic nonlinear dynamical systems are ubiquitous in modern, real-world applications. Yet, estimating the unknown parameters of stochastic, nonlinear dynamical models remains a challenging problem. The majority of existing methods employ maximum likelihood or Bayesian estimation. However, these methods suffer from some limitations, most notably the substantial computational time for inference coupled with limited flexibility in application. In this work, we propose DeepBayes estimators that leverage the power of deep recurrent neural networks in learning an estimator. The method consists of first training a recurrent neural network to minimize the mean-squared estimation error over a set of synthetically generated data using models drawn from the model set of interest. The a priori trained estimator can then be used directly for inference by evaluating the network with the estimation data. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We experiment with two popular recurrent neural networks -- long short term memory network (LSTM) and gated recurrent unit (GRU). We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches. We also provide a study on a real-world nonlinear benchmark problem. The experimental evaluations show that the proposed approach is asymptotically as good as the Bayes estimator.
|
|
| 2. |
- Anubhab, Ghosh, et al.
(författare)
-
Time-Varying Normalizing Flows for Dynamical Signals
- 2022
-
Konferensbidrag (refereegranskat)abstract
- We develop a time-varying normalizing flow (TVNF) for explicit generative modeling of dynamical signals. Being explicit, it can generate samples of dynamical signals, and compute the likelihood of a (given) dynamical signal sample. In the proposed model, signal flow in the layers of the normalizing flow is a function of time, which is realized using an encoded representation that is the output of a recurrent neural network (RNN). Given a set of dynamical signals, the parameters of TVNF are learned according to a maximum-likelihood approach in conjunction with gradient descent (backpropagation). Use of the proposed model is illustrated for a toy application scenario-maximum-likelihood based speech-phone classification task.
|
|
| 3. |
- Fontcuberta, Aleix Espuna, et al.
(författare)
-
Forecasting Solar Cycle 25 with Physical Model-Validated Recurrent Neural Networks
- 2023
-
Ingår i: Solar Physics. - : Springer Nature. - 0038-0938 .- 1573-093X. ; 298:1
-
Tidskriftsartikel (refereegranskat)abstract
- The Sun's activity, which is associated with the solar magnetic cycle, creates a dynamic environment in space known as space weather. Severe space weather can disrupt space-based and Earth-based technologies. Slow decadal-scale variations on solar-cycle timescales are important for radiative forcing of the Earth's atmosphere and impact satellite lifetimes and atmospheric dynamics. Predicting the solar magnetic cycle is therefore of critical importance for humanity. In this context, a novel development is the application of machine-learning algorithms for solar-cycle forecasting. Diverse approaches have been developed for this purpose; however, with no consensus across different techniques and physics-based approaches. Here, we first explore the performance of four different machine-learning algorithms - all of them belonging to a class called Recurrent Neural Networks (RNNs) - in predicting simulated sunspot cycles based on a widely studied, stochastically forced, nonlinear time-delay solar dynamo model. We conclude that the algorithm Echo State Network (ESN) performs the best, but predictability is limited to only one future sunspot cycle, in agreement with recent physical insights. Subsequently, we train the ESN algorithm and a modified version of it (MESN) with solar-cycle observations to forecast Cycles 22 - 25. We obtain accurate hindcasts for Solar Cycles 22 - 24. For Solar Cycle 25 the ESN algorithm forecasts a peak amplitude of 131 +/- 14 sunspots around July 2024 and indicates a cycle length of approximately 10 years. The MESN forecasts a peak of 137 +/- 2 sunspots around April 2024, with the same cycle length. Qualitatively, both forecasts indicate that Cycle 25 will be slightly stronger than Cycle 24 but weaker than Cycle 23. Our novel approach bridges physical model-based forecasts with machine-learning-based approaches, achieving consistency across these diverse techniques.
|
|
| 4. |
- Ghosh, Anubhab, et al.
(författare)
-
DANSE : Data-driven Non-linear State Estimation of Model-free Process in Unsupervised Bayesian Setup
- 2023
-
Ingår i: European Signal Processing Conference. - : Institute of Electrical and Electronics Engineers (IEEE). - 9789464593600 ; , s. 870-874
-
Konferensbidrag (refereegranskat)abstract
- We propose DANSE - a data-driven non-linear state estimation method. DANSE provides a closed-form posterior of the state of a model-free process, given linear measurements of the state in a Bayesian setup, like the celebrated Kalman filter (KF). Non-linear dynamics of the state are captured by data-driven recurrent neural networks (RNNs). The training of DANSE combines maximum-likelihood and gradient-descent in an unsupervised framework, i.e. only measurement data and no process data are required. Using simulated linear and non-linear process models, we demonstrate that DANSE - without knowledge of the process model - provides competitive performance against model-based approaches such as KF, unscented KF (UKF), extended KF (EKF), and a hybrid approach such as KalmanNet.
|
|
| 5. |
- Ghosh, Anubhab, et al.
(författare)
-
DANSE : Data-Driven Non-Linear State Estimation of Model-Free Process in Unsupervised Learning Setup
- 2024
-
Ingår i: IEEE Transactions on Signal Processing. - : Institute of Electrical and Electronics Engineers (IEEE). - 1053-587X .- 1941-0476. ; 72, s. 1824-1838
-
Tidskriftsartikel (refereegranskat)abstract
- We address the tasks of Bayesian state estimation and forecasting for a model-free process in an unsupervised learning setup. For a model-free process, we do not have any a-priori knowledge of the process dynamics. In the article, we propose DANSE - a Data-driven Nonlinear State Estimation method. DANSE provides a closed-form posterior of the state of the model-free process, given linear measurements of the state. In addition, it provides a closed-form posterior for forecasting. A data-driven recurrent neural network (RNN) is used in DANSE to provide the parameters of a prior of the state. The prior depends on the past measurements as input, and then we find the closed-form posterior of the state using the current measurement as input. The data-driven RNN captures the underlying non-linear dynamics of the model-free process. The training of DANSE, mainly learning the parameters of the RNN, is executed using an unsupervised learning approach. In unsupervised learning, we have access to a training dataset comprising only a set of (noisy) measurement data trajectories, but we do not have any access to the state trajectories. Therefore, DANSE does not have access to state information in the training data and can not use supervised learning. Using simulated linear and non-linear process models (Lorenz attractor and Chen attractor), we evaluate the unsupervised learning-based DANSE. We show that the proposed DANSE, without knowledge of the process model and without supervised learning, provides a competitive performance against model-driven methods, such as the Kalman filter (KF), extended KF (EKF), unscented KF (UKF), a data-driven deep Markov model (DMM) and a recently proposed hybrid method called KalmanNet. In addition, we show that DANSE works for high-dimensional state estimation.
|
|
| 6. |
- Ghosh, Anubhab, et al.
(författare)
-
DeepBayes—An estimator for parameter estimation in stochastic nonlinear dynamical models
- 2024
-
Ingår i: Automatica. - : Elsevier Ltd. - 0005-1098 .- 1873-2836. ; 159
-
Tidskriftsartikel (refereegranskat)abstract
- Stochastic nonlinear dynamical systems are ubiquitous in modern, real-world applications. Yet, estimating the unknown parameters of stochastic, nonlinear dynamical models remains a challenging problem. The majority of existing methods employ maximum likelihood or Bayesian estimation. However, these methods suffer from some limitations, most notably the substantial computational time for inference coupled with limited flexibility in application. In this work, we propose DeepBayes estimators that leverage the power of deep recurrent neural networks. The method consists of first training a recurrent neural network to minimize the mean-squared estimation error over a set of synthetically generated data using models drawn from the model set of interest. The a priori trained estimator can then be used directly for inference by evaluating the network with the estimation data. The deep recurrent neural network architectures can be trained offline and ensure significant time savings during inference. We experiment with two popular recurrent neural networks — long short term memory network (LSTM) and gated recurrent unit (GRU). We demonstrate the applicability of our proposed method on different example models and perform detailed comparisons with state-of-the-art approaches. We also provide a study on a real-world nonlinear benchmark problem. The experimental evaluations show that the proposed approach is asymptotically as good as the Bayes estimator.
|
|
| 7. |
- Ghosh, Anubhab, et al.
(författare)
-
Robust classification using hidden markov models and mixtures of normalizing flows
- 2020
-
Ingår i: 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP). - : Institute of Electrical and Electronics Engineers (IEEE).
-
Konferensbidrag (refereegranskat)abstract
- We test the robustness of a maximum-likelihood (ML) based classifier where sequential data as observation is corrupted by noise. The hypothesis is that a generative model, that combines the state transitions of a hidden Markov model (HMM) and the neural network based probability distributions for the hidden states of the HMM, can provide a robust classification performance. The combined model is called normalizing-flow mixture model based HMM (NMM-HMM). It can be trained using a combination of expectation-maximization (EM) and backpropagation. We verify the improved robustness of NMM-HMM classifiers in an application to speech recognition.
|
|
| 8. |
- Ghosh, Anubhab, et al.
(författare)
-
Time-varying Normalizing Flow for Generative Modeling of Dynamical Signals
- 2022
-
Ingår i: 2022 30Th European Signal Processing Conference (EUSIPCO 2022). - : IEEE. - 9789082797091 - 9781665467995 ; , s. 1492-1496
-
Konferensbidrag (refereegranskat)abstract
- We develop a time-varying normalizing flow (TVNF) for explicit generative modeling of dynamical signals. Being explicit, it can generate samples of dynamical signals, and compute the likelihood of a (given) dynamical signal sample. In the proposed model, signal flow in the layers of the normalizing flow is a function of time, which is realized using an encoded representation that is the output of a recurrent neural network (RNN). Given a set of dynamical signals, the parameters of TVNF are learned according to maximum-likelihood approach in conjunction with gradient descent (backpropagation). Use of the proposed model is illustrated for a toy application scenario - maximum-likelihood based speech-phone classification task.
|
|
| 9. |
- Honore, Antoine, et al.
(författare)
-
Compressed Sensing of Generative Sparse-Latent (GSL) Signals
- 2023
-
Ingår i: 31st European Signal Processing Conference, EUSIPCO 2023 - Proceedings. - : European Signal Processing Conference, EUSIPCO. ; , s. 1918-1922
-
Konferensbidrag (refereegranskat)abstract
- We consider reconstruction of an ambient signal in a compressed sensing (CS) setup where the ambient signal has a neural network based generative model. The generative model has a sparse-latent input and we refer to the generated ambient signal as generative sparse-latent signal (GSL). The proposed sparsity inducing reconstruction algorithm is inherently non-convex, and we show that a gradient based search provides a good reconstruction performance. We evaluate our proposed algorithm using simulated data.
|
|
