| 1. |
- Gustafsson, Stefan, et al.
(författare)
-
Development and validation of deep learning ECG-based prediction of myocardial infarction in emergency department patients
- 2022
-
Ingår i: Scientific Reports. - : Springer Nature. - 2045-2322. ; 12
-
Tidskriftsartikel (refereegranskat)abstract
- Myocardial infarction diagnosis is a common challenge in the emergency department. In managed settings, deep learning-based models and especially convolutional deep models have shown promise in electrocardiogram (ECG) classification, but there is a lack of high-performing models for the diagnosis of myocardial infarction in real-world scenarios. We aimed to train and validate a deep learning model using ECGs to predict myocardial infarction in real-world emergency department patients. We studied emergency department patients in the Stockholm region between 2007 and 2016 that had an ECG obtained because of their presenting complaint. We developed a deep neural network based on convolutional layers similar to a residual network. Inputs to the model were ECG tracing, age, and sex; and outputs were the probabilities of three mutually exclusive classes: non-ST-elevation myocardial infarction (NSTEMI), ST-elevation myocardial infarction (STEMI), and control status, as registered in the SWEDEHEART and other registries. We used an ensemble of five models. Among 492,226 ECGs in 214,250 patients, 5,416 were recorded with an NSTEMI, 1,818 a STEMI, and 485,207 without a myocardial infarction. In a random test set, our model could discriminate STEMIs/NSTEMIs from controls with a C-statistic of 0.991/0.832 and had a Brier score of 0.001/0.008. The model obtained a similar performance in a temporally separated test set of the study sample, and achieved a C-statistic of 0.985 and a Brier score of 0.002 in discriminating STEMIs from controls in an external test set. We developed and validated a deep learning model with excellent performance in discriminating between control, STEMI, and NSTEMI on the presenting ECG of a real-world sample of the important population of all-comers to the emergency department. Hence, deep learning models for ECG decision support could be valuable in the emergency department.
|
|
| 2. |
- Andersson, Carl
(författare)
-
Deep probabilistic models for sequential and hierarchical data
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Consider the problem where we want a computer program capable of recognizing a pedestrian on the road. This could be employed in a car to automatically apply the brakes to avoid an accident. Writing such a program is immensely difficult but what if we could instead use examples and let the program learn what characterizes a pedestrian from the examples. Machine learning can be described as the process of teaching a model (computer program) to predict something (the presence of a pedestrian) with help of data (examples) instead of through explicit programming.This thesis focuses on a specific method in machine learning, called deep learning. This method can arguably be seen as sole responsible for the recent upswing of machine learning in academia as well as in society at large. However, deep learning requires, in human standards, a huge amount of data to perform well which can be a limiting factor. In this thesis we describe different approaches to reduce the amount of data that is needed by encoding some of our prior knowledge about the problem into the model. To this end we focus on sequential and hierarchical data, such as speech and written language.Representing sequential output is in general difficult due to the complexity of the output space. Here, we make use of a probabilistic approach focusing on sequential models in combination with a deep learning structure called the variational autoencoder. This is applied to a range of different problem settings, from system identification to speech modeling.The results come in three parts. The first contribution focus on applications of deep learning to typical system identification problems, the intersection between the two areas and how they can benefit from each other. The second contribution is on hierarchical data where we promote a multiscale variational autoencoder inspired by image modeling. The final contribution is on verification of probabilistic models, in particular how to evaluate the validity of a probabilistic output, also known as calibration.
|
|
| 3. |
- Gustafsson, Fredrik K., 1993-
(författare)
-
Towards Accurate and Reliable Deep Regression Models
- 2023
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Regression is a fundamental machine learning task with many important applications within computer vision and other domains. In general, it entails predicting continuous targets from given inputs. Deep learning has become the dominant paradigm within machine learning in recent years, and a wide variety of different techniques have been employed to solve regression problems using deep models. There is however no broad consensus on how deep regression models should be constructed for best possible accuracy, or how the uncertainty in their predictions should be represented and estimated. These open questions are studied in this thesis, aiming to help take steps towards an ultimate goal of developing deep regression models which are both accurate and reliable enough for real-world deployment within medical applications and other safety-critical domains.The first main contribution of the thesis is the formulation and development of energy-based probabilistic regression. This is a general and conceptually simple regression framework with a clear probabilistic interpretation, using energy-based models to represent the true conditional target distribution. The framework is applied to a number of regression problems and demonstrates particularly strong performance for 2D bounding box regression, improving the state-of-the-art when applied to the task of visual tracking.The second main contribution is a critical evaluation of various uncertainty estimation methods. A general introduction to the problem of estimating the predictive uncertainty of deep models is first provided, together with an extensive comparison of the two popular methods ensembling and MC-dropout. A number of regression uncertainty estimation methods are then further evaluated, specifically examining their reliability under real-world distribution shifts. This evaluation uncovers important limitations of current methods and serves as a challenge to the research community. It demonstrates that more work is required in order to develop truly reliable uncertainty estimation methods for regression.
|
|
| 4. |
- Jidling, Carl, 1992-
(författare)
-
Tailoring Gaussian processes and large-scale optimisation
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- This thesis is centred around Gaussian processes and large-scale optimisation, where the main contributions are presented in the included papers.Provided access to linear constraints (e.g. equilibrium conditions), we propose a constructive procedure to design the covariance function in a Gaussian process. The constraints are thereby explicitly incorporated with guaranteed fulfilment. One such construction is successfully applied to strain field reconstruction, where the goal is to describe the interior of a deformed object. Furthermore, we analyse the Gaussian process as a tool for X-ray computed tomography, a field of high importance primarily due to its central role in medical treatments. This provides insightful interpretations of traditional reconstruction algorithms. Large-scale optimisation is considered in two different contexts. First, we consider a stochastic environment, for which we suggest a new method inspired by the quasi-Newton framework. Promising results are demonstrated on real world benchmark problems. Secondly, we suggest an approach to solve an applied deterministic optimisation problem that arises within the design of electrical circuit boards. We reduce the memory requirements through a tailored algorithm, while also benefiting from other parts of the setting to ensure a high computational efficiency. The final paper scrutinises a publication from the early phase of the COVID-19 pandemic, in which the aim was to assess the effectiveness of different governmental interventions. We show that minor modifications in the input data have important impact on the results, and we argue that great caution is necessary when such models are used as a support for decision making.
|
|
| 5. |
- Kudlicka, Jan
(författare)
-
Probabilistic Programming for Birth-Death Models of Evolution
- 2021
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Phylogenetic birth-death models constitute a family of generative models of evolution. In these models an evolutionary process starts with a single species at a certain time in the past, and the speciations—splitting one species into two descendant species—and extinctions are modeled as events of non-homogenous Poisson processes. Different birth-death models admit different types of changes to the speciation and extinction rates.The result of an evolutionary process is a binary tree called a phylogenetic tree, or phylogeny, with the root representing the single species at the origin, internal nodes speciation events, and leaves currently living—extant—species (in the present time) and extinction events (in the past). Usually only a part of this tree, corresponding to the evolution of the extant species and their ancestors, is known via reconstruction from e.g. genomic sequences of these extant species.The task of our interest is to estimate the parameters of birth-death models given this reconstructed tree as the observation. While encoding the generative birth-death models as computer programs is easy and straightforward, developing and implementing bespoke inference algorithms are not. This complicates prototyping, development, and deployment of new birth-death models.Probabilistic programming is a new approach in which the generative models are encoded as computer programs in languages that include support for random variables, conditioning on the observed data, as well as automatic inference. This thesis is based on a collection of papers in which we demonstrate how to use probabilistic programming to solve the above-mentioned task of parameter inference in birth-death models. We show how these models can be implemented as simple programs in probabilistic programming languages. Our contribution also includes general improvements of the automatic inference methods.
|
|
| 6. |
- Lima, Emilly M., et al.
(författare)
-
Deep neural network-estimated electrocardiographic age as a mortality predictor
- 2021
-
Ingår i: Nature Communications. - : Springer Nature. - 2041-1723. ; 12:1
-
Tidskriftsartikel (refereegranskat)abstract
- The electrocardiogram (ECG) is the most commonly used exam for the screening and evaluation of cardiovascular diseases. Here, the authors propose that the age predicted by artificial intelligence from the raw ECG tracing can be a measure of cardiovascular health and provide prognostic information. The electrocardiogram (ECG) is the most commonly used exam for the evaluation of cardiovascular diseases. Here we propose that the age predicted by artificial intelligence (AI) from the raw ECG (ECG-age) can be a measure of cardiovascular health. A deep neural network is trained to predict a patient's age from the 12-lead ECG in the CODE study cohort (n = 1,558,415 patients). On a 15% hold-out split, patients with ECG-age more than 8 years greater than the chronological age have a higher mortality rate (hazard ratio (HR) 1.79, p < 0.001), whereas those with ECG-age more than 8 years smaller, have a lower mortality rate (HR 0.78, p < 0.001). Similar results are obtained in the external cohorts ELSA-Brasil (n = 14,236) and SaMi-Trop (n = 1,631). Moreover, even for apparent normal ECGs, the predicted ECG-age gap from the chronological age remains a statistically significant risk predictor. These results show that the AI-enabled analysis of the ECG can add prognostic information.
|
|
| 7. |
- Osama, Muhammad
(författare)
-
Robust machine learning methods
- 2022
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- We are surrounded by data in our daily lives. The rent of our houses, the amount of electricity units consumed, the prices of different products at a supermarket, the daily temperature, our medicine prescriptions, our internet search history are all different forms of data. Data can be used in a wide range of applications. For example, one can use data to predict product prices in the future; to predict tomorrow's temperature; to recommend videos; or suggest better prescriptions. However in order to do the above, one is required to learn a model from data. A model is a mathematical description of how the phenomena we are interested in behaves e.g. how does the temperature vary? Is it periodic? What kinds of patterns does it have? Machine learning is about this process of learning models from data by building on disciplines such as statistics and optimization. Learning models comes with many different challenges. Some challenges are related to how flexible the model is, some are related to the size of data, some are related to computational efficiency etc. One of the challenges is that of data outliers. For instance, due to war in a country exports could stop and there could be a sudden spike in prices of different products. This sudden jump in prices is an outlier or corruption to the normal situation and must be accounted for when learning the model. Another challenge could be that data is collected in one situation but the model is to be used in another situation. For example, one might have data on vaccine trials where the participants were mostly old people. But one might want to make a decision on whether to use the vaccine or not for the whole population that contains people of all age groups. So one must also account for this difference when learning models because the conclusion drawn may not be valid for the young people in the population. Yet another challenge could arise when data is collected from different sources or contexts. For example, a shopkeeper might have data on sales of paracetamol when there was flu and when there was no flu and she might want to decide how much paracetamol to stock for the next month. In this situation, it is difficult to know whether there will be a flu next month or not and so deciding on how much to stock is a challenge. This thesis tries to address these and other similar challenges.In paper I, we address the challenge of data corruption i.e., learning models in a robust way when some fraction of the data is corrupted. In paper II, we apply the methodology of paper I to the problem of localization in wireless networks. Paper III addresses the challenge of estimating causal effect between an exposure and an outcome variable from spatially collected data (e.g. whether increasing number of police personnel in an area reduces number of crimes there). Paper IV addresses the challenge of learning improved decision policies e.g. which treatment to assign to which patient given past data on treatment assignments. In paper V, we look at the challenge of learning models when data is acquired from different contexts and the future context is unknown. In paper VI, we address the challenge of predicting count data across space e.g. number of crimes in an area and quantify its uncertainty. In paper VII, we address the challenge of learning models when data points arrive in a streaming fashion i.e., point by point. The proposed method enables online training and also yields some robustness properties.
|
|
| 8. |
- Sundström, Johan, Professor, 1971-, et al.
(författare)
-
Machine Learning in Risk Prediction
- 2020
-
Ingår i: Hypertension. - 0194-911X .- 1524-4563. ; 75:5, s. 1165-1166
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)
|
|
| 9. |
- Ancuti, Codruta O., et al.
(författare)
-
NTIRE 2023 HR NonHomogeneous Dehazing Challenge Report
- 2023
-
Ingår i: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). - Vancover : Institute of Electrical and Electronics Engineers (IEEE).
-
Konferensbidrag (refereegranskat)abstract
- This study assesses the outcomes of the NTIRE 2023 Challenge on Non-Homogeneous Dehazing, wherein novel techniques were proposed and evaluated on new image dataset called HD-NH-HAZE. The HD-NH-HAZE dataset contains 50 high resolution pairs of real-life outdoor images featuring nonhomogeneous hazy images and corresponding haze-free images of the same scene. The nonhomogeneous haze was simulated using a professional setup that replicated real-world conditions of hazy scenarios. The competition had 246 participants and 17 teams that competed in the final testing phase, and the proposed solutions demonstrated the cutting-edge in image dehazing technology.
|
|
| 10. |
- Andersson, Carl R., et al.
(författare)
-
Learning deep autoregressive models for hierarchical data
- 2021
-
Ingår i: IFAC PapersOnLine. - : Elsevier. - 2405-8963. ; , s. 529-534
-
Konferensbidrag (refereegranskat)abstract
- We propose a model for hierarchical structured data as an extension to the stochastic temporal convolutional network. The proposed model combines an autoregressive model with a hierarchical variational autoencoder and downsampling to achieve superior computational complexity. We evaluate the proposed model on two different types of sequential data: speech and handwritten text. The results are promising with the proposed model achieving state-of-the-art performance.
|
|
| 11. |
- Baumann, Dominik, et al.
(författare)
-
On the trade-off between event-based and periodic state estimation under bandwidth constraints
- 2023
-
Ingår i: IFAC-PapersOnLine. - : Elsevier. - 2405-8963. ; 56:2, s. 5275-5280
-
Tidskriftsartikel (refereegranskat)abstract
- Event-based methods carefully select when to transmit information to enable high-performance control and estimation over resource-constrained communication networks. However, they come at a cost. For instance, event-based communication induces a higher computational load and increases the complexity of the scheduling problem. Thus, in some cases, allocating available slots to agents periodically in circular order may be superior. In this article, we discuss, for a specific example, when the additional complexity of event-based methods is beneficial. We evaluate our analysis in a synthetical example and on 20 simulated cart-pole systems.
|
|
| 12. |
- Baumann, Dominik, Ph.D. 1991-, et al.
(författare)
-
Safe Reinforcement Learning in Uncertain Contexts
- 2024
-
Ingår i: IEEE Transactions on robotics. - : IEEE. - 1552-3098 .- 1941-0468. ; 40, s. 1828-1841
-
Tidskriftsartikel (refereegranskat)abstract
- When deploying machine learning algorithms in the real world, guaranteeing safety is an essential asset. Existing safe learning approaches typically consider continuous variables, i.e., regression tasks. However, in practice, robotic systems are also subject to discrete, external environmental changes, e.g., having to carry objects of certain weights or operating on frozen, wet, or dry surfaces. Such influences can be modeled as discrete context variables. In the existing literature, such contexts are, if considered, mostly assumed to be known. In this work, we drop this assumption and show how we can perform safe learning when we cannot directly measure the context variables. To achieve this, we derive frequentist guarantees for multiclass classification, allowing us to estimate the current context from measurements. Furthermore, we propose an approach for identifying contexts through experiments. We discuss under which conditions we can retain theoretical guarantees and demonstrate the applicability of our algorithm on a Furuta pendulum with camera measurements of different weights that serve as contexts.
|
|
| 13. |
- Bånkestad, Maria, et al.
(författare)
-
Variational Elliptical Processes
- 2023
-
Ingår i: Transactions on Machine Learning Research. - 2835-8856.
-
Tidskriftsartikel (refereegranskat)abstract
- We present elliptical processes—a family of non-parametric probabilistic models that subsumes Gaussian processes and Student's t processes. This generalization includes a range of new heavy-tailed behaviors while retaining computational tractability. Elliptical processes are based on a representation of elliptical distributions as a continuous mixture of Gaussian distributions. We parameterize this mixture distribution as a spline normalizing flow, which we train using variational inference. The proposed form of the variational posterior enables a sparse variational elliptical process applicable to large-scale problems. We highlight advantages compared to Gaussian processes through regression and classification experiments. Elliptical processes can supersede Gaussian processes in several settings, including cases where the likelihood is non-Gaussian or when accurate tail modeling is essential.
|
|
| 14. |
- Carlsson, Håkan, et al.
(författare)
-
Quantifying the Uncertainty of the Relative Geometry in Inertial Sensors Arrays
- 2021
-
Ingår i: IEEE Sensors Journal. - : Institute of Electrical and Electronics Engineers (IEEE). - 1530-437X .- 1558-1748. ; 21:17, s. 19362-19373
-
Tidskriftsartikel (refereegranskat)abstract
- We present an algorithm to estimate and quantify the uncertainty of the accelerometers' relative geometry in an inertial sensor array. We formulate the calibration problem as a Bayesian estimation problem and propose an algorithm that samples the accelerometer positions' posterior distribution using Markov chain Monte Carlo. By identifying linear substructures of the measurement model, the unknown linear motion parameters are analytically marginalized, and the remaining non-linear motion parameters are numerically marginalized. The numerical marginalization occurs in a low dimensional space where the gyroscopes give information about the motion. This combination of information from gyroscopes and analytical marginalization allows the user to make no assumptions of the motion before the calibration. It thus enables the user to estimate the accelerometer positions' relative geometry by simply exposing the array to arbitrary twisting motion. We show that the calibration algorithm gives good results on both simulated and experimental data, despite sampling a high dimensional space.
|
|
| 15. |
- Conde, Marcus V., et al.
(författare)
-
Lens-to-Lens Bokeh Effect Transformation : NTIRE 2023 Challenge Report
- 2023
-
Ingår i: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. - Vancover : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 1643-1659
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- We present the new Bokeh Effect Transformation Dataset (BETD), and review the proposed solutions for this novel task at the NTIRE 2023 Bokeh Effect Transformation Challenge. Recent advancements of mobile photography aim to reach the visual quality of full-frame cameras. Now, a goal in computational photography is to optimize the Bokeh effect itself, which is the aesthetic quality of the blur in out-of-focus areas of an image. Photographers create this aesthetic effect by benefiting from the lens optical properties. The aim of this work is to design a neural network capable of converting the the Bokeh effect of one lens to the effect of another lens without harming the sharp foreground regions in the image. For a given input image, knowing the target lens type, we render or transform the Bokeh effect accordingly to the lens properties. We build the BETD using two full-frame Sony cameras, and diverse lens setups. To the best of our knowledge, we are the first attempt to solve this novel task, and we provide the first BETD dataset and benchmark for it. The challenge had 99 registered participants. The submitted methods gauge the state-of-the-art in Bokeh effect rendering and transformation.
|
|
| 16. |
- Courts, Jarrad, et al.
(författare)
-
Gaussian Variational State Estimation for Nonlinear State-Space Models
- 2021
-
Ingår i: IEEE Transactions on Signal Processing. - : Institute of Electrical and Electronics Engineers (IEEE). - 1053-587X .- 1941-0476. ; 69, s. 5979-5993
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper, the problem of state estimation, in the context of both filtering and smoothing, for nonlinear state-space models is considered. Due to the nonlinear nature of the models, the state estimation problem is generally intractable as it involves integrals of general nonlinear functions and the filtered and smoothed state distributions lack closed-form solutions. As such, it is common to approximate the state estimation problem. In this paper, we develop an assumed Gaussian solution based on variational inference, which offers the key advantage of a flexible, but principled, mechanism for approximating the required distributions. Our main contribution lies in a new formulation of the state estimation problem as an optimisation problem, which can then be solved using standard optimisation routines that employ exact first- and second-order derivatives. The resulting state estimation approach involves a minimal number of assumptions and applies directly to nonlinear systems with both Gaussian and non-Gaussian probabilistic models. The performance of our approach is demonstrated on several examples; a challenging scalar system, a model of a simple robotic system, and a target tracking problem using a von Mises-Fisher distribution and outperforms alternative assumed Gaussian approaches to state estimation.
|
|
| 17. |
- Courts, Jarrad, et al.
(författare)
-
Variational State and Parameter Estimation
- 2021
-
Ingår i: IFAC PapersOnLine. - : Elsevier. - 2405-8963. ; , s. 732-737
-
Konferensbidrag (refereegranskat)abstract
- This paper considers the problem of computing Bayesian estimates of both states and model parameters for nonlinear state-space models. Generally, this problem does not have a tractable solution and approximations must be utilised. In this work, a variational approach is used to provide an assumed density which approximates the desired, intractable, distribution. The approach is deterministic and results in an optimisation problem of a standard form. Due to the parametrisation of the assumed density selected first- and second-order derivatives are readily available which allows for efficient solutions. The proposed method is compared against state-of-the-art Hamiltonian Monte Carlo in two numerical examples.
|
|
| 18. |
- Courts, Jarrad, et al.
(författare)
-
Variational system identification for nonlinear state-space models
- 2023
-
Ingår i: Automatica. - : Elsevier. - 0005-1098 .- 1873-2836. ; 147
-
Tidskriftsartikel (refereegranskat)abstract
- This paper considers parameter estimation for nonlinear state-space models, which is an important but challenging problem. We address this challenge by employing a variational inference (VI) approach, which is a principled method that has deep connections to maximum likelihood estimation. This VI approach ultimately provides estimates of the model as solutions to an optimisation problem, which is deterministic, tractable and can be solved using standard optimisation tools. A specialisation of this approach for systems with additive Gaussian noise is also detailed. The proposed method is examined numerically on a range of simulated and real examples focusing on the robustness to parameter initialisation; additionally, favourable comparisons are performed against state-of-the-art alternatives.
|
|
| 19. |
- Ferizbegovic, Mina, et al.
(författare)
-
Bayes control of hammerstein systems
- 2021
-
Ingår i: 19th IFAC Symposium on System Identification, SYSID 2021. - : Elsevier BV. ; , s. 755-760, s. 755-760
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we consider data driven control of Hammerstein systems. For such systems a common control structure is a transfer function followed by a static output nonlinearity that tries to cancel the input nonlinearity of the system, which is modeled as a polynomial or piece-wise linear function. The linear part of the controller is used to achieve desired disturbance rejection and tracking properties. To design a linear part of the controller, we propose a weighted average risk criterion with the risk being the average of the squared L2 tracking error. Here the average is with respect to the observations used in the controller and the weighting is with respect to how important it is to have good control for different impulse responses. This criterion corresponds to the average risk criterion leading to the Bayes estimator and we therefore call this approach Bayes control. By parametrizing the weighting function and estimating the corresponding hyperparameters we tune the weighting function to the information regarding the true impulse response contained in the data set available to the user for the control design. The numerical results show that the proposed methods result in stable controllers with performance comparable to the optimal controller, designed using the true input nonlinearity and true plant.
|
|
| 20. |
- Ferizbegovic, Mina, et al.
(författare)
-
Learning Robust LQ-Controllers Using Application Oriented Exploration
- 2020
-
Ingår i: IEEE Control Systems Letters. - : Institute of Electrical and Electronics Engineers (IEEE). - 2475-1456. ; 4:1, s. 19-24
-
Tidskriftsartikel (refereegranskat)abstract
- This letter concerns the problem of learning robust LQ-controllers, when the dynamics of the linear system are unknown. First, we propose a robust control synthesis method to minimize the worst-case LQ cost, with probability 1-δ , given empirical observations of the system. Next, we propose an approximate dual controller that simultaneously regulates the system and reduces model uncertainty. The objective of the dual controller is to minimize the worst-case cost attained by a new robust controller, synthesized with the reduced model uncertainty. The dual controller is subject to an exploration budget in the sense that it has constraints on its worst-case cost with respect to the current model uncertainty. In our numerical experiments, we observe better performance of the proposed robust LQ regulator over the existing methods. Moreover, the dual control strategy gives promising results in comparison with the common greedy random exploration strategies.
|
|
| 21. |
- Ferizbegovic, Mina, et al.
(författare)
-
Willems' fundamental lemma based on second-order moments
- 2021
-
Ingår i: 2021 60TH IEEE CONFERENCE ON DECISION AND CONTROL (CDC). - : Institute of Electrical and Electronics Engineers (IEEE). - 9781665436595 ; , s. 396-401
-
Konferensbidrag (refereegranskat)abstract
- In this paper, we propose variations of Willems' fundamental lemma that utilize second-order moments such as correlation functions in the time domain and power spectra in the frequency domain. We believe that using a formulation with estimated correlation coefficients is suitable for data compression, and possibly can reduce noise. Also, the formulations in the frequency domain can enable modeling of a system in a frequency region of interest.
|
|
| 22. |
- Gedon, Daniel, 1994-, et al.
(författare)
-
Deep State Space Models for Nonlinear System Identification
- 2021
-
Ingår i: IFAC PapersOnLine. - : Elsevier. - 2405-8963. ; , s. 481-486
-
Konferensbidrag (refereegranskat)abstract
- Deep state space models (SSMs) are an actively researched model class for temporal models developed in the deep learning community which have a close connection to classic SSMs. The use of deep SSMs as a black-box identification model can describe a wide range of dynamics due to the flexibility of deep neural networks. Additionally, the probabilistic nature of the model class allows the uncertainty of the system to be modelled. In this work a deep SSM class and its parameter learning algorithm are explained in an effort to extend the toolbox of nonlinear identification methods with a deep learning based method. Six recent deep SSMs are evaluated in a first unified implementation on nonlinear system identification benchmarks.
|
|
| 23. |
- Gedon, Daniel, 1994-, et al.
(författare)
-
First Steps Towards Self-Supervised Pretraining of the 12-Lead ECG
- 2021
-
Ingår i: 2021 Computing In Cardiology (CINC). - : Institute of Electrical and Electronics Engineers (IEEE). - 9781665479165
-
Konferensbidrag (refereegranskat)abstract
- Self-supervised learning is a paradigm that extracts general features which describe the input space by artificially generating labels from the input without the need for explicit annotations. The learned features can then be used by transfer learning to boost the performance on a downstream task. Such methods have recently produced state of the art results in natural language processing and computer vision. Here, we propose a self-supervised learning method for 12-lead electrocardiograms (ECGs). For pretraining the model we design a task to mask out subsegements of all channels of the input signals and try to predict the actual values. As the model architecture, we use a U-ResNet containing an encoder-decoder structure. We test our method by self-supervised pretraining on the CODE dataset and then transfer the learnt features by finetuning on the PTBXL and CPSC benchmarks to evaluate the effect of our method in the classification of 12-leads ECGs. The method does provide modest improvements in performance when compared to not using pretraining. In future work we will make use of these ideas in smaller dataset, where we believe it can lead to larger performance gains.
|
|
| 24. |
- Gedon, Daniel, 1994-, et al.
(författare)
-
Invertible Kernel PCA With Random Fourier Features
- 2023
-
Ingår i: IEEE Signal Processing Letters. - : Institute of Electrical and Electronics Engineers (IEEE). - 1070-9908 .- 1558-2361. ; 30, s. 563-567
-
Tidskriftsartikel (refereegranskat)abstract
- Kernel principal component analysis (kPCA) is a widely studied method to construct a low-dimensional data representation after a nonlinear transformation. The prevailing method to reconstruct the original input signal from kPCA-an important task for denoising-requires us to solve a supervised learning problem. In this paper, we present an alternative method where the reconstruction follows naturally from the compression step. We first approximate the kernel with random Fourier features. Then, we exploit the fact that the nonlinear transformation is invertible in a certain subdomain. Hence, the name invertible kernel PCA (ikPCA). We experiment with different data modalities and show that ikPCA performs similarly to kPCA with supervised reconstruction on denoising tasks, making it a strong alternative.
|
|
| 25. |
- Gedon, Daniel, 1994-
(författare)
-
On Deep Learning for Low-Dimensional Representations
- 2024
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- In science and engineering, we are often concerned with creating mathematical models from data. These models are abstractions of observed real-world processes where the goal is often to understand these processes or to use the models to predict future instances of the observed process. Natural processes often exhibit low-dimensional structures which we can embed into the model. In mechanistic models, we directly include this structure into the model through mathematical equations often inspired by physical constraints. In contrast, within machine learning and particularly in deep learning we often deal with high-dimensional data such as images and learn a model without imposing a low-dimensional structure. Instead, we learn some kind of representations that are useful for the task at hand. While representation learning arguably enables the power of deep neural networks, it is less clear how to understand real-world processes from these models or whether we can benefit from including a low-dimensional structure in the model.Learning from data with intrinsic low-dimensional structure and how to replicate this structure in machine learning models is studied within this dissertation. While we put specific emphasis on deep neural networks, we also consider kernel machines in the context of Gaussian processes, as well as linear models, for example by studying the generalisation of models with an explicit low-dimensional structure. First, we argue that many real-world observations have an intrinsic low-dimensional structure. We can find evidence of this structure for example through low-rank approximations of many real-world data sets. Then, we face two open-ended research questions. First, we study the behaviour of machine learning models when they are trained on data with low-dimensional structures. Here we investigate fundamental aspects of learning low-dimensional representations and how well models with explicit low-dimensional structures perform. Second, we focus on applications in the modelling of dynamical systems and the medical domain. We investigate how we can benefit from low-dimensional representations for these applications and explore the potential of low-dimensional model structures for predictive tasks. Finally, we give a brief outlook on how we go beyond learning low-dimensional structures and identify the underlying mechanisms that generate the data to better model and understand these processes.This dissertation provides an overview of learning low-dimensional structures in machine learning models. It covers a wide range of topics from representation learning over the study of generalisation in overparameterized models to applications with time series and medical applications. However, each contribution opens up a range of questions to study in the future. Therefore this dissertation serves as a starting point to further explore learning of low-dimensional structure and representations.
|
|
