| 1. |
|
|
| 2. |
- Lindholm, Andreas, et al.
(författare)
-
Machine learning : a first course for engineers and scientists
- 2022
-
Bok (övrigt vetenskapligt/konstnärligt)abstract
- This book introduces machine learning for readers with some background in basic linear algebra, statistics, probability, and programming. In a coherent statistical framework it covers a selection of supervised machine learning methods, from the most fundamental (k-NN, decision trees, linear and logistic regression) to more advanced methods (deep neural networks, support vector machines, Gaussian processes, random forests and boosting), plus commonly-used unsupervised methods (generative modeling, k-means, PCA, autoencoders and generative adversarial networks). Careful explanations and pseudo-code are presented for all methods. The authors maintain a focus on the fundamentals by drawing connections between methods and discussing general concepts such as loss functions, maximum likelihood, the bias-variance decomposition, ensemble averaging, kernels and the Bayesian approach along with generally useful tools such as regularization, cross validation, evaluation metrics and optimization methods. The final chapters offer practical advice for solving real-world supervised machine learning problems and on ethical aspects of modern machine learning
|
|
| 3. |
- Ahmadian, Amirhossein, 1992-, et al.
(författare)
-
Enhancing Representation Learning with Deep Classifiers in Presence of Shortcut
- 2023
-
Ingår i: Proceedings of IEEE ICASSP 2023.
-
Konferensbidrag (refereegranskat)abstract
- A deep neural classifier trained on an upstream task can be leveraged to boost the performance of another classifier in a related downstream task through the representations learned in hidden layers. However, presence of shortcuts (easy-to-learn features) in the upstream task can considerably impair the versatility of intermediate representations and, in turn, the downstream performance. In this paper, we propose a method to improve the representations learned by deep neural image classifiers in spite of a shortcut in upstream data. In our method, the upstream classification objective is augmented with a type of adversarial training where an auxiliary network, so called lens, fools the classifier by exploiting the shortcut in reconstructing images. Empirical comparisons in self-supervised and transfer learning problems with three shortcut-biased datasets suggest the advantages of our method in terms of downstream performance and/or training time.
|
|
| 4. |
- Ahmadian, Amirhossein, 1992-, et al.
(författare)
-
Likelihood-free Out-of-Distribution Detection with Invertible Generative Models
- 2021
-
Ingår i: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI 2021). - California : International Joint Conferences on Artificial Intelligence Organization.
-
Konferensbidrag (refereegranskat)abstract
- Likelihood of generative models has been used traditionally as a score to detect atypical (Out-of-Distribution, OOD) inputs. However, several recent studies have found this approach to be highly unreliable, even with invertible generative models, where computing the likelihood is feasible. In this paper, we present a different framework for generative model--based OOD detection that employs the model in constructing a new representation space, instead of using it directly in computing typicality scores, where it is emphasized that the score function should be interpretable as the similarity between the input and training data in the new space. In practice, with a focus on invertible models, we propose to extract low-dimensional features (statistics) based on the model encoder and complexity of input images, and then use a One-Class SVM to score the data. Contrary to recently proposed OOD detection methods for generative models, our method does not require computing likelihood values. Consequently, it is much faster when using invertible models with iteratively approximated likelihood (e.g. iResNet), while it still has a performance competitive with other related methods
|
|
| 5. |
- Ekström Kelvinius, Filip, et al.
(författare)
-
Discriminator Guidance for Autoregressive Diffusion Models
- 2024
-
Ingår i: Proceedings of The 27th International Conference on Artificial Intelligence and Statistics. - : PMLR. ; , s. 3403-3411
-
Konferensbidrag (refereegranskat)abstract
- We introduce discriminator guidance in the setting of Autoregressive Diffusion Models. The use of a discriminator to guide a diffusion process has previously been used for continuous diffusion models, and in this work we derive ways of using a discriminator together with a pretrained generative model in the discrete case. First, we show that using an optimal discriminator will correct the pretrained model and enable exact sampling from the underlying data distribution. Second, to account for the realistic scenario of using a sub-optimal discriminator, we derive a sequential Monte Carlo algorithm which iteratively takes the predictions from the discriminator into account during the generation process. We test these approaches on the task of generating molecular graphs and show how the discriminator improves the generative performance over using only the pretrained model.
|
|
| 6. |
- Glaser, Pierre, et al.
(författare)
-
Fast and Scalable Score-Based Kernel Calibration Tests
- 2023
-
Ingår i: Thirty-Ninth Conference on Uncertainty in Artificial Intelligence.
-
Konferensbidrag (refereegranskat)abstract
- We introduce the Kernel Calibration Conditional Stein Discrepancy test (KCCSD test), a non-parametric, kernel-based test for assessing the calibration of probabilistic models with well-defined scores. In contrast to previous methods, our test avoids the need for possibly expensive expectation approximations while providing control over its type-I error. We achieve these improvements by using a new family of kernels for score-based probabilities that can be estimated without probability density samples, and by using a conditional goodness-of-fit criterion for the KCCSD test’s U-statistic. The tractability of the KCCSD test widens the surface area of calibration measures to new promising use-cases, such as regularization during model training. We demonstrate the properties of our test on various synthetic settings.
|
|
| 7. |
- Govindarajan, Hariprasath, et al.
(författare)
-
DINO as a von Mises-Fisher mixture model
- 2023
-
Ingår i: The Eleventh International Conference on Learning Representations.
-
Konferensbidrag (refereegranskat)abstract
- Self-distillation methods using Siamese networks are popular for self-supervised pre-training. DINO is one such method based on a cross-entropy loss between K-dimensional probability vectors, obtained by applying a softmax function to the dot product between representations and learnt prototypes. Given the fact that the learned representations are L2-normalized, we show that DINO and its derivatives, such as iBOT, can be interpreted as a mixture model of von Mises-Fisher components. With this interpretation, DINO assumes equal precision for all components when the prototypes are also L2-normalized. Using this insight we propose DINO-vMF, that adds appropriate normalization constants when computing the cluster assignment probabilities. Unlike DINO, DINO-vMF is stable also for the larger ViT-Base model with unnormalized prototypes. We show that the added flexibility of the mixture model is beneficial in terms of better image representations. The DINO-vMF pre-trained model consistently performs better than DINO on a range of downstream tasks. We obtain similar improvements for iBOT-vMF vs iBOT and thereby show the relevance of our proposed modification also for other methods derived from DINO.
|
|
| 8. |
- Lindqvist, Jakob, 1992, et al.
(författare)
-
A General Framework for Ensemble Distribution Distillation
- 2020
-
Ingår i: 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP). - : IEEE. - 9781728166629 ; 2020-September
-
Konferensbidrag (refereegranskat)abstract
- Ensembles of neural networks have shown to give better predictive performance and more reliable uncertainty estimates than individual networks. Additionally, ensembles allow the uncertainty to be decomposed into aleatoric (data) and epistemic (model) components, giving a more complete picture of the predictive uncertainty. Ensemble distillation is the process of compressing an ensemble into a single model, often resulting in a leaner model that still outperforms the individual ensemble members. Unfortunately, standard distillation erases the natural uncertainty decomposition of the ensemble. We present a general framework for distilling both regression and classification ensembles in a way that preserves the decomposition. We demonstrate the desired behaviour of our framework and show that its predictive performance is on par with standard distillation.
|
|
| 9. |
- Olmin, Amanda, 1994-, et al.
(författare)
-
Active Learning with Weak Supervision for Gaussian Processes
- 2023
-
Ingår i: Communications in Computer and Information Science. - Singapore : Springer Nature. - 1865-0937 .- 1865-0929. ; 1792 CCIS, s. 195-204
-
Konferensbidrag (refereegranskat)abstract
- Annotating data for supervised learning can be costly. When the annotation budget is limited, active learning can be used to select and annotate those observations that are likely to give the most gain in model performance. We propose an active learning algorithm that, in addition to selecting which observation to annotate, selects the precision of the annotation that is acquired. Assuming that annotations with low precision are cheaper to obtain, this allows the model to explore a larger part of the input space, with the same annotation budget. We build our acquisition function on the previously proposed BALD objective for Gaussian Processes, and empirically demonstrate the gains of being able to adjust the annotation precision in the active learning loop.
|
|
| 10. |
- Olmin, Amanda, 1994-, et al.
(författare)
-
On the connection between Noise-Contrastive Estimation and Contrastive Divergence
- 2024
-
Konferensbidrag (refereegranskat)abstract
- Noise-contrastive estimation (NCE) is a popular method for estimating unnormalised probabilistic models, such as energy-based models, which are effective for modelling complex data distributions. Unlike classical maximum likelihood (ML) estimation that relies on importance sampling (resulting in ML-IS) or MCMC (resulting in contrastive divergence, CD), NCE uses a proxy criterion to avoid the need for evaluating an often intractable normalisation constant. Despite apparent conceptual differences, we show that two NCE criteria, ranking NCE (RNCE) and conditional NCE (CNCE), can be viewed as ML estimation methods. Specifically, RNCE is equivalent to ML estimation combined with conditional importance sampling, and both RNCE and CNCE are special cases of CD. These findings bridge the gap between the two method classes and allow us to apply techniques from the ML-IS and CD literature to NCE, offering several advantageous extensions.
|
|
