Search: hsv:(NATURVETENSKAP) hsv:(Data och informationsvetenskap) hsv:(Datavetenskap) >
Evaluating the Robu...
Evaluating the Robustness of ML Models to Out-of-Distribution Data Through Similarity Analysis
-
- Lindén, Joakim (author)
- Mälardalens universitet,Inbyggda system,Mälardalen University, Västerås, Sweden; Saab AB, Linköping, Sweden
-
- Forsberg, Håkan (author)
- Mälardalens universitet,Inbyggda system,Mälardalen University, Västerås, Sweden
-
- Daneshtalab, Masoud (author)
- Mälardalens universitet,Inbyggda system,Mälardalen University, Västerås, Sweden
-
show more...
-
- Söderquist, Ingemar (author)
- KTH,Elektronik och inbyggda system,Saab AB, Linköping, Sweden
-
show less...
-
(creator_code:org_t)
- Springer Science and Business Media Deutschland GmbH, 2023
- 2023
- English.
-
In: Commun. Comput. Info. Sci.. - : Springer Science and Business Media Deutschland GmbH. - 9783031429408 ; , s. 348-359, s. 348-359
- Related links:
-
https://urn.kb.se/re...
-
show more...
-
https://doi.org/10.1...
-
https://urn.kb.se/re...
-
show less...
Abstract
Subject headings
Close
- In Machine Learning systems, several factors impact the performance of a trained model. The most important ones include model architecture, the amount of training time, the dataset size and diversity. We present a method for analyzing datasets from a use-case scenario perspective, detecting and quantifying out-of-distribution (OOD) data on dataset level. Our main contribution is the novel use of similarity metrics for the evaluation of the robustness of a model by introducing relative Fréchet Inception Distance (FID) and relative Kernel Inception Distance (KID) measures. These relative measures are relative to a baseline in-distribution dataset and are used to estimate how the model will perform on OOD data (i.e. estimate the model accuracy drop). We find a correlation between our proposed relative FID/relative KID measure and the drop in Average Precision (AP) accuracy on unseen data.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Bioinformatik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Bioinformatics (hsv//eng)
Keyword
- accuracy estimation
- datasets
- neural networks
- similarity metrics
- Learning systems
- Dataset
- Distance measure
- Frechet
- Machine learning systems
- Modeling architecture
- Neural-networks
- Performance
- Similarity analysis
- Drops
Publication and Content Type
- ref (subject category)
- kon (subject category)
Find in a library
To the university's database