Sökning: onr:"swepub:oai:DiVA.org:mdh-64446" >
Evaluating the Robu...
Evaluating the Robustness of ML Models to Out-of-Distribution Data Through Similarity Analysis
-
- Lindén, Joakim (författare)
- Mälardalens universitet,Inbyggda system,Mälardalen University, Västerås, Sweden; Saab AB, Linköping, Sweden
-
- Forsberg, Håkan (författare)
- Mälardalens universitet,Inbyggda system,Mälardalen University, Västerås, Sweden
-
- Daneshtalab, Masoud (författare)
- Mälardalens universitet,Inbyggda system,Mälardalen University, Västerås, Sweden
-
visa fler...
-
- Söderquist, Ingemar (författare)
- KTH,Elektronik och inbyggda system,Saab AB, Linköping, Sweden
-
visa färre...
-
(creator_code:org_t)
- Springer Science and Business Media Deutschland GmbH, 2023
- 2023
- Engelska.
-
Ingår i: Commun. Comput. Info. Sci.. - : Springer Science and Business Media Deutschland GmbH. - 9783031429408 ; , s. 348-359, s. 348-359
- Relaterad länk:
-
https://urn.kb.se/re...
-
visa fler...
-
https://doi.org/10.1...
-
https://urn.kb.se/re...
-
visa färre...
Abstract
Ämnesord
Stäng
- In Machine Learning systems, several factors impact the performance of a trained model. The most important ones include model architecture, the amount of training time, the dataset size and diversity. We present a method for analyzing datasets from a use-case scenario perspective, detecting and quantifying out-of-distribution (OOD) data on dataset level. Our main contribution is the novel use of similarity metrics for the evaluation of the robustness of a model by introducing relative Fréchet Inception Distance (FID) and relative Kernel Inception Distance (KID) measures. These relative measures are relative to a baseline in-distribution dataset and are used to estimate how the model will perform on OOD data (i.e. estimate the model accuracy drop). We find a correlation between our proposed relative FID/relative KID measure and the drop in Average Precision (AP) accuracy on unseen data.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Bioinformatik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Bioinformatics (hsv//eng)
Nyckelord
- accuracy estimation
- datasets
- neural networks
- similarity metrics
- Learning systems
- Dataset
- Distance measure
- Frechet
- Machine learning systems
- Modeling architecture
- Neural-networks
- Performance
- Similarity analysis
- Drops
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)
Hitta via bibliotek
Till lärosätets databas