Sökning: onr:"swepub:oai:DiVA.org:ri-48930" >
Performance analysi...
Performance analysis of out-of-distribution detection on trained neural networks
-
- Henriksson, Jens, 1991 (författare)
- Semcon Sweden AB, Sweden; Chalmers Institute of Technology, Sweden,Chalmers tekniska högskola,Chalmers University of Technology,Semcon
-
- Berger, Christian, 1980 (författare)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU),Chalmers Institute of Technology, Sweden; University of Gothenburg, Sweden
-
- Borg, Markus (författare)
- RISE,Mobilitet och system,RISE Research Institutes of Sweden
-
visa fler...
-
- Tornberg, Lars, 1979 (författare)
- Volvo Cars, Sweden
-
- Sathyamoorthy, Sankar Raman, 1984 (författare)
- QRTech AB, Sweden,Qrtech AB
-
- Englund, Cristofer, 1977- (författare)
- RISE,Mobilitet och system,RISE Research Institutes of Sweden
-
visa färre...
-
(creator_code:org_t)
- Elsevier B.V. 2020
- 2020
- Engelska.
-
Ingår i: Information and Software Technology. - : Elsevier B.V.. - 0950-5849 .- 1873-6025.
- Relaterad länk:
-
https://urn.kb.se/re...
-
visa fler...
-
https://doi.org/10.1...
-
https://research.cha...
-
https://gup.ub.gu.se...
-
visa färre...
Abstract
Ämnesord
Stäng
- Context: Deep Neural Networks (DNN) have shown great promise in various domains, for example to support pattern recognition in medical imagery. However, DNNs need to be tested for robustness before being deployed in safety critical applications. One common challenge occurs when the model is exposed to data samples outside of the training data domain, which can yield to outputs with high confidence despite no prior knowledge of the given input. Objective: The aim of this paper is to investigate how the performance of detecting out-of-distribution (OOD) samples changes for outlier detection methods (e.g., supervisors) when DNNs become better on training samples. Method: Supervisors are components aiming at detecting out-of-distribution samples for a DNN. The experimental setup in this work compares the performance of supervisors using metrics and datasets that reflect the most common setups in related works. Four different DNNs with three different supervisors are compared during different stages of training, to detect at what point during training the performance of the supervisors begins to deteriorate. Results: Found that the outlier detection performance of the supervisors increased as the accuracy of the underlying DNN improved. However, all supervisors showed a large variation in performance, even for variations of network parameters that marginally changed the model accuracy. The results showed that understanding the relationship between training results and supervisor performance is crucial to improve a model's robustness. Conclusion: Analyzing DNNs for robustness is a challenging task. Results showed that variations in model parameters that have small variations on model predictions can have a large impact on the out-of-distribution detection performance. This kind of behavior needs to be addressed when DNNs are part of a safety critical application and hence, the necessary safety argumentation for such systems need be structured accordingly.
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Bioinformatik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Bioinformatics (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Datorsystem (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Computer Systems (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Software Engineering (hsv//eng)
Nyckelord
- Automotive perception
- Deep neural networks
- Out-of-distribution
- Robustness
- Safety-critical systems
- Anomaly detection
- Data handling
- Pattern recognition
- Safety engineering
- Statistics
- Supervisory personnel
- Detection performance
- Different stages
- Model parameters
- Network parameters
- Performance analysis
- Safety critical applications
- Small variations
- Trained neural networks
- Neural networks
- Deep neural networks
- Robustness
- Out-of-distribution
- Automotive
- perception
- Safety-critical systems
- Computer Science
Publikations- och innehållstyp
- ref (ämneskategori)
- art (ämneskategori)
Hitta via bibliotek
Till lärosätets databas