Search: LAR1:gu
> Halmstad University
> Chalmers University of Technology >
Towards Structured ...
Towards Structured Evaluation of Deep Neural Network Supervisors
-
- Henriksson, Jens, 1991 (author)
- Semcon AB, Gothenburg, Sweden,Semcon
-
- Berger, Christian, 1980 (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU)
-
- Borg, Markus (author)
- RISE,SICS,RISE Research Institutes of Sweden AB, Lund and Gothenburg, Sweden,RISE Research Institutes of Sweden
-
show more...
-
- Tornberg, Lars, 1979 (author)
- Machine Learning and AI Center of Excellence, Volvo Cars, Gothenburg, Sweden,AstraZeneca AB
-
- Englund, Cristofer (author)
- RISE,Viktoria,RISE Research Institutes of Sweden AB, Lund and Gothenburg, Sweden,RISE Research Institutes of Sweden
-
- Sathyamoorthy, Sankar Raman, 1984 (author)
- QRTech AB, Gothenburg, Sweden,Qrtech AB
-
- Ursing, Stig (author)
- Semcon AB, Gothenburg, Sweden,Semcon
-
show less...
-
(creator_code:org_t)
- New York : Institute of Electrical and Electronics Engineers Inc. 2019
- 2019
- English.
-
In: Proceedings - 2019 IEEE International Conference on Artificial Intelligence Testing, AITest 2019. - New York : Institute of Electrical and Electronics Engineers Inc.. - 9781728104928 ; 1
- Related links:
-
https://urn.kb.se/re...
-
show more...
-
https://doi.org/10.1...
-
https://urn.kb.se/re...
-
https://research.cha...
-
https://gup.ub.gu.se...
-
show less...
Abstract
Subject headings
Close
- Deep Neural Networks (DNN) have improved the quality of several non-safety related products in the past years. However, before DNNs should be deployed to safety-critical applications, their robustness needs to be systematically analyzed. A common challenge for DNNs occurs when input is dissimilar to the training set, which might lead to high confidence predictions despite proper knowledge of the input. Several previous studies have proposed to complement DNNs with a supervisor that detects when inputs are outside the scope of the network. Most of these supervisors, however, are developed and tested for a selected scenario using a specific performance metric. In this work, we emphasize the need to assess and compare the performance of supervisors in a structured way. We present a framework constituted by four datasets organized in six test cases combined with seven evaluation metrics. The test cases provide varying complexity and include data from publicly available sources as well as a novel dataset consisting of images from simulated driving scenarios. The latter we plan to make publicly available. Our framework can be used to support DNN supervisor evaluation, which in turn could be used to motive development, validation, and deployment of DNNs in safety-critical applications.
Subject headings
- TEKNIK OCH TEKNOLOGIER -- Annan teknik (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Other Engineering and Technologies (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Software Engineering (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Datorsystem (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Computer Systems (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)
Keyword
- Automotive perception
- Deep neural networks
- Out-of-distribution
- Robustness
- Supervisor
- Artificial intelligence
- Robustness (control systems)
- Safety engineering
- Statistical tests
- Supervisory personnel
- Evaluation metrics
- High confidence
- Performance metrices
- Safety critical applications
- Safety-related products
- Simulated driving
- Structured evaluation
- deep neural networks
- robustness
- out-of-distribution
- supervisor
- automotive perception
- OCEEDINGS7th Industrial Conference on Data Mining
- JUL 14-18
- 2007
- Leipzig
- GERMANY
- V4597
- OCESSING (WCSP)7th International Conference on Wireless Communications and Signal Processing
Publication and Content Type
- ref (subject category)
- kon (subject category)
Find in a library
To the university's database