Search: onr:"swepub:oai:research.chalmers.se:73bba664-d30a-4cae-ae35-5c94e50a87de" >
Deceiving Humans an...
Deceiving Humans and Machines Alike: Search-based Test Input Generation for DNNs Using Variational Autoencoders
-
Kang, Sungmin (author)
-
- Feldt, Robert, 1972 (author)
- Chalmers tekniska högskola,Chalmers University of Technology
-
- Yoo, Shin (author)
- Korea Advanced Institute of Science and Technology (KAIST)
-
(creator_code:org_t)
- 2024
- 2024
- English.
-
In: ACM Transactions on Software Engineering and Methodology. - 1049-331X .- 1557-7392. ; 33:4
- Related links:
-
https://doi.org/10.1...
-
show more...
-
https://research.cha...
-
show less...
Abstract
Subject headings
Close
- Due to the rapid adoption of Deep Neural Networks (DNNs) into larger software systems, testing of DNN-based systems has received much attention recently. While many different test adequacy criteria have been suggested, we lack effective test input generation techniques. Inputs such as images of real-world objects and scenes are not only expensive to collect but also difficult to randomly sample. Consequently, current testing techniques for DNNs tend to apply small local perturbations to existing inputs to generate new inputs. We propose SINVAD (Search-based Input space Navigation using Variational AutoencoDers), a way to sample from, and navigate over, a space of realistic inputs that resembles the true distribution in the training data. Our input space is constructed using Variational Autoencoders (VAEs), and navigated through their latent vector space. Our analysis shows that the VAE-based input space is well-aligned with human perception of what constitutes realistic inputs. Further, we show that this space can be effectively searched to achieve various testing scenarios, such as boundary testing of two different DNNs or analyzing class labels that are difficult for the given DNN to distinguish. Guidelines on how to design VAE architectures are presented as well. Our results have the potential to open the field to meaningful exploration through the space of highly structured images.
Subject headings
- TEKNIK OCH TEKNOLOGIER -- Samhällsbyggnadsteknik -- Geoteknik (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Civil Engineering -- Geotechnical Engineering (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorseende och robotik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Vision and Robotics (hsv//eng)
- TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Annan elektroteknik och elektronik (hsv//swe)
- ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Other Electrical Engineering, Electronic Engineering, Information Engineering (hsv//eng)
Keyword
- Test data generation
- deep neural network
- search-based software engineering
Publication and Content Type
- art (subject category)
- ref (subject category)
Find in a library
To the university's database