id:"swepub:oai:gup.ub.gu.se/316073"
Search: id:"swepub:oai:gup.ub.gu.se/316073" > A Multi-factor Appr...
- 1 of 1
- Previous record
- Next record
- To hitlist
A Multi-factor Approach for Flaky Test Detection and Automated Root Cause Analysis
-
- Ahmad, Azeem (author)
- Linköpings universitet,Programvara och system,Tekniska fakulteten
-
- de Oliveira Neto, Francisco Gomes (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU),Chalmers & Univ Gothenburg, Sweden
-
- Shi, Zhixiang (author)
- Linköpings universitet,Institutionen för datavetenskap,Tekniska fakulteten
- show more...
- show less...
- (creator_code:org_t)
- IEEE COMPUTER SOC, 2021
- 2021
- English.
- In: Proceedings - Asia-Pacific Software Engineering Conference, APSEC. - : IEEE COMPUTER SOC. - 1530-1362. ; , s. 338-348
- Conference paper (peer-reviewed)
Abstract
Subject headings
Close
- Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase, hence reducing the confidence of developers during maintenance as well as in the quality of a product. Re-running test cases to reveal flakiness is resource-consuming, unreliable and does not reveal the root causes of test flakiness. Our paper evaluates a multi-factor approach to identify flaky test executions implemented in a tool named MDF laker. The four factors are: trace-back coverage, flaky frequency, number of test smells, and test size. Based on the extracted factors, MDFlaker uses k-Nearest Neighbor (KNN) to determine whether failed test executions are flaky. We investigate MDFlaker in a case study with 2166 test executions from different open-source repositories. We evaluate the effectiveness of our flaky detection tool. We illustrate how the multi-factor approach can be used to reveal root causes for flakiness, and we conduct a qualitative comparison between MDF laker and other tools proposed in literature. Our results show that the combination of different factors can be used to identify flaky tests. Each factor has its own trade-off, e.g., trace-back leads to many true positives, while flaky frequency yields more true negatives. Therefore, specific combinations of factors enable classification for testers with limited information (e.g., not enough test history information).
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Software Engineering (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Keyword
- automated root-cause analysis
- flaky test detection
- flaky tests
- non-deterministic tests
- trace-back
- flaky tests; non-deterministic tests; flaky test detection; automated root-cause analysis; trace-back
Publication and Content Type
- ref (subject category)
- kon (subject category)
Find in a library
- Proceedings - Asia-Pacific Software Engineering Conference, APSEC (Search for host publication in LIBRIS)
To the university's database
- 1 of 1
- Previous record
- Next record
- To hitlist
Find more in SwePub
- By the author/editor
- Ahmad, Azeem
- de Oliveira Neto ...
- Shi, Zhixiang
- Sandahl, Kristia ...
- Leifler, Ola
- About the subject
-
- NATURAL SCIENCES
- NATURAL SCIENCES
- and Computer and Inf ...
- and Software Enginee ...
-
- NATURAL SCIENCES
- NATURAL SCIENCES
- and Computer and Inf ...
- and Computer Science ...
- Articles in the publication
- Proceedings - As ...
- 2021 28TH ASIA-P ...
- By the university
- University of Gothenburg
- Linköping University
Search outside SwePub
- Extend your search to:
- Google Book Search
- Google Scholar
Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.
- Copyright © LIBRIS - National Library Systems
- LIBRIS.kb.se
