Sökning: onr:"swepub:oai:gup.ub.gu.se/316073" >
A Multi-factor Appr...
A Multi-factor Approach for Flaky Test Detection and Automated Root Cause Analysis
-
- Ahmad, Azeem (författare)
- Linköpings universitet,Programvara och system,Tekniska fakulteten
-
- de Oliveira Neto, Francisco Gomes (författare)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU),Chalmers & Univ Gothenburg, Sweden
-
- Shi, Zhixiang (författare)
- Linköpings universitet,Institutionen för datavetenskap,Tekniska fakulteten
-
visa fler...
-
- Sandahl, Kristian (författare)
- Linköpings universitet,Programvara och system,Tekniska fakulteten
-
- Leifler, Ola (författare)
- Linköpings universitet,Programvara och system,Tekniska fakulteten
-
visa färre...
-
(creator_code:org_t)
- IEEE, 2021
- 2021
- Engelska.
-
Ingår i: Proceedings - Asia-Pacific Software Engineering Conference, APSEC. - : IEEE. - 1530-1362. - 9781665437844 - 9781665437851
- Relaterad länk:
-
https://gup.ub.gu.se...
-
visa fler...
-
https://doi.org/10.1...
-
https://urn.kb.se/re...
-
visa färre...
Abstract
Ämnesord
Stäng
- Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase, hence reducing the confidence of developers during maintenance as well as in the quality of a product. Re-running test cases to reveal flakiness is resource-consuming, unreliable and does not reveal the root causes of test flakiness. Our paper evaluates a multi-factor approach to identify flaky test executions implemented in a tool named MDF laker. The four factors are: trace-back coverage, flaky frequency, number of test smells, and test size. Based on the extracted factors, MDFlaker uses k-Nearest Neighbor (KNN) to determine whether failed test executions are flaky. We investigate MDFlaker in a case study with 2166 test executions from different open-source repositories. We evaluate the effectiveness of our flaky detection tool. We illustrate how the multi-factor approach can be used to reveal root causes for flakiness, and we conduct a qualitative comparison between MDF laker and other tools proposed in literature. Our results show that the combination of different factors can be used to identify flaky tests. Each factor has its own trade-off, e.g., trace-back leads to many true positives, while flaky frequency yields more true negatives. Therefore, specific combinations of factors enable classification for testers with limited information (e.g., not enough test history information).
Ämnesord
- NATURVETENSKAP -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Software Engineering (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Nyckelord
- automated root-cause analysis
- flaky test detection
- flaky tests
- non-deterministic tests
- trace-back
- flaky tests; non-deterministic tests; flaky test detection; automated root-cause analysis; trace-back
Publikations- och innehållstyp
- ref (ämneskategori)
- kon (ämneskategori)
Hitta via bibliotek
Till lärosätets databas