Sökning: id:"swepub:oai:gup.ub.gu.se/316073" >
A Multi-factor Appr...
-
Ahmad, AzeemLinköpings universitet,Programvara och system,Tekniska fakulteten
(författare)
A Multi-factor Approach for Flaky Test Detection and Automated Root Cause Analysis
- Artikel/kapitelEngelska2021
Förlag, utgivningsår, omfång ...
Nummerbeteckningar
-
LIBRIS-ID:oai:gup.ub.gu.se/316073
-
https://gup.ub.gu.se/publication/316073URI
-
https://doi.org/10.1109/APSEC53868.2021.00041DOI
-
https://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-186181URI
Kompletterande språkuppgifter
Ingår i deldatabas
Klassifikation
-
Ämneskategori:ref swepub-contenttype
-
Ämneskategori:kon swepub-publicationtype
Anmärkningar
-
Developers often spend time to determine whether test case failures are real failures or flaky. The flaky tests, also known as non-deterministic tests, switch their outcomes without any modification in the codebase, hence reducing the confidence of developers during maintenance as well as in the quality of a product. Re-running test cases to reveal flakiness is resource-consuming, unreliable and does not reveal the root causes of test flakiness. Our paper evaluates a multi-factor approach to identify flaky test executions implemented in a tool named MDF laker. The four factors are: trace-back coverage, flaky frequency, number of test smells, and test size. Based on the extracted factors, MDFlaker uses k-Nearest Neighbor (KNN) to determine whether failed test executions are flaky. We investigate MDFlaker in a case study with 2166 test executions from different open-source repositories. We evaluate the effectiveness of our flaky detection tool. We illustrate how the multi-factor approach can be used to reveal root causes for flakiness, and we conduct a qualitative comparison between MDF laker and other tools proposed in literature. Our results show that the combination of different factors can be used to identify flaky tests. Each factor has its own trade-off, e.g., trace-back leads to many true positives, while flaky frequency yields more true negatives. Therefore, specific combinations of factors enable classification for testers with limited information (e.g., not enough test history information).
Ämnesord och genrebeteckningar
Biuppslag (personer, institutioner, konferenser, titlar ...)
-
de Oliveira Neto, Francisco GomesGothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik (GU),Department of Computer Science and Engineering (GU),Chalmers & Univ Gothenburg, Sweden(Swepub:gu)xdeofr
(författare)
-
Shi, ZhixiangLinköpings universitet,Institutionen för datavetenskap,Tekniska fakulteten(Swepub:liu)n/a
(författare)
-
Sandahl, KristianLinköpings universitet,Programvara och system,Tekniska fakulteten(Swepub:liu)krisa34
(författare)
-
Leifler, OlaLinköpings universitet,Programvara och system,Tekniska fakulteten(Swepub:liu)olale55
(författare)
-
Linköpings universitetProgramvara och system
(creator_code:org_t)
Sammanhörande titlar
-
Ingår i:Proceedings - Asia-Pacific Software Engineering Conference, APSEC: IEEE COMPUTER SOC, s. 338-3481530-1362
-
Ingår i:2021 28TH ASIA-PACIFIC SOFTWARE ENGINEERING CONFERENCE (APSEC 2021): IEEE COMPUTER SOC, s. 338-34897816654378449781665437851
Internetlänk
Hitta via bibliotek
Till lärosätets databas