Search: hsv:(NATURVETENSKAP) hsv:(Data och informationsvetenskap) hsv:(Datorteknik) >
Applying test case ...
Applying test case prioritization to software microbenchmarks
-
- Laaber, C. (author)
- Universität Zürich,University of Zürich
-
- Gall, H. C. (author)
- Universität Zürich,University of Zürich
-
- Leitner, Philipp, 1982 (author)
- Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik, Software Engineering (GU),Institutionen för data- och informationsteknik, Software Engineering (GU),Chalmers tekniska högskola,Chalmers University of Technology
-
(creator_code:org_t)
- 2021-09-30
- 2021
- English.
-
In: Empirical Software Engineering. - : Springer Science and Business Media LLC. - 1382-3256 .- 1573-7616. ; 26:6
- Related links:
-
https://link.springe...
-
show more...
-
https://research.cha... (primary) (free)
-
https://gup.ub.gu.se...
-
https://doi.org/10.1...
-
https://research.cha...
-
show less...
Abstract
Subject headings
Close
- Regression testing comprises techniques which are applied during software evolution to uncover faults effectively and efficiently. While regression testing is widely studied for functional tests, performance regression testing, e.g., with software microbenchmarks, is hardly investigated. Applying test case prioritization (TCP), a regression testing technique, to software microbenchmarks may help capturing large performance regressions sooner upon new versions. This may especially be beneficial for microbenchmark suites, because they take considerably longer to execute than unit test suites. However, it is unclear whether traditional unit testing TCP techniques work equally well for software microbenchmarks. In this paper, we empirically study coverage-based TCP techniques, employing total and additional greedy strategies, applied to software microbenchmarks along multiple parameterization dimensions, leading to 54 unique technique instantiations. We find that TCP techniques have a mean APFD-P (average percentage of fault-detection on performance) effectiveness between 0.54 and 0.71 and are able to capture the three largest performance changes after executing 29% to 66% of the whole microbenchmark suite. Our efficiency analysis reveals that the runtime overhead of TCP varies considerably depending on the exact parameterization. The most effective technique has an overhead of 11% of the total microbenchmark suite execution time, making TCP a viable option for performance regression testing. The results demonstrate that the total strategy is superior to the additional strategy. Finally, dynamic-coverage techniques should be favored over static-coverage techniques due to their acceptable analysis overhead; however, in settings where the time for prioritzation is limited, static-coverage techniques provide an attractive alternative.
Subject headings
- NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datorteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Engineering (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Programvaruteknik (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Software Engineering (hsv//eng)
- NATURVETENSKAP -- Data- och informationsvetenskap -- Datavetenskap (hsv//swe)
- NATURAL SCIENCES -- Computer and Information Sciences -- Computer Sciences (hsv//eng)
Keyword
- performance testing
- software microbenchmarking
- test case
- prioritization
- regression testing
- JMH
- performance
- systems
- Computer Science
- JMH
Publication and Content Type
- ref (subject category)
- art (subject category)
Find in a library
To the university's database