Using application benchmark call graphs to quantify and improve the practical relevance of microbenchmark suites

↓ Direkt till sidans innehåll
↓ Direkt till sidans sekundära innehåll (sidomenyn)

Search: WFRF:(Kommula Snehitha M.) > Using application b...

1 of 1
Previous record
Next record
To hitlist

Using application benchmark call graphs to quantify and improve the practical relevance of microbenchmark suites

Grambow, M. (author): Einstein Center Digital Future,Technische Universität Berlin

Laaber, C. (author): Universität Zürich,University of Zürich

Leitner, Philipp, 1982 (author): Gothenburg University,Göteborgs universitet,Institutionen för data- och informationsteknik, Software Engineering (GU),Institutionen för data- och informationsteknik, Software Engineering (GU),Chalmers tekniska högskola,Chalmers University of Technology

Bermbach, D. (author): Einstein Center Digital Future,Technische Universität Berlin

show less...

(creator_code:org_t)

2021-05-28
2021
English.
In: Peerj Computer Science. - : PeerJ. - 2376-5992. ; 7

Related links:: https://doi.org/10.7...; show more...; https://research.cha... (primary) (free); https://gup.ub.gu.se...; https://doi.org/10.7...; https://research.cha...; show less...

Journal article (peer-reviewed)

Abstract Subject headings

Performance problems in applications should ideally be detected as soon as they occur, i.e., directly when the causing code modification is added to the code repository. To this end, complex and cost-intensive application benchmarks or lightweight but less relevant microbenchmarks can be added to existing build pipelines to ensure performance goals. In this paper, we show how the practical relevance of microbenchmark suites can be improved and verified based on the application flow during an application benchmark run. We propose an approach to determine the overlap of common function calls between application and microbenchmarks, describe a method which identifies redundant microbenchmarks, and present a recommendation algorithm which reveals relevant functions that are not covered by microbenchmarks yet. A microbenchmark suite optimized in this way can easily test all functions determined to be relevant by application benchmarks after every code change, thus, significantly reducing the risk of undetected performance problems. Our evaluation using two time series databases shows that, depending on the specific application scenario, application benchmarks cover different functions of the system under test. Their respective microbenchmark suites cover between 35.62% and 66.29% of the functions called during the application benchmark, offering substantial room for improvement. Through two use cases-removing redundancies in the microbenchmark suite and recommendation of yet uncovered functions-we decrease the total number of microbenchmarks and increase the practical relevance of both suites. Removing redundancies can significantly reduce the number of microbenchmarks (and thus the execution time as well) to similar to 10% and similar to 23% of the original microbenchmark suites, whereas recommendation identifies up to 26 and 14 newly, uncovered functions to benchmark to improve the relevance. By utilizing the differences and synergies of application benchmarks and microbenchmarks, our approach potentially enables effective software performance assurance with performance tests of multiple granularities.

Subject headings

NATURVETENSKAP -- Data- och informationsvetenskap (hsv//swe)
NATURAL SCIENCES -- Computer and Information Sciences (hsv//eng)
NATURVETENSKAP -- Data- och informationsvetenskap -- Datorteknik (hsv//swe)
NATURAL SCIENCES -- Computer and Information Sciences -- Computer Engineering (hsv//eng)
TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Inbäddad systemteknik (hsv//swe)
ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Embedded Systems (hsv//eng)
TEKNIK OCH TEKNOLOGIER -- Elektroteknik och elektronik -- Datorsystem (hsv//swe)
ENGINEERING AND TECHNOLOGY -- Electrical Engineering, Electronic Engineering, Information Engineering -- Computer Systems (hsv//eng)

Keyword

Benchmarking
Time series databases
Microbenchmarking
Performance
testing
performance
Computer Science
Microbenchmarking

Publication and Content Type

ref (subject category)
art (subject category)

Find in a library

Peerj Computer Science (Search for host publication in LIBRIS)

To the university's database

1 of 1
Previous record
Next record
To hitlist

Find more in SwePub

By the author/editor: Grambow, M.; Laaber, C.; Leitner, Philipp ...; Bermbach, D.

About the subject

NATURAL SCIENCES: NATURAL SCIENCES; and Computer and Inf ...

NATURAL SCIENCES: NATURAL SCIENCES; and Computer and Inf ...; and Computer Enginee ...

ENGINEERING AND TECHNOLOGY: ENGINEERING AND ...; and Electrical Engin ...; and Embedded Systems

ENGINEERING AND TECHNOLOGY: ENGINEERING AND ...; and Electrical Engin ...; and Computer Systems

Articles in the publication: Peerj Computer S ...

By the university: University of Gothenburg; Chalmers University of Technology

Search outside SwePub

Extend your search to:: Google; Google Book Search; Google Scholar

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.

LIBRIS.kb.se