| 1. |
- Laaber, C., et al.
(författare)
-
An Evaluation of Open-Source Software Microbenchmark Suites for Continuous Performance Assessment
- 2018
-
Ingår i: MSR '18 Proceedings of the 15th International Conference on Mining Software Repositories. - New York, NY, USA : ACM Digital Library. - 9781450357166 ; , s. 119-130, s. 119-130
-
Bokkapitel (övrigt vetenskapligt/konstnärligt)abstract
- Continuous integration (CI) emphasizes quick feedback to developers. This is at odds with current practice of performance testing, which predominantely focuses on long-running tests against entire systems in production-like environments. Alternatively, software microbenchmarking attempts to establish a performance baseline for small code fragments in short time. This paper investigates the quality of microbenchmark suites with a focus on suitability to deliver quick performance feedback and CI integration. We study ten open-source libraries written in Java and Go with benchmark suite sizes ranging from 16 to 983 tests, and runtimes between 11 minutes and 8.75 hours. We show that our study subjects include benchmarks with result variability of 50% or higher, indicating that not all benchmarks are useful for reliable discovery of slow-downs. We further artificially inject actual slowdowns into public API methods of the study subjects and test whether test suites are able to discover them. We introduce a performance-test quality metric called the API benchmarking score (ABS). ABS represents a benchmark suite's ability to find slowdowns among a set of defined core API methods. Resulting benchmarking scores (i.e., fraction of discovered slowdowns) vary between 10% and 100% for the study subjects. This paper's methodology and results can be used to (1) assess the quality of existing microbenchmark suites, (2) select a set of tests to be run as part of CI, and (3) suggest or generate benchmarks for currently untested parts of an API.
|
|
| 2. |
- Soto-Valero, C., et al.
(författare)
-
Detection and analysis of behavioral T-patterns in debugging activities
- 2018
-
Ingår i: MSR '18 Proceedings of the 15th International Conference on Mining Software Repositories. - New York, NY, USA : IEEE Computer Society. - 9781450357166 ; , s. 110-113
-
Konferensbidrag (refereegranskat)abstract
- A growing body of research in empirical software engineering applies recurrent patterns analysis in order to make sense of the developers' behavior during their interactions with IDEs. However, the exploration of hidden real-time structures of programming behavior remains a challenging task. In this paper, we investigate the presence of temporal behavioral patterns (T-patterns) in debugging activities using the THEME software. Our preliminary exploratory results show that debugging activities are strongly correlated with code editing, file handling, window interactions and other general types of programming activities. The validation of our T-patterns detection approach demonstrates that debugging activities are performed on the basis of repetitive and well-organized behavioral events. Furthermore, we identify a large set of T-patterns that associate debugging activities with build success, which corroborates the positive impact of debugging practices on software development.
|
|
