| 1. |
- Scheuner, Joel, 1991, et al.
(författare)
-
A Cloud Benchmark Suite Combining Micro and Applications Benchmarks
- 2018
-
Ingår i: ACM/SPEC International Conference on Performance Engineering Companion. - New York, NY, USA : ACM. - 9781450356299 ; , s. 161-166
-
Konferensbidrag (refereegranskat)abstract
- Micro and application performance benchmarks are commonly used to guide cloud service selection. However, they are often considered in isolation in a hardly reproducible setup with a flawed execution strategy. This paper presents a new execution methodology that combines micro and application benchmarks into a benchmark suite called RMIT Combined, integrates this suite into an automated cloud benchmarking environment, and implements a repeatable execution strategy. Additionally, a newly crafted Web serving benchmark called WPBench with three different load scenarios is contributed. A case study in the Amazon EC2 cloud demonstrates that choosing a cost-efficient instance type can deliver up to 40% better performance with 40% lower costs at the same time for the Web serving benchmark WPBench. Contrary to prior research, our findings reveal that network performance does not vary relevantly anymore. Our results also show that choosing a modern type of virtualization can improve disk utilization up to 10% for I/O-heavy workloads.
|
|
| 2. |
- Scheuner, Joel, 1991, et al.
(författare)
-
Estimating Cloud Application Performance Based on Micro-Benchmark Profiling
- 2018
-
Ingår i: 2018 IEEE 11th International Conference on Cloud Computing (CLOUD). - 2159-6190 .- 2159-6190. - 9781538672358 ; , s. 90-97
-
Konferensbidrag (refereegranskat)abstract
- The continuing growth of the cloud computing market has led to an unprecedented diversity of cloud services. To support service selection, micro-benchmarks are commonly used to identify the best performing cloud service. However, it remains unclear how relevant these synthetic micro-benchmarks are for gaining insights into the performance of real-world applications. Therefore, this paper develops a cloud benchmarking methodology that uses micro-benchmarks to profile applications and subsequently predicts how an application performs on a wide range of cloud services. A study with a real cloud provider (Amazon EC2) has been conducted to quantitatively evaluate the estimation model with 38 metrics from 23 micro-benchmarks and 2 applications from different domains. The results reveal remarkably low variability in cloud service performance and show that selected micro-benchmarks can estimate the duration of a scientific computing application with a relative error of less than 10% and the response time of a Web serving application with a relative error between 10% and 20%. In conclusion, this paper emphasizes the importance of cloud benchmarking by substantiating the suitability of micro-benchmarks for estimating application performance in comparison to common baselines but also highlights that only selected micro-benchmarks are relevant to estimate the performance of a particular application.
|
|
