PEAS : A Performance Evaluation framework for Auto-Scaling strategies in cloud applications

• Numerous auto-scaling strategies have been proposed in the last few years for improving various Quality of Service (QoS)indicators of cloud applications, e.g., response time and throughput, by adapting the amount of resources assigned to theapplication to meet the workload demand. However, the evaluation of a proposed auto-scaler is usually achieved throughexperiments under specific conditions, and seldom includes extensive testing to account for uncertainties in the workloads, andunexpected behaviors of the system. These tests by no means can provide guarantees about the behavior of the system in generalconditions. In this paper, we present PEAS, a Performance Evaluation framework for Auto-Scaling strategies in the presenceof uncertainties. The evaluation is formulated as a chance constrained optimization problem, which is solved using scenariotheory. The adoption of such a technique allows one to give probabilistic guarantees of the obtainable performance. Six differentauto-scaling strategies have been selected from the literature for extensive test evaluation, and compared using the proposedframework. We build a discrete event simulator and parameterize it based on real experiments. Using the simulator, each auto-scaler’s performance is evaluated using 796 distinct real workload traces from projects hosted on the Wikimedia foundations’servers, and their performance is compared using PEAS. The evaluation is carried out using different performance metrics,highlighting the flexibility of the framework, while providing probabilistic bounds on the evaluation and the performance of thealgorithms. Our results highlight the problem of generalizing the conclusions of the original published studies and show thatbased on the evaluation criteria, a controller can be shown to be better than other controllers.

Ämnesord

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik -- Datorsystem (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering -- Computer Systems (hsv//eng)

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering (hsv//eng)

Nyckelord

Performance evaluation

auto-scaling

randomized optimization

elasticity

cloud computing

Publikations- och innehållstyp

ref (ämneskategori)

art (ämneskategori)