| 1. |
- Al-Shishtawy, Ahmad, 1978-, et al.
(författare)
-
Achieving Robust Self-Management for Large-Scale Distributed Applications
- 2010
-
Ingår i: Self-Adaptive and Self-Organizing Systems (SASO), 2010 4th IEEE International Conference on. - : IEEE Computer Society. - 9781424485376 ; , s. 31-40
-
Konferensbidrag (refereegranskat)abstract
- Achieving self-management can be challenging, particularly in dynamic environments with resource churn (joins/leaves/failures). Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of robust management elements (RMEs), which are able to heal themselves under continuous churn. Using RMEs allows the developer to separate the issue of dealing with the effect of churn on management from the management logic. This facilitates the development of robust management by making the developer focus on managing the application while relying on the platform to provide the robustness of management. RMEs can be implemented as fault-tolerant long-living services. We present a generic approach and an associated algorithm to achieve fault-tolerant long-living services. Our approach is based on replicating a service using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. The algorithm uses P2P replica placement schemes to place replicas and uses the P2P overlay to monitor them. The replicated state machine is extended to analyze monitoring data in order to decide on when and where to migrate. We describe how to use our approach to achieve robust management elements. We present a simulation-based evaluation of our approach which shows its feasibility.
|
|
| 2. |
- Al-Shishtawy, Ahmad, 1978-, et al.
(författare)
-
Distributed Control Loop Patterns for Managing Distributed Applications
- 2008
-
Ingår i: SASOW 2008. - LOS ALAMITOS : IEEE Computer Society. - 9781424434688 ; , s. 260-265
-
Konferensbidrag (refereegranskat)abstract
- In this paper we discuss various control loop patterns for managing distributed applications with multiple control loops. We introduce a high-level framework, called DCMS, for developing, deploying and managing component-based distributed applications in dynamic environments. The control loops, and interactions among them, are illustrated in the context of a distributed self-managing storage service implemented using DCMS to achieve various self-* properties. Different control loops are used for different self-* behaviours, which illustrates one way to divide application management, which makes for both ease of development and for better scalability and robustness when managers are distributed. As the multiple control loops are not completely independent, we demonstrate different patterns to deal with the interaction and potential conflict between multiple managers.
|
|
| 3. |
- Al-Shishtawy, Ahmad, 1978-, et al.
(författare)
-
ElastMan : Autonomic Elasticity Manager for Cloud-Based Key-Value Stores
- 2012
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- The increasing spread of elastic Cloud services, together with the pay-asyou-go pricing model of Cloud computing, has led to the need of an elasticity controller. The controller automatically resizes an elastic service, in response to changes in workload, in order to meet Service Level Objectives (SLOs) at a reduced cost. However, variable performance of Cloud virtual machines and nonlinearities in Cloud services, such as the diminishing reward of adding a service instance with increasing the scale, complicates the controller design. We present the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores. ElastMan combines feedforward and feedback control. Feedforward control is used to respond to spikes in the workload by quickly resizing the service to meet SLOs at a minimal cost. Feedback control is used to correct modeling errors and to handle diurnal workload. To address nonlinearities, our design of ElastMan leverages the near-linear scalability of elastic Cloud services in order to build a scale-independent model of the service. Our design based on combining feedforward and feedback control allows to efficiently handle both diurnal and rapid changes in workload in order to meet SLOs at a minimal cost. Our evaluation shows the feasibility of our approach to automation of Cloud service elasticity.
|
|
| 4. |
- Al-Shishtawy, Ahmad, 1978-, et al.
(författare)
-
ElastMan : Autonomic elasticity manager for cloud-based key-value stores
- 2013
-
Ingår i: HPDC 2013 - Proceedings of the 22nd ACM International Symposium on High-Performance Parallel and Distributed Computing. - New York, NY, USA : ACM. - 9781450319102 ; , s. 115-116
-
Konferensbidrag (refereegranskat)abstract
- The increasing spread of elastic Cloud services, together with the pay-as-you-go pricing model of Cloud computing, has led to the need of an elasticity controller. The controller automatically resizes an elastic service in response to changes in workload, in order to meet Service Level Objectives (SLOs) at a reduced cost. However, variable performance of Cloud virtual machines and nonlinearities in Cloud services complicates the controller design. We present the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores. ElastMan combines feedforward and feedback control. Feedforward control is used to respond to spikes in the workload by quickly resizing the service to meet SLOs at a minimal cost. Feedback control is used to correct modeling errors and to handle diurnal workload. We have implemented and evaluated ElastMan using the Voldemort key-value store running in a Cloud environment based on OpenStack. Our evaluation shows the feasibility and effectiveness of our approach to automation of Cloud service elasticity.
|
|
| 5. |
- Al-Shishtawy, Ahmad, 1978-
(författare)
-
Enabling and Achieving Self-Management for Large Scale Distributed Systems : Platform and Design Methodology for Self-Management
- 2010
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Autonomic computing is a paradigm that aims at reducing administrative overhead by using autonomic managers to make applications self-managing. To better deal with large-scale dynamic environments; and to improve scalability, robustness, and performance; we advocate for distribution of management functions among several cooperative autonomic managers that coordinate their activities in order to achieve management objectives. Programming autonomic management in turn requires programming environment support and higher level abstractions to become feasible. In this thesis we present an introductory part and a number of papers that summaries our work in the area of autonomic computing. We focus on enabling and achieving self-management for large scale and/or dynamic distributed applications. We start by presenting our platform, called Niche, for programming self-managing component-based distributed applications. Niche supports a network-transparent view of system architecture simplifying designing application self-* code. Niche provides a concise and expressive API for self-* code. The implementation of the framework relies on scalability and robustness of structured overlay networks. We have also developed a distributed file storage service, called YASS, to illustrate and evaluate Niche. After introducing Niche we proceed by presenting a methodology and design space for designing the management part of a distributed self-managing application in a distributed manner. We define design steps, that includes partitioning of management functions and orchestration of multiple autonomic managers. We illustrate the proposed design methodology by applying it to the design and development of an improved version of our distributed storage service YASS as a case study. We continue by presenting a generic policy-based management framework which has been integrated into Niche. Policies are sets of rules that govern the system behaviors and reflect the business goals or system management objectives. The policy based management is introduced to simplify the management and reduce the overhead, by setting up policies to govern system behaviors. A prototype of the framework is presented and two generic policy languages (policy engines and corresponding APIs), namely SPL and XACML, are evaluated using our self-managing file storage application YASS as a case study. Finally, we present a generic approach to achieve robust services that is based on finite state machine replication with dynamic reconfiguration of replica sets. We contribute a decentralized algorithm that maintains the set of resource hosting service replicas in the presence of churn. We use this approach to implement robust management elements as robust services that can operate despite of churn.
|
|
| 6. |
- Al-Shishtawy, Ahmad, 1978-
(författare)
-
Self-Management for Large-Scale Distributed Systems
- 2012
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Autonomic computing aims at making computing systems self-managing by using autonomic managers in order to reduce obstacles caused by management complexity. This thesis presents results of research on self-management for large-scale distributed systems. This research was motivated by the increasing complexity of computing systems and their management.In the first part, we present our platform, called Niche, for programming self-managing component-based distributed applications. In our work on Niche, we have faced and addressed the following four challenges in achieving self-management in a dynamic environment characterized by volatile resources and high churn: resource discovery, robust and efficient sensing and actuation, management bottleneck, and scale. We present results of our research on addressing the above challenges. Niche implements the autonomic computing architecture, proposed by IBM, in a fully decentralized way. Niche supports a network-transparent view of the system architecture simplifying the design of distributed self-management. Niche provides a concise and expressive API for self-management. The implementation of the platform relies on the scalability and robustness of structured overlay networks. We proceed by presenting a methodology for designing the management part of a distributed self-managing application. We define design steps that include partitioning of management functions and orchestration of multiple autonomic managers.In the second part, we discuss robustness of management and data consistency, which are necessary in a distributed system. Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of Robust Management Elements, which are able to heal themselves under continuous churn. Our approach is based on replicating a management element using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. For data consistency, we propose a majority-based distributed key-value store supporting multiple consistency levels that is based on a peer-to-peer network. The store enables the tradeoff between high availability and data consistency. Using majority allows avoiding potential drawbacks of a master-based consistency control, namely, a single-point of failure and a potential performance bottleneck.In the third part, we investigate self-management for Cloud-based storage systems with the focus on elasticity control using elements of control theory and machine learning. We have conducted research on a number of different designs of an elasticity controller, including a State-Space feedback controller and a controller that combines feedback and feedforward control. We describe our experience in designing an elasticity controller for a Cloud-based key-value store using state-space model that enables to trade-off performance for cost. We describe the steps in designing an elasticity controller. We continue by presenting the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores that combines feedforward and feedback control.
|
|
| 7. |
- Brand, Per, et al.
(författare)
-
The Role of Overlay Services In a Self-Managing Framework for Dynamic Virtual Organizations
- 2008
-
Ingår i: Making Grids Work. - Boston, MA : Springer-Verlag New York. - 9780387784489 ; , s. 153-164
-
Konferensbidrag (refereegranskat)abstract
- We combine and extend recent results in autonomic computing and structuredpeer-to-peer to build an infrastructure for constructing and managing dynamic vir-tual organizations. The paper focuses on the middle layer of the proposed infras-tructure, in-between the Niche overlay system on the bottom, and an architecture-based management system based on Jade on the top. The middle layer, theoverlay services, are responsible for all sensing and actuation carried out by theVO management. We describe in detail the API of the resource and componentoverlay services both on the management node and the nodes hosting resources.We present a simple use case demonstrating resource discovery, initial deploy-ment, self-configuration as a result of resource availability change, self-healing,self-tuning and self-protection. The advantages of the design are 1) the overlayservices are in themselves self-managing, and sensor/actuation services they pro-vide are robust, 2) management can be dealt with declaratively and at a high-level,and 3) the overlay services provide good scalability in dynamic VOs.
|
|
| 8. |
- Holst, Anders, et al.
(författare)
-
Eliciting structure in data
- 2019
-
Ingår i: CEUR Workshop Proceedings. - Aachen : CEUR-WS.
-
Konferensbidrag (refereegranskat)abstract
- This paper demonstrates how to explore and visualize different types of structure in data, including clusters, anomalies, causal relations, and higher order relations. The methods are developed with the goal of being as automatic as possible and applicable to massive, streaming, and distributed data. Finally, a decentralized learning scheme is discussed, enabling finding structure in the data without collecting the data centrally.
|
|
| 9. |
- Hägglund, Marcus, et al.
(författare)
-
COCLUBERT : Clustering Machine Learning Source Code
- 2021
-
Ingår i: 20th IEEE international conference on machine learning and applications (ICMLA 2021). - : Institute of Electrical and Electronics Engineers (IEEE). ; , s. 151-158
-
Konferensbidrag (refereegranskat)abstract
- Nowadays, we can find machine learning (ML) applications in nearly every aspect of modern life, and we see that more developers are engaged in the field than ever. In order to facilitate the development of new ML applications, it would be beneficial to provide services that enable developers to share, access, and search for source code easily. A step towards making such a service is to cluster source code by functionality. In this work, we present COCLUBERT, a BERT-based model for source code embedding based on their functionality and clustering them accordingly. We build COCLUBERT using CuBERT, a variant of BERT pre-trained on source code, and present three ways to fine-tune it for the clustering task. In the experiments, we compare COCLUBERT with a baseline model, where we cluster source code using CuBERT embedding without fine-tuning. We show that COCLUBERT significantly outperforms the baseline model by increasing the Dunn Index metric by a factor of 141, the Silhouette Score metric by a factor of two, and the Adjusted Rand Index metric by a factor of 11.
|
|
| 10. |
- Moulavi, M. Amir, et al.
(författare)
-
State-Space Feedback Control for Elastic Distributed Storage in a Cloud Environment
- 2012
-
Ingår i: ICAS 2012. - St. Maarten, Netherlands Antilles. - 9781612081878 ; , s. 589-596
-
Konferensbidrag (refereegranskat)abstract
- Elasticity in Cloud computing is an ability of asystem to scale up and down (request and release resources) in response to changes in its environment and workload. Elasticity can be achieved manually or automatically. Efforts arebeing made to automate elasticity in order to improve system performance under dynamic workloads. In this paper, we reportour experience in designing an elasticity controller for a key-value storage service deployed in a Cloud environment. To design our controller, we have adopted a control theoretic approach. Automation of elasticity is achieved by providing a feedback controller that automatically increases and decreases the number of nodes in order to meet service level objectives under high load and to reduce costs under low load. Every step in the building of a controller for elastic storage, includingsystem identification and controller design, is discussed. We have evaluated our approach by using simulation. We have developed a simulation framework EStoreSim in order to simulate anelastic key-value store in a Cloud environment and be able to experiment with different controllers. We have examined the implemented controller against specific service level objectives and evaluated the controller behavior in different scenarios. Our simulation experiments have shown the feasibility of our approach to automate elasticity of storage services using state-space feedback control.
|
|
| 11. |
- Pena, Francisco J., et al.
(författare)
-
Siambert : Siamese Bert-based Code Search
- 2022
-
Ingår i: 34th Workshop of the Swedish Artificial Intelligence Society, SAIS 2022. - : Institute of Electrical and Electronics Engineers Inc.. - 9781665471268
-
Konferensbidrag (refereegranskat)abstract
- Code Search is a practical tool that helps developers navigate growing source code repositories by connecting natural language queries with code snippets. Platforms such as StackOverflow resolve coding questions and answers; however, they cannot perform a semantic search through the code. Moreover, poorly documented code adds more complexity to search for code snippets in repositories. To tackle this challenge, this paper presents Siambert, a BERT-based model that gets the question in natural language and returns relevant code snippets. The Siambert architecture consists of two stages, where the first stage, inspired by Siamese Neural Network, returns the top K relevant code snippets to the input questions, and the second stage ranks the given snippets by the first stage. The experiments show that Siambert outperforms non-BERT-based models having improvements that range from 12% to 39% on the Recall@1 metric and improves the inference time performance, making it 15x faster than standard BERT models
|
|
| 12. |
- Vlassov, Vladimir, 1957-, et al.
(författare)
-
Niche : A Platform for Self-Managing Distributed Applications
- 2012
-
Ingår i: Formal and Practical Aspects of Autonomic Computing and Networking. - : IGI Global. - 9781609608453 - 1609608453 ; , s. 241-283
-
Bokkapitel (refereegranskat)abstract
- We present Niche, a general-purpose, distributed component management system used to develop, deploy,and execute self-managing distributed applications. Niche consists of both a component-based programming model as well as a distributed runtime environment. It is especially designed for complex distributed applications that run and manage themselves in dynamic and volatile environments. Self-management in dynamic environments is challenging due to the high rate of system or environmental changes and the corresponding need to frequently reconfigure, heal, and tune the application. The challenges are met partly by making use of an underlying overlay in the platform to provide an efficient, location-independent,and robust sensing and actuation infrastructure, and partly by allowing for maximum decentralization of management. We describe the overlay services, the execution environment, showing how the challengesin dynamic environments are met. We also describe the programming model and a high-level design methodology for developing decentralized management, illustrated by two application case studies.
|
|
