| 1. |
- Apolonia, Nuno, 1984-
(författare)
-
On Service Optimization in Community Network Micro-Clouds
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Internet coverage in the world is still weak and local communities are required to come together and build their own network infrastructures. People collaborate for the common goal of accessing the Internet and cloud services by building Community networks (CNs).The use of Internet cloud services has grown over the last decade. Community network cloud infrastructures (i.e. micro-clouds) have been introduced to run services inside the network, without the need to consume them from the Internet. CN micro-clouds aims for not only an improved service performance, but also an entry point for an alternative to Internet cloud services in CNs. However, the adaptation of the services to be used in CN micro-clouds have their own challenges since the use of low-capacity devices and wireless connections without a central management is predominant in CNs. Further, large and irregular topology of the network, high software and hardware diversity and different service requirements in CNs, makes the CN micro-clouds a challenging environment to run local services, and to achieve service performance and quality similar to Internet cloud services. In this thesis, our main objective is the optimization of services (performance, quality) in CN micro-clouds, facilitating entrance to other services and motivating members to make use of CN micro-cloud services as an alternative to Internet services. We present an approach to handle services in CN micro-cloud environments in order to improve service performance and quality that can be approximated to Internet services, while also giving to the community motivation to use CN micro-cloud services. Furthermore, we break the problem into different levels (resource, service and middleware), propose a model that provides improvements for each level and contribute with information that helps to support the improvements (in terms of service performance and quality) in the other levels.At the resource level, we facilitate the use of community devices by utilizing virtualization techniques that isolate and manage CN micro-cloud services in order to have a multi-purpose environment that fosters services in the CN micro-cloud environment.At the service level, we build a monitoring tool tailored for CN micro-clouds that helps us to analyze service behavior and performance in CN micro-clouds. Subsequently, the information gathered enables adaptation of the services to the environment in order to improve their quality and performance under CN environments. At the middleware level, we build overlay networks as the main communication system according to the social information in order to improve paths and routes of the nodes, and improve transmission of data across the network by utilizing the relationships already established in the social network or community of practices that are related to the CNs. Therefore, service performance in CN micro-clouds can become more stable with respect to resource usage, performance and user perceived quality.
|
|
| 2. |
- Al-Shishtawy, Ahmad, 1978-
(författare)
-
Self-Management for Large-Scale Distributed Systems
- 2012
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Autonomic computing aims at making computing systems self-managing by using autonomic managers in order to reduce obstacles caused by management complexity. This thesis presents results of research on self-management for large-scale distributed systems. This research was motivated by the increasing complexity of computing systems and their management.In the first part, we present our platform, called Niche, for programming self-managing component-based distributed applications. In our work on Niche, we have faced and addressed the following four challenges in achieving self-management in a dynamic environment characterized by volatile resources and high churn: resource discovery, robust and efficient sensing and actuation, management bottleneck, and scale. We present results of our research on addressing the above challenges. Niche implements the autonomic computing architecture, proposed by IBM, in a fully decentralized way. Niche supports a network-transparent view of the system architecture simplifying the design of distributed self-management. Niche provides a concise and expressive API for self-management. The implementation of the platform relies on the scalability and robustness of structured overlay networks. We proceed by presenting a methodology for designing the management part of a distributed self-managing application. We define design steps that include partitioning of management functions and orchestration of multiple autonomic managers.In the second part, we discuss robustness of management and data consistency, which are necessary in a distributed system. Dealing with the effect of churn on management increases the complexity of the management logic and thus makes its development time consuming and error prone. We propose the abstraction of Robust Management Elements, which are able to heal themselves under continuous churn. Our approach is based on replicating a management element using finite state machine replication with a reconfigurable replica set. Our algorithm automates the reconfiguration (migration) of the replica set in order to tolerate continuous churn. For data consistency, we propose a majority-based distributed key-value store supporting multiple consistency levels that is based on a peer-to-peer network. The store enables the tradeoff between high availability and data consistency. Using majority allows avoiding potential drawbacks of a master-based consistency control, namely, a single-point of failure and a potential performance bottleneck.In the third part, we investigate self-management for Cloud-based storage systems with the focus on elasticity control using elements of control theory and machine learning. We have conducted research on a number of different designs of an elasticity controller, including a State-Space feedback controller and a controller that combines feedback and feedforward control. We describe our experience in designing an elasticity controller for a Cloud-based key-value store using state-space model that enables to trade-off performance for cost. We describe the steps in designing an elasticity controller. We continue by presenting the design and evaluation of ElastMan, an elasticity controller for Cloud-based elastic key-value stores that combines feedforward and feedback control.
|
|
| 3. |
- Issa, Shady, 1989-
(författare)
-
Techniques for Enhancing the Efficiency of Transactional Memory Systems
- 2018
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Transactional Memory (TM) is an emerging programming paradigm that drastically simplifies the development of concurrent applications by relieving programmers from a major source of complexity: how to ensure correct, yet efficient, synchronization of concurrent accesses to shared memory. Despite the large body of research devoted to this area, existing TM systems still suffer from severe limitations that hamper both their performance and energy efficiency.This dissertation tackles the problem of how to build efficient implementations of the TM abstraction by introducing innovative techniques that address three crucial limitations of existing TM systems by: (i) extending the effective capacity of Hardware TM (HTM) implementations; (ii) reducing the synchronization overheads in Hybrid TM (HyTM) systems; (iii) enhancing the efficiency of TM applications via energy-aware contention management schemes.The first contribution of this dissertation, named POWER8-TM (P8TM), addresses what is arguably one of the most compelling limitations of existing HTM implementations: the inability to process transactions whose footprint exceeds the capacity of the processor's cache. By leveraging, in an innovative way, two hardware features provided by IBM POWER8 processors, namely Rollback-only Transactions and Suspend/Resume, P8TM can achieve up to 7x performance gains in workloads that stress the capacity limitations of HTM.The second contribution is Dynamic Memory Partitioning-TM (DMP-TM), a novel Hybrid TM (HyTM) that offloads the cost of detecting conflicts between HTM and Software TM (STM) to off-the-shelf operating system memory protection mechanisms. DMP-TM's design is agnostic to the STM algorithm and has the key advantage of allowing for integrating, in an efficient way, highly scalable STM implementations that would, otherwise, demand expensive instrumentation of the HTM path. This allows DMP-TM to achieve up to 20x speedups compared to state of the art HyTM solutions in uncontended workloads.The third contribution, Green-CM, is an energy-aware Contention Manager (CM) that has two main innovative aspects: (i) a novel asymmetric design, which combines different back-off policies in order to take advantage of Dynamic Frequency and Voltage Scaling (DVFS) hardware capabilities, available in most modern processors; (ii) an energy efficient implementation of a fundamental building block for many CM implementations, namely, the mechanism used to back-off threads for a predefined amount of time. Thanks to its innovative design, Green-CM can reduce the Energy Delay Product by up to 2.35x with respect to state of the art CMs.All the techniques proposed in this dissertation share an important common feature that is essential to preserve the ease of use of the TM abstraction: the reliance on on-line self-tuning mechanisms that ensure robust performance even in presence of heterogeneous workloads, without requiring prior knowledge of the target workloads or architecture.
|
|
| 4. |
- Xhagjika, Vamis, 1986-
(författare)
-
Resource, Data and Application Management for Cloud Federations and Multi-Clouds
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Distributed Real-Time Media Processing refers to classes of highly distributed, delay no-tolerant applications that account for the majority of the data traffic generated in the world today. Real-Time audio/video conferencing and live content streaming are of particular research interests as technology forecasts predict video traffic surpassing every other type of data traffic in the world in the near future. Live streaming refers to applications in which audio/video streams from a source need to be delivered to a set of geo-distributed destinations while maintaining low latency of stream delivery. Real-time conferencing platforms are application platforms that implement many-to-many audio/video real-time communications. Both of these categories exhibit high sensitivity to both network state (latency, jitter, packet loss, bit rate) as well as stream processing backend load profiles (latency and jitter introduced as Cloud processing of media packets). This thesis addresses enhancing real-time media processing both at the network level parameters as well as Cloud optimisations.We provide a novel, bandwidth management algorithm, for cloud services sharing the same network infrastructure, which provides a 2x improvement in system stability. Further examining network impact on cloud services, we provide a novel hybrid Cloud-Network distributed Cloud architecture to enable locality aware, application enhancements. This architecture led to a multi-cloud management overlay algorithm that maintains low management overhead on large scale cloud deployments. On the application level we provide a study of Media Quality parameters for a WebRTC enabled Media Cloud back-end, and provide patterns of quality metrics with respect to back-end stream load and network parameters. Additionally we empirically show that a "minimal load" algorithm for stream allocation, outperforms other Rotational, or Static Threshold based algorithms.
|
|
| 5. |
- Zeng, Jingna, 1985-
(författare)
-
Augmenting Transactional Memory with the Future Abstraction
- 2020
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The advent of multicore systems spurred great interest in Transactional Memory (TM). TM is a parallel programming paradigm that coordinates concurrent threads by incorporating transactions into programming languages. TM shifts the burden of coordinating and synchronizing concurrent threads from programmers to the compiler, run-time environment, or even hardware.Unfortunately, though, current state-of-the-art TM implementations lack support for a powerful and intuitive abstraction that is widely used in modern parallel programming environments (e.g., C++, Java and JavaScript), namely "futures". The future abstraction is widely used due to its ability to express in a natural way opportunities for parallelism as well as logical dependencies among parallel tasks. Yet, perhaps surprisingly, the problem of how to support the future abstraction in a TM implementation has not been studied in the literature, although futures represent a natural and convenient means to enable intra-transaction parallelism in long-running transactions.This dissertation aims at filling precisely this relevant gap in the literature by investigating how to reconcile the TM and the future abstractions.This limitation is tackled by introducing a novel abstraction, called transactional futures,i.e., transactions that execute wrapped within futures and that are spawned and evaluated by other transactions or transactional futures.The semantics of transactional futures describes the allowed behaviors with regards to properties such as atomicity and isolation. Multiple semantics are defined, and the trade-offs between ease of use and efficiency are explored for each. Based on these semantics, this dissertation presents two novel TM implementations. The efficiency of the proposed transactional futures abstraction is evaluated in a real system using the TM implementations.Finally, this dissertation addresses the problem of self-tuning the parallelism degree in TM systems that support intra-transaction parallelism. This goal is achieved by presenting an online learning system that combines model-driven (Sequential Based Bayesian Optimization) and local searches techniques, as well as adaptive performance monitoring techniques.
|
|
| 6. |
- Awan, Ahsan Javed, 1988-
(författare)
-
Performance Characterization of In-Memory Data Analytics on a Scale-up Server
- 2016
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- The sheer increase in volume of data over the last decade has triggered research in cluster computing frameworks that enable web enterprises to extract big insights from big data. While Apache Spark defines the state of the art in big data analytics platforms for (i) exploiting data-flow and in-memory computing and (ii) for exhibiting superior scale-out performance on the commodity machines, little effort has been devoted at understanding the performance of in-memory data analytics with Spark on modern scale-up servers. This thesis characterizes the performance of in-memory data analytics with Spark on scale-up servers.Through empirical evaluation of representative benchmark workloads on a dual socket server, we have found that in-memory data analytics with Spark exhibit poor multi-core scalability beyond 12 cores due to thread level load imbalance and work-time inflation. We have also found that workloads are bound by the latency of frequent data accesses to DRAM. By enlarging input data size, application performance degrades significantly due to substantial increase in wait time during I/O operations and garbage collection, despite 10% better instruction retirement rate (due to lower L1 cache misses and higher core utilization).For data accesses we have found that simultaneous multi-threading is effective in hiding the data latencies. We have also observed that (i) data locality on NUMA nodes can improve the performance by 10% on average, (ii) disabling next-line L1-D prefetchers can reduce the execution time by up-to 14%. For GC impact, we match memory behaviour with the garbage collector to improve performance of applications between 1.6x to 3x. and recommend to use multiple small executors that can provide up-to 36% speedup over single large executor.
|
|
| 7. |
- Dokoohaki, Nima, 1980-
(författare)
-
Trust-Based User Profiling
- 2013
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- We have introduced the notion of user profiling with trust, as a solution to theproblem of uncertainty and unmanageable exposure of personal data duringaccess, retrieval and consumption by web applications. Our solution sug-gests explicit modeling of trust and embedding trust metrics and mechanismswithin very fabric of user profiles. This has in turn allowed information sys-tems to consume and understand this extra knowledge in order to improveinteraction and collaboration among individuals and system. When formaliz-ing such profiles, another challenge is to realize increasingly important notionof privacy preferences of users. Thus, the profiles are designed in a way toincorporate preferences of users allowing target systems to understand pri-vacy concerns of users during their interaction. A majority of contributionsof this work had impact on profiling and recommendation in digital librariescontext, and was implemented in the framework of EU FP7 Smartmuseumproject. Highlighted results start from modeling of adaptive user profilesincorporating users taste, trust and privacy preferences. This in turn led toproposal of several ontologies for user and content characteristics modeling forimproving indexing and retrieval of user content and profiles across the plat-form. Sparsity and uncertainty of profiles were studied through frameworksof data mining and machine learning of profile data taken from on-line so-cial networks. Results of mining and population of data from social networksalong with profile data increased the accuracy of intelligent suggestions madeby system to improving navigation of users in on-line and off-line museum in-terfaces. We also introduced several trust-based recommendation techniquesand frameworks capable of mining implicit and explicit trust across ratingsnetworks taken from social and opinion web. Resulting recommendation al-gorithms have shown to increase accuracy of profiles, through incorporationof knowledge of items and users and diffusing them along the trust networks.At the same time focusing on automated distributed management of profiles,we showed that coverage of system can be increased effectively, surpassingcomparable state of art techniques. We have clearly shown that trust clearlyelevates accuracy of suggestions predicted by system. To assure overall pri-vacy of such value-laden systems, privacy was given a direct focus when archi-tectures and metrics were proposed and shown that a joint optimal setting foraccuracy and perturbation techniques can maintain accurate output. Finally,focusing on hybrid models of web data and recommendations motivated usto study impact of trust in the context of topic-driven recommendation insocial and opinion media, which in turn helped us to show that leveragingcontent-driven and tie-strength networks can improve systems accuracy forseveral important web computing tasks.
|
|
| 8. |
- Paul, Ruma, 1985-
(författare)
-
Building Distributed Systems for High-Stress Environments using Reversibility and Phase-Awareness
- 2016
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Large-scale applications for mobile devices and Internet of Things live in stressful real-world environments: they have both continuous faults and bursts of high faults. Typical faults are node crashes, network partitions, and communication delays. In this thesis, we propose a principled approach to build applications that survive in such environments by using the concepts of Reversibility and Phase. A system is Reversible if the set of operations it provides depends on its current stress, and not on the history of the stress. By stress we imply all the potential perturbing effects of the environment on the system, which includes both faults and other nonfunctional properties such as communication delay and bandwidth. Reversibility generalizes standard fault tolerance with nested fault models. When the stress causes the fault rate to go outside one model then it is still inside the scope of the next model. As stress is a global condition that cannot easily be measured by individual nodes, we propose the concept of Phase in order to approximate the set of available operations of the system at each node. Phase is a per-node property, and can be determined with no additional distributed computation. We present two case studies. First, we present a transactional key-value store built on a structured overlay network and we explain how to make it Reversible. Second, we present a distributed collaborative graphic editor built on top of the key-value store, and we explain how to make it Phase-Aware, i.e., it optimizes its behavior according to a real-time observation of phase at each node using a Phase API. This shows the usefulness of Reversibility and Phase-Awareness for building large-scale Internet applications.
|
|
| 9. |
- Paul, Ruma R., 1985-
(författare)
-
An Empirical Study of the Global Behavior of Structured Overlay Networks as Complex Systems
- 2015
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Distributed applications built on top of Structured Overlay Networks (SONs) operate based on certain self-* behaviors of the underlying Peer-to-Peer network. Among those, self-organization and self-healing are the two most prominent and assumed properties. The operating environment of distributed systems continues to be more inhospitable with the advance and demand of new technologies; for example in case of mobile and ad hoc networks Churn (node turnover) can be extremely high due to node mobility, frequent disconnects/reconnects and configuration changes. Also, in such dynamic environments, the system may face high Churn (node turnover) and Network partition in a frequent manner. The situation becomes worse if the self-healing behavior of underlying SON is not complete and well defined. This implies the following non-trivial questions: Can the maintenance mechanism of a SON heal the damage to the structure due to harshness of the operating environment and reverse it back? What are the pre-conditions; in other words, what properties the healing mechanism should possess in order to achieve reversibility against stressful environments? Existing literature lacks such assessment and verification study of the self-healing property of a SON.In this thesis, we investigate both the behavior and design of a system that operate in inhospitable environments. This work is relevant to systems with both peaks of high stress (e.g. partitions, churn, network dynamicity etc.) and continuous high stress. We evaluate existing overlay maintenance strategies, namely Correction-on-Change, Correction-on-Use, Periodic Stabilization, and Ring Merge. We define the reversibility property of a system as its ability to repair itself to its original state. We propose a new strategy, called Knowledge Base, to improve conditions for reversibility against inhospitable environments. By means of simulations, we demonstrate reversibility for overlay networks with high levels of partition and churn. We make general conclusions about the ability of the maintenance strategies to achieve reversibility.Identification of Phase Transitions in a SON can provide useful information about the properties of each state of the system. Also, this enables to find the critical points in the operating space and parameters influencing them. The applications running on top of the SON can potentially utilize this knowledge to adapt its operation accordingly in different system states. In this thesis, a representative ring-based SON, namely Beernet is chosen and extended to achieve reversibility. The resulting overlay, Beernet++ exhibits reversible phase transitions under churn. We analyze the critical points observed during such transitions. We present the behavior of Beernet++ for high level of churn and network partitioning, along with their interaction.
|
|
| 10. |
- Kalavri, Vasiliki, 1986-
(författare)
-
Performance Optimization Techniques and Tools for Data-Intensive Computation Platforms : An Overview of Performance Limitations in Big Data Systems and Proposed Optimizations
- 2014
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Big data processing has recently gained a lot of attention both from academia and industry. The term refers to tools, methods, techniques and frameworks built to collect, store, process and analyze massive amounts of data. Big data can be structured, unstructured or semi-structured. Data is generated from various different sources and can arrive in the system at various rates. In order to process these large amounts of heterogeneous data in an inexpensive and efficient way, massive parallelism is often used. The common architecture of a big data processing system consists of a shared-nothing cluster of commodity machines. However, even in such a highly parallel setting, processing is often very time-consuming. Applications may take up to hours or even days to produce useful results, making interactive analysis and debugging cumbersome.One of the main problems is that good performance requires both good data locality and good resource utilization. A characteristic of big data analytics is that the amount of data that is processed is typically large in comparison with the amount of computation done on it. In this case, processing can benefit from data locality, which can be achieved by moving the computation close the to data, rather than vice versa. Good utilization of resources means that the data processing is done with maximal parallelization. Both locality and resource utilization are aspects of the programming framework’s runtime system. Requiring the programmer to work explicitly with parallel process creation and process placement is not desirable. Thus, specifying good optimization that would relieve the programmer from low-level, error-prone instrumentation to achieve good performance is essential.The main goal of this thesis is to study, design and implement performance optimizations for big data frameworks. This work contributes methods and techniques to build tools for easy and efficient processing of very large data sets. It describes ways to make systems faster, by inventing ways to shorten job completion times. Another major goal is to facilitate the application development in distributed data-intensive computation platforms and make big-data analytics accessible to non-experts, so that users with limited programming experience can benefit from analyzing enormous datasets.The thesis provides results from a study of existing optimizations in MapReduce and Hadoop related systems. The study presents a comparison and classification of existing systems, based on their main contribution. It then summarizes the current state of the research field and identifies trends and open issues, while also providing our vision on future directions.Next, this thesis presents a set of performance optimization techniques and corresponding tools fordata-intensive computing platforms;PonIC, a project that ports the high-level dataflow framework Pig, on top of the data-parallel computing framework Stratosphere. The results of this work show that Pig can highly benefit from using Stratosphereas the backend system and gain performance, without any loss of expressiveness. The work also identifies the features of Pig that negatively impact execution time and presents a way of integrating Pig with different backends.HOP-S, a system that uses in-memory random sampling to return approximate, yet accurate query answers. It uses a simple, yet efficient random sampling technique implementation, which significantly improves the accuracy of online aggregation.An optimization that exploits computation redundancy in analysis programs and m2r2, a system that stores intermediate results and uses plan matching and rewriting in order to reuse results in future queries. Our prototype on top of the Pig framework demonstrates significantly reduced query response times.Finally, an optimization framework for iterative fixed points, which exploits asymmetry in large-scale graph analysis. The framework uses a mathematical model to explain several optimizations and to formally specify the conditions under which, optimized iterative algorithms are equivalent to the general solution.
|
|
| 11. |
|
|
| 12. |
- Peiro Sajjad, Hooman
(författare)
-
Towards Unifying Stream Processing over Central and Near-the-Edge Data Centers
- 2016
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- In this thesis, our goal is to enable and achieve effective and efficient real-time stream processing in a geo-distributed infrastructure, by combining the power of central data centers and micro data centers. Our research focus is to address the challenges of distributing the stream processing applications and placing them closer to data sources and sinks. We enable applications to run in a geo-distributed setting and provide solutions for the network-aware placement of distributed stream processing applications across geo-distributed infrastructures. First, we evaluate Apache Storm, a widely used open-source distributed stream processing system, in the community network Cloud, as an example of a geo-distributed infrastructure. Our evaluation exposes new requirements for stream processing systems to function in a geo-distributed infrastructure. Second, we propose a solution to facilitate the optimal placement of the stream processing components on geo-distributed infrastructures. We present a novel method for partitioning a geo-distributed infrastructure into a set of computing clusters, each called a micro data center. According to our results, we can increase the minimum available bandwidth in the network and likewise, reduce the average latency to less than 50%. Next, we propose a parallel and distributed graph partitioner, called HoVerCut, for fast partitioning of streaming graphs. Since a lot of data can be presented in the form of graph, graph partitioning can be used to assign the graph elements to different data centers to provide data locality for efficient processing. Last, we provide an approach, called SpanEdge that enables stream processing systems to work on a geo-distributed infrastructure. SpenEdge unifies stream processing over the central and near-the-edge data centers (micro data centers). As a proof of concept, we implement SpanEdge by extending Apache Storm that enables it to run across multiple data centers.
|
|
