| 1. |
- Aguilar, Xavier, et al.
(författare)
-
MPI Trace Compression Using Event Flow Graphs
- 2014
-
Konferensbidrag (refereegranskat)abstract
- Understanding how parallel applications behave is crucial for using high-performance computing (HPC) resources efficiently. However, the task of performance analysis is becoming increasingly difficult due to the growing complexity of scientific codes and the size of machines. Even though many tools have been developed over the past years to help in this task, current approaches either only offer an overview of the application discarding temporal information, or they generate huge trace files that are often difficult to handle.In this paper we propose the use of event flow graphs for monitoring MPI applications, a new and different approach that balances the low overhead of profiling tools with the abundance of information available from tracers. Event flow graphs are captured with very low overhead, require orders of magnitude less storage than standard trace files, and can still recover the full sequence of events in the application. We test this new approach with the NERSC-8/Trinity Benchmark suite and achieve compression ratios up to 119x.
|
|
| 2. |
- Aguilar, Xavier, et al.
(författare)
-
Online Performance Data Introspection with IPM
- 2014
-
Ingår i: Proceedings of the 15th IEEE International Conference on High Performance Computing and Communications (HPCC 2013). - : IEEE Computer Society. - 9780769550886 ; , s. 728-734
-
Konferensbidrag (refereegranskat)abstract
- Exascale systems will be heterogeneous architectures with multiple levels of concurrency and energy constraints. In such a complex scenario, performance monitoring and runtime systems play a major role to obtain good application performance and scalability. Furthermore, online access to performance data becomes a necessity to decide how to schedule resources and orchestrate computational elements: processes, threads, tasks, etc. We present the Performance Introspection API, an extension of the IPM tool that provides online runtime access to performance data from an application while it runs. We describe its design and implementation and show its overhead on several test benchmarks. We also present a real test case using the Performance Introspection API in conjunction with processor frequency scaling to reduce power consumption.
|
|
| 3. |
- Aguilar, Xavier, et al.
(författare)
-
Scalability analysis of Dalton, a molecular structure program
- 2013
-
Ingår i: Future Generation Computer Systems. - : Elsevier BV. - 0167-739X .- 1872-7115. ; 29:8, s. 2197-2204
-
Tidskriftsartikel (refereegranskat)abstract
- Dalton is a molecular electronic structure program featuring common methods of computational chemistry that are based on pure quantum mechanics (QM) as well as hybrid quantum mechanics/molecular mechanics (QM/MM). It is specialized and has a leading position in calculation of molecular properties with a large world-wide user community (over 2000 licenses issued). In this paper, we present a performance characterization and optimization of Dalton. We also propose a solution to avoid the master/worker design of Dalton to become a performance bottleneck for larger process numbers. With these improvements we obtain speedups of 4x, increasing the parallel efficiency of the code and being able to run in it in a much bigger number of cores.
|
|
| 4. |
- Aguilar, Xavier, et al.
(författare)
-
Scaling Dalton, a molecular electronic structure program
- 2011
-
Ingår i: Seventh International Conference on e-Science, e-Science 2011, 5-8 December 2011, Stockholm, Sweden. - : IEEE conference proceedings. - 9781457721632 ; , s. 256-262
-
Konferensbidrag (refereegranskat)abstract
- Dalton is a molecular electronic structure program featuring common methods of computational chemistry that are based on pure quantum mechanics (QM) as well as hybrid quantum mechanics/molecular mechanics (QM/MM). It is specialized and has a leading position in calculation of molecular properties with a large world-wide user community (over 2000 licenses issued). In this paper, we present a characterization and performance optimization of Dalton that increases the scalability and parallel efficiency of the application. We also propose asolution that helps to avoid the master/worker design of Daltonto become a performance bottleneck for larger process numbers and increase the parallel efficiency.
|
|
| 5. |
- Ahmed, Laeeq, et al.
(författare)
-
Using iterative MapReduce for parallel virtual screening
- 2013
-
Ingår i: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom). - : IEEE Computer Society. - 9780769550954 ; , s. 27-32
-
Konferensbidrag (refereegranskat)abstract
- Virtual Screening is a technique in chemo informatics used for Drug discovery by searching large libraries of molecule structures. Virtual Screening often uses SVM, a supervised machine learning technique used for regression and classification analysis. Virtual screening using SVM not only involves huge datasets, but it is also compute expensive with a complexity that can grow at least up to O(n2). SVM based applications most commonly use MPI, which becomes complex and impractical with large datasets. As an alternative to MPI, MapReduce, and its different implementations, have been successfully used on commodity clusters for analysis of data for problems with very large datasets. Due to the large libraries of molecule structures in virtual screening, it becomes a good candidate for MapReduce. In this paper we present a MapReduce implementation of SVM based virtual screening, using Spark, an iterative MapReduce programming model. We show that our implementation has a good scaling behaviour and opens up the possibility of using huge public cloud infrastructures efficiently for virtual screening.
|
|
| 6. |
- Apostolov, Rossen, et al.
(författare)
-
Scalable Software Services for Life Science
- 2011
-
Ingår i: Proceedings of 9th HealthGrid conference.
-
Konferensbidrag (refereegranskat)abstract
- Life Science is developing into one of the largest e- Infrastructure users in Europe, in part due to the ever-growing amount of biological data. Modern drug design typically includes both sequence bioinformatics, in silico virtual screening, and free energy calculations, e.g. of drug binding. This development will accelerate tremendously, and puts high demands on simulation software and support services. e-Infrastructure projects such as PRACE/DEISA have made important advances on hardware and scalability, but have largely been focused on theoretical scalability for large systems, while typical life science applications rather concern small-to-medium size molecules. Here, we propose to address this with by implementing new techniques for efficient small-system parallelization combined with throughput and ensemble computing to enable the life science community to exploit the largest next-generation e-Infrastructures. We will also build a new cross-disciplinary Competence Network for all of life science, to position Europe as the world-leading community for development and maintenance of this software e-Infrastructure. Specifically, we will (1) develop new hierarchical parallelization approaches explicitly based on ensemble and high-throughput computing for new multi-core and streaming/GPU architectures, and establish open software standards for data storage and exchange, (2) implement, document, and maintain such techniques in pilot European open-source codes such as the widely used GROMACS & DALTON, a new application for ensemble simulation (DISCRETE), and large-scale bioinformatics protein annotation, (3) create a Competence Centre for scalable life science software to strengthen Europe as a major software provider and to enable the community to exploit e-Infrastructures to their full extent. This Competence Network will provide training and support infrastructure, and establish a long-term framework for maintenance and optimization of life science codes.
|
|
| 7. |
- Espling, Daniel, 1983-
(författare)
-
Metadata Management in Multi-Grids and Multi-Clouds
- 2011
-
Licentiatavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Grid computing and cloud computing are two related paradigms used to access and use vast amounts of computational resources. The resources are often owned and managed by a third party, relieving the users from the costs and burdens of acquiring and managing a considerably large infrastructure themselves. Commonly, the resources are either contributed by different stakeholders participating in shared projects (grids), or owned and managed by a single entity and made available to its users with charging based on actual resource consumption (clouds). Individual grid or cloud sites can form collaborations with other sites, giving each site access to more resources that can be used to execute tasks submitted by users. There are several different models of collaborations between sites, each suitable for different scenarios and each posing additional requirements on the underlying technologies.Metadata concerning the status and resource consumption of tasks are created during the execution of the task on the infrastructure. This metadata is used as the primary input in many core management processes, e.g., as a base for accounting and billing, as input when prioritizing and placing incoming task, and as a base for managing the amount of resources allocated to different tasks.Focusing on management and utilization of metadata, this thesis contributes to a better understanding of the requirements and challenges imposed by different collaboration models in both grids and clouds. The underlying design criteria and resulting architectures of several software systems are presented in detail. Each system addresses different challenges imposed by cross-site grid and cloud architectures:The LUTSfed approach provides a lean and optional mechanism for filtering and management of usage data between grid or cloud sites.An accounting and billing system natively designed to support cross-site clouds demonstrates usage data management despite unknown placement and dynamic task resource allocation.The FSGrid system enables fairshare job prioritization across different grid sites, mitigating the problems of heterogeneous scheduling software and local management policies.The results and experiences from these systems are both theoretical and practical, as full scale implementations of each system has been developed and analyzed as a part of this work. Early theoretical work on structure-based service management forms a foundation for future work on structured-aware service placement in cross- site clouds.
|
|
| 8. |
- Gholami, Ali, et al.
(författare)
-
Privacy Threat Modeling for Emerging BiobankClouds
- 2014
-
Ingår i: Procedia Computer Science. - : Elsevier. ; 37, s. 489-496
-
Konferensbidrag (refereegranskat)abstract
- There is an increased amount of data produced by next generation sequencing (NGS) machines which demand scalable storage and analysis of genomic data. In order to cope with this huge amount of information, many biobanks are interested in cloud computing capabilities such as on-demand elasticity of computing power and storage capacity. There are several security and privacy requirements mandated by personal data protection legislation which hinder biobanks from migrating big data generated by the NGS machines. This paper describes the privacy requirements of platform-as-service BiobankClouds according to the European Data Protection Directive (DPD). It identifies several key privacy threats which leave BiobankClouds vulnerable to an attack. This study benefits health-care application designers in the requirement elicitation cycle when building privacy-preserving BiobankCloud platforms.
|
|
| 9. |
- Gholami, Ali, et al.
(författare)
-
ScaBIA : Scalable brain image analysis in the cloud
- 2013
-
Ingår i: CLOSER 2013 - Proceedings of the 3rd International Conference on Cloud Computing and Services Science. - 9789898565525 ; , s. 329-336
-
Konferensbidrag (refereegranskat)abstract
- The use of cloud computing as a new paradigm has become a reality. Cloud computing leverages the use of on-demand CPU power and storage resources while eliminating the cost of commodity hardware ownership. Cloud computing is now gaining popularity among many different organizations and commercial sectors. In this paper, we present the scalable brain image analysis (ScaBIA) architecture, a new model to run statistical parametric analysis (SPM) jobs using cloud computing. SPM is one of the most popular toolkits in neuroscience for running compute-intensive brain image analysis tasks. However, issues such as sharing raw data and results, as well as scalability and performance are major bottlenecks in the "single PC"-execution model. In this work, we describe a prototype using the generic worker (GW), an e-Science as a service middleware, on top of Microsoft Azure to run and manage the SPM tasks. The functional prototype shows that ScaBIA provides a scalable framework for multi-job submission and enables users to share data securely using storage access keys across different organizations.
|
|
| 10. |
- Jayawardena, Mahen, 1977-
(författare)
-
An e-Science Approach to Genetic Analysis of Quantitative Traits
- 2010
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- Many important traits in plants, animals and humans are quantitative, and most such traits are generally believed to be affected by multiple genetic loci. Standard computational tools for mapping of quantitative traits (i.e. for finding Quantitative Trait Loci, QTL, in the genome) use linear regression models for relating the observed phenotypes to the genetic composition of individuals in an experimental population. Using these tools to simultaneously search for multiple QTL is computationally demanding. The main reason for this is the complex optimization landscape for the multidimensional global optimization problems that must be solved. This thesis describes parallel algorithms, implementations and tools for simultaneous mapping of several QTL. These new computational tools enable genetic analysis exploiting new classes of multidimensional statistical models, potentially resulting in interesting results in genetics. We first describe how the standard, brute-force algorithm for global optimization in QTL analysis is parallelized and implemented on a grid system. Then, we also present a parallelized version of the more elaborate global optimization algorithm DIRECT and show how this can be efficiently deployed and used on grid systems and other loosely-coupled architectures. The parallel DIRECT scheme is further developed to exploit both coarse-grained parallelism in grid systems or clusters as well as fine-grained, tightly-coupled parallelism in multi-core nodes. The results show that excellent speedup and performance can be archived on grid systems and clusters, even when using a tightly-coupled algorithm such as DIRECT. Finally, we provide two distinctly different front-ends for our code. One is a grid portal providing a graphical front-end suitable for novice users and standard forms of QTL analysis. The other is a prototype of an R-based grid-enabled problem solving environment. Both of these front-ends can, after some further refinement, be utilized by geneticists for performing multidimensional genetic analysis of quantitative traits on a regular basis.
|
|
