| 1. |
- Konstantinov, A., et al.
(författare)
-
Data management services of NorduGrid
- 2005
-
Ingår i: CERN-2005-002. ; 2, s. 765-766
-
Konferensbidrag (refereegranskat)abstract
- In common grid installations, services responsible for storing big data chunks, replication of those data and indexing their availability are usually completely decoupled. And a task of synchronizing data is passed to either user-level tools or separate services (like spiders) which are subject to failure and usually cannot perform properly if one of underlying services fails too. The NorduGrid Smart Storage Element (SSE) was designed to try to overcome those problems by combining the most desirable features into one service. It uses HTTPS/G for secure data transfer, Web Services for control (through same HTTPS/G channel) and can provide information to indexing services used in middlewares based on the Globus Toolkit (TM). At the moment, those are the Replica Catalog and the Replica Location Service. The modular internal design of the SSE and the power of C++ object programming allows to add support for other indexing services in an easy way. There are plans to complement it with a Smart Indexing Service capable of resolving inconsistencies hence creating a robust distributed data storage system.
|
|
| 2. |
- Langgard Nielsen, J., et al.
(författare)
-
Experiences with Data Indexing services supported by the NorduGrid middleware
- 2005
-
Ingår i: CERN-2005-002. ; 2, s. 673-675
-
Konferensbidrag (refereegranskat)abstract
- The NorduGrid middleware, ARC, has integrated support for querying and registering to Data Indexing services such as the Globus Replica Catalog and Globus Replica Location Server. This support allows one to use these indexing services for for example brokering during jobsubmission, automatic registration of files and many other things. This integrated support is complemented by a set of command-line tools for registering to and querying these Data Indexing services. In this article we will describe experiences with these indexing services both from a daily work point of view and in production environments such as the Atlas Data- Challenges 1 and 2. We will describe the advantages of such Data Indexing services as well as their shortcomings. Finally we will present a proposal for an extended Smart Indexing Service which should deal with the shortcomings described. Such an indexing service is being designed at the moment.
|
|
| 3. |
- Pajchel, K., et al.
(författare)
-
Usage statistics and usage patterns on the NorduGrid: Analyzing the logging information collected on one of the largest production Grids of the world
- 2005
-
Ingår i: CERN-2005-002. ; 2, s. 711-714
-
Konferensbidrag (refereegranskat)abstract
- The Nordic Grid facility (NorduGrid [1]) came into operation during summer 2002 when the Scandinavian ATLAS HEP group started to use the Grid for the ATLAS Data Challenges (DC) and was thus the first Grid ever contributing to an ATLAS production. Since then, the Grid facility has been in continuous 24/7 operation. NorduGrid is being used by a growing set of active users from various scientific areas including physics, chemistry, biology and informatics. It has given major contributions to the ATLAS Data Challenge 1 [2] and the ongoing Data Challenge 2. · The increasing number of resources has made NorduGrid one of the largest production Grids in the world, continuously running on more than 30 sites more than 3000 CPUs. · The resources range from small test clusters at academic institutions to large farms at several supercomputer centers and the NorduGrid software runs on clusters with very different Linux distributions. This presentation gives a short overview of the design and implementation of the NorduGrid middleware, logging and monitoring facilities. It will be followed by a description of a typical job on NorduGrid and the information about its parameters which are monitored online and persistified in the logging service.
|
|
| 4. |
- Sturrock, R., et al.
(författare)
-
Performance of the NorduGrid ARC and the Dulcinea Executor in ATLAS Data Challenge 2
- 2005
-
Ingår i: CERN-2005-002. ; 2, s. 1095-1098
-
Konferensbidrag (refereegranskat)abstract
- This talk describes the various stages of ATLAS Data Challenge 2 (DC2) in what concerns usage of resources deployed via NorduGrid's Advanced Resource Connector (ARC). It also describes the integration of these resources with the ATLAS production system using the Dulcinea executor. ATLAS Data Challenge 2 (DC2), run in 2004, was designed to be a step forward in the distributed data processing. In particular, much coordination of task assignment to resources was planned to be delegated to Grid in its different flavours. An automatic production management system was designed, to direct the tasks to Grids and conventional resources. The Dulcinea executor is a part of this system that provides interface to the information system and resource brokering capabilities of the ARC middleware. The executor translates the job definitions recieved from the supervisor to the extended resource specification language (XRSL) used by the ARC middleware. It also takes advantage of the ARC middleware's built-in support for the Globus Replica Location Server (RLS) for file registration and lookup. NorduGrid's ARC has been deployed on many ATLAS-dedicated resources across the world in order to enable effective participation in ATLAS DC2. This was the first attempt to harness large amounts of strongly heterogeneous resources in various countries for a single collaborative exercise using Grid tools. This talk addresses various issues that arose during different stages of DC2 in this environment: preparation, such as ATLAS software installation; deployment of the middleware; and processing. The results and lessons are summarized as well.
|
|
