Computational Grids: A New Infrastructure for High-Performance Computing - Prof. Alan L. S, Study notes of Computer Science

Download Computational Grids: A New Infrastructure for High-Performance Computing - Prof. Alan L. S and more Study notes Computer Science in PDF only on Docsity!

Computational Grids

Chapter 2 of The Grid: Blueprint for a New Computing Infrastructure, by Ian Foster and Carl Kesselman Presented by Adam Bazinet

Six Questions

• Why do we need computational grids?
• What types of applications will grids be used for?
• Who will use grids?
• How will grids be used?
• What is involved in building a grid?
• What problems must be solved to make grids commonplace?

Increasing Delivered Computation - How?

• End systems will continue to increase their computational capacity
• Easy demand-driven access to computational resources will improve
• Better utilization of idle cycles, as computers sit idle (but on) much of the time
• Better sharing of computational results will result in more efficient and productive collaborations
• Such new technology may lead to innovative new problem-solving techniques

Definition of Computational Grids

• By analogy with the electrical grid circa 1910
• A computational grid: a hardware and software infrastructure that provides dependable , consistent , pervasive , and inexpensive access to high-end computational capabilities.
• It is this combination of features that will have a transformative effect on how computation is performed.

Four Grid Communities

• Government
• A Health Maintenance Organization
• Materials Science Collaboratory
• Computational Market Economy

People Using Grids

• Grid Developers
• Tool Developers
• Application Developers
• End Users
• System Administrators

End Systems

• Comprises computers, storage systems, sensors, and other devices
• Small in scale, highly homogeneous, highly integrated
• Simplest and most intensively studied environment in which to provide basic services
• Do not necessarily integrate easily into larger clusters, intranets, and internets

Clusters

• A network of workstations connected by a high-speed LAN
• Typically homogeneous
• Principal complicating factors:
  1. increased physical scale
  2. reduced integration
• Software architectures for clusters may converge with those for end systems, as end-system architectures address issues of network operation and scale.

Internets

• A grid of internetworked systems that span multiple organizations.
• Principal complicating factors:
  1. lack of centralized control
  2. geographical distribution
  3. international issues
• Security is of primary concern
• Policy issues between organizations become important

Research Challenges

• Many and varied:
  1. the nature of applications may change
  2. new programming models and tools
  3. system architecture
  4. algorithms and problem-solving methods
  5. resource management
  6. security
  7. instrumentation and performance analysis
  8. end systems
  9. network protocols and infrastructure

The Lattice Project

• The Lattice Project is primarily aimed at effectively sharing computational resources between departments and institutions, starting with those in the University System of Maryland.
• The Grid is focused on computation , and we have not yet made efforts to enable large-scale data access, storage, or replication.
• The Grid has transitioned from a research project which started in 2003 to a production system that has been used by a number of researchers, racking up many hundreds of CPU years in the process.

Grid Software

• The backbone of the Grid system is Globus Toolkit software, which provides mechanisms for job submission, file transfer, and authentication and authorization of entities on the Grid.
• The most novel feature of The Lattice Project is our Globus-BOINC interface, which enables Grid jobs to flow into a BOINC pool. The Lattice BOINC Project (http://boinc.umiacs.umd.edu) is our active BOINC project for this purpose. Anyone can participate.
• We also work with scheduling software that controls local resources. Our resource base is currently composed of Condor pools and clusters running variants of PBS.

Grid Services

• Applications are Grid-enabled and made into a Grid service. Such trusted applications are then made available to run on Grid resources.
• To date, we have Grid-enabled approximately 25 applications, mostly life science applications, with notable exceptions. Only a subset of these have been run a significant amount.
• We have developed a software stack that allows us to Grid-enable applications quickly and easily. We call this software GSBL (Grid Services Base Library) and GSG (Grid Services Generator).

User Interfaces

• Our primary Grid interface is command-line based. Grid users log on to a specific machine, upload their input data, and then submit and monitor jobs using our tools.
• We also provide a Web interface for monitoring job status, which is located on The Lattice Project intranet.
• Future work might see job submission and other operations take place via a Web interface or portal of sorts. We may also make the command-line interface more widely available.