BIMA

October 2002

Steve Engelhardt

The Berkeley-Illinois-Maryland-Array (BIMA) is a consortium which operates a millimeter-wave radio interferometer at Hat Creek, California. This "radio telescope" generates a constant stream of data, which must then be processed to understand its meaning. NLANR/DAST is collaborating with the BIMA team, primarily Ray Plante, to redesign and reimplement their job processing and queueing system to be component-based and Globus-enabled, with the additional goal of submitting useful components back to the Globus community.

Introduction

NLANR/DAST has been collaborating with the BIMA team over the past year in redesigning and reimplementing their job processing and queueing system. We are developing an extensible SOAP-based queue management service in Java which will be used in the BIMA image processing pipeline to replace the current infrastructure. This queue manager allows the programmatic submission of jobs using a well-defined interface described in WSDL, which will be made public for review and possible future standardization. The new queue manager is based off previous work, but improves upon it in a number of important areas.

In addition to the queue manager, we are also building a simple web front end so that end users can easily monitor and submit jobs. Since the queue manager is written using SOAP, writing multiple front ends, even in different programming languages, is very easy.

Once this project has reached a sufficient level of maturity, we plan on submitting it to the Globus community at large, as it may be very useful as a job queueing system to a number of different projects.

Furthermore, if it becomes technically feasible and reasonable in the future, the queue manager may be integrated with other queueing systems such as Condor, simply by changing how the SOAP interface communicates with the actual queue implementation.

Features

SOAP based
Interface standardized in WSDL, which may be reviewed and improved by other teams
Ability to submit jobs to multiple different destinations, including Globus and using rsh
Uses XML heavily: the queues and jobs are all represented by human-readable XML documents
Ability to edit how jobs are processed by editing an XML document
Supports multiple jobs running simultaneously using multiple queues; adding a new queue is extremely easy

Current Status

NLANR/DAST is currently trying to round out a number of features so that the BIMA team may test and deploy the queue manager, at which time the BIMA team will take over development.

Contact DAST Blank Space Last reviewed: December 31, 1969
NLANR || Applications Support || Engineering Support || Measurement and Network Analysis