NLANR/DAST Projects

Completed Projects passed on to other Projects
Advanced Applications Database | GPDK | Myproxy | Web100

Completed Projects
Alliance Portal | Cactus | DSRT | GARA | libdast library | Mapping the Universe | Netart | Terabyte Challenge | vBNS Visualization | Widearea High-Speed Network Transfer

Projects currently under development

NextINet

NextINet is a project dedicated to serving a community of scientists, educators, engineers, private citizens, businesses, non-profits and government organizations focused on building Cyberinfrastructure.

NextINet tools include the website (a so-called "curated repository" of useful information resources updated daily), a "current awareness" service by which information is selectively targeted to those with interest, an electronic journal, a series of regularly staged events and discussions about where technologies are taking us (and how we can best develop and harness them), and a set of communication services to encourage cooperation across the organizational and disciplinary lines that may slow and hinder work on development of a global advanced communications and computing network.

Network Performance Advisor

The NLANR Network Performance Advisor is a single application which integrates the measuring, analyzing, and displaying of network performance statistics. The Advisor enables the writing of the analysis and display portions by providing a platform to allow easy integration of any number of network diagnostic tools, combined with the ability to uniformly query the results of these tools. It will ship with a network performance analysis tool and a network debugging utility aimed for network engineers, and the knowledge about a number of diagnostic tools, likely including ping, ifconfig, iperf, AMP, Surveyor, and the Web100 suite of tools. Due to the Advisor's design, new analysis and display tools will be easy to write, and new network diagnostic tools will be straightforward to integrate. The Advisor distinguishes itself by the ability to display and analyze an extremely broad set of network statistics, due to its ability to integrate any network diagnostic tool.

Multicast Beacon

Multicast is a way of distributing IP packets to a set of machines that have expressed an interest in receiving them. It is a scalable, one-to-many distribution method suitable for video conferencing and data sharing over the network. The Multicast Beacon is active measurement software which monitors the performance of a given multicast group. Most often seen teamed up with the Access Grid, the Multicast Beacon provides measurements about the current multicast traffic on the network.

Background on the Access Grid:
The Access Grid is an effort to build up a grid of distributed human interactions across the network. Examples of such interaction include video conference, data sharing, group discussion, etc. Led by ANL, the Access Grid involves many universities and organizations across the US. The goal is to prototype a number of grid nodes and uses these nodes to conduct meetings and lectures for practical purposes. The Access Grid uses the multicasting infrastructure enabled by today's high performance networks. As part of the grid, a Beacon client is running at each site to monitor current multicast conditions.

BIMA

The Berkeley-Illinois-Maryland-Array (BIMA) is a consortium which operates a millimeter-wave radio interferometer at Hat Creek, California. This "radio telescope" generates a constant stream of data, which must then be processed to understand its meaning. NLANR/DAST is collaborating with the BIMA team, primarily Ray Plante, to redesign and reimplement their job processing and queueing system to be component-based and Globus-enabled, with the additional goal of submitting useful components back to the Globus community.

Data Mover

Data Mover is an external module for the Internet Backbone Plane which moves data between server depots. The Internet Backbone Plane (IBP) is middleware for controlling remote storage. Currently, TCP only allows an application to specify the end points of a communication stream. The IBP allows an application to specify the intermediate data staging locations for communication between processes. The Data Mover module implements the IBP_mcopy() service for the transfer of data. The Data Mover supports point to multipoint copy requests from a client, and hides data processing and performance issues from the user. The DAST group is attempting to integrate Web100 with the Data Mover module to increase data transfer speeds.

Enhancing FTP Performance - Working with GridFTP centered at ANL.

FTP is the single most common TCP bulk data transfer application in use today, yet its performance is notoriously poor. NLANR is investigating how to improve that performance, both with current implementations and by proposing new RFC's to fix deficiencies in the FTP protocol. A survey of existing implementations has been done, and NLANR is working with both the SURA high-performance file transfer working group and the Web100 group to develop a standard and reference implementations.

Much of the problem stems from Relying on the static system socket buffer size for TCP data transfers, which seriously limits the efficient use of high-speed network bandwidth and system memory. The performance of FTP is directly affected by TCP window size. NLANR has developed an Automatic Buffer Tuning method from the application level as an attempt to solve this problem. The Automatic Buffer Tuning-enabled FTP client demonstrates that dynamic adjustment of TCP socket buffer size at TCP connection setup time causes enhanced FTP application performance, improved utilization of network bandwidth, flexible reservation of system memory, and requires no kernel modification.

Grid Forum: Performance Working Group and Application Requirements Working Group

The Grid Forum is an informal consortium of institutions and individuals working on wide area computing and computational grids. NLANR is working with the Performance Working Group to create a repository and taxonomy of performance tools and define a schema for performance monitoring data. The Applications Requirements Working Group promotes the development of Grid software and infrastructure which promotes application performance, portability and ease-of-use.

iGrid 2002

Writeup pending.

Iperf - Network Performance Testing

While tools to measure network performance, such as ttcp, exist, most are very old and have confusing options. Iperf was developed as a modern alternative for measuring TCP and UDP bandwidth performance. The primary goal of Iperf is to help in tuning TCP connections over a particular path. The most fundamental tuning issue for TCP is the TCP window size, which controls how much data can be in the network at any one point. If it is too small, the sender will be idle at times and get poor performance.

Iperf is a tool to measure maximum TCP bandwidth, allowing the tuning of various parameters and UDP characteristics. Iperf reports bandwidth, delay jitter, datagram loss.

Jperf - Graphical Front-End to Iperf

Jperf is a Java-based graphical front-end to Iperf. It allows for easier use of Iperf, as well as the ability to see all Iperf options visually. Jperf detects the version of Iperf it's running in front of, and only allows those options that are actually available.

Netlog and Viznet: Application Network Performance Monitoring

One of the first steps of tuning network applications is to instrument them to find out what performance is being obtained and to see what the general traffic characteristics are. Netlog provides easy, rapid instrumentation and visualization of performance data.

Netlog is a C library that can be linked into an existing network application to provide instrumentation on network performance.

Viznet is a Java application designed to work with the Netlog library to visualize network bandwidth performance over time.

Completed Projects passed on to other Projects

Advanced Applications Database

The Advanced Applications Database is the core of the NLANR Clearinghouse project that was conceived as an online resource for people using high performance networks who need information about projects, applications, and other related resources. Currently there are no other data collections useful for locating applications, advanced resources, or people who are using/developing them. At present the collection includes more than 2,000 applications, projects, and resources from over 900 institutions in 30 countries worldwide.

Initially the Clearinghouse will provide access to information collection. Later it will serve as an online collaboration space -- a Collaboratory -- for various communities that can be found within and among those using the NSF vBNS and UCAID Abilene networks.

At first, the Advanced Applications database included basic information about the projects, participants, and institutions. Later the database will grow to include more detailed information about technical requirements of the applications and results of application research. Once the NSF HPC information is in place, we will work with staff of the Internet 2/Abilene community to see that their data are incorporated.

Cactus - Completed and spun off into its own Cactus project.

Cactus is a numerical relativity simulation capable of solving Einstein's field equations in three dimensions. A wide variety of astrophysical phenomenon including colliding neutron stars, black holes and the formation of singularities can be studied using Cactus. Visualization techniques including iso-surfacing can be performed in parallel with the simulation providing immediate feedback to the ongoing simulation.

By coupling the Cactus code with the advanced capabilities of the Globus toolkit, we used MPICH-G to transform Cactus into a powerful distributed application capable of harnessing multiple compute resources without modifying the code. MPICH-G extends the MPI standard for message passing by allowing standard MPI codes to run efficiently on heterogenous resources. Globus provides a transparent mechanism for submitting jobs to remote compute resources by interfacing to most resource schedulers including LSF, NQE and LoadLeveler.

GPDK - Completed and passed off to DOE Science Grid Grid Portal Development Kit page

The Grid Portal Development Kit provides both a portal development environment for the creation of new portals as well as a collection of Grid service beans used to accomplish basic operations such as job submission, file transfer and querying of information services.

 

Myproxy - Completed and passed off to Grid-in-a-Box

Myproxy provides a secure method for portal users to access resources using a limited proxy using the Globus Security Infrastructure.

Web100 - Completed and spun off into its own independently funded project, www.web100.org.

The Web100 concept addresses the problem of researchers being lucky if they could see even 10-Mbps rates out of 100-Mbps wide-area networks. NLANR addressed the networking issues in a comprehensive manner, fixing the TCP/IP implementation and several key network applications such as FTP in on open source operating system, and providing a widely available reference implementation for high-performance networking. (See also the Enhancing FTP Performance project.)

Alliance Portal: Network Performance

As a part of the Alliance Portal at SC'99, NLANR worked on having a web accessible summary of network performance. The Java Network Weather Service was used as a tool to measure bandwidth and delay. Also used was performance data published in the Globus Information Services LDAP database.

DSRT Monitor

DSRT was a project to build a Remote Monitor System for distributed applications using the Dynamic Soft Real-Time (DSRT) scheduler and a centralized directory server as a resource directory. The idea was to achieve guaranteed CPU allocations to real-time applications while sharing processing bandwidth with time-sharing applications.

GARA GUI

The Globus Architecture for Reservations and Allocations supports end-to-end QoS guarantees for bandwidth and CPU utilization. The GARA framework allows users to co-allocate network and CPU resources through a timeslot table used for managing requests for reservations. The layered structure of GARA supports reservations through a lower-level Local Resource Allocation Manager (LRAM). Currently, three LRAM's exist to interface to the RSVP protocol for bandwidth reservation, the DSRT for cpu reservation/task scheduling and the SMP LRAM to support process reservation on a shared memory multiprocessor.

libdast library

In developing Iperf and other network programs, NLANR/DAST developed a number of C++ classes, C utilities, and autoconf macros that are sufficiently useful and generic that we wanted to release them as a separate library to help other developers avoid having to re-invent the wheel for their own projects.

The C++ classes are intended to be similar in design to the corresponding Java classes within Iperf. They are packaged here, and may be included in other projects subject to the standard UI license.

Mapping the Universe: Cosmology

"Mapping the Universe" involved multiple Alliance sites, and resulted in cosmological animations which were shown on a PBS HDTV NOVA show in the year 2000.

Brent Tully, astronomer, and Tom Lucas, writer/producer, were funded by PBS NOVA and the informal science division of the National Science Foundation to execute one of the first science educational HDTV productions for broadcast. NCSA's role was to run AMR simulations and 'package' computational simulations for broadcast HDTV quality generating about 10 minutes of imagery.

Alias/Wavefront Maya software was used to develop an animation of an exploding supernova, which required rigorous testing of the built-in software dynamics as well as collaboration with astronomers in order to achieve the most realistic product. In addition to animation, NLANR also assisted the project in its distributed computing needs.

Netart

Netart is a project led by Victoria Vesna at UCLA. It provided interconnected agents for the user community. This package, called netnode, is the network messaging system used to support the underlying networking functions.

Terabyte Challenge and Netstorm

The Terabyte Challenge is a project led by Bob Grossman and Emory Creel at the National Center for Data Mining (NCDM) at the University of Illinois at Chicago. The data mining infrastructure creates a flexible way to do data transfer and analysis across multiple domains. A number of applications from diverse disciplines were identified to take advantage of this project.

The Terabyte Challenge team coordinated distributed data nodes at various institutions to do data mining across high-speed networks. NLANR set up one node at NCSA Access center to participate in this effort.

NLANR developed a distributed network monitor to generate data based on network latency in a collaboration with Emory Creel at NCDM. A set of monitors were run with a DSTP (Data Space Transfer Protocol) server at each site to generate data in real-time, and served data to the clients by the DSTP protocol. A prototype Java program was written to collect basic network latency data. myping is a program to measure round-trip-time, mytrace is a program to measure routing information, and myudp is a program to measure one-way delay.

vBNS Visualization

NLANR created a 3D, interactive, immersive visualization of the vBNS and Abilene networks. Our first step was to develop a visual traceroute, in Java, which plotted a network path against a geographic map. An initial version was shown at the Boston Chautauqua, and at Distributed Computing Workshops. This is a sample snapshot of the route between NCSA in Illinois and SDSC in California. This is an MPEG of the vBNS growth.

Wide Area High-Speed Network Transfer

Gigabit Ethernet remains a very attractive option for high-speed connection to the desktop workstation. With the improvement of CPU and OS, a commodity PC can achieve much higher throughput than before. This potentially benefits bandwidth critical applications, such as high quality network video, large file transfer, and distributed simulation.

NLANR conducted tests between machines with Gigabit Ethernet connections, optimizing throughput locally, and then moved on to test on wide-area networks.