NLANR/DAST - Guide to Writing High-Performance Network Applications

As we talk with users about writing applications for high-performance research networks, it's clear that it may be helpful to explicitly think about the minimum set of requirements any prospective application author needs to keep in mind.

1. One end of the application needs to be homed at your home institution.

   It may sound self-evident, and we may very well be stating the obvious, but one end of the application needs to be at your home institution, and your home institution needs to have a connection to a high-performance research network. If you work with a community or industrial network partner this may be an issue for you, since in those cases being "closely affiliated" with a partner typically isn't "close enough" when it comes to the high-performance research network community.

2. The other end of the application needs to be at a site with live high performance connectivity.

   Another seemingly self-evident requirement which we nonetheless must mention: the other end of the application must be at a site that has high performance connectivity with a high-performance research network. (i.e., sites connected via I2/Abilene, the vBNS, Canarie, ESNet, NREN, DREN, etc.).

   Why is this an issue? Well, you may run into sites that are members but are still working on getting physical high-performance network connectivity deployed; obviously they won't work, at least not yet. You will also run into other sites on foreign high-performance networks which may not peer with your high-performance research network yet. Again, those sites won't work, at least not yet. For information on sites that are connected to or peer with various high-performance research networks, contact NLANR/DAST by email at dast@nlanr.net. It can be frustrating to have data from a telescope in Hawaii, for example, that you'd like to retrieve via high-performance research network, only to find that the Hawaiian instrument doesn't have the appropriate connectivity-- but please remember that all of the high-performance research networks are evolving and developing, and institutional eligibility for connection high-performance networks isn't something we control.

3. Your application should (ideally) have characteristics which take advantage of the high-performance research network's special capabilities.

   An application engineered for a high-performance research network should (ideally) have characteristics which take advantage of the network's special capabilities, i.e., your application should require high bandwidth, low latency, low jitter, or be otherwise particularly well suited to big pipes. Put another way, a good high-performance research network application is usually an application that doesn't work well over the commodity/commercial Internet.

4. Your application needs to be able to differentiate between high performance Internet connections and commodity Internet connections.

   Implicitly or explicitly, you and your application need to be able to differentiate between high-performance network-connected partners and commodity Internet network-connected partners. Consider, for example, a web server that can deliver high bit rate video on demand. A high-performance research network connection is designed for and has the capacity available to service those high bit rate streams. But what if a person comes in via a commodity Internet connection and requests those high bit rate streams? Commodity network capacity can't satisfy that demand, nor will a user trying to view that video at the other end of a commodity Internet connection get the high bit rate multimedia experience you probably want them to have. Thus, your application needs to be aware of where its peers are coming from, either because it explicitly selects the remote peers it works with itself (e.g., a web crawling spider that crawls only high-performance research network-attached web sites), or because it controls which sites can access it via a semiautomatic mechanism (such as a generated .htaccess file which "knows" what network prefixes are via the high-performance research network), or via a manual mechanism (such as a person-to-person agreement to exchange data, reached by you with a remote collaborator at another high-performance research network site).

5. Applications should be ongoing (or time critical).

   Applications particularly well suited to high-performance networking connections should be ongoing (rather than being one-time events), or they should be time critical (i.e., you need access to the data ASAP).

   Put another way, if you have a one-time, non-time critical need to move data between two sites, it's hard to beat the throughput/cost efficiency ratio of a box full of DLT tapes sent overnight via Federal Express. On the other hand, if you're moving data every day, or the data needs to be available virtually immediately, then think "let's try delivering this data via big pipes."

6. Applications can't be for commercial purposes, nor can they involve classified data.

   Most universities' Acceptable Use Policy prohibits commercial use of university resources, which means that commercial projects are unacceptable. Similarly, high-performance research networks aren't certified for use for classified research purposes, so don't plan to use it to move classified data, even though they may route to places well known for doing classified research (such as Lawrence Livermore Lab, Los Alamos, Sandia, Kirtland Air Force Base, etc.).