All Aboard! the National LambdaRail High-performance Computing Now Available to Researchers
Photo by Dan Addison
Big scientific journeys at the University of Virginia, once limited by computing capacity, now have a ticket to ride on the National LambdaRail, a nationwide high-performance computing network designed specifically to support research.
U.Va. is now fully connected to NLR, providing University researchers with access from their desktops to some of the most important research sites and databases in the world, including national labs such as Oak Ridge, supercomputing centers in California and Illinois, particle accelerators, major observatories and environmental monitoring sites.
The University is inviting researchers in all disciplines to consider tapping into the world’s data resources via this new connection to the super high-speed nationwide network.
NLR is the first high-performance national research network to use transcontinental optical circuits (light waves, or lambdas) for transporting massive data over fiber-optic cable specifically for scientific research. The $100 million network took three years to develop, and as a member, U.Va. has full access.
“It’s like having all the computers in the world right in the office with you,” said John Hawley, a professor of astronomy who is preparing to use the connection for access to supercomputers in California and Illinois for his studies on the behavior of gas orbiting black holes in deep space. “By having nearly instant access to the world’s data we are able to do more creative science.”
Major research is ever-increasingly complex, collaborative, cross-disciplinary and multi-institutional. Connection to NLR is considered essential to the University’s success at conducting — and receiving funding for — major research projects in the coming years.
“Our use of NLR will be transformational for doing big science projects at the University,” said Dr. R. Ariel Gomez, vice president for research and graduate studies. “This will allow greater collaboration with colleagues at other institutions and will help us greatly to apply for big scientific research grants. It will enable us to do large and highly creative experiments with the power to change the world for the better.”
The Commonwealth of Virginia has provided $2.4 million to help connect U.Va. and five other institutions — Virginia Tech, Virginia Commonwealth University, the College of William & Mary, George Mason University and Old Dominion University — to NLR in an effort to increase academic research and development in the state by $1 billion in the next four years. Major research is essential to the continuing economic development of the commonwealth in high-tech fields of industry.
According to Robert Reynolds, vice president and chief information officer, U.Va. will contribute $100,000 per year for five years to the cost that was needed to build an NLR connection in Virginia, and there are additional costs to maintain and operate the U.Va. connection.
But the benefits far exceed the costs. Reynolds expects research funding to jump dramatically in the coming years because of U.Va.’s NLR connection.
“We would be noncompetitive without this facility,” he said. “There is a significant economic multiplier for U.Va. and the commonwealth in our ability to compete for and win the large research grants from the Departments of Energy, the National Science Foundation, the National Institutes of Health and the Department of Defense.”
U.Va., along with its partners, are members of the MidAtlantic Terascale Partnership, a group of Virginia, Maryland and Washington, D.C. institutions linked to NLR, along with NASA and the Thomas Jefferson National Accelerator Facility.
Connection to NLR allows vast amounts of information to be moved faster and at far less cost than through the Internet — on the order of 700 times less expensive per unit of data than the cost of the same unit of information sent over traditional high-speed links.
And the data will travel faster. As an example, “broadband” home modems transmit data at approximately one megabit per second. At that rate, it would take close to 28 minutes to transmit a 200 megabyte file that would roar across the LambdaRail — with its multi-gigabit per second capacity — in about one-sixth of a second. NLR has the capability to transmit in a single connection more information than 160,000 dial-up Internet connections or 7,000 broadband connections.
For scientists who are accessing enormous databases at other locations, the time savings is extraordinary. Data that previously might have taken days just to download can now be accomplished in a couple of hours. That leaves a lot more time for conducting data analysis, running models, doing the actual research.
“The high speed of the LambdaRail removes a layer of complexity from the task of handling lots of data,” Hawley said. “We no longer need to think about speed or distance. It’s as if everything is right here, no matter the size of the data base or its location.”
“There are many potential projects that could benefit from the use of NLR,” adds Malathi Veeraraghavan, associate professor of electrical and computer engineering. “These include both high-end science projects, such as astronomy, computational biology and genomics, but also commercial applications in the multimedia (audio and video) domain.”
Even visual projects in the humanities that are data-heavy might be accomplished more easily via an NLR connection.
There is a cost for many uses on the LambdaRail, much like buying fare on a passenger train. The further you go and the more service you want, the higher the price. But for big science projects, there is little doubt that NLR connection is worth the fare.
Veeraraghavan is even offering investigators help on learning how to connect to NLR and make full use of it.
“It’s a powerful tool,” she said, “one we should be taking full advantage of.”
Bruce Hayden, professor of environmental sciences, is working with the National Science Foundation on a project to establish the National Ecological Observatory Network (NEON). The 30-year project will consist of 60 research stations across North America, from the arctic to the tropics, all linked by state-of-the-art communications and data-management equipment.
The long-term goal of NEON is to predict future states of ecological systems. To achieve that goal, NEON scientists like Hayden will investigate the interactions between climate change, patterns of land use, invasive species and the spread of infectious disease. Their research one day may answer critical questions, such as the ecological consequences of global warming, or how human activity, such as managing and urbanizing landscapes, affects ecological processes.
“With NEON, people at observatories across the continent will be using the same equipment, working with the same missions and data protocols, working on big-scale projects,” Hayden said. Ecological science over the next 30 years will not be possible without the connectivity and speed of National LambdaRail. “The ability to quickly move and equally share massive and complex data is critical to the success of this kind of complex long-term science. It will transform the way ecological science is done.”
John Hawley, VITA Professor of Astronomy, studies the behavior of gas orbiting black holes in deep space. By observing this swirling gas, astronomers are better able to understand how black holes are formed, and how galaxies interact. Hawley uses three-dimensional time-dependent computational simulations to model observations from telescopes at several locations.
The models are sufficiently complex that he must use the computational capabilities of supercomputers in Illinois and California, which produce massive quantities of data. But that data, when accessed from U.Va., comes in relatively slowly over regular broadband and requires tedious management.
With the LambdaRail network, Hawley says, “A huge layer of complexity in data access, handling and management will be removed. This will allow more time for the creative work of science.”
Hawley said the network will bring in a “new era” for researchers at the University, from molecular medicine to physics and astronomy and beyond. “Investigators will be able to access all sorts of data for all sorts of disciplines, as if the data is right here on our laptops. We’ll be able to do work that we have always wanted to do but couldn’t because of the time constraints imposed by slower access to data and the problems of data storage and management.
“Thirty years ago we traveled to the computers. Ten years ago the computers were in our offices. Now (with LambdaRail), all the computers in the world are in our office, at our fingertips.”
Malathi Veeraraghavan, associate professor of electrical and computer engineering, is working on the creation of an optical computer network called CHEETAH (Circuit-switched High-Speed End-to-End Transport Architecture) that uses NLR links. The CHEETAH network connects U.Va., the City University of New York, North Carolina State University and Oak Ridge National Laboratory.
CHEETAH is being designed for potential future consumers to complement the Internet by allowing a user to make a reservation for bandwidth before sending data.
“Users will be able to receive rate-guaranteed connectivity between computers, storage devices and visualization displays,” Veeraraghavan said. “This type of service will be useful in the commercial sector, for example, for computer gaming and high-quality video-telephony/video-conferencing where low latency [delay] is important and consumers would be willing to pay extra for the capability.”
Veeraraghavan and her colleagues are using NLR high-speed links for the creation of the CHEETAH network, which, she says, is a research testbed to experiment with networking technologies for future designs of the Internet. The first customers will be scientific researchers who need higher bandwidth and predictable delays. Slowly these benefits could be extended to the average user.
“NLR provides us the high-speed connectivity needed for experimental networking research at an affordable cost,” she said.
Researchers in U.Va.’s High Energy Physics Group are seeking evidence for physics beyond the Standard Model (the current theory of fundamental particles and how they interact), such as supersymmetric particles (possibly the “dark matter” of the universe), extra dimensions beyond the four of space-time and micro black holes. The group is a member of a major physics undertaking, the Compact Muon Solenoid Detector (CMS) experiment, which will start taking data at the Large Hadron Collider (LHC) in Geneva in 2007.
Via an international computer grid, U.Va. physicists will have access to huge amounts of data from the highest energy interactions ever produced by man.
“The high-speed connection via LambdaRail to this international grid is essential for our group to be competitive in getting access to this data,” said Brad Cox, professor of physics and a principal investigator with the High Energy Physics Group. “The high-speed data transmission of LambdaRail is a required capability for the University to be full partners in this exciting era of physics that is just getting under way.”
Marty Humphrey, assistant professor of computer science, runs large computational experiments as part of his research group’s Microsoft Institute for High-Performance Computing, as well as for his research supported by the National Science Foundation and the Department of Energy. For their complex simulation experiments, they frequently need large input data from remote sites, such as the San Diego Supercomputing Center.
But it can take hours to download that data through the Internet.
“This really slows down our experimentation,” Humphrey said. “But NLR provides much better connectivity than the existing networking, so it might now be possible to leave the data where it is — at the supercomputing center — and access it dynamically, across the network. This makes things much better. We don’t have to worry that our local data is out-of-sync with the main data, we don’t have to download massive data onto our own computers. We have a much better quality of service; it’s much faster and more predictable.”