The shadow network
Building global scientific experiments using private-wavelength networks.
Special packet-routing schemes made it possible to build an Internet that spans the world and serves billions of active users. But they're bad news when you need to send data in gigabit chunks. For that you need something that involves a lot less processing. Scientists building a new generation of high-bandwidth networks have resorted to switching not packets, but complete optical channels.
Röland Nuijts of the University of Amsterdam points to a massive growth in the data capacity of research networks as one reason why so much work is underway on new ways of building networks.
"Over the past ten years, capacity has doubled roughly every 40 weeks," he says.
Nuijts splits scientific users into three classes. The first is made up of teachers and students, who are numerous but do not use much bandwidth each. The second consists of institutions' virtual private networks, which each need hundreds of megabits per second but are much rarer. The last is populated with super-users conducting huge experiments in physics and biology: "They need the big fat data pipes."
One of the world's largest experiments is about to begin at CERN, the European Organisation for Nuclear Research, where the Large Hadron Collider (LHC) will look for clues to fundamental questions in physics - such as why matter has mass.
"Experiments at the LHC create a huge amount of data. It isn't stored locally but distributed around Europe and a few sites outside Europe," says Ronald van der Pol of Dutch supercomputer support firm SARA.
Nuits adds: "We thought it would be a good idea for these users to have their own lightpaths so that their data-hungry applications are detoured around the IP core."
More than ten centres collect data from the LHC and pass it over dedicated optical channels, or lightpaths, operating at 10Gb/sec. In the LHC's private network, the optical paths are fixed. The links run over a fibre backbone rented from a variety of telecom companies, which is cheaper than CERN building its own physical network. Because the rented links can cope with many wavelengths of light, it is possible to run the LHC's traffic through the same fibres as those used for regular Internet and telecom traffic, without the two interfering.
Switches and multiplexers at regular intervals in those networks - most often spaced at around 80km - make it possible for partner research institutes to access the LHC's network as it runs over the operators' multi-gigabit backbone networks.
The organisations around the LHC need a full-time lightpath network. But many scientific users only need that kind of bandwidth during an experiment. So they want to be able to set up ad hoc networks.
The answer may lie in dynamic lightpaths.
Researchers are trying various approaches. The University of Amsterdam is using specialised switches in its StarPlane project to provide a reconfigurable interconnect for individual wavelengths of light. The wavelength switching system uses a diffraction grating to split the light beams as they enter the switch, directing them on to an array of mirrors driven by micro-electromechanical system (MEMS) levers. Each mirror can steer a different colour beam, or lambda, to a port, where it continues on its way through the network.
Normally, the paths would be set and left as they were until a problem appeared on the network and the wavelengths needed to be rerouted. In the StarPlane approach, the mirrors enable the paths to be switched in minutes. It is hoped that this will be cut to seconds as the management software improves.
Huib Jan van Langevelde wants to use dynamic lightpaths to support an 11,000km diameter virtual radio telescope his team has constructed. The eVLBI (electronic very long baseline interferometry) project makes it possible to merge the data from many telescopes on different continents into a cohesive image that has much greater resolution than even giant installations such as the instrument at Arecibo.
Every piece of data that each telescope captures is correlated for every possible combination of tele-scopes. The result is a set of images a hundred times better than any achievable with optical telescopes.
VLBI has been used before, but it took time - a lot of time. So much data needs to be transported between sites that it was often easier to put it onto a disk or tape and pack it on a plane.
"Schoolchildren ten years ago would ask us why we didn't send the data over the Internet. And we said: that's just not possible," says van Langevelde. But dynamic lightpaths allow the network to be built when needed and the bandwidth reallocated once the eVLBI goes offline.
Call the management
The construction of ad hoc networks, such as that created for the eVLBI experiment, will provide many more science and technology researchers with access to the kind of high-bandwidth networking that super-users currently enjoy.
But there are issues facing dynamic lambda switching, and they're not just technical. One that may be more difficult to solve than the technical challenges is working out how to manage end users setting up virtual circuits across networks that call on fibres owned by many different operators.
Marian Garcia Vidondo, chief operations officer of the Dante organisation, which operates pan-European research networks, says that it takes five networks to link researchers in Poland to colleagues in Canada. "Each one has a different way of managing the connectivity," she says. "We are using domain-monitoring tools that may not be useful when we are dealing with a multi-domain environment."
It used to take months to set up channels through all of these networks, although it is getting easier. Ultimately, though, lambda switching will need automation that works internationally.
"Unless the networks can be automated in the way that IP can be automated, it won't happen," says Dai Davies, general manager of Dante. "IP is very much more friendly to work with than point-to-point management services."
But Dante is far from alone. There are other groups around the world working on ways to use dynamic circuits to build more efficient high-bandwidth networks for the academic community. This technology may spill over into the commercial world as companies start to make use of grid computing and, potentially, virtual supercomputers.
The researchers probably won't care who wins the commercial battle. "Astronomers and physicists want to share resources around the world. Borders between countries are not relevant to them. They want seamless connectivity," says Vidondo.