Duke Computer Scientists Exceed "Gigabit" Data Processing Speeds With Internet Software

This system essentially doubles the current speed at which data can be transferred over the fastest LANs with TCP/IP, the communications standard used for the Internet and the World Wide Web. It is 20,000 times faster than communication through a telephone modem.

The system uses a special high-speed Myrinet LAN operating at Duke's computer science department. Duke's Myrinet system was supplied by Myricom Inc. of Arcadia, Calif., as part of an experimental project, funded by the National Science Foundation, to develop new techniques for high-speed communications.

This Myrinet network is itself rated at more than 1 gigabit. But system bottlenecks limit the rate at which data can move between the network and the computers connected to it, said Jeff Chase, Duke assistant computer science professor.

Using the fastest LANs now on the market, "you'll get about a half a gigabit per second through TCP," Chase said in an interview. However, by using the latest newly released Myrinet network cards together with their own modifications, the Duke team achieved speeds of 1.147 billion bits a second by mid-May, added Andrew Gallatin, a senior systems programmer within Duke's computer science department who works with Chase.

Other members of the Duke group include computer science graduate student Kenneth Yocum and Alvin Lebeck, also an assistant computer science professor.

"It's the first demonstration on public record of TCP/IP running faster than a gigabit per second, end-to-end, one host (network workstation) to another," Chase said. "What we have done is provide the software support that's needed to allow others to achieve similar speeds on other networks that will arrive in the future."

More details can be found at the Duke department of computer science web site at www.cs.duke.edu/ari/trapeze.

LANs are groups of computers that are wired together to allow them to exchange messages and data. They range in speed and complexity from commonplace office networks to the array of high-end Digital Alpha workstations currently connected by Myrinet in a glassed-in "fishbowl lab" in Chase's department.

Those machines and associated equipment are part of a larger Duke computer science testbed cluster funded by grants from the National Science Foundation, Myricom and Intel Corp.

While the Myricom LAN is experimental and operates within a small space, the techniques developed there could eventually help computer users obtain more efficient access to larger scale networks, including a future version of the Internet, Chase said.

It might also mean that standard TCP/IP type software could be used for such cutting edge applications as wiring together individual desktop computers into a massively parallel supercomputer.

"What we've done is narrow the gap between standard TCP/IP communications that everybody loves and knows how to use and have the software to use and these more cutting edge technologies that are harder to use and difficult for people to program," he said.

TCP/IP communications software operates in software layers called a "protocol stack" inside individual LAN computers. That protocol software works in coordination with other software in sending and receiving data across the network connecting each computer.

"The sending and receiving software must be in synch to make sure that they are carrying as many bits as they can carry," Chase added. "That software has to run very efficiently or else the computers won't be able to keep up."

What the Duke team did is "streamline" software operations on both the sending and receiving sides of the central protocol stacks through a variety of modifications.

One change, called "zero-copy data movement," circumvents the time-consuming step of reading data from one area of computer network memory and writing it into another, which taxes a computer's central processing unit (CPU).

"One might think we could fix this problem by using faster CPUs," Chase added. "As it turns out, memory speeds are not growing as fast as CPU speed. As CPU speed increases relative to memory speed, your fast CPU spends a larger share of its processing power waiting for the slow memory to respond to these copy operations."

A related feature, called "scatter/gather input/output," allows data in various locations of computer memory to be rounded up and sent together as large messages. A third, called "checksum offloading," enables computers to use special hardware on their network cards to speed up error checking.

Another innovation, for which the Duke group has filed for a patent, is "adaptive message pipelining," which schedules the movement of data between the network and an individual computer's memory to deliver high performance.

Some of these changes involved modifying software codes. Others involved changing "firmware," codes in network cards that programmers ordinarily cannot alter. By special agreement, Myricom provided the Duke researchers the tools to alter the firmware.

Major components of the network system that the Duke team did not alter are the protocol stacks themselves, obtained from a standard Unix operating system public domain TCP/IP source called FreeBSD.

"A lot of very smart people at a lot of places over a period of decades have done a lot of work trying to write the software that allows TCP communications at very high speeds," Chase said. "In some sense, what we have done really is show that they got it right."