P A P E R S

U-Net

Incorporating Memory Management into User-Level Network Interfaces, Anindya Basu, Matt Welsh, Thorsten von Eicken, Presented at Hot Interconnects V, August 1997, Stanford University.
Also available as Department of Computer Science, Cornell University, Technical Report, TR97-1620. [pdf]

User-level network interfaces allow applications direct access to the network without operating system inter-vention on every send and receive. Messages are transferred directly to and from user-space by the network interface while observing the traditional protection boundaries between processes. Current user-level network interfaces limit this message transfer to a per-process region of permanently-pinned physical memory to allow safe DMA. This approach is inflexible in that it requires data to be copied into and out of this memory region, and does not scale to a large number of processes.

This paper presents an extension to the U-Net user-level network architecture (U-Net/MM) allowing messages to be transferred directly to and from any part of an application’s address space. This is achieved by integrating a translation look-aside buffer into the network interface and coordinating its operation with the operating system’s virtual memory subsystem. This mechanism allows network buffer pages to be pinned and unpinned dynamically. Two implementations of U-Net/MM are described, demonstrating that existing commodity hardware and commercial operating systems can efficiently support the architecture.

ATM and Fast Ethernet Network Interfaces for User-level Communication, Matt Welsh, Anindya Basu, and Thorsten von Eicken. Proceedings of the Third International Symposium on High Performance Computer Architecture (HPCA), San Antonio, Texas, February 1-5, 1997.

Fast Ethernet and ATM are two attractive network technologies for interconnecting workstation clusters for parallel and distributed computing. This paper compares network interfaces with and without programmable co-processors for the two types of networks using the U-Net communication architecture to provide low-latency and high-bandwidth communication. U-Net provides protected, user-level access to the network interface and offers application-level round-trip latencies as low as 60 usec over Fast Ethernet and 90 usec over ATM.

The design of the network interface and the underlying network fabric have a large bearing on the U-Net design and performance. Network interfaces with programmable co-processors can transfer data directly to and from user space while others require aid from the operating system kernel. The paper provides detailed performance analysis of U-Net for Fast Ethernet and ATM, including application-level performance on a set of Split-C parallel benchmarks. These results show that high-performance computing is possible on a network of PCs connected via Fast Ethernet.

Copyright 1997 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from the IEEE. Contact: Manager, Copyrights and Permissions / IEEE Service Center / 445 Hoes Lane / P.O. Box 1331 / Piscataway, NJ 08855-1331, USA. Telephone: + Intl. 908-562-3966.

Low-Latency Communication over Fast Ethernet, Matt Welsh, Anindya Basu, Thorsten von Eicken. Proceedings of Euro-Par '96, Lyon, France, August 27-29, 1996.

Fast Ethernet (100Base-TX) can provide a low-cost alternative to more esoteric network technologies for high-performance cluster computing. We use a network architecture based on the U-Net approach to implement low-latency and high-bandwidth communication over Fast Ethernet, with performance rivaling (and in some cases exceeding) that of 155 Mbps ATM. U-Net provides protected, user-level access to the network interface and enables application-level round-trip latencies of less than 60 usec over Fast Ethernet.

U-Net: A User-Level Network Interface for Parallel and Distributed Computing, Anindya Basu, Vineet Buch, Werner Vogels, Thorsten von Eicken. Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP), Copper Mountain, Colorado, December 3-6, 1995.

The U-Net communication architecture provides processes with a virtual view of a network interface to enable user-level access to high-speed communication devices. The architecture, implemented on standard workstations using off-the-shelf ATM communication hardware, removes the kernel from the communication path, while still providing full protection.

The model presented by U-Net allows for the construction of protocols at user level whose performance is only limited by the capabilities of network. The architecture is extremely flexible in the sense that traditional protocols like TCP and UDP, as well as novel abstractions like Active Messages can be implemented efficiently. A U-Net prototype on an 8-node ATM cluster of standard workstations offers 65 microseconds round-trip latency and 15 Mbytes/sec bandwidth. It achieves TCP performance at maximum network bandwidth and demonstrates performance equivalent to Meiko CS-2 and TMC CM-5 supercomputers on a set of Split-C benchmarks.

Thorsten von Eicken

DEPARTMENT OF
COMPUTER SCIENCE,
CORNELL UNIVERSITY.

Last updated: 11/25/96.