The Ensemble Project

Cornell University

Abstract

The Ensemble System is a high performance, reconfigurable, Plug'n'Play network protocol architecture. Modules from a large collection of micro-protocols can be stacked and restacked in a variety of ways, allowing applications to adapt, at run time, to changes in network bandwidth and latency, to processor and network failures, to changing security requirements, etc. All currently supported micro-protocols deal with groups of processes, rather than just two. This supports distributed applications better, and allows for better resource utilization and management. Ensemble is supported under Unix and NT. Drivers exists for UDP (with or without multicast extensions) through BSD sockets, or for ATM through U-Net.

Principal Investigators

• Kenneth P. Birman.
• Robbert van Renesse.
• Werner Vogels.

Links to Ensemble places of interest

• Glossy brochure.
• License and Getting Software.
• Timeline and Milestones.
• Frequently Asked Questions.
• Security.
• Active Networking.
• ML.
• ....

Recent Highlights

• Ensemble software distribution. We have a solid version of the Ensemble system now, usable on a wide variety of platforms (including several versions of UNIX, NT and NT Clusters) and from a wide range of languages (C++, C, Java, ML/O'Caml, ADA) and systems (CORBA through Electra or directly via IIOP, COM/DCOM, Tcl/Tk). A link to the download area can be found above.
• Adaptive networks technology. We have developed tools with which Ensemble can be used to ``manage'' networks with large numbers of endpoints. This work is novel because we are able to address not just management of membership, but also dynamic reconfiguration of the protocols in use by Ensemble itself, protocol stacks external to Ensemble, and have done this in a way that integrates well with security tools. A paper on this work can be found here.
• A dynamic virtually private networks technology. We have been using the adaptation features of Ensemble to manage security keys in a virtual private network architecture, which currently runs on the Linux system (it will be ported to NT and perhaps other systems too in the future). A DVPN supports dynamically changing keys (we change ours every 30 seconds), the possibility to exclude an untrusted machine from the network so that it will lose access to the DVPN even if the key it was using has been compromised, authentication, and high availability. We are extending the DVPN architecture to also address the protection of critical network services (routing, name services, etc), user-level servers (file systems, databases), and to automate the management of quality of service properties as well as security properties. Conventional VPN's lack these properties: they are static, may use the same key for years at a time, and have no provision for excluding untrusted machines. A paper on this work is under preparation and will shortly be added to the archive of Horus and Ensemble papers.
• We have developed a new, extremely scalable, stable probablistic multicast protocol. Similar in spirit to the USENET protocol called MUSE, our ``pbcast'' protocol employs a gossip technique to run on very large networks with purely local costs. Unlike traditional reliable multicast protocols, pbcast doesn't degrade (in terms of throughput or latency) when individual machines suffer from performance or responsiveness problems -- a condition under which many reliable multicast protocols exhibit extreme performance anomalies. An initial paper on this work can be found in our archive of Horus and Ensemble papers, but a much more complete one is still under preparation. The protocol itself is included as part of our software distribution.
• We've extended Friedman's work on secure, real-time fault-tolerance in scalable servers to include IP failover for TCP connections, such as arise in HTTP servers. We are wrapping this up now, and will soon have a software distribution of a Maestro tool for implementing IP failover in new servers, and a comprehensive paper on the technique. The method requires no changes on the client side -- a client that uses TCP to connect to a server, for example, employs a standard TCP stack and sees no disruptions even if failures and restarts occur.
• We've delivered Ensemble into DARPA's Quorum program through BBN's AquA and QuO effort. Over time, this should make Ensemble available to a large community working on the Next Generation Internet Initiative, the Naval SC-21 ``Surface Combatant'' communication standards, and other members of the DARPA networking community.

General Research and Development Goals and Demonstrations for 1997

• Development of a Web service with predictable access times over Ensemble. This uses a TCP/IP roll-over scheme that is transparent to Web browsers. We also plan to provide some form of safe Web transactions.
• When a process outside a physical firewall wants to join an Ensemble application process group that is running inside the firewall, we can dynamically reconfigure the application to install microprotocols that provide signing and/or encryption, that is, switch to a dynamically created virtual private network. We plan to demonstrate this, and analyse the robustness against adversarial attacks.
• Group rekeying--the ability for a group to generate and agree on a new key. This can be compared to revocation: after a group member leaves (or crashes, or is thrown out), it may be important that he no longer is able to receive messages sent within the group, or be able to sent messages to the group.
• Demonstration of secure group merges: Ensemble groups can merge, for example, after a partition resolves. Doing so securely is somewhat tricky, and is similar to security amplification. We cooperate with the Hebrew University of Jerusalem on the issues of secure group communication.
• Ensemble is now loaded into NuPrl, a proof engine. We plan to demonstrate formal proofs of Ensemble protocol properties, as well as applying partial evaluation to improve Ensemble performance.
• Support for GSS. Currently, we're only supporting Kerberos.
• Porting Ensemble to run over an Active Network. This, in turn, will allow us to experiment with migrating group members, new multicast routing protocols, internet resource reservation, and dealing with so-called message implosion problems.
• Analysis of scalability of key group communication protocols. In particular, message stability detection is relatively hard to scale. We are currently simulating a selection of these protocols.
• Development of a secure, fault-tolerant, scalable parallel execution facility that will demonstrate many Ensemble features: replication, secure groups, RPC, Unix and NT support, migration, etc.
• We are completing a port of Electra, a CORBA-compliant ORB, to Ensemble. Electra supports object groups, which may be used to implement fault-tolerant objects. We are also investigating if we can support fault-tolerant object invocations through IIOP.

General Introduction and Papers

• Building Secure and Reliable Network Applications (book)
• Software for Reliable Networks (Scientific American)
• Building Adaptive Systems Using Ensemble
• The Maestro Group Manager: A Structuring Tool For Applications With Multiple Quality of Service Requirements
• Optimizing Layered Communication Protocols
• Probabilistic Broadcast
• Design and Performance of Horus: A Lightweight Group Communications System (html version).
• Design and Performance of Horus: A Lightweight Group Communications System (GIF version).
• A Framework for Protocol Composition in Horus.
• Protocol Composition in Horus.
• Incorporating System Resource Information into Flow Control.
• Strong and Weak Virtual Synchrony in Horus.
• Horus: A Flexible Group Communications System.
• A Security Architecture for Fault-Tolerant Systems.
• Support for Complex Multi-Media Applications using the Horus system.
• Operating Support for Mobile Agents.