Ensemble 0.70p1 - Patch Release 1

Release Notes and Installation Instructions

Copyright © 1997 Cornell University, Ithaca, NY, USA
All rights reserved.

Last Revised: Tues May 2, 15:33:48 EDT 2000
Maintainers: Mark Hayden, Tim Clark, Ohad Rodeh. 

Introduction

Thank you for choosing Ensemble as your message transport system. Its platform independence allows you to interconnect a wide range of computing environments.

The following instructions guide you through the process of installing Ensemble. Click on the hypertext links for the details about each step. The installation instructions are designed to be as simple as possible. However, please read them through carefully before starting and follow them carefully.  Also, please read the file RELEASE_NOTES for the latest information on this release.

These installation instructions are aimed at getting Ensemble up and running on your system as quickly as possible. After successfully installing the system, you may wish to compile Ensemble for native instruction set of your computer and to use an optimized C library we provide in order to improve performance. The difference is only significant when Ensemble is used in high-performance settings and many users find the performance of bytecode satisfactory.

Required Software

To compile Ensemble you will need either version 2.04 or version 3.00 of the Objective Caml (O'Caml) compiler.

O'Caml is freely available via the World Wide Web at http://pauillac.inria.fr/ocaml Please ensure you have the correct version. See the instructions below on how to obtain it. This is a different compiler version than required by previous versions of Ensemble! Version 0.40 of Ensemble used ocaml-1.05, Version 0.50 of Ensemble used ocaml-1.07, version 0.60 used ocaml-2.01, and we are now up to ocaml-3.00.

On Unix platforms, to compile the HOT C interface, you need a POSIX compliant threads package. (On Solaris, you can use the native threads library).  More recent versions of Linux now provide a POSIX compliant pthreads package (pthreads), however, Linux users with older versions of the operating system, or users who still use SunOS, will need to acquire a threads package, such as the FSU threads package or the rvr_threads package.  Click here for information on how to acquire and install the FSU pthreads package.  The rvr_threads package will be available from Cornell in a separate tar file of contributed code.

To compile the Maestro tools C++ interface, you will need a C++ compiler. We use the Microsoft C++ compiler on Windows NT and gcc on Unix platforms, but other compilers should work.  On NT to build the optimized versions of HOT C and Maestro, you will also need the masm (MS Compatibility Driver) and ml (MS Macro Assembler) executables installed and located in your path.

Upgrade Information

If you are upgrading from an older version, due to problems with long environment variable names on certain platforms, we have changed the names for the Ensemble environment variables to use the prefix 'ENS_' instead of 'ENSEMBLE_'. Please update your configuration.

The name of the group membership service has been changed from 'domain' to 'groupd'. The names of the corresponding environment variables have changed from 'ENS_DOMAIN_' to 'ENS_GROUPD_' , as shown above.

After you have made these changes, to upgrade your system simply install the new version of Ensemble as described below. For further information on the changes in this release, please read the RELEASE_NOTES found in the ensemble directory.

Some environment variables will need to be set before running Ensemble applications which need to communicate with other processes (via the gossip or groupd servers) - see ensemble/doc/tut.ps for more information:

ENSROOT (set to root directory of ensemble distribution, e.g: /usr/u/ensemble)

ENS_ID (set to your user login id)

ENS_PORT (for all Ensemble applications, set to an unused port number)

ENS_GOSSIP_PORT, ENS_GOSSIP_HOSTS (for gossip applications, an unused port number and a list of colon-separated host names, e.g.: mymachine1:mymachine2:mymachine3)

ENS_GROUPD_PORT, ENS_GROUPD_HOSTS (for groupd applications, an unused port number and a list of colon-separated host names)

ENS_DEERING_PORT (for IP Multicast, an unused port number)

ENS_OUTBOARD (if you will be using the HOT outboard application, a full pathway to the outboard executable, e.g: /usr/u/ensemble/demo/outboard)

Licensing

Ensemble code is copyrighted by Cornell University. To use Ensemble you must agree to the terms of the license. Ensemble is freely available software, but you must agree to the terms of the license before you can install it. We ask that you read the licensing agreement before installing Ensemble.  It can be found in ensemble/LICENSE.

Installation

Install Ensemble by following the step-by-step instructions. You need to get the O'Caml compiler, extract the Ensemble software, and then compile the portions you wish to use. In addition, instructions for building Ensemble on multiple platforms.

How to obtain Objective Caml 2.04/3.00

To compile Ensemble you need Objective Caml version 2.04 or 3.00. Other versions of the compiler may NOT work. This is because the O'Caml bytecode format and libraries are not guaranteed to be compatible across versions.

If you already have O'Caml you can check which version you have by typing: 

     ocamlc -v
This gives you the version of the compiler in your path (if there is one). The first line of the output should be: 
     The Objective Caml compiler, version ocaml 2.04
or 
     The Objective Caml compiler, version ocaml 3.00
If you have some other version, then first obtain version 2.04 or 3.00 and install it. O'Caml is freely available from http://pauillac.inria.fr/ocaml and is easy to install (precompiled binaries are available for Windows NT). You do not need the native code compiler, although you can compile Ensemble with it.

The O'Caml Tcl/Tk Interface

If you want to compile the white board demo or wish to use the O'Caml Tcl/Tk interface, you will also need the Ocamltk library. This library is entirely optional. You can install Ensemble without it and install it later if you change your mind. The library can be obtained from the O'Caml Web site, under the distrib directory:

http://pauillac.inria.fr/ocaml/distrib

In order to build the whiteboard demo (wbml) in the demo/tk directory, you will have to make sure that the camltk41 libraries are installed into the CAMLLIB directory.  You may also need to modify the mk/config.mk file to add the path to the Tcl/Tk libraries (TKLIBS).

Back to installation guide.

How to download and extract the Ensemble software

Download Ensemble from http://www.cs.cornell.edu/Info/Projects/Ensemble/ftp.html.  Ensemble is no longer encrypted.  Simply unzip and untar the distribution. It will put the sources in a directory called "ensemble". On Windows NT you can use the winzip program. On Unix platforms, do it this way: 
     # uncompress and untar the distribution
     gunzip ensemble-0_70p1.tar.gz
     tar xvf ensemble.tar
Back to installation guide.

Building Ensemble for Windows NT

** NOTE ** This distribution of Ensemble requires users to build the bytecode libraries - they are not supplied pre-compiled as with previous distributions. Directions to do so are included below. Some demonstration applications have been provided which you can use to test the distribution once the first parts are installed.

You can also compile everything to run on the native instruction set of your computer instead of using an interpreter. This will run faster.

Building on Windows NT

In the configuration file, ensemble\mk\config.nmk, you may need to edit the following macro definitions (you may wish to try the default settings first). MSDEV and LIBSYS should have appropriate values for your C compiler. Set the CAMLLIB environment variable to be the directory where you installed the O'Caml libraries (this is used to set the OCAML_LIB macro. CAMLLIB should have been set when O'Caml was installed -- see the O'Caml README.win32 file).  However, note that this environment variable must be set (via Control Panel, System, Environment) before the DOS console on which you will compile Ensemble is opened.

Compile the bytecode and demonstration programs as follows. Note that on Windows NT, you need to use nmake and the Makefile.nt versions of the makefiles: 

     # build bytecode ensemble
     # directory: ensemble\def
     nmake -f Makefile.nt
If all went smoothly, the initial installation is now complete.

How to run applications

You can now refer to the file 'ensemble\doc\tut.ps' for instructions on executing the demo applications. This involves setting a few environment variables and then executing the applications.

Building the native code version for Windows NT

You can also compile everything to run on the native instruction set of your computer instead of using an interpreter. This will run faster.

To use the native code version of Ensemble, compile it in the directory ensemble\opt. This step compiles all of the Ensemble sources and will take awhile. This is also where you build the native code versions of the HOT tools.  *NOTE* You must make sure that the Microsoft executables, masm.exe and ml.exe are installed and in your path.

In order to compile the HOT tools, you would do the following: 

    # build native code libraries in opt directory
    # directory: ensemble\opt
    nmake -f Makefile.nt
    
    # change directory to demo and build native code demo programs 
    # directory: ensemble\demo
    nmake -f Makefile.nt socketopt

    # change back to opt and build the HOT tools
    # directory: ensemble\opt
    nmake -f Makefile.nt libhot
Once you have built the HOT tools, there will be 3 executable files placed in the ensemble\demo directory: hot_test.exe, hot_testo.exe, and outboard.exe. The file hot_test.exe runs basic single-process sanity checks of the HOT C / Ensemble interface. The files hot_testo.exe ("o" is for outboard) and outboard.exe perform the same sanity checks, however, the Ensemble protocols are isolated in a separate process (outboard.exe) and communication takes place via TCP sockets.  The hot library file, ensemble\lib\nt\libhot.a will  be placed in the ensemble\lib\nt directory.  You will need to link this hot library with your own "C" applications.

There is also an auxilliary HOT test program, called hot_test2.c, which can be compiled as follows: 

    # build the hot_test2 demo program
    # directory: ensemble\opt
    nmake -f Makefile.nt hot_test2
This will build and install the program hot_test2.exe in the demo directory.  The program hot_test2 is a two-process RPC performance test.

In order to run hot_testo.exe, you first have to set environment variables. This is done through the Control Panel, by selecting the System icon, and then selecting the Environment tab. You then enter the environment variable ENS_OUTBOARD and for its value, you enter the complete path to the outboard.exe file, e.g.:

ENS_OUTBOARD

c:\ensemble\demo\outboard.exe

Then select "Set" and "OK".  The Environment data entry window will close.  You may now open a MS DOS Console window and run hot_testo.exe.

If you attempt to run hot_testo.exe and you get the following output, it means that you did not set the ENS_OUTBOARD environment variable before you opened the Command prompt window (it needs to be set before the Command prompt is opened in order to be exported):

HOT_TEST: CDEMO: starting

PANIC: spawnvp

Close the Command prompt window, go to the Control Panel, open the System icon, select Environment, add the ENS_OUTBOARD variable, then select "Set", and "OK". Open a new Command prompt window and try again.

The HOT tools can also be built with cryptographic support as follows.  First you will need to edit the file, config.nmk, and uncomment the CRYPTO_LINK lines.  Then perform the following at the command line: 

    # build crypto library
    # directory: ensemble\opt
    nmake -f Makefile.nt crypto
    # build HOT tools with crypto support
    # directory: ensemble\opt
    nmake -f Makefile.nt libhot-crypto
This will build versions of the hot_tests and outboard executables with a "-crypto" extension and place them in the demo directory.

Building the Maestro tools for Windows NT

** NOTE **  The Maestro IIOP software, built when compiling on Unix, has not yet been ported to NT.
You need to have compiled the HOT library before compiling the Maestro C++ interface. This step generates the file ensemble\maestro\maestro-nt\libmae-nt.lib as well as the maestro test programs: 
    # build the Maestro tools
    # directory: ensemble\opt
    nmake -f Makefile.nt maestro-nt
You should now have three test programs, ensemble\maestro\maestro-nt\maestro_test.exe, ensemble\maestro\maestro-nt\maestro_perf.exe, and ensemble\maestro\maestro-nt\group.exe.  maestro_test is a single process sanity test in which several group-member objects are created which then randomly send messages and leave/rejoin the group.  maestro_perf is a two-process RPC performance test.  group is an application which tests the new, simpler state transfer algorithm. To get started with Maestro and Ensemble, we recommend you read the Maestro documentation and start from the test programs (in the corresponding .C files) -- you can modify and experiment with them.

Building the EJava Interface for Windows NT

The EJava interface to Ensemble was built and tested using Sun Microsystem's JDK1.2.  Before trying to build this interface, please check and make sure you have access to JDK1.2.  Then follow these steps: 
    # build the EJava interface
    # directory: ensemble\ejava
    nmake -f Makefile.nt
You can then run one of the demo programs, EnsDemoOutboard or ThreadTest as follows: 
    java EnsDemoOutboard
EnsDemoOutboard is a graphical version of the mtalk demo.  It allows processes to "chat" to each other. The demo application, ThreadTest, is a test which stresses multi-threading.   It requires one parameter, the number of group members to wait for before beginning the test.  For example, to run the test with 2 members, start the test on 2 machines (or 2 consoles) as follows: 
    java ThreadTest 2

Installation is complete

This concludes the instructions for building Ensemble on Windows NT systems.
Back to installation guide.

Building Ensemble on Unix Systems

** NOTE ** This distribution of Ensemble requires users to build the bytecode libraries - they are not supplied pre-compiled as with previous distributions. Directions to do so are included below. Some demonstration applications have been provided which you can use to test the distribution once the first parts are installed.

You can also compile everything to run on the native instruction set of your computer instead of using an interpreter. This will run faster. You can also use a special Socket library to improve the performance of network operations. Details on how to build all of these things are given below.

Building on Unix Systems

First, make sure the CAMLLIB environment variable is set, and the Ocaml binaries are accessible from your PATH.  Then compile the Ensemble system and demonstration programs as follows (on UNIX, the make automatically builds the demo programs): 
     # make the Ensemble system
     # directory: ensemble/def
     make
If all went smoothly, the initial installation is now complete, and you can try out the demo programs following the instructions below.

How to run applications

You can now refer to the tutorial, 'ensemble/doc/tut.ps', for instructions on executing the demo applications. This involves setting a few environment variables and then executing the applications. Come back to here to install additional parts of Ensemble.

Graphical Demos

If you installed the O'caml Tcl/Tk interface, compile the white board demo and "Game of Life" demo. (Note: In order to use the Tcl/Tk interface you may need to set the TKLIB macro in 'ensemble/mk/config.mk' and you must have installed the camltk41 libraries).  These applications will not compile if the Ocaml TK interface has not been installed. 
     # make the Ensemble tk library
     # directory: ensemble/def
     make tk
     # make the white board demo
     # directory: ensemble/demo/tk
     make

     # make the game-of-life demo
     # directory: ensemble/demo/life
     make

Environment variables

In order to compile platform-dependent parts (the Socket library, the native code, the HOT C interface, and the Maestro C++ interface) some configuration needs to be done. In sh, you set them like this: In tcsh, set them like this: 
         setenv CAMLLIB /usr/local/lib/ocaml
         setenv PATH /usr/local/bin/ocaml:$PATH
We recommend you add these to your '.profile' or equivalent for your shell.

Building the Socket library

The Socket library supports extensions to the Unix library concerned with communication. It is derived in large part from the O'Caml Unix library. It is not necessary on Unix (the O'Caml Unix library is used by default), but using the Socket library increases Ensemble performance and decreases memory usage. Note that you only need to compile this separately if you are using the bytecode version of Ensemble. The build for the native code system compiles the Socket library by default. 
    # In ensemble/mk/config.mk, edit the macro definition for HSYS_TYPE to be "skt".
    # directory: ensemble/def
    make socket
    # directory: ensemble/def
    make clean; make
When you compile other parts of Ensemble in the future, the Socket library will be used in place of the Unix library.

Building the HOT tools

The HOT tools can be compiled from either the bytecode or native code version of Ensemble. Compiling from 'ensemble/def' builds the bytecode version. Compiling from 'ensemble/opt' builds the native version. Either way the C library file that is generated is platform dependent.

This step generates the file 'ensemble/lib/PLATFORM/libhot.a' (where PLATFORM is the name of your platform) and installs the HOT library files into 'ensemble/lib/PLATFORM'.

**NOTE ** For users on SunOS, lacking Solaris threads,  please contact us for a thread package to use (the rvr_threads package).  Alternatively, please see the section below on POSIX threads.  You will have to obtain this threads package, install it, and modify the ensemble/mk/config.mk file to reference the corresponding threads interface file (either ensemble/hot/rvr_intf.c or pthread_intf.c). 

    # build the HOT tools
    # directory: ensemble/def (or ensemble/opt)
    make libhot
There is also an auxilliary HOT test program, called hot_test2.c, which can be compiled as follows: 
    # build the hot_test2 demo program
    # directory: ensemble/def
    make hot_test2
This will build and install the program hot_test2 in the demo directory.  The program hot_test2 is a two-process RPC performance test.
The HOT tools can also be built with cryptographic support as follows.  First you will need to edit the file, config.mk, and uncomment the CRYPTO_LINK lines.  Then perform the following at the command line: 
    # build the HOT tools with crypto
    # directory: ensemble/def (or ensemble/opt)
    make libhot-crypto
This will install the hot_test-crypto, hot_testo-crypto and outboard-crypto files in the demo directory.

Building the Maestro tools

You need to have compiled the HOT library before compiling the Maestro C++ interface. You need a C++ compiler in order to compile Maestro. This step generates the file 'ensemble/maestro/conf/PLATFORM/libmae.a' (where platform is the name of your platform). 
    # build the Maestro and IIOP libraries and utilities
    # directory: ensemble/def 
    make maestro
    # build the Maestro and IIOP test applications
    # directory: ensemble/def 
    make maestro_test
(** NOTE ** You can also run "make" in the maestro directory)

You should now have five test programs, ensemble/maestro/test/maestro-test , ensemble/maestro/test/maestro-perf,  ensemble/maestro/test/test-replicated, ensemble/maestro/test/test-simple, and ensemble/maestro/test/group.  maestro-test is a single process sanity test in which several group-member objects are created which then randomly send messages and leave/rejoin the group.  maestro-perf is a two-process RPC performance test.  Both test-replicated and test-simple are Maestro IIOP test applications. group is an application which tests the new, simpler state transfer algorithm. See ensemble/maestro/REF.ps for more information on the Maestro IIOP tools. To get started with Maestro and Ensemble, we also recommend you read the Maestro documentation and start from the test programs (in the corresponding .C files) -- you can modify and experiment with them.

Building the native code version

To use the native code version of Ensemble, compile it in the directory 'ensemble/opt'. This is also where you build the native code versions of the HOT tools. Note that this automatically compiles the native code Socket library for you. This step compiles all of the Ensemble sources (including the demos) and will take awhile. 
    # build native code libraries
    # directory: ensemble/opt
    make

    # build the HOT tools for native code
    # directory: ensemble/opt
    make libhot

    # build the Maestro and IIOP libraries and utilities
    # directory: ensemble/opt 
    make maestro
    # build the Maestro and IIOP test applications
    # directory: ensemble/opt 
    make maestro_test

Building the EJava Interface

The EJava interface to Ensemble was built and tested using Sun Microsystem's JDK1.2.  Before trying to build this interface, please check and make sure you have access to JDK1.2.  Then follow these steps: 
    # build the EJava interface
    # directory: ensemble/ejava
    make
You can then run one of the demo programs,  EnsDemoOutboard or ThreadTest as follows: 
    java EnsDemoOutboard
EnsDemoOutboard is a graphical version of the mtalk demo.  It allows processes to "chat" to each other. The demo application, ThreadTest, is a test which stresses multi-threading.  ThreadTest, requires one parameter, the number of group members to wait for before beginning the test.  For example, to run the test with 2 members, start the test on 2 machines (or 2 consoles) as follows: 
    java ThreadTest 2

Installation is complete

This concludes the instructions for building Ensemble on Unix systems.
Back to installation guide.

"Contrib" Code Removed,  Packaged Separately

With Ensemble version 0.70p1, we only include the Ensemble core toolkit.  All other contributed work will be available separately by request.  Separate tar (or zip) files will be generated for the add-on work, including  the Dynamic Virtual Private Network (DVPN) code, the rvr_threads package, and a related project JavaGroups . These add-ons will be available upon request.
 

Java Interface Updated

Due to popular demand, the Ensemble Java interface, Ejava, is now included in the core Ensemble release.  The interface and demo programs can be found in the ensemble/ejava directory.  The ejava interface has been updated with new functions to allow it to inter-operate with HOT "C"/Maestro applications.  See the RELEASE_NOTES for more information.

NOTE: This Java interface was developed using  Sun Microsystem's JDK 1.2.  Other versions of Java or earlier versions of the JDK may not work properly.

Building Ensemble on Multiple Platforms

Multiple versions of Ensemble can co-exist within the same directory tree. All platform-dependent files now have a separate directory describing the platform for which they were compiled. For Unix platforms, you just need to ensure that the 'MACHTYPE' and 'OSTYPE' environment variables are correctly set. For Windows NT, the name of the platfrom ('nt') is embedded in the Windows NT makefiles and does not need to be set. For instance, the platform-independent bytecode library for Ensemble is called 'libens.cma' ('cma' is the O'Caml suffix used for bytecode libraryies). The platform-dependent, native code library for Ensemble on i386, Linux platforms is called 'lib/i386-linux/libens.cmxa' ('cmxa' is the O'Caml suffix for native code libraries').

The only exceptions are the demo programs. These are platform independent but do not follow the naming convention. Whenever you compile executables for a new platform, the old executable demos are removed and replaced with the new ones.

This concludes the instructions for building Ensemble on multiple platforms.
Back to installation guide.
 

"Make Clean" Changes for Removing Object Files/Executables

Since the directory tree has changed to accomodate platform-specific builds, the behavior of the Make command, "make clean" (or "nmake -f Makefile.nt clean" on NT) has changed as well.  Now, "make clean" from the top-level directory (ensemble) will call "make clean" in each of the def and opt directories.  "make clean" in each of these directories will remove the platform-specific object code directories, e.g. def/obj/sparc-solaris, as well as the platform-specific library files, e.g: ensemble/lib/sparc-solaris, the compiled demo programs, and the compiled platform-specific maestro files, e.g. ensemble/maestro/conf/sparc-solaris.

The command "make realclean"  will  attempt to perform the "make clean" operations ON ALL PLATFORMS, e.g. sparc-solaris, linux, and NT, and so is to be used only when cleaning up the entire distribution directory of ALL compiled object files and executables.

POSIX threads

If your system does not have a native POSIX threads library, install the FSU pthreads.tar.gz package. You can get it from: After installing the library. Update in ensemble/mk/config the macros GTHREADS, GTHREADS_INC to point to your installation. If necessary, add new HOT_CFLAGS_arch_os, HOT_MLLINK_arch_os, HOT_LINK_arch_os, and HOT_THREAD_OBJ_arch_os macros to config.mk for your platform (and send them to us so we can add them to the next distribution).
Back to installation guide.


SECURITY

Ensemble version 0.70p1 supports several security properties, including user message encryption, authentication, and group member authorization. In order to make ensemble with cryptographic support,  FIRST, you must edit config.mk.  You will see the CRYPTO_LINK define commented out by default.  Comment out the top (empty) CRYPTO_LINK line and uncomment the next 3 CRYPTO_LINK lines.  On NT, simply uncomment the lines following the CRYPTO_LINK line.  This will now link in the crypto libraries when you build executables.  Now perform the following steps:

(** NOTE:  This version of the security code requires PGP5.0 on Unix.)

Unix: 

    # Make the regular crypto library
    # directory: ensemble/def
    make crypto

    # Make the demo's with the crypto library
    # directory: ensemble/def
    make

    # Make the optimized crypto library
    # directory: ensemble/opt
    make crypto
    # Make the demo's with the optimized crypto library
    # directory: ensemble/opt
    make

    # Make libhot with the optmized crypto library
    # directory: ensemble/opt
    make libhot-crypto
After following the above steps, to build maestro test applications with crypto support, edit the file: 
ensemble/maestro/test/Makefile
and uncomment out the crypto versions of CRYPTOLIB and HOTLIB, commenting out the default versions, e.g: 
#HOTLIB		= $(ENS)/lib/$(PLATFORM)/libhot.a
HOTLIB		= $(ENS)/lib/$(PLATFORM)/libhot-crypto.a

#CRYPTOLIB	=
CRYPTOLIB	= $(ENS)/lib/$(PLATFORM)/libcryptoc.a
Then: 
    # Make maestro with the crypto library
    # directory: ensemble/def or opt
    make maestro_test
** NOTE:  on NT, change "make" to:
nmake -f Makefile.nt
and things should compile smoothly.
 

Back to installation guide.