9 Horus Object Tools (HOT)
HOT, the Horus Object Tools, forms the Ensemble C interface. This
chapter describes its interfaces, and how to use it.
9.1 Introduction
The Horus Object Tools (HOT) is a toolkit developed for
the Horus group communication system. Horus is the predecessor of
Ensemble, developed at Cornell university in the years 1990-1995.
HOT was ported to Ensemble, and it now forms its C interface. On top
of HOT, the maestro toolkit provides a C++ class hierarchy. This
document walks through the major HOT interface file - hot_ens.h.
9.2 The major interface
File hot_ens.h is located in the include subdirectory
of hot.
#define HOT_ENS_DEFAULT_TRANSPORT "UDP"
The HOT_ENS_DEFAULT_TRANSPORT parameter designates which
transport the system is to use. The major options are UDP, TCP,
or IP-multicast (DEERING). The default is UDP.
#if 0
#define HOT_ENS_DEFAULT_PROTOCOL
"Top:Heal:Switch:Leave:Inter:Intra:Elect:Merge:Sync:Suspect:"
"Top_appl:Pt2pt:Frag:Stable:Mnak:Bottom"
#else
#define HOT_ENS_DEFAULT_PROTOCOL
"BOTTOM:MNAK:PT2PT:CHK_FIFO:CHK_SYNC:"
"TOP_APPL:STABLE:VSYNC:SYNC:"
"ELECT:INTRA:INTER:LEAVE:SUSPECT:"
"HEAL:TOP"
#endif
The HOT_ENS_DEFAULT_PROTOCOL parameter designates which
Ensemble protocol stack to use. There are many options, one may choose
to use state-transfer, total-ordering, security, probabilistic
protocols, debugging layers and more. This parameter is used, only if
the next parameter HOT_ENS_DEFAULT_PROPERTIES is not used.
#define HOT_ENS_DEFAULT_PROPERTIES "Gmp:Sync:Heal:Switch:Frag:Suspect:Flow"
#define HOT_ENS_DEFAULT_HEARTBEAT_RATE 5000
#define HOT_ENS_DEFAULT_GROUP_NAME "<Ensemble_Default_Group>"
Since Ensemble supports about 50 layers, and many more layer
combinations, simply requesting a layer combination is too
low-level. A higher level interface is provided by the PROPERTIES
parameter. Each property describes a set of layers to use, and a set
of properties specifies a layer combination. The default is to use:
Property |
description |
Gmp |
group membership |
Sync |
Virtual Synchrony |
Switch |
layer switching on the fly |
Frag |
message fragmentation |
Suspect |
Failure suspicion |
Flow |
Flow control |
|
/* Group context descriptor.
*/
typedef struct hot_gctx *hot_gctx_t ;
It is possible to join several groups in a HOT application.
/* Endpoint id.
*/
typedef struct {
char name[HOT_ENDP_MAX_NAME_SIZE] ;
} hot_endpt_t ;
The type of an endpoint.
#define HOT_ENS_MAX_PROTO_NAME_LENGTH 256
#define HOT_ENS_MAX_PARAMS_LENGTH 256
#define HOT_ENS_MAX_PRINCIPAL_NAME_LENGTH 64
#define HOT_ENS_MAX_KEY_LEGNTH 40
#define HOT_ENS_MAX_PROPERTIES_LENGTH 256
#define HOT_ENS_MAX_VERSION_LENGTH 128
#define HOT_ENS_MAX_GROUP_NAME_LENGTH 128
#define HOT_ENS_MAX_IO_NAME_LENGTH 128
typedef struct {
int ltime ;
hot_endpt_t coord ;
} hot_view_id ;
typedef unsigned hot_rank_t ;
typedef int hot_bool_t ;
Some constants, the view logical time, a member rank, and the boolean
type. A view logical time (lt) is an identifier that is attached to a
view. A view is the current membership. As group
membership changes over time, so do group views. An lt is a logical
time that allows comparing two views and determining which view is
older.
/* View state.
*/
typedef struct {
char version[HOT_ENS_MAX_VERSION_LENGTH] ;
char group_name[HOT_ENS_MAX_GROUP_NAME_LENGTH] ;
hot_endpt_t *members ;
hot_rank_t nmembers ;
hot_rank_t rank ;
char protocol[HOT_ENS_MAX_PROTO_NAME_LENGTH] ;
hot_bool_t groupd ;
hot_view_id view_id ;
char params[HOT_ENS_MAX_PARAMS_LENGTH] ;
hot_bool_t xfer_view ;
hot_bool_t primary ;
hot_bool_t *clients;
char key[HOT_ENS_MAX_KEY_LEGNTH];
} hot_view_state_t ;
The view data structure detailing view information. For example: the
Ensemble version number, group name, a list of members, etc.
9.3 Application callback types
This section is dedicated to a list of callbacks that a HOT
application must provide. These functions are called by Ensemble
whenever certain events occur. For example, the view callback is
invoked whenever a view change occurs.
typedef void (*hot_ens_ReceiveCast_cback)
(hot_gctx_t gctx, void *env, hot_endpt_t *origin, hot_msg_t msg) ;
A multicast message has been received. The parameters are:
-
gctx:
- group context.
- env:
- an context parameter.
- origin:
- the origin of the message.
- msg:
- the received message.
typedef void (*hot_ens_ReceiveSend_cback)
(hot_gctx_t gctx, void *env, hot_endpt_t *origin, hot_msg_t msg) ;
A point-to-point message has been received, as above.
typedef void (*hot_ens_AcceptedView_cback)
(hot_gctx_t gctx, void *env, hot_view_state_t *view_state) ;
A new view has been accepted.
typedef void (*hot_ens_Heartbeat_cback)
(hot_gctx_t gctx, void *env, unsigned rate) ;
Got a heartbeat event (invoked periodically). Current time (in
milliseconds) is specified.
typedef void (*hot_ens_Block_cback)
(hot_gctx_t gctx, void *env) ;
The group is about to block for a view change
typedef void (*hot_ens_Exit_cback)
(hot_gctx_t gctx, void *env) ;
The member has left the group.
9.4 Structures and join options
/* Application callback configuration.
*/
typedef struct {
hot_ens_ReceiveCast_cback receive_cast ;
hot_ens_ReceiveSend_cback receive_send ;
hot_ens_AcceptedView_cback accepted_view ;
hot_ens_Heartbeat_cback heartbeat ;
hot_ens_Block_cback block ;
hot_ens_Exit_cback exit ;
} hot_ens_cbacks_t ;
This is a structure that must be defined by any HOT application. It
is passed as an argument to HOT, in the join options (see below).
typedef struct {
unsigned heartbeat_rate; /* Rate of heartbeat upcalls, in millisec. */
char transports[HOT_ENS_MAX_TRANSPORTS_LENGTH]; /* List of transports */
char group_name[HOT_ENS_MAX_GROUP_NAME_LENGTH]; /* ASCII name of the group */
char protocol[HOT_ENS_MAX_PROTO_NAME_LENGTH]; /* The protocol stack */
char properties[HOT_ENS_MAX_PROPERTIES_LENGTH]; /* Protocol properties */
hot_bool_t use_properties; /* Set if properties are to be used */
char params[HOT_ENS_MAX_PARAMS_LENGTH]; /* Protocol parameters */
hot_bool_t groupd; /* Set if group daemon is in use */
hot_ens_cbacks_t conf;
void *env;
char **argv;
hot_bool_t debug;
hot_bool_t client;
char outboard[HOT_ENS_MAX_IO_NAME_LENGTH]; /* Outboard module to use */
/* Normally, Ensemble generates a unique endpoint name for each
* group an application joins (this is what happens if you leave
* 'endpt' unmodified). The application can optionally provide
* its own endpoint name. It can, for instance, reuse an
* endpoint name generated by Ensemble for another group (the
* same endpoint name can be used to join any number of groups).
* The application can even generate an endpoint on its own.
* Such names should be unique. It is best if they contain only
* printable characters and do not contain spaces because
* Ensemble my print them out in debugging or error messages.
* (The names generated by Ensemble fit these characteristics.)
* See Endpt.extern in ensemble/type/endpt.mli for more
* information.
*/
hot_endpt_t endpt ;
/* [OR] Security options.
*/
char princ[HOT_ENS_MAX_PRINCIPAL_NAME_LENGTH]; /* Principal name */
char key[HOT_ENS_MAX_KEY_LEGNTH]; /* key length */
hot_bool_t secure; /* Secure group. */
} hot_ens_JoinOps_t;
A list of join options. This includes the transport to be used, the
heartbeat rate, the group name, a parameter list, a list of callbacks,
an environment variable (used in the callbacks), and more.
The outboard parameter designates whether to use the outboard or not.
There are two modes in which HOT can be run, a standard inboard mode,
and a debugging outboard mode. In the inboard mode the HOT C
code is directly linked with the Ensemble ML code. In order to allow
separately debugging the C code without complications of a shared heap
between ML and C objects the outboard mode is provided. In this mode,
the HOT C code communicates with Ensemble thorough a TCP/IP
socket. The outboard parameter describes the path to the
outboard executable. This executable is created by the HOT makefile,
and it's standard location is in the demo subdirectory.
typedef enum {
HOT_ENS_MSG_SEND_UNSPECIFIED_VIEW,
HOT_ENS_MSG_SEND_CURRENT_VIEW,
HOT_ENS_MSG_SEND_NEXT_VIEW
} hot_ens_MsgSendView ;
Messages can be sent in the current view, in the next view, etc.
9.5 Application calls
This section describes a list of calls usable directly from the
application.
hot_err_t hot_ens_Start(
char **argv
) ;
Start the Ensemble thread. The current design is that Ensemble is
started in one thread, and HOT code and application are run in another
thread. This allows running several application threads using Ensemble
in a thread-safe manner.
void hot_ens_InitJoinOps(
hot_ens_JoinOps_t *ops
) ;
Initialize an options structure.
hot_err_t hot_ens_Join(
hot_ens_JoinOps_t *ops,
hot_gctx_t *gctxp /*OUT*/
) ;
Join a group. On success, returns group context in *gctxp.
hot_err_t hot_ens_Leave(hot_gctx_t gctx) ;
Leave a group. After this downcall, the context becomes invalid.
hot_err_t hot_ens_Cast(
hot_gctx_t g,
hot_msg_t msg,
hot_ens_MsgSendView *send_view /*OUT*/
) ;
Send a multicast message to the group.
The view in which the message is going to be sent
is returned in send_view, if send_view is not NULL.
This allows sending a message in the next view.
hot_err_t hot_ens_Send(
hot_gctx_t g,
hot_endpt_t *dest,
hot_msg_t msg,
hot_ens_MsgSendView *send_view /*OUT*/
) ;
Send a point-to-point message to the specified group member.
The view in which the message is going to be sent
is returned in send_view, if send_view is not NULL.
This allows sending a message in the next view.
hot_err_t hot_ens_Suspect(
hot_gctx_t gctx,
hot_endpt_t *suspects,
int nsuspects
) ;
Report specified group members as failure-suspected.
hot_err_t hot_ens_XferDone(
hot_gctx_t gctx
) ;
Inform Ensemble that the state-transfer is complete.
This callback should be used when the state-transfer layer,
XFER, is in the protocol stack. This layer creates a special
state-transfer view after a view-change when new members arrive.
In this state-transfer view, the application can perform arbitrary
communication. Once all application in the view perform an XferDone
downcall, the view is terminated, and a new, regular view is
installed.
hot_err_t hot_ens_ChangeProtocol(
hot_gctx_t gctx,
char *protocol_name
) ;
Request a protocol change.
hot_err_t hot_ens_ChangeProperties(
hot_gctx_t gctx,
char *properties
) ;
Request a protocol change (specify properties).
hot_err_t hot_ens_RequestNewView(
hot_gctx_t gctx
) ;
Request a new view to be installed.
hot_err_t hot_ens_Rekey(
hot_gctx_t gctx
) ;
Request a rekey operation.
hot_err_t hot_ens_MLPrintOverride(
void (*handler)(char *msg)
) ;
hot_err_t hot_ens_MLUncaughtException(
void (*handler)(char *info)
) ;
These are functions used to define an action taken when ML throws an
exception, or prints out a value. These functions should not be
tampered with by the casual user.