 |
Maestro Open Toolkit: Group Members
|
This document is a part of online
Ensemble documentation, under
Maestro Open Toolkit.
The Maestro_GroupMember class implements an abstract data type for
group members in the
Ensemble system.
There are public downcalls
for joining groups, sending messages, etc., and (protected) callbacks that are
invoked when group events arive. Typically an application will define a
subclass of the Maestro_GroupMember class, which will overload
callbacks/downcalls of Maestro_GroupMember to implement
application-specific functionality.
Code Example:
Using Maestro_GroupMember
Index of Topics:
Starting Ensemble
Creating Maestro_GroupMember Objects
Joining and leaving groups
Sending and receiving messages
Participating in membership changes
Using external failure detectors
Changing protocol stacks
Index of Downcalls/Callbacks:
start
join
leave
send
cast
suspect
changeProtocol
requestNewView
myEndpID
grpMemb_ReceiveCast_Callback
grpMemb_ReceiveSend_Callback
grpMemb_ViewMsg_Callback
grpMemb_AcceptedView_Callback
grpMemb_Block_Callback
grpMemb_Exit_Callback
grpMemb_Heartbeat_Callback
The Maestro_GroupMember class provides the following constructor:
Maestro_GroupMember(Maestro_GrpMemb_Options &membOps);
Initialization parameters are specified with the membOps argument.
The Maestro_GrpMemb_Options structure contains the following fields:
unsigned heartbeatRate -- The frequency of heartbeat callbacks, in milliseconds.
Maestro_String groupName
-- The name of the group to be joined.
Maestro_String protocol -- The Ensemble protocol stack to be used.
Protocols in the stack are separated by colons, for example:
"Top:Heal:Switch:Leave:Inter:Intra:Elect:Merge:Sync:Suspect:Top_appl:Pt2pt:Frag:Stable:Mnak:Bottom", The use of raw protocol stacks is
discouraged. Instead, protocol properties should be specified
(see below).
Maestro_String properties -- Group protocol properties required
by the application.
The following properties are supported:
Agree -- Agreed (safe) delivery
Gmp -- Group-membership properties
Sync -- View synchronization
Total -- Totally ordered messages
Heal -- Partition healing
Switch -- Protocol switching
Auth -- Authentication
Causal -- Causally ordered broadcasts
Subcast -- Subcast pt2pt messages
Frag -- Fragmentation-reassembly
Scale -- Scalability
Suspect -- Failure detection
Flow -- Flow control
Migrate -- Process migration
Privacy -- Encryption of application data
Rekey -- Support for rekeying the group
Primary -- Primary partition detection
Properties are separated by colons, for example:
"Gmp:Sync:Heal:Switch:Suspect:Flow:Primary".
Maestro_String params -- Parameters for the group.
Maestro_String transports -- The list of available transport protocols. Protocols in the
list are separated by colons, for example: "UDP:ATM".
Maestro_ErrorHandler *errorHandler
-- The error handler to be used by the group-member object.
If errorHandlerset field is set to NULL, the default error handler is used.
int groupdFlag -- The flag should be set if
Ensemble Membership Service
is to be used.
int viewMsgFlag -- The flag should be set if view messages
are required, in which case the
grpMemb_ViewMsg_Callback
is invoked at the coordinator allowing it to distribute additional
data to view members along with the view structure.
char **argv -- Command-line parameters.
A Maestro_GroupMember object does not automatically join the group
at the initialization time. To join the group, the join() method
should be invoked:
void join();
A call to join() is eventually followed by an invocation of the
grpMemb_AcceptedView_Callback() by Ensemble.
A call to join() returns only
after the grpMemb_AcceptedView_Callback() method
has been invoked by Ensemble.
A member object cannot join multiple groups and cannot join a group more than once.
A member can leave the group by calling the leave() method:
void leave():
After the member completes leaving the group, the
grpMemb_Exit_Callback() method is invoked by Ensemble.
grpMemb_Exit_Callback() is defined in the Maestro_GroupMember class
as a no-op function (to be overloaded in subclasses of
Maestro_GroupMember):
void grpMemb_Exit_Callback() {}
It is guaranteed that grpMemb_Exit_Callback() will be the last callback
received by a group member.
After a group member leaves the group, it can rejoin by calling the
join() method again.
A Maestro_GroupMember object leaves the
group automatically at the time of deallocation, by calling the
leave() method from the destructor.
After a Maestro_GroupMember object has
joined a group, it can
send point-to-point and multicast messages to it.
The Maestro_GroupMember class provides the following downcall methods
for sending point-to-point (send) and multicast (cast) messages:
void send(Maestro_EndpID &dest,
Maestro_Message &msg);
void send(Maestro_EndpID &dest,
Maestro_Message &msg,
Maestro_MsgSendView &sendView);
void send(Maestro_EndpID &dest,
Maestro_Message &msg,
Maestro_MsgSendView &sendView,
Maestro_GrpMemb_MsgOptions &msgOps);
void cast(Maestro_Message &msg);
void cast(Maestro_Message &msg,
Maestro_MsgSendView &sendView);
void cast(Maestro_Message &msg,
Maestro_MsgSendView &sendView,
Maestro_GrpMemb_MsgOptions &msgOps);
The dest argument in send methods specifies the
endpoint ID of the
destination.
The msg argument contains the
message to be sent.
sendView is an "out" argument. It specifies the view in which the message
will be delivered to its destination(s). The Maestro_MsgSendView type is
defined as follows:
enum Maestro_MsgSendView {
MAESTRO_MSG_SEND_UNSPECIFIED_VIEW, // The view of msg delivery is unspecified
MAESTRO_MSG_SEND_CURRENT_VIEW, // The msg will be delivered in the current view
MAESTRO_MSG_SEND_NEXT_VIEW // The msg will be delivered in the next view
};
If a Maestro_GroupMember object is created over a protocol stack that
provides the
Virtual Synchrony
property, all messages are delivered in the same
view in which they are sent. However, if a send or a cast method is
invoked during a view change, the message will be sent (and hence delivered) in the future
view. The sendView argument can then be used to determine the view in which a
given message will be sent and delivered.
The msgOps argument has no effect
(the Maestro_GrpMemb_MsgOptions structure is empty).
It is intended to be overloaded in subclasses of the Maestro_GroupMember class.
When a point-to-point or a multicast message is received by a group member,
the grpMemb_ReceiveSend_Callback (respectively,
grpMemb_ReceiveCast_Callback)
method is invoked by Ensemble. The message callbacks are defined in
Maestro_GroupMember as follows:
void grpMemb_ReceiveCast_Callback(Maestro_EndpID &origin,
Maestro_Message &msg) {}
void grpMemb_ReceiveSend_Callback(Maestro_EndpID &origin,
Maestro_Message &msg) {}
By default, no action is taken when a message is received. Message callbacks
are intended to be overloaded in subclasses of the Maestro_GroupMember class
to implement application-specific functionality.
When member objects join or leave the group (or become unavailable),
Ensemble starts the view-change protocol to bring the view of
group members up to date.
In the first phase of the view-change
protocol, the group is blocked, so that members
cannot send any new messages.
Ensemble invokes the grpMemb_Block_Callback() method of a
Maestro_GroupMember object to notify it
that the group is about to be blocked.
The application can send messages for the current view from within
grpMemb_Block_Callback().
However, after the callback returns, all subsequent message downcalls will be delayed by
Ensemble until the next view is installed.
The grpMemb_Block_Callback() method is defined in
Maestro_GroupMember as follows:
void grpMemb_Block_Callback() {}
By default, no action is taken when a group is being blocked.
However, grpMemb_Block_Callback() can be overloaded in
subclasses of the Maestro_GroupMember class to implement
application-specific behavior.
When the new view is ready to be installed, the
grpMemb_ViewMsg_Callback() method is invoked at the coordinator.
The grpMemb_ViewMsg_Callback() function is defined in
Maestro_GroupMember as follows:
void grpMemb_ViewMsg_Callback(Maestro_GrpMemb_ViewData &viewData,
Maestro_Message &viewMsg) {}
viewData describes the new view.
viewMsg is an "out" parameter. The coordinator can include
some data with the view message (by writing to viewMsg),
which will be delivered to group members
along with the new view.
By default, the grpMemb_ViewMsg_Callback
is implemented in
Maestro_GroupMember as a no-op function. However,
it can be overloaded in subclasses of the Maestro_GroupMember class
to implement application-specific behavior.
After the call to grpMemb_ViewMsg_Callback returns,
the new view is installed at group members included in it.
When a group member receives a new view,
the grpMemb_AcceptedView_Callback
method is invoked. Upon returning from the call to
grpMemb_AcceptedView_Callback, Ensemble unblocks the group, so that
messages can be sent again. The grpMemb_AcceptedView_Callback
method is defined in Maestro_GroupMember as follows:
void grpMemb_AcceptedView_Callback(Maestro_GrpMemb_ViewData &viewData,
Maestro_Message &msg) {}
viewData describes the new view. The Maestro_GrpMemb_ViewData
structure has the following fields:
Maestro_String version -- The version of Ensemble in use.
Maestro_String groupName -- Name of the group.
Maestro_EndpID myEndpID -- The endpoint ID of the member.
Maestro_EndpList members --
The list of members in the new view.
Maestro_EndpList newMembers -- The list of members added to the member's old view.
Maestro_EndpList departedMembers -- The list of members removed from the member's
old view.
int nmembers -- The size of the new view.
int myRank -- The member's rank in the new view.
int myOldRank -- The member's rank in its old view.
Maestro_EndpID coordinator -- The endpoint ID of the coordinator of the new view.
Maestro_EndpID oldCoordinator -- The endpoint ID of the coordinator of the
member's old view.
Maestro_String protocol -- The protocol stack used by Ensemble in the new view.
int groupdFlag -- The flag is set if group daemon
(Ensemble Membership Service) is in use.
Maestro_ViewID viewID -- The ID of the view.
Maestro_String params -- Group parameters.
int primaryFlag -- The flag is set if the view is primary.
msg contains the view message filled in by the coordinator.
Maestro supports external failure detectors via the suspect() method,
which is defined in Maestro_GroupMember as follows:
void suspect(Maestro_EndpList &suspects);
The suspects parameter contains the list of members which are believed to be
faulty and should be removed from the view.
The protocol stack of a view can be changed with a call to the changeProtocol
method of Maestro_GroupMembers. The changeProtocol function is
declared as follows:
void changeProtocol(Maestro_String &protocol);
The protocol parameter specifies the new stack. Protocols in a stack are
separated by colons, such as in
"Top:Heal:Switch:Leave:Inter:Intra:Elect:Merge:Sync:Suspect:Top_appl:Pt2pt:Frag:Stable:Mnak:Bottom".
Only the coordinator of a view can request a replacement of a protocol stack.
Calls to changeProtocol made by non-coordinator view members are ignored.
The coordinator of a view can request a view change with a requestNewView() method,
which is declared as follows:
void requestNewView();
The endpoint ID of a group member object can be obtained (after the member has
joined the group) with the myEndpID() method, which is declared as follows:
void myEndpID(Maestro_EndpID &eid);
The Maestro_GroupMember class defines the grpMemb_Heartbeat_Callback()
function, which is periodically invoked by Ensemble as a callback.
grpMemb_Heartbeat_Callback() is defined as a no-op function as follows:
void grpMemb_Heartbeat_Callback(unsigned time) {}
It is intended that grpMemb_Heartbeat_Callback() will be overloaded in subclasses
of Maestro_GroupMember to implement application-specific functionality.
send mail to
alexey@cs.cornell.edu