STATE_TRANSFER

The STATE_TRANSFER layer is responsible to reply to GET_STATE events sent from the channel and return the state in a GET_STATE_OK event. It will interact with other group members to get the group state. The protocol interaction is shown in fig. 7.2.

  

Figure 7.2: State transfer protocol interaction

A state transfer is initiated by multicasting a MAKE_COPY message to all members (including the initiating member). When the initiating member receives this message, it starts queueing subsequent messages. All other member, upon reception of the message, make a copy of the current state.

In the second phase, the initiator selects one member (possibly the oldest, or the coordinator) and sends a RETURN_STATE message to it. The receiver returns its copy of the state that was created upon reception of MAKE_COPY. The initiator sets its state and then replays all queued messages before accepting regular messages again. (Message will not be queued after replaying the queued messages).

The above scheme requires that messages are received by all members in the same order (total ordering). Thus, message MAKE_COPY is received by all members at the same logical time, allowing the initiator of a state transfer to queue all subsequent messages until reception of the state.

The events exchanged between peer STATE_TRANSFER layers and their JChannel instances are shown in fig. 7.3.

  

Figure 7.3: Events between STATE_TRANSFER layers and JChannel

The protocol on the left side represents the initiator of a state transfer, the one on the right the participant (e.g. an existing group member). The protocol is started when a GET_STATE event is received. It causes 2 messages to be sent to a) all members and b) one selected member. The first message is MAKE_COPY (1) and it is sent to all members. Upon reception, the initiator reacts differently from the participants: it starts queuing all subsequent messages. Participants try to retrieve the state from the application. This is done by sending up a GET_APPLSTATE event and waiting for a GET_APPLSTATE_OK event to be returned. If the channel does not support state transfer events (corresponding option is not set), then the GET_APPLSTATE_OK response will contain a null state. Otherwise it will be a copy of the application's current state^7.4. The copy (if valid) is then stored in the STATE_TRANSFER layer for subsequent retrieval by the initiator.

The second message is a RETURN_STATE message (2) sent only to one selected member^7.5. This is currently the oldest member (usually the coordinator). Note that the message is never sent to the initiator itself. The receiver just returns the previously saved copy of the state.

When the initiator receives the copy from the selected member, it does two things: first it sets its own state from the data received. Then, it blocks all message reception, replays the message queue (to which all messages received after MAKE_COPY were queued), disables queueing and unblocks message reception. From now on, messages received will not be queued, but passed up to the channel and on to the application where they will be applied to the application's current state. Queueing ensures that any message sent after MAKE_COPY and before receiving the state will not be lost, but properly applied to construct a valid state.

When the state was received, a GET_STATE_OK event is passed up the stack. When processed by JChannel, a STATE_RECEIVED event is created and inserted into JChanel's event queue. After this, a caller blocked on the GetState method will return (true or false). One of the next calls to Receive will return a SetStateEvent object containing the state, which can then be set by the application.

The implementation of STATE_TRANSFER relies on total ordering of messages multicast to all group members: it is based on the assumption that all members receive the MAKE_COPY message at the same logical time. If this was not the case, then the different participants might save different versions of the state. For example if participant Q received messages m1, MAKE_COPY and m2, and participant R received m1, m2 and MAKE_COPY then, the state saved by Q would include m1, but R's state would include m1 and m2.