
Variables
---------

- groups: a hashtable of all groups in the system. This is essentially the replicated
  information shared between all GroupServices.
  Key: group name, values: list of GroupInfos. A GroupInfo object
  contains the name of the group, its address (McastAddress or
  MultipleAddress), all members (hashtable of group names and list of
  UnicastAddresses)

- A list of groups with local members is maintained by the
  Dispatcher





Group Membersip Service (GMS)
-----------------------------

The task of GMS is to keep track of the groups that exist and their
members. It is a distributed directory system, cooperating with other
GMSs to maintain a replicated consistent state of all groups. Group
members may choose to maintain their group membership themselves
(e.g. a list of all members of their group), in which case they would
not need to consult the GMS to query group membership every time.

The GMS is represented by a GMS server object which (currently) is in
the client's address space. (Later, RMI access will be offered).

The GMS uses a Transport object (*no* Dispatcher !) to send/receive
messages to/from other GMSs. Thus, it doesn't have to take care of
fragementation/defragmentation, retransmission, ordering etc. The
Transport must provide *total ordering* (yet to be implemented),
frag/defrag, NAK and UDP.

A GMS has a Transport connected underneath it which has 2 sockets: an
incoming (multicast) socket for receiving messages) and an outgoing
socket (for sending messages).



Information Maintained by a GMS
-------------------------------

Each GMS has 2 pieces of data it maintains:

 a.) a list of all groups it knows about and their members and
 b.) a list of all other GMSs it cooperates with

The list of groups is a hashtable with group names as keys and a list
of UnicastAddresses for each group as values. The list of other GMSs
is represented by either an McastAddress (multicast address) to which
all GMSs listen, or a MultipleAddress. The latter is a list of
UnicastAddresses (host:port) of each GMS. Usually UnicastAddresses
will be used, as multicast broadcasting is (yet) problematic in wide
area networks (routers may drop mcast packets)



Initial Bootstrapping:
----------------------

- A GMS object is created

- A multicast message is sent to address GMS_MCAST_ADDRESS
  (default:11000 if not defined). 

  If no response is received (e.g. no servers running, or network does
  not support mcast), a unicast message is sent to all known bootstrap
  servers. The list of these servers is defined in GMS_INITIAL_SERVERS
  and looks like "janet:10000;gulag:3564;mojave:5555". The message sent
  is a request for the list of all GMS servers

- In case there is no response (or no bootstrap servers have been
  defined), the newly instantiated GMS assumes that there are
  currently no other GMSs started. The newly created object then joins
  the multicast group at address GMS_MCAST_ADDRESS

- If one or multiple responses are received, the newly created GMS
  server joins the list of GMS servers (represented by a Group
  object). This is done by either joining the IP multicast group, or
  by adding itself to the MultipleAddress group

- A GMS_SERVER_JOINED message is sent to all member of the GMS group,
  either via multicast or via multiple unicasts. Each GMS updates its
  view upon reception of the GMS_SERVER_JOINED message

- The newly created GMS picks one or two GMSs from the received list
  and sends them a GetState message. It repeats this as long as a
  response is received (taking turns in choosing members). When a
  response is received the local database is initialized

- The idea is that a server containing *only* a GMS object will always
  be running on the mentioned addresses/ports



Group Membership Changes
------------------------

- When a new object joins a group, or leaves it, or when new groups
  are created or old ones deleted, all GMS are notified using an
  atomic, totally ordered broadcast to the GMS group address. This
  means that either all or zero GMSs receive it, and that all GMSs
  receive the message in the same order (these properties are taken
  care of in the protocol stack of the Transport)

- The protocol stack is set up for all GMSs identically


Heartbeat
---------

- The first GMS in a group is its *coordinator* and sends out a
  heartbeat message periodically to all members

- When a member doesn't respond within a certain timeframe, it is
  suspected dead and removed from the GMS group

- When such a GMS later rejoins (e.g. after a network partition), a
  merge algorithm has to take care that the correct information will
  be in the database (maybe use primary partitions)

- If the coordinator dies itself, this has to be detected and voting
  has to take place to elect a new coordinator. We should use the
  Voting/Election service of JavaGroups later to do this.















	


Create a new group (GroupService.CreateGroup)
---------------------------------------------

- If the group is already present in the Dispatcher's local groups:
  throw GroupAlreadyExists exception

- Choose a group address (either MultipleAddress or McastAddress)

- Create a new entry in groups. Key is group name, value is GroupInfo
  (containing group name and address).

- Multicast a CREATE_GROUP message to all GroupServices (send the GroupInfo
  object)

- Return a Group object with the correct name and address to caller (taken from
  GroupInfo). Add this Group object to the Dispatcher's local groups list





Find a group (GroupService.FindGroup)
-------------------------------------

- Find the group in the local groups

- Return Group object to caller (if found)





Add a new member (Group.Join)
-----------------------------

- Find group in Dispatcher's list of local groups

- If null: throw GroupNotExists exception

- Assign UnicastAddress to member (host:transport_id:object_id:group_name). Id
  is assigned by Dispatcher (GetNewObjectId).

- Add UnicastAddress as new value to GroupInfo (members). Throw exception
  AlreadyMember if already present in group

- Add new member to Dispatcher's local groups list

- Multicast a MemberJoined message to all GroupServices

- If Group's address is McastAddress: join multicast address. This is done using
  UDP.JoinMulticastGroup(): a JOIN to the same group will only increase a
  counter, but not start a new thread





Remove a member (Group.Leave)
-----------------------------

- Find group in Dispatcher's local groups list

- Remove entry from Dispatcher's list

- Remove entry from GMS list

- Propagate change to all GMSs

- If group's address is a McastAddress: leave multicast group. This is done
  using UDP.LeaveMulticastGroup. The last member to leave will destroy the
  thread listening at the multicast socket






Send a message to all members of the group (Group.Mcast)
--------------------------------------------------------

- This method will set the destination address of the Message to be the group
  address (McastAddress or MultipleAddress)

- Call Dispatcher.Send(msg). Dispatcher sets source address

- Dispatcher uses the GMS to retrieve all members of the group (either
  McastAddress or MultipleAddress)

- This will either send a multicast message (McastAddress) or several unicast
  messages (MultipleAddress) to all members of the group . Note that we
  effectively send only one message to each Dispatcher. The Dispatcher itself
  will then distribute it to all members of that group





Send a message to some members of the group (Group.Send)
--------------------------------------------------------

- Prepare a Message object and set destination (MultipleAddress). This is done
  by adding several UnicastAddresses to the MultipleAddress object. Ths list of
  all members can be retrieved by the client from the GMS

- Call Dispatcher.Send(msg). Dispatcher sets source address

- A Unicast message will be sent to all the Dispatchers whose host:port part
  matches the one of each UnicastAddress in MultipleAddress. Each Dispatcher
  will then dispatch the message to the object that matches the object_id of the
  UnicastMessage





Transfer state to a new GroupService (GroupService.GetState)
------------------------------------------------------------


- Dump the 'groups' member variable to a byte stream and send it to the source
  address of the message
