Tron Architecture

(Back to Main)

Contents


General Overview

The tron architecture consists of a series of the protocol stack itself, a series of interfaces, and the actual implementation of some standard layers. The main idea behind the architecture is that the programmer creates a cProtocolStack object and a set of layers that implement the cLayer interface that the protcol stack is to use. The programmer then sets up the protocol stack using cProtocolStack::AddLayer() for each of the layers and then registers a series of desired callbacks for observing view changes, message delivery, and asynchronous error notification. All sends are performed though the protocol stack, and recieves occur through the cDeliver callback. The layers themselves and their composition in the protocol stack provide the desired networking properties and guarantees. Messages are sent to and received from a generic cEndpoint type, which provides a nice abstraction for endpoints and enables the use of different kinds of network layers (for example, IP, simulation, VI, etc) even within the same protocol stack, if necessary.

The simplest way to learn how to program using that standard tron facilities is to look at the example applications. The easiest way to gain an understanding of how tron works, along with how to create new layers is to read these pages and look at the source code itself. Almost every function located in a .cpp file has an elaborate function header which explains what the inputs are and what the function does. It should be rather self-explanatory. These web pages will give a brief overview and put things in perspective, but the source code should be consulted for more in-depth research.

Protocol Stack

The protocol stack composes a series of layers and provides an interface for setting up the layer stack, sending, event handling, and scheduling. The event handling and scheduling of the protocol stack make it easier to create layers that do not need their own threads and require little, if any, locking. The protocol stack has two main modes of operation. If the cProtocolStack::Start() function is called, a new thread is spawned and the protocol stack takes care of scheduling its layers internally. Otherwise, an application can specifically call the cProtocolStack::Schedule() method to provide scheduling for the protocol stack and all of its layers. With either of these methods, each of the layers will be given a regular time slice in which to perform any necessary activities. In addition, the protocol stack allows Event objects to be registered with an associated callback that occurs in the same thread as the protocol stack's schedule. In this manner, a layer can be guaranteed that only a single callback (for any layer) will be active at a time, along with the guarantee that no layer schedules and callbacks occur simultaneously. This often removes the need for any locking within the layer at all, although a single Send operation can occur simultaneously since it is initiated by an application thread. For a more clear example of this, see the network layer description.

Protocol Stack

The protocol stack has the following interface:
Method Description
Send Two different versions- one which takes a preallocated message buffer, and another which takes a byte buffer and size. Both send to all of the members of the given group.
AllocateBuffer Allows the application to preallocate message buffers, allowing for zero copy sends.
AddLayer Adds a layer to the protocol stack from the bottom up. Must be done before Start() or Schedule() is called.
RemoveLayer Removes the top-most layer that is left on the protocol stack.
RegisterDeliverCallback
UnregisterDeliverCallback
Used to set up cDeliver callback.
RegisterViewCallback
UnregisterViewCallback
Used to set up the cView callback.
RegisterErrorCallback
UnregisterErrorCallback
Used to set up cErrorCallback, the asynchronous error handler.
Schedule Allows the application to schedule the protocol stack by hand, so that no new threads are created. This is recommended for advanced users only.
Start Starts the protocol stack's scheduling thread.
Stop Stops the protocol stack's scheduling thread.
Cleanup Cleans up the protocol stack. This should be called before the stack is deleted.
AddEvent Adds the event and its associated callback to the protocol stack's event loop.
RemoveEvent Removes the event and its associated callback from the protocol stack's event loop.
GetMTS Gets the maximum transfer size of a packet. Derived from the information provided by the stack's layers.
TimeStep Allows manual control of the protocol stack's time facility. Once this is called, time must be stepped from then on using this function.
GetTime Gets the current time value (in milliseconds since the system was started).
GetLocalAddress Gets a group containing all of the local address for this instance of the stack.
AddLocalAddress Adds a local endpoint address to this protocol stack (should be called at least by the bottom most layer).

MsgBuffer and Buffer Manager

The protocl stack uses a buffer manager to allocate all of its message buffers. It uses the cMsgBuffer object to hold msg data. When the protocol stack is composed, the cLayerInfo class gathers information about the MTS and header sizes required by each layer. When a new message buffer is allocated by the protocol stack (through the buffer manager), the total sum of the header sizes of all of the layers are reserved within the buffer along with the expected message size. In this manner, when the cMsgBuffer is passed down through the layers, each layer can tack its own header on the front without having to realloc or copy the payload. If the application preallocates its buffers, then zero copy send and receive can be accomplished. The cMsgBuffer uses its own reference counting so that on receive, if a layer wants to keep a copy of a cMsgBuffer it merely has to add a reference. The cMsgBuffer provides the following set of methods:
Method Description
AddRef Increases the reference count on this buffer.
Release Decreases the reference count on this buffer. If the reference count is equal to or less than zero then the buffer is freed.
Init Initializes the buffer, optionally reserving the given amount of space for headers.
AddHeader Copies the given header onto the front of the buffer, decreasing the amt of free header space.
RemoveHeader Removes the given amount of header from the front of the buffer, increasing the amt of free header space.
SetPayload Copies the given byte buffer into the msg buffer and sets the payload size.
GetPayload Gets a pointer to the byte buffer that is the potential payload for this buffer, along with its size.
SetPayloadSize
GetPayloadSize
Sets and gets the size of the payload, respectively.
Clone Allocates and creates an exact clone of this message buffer (including amt of header reserve space).

Groups

A cGroup is a group of endpoints. A cGroup object is used as the destination for a message. The cGroup class provides the following methods:
Method Description
AddEndpoint Adds the given endpoint to the group.
RemoveEndpoint Removes the given endpoint from the group.
IsMember Returns true if the given endpoint is a member of the group.
GetNumElements Returns the number of elements in the group.
GetSize Returns the total serialization size of all of the members.
Print Prints a text representation of all of the members of the group to the given ostream.
Clear Removes all of the endpoints in the group.
GetIterator Returns an iterator of all of the groups elements. Note:The should be locked before this is called and while the iterator is in use if necessary.
Lock
Unlock
Used to lock and unlock the group for synchronization purposes.

Endpoints

The endpoint interface provides the notion of an abstract endpoint object as a destination for a message. It implements the cHashObject interface and also has the following methods:
Method Description
Serialize
Deserialize
Used to transfer representation of endpoint to and from byte buffers.
GetSize Returns the amount of bytes that a serialized version of the endpoint would take.
Print Pushes a text-based representation of the endpoint on the given ostream.
IsSameSubnet Returns true if the endpoints are on the same subnet (if there is a notion of same subnet!).
SubnetLessThan Used to provide ordering on subnets (if there is a notion of same subnet!).
AllocCopy Creates a copy of the given endpoint.
GetType Returns an integer representation of the endpoint type.

Endpoints are created by the cEndpointFactory object using static methods. The cEndpointFactory is the only way in which to correctly create endpoints of a given type, as well as to deserialize a byte-representation of an endpoint into an actual endpoint. Tron currently supports cIPEndoint and cSimEndpoint endpoint types, which are IP and simulation endpoints respectively. If new endpoint types are added, the cEndpointFactory class must be modified so that they can be created.

Interfaces

cObject

The cObject interface provides a common base class that all Tron classes can derive from.

cHashObject

The cHashObject interface provides common methods for an object so that it can be used in comparison, etc:
Method Description
HashCode A hashcode for this object. Two different objects may have the same hashcode.
Equals Returns true if endpoint is the same as the given endpoint.
LessThan Used to order endpoints of the same type.
AddRef Adds a reference so that the object won't be garbage collected until release is called. Increases the reference count by one.
Release Decreases the reference count by one. Garbage collects the object if the reference count is less than or equal to zero. (Note:Objects start with a reference count of zero).

cLayer

The cLayer interface provides the glue that holds layers together in the protocol stack. It consists of the following members:
Method Description
Init Initializes the layer with the given cParam.
Cleanup Cleans up the layer. The layer should be able to handle another Init() call after cleanup.
Send Sends the given message. Will pass the msg down the stack if necessary.
RegisterDeliverCallback
UnregisterDeliverCallback
Used to set up cDeliver callback.
RegisterViewCallback
UnregisterViewCallback
Used to set up the cView callback.
RegisterErrorCallback
UnregisterErrorCallback
Used to set up cErrorCallback, the asynchronous error handler.
Schedule Called on the layer by the protocol stack. This will be called regularly and gives the layer a reasonable, but small, timeslice in which to do some extra work.
GetLayerInfo Returns the layer info (usually to be propagated up the stack).

cParam

This provides an interface for the parameter set to be used on a layer in its cLayer::Init() method. cParam's single method, cParam::ResetToDefault(), should reset all of the parameters to their default values.

cDeliver

This provides the callback for message delivery. It will be called whenever there is a message ready for delivery. An observer registers for a deliver callback either through the top of the protocol stack or through a layer directly. A layer will usually register itself for delivery with the layer below.

cView

This provides the callback for view membership information. An observer registers for a view callback either through the top of the protocol stack or through a layer directly.

cErrorCallback

This provides the callback for an asynchronous error handler. An observer registers for an error callback either through the top of the protocol stack or through a layer directly.

cCallback

This provides a generic callback method. Currently it is used so that an event can be added to the protocol stack and the given callback will be called when the event is triggered.

Pre-Built Layers

The pre-built layers should provide excellent examples on how to make layers for the tron system. The following are the layers included in the distribution along with explanations of the working of each:

  • Network Layer - An IP based network layer that supports unicast, broadcast, and multicast.
  • SimNet Layer - A simulated network layer that supports unicast, broadcast, and multicast.
  • Debug Layer - A layer used for debugging that can print and/or probabilistically drop incoming and outgoing messages.
  • Pbcast Layer - A probabilistic broadcast layer that includes a gossip-style failure detection service.

    Utilitiy Classes

    The util classes are a set of useful classes that are used by the architecture but not specific to any particular aspect. Examples include FIFO queus, stacks, CoolHash (a generic hash function...check it out). One class of note is the gError set of global functions. They are used for generic information and error reporting:
    Method Description
    gSetErrorStream Sets the error output stream to be the given ostream reference.
    gError Prints the given error string, along with the __LINE__ and __FILE__ information.
    gAbort Prints the given error string, along with the __LINE__ and __FILE__ info, and aborts the program.
    gInfo Prints the given information string, along with the __LINE__ and __FILE__ info.
    gSetPrintInfo if true, allows info msgs to be printed. if false, suppresses info messages.

    Website created by: Ted Bonkenburg (
    tjb13@cornell.edu )