• Introduction
• Description

2.    Design and Specification

• Design Philosophy
• The Definition of "group"
• Abstract class Manager
• Architecture

3.    The Implementation

• The "Persistent" Control Layer

• Inter-group Membership Component: GroupsManager
• Group List Control Message

• The "Volatile" Control Layer

• Intra-group Membership Component: MembershipManager
• Membership Control Message
• Media Control Message
• The Data Layer
  
  • VideoManager
  • AudioManager
  • TextManager

• The Graphical User Interface Layer
  
  • ChatPanel
  • GroupsPanel
  • MembershipPanel
  • GUIManager

4.    Limitations and Future Extensions

7.    Download

JavaGroupsVC is an open source program designed to provide a multi-platform video conferencing solution. The motivation behind this project comes from the lack of platform independent solution for videoconference. Proprietary hardware and platform specific video formats requires video conferencing application to be tailored and rewritten for each computing platform.

The emergence of Java technology provides tools to create platform independent applications. However, it is the general belief among technologists is that Java technology is not suitable for computational intensive tasks, such as encoding and decoding video or audio, which are required in a video conferencing application. The ultimate goal of this project is to explore the feasibility of creating video conferencing application using Java’s latest multimedia technology, Java Media Framework, over JavaGroups, a Java-based reliable group communication toolkit developed at Cornell University.

In this release, JavaGroupsVC supports three data channels for its operations: a video channel for video broadcast, an audio channel for audio broadcast and a text channel for chat session. A user who joins a group will be able to share video and audio with all members of the group. Each user will be asked to choose a username and a cartoon icon to represent him.

Usernames and user icons are unique only within a particular group. If a particular username or user icon is already being used by another user in the group, the newcomer will be asked to change his. All the members in the group will be displayed using Group members can choose to send chat message to all members or to a specific individual. All the groups available for joining and the group members currently in the groups will be displayed in an expandable tree structure, so that users can browse for groups that interest them before joining any group.

Users will be able to join and rejoin groups for unlimited number of times. However, a user can only be in one group at any given time. A user will have to leave his current group, if any, before he can join another group.

At the highest level, this project is designed to be an extension package to the standard JavaGroups source distribution. It is not part of the JavaGroups standard distribution. In other words, end users that are not interested in the videoconference features should not have to download the package. On the other hand, for those end users that are interested in trying out the videoconference features, they should be able to just download the extension package, put it in the JavaGroups root directory and run the application together with the standard JavaGroups distribution.

At the lower level, this project is organized into individual components by their functions. And each of the components has its own handler, which is called "manager" that handles the manipulation and reproduction of each type of data.

There are two types of "manager" in JavaGroupsVC. The first type is a "channel manager". For example, for this release, there are three data components that are visible to the users. They are the video component, the audio component and the text component. Each of these components has its "channel manager" which are VideoManager, AudioManager, and TextManager respectively. Each "channel manager" takes care of its channel connection, its data type specific operation and its communication with the Graphical Use Interface. The motivation behind this type of code organization is that it will be easier for potential developers to add new features in the future by simply create a new handler for the new component.

The second type of "manager" in JavaGroupsVC is a "data structure manager". Each data structure in JavaGroupsVC has its "data structure manager" to handles any operation that alters the data it stores. For instance, the data structure class to store all the user information is called UserInfo. Its data structure manager is called UserInfoManager that handles the operations that alter the user information database such as adding new user, deleting current user, and changing username or user icon

At the lowest level, potential developers might find some variable names and class names are longer than usual. The reason behind long names is to make the code more readable without making use of excessive comments.

In this release, user can only be in one group at any given time. Therefore, user needs to leave the current group before joining a new group. As in JavaGroups, a group is defined by a character string, which is the group name. Note that, in this release, a "group" consists of three data channels (video, audio and text channels) and two control channels (membership channel and groups channel). When a user leaves a group, it disconnects from all three of the data channels and one of the two control channels (in this case, the membership channel). When a user joins a group, it connects to all data channels and the membership channel of the group.

JavaGroupsVC is designed using a layered architecture. There are three major layers: Graphical User Interface Layer (GUI Layer), Control Layer (includes both the "persistent" and "volatile" control layers) and Data Layer.

Figure 1: The layered architecture of JavaGroupsVC.

In this release, there are six major components in the application, four of which are visible to the end users. The four visible components are the GUI component in the GUI Layer, and the video component, the audio component and the text component in the Data Layer. The other two components that are transparent to the end users are the intra-group membership component and the inter-group membership component in the Control Layer.

The Control Layer consists of two sub-layers: the "persistent" control layer and the "volatile" control layer.

Figure 2: The control layer of JavaGroupsVC

In the "persistent" control layer, there can be only one channel manager at a time and the channel manager does not exit until the application itself exits. This layer is used to coordinate and exchange information between the groups.

In the "volatile" control layer, there can be more than one "channel manager" at a time (however, in this release, only one channel manager will be active at a time in this layer as well, see the "limitations" section at the end of this report). New channel manager can be instantiated and old channel managers can exit as needed. This layer is used to coordinate and exchange information between members in a group.

The inter-group membership handler in the "persistent" Control Layer is GroupsManager. It runs the inter-group membership protocol to ensure that it has the most current information about all the groups as well as each of their group members on the server. It listens for control messages in the default channel called grouplist control messages.

All the grouplist control messages are subclass of Message class in JavaGroups. There are five different grouplist control messages in this layer:

1. BcastNewGroupMsg: broadcast message by the creator of a new group to notify all users in different groups to about the creation of a new group.
2. GroupListRequestMsg: point to point message from a new member to the coordinator of all groups to request a current group list.
3. GroupListReplyMsg: point to point reply message from the coordinator of all groups that contains the current group list to the requestor of group list.
4. MemberJoinGroupMsg:: broadcast message by the coordinator of a group to notify all users in different groups about a new member who joins the group.
5. MemberLeaveGroupMsg: broadcast message by the coordinator of a group to notify all users in different groups about an old member who leaves the group.

Figure 3: The "persistent" control layer

In this release, the default channel is called "DefaultGroup@server". All clients will join this default channel at start-up. GroupsManager is not group specific. Therefore, each client will have only one GroupsManager during its entire video conference session no matter how often it joins or leaves various groups. GroupsManager will exit only when the application itself exits.

The intra-group membership handler in the "volatile" Control Layer is MembershipManager. It runs the intra-group membership protocol to ensure that it has the most current information about all the group members. It listens for control messages in a dedicated channel using the naming convention of <groupname>@control. For example, in a group called "movie", the dedicated channel for its MembershipManager will be called movie@control. MembershipManager is group specific. If a user joins a new group, a new MembershipManager will be instantiated to handle the membership information in the new group and the old MembershipManager will be garbage collected.

All the control messages in this layer are subclass of Message class in JavaGroups. There are two types of control message in this layer. The first type of control message in this layer is called membership control message that deals with membership protocol. There are eleven different control messages in this category:

1. BcastUserInfoMsg: broadcast message from a new member to publish information about itself to all members in the group
2. AudioChannelBcastMsg: broadcast message from a new member to publish its audio channel address to all members in the group
3. VideoChannelBcastMsg: broadcast message from a new member to publish its video channel address to all members in the group
4. TextChannelBcastMsg: : broadcast message from a new member to publish its text channel address to all members in the group
5. InitialMembersRequestMsg: point to point message from a new member to request the current member list from the coordinator of the group
6. InitialMembersReplyMsg: point to point reply message that contains the current member list from the group coordinator to the requestor
7. IconInquiryMsg: point to point message from a new member to request the list of available icons from the group coordinator
8. IconInquiryReplyMsg: point to point reply message that contains the list of available icons from the group coordinator to the requestor
9. JoinRequestMsg: point to point message from a new member to the group coordinator to request permission to join the group
10. JoinRequestGrantedMsg: point to point message from the group coordinator to the new member that grants the permission to join the group
11. JoinRequestDeniedMsg: point to point message from the group coordinator to the new member that denies the permission to join the group

Figure 4: The membership control message in the "volatile" control layer

The second type of control message in this layer is called media control message that deals with media transmission coordination protocol. There are thirteen different control messages in this category:

1. StartVideoConferenceRequestMsg: point to point message from a member (requestor) to request a videoconference session with another member (receiver).
2. StartVideoConferenceRequestGrantedMsg: point to point message from a member (receiver) who agrees to hold a videoconference with another member (requestor)
3. StartVideoConferenceRequestDeniedMsg: point to point message from a member (receiver) who refuses to hold a video conference with another member (requestor)
4. StartAudioBcastMsg: broadcast message from group coordinator about the beginning of a audio broadcast session to all members in the group
5. EndAudioBcastMsg: broadcast message from group coordinator about the end of an audio broadcast session to all members in the group
6. StartAudioRequestMsg: point to point message from a member (requestor) to the group coordinator to request the permission to broadcast audio in the group
7. StartAudioRequestGrantedMsg: point to point message from the group coordinator to the requestor to grant the permission to broadcast audio in the group
8. StartAudioRequestDeniedMsg: point to point message from the group coordinator to the requestor to denied the permission to broadcast audio in the group
9. StartVideoBcastMsg: broadcast message from group coordinator about the beginning of a video broadcast session to all members in the group
10. EndVideoBcastMsg: broadcast message from group coordinator about the end of a video broadcast session to all members in the group
11. StartVideoBcastRequestMsg: : point to point message from a member (requestor) to the group coordinator to request the permission to broadcast video in the group
12. StartVideoBcastRequestGrantedMsg: point to point message from the group coordinator to the requestor to grant the permission to broadcast video in the group
13. StartVideoBcastRequestDeniedMsg: point to point message from the group coordinator to the requestor to denied the permission to broadcast video in the group

Figure 5: The media control message in the "volatile" control channel.

The major assumption in designing the intra-group membership protocol as well as the inter-group membership protocol is that the underlying JavaGroups framework provides total-order and loss-less delivery of messages. The consistency and integrity of the membership information between various members in a group relies on the reliability guarantees of the underlying JavaGroups.

Under this assumption, the design of the intra-group membership protocol takes on a distributed approach. All members in the group have the same membership information about the group. New members who join later simply query the coordinator of the group (the first address in the new view) and therefore acquire all the membership information about the group before they can be admitted as new members. If in case of the failure or departure of the coordinator, the new view will have a new coordinator and since every member has the same membership information about the group, the new coordinator will take over immediately as the official coordinator of the group.

MediaManager class is the super class for all data-channel managers, such as VideoManager, AudioManager and TextManager. MediaManager extends Manager class and defines more instance variables and methods that are common to all data-channel managers, such as data packet size/delay, capturing and streaming data, as well as playing-back data.

VideoManager is the channel manager for the video channel. It deals with the capture and playback of video data. The initial design for JavaGroupVC is to use JMF 2.0 to capture and playback of video data. However, due to the delay in the release of JMF2.0, the video capture feature is not implemented in this release. JMF1.1 player is used to playback the video broadcast.

All the video data messages are subclass of Message class in JavaGroups. There are three different data messages in the video channel:

1. StartVideoMsg: broadcast message that represents the very first video packet from the member who is broadcasting video to all members so that they can start a new player
2. VideoMsg: broadcast message that contains the video data
3. EndVideoMsg: broadcast message that represents the very last video packet from the member who is broadcasting video to all members so that they can kill the player

AudioManager is the channel manager for the audio channel. It deals with the capture and playback of audio data. Just like in the video data, the initial design for JavaGroupVC is to use JMF 2.0 to capture and playback of audio data. However, due to the delay in the release of JMF2.0, the audio capture feature is not implemented in this release. JMF1.1 player is used to playback the audio broadcast.

All the audio data messages are subclass of Message class in JavaGroups. There are three different data messages in the Audio channel:

1. StartAudioMsg: broadcast message that represents the very first audio packet from the member who is broadcasting audio to all members so that they can start a new audio player
2. AudioMsg: broadcast message that contains the audio data
3. EndAudioMsg: broadcast message that represents the very last video packet from the member who is broadcasting audio to all members so that they can kill the audio player

Figure 7: The data messages for AudioManager in the audio channel

TextManager is the channel manager for the text channel. It deals with the broadcast and receipt of text data. Just like in the video data, the initial design for JavaGroupVC is to use JMF 2.0 to capture and playback of audio data. However, due to the delay in the release of JMF2.0, the audio capture feature is not implemented in this release. JMF1.1 player is used to playback the audio broadcast.

All the text data messages are subclass of Message class in JavaGroups. There is only one data message in the text channel:

1. TextMsg: broadcast message (public message) to all members in the group or point to point message (private message) to a specific recipient that contains text data.

Figure 8: The data message for TextManager in the text channel

The main Graphical User Interface component in this project is the GUIManager that handles all the user events through the lightweight Java Swing component. There are four major sub-components that are one level below GUIManager: ChatPanel, MediaPanel, MembershipPanel and GroupsPanel.

Figure 9: The Graphical User Interface Layer

ChatPanel is the GUI sub-component for displaying and recording the activities in the chat session ( text channel). ChatPanel consists of a text field for typing in messages and a text area for displaying messages received from all members.

GroupsPanel is the GUI sub-component for displaying all the groups together with all the members in those groups. It has a group tree structure that can be expanded to see the members in each group.

MembershipPanel is the GUI sub-component for displaying all the members in the current group. Each member in the group is represent as a clickable button with username and user-icon. Private message in a chat session is sent by clicking on the button that represents a particular member.

MediaPanel is the GUI sub-component for displaying both the video and audio player. However, since JMF 1.1 only supports heavyweight component in Java AWT, the video and audio players used in this release are both heavyweight component. This creates a problem when the player is overlapping with the border (which is lightweight Swing component), the player is always on top.

In order to make the application truly interactive and responsive to the user, user events have to be communicated through the four sub-components to the underlying JavaGroups channels. On the other hand, in order for to show the activities or events occur in the underlying JavaGroups framework, various channel managers need a way to communicate with the four sub-components. GUIManager handles all the communication between the GUI layer and the control/data layer. The goal is to make sure that we have a centralized control over the GUI layer so that the codes will be more manageable as they become more complex.

Figure 10: The communication between the control/data layer and the GUI layer

Limitations

In this release, there are several limitations as listed below

1. Only audio and video broadcast is support. No point to point audio or video is supported.
2. Capture of audio and video is not implemented ( due to the delay in the release of JMF2.0)
3. The only video format supported is MPEG
4. The only audio format supported is AU
5. The packet delay of 500ms is deliberately added while broadcasting audio or video. JMF 1.1 player crashes before the player can be realized if the delay is not in place.

1. Video and audio capture

With the release of JMF 2.0 this summer, the capture feature should implemented in package JavaGroups.VideoConference.Media.Capture. Note that the playback of video and audio is implemented in package JavaGroups.VideoConference.Media.Playback and it implements a pull data source model.

2. Multi-party video conferencing session

In this release, only two-party video conferencing session is possible. This limitation can be easily overcome in one of the two ways:

• There is only one video channel and one audio channel in each group. While receiving messages for video and audio, parse the headers to determine the source.
• Create one video channel and one audio channel for each source (each member participating in the videoconference). Within the individual channel, it is just like a one-way broadcast of video or audio, which is already implemented in this release.

0. Client gets Username, Groupname, UserIcon from the end user.

1. When a client connects to a group, it checks if it is the only member.  

   1a. If it is, it adds itself to its hash-table. Note that it is accepted to a group. Starts listening to messages. Go to (3)   

   1b. If it is not, make a request to the coordinator to join the group.Ignore all messages for now (except for the request reply from coordinator). Go to (2)

2. Listening to coordinator’s reply. If receive a request reply from coordinator

    2a. If it is a request denied reply, and if the problem is in name duplication, get a different username from user and go to (1b). If the problem is in icon duplication, send icon request message to coordinator and loop back to (2)

    2b. If it is a request granted reply, set its own user-info. Broadcast its user-info to the group. Request initial members info in the group from coordinator. Loop back to (2).

    2c. If it is an initial member info reply, set hash-table to the one from coordinator. Add own user-info to the hash-table. Note that it is accepted to a group. Go to (3)

    2d. If it is an icon reply message, display available icon and get a different icon from user. Go to (1b)

3. Listening to messages. If receive a message,

    3a. If it is a join request from a new member, go to (4)

    3b. If it is a member info request from a newly accepted member, go to (5)

    3c. If it is a user-info broadcast message from a newly accepted member, go to (6)

    3d. If it is a text-channel address broadcast message from a newly accepted member, go to (7)

    3e. If it is a video-channel address broadcast message from a newly accepted member, go to (8)

    3f. If it is a audio-channel address broadcast message from a newly accepted member,  go to (9)

4. Checks the proposed user-info against other user-info in hash-table.

    4a. If the proposed user-info is unique, sends request granted reply to new member. Loops back to (3)

    4b. If the proposed user-info is not unique, sends request denied reply to the new member. Loops back to (3)

5. Receives a member info request from a newly accepted member,

    5a. If itself is a coordinator, send hash-table in the reply to this newly accepted member. Loops back to (3)

    5b. It itself is not a coordinator, then do nothing. Loops back to (3)

6. Receives a user-info broadcast message,

    6a. If the message is a broadcast from itself, ignores the message. Loops back to (3)

    6b. If the message is a broadcast from someone else, add the user-info to hash-table. Loops back to (3)

7. Receives a text-channel address broadcast message,

    7a. If the message is a broadcast from itself, ignores the message. Loops back to (3)

    7b. If the message is a broadcast from someone else, add the text-channel address to the appropriate user-info in the hash-table. Loops back to (3)

8. Receives a video-channel address broadcast message,

    8a. If the message is a broadcast from itself, ignores the message. Loops back to (3)

    8b. If the message is a broadcast from someone else, add the video -channel address to the appropriate user-info in the hash-table. Loops back to (3)

9. Receives a audio-channel address broadcast message,

    9a. If the message is a broadcast from itself, ignores the message. Loops back to (3)

    9b. If the message is a broadcast from someone else, add the audio -channel address to the appropriate user-info in the hash-table. Loops back to (3)

1. It checks if it is the only member in the default group.

     1a. If it is, it does nothing (meaning there are no groups active yet). Starts listening to messages. Go to (3)

     1b. If it is not, make a request to the coordinator to get a list of all groups. Go to (2)

2. Listening to coordinator’s reply. If receives a list of all groups, update its hashtable. Starts listening to messages. Go to (3)

3. Listening to messages. If receives a message and the message is a

    3a. GroupListRequestMsg, and client is the coordinator, reply to sender a current group list in a GroupListReplyMsg. Loops back to (3).

    3b. MemberJoinGroupMsg, update the hashtable by adding the newly join member. Loop back to (3).

    3c. MemberLeaveGroupMsg, update the hashtable by deleting the member who just left. Loop back to (3).