|
|
IMSC Communicator |
The IMSC Communicator is an experimental extensible platform for remote, collaborative data sharing. The goal of this project is to explore and formalize architectural patterns for true multi-stream systems.
Popular architectures for communication applications include Client/Server and Peer To Peer. Different elements of the overall system are considered separate applications. Although these models can either be encapsulated or implemented in the SAI style, a communication application designed in the SAI style can also be considered a single distributed application graph, in which some cell to cell connections are replaced with network links. From this point of view, specific network architectures would be implemented in the SAI style, as part of the overall distributed application.
The core pattern of the Communicator is a sequence of cells introducing network communication between two independent subgraphs. Figure 1 shows an example sequence comprising encoding, compression and networking (send) on the emitting side, networking (receive), decompression and decoding on the receiving side. The encoding cell flatten a node structure into a linear buffer so that the structure can later be regenerated. This encoding can be specific of various data types. A general encoding scheme could for example be based on XML. The output of an encoding cell is a character string (binary or text). The compression cell takes as input the encoded character string produces a compressed buffer. The compression scheme used can be input data dependent, or generic, in which case it should be lossless. Note that the compression step is optional. The networking cells are responsible for packetizing and sending incoming character strings on one side, and receiving the packets and restoring the string on the other side. Different modalities and protocols can be implemented and tested. The decompression cell regenerates the original character string from the compressed buffer. The decoding cells regenerate the node structure into a pulse, from the encoded character string.
Figure 1: Simple generic networking for distributed applications.
This communication core is used as a generic platform for designing and testing data transfer modalities, synchronization schemes and corresponding architectural patterns.
The very first test system supported video only, using existing live video capture and image display cells. A later demonstration included a new live capture cell for both image and sound, developed using Microsoft DirectShow, and a new cell for synchronized rendering of sound and video, also developed using DirectShow. Figure 2 shows the conceptual graph for early embodiments of the 2-way communicator, with example screen shots. For this experiment, the compression cell encapsulates the open source LZO library for real-time lossless compression/decompression. Note that the compression step is optional. Networking cells were implemented using Windows Sockets, using either TCP/IP or UDP. A background replacement unit, based on the segmentation by change detection, was added to the capture side to illustrate how the modularity of the architecture allows to ``plug-and-play'' subgraphs developed independently.
Figure 2: Conceptual graph for an early version of the IMSC Communicator, with support for video (synchronized image and sound).
In this particular architectural design, the combination of image and sound streams is treated as a single input data stream produced by the capture cell and used by the rendering cell. This simple choice, adopted as a preliminary test design, does not address directly the most interesting and critical synchronization issues of multi-stream systems. Going beyond the external alignment of input data requires the design of independent input cells for image, sound and other data types (e.g. MIDI data). A first technical issue, not directly related to system design, is that of accurate time-stamping of these independent streams. Then, assuming some method exists that provides sufficient accuracy for a given application, any set of externally synchronized input streams, even if they are delivered to the system in the form of separate streams, only amount to a single internal stream. An important family of design patterns and algorithms to be explored therefore deals with temporal alignment of data streams. Other promising research directions include investigation of data aware scheduling strategies for network transmissions.
The following graduate students worked on aspects of this project (listed in chronological order):
This research was funded in part by the Integrated Media Systems Center, a National Science Foundation Engineering Research Center, Cooperative Agreement No. EEC-9529152. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect those of the National Science Fundation.