sai || software architecture for immersipresence
Alexandre R.J. François
|
SAI (Software Architecture for Immersipresence) is an architectural
framework for the design, analysis and implementation of interactive
software systems.
SAI defines a formal architectural style whose
data and processing models capture temporal properties of
computational primitives. SAI's asynchronous concurrent processing
model allows designing for optimal (theoretical) system latency and
throughput. The modularity and scalability of the style facilitate
distributed code development, testing, and reuse, as well as fast
system design and integration, efficient maintenance and evolution. A
graph-based notation for architectural designs affords intuitive and
scalable system representation.
The formalism allows a range of intermediate-level representations
from conceptual to logical level. SAI favors the encoding of system
logic in the structural organization of simple computing components,
rather than in computationally complex individual components. Designs
exhibit a rich variety of structural and functional architectural
patterns, suitable for systematic study and re-use.
The open source Modular Flow Scheduling
Middleware (MFSM) provides a multi-threaded, cross-platform
implementation of SAI's primitives.
The SAI/MFSM framework has been used in the design and implementation
of numerous interactive systems in
research, education and performance settings.
|
SAI specifies a formal architectural style comprised of an extensible
data model and a hybrid (shared memory and message-passing)
distributed asynchronous concurrent processing
model. Figure 1 presents an overview of SAI
defining elements in their standard notation.
Figure 1: Overview of SAI elements. Cells are represented as
squares, sources as circles. Source-cell connections are drawn as fat
lines, while cell-cell connections are drawn as thin arrows crossing
over the cells. When color is available, cells are colored in green
(reserved for processing); sources, source-cell connections, passive
pulses are in colored in red (persistent information); streams and
active pulses are colored in blue (volatile information).
In SAI, all data is encapsulated in pulses. A pulse is the
carrier for all the synchronous data causally related to a given time
stamp. Information in a pulse is organized as a mono-rooted
composition hierarchy of
node instances. The nodes constitute an extensible set of
atomic data units that implement or encapsulate specific data
structures. Pulses holding volatile data flow down streams defined by
connections between processing centers called cells, in a
message passing fashion. They trigger computations, and are thus
called active pulses. In contrast, pulses holding persistent
information are held in repositories called sources, where
the processing centers can access them in a concurrent shared memory
access fashion. Processing in a cell may result in the augmentation
of the active pulse (input data), augmentation and/or update of the
passive pulse (process parameters). The processing of active pulses
is carried concurrently, as they are received by the cell. Data
binding is performed dynamically in an operation
called filtering. Active and passive filters
qualitatively specify, for each cell, the target data in respective
pulses. This hybrid model combining message passing and shared
repository communication, combined with a unified data model,
constitutes a universal processing framework.
A particular system architecture is specified at the conceptual level
by a set of source and cell instances, and their inter-connections.
Specialized cells may be accompanied by a description of the task they
implement. The logical level specification of a design describes, for
each cell, its active and passive filters and its output structure,
and for each source, the structure of its passive pulse.
An open source architectural middleware called Modular Flow Scheduling
Framework (MFSM), developed in C++, provides cross-platform code
support for SAI's architectural abstractions, in the form of an
extensible set of classes. A number of software modules regroup
specializations that implement specific data structures, algorithms
and/or functionalities. They constitute a constantly growing base of
open source, reusable code, maintained as part of the MFSM project.
The project also comprises extensive documentation, including user
guide, reference guide and tutorials, all available on the project
Website: mfsm.sourceforge.net
The SAI framework has been used in the design and implementation of
numerous interactive systems.
In the computer vision domain, early interactive systems designed and
developed with the SAI framework include Video Painting, a real-time
mosaicing system,
and Virtual
Mirror, a handheld mirror simulation.
Stevi, a high-level
vision system for a personal service robot shows the potential of the
SAI approach for building ambitious vision systems that integrate with
other software and hardware components in interactive applications.
An early demonstration that performed live video
mapping on an animated 3D model illustrated the design of
asynchronous concurrent component for interactive computer graphics.
As SAI and associated tools became more mature, they allowed more
ambitious cross-disciplinary projects. The SAI framework has
facilitated the collaborative design and successful implementation of
original and innovative interactive music
systems. Representative projects
include Music on the
Spiral Array . Real-Time (MuSA.RT), an interactive music analysis
and visualization system;
the Expression Synthesis
Project (ESP), a driving interface for generating expressive
musical performances;
and, Multimodal
Interaction for Musical Improvisation (MIMI), an interactive
musical improvisation system that provides visual feedback to the
performer.
The SAI framework was instrumental in the successful implementation of
graduate and undergraduate courses that center on a class-wide term
project. A graduate level seminar course,
titled Integrated Media
Systems, taught in Fall 2002 at the University of Southern
California (USC), introduced students to the specific technological
and organizational difficulties of designing and building interactive
and media-rich integrated systems. Using SAI and MFSM, through a
supervised, systematic, iterative and modular process, 24 students
developed a playable on-line 3D soccer game in only two months. None
of the students had used SAI before, and none had ever participated in
such a project. The design of a special session of the undergraduate
course Principles of
Software Development (the last of a series of four required
programming courses at USC) for students enrolled in, or interested in
pursuing, USC's new computer science Games major, adopted a similar
class-wide collaborative project. In the Spring 2008 session, 8
students, using the SAI/MFSM framework, collaboratively designed and
implemented Crosswinds,
a networked video adaptation of an original student-designed board
game. The SAI formalism allows high level understanding of the
overall system organization, precise composition rules, and strong
structure for code development, a combination that was, in both cases,
essential in the successful completion of a class-wide collaborative
project. The class project in these courses represented a major
innovation compared to traditional class projects, in which
individuals or small teams develop small-scale independent or
identical projects.
software architecture
Alexandre R.J. François,
"An Architectural Framework for the Design, Analysis and Implementation of Interactive Systems,"
IEEE Transactions on Software Engineering, under review.
Alexandre R.J. François,
"A Hybrid Architectural Style for Distributed Parallel Processing of Generic Data Streams,"
Proceedings of the International Conference on Software Engineering, pp. 367-376, Edinburgh, Scotland, UK, May 2004.
[pdf]
[BibRef]
Alexandre R.J. François,
Software Architecture for Immersipresence,
IMSC Technical Report IMSC-03-001,
University of Southern California, Los Angeles,
December 2003.
[pdf]
[BibRef]
music computing
Alexandre R. J. François, Elaine Chew and Dennis Thurmond,
"Visual Feedback in Performer-Machine Interaction for Musical Improvisation,"
Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 277-280, New York, NY, USA, June 2007.
[pdf]
[BibRef]
Alexandre R. J. François and Elaine Chew,
"An Architectural Framework for Interactive Music Systems,"
Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 150-155, Paris, France, June 2006.
[pdf]
[BibRef]
Elaine Chew and Alexandre R. J. François,
"Interactive multi-scale visualizations of tonal evolution in MuSA.RT Opus 2,"
ACM Computers in Entertainment Special Issue on Music Visualization, vol. 3, no. 4, October-December 2005, 16 pages. [ACM Digital Library]
Elaine Chew, Alexandre R. J. François, Jie Liu and Aaron Yang,
"ESP: A Driving Interface for Expression Synthesis,"
Proceedings of the International Conference on New Interfaces for Musical Expression, Vancouver, B.C., Canada, May 2005.
[Abstract]
[pdf]
[ACM Digital Library]
interactive systems
Alexandre R.J. François,
"Components for Immersion,"
Proceedings of the IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland, August 2002.
[pdf]
[BibRef]
Alexandre R.J. François, Elaine Kang and Umberto Malesci,
"A Handheld Virtual Mirror,"
SIGGRAPH Conference Abstracts and Applications proceedings, p.140, San Antonio, TX, July 2002.
[see conference paper]
[pdf]
[BibRef]
computer vision
Gérard G. Medioni,
Alexandre R.J. François,
Matheen Siddiqui, Kwangsu Kim and Hosub Yoon,
"Robust Real-Time Vision for a Personal Service Robot,"
to appear, Computer Vision and Image Understanding, special issue on Human-Computer Interaction.
[doi]
[BibRef]
Alexandre R.J. François,
"Software Architecture for Computer Vision,"
Emerging Topics in Computer Vision, G.Medioni and S.B. Kang Eds., Prentice Hall, 2004, pp. 585-654.
This research was funded in part by
the Integrated Media Systems Center,
a National Science Foundation
Engineering Research Center, Cooperative Agreement
No. EEC-9529152. Any opinions, findings and conclusions or
recommendations expressed in this material are those of the author and
do not necessarily reflect those of the National Science Fundation.