This page lists my past academic research projects. More recent projects are listed on my Home page

Research Interests

  • Software architecture for interactive systems
  • Dynamic models of perceptual and cognitive processes
  • Design and creativity

Software architecture

Software Architecture for Immersipresence (SAI)

SAI is a software architecture framework for the design, analysis and implementation of integrated media systems.

Publications: ComJnl 2011, NIME 2006, SACV, IMSC-05-003, ICSE 2004, IMSC-03-001, MM 2000.

Funding: IMSC (NSF-ERC)

MFSM

MFSM (Modular Flow scheduling Middleware) is an architectural middleware implementing the core elements of the SAI style. [mfsm.sourceforge.net]

Publications: SACV.

Funding: IMSC (NSF-ERC)

VisualSAI

VisualSAI is a graphical environment for visually creating and analysing component-based architectural designs in the SAI style. [visualsai.sourceforge.net]

Publications: IMSC-05-003.

Funding: IMSC (NSF-ERC)

Music computing

MIMI: Multi-modal Interaction for Musical Improvisation (with Elaine Chew and Dennis Thurmond)

MIMI is a multi-modal interactive musical improvisation system that explores the potential and powerful impact of visual feedback in performer-machine interaction. [website]

Publications: IJART 2013, CIE 2011, IMIDA 2010, TPMC, NIME 2007, CS-07-889.

Funding: NSF

MuSA.RT: Music on the Spiral Array . Real-Time (with Elaine Chew)

MuSA.RT is a collaborative research project that integrates real-time music processing and content-based graphical rendering in interactive immersive environments.

Publications: ICMPC 2008, MCM 2007, CIE 2005, MM 2003.

Funding: IMSC (NSF-ERC), NSF, Radcliffe Institute for Advanced Study

ESP: Expression Synthesis Project (with Elaine Chew)

The Expression Synthesis Project (ESP) explores the use of a driving interface for expression synthesis, also known as performance rendering. ESP aims to provide a compelling metaphor for expressive performance so as to make high-level expressive decisions accessible to non-experts.

Publications: AMC 2006, ACE 2006, NIME 2005.

Funding: IMSC (NSF-ERC) and NSF

Interactive systems

IMSC Communicator

The IMSC Communicator is an experimental extensible platform for remote, collaborative data sharing. The goal of this project is to explore and formalize architectural patterns for true multi-stream systems.

Publications: IMSC-03-001.

Funding: IMSC (NSF-ERC)

The Virtual Mirror: A Handheld Mirror Simulation

The Virtual Mirror is a system simulating a handheld mirror.

Publications: SACV, ICME 2003, IMSC-03-001, SIGGRAPH 2002.

Funding: IMSC (NSF-ERC) and USC Arts Initiative

Computer vision

Visual Sensing for Natural Human-Robot Interaction (with Gérard Medioni)

This project addresses the design and implementation of a computer vision system to empower personal robots with advanced vision capabilities.

Publications: ComJnl 2011, CVIU 2007, URAI 2006, CS-06-880.

Funding: ETRI

Video Event Markup Language (with Ram Nevatia)

The specification and development of a Video Event Markup Language (VEML) is part of a large scale effort supported by the Advanced Research and Development Activity (ARDA) of the U.S. Government to promote video understanding at the event level.

Publication: MM 2005.

Funding: ARDA

Real-Time Video Analysis: Segmentation and Tracking (with Gérard Medioni)

This series of projects leverage color background model-based segmentation (change detection), and an original multi-resolution approach to tracking. Applications include background substitution (``blue-screen without a blue screen’’) and player and ball tracking in racquet sports (tennis, racquetball).

Publications: SACV, IRIS-04-422, IRIS-03-420, ICVS 2001, CISST 1999, Ph.D. Dissertation.

Invention disclosure: A System to Perform Real-time Video Stream Segmentation Using an Adaptive Statistical Background Model in HSV Color Space, Alexandre R.J. François and Gérard G. Medioni, file #2937, USC Office of Technology Licensing.

Funding: IMSC (NSF-ERC)

3-D Object Descriptions from a single image (with Gérard Medioni)

This series of projects combine vision and graphics in the design of interactive systems and algorithms for 3-D modeling from a single image. Projective properties of classes of Generalized Cylinders and bilateral-symmetric objects provide constraints that allow geometric reconstruction. Sparse and intuitive user input allows disambiguation and efficient computation.

Publications: IVC 2003 (ICPR 2002), IVC 2001 (ICMCS 1999), Ph.D. Dissertation.

Invention disclosure: A Human-assisted System to Build 3-D Models from a Single Image, Alexandre R.J. François and Gérard G. Medioni, file #2936, USC Office of Technology Licensing.

Funding: IMSC (NSF-ERC)

Shape/Object Recognition (with Gérard Medioni)

This project tackles the problem of generic shape recognition, in which exact models are not available. A Case-Based Reasoning approach performs learning of segmented, hierarchical shape descriptions, and recognition based on previous observation.

Publications: ICPR 2000, IWOR 1996, M.S. Thesis.

Funding: DARPA