From remote media immersion to distributed immersive performance

Abstract

We present the architecture, technology and experimental applications of a real-time, multi-site, interactive and collaborative environment called Distributed Immersive Performance (DIP). The objective of DIP is to develop the technology for live, interactive musical performances in which the participants - subsets of musicians, the conductor and the audience - are in different physical locations and are interconnected by very high fidelity multichannel audio and video links. DIP is a specific realization of broader immersive technology - the creation of the complete aural and visual ambience that places a person or a group of people in a virtual space where they can experience events occurring at a remote site or communicate naturally regardless of their location. The DIP experimental system has interaction sites and servers in different locations on the USC campus and at several partners, including the New World Symphony of Miami Beach, FL. The sites have different types of equipment to test the effects of video and audio fidelity on the ease of use and functionality for different applications. Many sites have high-definition (HD) video or digital video (DV) quality images projected onto wide screen wall displays completely integrated with an immersive audio reproduction system for a seamless, fully three-dimensional aural environment with the correct spatial sound localization for participants. The system is capable of storage and playback of the many streams of synchronized audio and video data (immersidata), and utilizes novel protocols for the low-latency, seamless, synchronized real-time delivery of immersidata over local area networks and wide-area networks such as Internet2. We discuss several recent interactive experiments using the system and many technical challenges common to the DIP scenario and a broader range of applications. These challenges include: (1). low latency continuous media (CM) stream transmission, synchronization and data loss management; (2). low latency, real-time video and multichannel immersive audio acquisition and rendering; (3). real-time continuous media stream recording, storage, playback; (4). human factors studies: psychophysical, perceptual, artistic, performance evaluation; (5). robust integration of all these technical areas into a seamless presentation to the participants. Copyright 2003 ACM.

Publication Title

Proceedings of the 2003 ACM SIGMM Workshop on Experiential Telepresence, ETP '03

Link to Full Text

Share

COinS