This paper presents some recent developments at DISTInfoMus Lab on multimodal and cross-modal processing of multimedia data streams with a particular focus on interactive systems exploiting Tangible Acoustic Interfaces (TAIs). In our research multimodal and cross-modal algorithms are employed for enhancing the extraction and analysis of the expressive information conveyed by gesture in non-verbal interaction. The paper discusses some concrete examples of such algorithms focusing on the analysis of high-level features from expressive gestures of subjects interacting with TAIs. The features for explicit support of multimodal and cross-modal processing in the new EyesWeb 4 open platform (available at www.eyesweb.org) are also introduced. Results are exploited in a series of public events in which the developed techniques are applied and evaluated with experiments involving both experts and the general audience. Research is carried out in the framework of the EU-IST STREP Project TAI-CHI (Tangible Acoustic Interfaces for Computer-Human Interaction).

Multimodal and cross-modal processing in interactive systems based on tangible acoustic interfaces

CAMURRI, ANTONIO;MAZZARINO, BARBARA;VOLPE, GUALTIERO
2005-01-01

Abstract

This paper presents some recent developments at DISTInfoMus Lab on multimodal and cross-modal processing of multimedia data streams with a particular focus on interactive systems exploiting Tangible Acoustic Interfaces (TAIs). In our research multimodal and cross-modal algorithms are employed for enhancing the extraction and analysis of the expressive information conveyed by gesture in non-verbal interaction. The paper discusses some concrete examples of such algorithms focusing on the analysis of high-level features from expressive gestures of subjects interacting with TAIs. The features for explicit support of multimodal and cross-modal processing in the new EyesWeb 4 open platform (available at www.eyesweb.org) are also introduced. Results are exploited in a series of public events in which the developed techniques are applied and evaluated with experiments involving both experts and the general audience. Research is carried out in the framework of the EU-IST STREP Project TAI-CHI (Tangible Acoustic Interfaces for Computer-Human Interaction).
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11567/852416
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? ND
social impact