A system for mobile music authoring and active listening

We propose a paradigm and an end-to-end system to support authoring and real-time active listening experience of prerecorded music. Users are prosumers who navigate and express themselves in a shared (physical or virtual) orchestra space, populated by the sections of an orchestra playing a prerecorded music piece. The system consists of two components: an authoring system enabling the user to edit and shape personalized versions of active listening experiences, and a run-time environment implementing the mechanisms of interaction with the active music content. Both components exploit metadata defining the mapping rules of users expressive gesture and context information onto sound and music processing. A user interacts with content by holding her smartphone in her hand and by using environmental sensors and context-aware information to shape the experience, possibly in cooperation with other users.