Interactive Orchestra

PURPOSE

The Interactive Orchestra project is a virtual reality experience that aims to provide a podium for music students who wish to gain experience conducting an orchestra. It creates a more convenient bridge between student learning in the classroom and experiential learning in front of an orchestra through the utilization of 3D space and sound.

 

 

 

 

THE EXPERIENCE

The experience simulates the response latency from a symphony orchestra relative to the beat of the conductor, allow the user to influence the music elements such as tempo and dynamics, and provide instantaneous feedback on gestures.

Starting with “O Canada” as a backing track, the experience is focused on tracking the user’s gestures and mapping them to individual beats, while collecting corresponding data such as velocity.

Background

The Interactive Orchestra project is a virtual reality experience that aims to provide a virtual podium for music students who wish to gain experience conducting an orchestra. It seeks to create a more convenient bridge between student learning in the classroom and experiential learning in front of an orchestra.

The experience will simulate the response latency from a symphony orchestra relative to the beat of the conductor, allow the user to influence the music elements such as tempo and dynamics, and provide instantaneous feedback on gestures. To learn more, read our project spotlight on Interactive Orchestra.

Check out the 360 video below showing Dr. Girard conducting an orchestra of his students.

 

THE PROCESS

The project was originally built on HTC Vive but has since been ported to the Oculus Rift Touch.

An asset called AirSig was used to collect each gesture of the 4/4 legato beat and recognize sections of a user’s gesture.

 

FEATURES

Notable features currently include:

  • Tempo changes based on the speed of user gestures
  • Custom UBC engraved baton
  • Improved menu user interface with beats-per-minute display and progress bar
  • Metronome implementation
  • Static 4/4 beat guide
  • Practice room environment

NEXT STEPS

The experience will include an environment created from 3D model captures of the stage at the Chan Centre for the Performing Arts. To improve the first-time user’s experience, an animated guide will introduce the gestures in a dynamic manner.

To echo a more realistic conducting experience, the audio file will be separated and linked to models for individual orchestral sections.

Faculty

Dr. Jonathan Girard | Director of Orchestral Activities, Assistant Professor of Conducting and Ensembles

Student Team

  • Serena Chao
  • Michael Goh
  • Angel Wen
  • Vanessa Bayubaskoro
  • William Beltran
  • Farhan Kassam
  • Jonathan Kay
  • Conrad Sosnowski
  • Jessica Wu