Purpose

The Interactive Orchestra project is a virtual reality experience that aims to provide a podium for music students who wish to gain experience conducting an orchestra. It creates a more convenient bridge between student learning in the classroom and experiential learning in front of an orchestra through the utilization of 3D space and sound.

The Experience

The experience simulates the response latency from a symphony orchestra relative to the beat of the conductor, allows the user to influence the musical elements such as tempo and dynamics, and provides instantaneous feedback on gestures.

Starting with “O Canada” as a backing track, the experience is focused on tracking the user’s gestures and mapping them to individual beats, while collecting corresponding data such as velocity.

Background

The Interactive Orchestra project is a virtual reality experience that aims to provide a virtual podium for music students who wish to gain experience conducting an orchestra. It seeks to create a more convenient bridge between student learning in the classroom and experiential learning in front of an orchestra.

The experience will simulate the response latency from a symphony orchestra relative to the beat of the conductor, allow the user to influence the musical elements such as tempo and dynamics, and provide instantaneous feedback on gestures. To learn more, read our project spotlight on Interactive Orchestra.

Check out the 360 video below showing Dr. Girard conducting an orchestra of his students.

The Process

The project was originally built on HTC Vive but has since been ported to the Oculus Rift Touch.

An asset called AirSig was used to collect each gesture of the 4/4 legato beat and recognize sections of a user’s gesture.

Features

Notable features currently include:

• Tempo changes based on the speed of user gestures
• Custom UBC engraved baton
• Improved menu user interface with beats-per-minute display and progress bar
• Metronome implementation
• Static 4/4 beat guide
• Practice room environment

The Team

Faculty

• Dr. Jonathan Girard | Director of Orchestral Activities, Assistant Professor of Conducting and Ensembles
• Dr. Patrick Pennefather | Assistant Professor, UBC Theatre and Film

Students

• Serena Chao
• Michael Goh
• Angel Wen
• Vanessa Bayubaskoro
• William Beltran
• Farhan Kassam
• Jonathan Kay
• Conrad Sosnowski
• Jessica Wu
• Georgette Espina