Spatial Audio & Invisible Musicians in Tosca VR

My colleagues and I at Relative Motion all come from a background where we know that the best performance is a live dialogue between performers, musicians, those cueing and leading tempo and dynamics and, crucially, an audience. In creating Tosca VR, despite it being a pre-recorded piece, we wanted to ensure the audience felt an authentic, beating heart at its core – a sense of liveness as well as complete immersion. In an opera house, the visual and aural connection between conductor, performers and members of the orchestra is paramount – it’s highly unusual that any visual or audio monitoring is used. Our design for Tosca VR was to place a single audience member (which we abbreviate to SAM) in the centre of the action, as it  would play out in a real location.  There'd be no 

convenient giant pit in which to hide even the smallest of orchestras and conductors! Add into the mix the need to record our singers in high quality at the same time as being able to place their voices correctly in a sphere of audio surrounding our SAM and we had a set of tricky technical and creative challenges to solve.
    
Pre-recording an orchestra was our initial instinct but our musical director Joe Louis Robinson helped us refine a better solution. He and the singers needed a more responsive relationship both as their performances developed in rehearsal and during recordings; singing to a backing track would be restrictive. Liveness was crucial. Just as a rehearsal pianist or repetiteur sits in for a full orchestra in an opera house, Joe would accompany the singers live on a keyboard, in rehearsals and during the final recording.
 
We used a combination of microphone techniques. An Ambisonic mic (which contains 4 separate mic capsules in a tetrahedral arrangement) sat, along with its recorder just under our 360 camera, capturing a full sphere of room audio. Our singers were also fitted with radio microphones, their tiny capsules hidden in choice places under clothing. Clean, clear and separate vocal capture was vital but how could the singers hear Joe’s keyboard and spoken cues without the sound bleeding into the mics? We didn’t want an ever-present “roomy” version of Joe’s keyboard to cause headaches in post-production.
 
In-ear monitors are now commonplace in live sound production but they’re big earplugs with a cable – quite visible on camera to our SAM, which would somewhat spoil the look of the piece. We knew the solution might lie in a new range of extremely tiny wireless earpieces made by a manufacturer of hearing aids that sit right in the ear canal. Asking an opera singer to block up one ear with an earpiece, no matter how tiny, is a delicate matter. Vocal aural feedback is crucial for a singer’s intonation, or pitching. Our excellently resilient singers spent two days “living” with their earpieces as we made fine adjustments to levels and positioning leading up to a series of single-take captures. 
 
With Joe’s accompaniment and spoken cues transmitted direct to the performers’ ears (with virtually zero latency) and with Joe monitoring his keyboard and the singers in headphones, we experienced the uncanny effect of hearing only their voices in the space, yet perfectly synchronised, responsive to one another and live!
 
Near flawless vocal performance capture achieved, musical director Joe took the vocal recordings into his studio, along with his reference keyboard track to create the time-base for the orchestral recording sessions. Final tweaks were made to his arrangements before we arrived on the doorstep of Sonica Studios in London with our assembled compact chamber orchestra. Sonica’s Mat and Paul are no strangers to projects with somewhat interesting briefs and they completely understood the production process we were undergoing. Our orchestra were separated into different live rooms as much as possible, all with sightlines through windows to Joe, conducting, who monitored tracks of the singers, a tempo-setting click and his piano from the shoot in headphones. In a series of cues (self-contained sections of the music) the score for Tosca VR came to life, hand in hand, beat to beat with the vocal performances of our singers

​Written by Andy Purves