Creating a Multi-Channel, Multi-Platform Mix
A Report from the AES/SMPTE Joint Meeting
by Mel Lambert
Photos by Mel Lambert and Richard A.Wollrich, courtesy of AES LA Section
Taking as its focus the various ways in which the TV and film community can create a single, object-based immersive mix that will translate everywhere — from cinema to home theatre to tablets with headphones — a recent AES/SMPTE monthly meeting held at the Sportsmen’s Lodge in Studio City in late January offered a valuable analysis from industry practitioners.
Entitled “The Mix: The Challenges of Creating a Multi-Channel, Multi-Platform Product,” the meeting was moderated by Jim DeFilippis from TMS Consulting, who has experience of immersive 3D audio. DeFilippis artfully steered the discussions from sound acquisition via post-production to media distribution, “from the vantage point of the creators and through the various crafts,” while reviewing available formats and workflow solutions and “what decisions need to be made with respect to creating immersive, object-based and/or multi-channel sound” that can be delivered over a wide variety of distribution channels.
“I have been creating immersive experiences since before [the advent of] Dolby Atmos,” recalled Oscar- and Emmy- nominated re-recording mixer Greg P. Russell, CAS, from Technicolor at Paramount post facility. “The early Lt-Rt Dolby Stereo with mono surrounds created sound from all around the audience, while 5.1- and 7.1-channel formats offered more potential for immersive experiences,” especially the 7.1 format, “with its quad surrounds that offer better separation.” During his long career, Russell has mixed more than 200 feature films, including Spider-Man, Armageddon, The Mask of Zorro and director Michael Bay’s Transformers franchise, with 16 Academy nominations, 11 CAS Award nominations, two BAFTA nominations and two Emmy nominations — with one win.
The other panelists were re-recording mixer Tony Lamberti, CAS, who has more than 30 years of mixing experience in this complex art form with Emmy Awards for his work on And Starring Pancho Villa as Himself and the John Adams miniseries, in addition to an Oscar nomination for Inglourious Basterds and credits for Django Unchained, Shrek, Mission: Impossible II and The Twilight Saga: Breaking Dawn (Parts 1 & 2); David Gould, director of audio content solutions at Dolby Laboratories, where he is responsible for creating products and solutions for the audio post community in Atmos format; Michael Karagosian, whose experience with cinema sound dates from 1979’s Apocalypse Now with Dolby Laboratories and who represented the National Association of Theatre Owners (NATO) during the development of digital cinema; and Lon Neumann from Neumann Technologies, which offers audio training seminars, standards development, loudness measurement/management and surround-sound production/management.
Lon Neumann, left, Michael Karagosian, Greg P. Russell, Tony Lamberti, David Gould and Jim DeFilippis.
“The Atmos format’s ceiling speakers offer additional creative opportunities,” Russell continued. “Sound effects mixers — like me — can now use additional surround channels to place the audience within the action, including rain and thunder coming from above them, to create an even more exciting and realistic experience. And the use of full-range surround loudspeakers [for Atmos playback] means that we retain the same sound characteristic no matter where the sound is moved around the auditorium. This is very, very cool!”
“While there are very exciting opportunities available with full-range surround speakers," Lambert agreed, "the added ‘depth’ available from an Atmos mix really lets the re-recording mixer open up a soundtrack." "And the enhanced resolution of object-based channels lets me place the sound at precise locations within a playback environment,” Russell added. “We have enhanced clarity [from an Atmos mix] because sounds can be moved apart from one another” to extra playback speakers.
Because the Dolby CP850 Atmos-capable cinema processor is designed to deliver an object-based film soundtrack into appropriate loudspeaker channels, dependent upon the theatre size and speaker locations, it is sometimes argued that re-recording mixers are no longer in control of their mixes. “That is simply not true,” Dolby’s Gould emphasized. “Since the [Atmos soundtrack] is rendered specifically to accurately match the playback environment, and is scalable between different playback spaces,” while fully maintaining the mixer’s creative intentions and placement decisions made on the re-recording stage.
“A Native Atmos mix offers enhanced flexibility,” Gould stressed, “because assignment decisions can be made from the very beginning of the soundtrack project,” with less compromises to subsequent 5.1- and 7.1-channel versions. “An Atmos soundtrack is the optimum mix to be played back anywhere, and can be down mixed [in the theatre] to 5.1 or 7.1 with no tweaks; it just works.”
As well as outputting an Atmos object-based mix to a maximum of 64 speaker feeds, the CP850 cinema processor can also replay traditional 7.1 and 5.1 formats, plus render selected Dolby Atmos films to a 7.1 configuration. The majority of DCPs delivered to theatres also contain a dedicated 5.1 or 7.1 soundtrack mix, in addition to an immersive Atmos mix, if available. “We like to QC that mix down [on the stage],” Lamberti stressed, “but we are also leaving it up to the renderer,” further underscoring the “One Mix” release philosophy for film studios.
Michael Karagosian, left, Greg P. Russell and Tony Lamberti.
“Although Native Atmos mixes starting with a 9.1-channel bed are more fulfilling, I have worked from a 7-1-channel soundtrack and up-mixed to Atmos,” Russell argued. “Since all the elements are virtual, we can find extra sources and treat them as objects. But with only four days to handle an Atmos mix [as was the case during a recent project] we needed to be selective about what we used. And picture changes during a Native Atmos mix mean we need to re-conform the [Avid Pro Tools sessions]; a 7.1 mix offers an easier workflow.”
“We have pre-dubbed in Dolby Atmos and then made a temp pass to check that it works in 7.1,” added Lamberti. “Having done the 7.1 final we can then up-mix for an Atmos release.” “For [director Michael Bay’s recent] 13 Hours: The Secret Soldiers of Benghazi,” Russell recalled, “we started in 7.1-channel and prepared our Atmos soundtrack from that starting point, by lifting sound effects from the beds into the object channels.” “A rock-solid 5.1 mix represents an excellent starting point for an Atmos mix,” Lamberti said.
“Mixers can start laying out the soundtrack in Atmos but listen in 5.1 or 7.1-channel using the Monitor Mode option,” Gould explained. “Then, by selecting [Amos] mode, you can hear the changes with elevation moves, for example. Up-mixing 7.1 produces excellent results.”
Regarding other immersive formats, including IMAX, DTS MDA and Barco Auro, Russell explained that while he has yet to undertake a Native Auro mix, “For one project, we went from [object-based] Atmos to [channel-based] Auro, using the added height channels to retain the overall imagery. It was a pretty decent conversion,” he stated.
“The format choice often comes down to the post-production schedule,” Russell continued. “Will we have the time to do what we need to do?” “Those time pressures can be overwhelming,” Lamberti agreed. “We always want to serve our filmmakers with an awesome mix that supports the narrative, while creating sound movements within the mix.”
“We need to educate filmmakers about the ramifications of that time crunch,” Lamberti added. “After all, how many Atmos theatres are there out there? Will the majority of audiences get to hear our Atmos mixes; maybe we should mix for the primary market?” As Gould revealed, currently there are some 1,600 Atmos-capable screens around the world — out of a total population approaching 140,000 — with close to 400 Atmos theatrical titles announced or in release. There are reportedly more than 100 Atmos-equipped mixing facilities worldwide.
Of course, a single soundtrack mix could also deliver the filmmaker’s intent to consumer audiences; Dolby has announced that more than 60 Blu-Ray Discs featuring Atmos soundtracks are available globally, with 44 released or announced domestically, in addition to some 14 titles available on Vudu’s OTT steaming service.
“I’m a firm believer in being involved in that translation process from theatrical to home Atmos release,” Russell advised, “since the dynamic range for the home is very different from a movie theatre. The low-level parts need to be boosted a bit and the high levels reduced a little. For a [Home Atmos] mix for Spider-Man, mixed on near-field monitors at 80/82 dBA levels rather than the conventional 85 dBA we use for the theatrical mix; we also made a stereo mix on bookshelf speakers.”
Movie Atmos mixes can also be remastered relatively easily for consumer playback using the 7.1.4 Home Atmos format delivered via Blu-Ray or OTT services, Gould explained. “Using spatial coding for the audio tracks and the [companion] metadata, we can deliver immersive soundtracks to flexible home environments,” as well as tablets and steaming services such as Amazon Prime.
Media consultant Karagosian wondered how well immersive theatrical mixes translate to other playback formats. “My mixes have translated very well to Home Atmos,” Russell stated. “Although it’s all downstream, we need to QC the result and accept or reject the result. After all, given the opportunity, we want to make the best presentation in any format.”
“The shift to object-based mixing and higher channel counts is a challenge for theatrical mixers,” Gould said. “Mixers are more familiar with channel-based systems and often wonder how we can keep the metadata in sync with the 128 objects. But Dolby has put into place workflows for re-recording soundtracks that deliver an outstanding product, with content compatibility from editorial to the audience.”
As secretary of the SMPTE Working Group: Interoperability of Immersive Sound for Cinema, Neumann reported that seven different immersive/object-based formats have been identified for consideration: Dolby Atmos, Auro 3D, DTS MDA, USL, Higher Order Ambisonics, NHK 22.2 and Iosono. “Motion picture and home theaters are two, very different spaces,” he explained. “Cinema mixes have to be remastered for the consumer space — at the very least quality-checked for playback loudness, dynamic range, and imaging.” He also advised mixers to convert legacy mono and stereo soundtracks to a discrete 5.1-channel format so that they have a hard-wired center channel. “Turner Classics has done that with its entire mono black-and-white library” to ensure compatibility with delivery media. “Dialogue is where the movie exists for most home audiences.”
Audience member Gary Bourgeois, CAS,a veteran re-recording mixer who has worked on a number of films while at Todd-AO Hollywood and then Sony Pictures Studios, pointed out that budgeting considerations often affect the format in which a film will be mixed. “We need to involve post-production supervisors and sound designers in our discussions, because they often do not understand how much our workflows have and continuously keep changing,” he said. “During budget and scheduling discussions, we need to get away from labeling mix-stage days as Pre-dub or Finals; just simply ‘Stage Days.’ It can make a major difference to the end result.”
“Communication is the key,” Russell concluded. “What are we doing, and what is the best way to do it? Look at the film: Is there a location you want to stand out? That is an important dialogue among the creative team. Talk to the production mixer and secure tracks that are as wide as possible; sound separation is the key for us.”
Mel Lambert has been intimately involved with production industries on both sides of the Atlantic for more years than he cares to remember. He is principal of Content Creators, a Los Angeles-based copywriting and editorial service, and can be reached at email@example.com
. He is also a 30-year member of the UK’s National Union of Journalists.