{"id":247,"date":"2014-03-18T19:37:58","date_gmt":"2014-03-18T19:37:58","guid":{"rendered":"http:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/?p=247"},"modified":"2017-02-28T13:39:43","modified_gmt":"2017-02-28T13:39:43","slug":"interactive-and-dynamic-mixing","status":"publish","type":"post","link":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/2014\/03\/18\/interactive-and-dynamic-mixing\/","title":{"rendered":"Interactive and Dynamic Mixing"},"content":{"rendered":"<p>&#8216;Mixing&#8217; spreads itself across the whole life-cycle of any audio project, and involves an amalgam of all our perceptual, creative and technical skills. We&#8217;re all quite aware of how challenging mixing is (or, at least, can be) for fixed media situations but how are these challenges affected when we&#8217;re presented with a (literally) moving target?<\/p>\n<p>This session we&#8217;ll look at these challenges: what&#8217;s similar and distinct about mixing for dynamic situations, and what things to bear in mind. We&#8217;ll look a bit at a couple of approaches taken in the games industry, and finally have a look at some of FMOD&#8217;s facilities.<\/p>\n<h1>First Bit: An Exercise<\/h1>\n<p>You all have experiences mixing. As such it seems a bit redundant, not to mention inefficient, for me to stand at the front and tell you about it. Rather, we&#8217;re going to pool our experience.<\/p>\n<ol>\n<li>In groups, first consider mixing in the general sense. What are four priorities for making a &#8216;good&#8217; mix? What are four priorities for avoiding a &#8216;bad&#8217; mix?These priorities shouldn&#8217;t be qualities of what qualifies a good\/bad mix, rather I want to you think in practical terms about the sorts of thing you need to\u00a0<em>do\u00a0<\/em>(or not do) as you work.<\/li>\n<li>Still considering the general case: what sorts of\u00a0 knowledge do we bring to our practice as mixers? Look over your eight priorities from above, and jot down the different kinds of skill \/ knowledge required to enact them. Is it all just technical, or do specific forms of cultural, social, personal knowledge play a part? <em>Why am I asking you this? <\/em><\/li>\n<li>Now, agree between your group members five qualities of a good mix, in a general sense (any medium, any genre?)<\/li>\n<\/ol>\n<p>(Class discussion to follow)<\/p>\n<p>Back in to groups:<\/p>\n<ul>\n<li>What&#8217;s different for interactive \/ dynamic mixing? Agree on a list of (4-5) distinct challenges presented by this medium. (For those doing Sound and Fixed Media, it may be valuable to consider the kind of work you&#8217;ve been doing on that course and consider how it might need to change in an interactive context)<\/li>\n<li>What are the particular priorities for an effective game mix?<\/li>\n<\/ul>\n<p>(Some further discussion)<\/p>\n<h1>Dynamic Mixing Perspectives<\/h1>\n<p>A quick survey of perspectives and approaches to game mixing.<\/p>\n<p><a href=\"http:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1.png\" rel=\"attachment wp-att-350\"><img decoding=\"async\" loading=\"lazy\" class=\"alignleft size-full wp-image-350\" src=\"http:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1.png\" alt=\"ISE_dynamicMix_gameShot\" width=\"1525\" height=\"1145\" srcset=\"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1.png 1525w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-300x225.png 300w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-768x577.png 768w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-1024x769.png 1024w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-100x75.png 100w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-150x113.png 150w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-200x150.png 200w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-450x338.png 450w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-600x450.png 600w, https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/files\/2014\/03\/ISE_dynamicMix_gameShot-1-900x676.png 900w\" sizes=\"(max-width: 1525px) 100vw, 1525px\" \/><\/a><\/p>\n<p>This picture gives some indication of the mixing task that needs to be dealt with at a given <em>instant<\/em>. One helpful way of thinking about the job at hand comes from <a href=\"http:\/\/gamasutra.com\/view\/feature\/172660\/\" target=\"_blank\">Bridgett&#8217;s perspectives on Horizontal and Vertical mix elements<\/a>. Bridgett denotes &#8216;horizontal&#8217; elements of a mix as concerning the consistency between assets (e.g. loudness normalisation of dialogue), and their ability to fit. Vertical elements concern the moment-by-moment combination of assets in order to present the game narrative appropriately. (This perspective seems to me also applicable more generally to <em>asset-based<\/em> mixes where we&#8217;re building things from a collection of materials, e.g. much sample-based music).<\/p>\n<p>Considering, in particular, &#8216;vertical&#8217; elements we have to bear the following in mind:<\/p>\n<ul>\n<li>SFX \u2013 how do these fit into the wider sound design schema? what resources and sonic bandwidth are available? Often SFX are given a lower priority than the other uber-categories&#8230; choosing priority: what conveys essential information and what is purely for aesthetic\/immersive purposes&#8230;<\/li>\n<li>Interface sounds \u2013 are these clear enough above any other elements in order that their informational content is effectively delivered? Do they spoil immersion by being too disruptive?<\/li>\n<li>Dialogue \u2013 what frequency range, priority? spatialize? send to centre speaker?<\/li>\n<li>Music \u2013 eg. level of intensity \u2013 what is happening when this cue is playing back? Can different layers be called or excluded based on other elements? Will the player be allowed to substitute their own music into the game?<\/li>\n<li>Switching components of a sound based on various criteria eg. distance from the listener object, switching tails \/ indoor\/ outdoor<\/li>\n<li>On a global level, usually attenuation and occasionally boosting of levels\/frequency ranges based on priority, state (e.g.. night, day, player health etc), resources, player location, object location etc \u2013 ducking, dynamic filtering and multi-band compression, side-chaining<\/li>\n<li>Depending on the platform some sound assets might be discarded so its important for a mix to work even at its bare minimum<\/li>\n<\/ul>\n<h2>General Techniques<\/h2>\n<ul>\n<li>&#8216;Normal&#8217; asset preparation (this is where mixing starts!)<\/li>\n<li>Conventional &#8216;linear&#8217; techniques<\/li>\n<li>Location-based mixing<\/li>\n<li>Dynamic range based manipulations<\/li>\n<\/ul>\n<h2>Dynamics<\/h2>\n<ul>\n<li>Straightforward &#8216;side-chaining&#8217; (actually, external keying for those of us who are pedantic)<\/li>\n<li>Meta-data tagging<\/li>\n<li>Snapshots, priority based systems<\/li>\n<\/ul>\n<p>The &#8216;High Dynamic Range&#8217; approach from Dice uses a tagging system to mark assets with a nominal loudness. These are given priority at runtime, and other elements are simply muted to reduce clutter:<\/p>\n<p><iframe loading=\"lazy\" style=\"border: 1px solid #CCC; border-width: 1px; margin-bottom: 5px; max-width: 100%;\" src=\"\/\/www.slideshare.net\/slideshow\/embed_code\/key\/k9cwkolLnd00Tu\" width=\"595\" height=\"485\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\" allowfullscreen=\"allowfullscreen\"> <\/iframe><\/p>\n<div style=\"margin-bottom: 5px;\"><strong> <a title=\"How High Dynamic Range Audio Makes Battlefield: Bad Company Go BOOM\" href=\"\/\/www.slideshare.net\/aclerwall\/how-high-dynamic-range-audio-makes-battlefield-bad-company-go-boom-1292018\" target=\"_blank\">How High Dynamic Range Audio Makes Battlefield: Bad Company Go BOOM<\/a> <\/strong> from <strong><a href=\"\/\/www.slideshare.net\/aclerwall\" target=\"_blank\">Anders Clerwall<\/a><\/strong><\/div>\n<p>&nbsp;<\/p>\n<p><iframe loading=\"lazy\" width=\"605\" height=\"340\" src=\"https:\/\/www.youtube.com\/embed\/nKDzklyIHHM?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p>See <a href=\"http:\/\/www.gameaudiopodcast.com\/?p=210\">here<\/a> for a really interesting podcast discussion about the challenges \/ techniques of dynamic mixing.<\/p>\n<h1>Dynamic Mixing and FMOD<\/h1>\n<p>Most of you have encountered some of the tools FMOD offers, especially the use of distance for location based mixing, and custom parameters to achieve more complex states (e.g. different sonic scenes at different elevations).<\/p>\n<p>FMOD has some additional features though:<\/p>\n<h3>Live Mixing<\/h3>\n<p>A cycle of run, tweak, build, refresh, run&#8230; is a pretty slow way to work. Fortunately we can connect directly to the game&#8230;<\/p>\n<p>Also, we can connect a control surface (that speaks Mackie Control) and actually get some faders under our fingers. (In the podcast above, Rob Bridgett discusses how important this was when they were working with Randy Thom on <em>Scarface<\/em>)<\/p>\n<h3>Loudness Metering<\/h3>\n<p>FMOD has a ITU 1770 \/ EBU R128 meter for &#8216;loudness&#8217; metering of your material. For an explanation of this technology, see <a title=\"Meters, Mixing Levels, Loudness (and the new ITU spec)\" href=\"http:\/\/designingsound.org\/2013\/02\/loudness-and-metering-part-1\/\">this. <\/a><\/p>\n<p>Bridgett&#8217;s article above has the following quite informative selection of loudness levels that they worked to:<\/p>\n<blockquote><p>The long term metering expressed in the ITU spec was developed for broadcast program content, and suits &#8216;predictable&#8217; program content usually around one hour or half an hour in duration, and in this respect a single long-term number can&#8217;t be applied as easily to video games that have indeterminate lengths and unpredictable content.<\/p>\n<p>The strategy we adopted wasn&#8217;t a conscious one, but more an observation based on what sounded good. We noticed different sections of the game naturally pooling into different loudness ranges by simply mixing to what sounded right. We noted a &#8216;range&#8217; of long term loudness measurements which can be anywhere between -13 and -23 LU, based on the nature of the action.<\/p>\n<ul>\n<li>story cutscenes -23<\/li>\n<li>in-game cut scenes -19<\/li>\n<li>background ambience -23.<\/li>\n<li>&#8216;Talky&#8217; missions, or &#8216;stealth&#8217; missions between -23 and -19 depending on context.<\/li>\n<li>Action missions -19<\/li>\n<li>Insane action missions -13<\/li>\n<\/ul>\n<p>These numbers are what we are calling, for now, our &#8220;Long Term Dynamic-Range&#8221;, a grouping of loudness measurements that apply to certain types of game play or presentation elements, in our case -13 to -23. In essence I think what is required here is a method of measuring all these different aspects of the game within a short time window, around 30 mins to make a single overall loudness measurement useful. This is not always an easy approach when the game isn&#8217;t structured in that way.<\/p>\n<p style=\"text-align: right;\">(<a href=\"http:\/\/gamasutra.com\/view\/feature\/172660\/notes_from_the_mix_prototype_2.php?page=4\">gamasutra.com\/view\/feature\/172660\/notes_from_the_mix_prototype_2.php?page=4<\/a>)<\/p>\n<\/blockquote>\n<p style=\"text-align: left;\">Elsewhere, Bridgett discusses how he used different loudness profiles for a kids&#8217; game in order to try and mitigate the danger of headphone use: <a href=\"http:\/\/gamasutra.com\/blogs\/RobBridgett\/20140405\/208210\/ReImagining_the_Sound_of_PreSchool_Games.php?print=1\">gamasutra.com\/blogs\/RobBridgett\/20140405\/208210\/ReImagining_the_Sound_of_PreSchool_Games.php?print=1<\/a><\/p>\n<h3>Submixes, Groups and VCAs<\/h3>\n<p>Once our projects get beyond a certain size (dynamic or not) we need to make the number of things to manage copeable-with.<\/p>\n<ul>\n<li>We can make sub-mixes within events by routing audio away from the event master<\/li>\n<li>FMOD has a very flexible grouping \/ bussing mechanism<\/li>\n<li>VCAs give you an additional layer on top of groups that sit outside the rest of the routing system<\/li>\n<\/ul>\n<p>Some of you may already have watched Jules&#8217; video on this:<\/p>\n<p><iframe loading=\"lazy\" src=\"https:\/\/player.vimeo.com\/video\/89434247\" width=\"605\" height=\"340\" frameborder=\"0\" webkitallowfullscreen mozallowfullscreen allowfullscreen><\/iframe><\/p>\n<h3>Snapshots<\/h3>\n<p>These are more powerful than they sound! Snapshots allow us to store and recall particular mixer settings, including arbitrary parameters<\/p>\n<ul>\n<li>Anything that is to be included in a snapshot has to be manually &#8216;scoped-in&#8217;. That is, nothing is included by default. This gives us a high degree of flexibility<\/li>\n<li>Snapshots are <em>attached to events. <\/em>This might seem counter-intuitive, but is very powerful indeed<\/li>\n<li>As a consequence, multiple snapshots can be in effect at any given moment<\/li>\n<li>FMOD then does behind the scenes magic to run a prioritising system, based on which snapshots have greater priority (denoted by their place in the snapshot list)<\/li>\n<li>Most commonly this would be used for ducking, but all kinds of creative possibilities are available<\/li>\n<\/ul>\n<p>These tutorial videos are useful. Even though done in the context of UE4, most of the action is in FMOD:<\/p>\n<p><iframe loading=\"lazy\" width=\"605\" height=\"340\" src=\"https:\/\/www.youtube.com\/embed\/TJCqnuC8HDQ?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<p><iframe loading=\"lazy\" width=\"605\" height=\"340\" src=\"https:\/\/www.youtube.com\/embed\/3KK9PIm3MEg?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<h1>Now, go play&#8230;<\/h1>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>&#8216;Mixing&#8217; spreads itself across the whole life-cycle of any audio project, and involves an amalgam of all our perceptual, creative and technical skills. We&#8217;re all quite aware of how challenging mixing is (or, at least, can be) for fixed media &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"more-link\" href=\"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/2014\/03\/18\/interactive-and-dynamic-mixing\/\"> <span class=\"screen-reader-text\">Interactive and Dynamic Mixing<\/span> Read More &raquo;<\/a><\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/posts\/247"}],"collection":[{"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/comments?post=247"}],"version-history":[{"count":16,"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/posts\/247\/revisions"}],"predecessor-version":[{"id":362,"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/posts\/247\/revisions\/362"}],"wp:attachment":[{"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/media?parent=247"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/categories?post=247"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/digital.eca.ed.ac.uk\/interactivesoundenvironments\/wp-json\/wp\/v2\/tags?post=247"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}