Adaptive Music with Elias – Tutorial Series – Part 1

(This article was previously published on DesigningMusicNOW on September 17,2018. For full disclosure, this article has been republished with permission from Scott Looney by Dale Crowley who currently works for Elias)

Adaptive music has been a staple of AAA dynamic game audio for more than two decades, when it was used in LucasArts games such as Monkey Island 2. Featuring the in-house iMUSE music system, combined with MIDI data, and downloadable sound banks, this game achieved a notable level of musical immersion, with flexible responsiveness and organic development that has only recently been equaled. The emergence of third party audio middleware around that time has since slowly developed in sophistication to the level where adaptive audio creation is now easier and more affordable than ever, yet middleware tools written specifically for composers are still somewhat lacking in terms of features, flexibility, and usability.

The Elias Adaptive Music Engine aims to correct this oversight using terminology and functionality familiar to composers who are experienced with adaptive music creation. In this article, we’ll introduce you the concepts that Elias uses for organizing various elements in the project, and we’ll also look into recreating the basic structure in the Elias Official Demo Theme, by breaking it down into its various components. In future articles we will dive into the sophisticated and cutting edge way that Elias handles MIDI, motifs and more.   Like a well designed video game, Elias is easy to learn but takes time to master.  Hopefully this article will start you on an exciting journey of adding adaptive music to your games.  Let’s get started!

Getting Elias Studio and the Demo project

Before we get started, there are a few things you will need.  Those are the Elias Studio, The Official Theme, and the Official Theme Re-Creation.

For directions on downloading and installing Elias Studio, watch this video.

Here is a walkthrough of all the steps detailed in this article which you can use  as a companion as you step through them:

Also, on the website, under the Resources heading, you’ll find the Tutorial section with many other example videos to help you out – and of course don’t forget the full user guide available from the site as well as the downloaded package.

Elias Conceptual Framework 

As stated earlier Elias is a tool for configuring and arranging adaptive music. This means that a composer or editor will generally have already recorded and produced the music content in question and need only bounce it out in their DAW to be able to use it in Elias Studio, which arranges and configures the existing content. To do this, Elias uses a hierarchy of terms that we will become familiar with. In order of size these are Project, Theme, Track, Level, Variation and Segment. Let’s break these down in order:

UI Overview

To get started we’re going to open the Official Elias Audio Demo, so you’ll need to uncompress the zip file called ‘official-elias-demo’ which you can get from that link. Next, open up Elias Studio and we’ll open that Project file to introduce you to the overall UI and look and feel.

To open a Project, click the Open Project button, then browse for the folder you just uncompressed. Inside are two subfolders. You’ll want the one labeled ‘Official Elias Audio Demo’ and then inside that folder is the project file you’ll want to open. You should probably see something like this when it opens the first time:

The Project window has a few significant subcomponents. We’ll cover the major areas of focus:

Each of these tabs can be independently broken out and placed if desired, and the desired window layouts saved. For now we’ll stick with this more compact approach, but if you accidentally end up with windows in undesired places you can reset your layout by going to Windows > Reset Layout – or just use Cmd Shift-L.

Setting Up The New Project

To download a simplified Official Demo that we are going to be using for these tutorials, click here:  Elias Official Demo - Recreation

Note that this project is not complete, but it contains all the files you will need to complete the steps in this blog.  It also has all the audio files organized in such a way that it makes it easier to rebuild the project from scratch.

For this new one, you’ll select New Project and have the option to name your project in a particular location. Desktop is a good place to put it. For this, we should create a new folder so click the New Folder button on the left and Name your folder. Here I named mine ‘Elias Demo Re-Creation’.

When you create this project, note that Elias by default only creates the meproj file – it does not create a folder for audio files. That is left up to the user to further organize as they see fit.

For this series we’re going to to be using the Official Elias audio demo, and its files have been presorted into previously created folders in order to more easily be able to find and organize all the files needed. So, we need to copy the audio files from the Official Audio Demo folder to the location of your new project where we can import them. Switch to the Finder (Mac) or Explorer (PC) and find the folder you uncompressed. Find the Official Elias Audio Demo folder and inside there is a folder named ‘audio’. Right click on this to copy and then find the folder where you created your project, right click and select Paste.

Let’s take a look at how things are organized. Open the folder with your new project and you should see something like this:

Note how the files are organized by Theme first and then Loop and Stinger Tracks afterwards. This will hopefully help in terms of keeping the project together and help you to more quickly find files to use in each Theme’s Loop and Stinger Tracks.

Create Themes

Okay we’re ready to get going on this re-creation. First, we’ll need to create our Themes, which are our largest and most important organizational element in any Elias project. The Theme is the main musical building block consisting of one or more Tracks, and only one Theme can be active at a time generally. Themes have a specific bar length, tempo, and time signature associated with them. You also cannot currently create tempo maps or have changing time signatures within a single Theme.

Click on the window for the new project you created and go to the Theme menu to create a New Theme. Name this Theme ‘Objective’. Do this again to create another Theme called ‘Objective Battle’.

Next go back to the Theme menu select the desired Theme and choose Theme Settings. You’ll set the settings for each theme, so you’ll be doing this twice.

The Theme settings display a length in bars or measures, a time signature and a tempo in BPM. Set your Themes to the following settings:

Note that if you change the length, you may get a confirmation dialog explaining that you are changing the structure of your Theme, which is a somewhat significant action and will change a number of other settings and timings of sub-elements like Stingers. Thus it’s best to set the length early on and not change it too much if possible later on – especially if you shorten the length. For example if you had a 32 bar Theme with a Stinger that was playing at bar 25 and then shortened the Theme length to 24 bar, that Stinger’s settings would be void and may have to be set again from scratch.

Adding Tracks

Go to the Themes menu and select the Objective Theme. Now we can get to adding Tracks to our Themes. As stated before, Tracks are the main organizational element within a Theme and there are four types of these:

To create Tracks go to the Track menu and select Add New Track:

Notice that each Track allows Solo and Mute options. The icon on the right side indicates that the Track is a Audio Track. MIDI Tracks would have a different icon.

Next, create and name the following Audio Stinger Tracks:‘1 Bar Pickup Stinger, ‘Hits’, ‘Tonal Brass Stinger’, ‘Level Complete’,’Game Over’ and ‘Theme Transition Stinger’.

Note that you can resize the width of any Track header on the right side, so as to see its title or Level Variations below more clearly if needed. Your Stinger Tracks should look something like this when finished:

Adding Variations To Tracks

Now we get to adding our Variations. To simplify things we’re just going to start with the Loop Tracks and to simply focus on one Track for the moment, the Strings track. To add a Variation, all you need to do is to drag over the sound file you want to the Level slot on that particular Track you want. But seeing as how there are quite a few files to look through, let’s look at the contents of the audio folder we copied from the Official Demo.

Since we’re on the Objective Theme and we’re filling a Loop Track we’ll be looking here in the ‘LoopTracks’ folder for what we want. We’ll need a file called ‘Strings 1 p’  so search for that, and when you’ve found it, drag it over the first slot on the Strings track like so:

(make gif of dragging and placing clip as Variation)

Congrats! You just created a Single Variation, the most basic and common type of all Variations. You can right click to Audition the file, make it Silent, or Clear the contents of the Variation, leaving it blank.

Now do this again, this time we’ll use the ‘Strings 2 mf’ file in that same subfolder. Drag this over to the slot immediately below the occupied slot. Your track should look like this when finished, with the two variations on top of each other:

Let’s do one more. Find the ‘Strings 4 mf melody’ file, and drag it over the Level 4 slot. Your track should now look like this:

Note that I’ve manually resized the track width as well to expose the file name and relative path of each of these Variations as well as the Level indicator on the left side. This can easily be done by clicking the triangle icon on the Level. Any active Variations on a Track at that Level will be visible.

This is also a good time to point out the paramount importance of file naming and organization when working with Elias Studio. Since this demo consists of nearly 100 sound files, if we had named any of these files generically like ‘Audio 1’ or something like that, we would not know its content without having to listen to those files, and also note that by naming files in the Finder or Explorer after the fact we would change its reference in Elias Studio and it would no longer work properly when re-opened. This restriction is the same if we were to create folders in the Finder or Explorer after we imported our files, to organize them. So the Pro tip here is to have all the naming and folder organization done before you import these into Elias Studio.

Let’s hear what this sounds like using the Player section (that’s Section B in our UI screenshot)

For this we’ll use the green playback button in the upper right corner to start playback (or just press spacebar), and when we do this we’ll hear our Strings playing at Level 1. Now, while the track is playing, look at the bottom of the area and you’ll see a long slider and a number on the right.

That’s the level number that we’re playing. Change this value to 2 and click the (Re-)Activate Level button. You can also press PageUp key on your keyboard, or drag the slider. You should hear an immediate change. While the change in question is the correct segment, it does so quickly and without regard to the tempo. We don’t want that in this case. We want a gradual change instead of an immediate one, and to do this we’ll click on the Strings Track header and look in the Inspector to see its settings.

The settings you can configure are one of the things that make the Elias Engine so flexible. For now we’re just going to focus on the Agility settings and the Fade Settings. The rest of these settings are covered thoroughly in the User Guide.

Agility Settings – this covers exactly when Elias Studio will trigger the Variation once its Level is active. You can set it to Anytime(immediately), on the Beat, on the Bar, or at a Custom time. For these String clips the composer needs them to be triggered in every other bar, so we’ll need to select Custom for this Track. When this is done you’ll see a window where you can insert the exact bars you want to trigger the clips.

To add a point, click the Add button. You’ll see the first entry showing up as 1:1. The indicator above is what bar and beat you’ll set, and when you click the Add button the value will show up. The screen below shows that I’ve set up the first four trigger points at Bar 1,3,5 and 7, all on Beat 1 of the bar. Keep adding these until you’ve got a list with the following progression: 1:1,3:1,5:1,7:1,9:1,11:1,13:1,15:1,17:1,21:1,25:1,29:1. Note that those last three starting from Bar 17 are four bars apart, not two. Obviously you can see this requires very intimate knowledge of the music in advance, but you can also try different settings to see what works best for you.

With this set let’s look further down at the Fade Settings. These are fairly easy to determine the function of. Enable this and set it to the following:

In this case, Fade In means if the previous Level was silent it would fade in in 100 ms. If the transition is between two clips of audio it will crossfade over 1000 ms, and if the next Level is silent, it will Fade Out in 100 ms.  

All right, let’s go back and listen to the music again at Level 1, and this time slowly keep changing our Level to Level 3. Notice that our transitions are much smoother and more well placed, and that both adjacent Levels flash during the transition period and turn solid when the next Level is playing. Also notice what happens at Level 3. We’re still hearing the Level 2 music. Now change to Level 4 and listen to that, then switch back to Level 3. Notice that you’re still hearing Level 4’s music.

This brings up a useful point. Blank Level Variations are not silent. They will usually contain the music of the Level that was most recently triggered. However if you want a blank Variation to be silent you can right click and select Make Silent. Try this out. The Variation will be red colored. At that point Level 3 will in fact be silent at all times when the Level arrives, so you’ll get to hear the more immediate Fade Out and Fade In time. To undo this, select that Level, right click again and choose Clear.

Level selection on blank Level Variations is actually a pretty complex thing in Elias Studio, because each Track can have its own Level Selection Strategy in the Track Inspector – they don’t all have to change to the same Level in the same way.

Let’s quickly cover the four major selection methods available:

For more info on this, please consult the User Guide.

Multiple Variations

We’ve covered Single and Silent Variations, and now we’ll look a Multiple Variation. To do this we’ll first switch to the Choir Loop Track by clicking its header, then look at your Track Settings in the Inspector and set them to the following:

Next, we need to bring in audio elements to our Choir Loop Track. First, look in the LoopTracks folder under the ObjectiveTheme folder. We need to find the Choir loops shown below:

Drag ‘Choir 1 Boy’ to Level 1, and ‘Choir 3 Boy and Women’ to Level 3. Next, drag ‘Choir 5 plus flute V2’ and ‘Choir 5 plus flute’ to Level 5. That’s right – you’re going to drag two files over the same Level location. Do this one file at a time, dragging the second file directly over the slot occupied by the first file.  Once you do this, the Variation will turn into a Multiple Variation and be colored blue. You can also expand the level before dragging, and then drag the two files over the darker area below the level. If you do this correctly, the two files will become variations. If not, they will be dropped into two adjacent levels. You can also edit this behavior in your Studio Settings, as shown here (Edit->Studio Settings):

By default, “Add across levels” is selected.  This can be overridden when you expand the level – Elias simply detects that and adds them to new levels.

Once you have added the clips to the level, on the far left of the level text, a number will be shown, indicating the number of items in this level.  In our case, this should look like the image below, with the <2> in front of the level name. You can use the arrow on the left side of the Track Loops window to see the contents of any Multiple Variation.

When Elias gets to a Level with a Multiple Variation on it, it will select a Level based on the Progression Mode settings, which we’ll look at here, just below the Level Selection Strategy:

These can be set in one of three different ways:

Let’s work on the Brass Track next. First though, let’s go over to the Strings track and we’ll look at another feature for Track Agility Settings. You recall the Custom Agility settings we did for the Strings? We are going to need something like that on the Brass Track. However, rather than having to enter all of our data from scratch, we can start from a copy of the Custom data from another Track and then edit it. All we need to do is click the Copy button on the side and we can then switch to the other Track and on its Custom settings, click Paste.

As mentioned, the Brass Track has different Custom Agility Settings. These should be as follows:  1:1,2:1,3:1,5:1,7:1,9:1,11:1,13:1,17:1,19:1,21:1,23:1,25:1,27:1,29:1. Hopefully the editing of these should be easier starting from the Strings settings, though as you can see they’re clearly not identical. Fade Settings should be set as follows:

Once you’re finished find the ‘Brass Horn melody’ file and drag it over Level 5. Here’s what the first three Tracks should look like up to Level 5:

Okay we’re in the home stretch. We have one more Variation type to cover and that’s the Segmented Variation. We’ll switch to the Percussion Track for this one. You should have already created this track earlier, but if not use Cmd-T(Ctrl-T on PC) to create a Loop Track called ‘Percussion’. Set the Track Settings in the Inspector as shown here:

Okay we’re ready to add our Segmented Variation. To do this, we don’t actually drag our file over the Level in question as we did for Single and Multiple Variations. Instead we use the Segment View located at the bottom as area D in our overview screenshot:

The Segmented Variation is useful when a particular timed arrangement or order of clips is desired. To start this out, find Level 6 on the Percussion Track and click on it. You’ll see a frame will highlight around this Level.

Next find the ‘Perc 6 S 4bars b1 b5 b17.ogg’ file and drag it over the Segment View. be sure to release the cursor at the location you want. It’s a bit finicky but you can move the position later on. You’ll get a dialog asking you to fill in some details:

Notice you cannot edit the Start Bar – it’s determined by where your cursor was when you released it. You do have to indicate the length of the clip, which in this case is 4 bars long (lucky it’s in the filename as well), and we only want it played one time so we set the repeat count to 1. The End bar can also not be edited, but it takes into account both the length and the Repeat count. Click OK and here’s what you should see:

You can move the location of this segment to bar 1 but be careful – if you drag it too far left there’s a bug that will make the placement invalid and it will turn your Variation Silent. Just get the left edge of the segment as close to Bar 1 as possible. Hopefully this interaction will be more robust in the future, but for now just be wary of Segments starting right at Bar 1.

Do this action again with the same file at Bar 7, and again at Bar 17. This gives you three copies of the same file placed as segments in various places.

Last we’re going to find the ‘Perc 6 S 12bars b21’ and drag this over to Bar 21 right after the last placed Segment. So, overall you should have the following 4 segments as shown:

Also notice that the Percussion track’s Level 6 slot shows <1>: for a Single Variation and the next number is the number of Segments in that Variation. Notice also that the Level is colored yellow instead of green, blue or red. Any yellow colored Level slots are automatically Segmented Variations.

For the last bit we’ll look at the last Track here called ‘Other’. Click its header and change the Inspector settings as shown:

Last, find the ‘Other 1 Low Pad’ file and drag it over Level 1. Your Loop Tracks should look like this up to Level 6:

Give this a playback and play around with Level changes and you’ll see how it’s already beginning to sound a bit like music!

Exporting Your Work From Elias Studio

We’ll be covering this in more detail later, but one of the strengths of Elias is that you can export the project and even convert and rename your files, and then archive everything into a compact zip file. To do this, go up and look at the icons in the Toolbar. Look for the rocket ship icon.

You’ll get this dialog:

From here you can prepare the project for export. In the top section you can browse to add additional files like directions for the integration/programming team, asset lists, or other things more external to the project that may be useful. Under the Export settings you can Zip the entire project (as long as it’s under 2GB of data). The Rename tracks option will rename all files used in the project based on a numbered naming convention, and should be avoided for now. Lastly, you can export in either WAV (uncompressed) format or Ogg Vorbis, using a selectable bitrate. Note that the exported project will only have the files or clips that are used in a Variation on a Level, and the folder organization you saw earlier will be lost. It will instead place all items under the ‘audio’ folder. We’ll look at bit more in depth at this in the future.

So we’ve just given you a brief exposure to the inside workings of Elias Studio and you can already see what a flexible and versatile tool it is for game composers. Never fear though – we’ll be back to flesh out many more features in the next installment. We’ll be covering Stingers, Action Presets and much more. Stay tuned!

About the Author

Scott R. Looney has been educating students and teachers alike about game audio for over 7 years. He is the co-founder of the Game Audio Institute (GAI), seeking to de-mystify the process and methods behind interactive and adaptive sound and music, as well as co-author of the Essential Guide To Game Audio (2014 published by Focal Press). Follow GAI for exciting news and announcements in the coming months! www.gameaudioinstitute.com