Game Audio 101 - The Insiders Guide to Music and Sound for Mobile Games

The Insider’s Guide to Music and Sound for Mobile Games

Written by B. Long Edited by Devin Monnens and Lorenz Rychner

The author makes no representation or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaims any implied warranties of merchantability or fitness for any particular purpose and shall in no event be liable for any loss of profit or any other commercial damage, including but not limited to special, incidental, consequential, or other damages.

Trademarks: This book identifies product names and services known to be trademarks, registered trademarks, or service marks of their respective holders. They are used throughout this book in an editorial fashion only. In addition, terms suspected of being trademarks, registered trademarks, or service marks have been appropriately capitalized, although the author cannot attest to the accuracy of this information. Use of a term in this book should not be regarded as affecting the validity of any trademark, registered trademark, or service mark. The Author is not associated with any product or vendor mentioned in this book.

https://www.gameaudio101.com

Table of Contents Introduction...............................................................................4

Sector 1 What the Beep?

The Mobile Landscape.............................................................6 Applitainment............................................................................8 Game Sound Basics................................................................9 A Brief History of Mobile.........................................................10 Back to the Future – MIDI 101...............................................11 Multi Platform Games in 2007................................................13

Sector 2 The iRevolution – Apple Development Apple Raises the Bar.............................................................14 Puzzle Games: “Gravity Sling”...............................................16 Kids Games: “Payload Extreme”............................................18 Edu Games: “8 Planets”.........................................................21 Casual Games: “Backflip Slots”.............................................23 Angry Birds: Making a Mega-Hit............................................26 DrumKit: Inside Music apps...................................................29

Sector 3 Everything Else!

WP7 Development “The Harvest”..........................................31 WP7 Development “Retrospaced”.........................................37 The #1 Secret to Mixing for Mobile........................................38 Android Development tips......................................................39 Breaking Into Game Audio.....................................................41 Resources for Musicians and Developers..............................42

Introduction When I tell people that my job is creating music and sounds for video games, their responses range from saying "That’s neat" to just staring blankly. This is usually followed by “how did you get into that?” It's a long story, but my response is: "I have musical A.D.D. and the video game industry was a perfect fit.”

The truth is, for years I worked IT jobs during the day and played live gigs at night.

This was my life for a solid 10 years. My very first 'studio' consisted of the Roland VS880, which was a digital 8-track with a 2-inch (non-backlit) LCD screen. Mixing on this thing was like brushing your teeth with a tree branch, but hey, it was digital baby! and this was back when a CD burner was $600.

I got used to being surrounded by bleeding-edge technology and noticed that the game industry was leading the pack. Being a lifelong gamer, it wasn't long before I had my first gig creating MIDI pirate music….the rest is history.

This book reveals the challenges one faces when trying to make a game sound awesome. It is often these challenges that bring out the best results.

The last few years have seen seismic shifts in mobile technology and I’m fortunate to have worked with some of the world’s most talented game developers during this period. There have been many projects where I thought “hey, I should write about this” so here it is. I hope to shed some light on the various challenges mobile game developers face these days.

Creating audio for games is, in my opinion, the highest form of sonic expression.

Nowhere else in audio production do you see this kind of creative freedom. Game Audio is more challenging than any other form of music or sound creation. Why? Because you watch your favorite movie maybe 5 times tops, where as a favorite game can easily be played hundreds of times. This means the audio must not only entertain, but it must do it again and again for the same person. This is what I call ‘sonic appeal’ Trying to get mobile phones to “sing” the same song can be a lot of work. It’s important to understand how incredibly difficult it was to create audio for a mobile game a few years back. Many of the challenges faced in video game development in the 1980s came full circle and returned to mobile. Creating custom music, sound and dialogue content for video games requires patience and a willingness to work within technical limitations. It’s the ultimate marriage of cutting-edge technology and boundless creativity.

This guide was written not only for composers and sound designers but also game developers. Not everyone has a sound designer or composer at their disposal. In many cases a development team has to tackle game audio on their own and take the DIY (Do It Yourself) approach.

Can creativity be taught? That’s a tough question. However, you can stumble into creative situations by simply letting go of all expectations and allowing ‘accidents’ to happen – this basically means experimenting while listening closely.

( ( (( turn up your ears)) ) )

Sector 1 - What the Beep?

The Mobile Landscape The mobile industry moves fast, and games are often developed at breakneck speeds. Audio is often overlooked as developers race to release their product. Think about how long it takes for a next-gen console to come out? Years. Mobile

devices, on the other hand, seem to hit the market every week with the promise of greater capabilities. The smartphone has indeed become a magical device, introducing games to a whole new audience. It seems the smartphone may soon outpace the toothbrush as a standard household item!

There are a multitude of mobile devices on the market, but as far as games are concerned, the platform usually falls in one these flavors: iPhone iOS (Apple) - The iPhone, iPod-Touch and the iPad account for the vast majority of games and apps being downloaded today. This OS was released in 2007 with the iPhone and has introduced mobile games to the masses. Apple has gone to great lengths to make the entire audio experience smooth and appealing. For example, when quitting an app, you’ll notice upon exit that the sound fades out smoothly to silence. This close attention to design detail is just another reason why the iPhone has been so well received.

Android OS (Google) - Android was released in 2007 and is quickly saturating the market, accounting for the majority of new phones being sold today. Android has a real future in the gaming realm. Google actually handed out free Droid and Nexus One phones to all attendees of GDC 2010 to get them more involved with the platform. It’s certainly not going away, in fact, it has already eclipsed the iPhone in total devices sold.

Windows Phone 7 (Microsoft) – Windows Mobile first hit the scene in 2003 and has evolved into the current version which is part of the Windows Phone 7 line of devices.

Although, the phone and platform are brand new, features like Xbox-Live integration hold great promise. Sound is obviously important to them and this is evident in the latest release of the HTC 7 Surround phone, which features “Dolby Mobile”.

SymbianOS (Nokia) – Phones based on Symbian OS account for around 46.9% of all mobile phone sales but this number is rapidly dropping. These are usually the “free” phones included with a basic mobile service plan. These tiny casual games still provide great entertainment, although they seem crude in comparison to today’s polished smartphone games. These feature low-res graphics and sound, mainly due to the severe lack of space, system resources and computing power.

BlackberryOS (RIM) – Blackberry started around 1998 and now has a loyal following, who affectionately refer to them as “crackberries” due to their addictive trackball and ease of use. These phones tend to cater more to the business crowd and have a weak presence in the gaming world. Maybe this is why RIM now has a booth at GDC and other gaming events.

Sure, there are others like MeeGo and Bada, but it’s way too early to be talking about those. Here’s a quick look at the predicted revenue for apps associated with each OS:

It doesn’t take a pie chart to see that the iPhone is dominating the industry!

The truth is that all mobile gaming platforms need quality audio if they hope to compete in the growing marketplace. This content can be menu sounds, music, ambience and dialogue content. Developers are primarily occupied with the code, art and design, which is a huge task in itself. Most game developers outsource the production of new music and sound to a professional composer or sound designer. They know that it will make their game that much more unique. There is a reason why every movie has a Composer, Sound Designer and Audio Engineer, etc….They require an intensely unique experience and original audio plays a major role. The basic idea you’ll hear throughout this book is: “If there are moving images on a screen, people expect to hear something as well.”

The reason for this is simple: Everyone grew up watching Film and TV, which always has music, sound and voice to reinforce the visual realism. Nothing in life is silent either. The closest I’ve come to hearing silence was in the middle of a desert, but even that had a sound! Sure, dead silence has its place in interactive media, but for the most part…games need audio.

Applitainment The line between games and apps has been blurred. Now we have music apps and “non-game” entertainment apps. If it has the element of interactivity, then it can benefit from sound. The lessons learned in mobile game development apply to all apps. Applitainment is a word I use to describe how even the most boring applications now have an element of entertainment. Ever notice

how even the most boring ‘productivity’ apps now have slick graphics and sound to enhance the user experience? All of this came from the game industry. Consumers these days expect apps to have captivating sights and sounds. Developers are taking advantage of better graphics and audio to heighten the appeal of their apps.

There is an endless list of non-game apps that are prime targets for original audio. A few that come to mind are astronomy, educational, exercise, bird watching and the list goes on. These products use everything from bird chirps to dialogue to swirling soundscapes and music. That’s what it takes to compete in the growing marketplace.

Traditionally, audio was an afterthought to game developers and a last-minute affair, but today’s gamer simply expects more. Some developers get lucky by “throwing in some sounds” at the last minute, but even that requires a musical ear. A little more time spent listening can be a big help.

The main challenge with “canned” music and sfx is that other developers can buy those same tracks. However, if you’re a solo developer with no audio team, this option can be very convenient. Sometimes developers find content that works but maybe the volume is too loud. In this case, the volume can be ‘turned down’ in the code. Small changes like this can make a HUGE impact and heighten the playability of a game.

Remember that a game should sound good on multiple listens. Since pre-existing content cannot be changed, we are limited to basic volume adjustments and other manipulations via the game engine. One interesting thing developers can do is pitch shifting the file from within the game code. If you’re a programmer using the Core Audio engine, then be sure to get the book: “Core Audio” by Kevin Avila and Chris Adamson.

Of course, most developers realize that their game should have a unique sound which can’t be heard anywhere else. This option gives them much more control over the final sound. Revisions to the music or sounds can be requested and this means they can get several versions to choose from. These final selections can then be tweaked until everyone is happy.

Game Sound Basics

1. Splash screen at launch

2. Game Title Screen

3. Menu Screen

The very first opportunity for audible content appears at game launch. When an app launches, sometimes a ‘splash screen’ will display the company logo. Often times,

you’ll hear an accompanying “sound logo” when seeing this screen. This heightens impact and gives the company a professional image.

Typically, the game title screen will come next, followed by the main menu screen.

Sometimes these two are combined into one screen. The main menu will almost always have sounds for the UI (User Interface). Even something as subtle as a navigation button needs quality sound. Usually, a simple click or bubble pop is used, depending on the game. However, you can also use musical notes or even a spoken word or phrase.

Sonic appeal is what makes them enjoy pushing that button….over and over again. Subtlety is the easiest way to achieve this – use your ears!

There can also be ambience and music in the menu background. All of this happens prior to gameplay and sets the tone for the entire experience. The menu is like a lounge

or lobby for the player, so the audio should reflect the overall theme of the game. When working with developers on their sonic logo, ask for video footage of the animated sequences. This allows you to really nail the timing. Typically, these are quite short and happen quickly upon launching the game. A company logo can convey a playful image or something more serious. Take the visual information and give it a sonic counterpart.

Before we plunge further into smartphone games, let’s go all the way back to 1997

A Brief History of Mobile The first mobile phones where giant, heavy and expensive compared to today’s tiny devices. In the early days of cellular technology, most cell phones were only able to emit basic tones on different frequencies. The monophonic ringtone uses basic sequencing technology to play a song: it’s a one instrument, one note at a time thing.

Most monophonic ringtones have since been outclassed by the more popular polyphonic and true tone ringtones. Polyphonic ringtones can also emulate different musical instruments, simulating a ‘digital orchestra’ in your pocket when the phone rings, however they lack the ability to use digitalized audio samples.

The first mobile game in history was called “Snake” which was available on select Nokia phones in 1997. It wasn’t too long before mobile games had color and sound.

These simple java games are still available on today’s cheaper phones. These are typically the ‘free’ phones you get when signing up for the cheapest calling plan. Most of today’s modern phones support ringtones called TrueTones, aka: Real Tones, which can accommodate full resolution audio files. These have become very popular, allowing

people to use mp3 snippets of a favorite song.

It’s important to understand that the sheer computing power of today’s smartphone rivals that of desktop computers in 2001, and anything launched into space before 1990! The recent leap in technology has fueled mobile game development in a big way.

Modern mobile games can now handle full-resolution audio files, however, the physical size of the speaker has not really changed and therefore this limitation of the playback hardware remains. Everything is still being squeezed through a tiny mono speaker and there can be steep file-size limitations too.

Back to the Future – MIDI 101

Believe it or not, there are games being developed today that still use audio technology from 1983. Here’s an example of a side-scrolling platform game made with Symbian.

Game developers will often send the composer an example of some music they have in mind for the game. The composer will use this as a guideline when creating new music.

In this case, the developer sent an epic, orchestral piece from a popular TV series which featured a live orchestra. They wanted something similar and asked that it be delivered as a .MID file. Unfortunately, General MIDI has a hard time recreating the lush sound of a full orchestra. It sounds more like a toy music box in comparison!

Nevertheless, MIDI was required due to the severe size limitations.

A MIDI file itself has no audio, just data that tells the instruments what to play. It simply contains the MIDI instructions for notes, volumes, sounds, and even effects. Ok, so where are the instruments? Every computer has a collection of General MIDI instruments often called a ‘Soundset’. Even the computer you are reading this on right now has a standard bank of General MIDI instruments. The General MIDI soundset was established in 1983: sounds 1-8 9-16 17-24 25-32 33-40 41-48 49-56

Instrument Name Piano Chromatic Percussion Organ Guitar Bass Strings Ensemble

sounds Instrument Name 65-72 Reed 73-80 Pipe 81-88 Synth Lead 89-96 Synth Pad 97-104 Synth Effects 105-112 Ethnic 113-120 Percussive

57-64

Brass

121-128 Sound Effects

The actual sound of each instrument varies from one computer to the next and could be described as ‘generic sounding’ and much like a ringtone.

Ok, back to Battal Gazi, the MIDI file itself could be no larger than 4 Kilobytes. (1MB = 1024 Kilobytes) After creating the piece with as much expression as General MIDI would allow, I then exported it as a standard .MID file. Here is the final piece remember that what you hear is my data playing the instruments inside your computer: https://www.gameaudio101.com/CastleActionNEW.mid Although the final piece sounded halfway decent on my audio workstation, playing it on different phones yielded completely different results. For example: When playing the file on other phones, there would be entire instrument parts missing or the volume of each channel would be totally wrong. This is because each phone has a slightly different bank of instruments with different number assignments, etc.

This game would be played on 50+ different phone models, so I had to do some testing. To complicate things even more, the developer was in Eastern Europe where cellular bandwidth was extremely limited, even in 2007. The entire game had to be incredibly small in size.

It’s no wonder the MP3 is so popular these days. Now you should be getting an idea of how far we’ve come with the iPhone. It all happened extremely fast.

MOD Music Some games still use “MOD trackers” which are similar to MIDI and hail from the Commodore 64 days. These usually fall into one of these formats: .IT, .S3M, .XM and .MOD. The sound is similar to MIDI and creating music in these formats requires use of a ‘tracker’. The advantages are tiny file size and consistency in playback sound.

Making music in these formats can be extremely difficult compared to today’s slick DAW process. Although crude in sound quality, MOD music has a loyal following. The sound is not for everyone, but extremely unique nonetheless. These formats are still in use and supported by middleware engines such as FMOD. If you’d like to play around with them there are two free programs which are popular: MilkyTracker and OpenMPT. MOD tracker creation is a good skill to have, especially since advanced engines like Unity 3 support its various formats.

Now then, let’s take a trip back to 2007, to the birth of smartphones:

Multi-platform development in 2007

Mytopia’s mobile games now run on Facebook, Myspace, Bebo, Hi5 and Friendster

Back in 2007, making a game run on every mobile platform was a challenging task. In the case of Mytopia, the games were developed first for Windows Mobile 6.1 and then transferred or ‘ported’ to all other mobile platforms. This process of ‘porting’ a game to a new platform is a technical affair often requiring a rebuild of the code.

During this period, I was the audio architect for Mytopia, tasked with creating the sound effects and music for all 14 of their mobile games. This was back when MP3 wasn’t yet supported for mobile, so everything had to be in WAV format...and very, very small.

To the uninitiated, WAV is an uncompressed format with near-CD quality.

Unfortunately, the standard 16bit/44.1khz WAV files use a massive amount of space.

For these games, the file had to be downsampled, which heavily distorts and degrades the sound. The audio files get converted into a wide array of proprietary formats, so the source audio files must be as clean as possible and sound good on a tiny mobile speaker. Once again, this means lots of testing! In this case, the size limit for each WAV file was 40K, which is the size of a small photograph.

When rendering audio down to mono tracks with a sample rate of 16bit/16khz, things can get ugly. The file size is nice and small, but unpleasant artifacts are introduced, making it sound “crunchy”. A sound effect of jingling coins will often sound like rocks when downsampled and played back through a mobile phone speaker.

The frequency range of a mobile speaker is still between 100Hz and 4 KHz, so keep this in mind as you create mobile content. When listening through those fancy studio monitors, don’t forget that it will end up playing through a tiny mono speaker.

Sector 2 – Apple raises the bar

The i-Revolution: By now, you’ve noticed that I haven’t mentioned much about the creative process. This is because I wanted to point out how technical concerns can often take center stage in video game development, especially just a few years back. Creative rules can (and should) be broken, but technical rules cannot.

Mobile game development has always been challenged with ‘disk space, processing power, system resources and cellular bandwidth’ – all of which determine how great a game can look and sound. Luckily, Apple resolved many old standing problems and opened the door to ‘full quality’ audio and larger files. The iPhone can also playback several uncompressed audio files simultaneously, giving the developer more control over the sonic landscape Today’s audio composers now have the luxury of working with all the latest virtual instruments and effect plugins. Now we can work without the extreme file size limitations, but keeping things small is still a great skill. Being able to create audio with fewer restrictions and render to an MP3 opens many creative doors. Although MP3 is a compressed format, there is very little loss of quality, even at very low sample rates.

This means that files can be shrunk down while retaining the integrity of the original content.

The App Store In the beginning of the app store, Apples size limit for downloading apps over the cellular network was 10MB. After working on early smartphone games, this was a huge amount of space. If the total app size exceeded 10MB, it could still be downloaded, but only via wi-fi at home or a coffee shop, etc. Developers who stayed under the 10MB limit could reach those roaming the earth with their phones.

This quickly became a priority and to help reach this goal, I often had to render my music as MP3s at 96kbps (mono) which can still sound amazing compared to General MIDI, or downsampled WAV files. Apple has since raised that 10MB limit to 20MB, so we now have even more space to fill with audio. The maximum app size is 2GB. Although the app store limit for cellular downloads was raised to 20MB, keep in mind that audio will typically occupy 50% of a game’s overall size. Keep your audio files a small as possible. Get used to mixing down to mono, 96kbps, etc.

Even the simplest of iPhone games can involve a huge amount of work for a development team. In some cases, the development team is just a few people, coding, designing art and making it all function as a game. Oftentimes, developers are so busy with the code and art that music becomes an afterthought. The top game developers, however, make it a point to have incredible audio as they know it leads to an incredible game.

The team has an enormous amount of work to do, so the audio contractor must deliver the goods in a timely fashion and communicate effectively. This also means changing their compositions on the fly and handling any criticism of the content. You may be advised to trash a masterpiece you just spent a week creating. It’s just not working in the game, but the good news is you can shelve it for later use in a future project.

Communication is paramount - if it can be described, then it can be created. The development team should give the composer and sound designer as much description as possible. Game art is good to have, but video footage of gameplay is always best. This allows for more accuracy when matching sight with sound.

You can imagine how much communication happens on a larger AAA console title.

These can have 200 team members working on a game for three years before it ships, whereas an iPhone game can be completed in three weeks with three people…..or overnight…

GravitySling: Gravity Sling was made overnight by 5 people during the 360 i-Dev developer conference in Denver. I was speaking at this 4-day event and ran into fellow developer Brian Robbins who said, “Hey, we are making a game overnight and could use some music and sound.” The idea was that several teams would stay up all night at the hotel with one purpose: “Make a functional game and show it off the next day” It sounded both mad and incredibly fun. The next day, each game was presented on a big projection screen with an explanation by the developers.

Every game was innovative enough to get big applause from the crowd. One thing I noticed was that just about every game was totally silent. Some were simple, casual

games and others had rich 3D environments. Most of them were silent though and ripe for music and sound! It all seemed impossible, but each team cranked out a working game over night.

Over the next week, the five of us held Skype conferences and discussed the direction of the game, touching on every possible way to improve it before release. This was great because each one of us could chime in with ideas for gameplay. The final result was Gravity Sling: an intertia-based puzzle game set in deep space.

The music I created was an upbeat trance-type track that fades into an extended, deepspace ambience. The track ended up lasting about 4 minutes and, after several versions, was bounced to a mono, 96kbps MP3 file. The iPhone can only play one compressed audio file at once, so music or ambience is best.

Listen to Gravity Sling here: http://www.youtube.com/watch?v=RPloxjDPRzs

The first thing you hear is a ‘space bubbling ambience in the background. This was a short, looping mp3 and the button dropdown sounds are all WAV files. The music track then starts with gameplay and continues for 4 minutes until fading out. The star ‘blip’ sounds you hear came from just one sound. I used ‘pitch shift’ to make each one progressively higher which gives a nice ascending, reward sound when several stars are collected in a row. Pitch shift is another feature included with every DAW today.

When you launch the spaceman with too much velocity, he gets ‘Lost in Space’ and you hear a matching sound. I used an ambient, virtual instrument called FM8 for most of the sound effects in Gravity Sling. This VI is made by Native Instruments and is part of the Komplete series. The drop down menu sound you hear is a 700k WAV file that gets quickly repeated. The clicks are simple and let the player know they are in control.

The UI (user interface) will almost always require button sounds. For these sounds, it’s best to use uncompressed files such as .WAV, .AIFF or .CAF files, which play back immediately upon touching the screen. Avoid using MP3 files for this since they take a while to load.

Now you can see and hear, how quickly mobile game audio has advanced since the early days of the smartphone. Since hitting the app store, Gravity Sling has been downloaded over 500,000 times by players worldwide. Not bad for a game that was essentially made in a few nights!

The audio in this game occupied most of the total app size, but we managed to get the game right under the 10MB limit. The final app weighed in at 9.8 MB and barely made the size requirement. Since then, the limit has been raised to 20MB which allows more even more expressive audio.

The Gravity Sling team has since ported the game to the Palm OS. This brought some big issues – one being that the audio could not be included! Listen to Brian Robbins tell the story here: http://noisebuffet.com/musicplayer/BRobbins.html (read along below) “Basically what it came down to is we could get the audio to play, but when we did; it killed the frame rate of the game. So the game itself was running about twenty to twenty-five frames per second, and if we played any audio files, even in the menu, we would drop to about fifteen frames a second. This slows the whole experience down.

And the most disappointing part of it for us was, we found that slow down even if we played that audio in the menus. So certainly we figured that if you were playing audio while you’re trying to play the game, that could be a performance hit. But we found that even if we played a little button-click sound, when you clicked the button in the menu, we still saw it slow down when you were playing the game later, even if we weren’t playing audio at the time. We did this using the HTML 5 app.

Since then, Palm has released the PDK, which is their Palm Developers Kit you can write like native C, C++ code. That shouldn’t have these issues. So there is a way on Palm that you can do it that we didn’t use and we didn’t have experience with. Our problems stem from doing it as an HTML 5 app and having to use the HTML 5 media playback tab. So that was the disappointing thing, but again, they do have the PDK now which should make most of these problems go away. It’s just that we weren’t in a position that we wanted to rewrite the app again for the PDK. So that’s why we never explored that, but that was our experience with the HTML audio on the Palm Pre.” As you can see, audio support is often the last thing to be added to an OS. Unfortunately, the Palm OS was soon dropped from mobile devices. It couldn’t compete in the crowded marketplace.

Looping Typically, today’s games require the music or ambience to loop. This saves space and keeps the overall game size small. It can be challenging to seamlessly loop an mp3.

Even though your creation loops perfectly inside the DAW, when rendered to MP3 it now has an annoying gap of silence added between loop points.

The reason for this is that MP3 files contain extra "frames" causing a slight gap either at the end or beginning of a track -- this extra data is added when you create the MP3 which changes your perfectly timed creation…

One tactic to avoiding the loop issue is to create your music with ambience fading in at the beginning and out at the end. Another strategy used by composers is to blend background ambience and music together in one track, thus saving space and processing power.

Payload: During the same 360i-dev event, I met Rod Strougo who mentioned a new truck game his team had been developing. The sound hadn’t been addressed and he knew that original audio could breathe some life into the game. It wasn’t long before I was part of the team, which was spread across the country. We used Skype to discuss the overall direction on the game and brainstorm.

Payload is a 2D kid’s game with a truck you control via tilting the screen. The goal is to catch falling objects before they hit the ground. Touching the screen will dynamite all objects before they land. While playing the silent build of the game, my first thought was that the truck itself could use some realistic sounds. I wanted to match the cartoonlike graphics with more realistic sounds. Your first ideas when seeing the art or gameplay footage are usually the strongest..

Since the truck moves back and forth, smashing into the side walls constantly, it only made sense to give it a nice impact sound that was solid, but not too overwhelming.

Also, there are different sized boulders that drop onto the hood of the truck, each requiring their own impact sounds. I used a combination of steel impact sounds and crashes from my own sound effects library.

Take a listen: http://www.youtube.com/watch?v=4WaP3T1lBK8 In the video, you’ll notice that when the truck hits the sides of the screen there is a matching sound. Also, when the truck accelerates you can hear the engine. The green bonus item that is caught at 00:26 in the video has a positive, reward-based sound. This sound was made in the same key as the music tracks to keep everything consistent.

The #1 skill a sound designer is simply his or her ability to listen. It’s easy to go overboard and process a sound to death. Know when you have reached this point and be honest with yourself. Often times, moving along to the next sound can keep your momentum. Step back occasionally and look at the big picture.

Since the player would be crashing into the side walls over and over, we needed plenty of sonic appeal  Once each sound was created, I would experiment with EQ and effects to get closer to the desired outcome. Once you have settled on a certain sound, experiment with volume to settle the sound into the world.

Developers can even do this within the

game code. We also needed to give the truck an ‘engine idle’ sound.

This required a short, looping WAV file. Then, a funny thing happened - A glitch in the game code, caused this sound to play during the menu. We liked it so much that we kept it playing throughout the entire game! Mistakes will often bring the most creative of outcomes. Even in the menu you will hear the engine sound constantly looping. This is short WAV and very subtle in volume.

When you tilt the iPhone and move the truck, a second ‘vroom’ sound is played over top of the idle sound. This second sound only plays when the truck moves. Since there would also be background music, it made sense to give these sound effects more of a dominant presence volume-wise. The truck also received a nice ignition sound for starting each level.

The game over screen features a truck driver with no money in his pockets or one with plenty of cash! In this screen the player hears a truck driving by:

Payload Levels Another fun part of Payload is that each level has a different background theme with matching environments. For this section, you’ll need to download the game and follow along: Quarry: This level features a rocky canyon landscape. The background ambient track features rockslides and jackhammer drilling.

When the crow flies across the screen, you hear the ‘caaw’ sound which has a bit of reverb added to imply distance. There is even a UFO that appears sometimes. For the music, I made a traditional rock track using guitar, keyboard and a looping drum track. There is even a UFO that flies by with its own sound.

Jungle: This level features a forest background with a jungle ambience of birds and insects. The monkeys received several different sounds from Rod which worked great.

We only needed to pull them down in volume inside the code. The music has a jungle groove with elements of percussion. I felt the need to include congos/woodblocks to fit the natural element. Each level needed to be as unique as possible for the kids playing!

City: This level was a downtown neighborhood with traffic, police sirens and general city noise. The background ambience contains all of these things and loops, or course. The cat in the window meows ever so often and walks around.

After a while it begins to rain which triggers a gentle rain loop. The music for this one was more of a pop-jazz track. It seemed smart to use a structured genre like that for a city theme Ice: In the Ice level we see an arctic scene with wind ambience and ice cracking. Again, these are combined in the background soundscape. Of course, the penguin had to squeak a few times and for the music, I made an ambient electronic piece. This was my idea of music for an icy atmosphere and seemed to fit. The sharp, rhythmic style matched the visuals and the whole team agreed on it.

All of the art and animation in Payload was made by Eric Stevens and seeing his new characters and backgrounds played a big role in my sonic decisions. Art assets are similar to audio and must be shrunk down to fit inside the game. As of this writing, we are working on Payload Xtreme, which will feature some cool new features like several monster trucks. http://www.prop.gr

8 Planets: Edu-games and Dialogue: 8 Planets is an edu-game covering the planets of our solar system. I was tasked with creating music, sound and dialogue for the developer. They asked that all audio content be delivering as uncompressed WAV files. This made the game around 70MB in size! They decided to ignore the 10MB limit and go for a higher quality experience instead.

All of the dialogue content was in the standard 44.1/16 bit uncompressed WAV format which gave this one great sound quality. This project was unique because my audio was created first, before the gameplay animations. The programmer and artist took my audio and crafted animated content around it, fusing sight with sound.

8 Planets contains a large amount of dialogue, so the words had to be pronounced slowly and clearly, especially since the players are children!

It kind of goes without saying, but main speaking parts should always be clearly audible above every other sound, including music, foreground sound effects, and background ambience. The VO delivery should be believable and fit the game!

Here we see the word Jupiter being animated as it is phonetically spelled out. Each section animates in sync with the spoken syllable. In this case, the screen shot was taken at the beginning of the voice talent saying: “Ju-PiTer”. The voice talent was a perfect fit for the project and had plenty of experience working with kids. The vocal sessions went rather quickly but the editing process was more time consuming. Although the session was basically one long take, I had to go back later and single out each phrase. In addition to adding basic EQ and voice compression to these tracks, I used a great mastering plugin called “PSP Vintage Warmer” to add a bit of warmth to her voice. This gave the overall dialogue a nice consistency and kept it strong.

The final step for dialogue files was making sure each phrase had a nice fade on each end before rendering to WAV files. The bulk of my work for this game was in recording & editing the 40 different voice parts.

For the intro theme music, I used the virtual instruments Xpand2 & Analog Factory to create a looping intro theme track. Again, looping is a constant in mobile games and helps to stretch the soundscape, while accommodating the space limitations. In this case, it was used to keep the main screen fun and engaging as the player makes their choice. The developer wanted a bouncy, space-island feel for the music track so I used the mental image of a kids cartoon while composing. I am very visual and have developed a good knack for looking at a screenshot and interpreting it sonically. This is a quality that most game audio professionals develop over their career. It’s not rocket science though – observe and interpret.

Be sure to download the free version of 8 Planets for your iPhone or iPod Touch More on looping The loop playback function is a powerful feature found in most DAWs. For the blue

ribbon, black hole and menu swooshes, I used the loop-playback function to sculpt the sound in realtime. This endless repetition forces you to make the content bearable when heard over and over. The end result must be pleasing to the ears, but this method will test your nerves because the repetitions can drive you nuts!

Once you have created a short sound effect or musical flourish, set your workstation to “loop playback”. Then highlight the piece and hit play. Now you can make changes in realtime, while it’s playing over and over! Listen closely and experiment with arrangements, instrumentation as well as EQ, volume and compression. This is where the magic happens and sonic appeal begins to form.

Backflip Slots: When using headphones, we hear most of the sonic spectrum. One challenge of creating audio content for mobile is taking into account the possibility of headphone usage. The iPad allows for a bit more mid-range frequency content, so I considered this when working with Backflip Studios on their HD stuff.

Typically, a mobile game will have a looping background ambience, background music, dialogue and UI/gameplay sounds. For the menu ambience in Backflip Slots, I visited a casino and brought along my Zoom H4 handheld recorder to capture the ambience.

Luckily, security didn’t ask any questions as I entered the building with this strange little device.

After finding the rows of slots machines, I sat in the middle and started playing. This location gave me a nicely-balanced ambience, so I placed the recorder on the seat next to me and started experimenting with different configurations. All this was done without headphones, so I had to use my best judgment and just go for it.

The funny thing about casinos is that each slot machine plays jingles in the key of C. The result is a hypnotizing cacophony that keeps the players hooked. I have actually played live gigs at casinos before and been instructed to keep every song in the key of C. This is surreal, much like playing alongside a choir of robots!

Since “Backflip Slots” was getting a more traditional look, we needed the sound to follow suit. For the reel spin, I went for a looping mechanical sound with a subtle friction texture. To achieve this, I combined the sound of a tractor engine with factory machinery samples from my own sound library. The two sounds were then mixed together and combined with elements from Sonic Fiction.

Seamless looping is commonplace in games but often brings technical challenges. That can detract from the creative focus, but it’s a necessary evil, given the technical

limitations of the hardware platforms. The reel landing sounds needed more of a chunky ‘click’ feel that would not be too overbearing.

Each bonus spin-character icon has an animated sequence in which they come to life and jump off the screen. These actions received everything from an 8-bit flamethrower to the sound of a samurai sword being unsheathed.

One of the keys to creating sonic appeal is subtlety and this can require extensive testing. Game sounds should not grate on the nerves even after being heard hundreds of times. This usually entails experimenting with volume, EQ and pitch shifting. The end result should always be a pleasant listening experience for the development team and ultimately, the player.

The win sequences were also in sharp contrast to the reel spins and button presses.

Win animations explode on the screen with coins raining down on top of an animated logo. Add pulsating lasers and lightning strikes and you now have some serious eye candy! These received a good amount of ‘bling’ on the sonic end and went through extensive revisions before completion.

Even when being piled on top of each other, audio files must be audible and clean. Since the game featured all of their IP (Paper Toss, NinJump, Graffiti Ball, etc), I used the existing sounds from their other games and remixed them to work within Slots. In the case of “NinJump”, I blended the hiyaahh with a gong cymbal for the combo win. Some of the combinations are buried deep in the game and will only be heard by top players.

The best way to test for this was to play them simultaneously in Pro Tools and listen for any breakup in the sound. Since the coins are the theme of the game, I gave them most of the higher frequencies while cutting the highs out of some of the pre-existing sound effects from all the games represented Notice that I’m not going into painstaking detail on how I created the audio. That is because my entire process of working revolves around listening. A fellow guitarist once told me that creating music essentially boils down to 90% listening and 10% doing.

When I get writers block with my own guitar playing, the solution that works best is this: Tune the guitar down (or up) to a new tuning. This is much like picking up an instrument for the first time. The chords or scales you once knew are now worthless!

This forces you to rely more on listening and approach the instrument with more of a child-like wonder.

The point here is that restrictions often bring out the best material. Try not to get too overwhelmed with all of todays fancy gear. You can make amazing sound or music with a limited set of tools. Listen to the Super Mario Brothers theme song. It was created in

an environment of staggering limitations. Koji Kondo rose above them and worked with what he had to create one of the most memorable pieces of music in modern history.

It never ceases to amaze me what can be done with today's virtual instruments.

There are so many great products out there that you really have to focus on the game project. It's easy to become distracted by attention-grabbing effects. Scroll through any of todays modern VI's (virtual instruments) and you'll see exactly what I mean. Always keep a screenshot or video of the game visible to keep you focused.

Pacing When creating music for a game, observe the speed of everything. What is the pace of the game? Answering that question will tell you a lot about the tempo and genre of any music needed. A slower, contemplative game will do well with a more thoughtful, medium to slow paced track. The faster moving games will almost have something more energetic. Of course, these old rules can always be broken, but put yourself in the place of the player.

Emotion Even slot machines create emotional responses from the player. An MMO is no different and this human factor should always be considered. There are only a few emotions we humans can experience and the game developers are always looking to bring them into the gameplay experience. When working on Payload, I noticed that the Ice level felt a bit desolate, so I choose minimalist electronic music to capture that. It was my own interpretation of the graphics and required a non-threatening genre that can fit the sounds.

Now let’s jump right into Angry Birds and talk with Ari:

ANGRY BIRDS - Conversation with Ari Pulkinnen: Angry Birds has already surpassed more than 50 million downloads on mobile platforms, which include devices running iOS, Android, Symbian and WebOS. With a movie and a port to Facebook coming soon, we will see the game’s dominance hit an all-time high. Let’s get creative now..

Ben: When Angry Birds was first in development, did they come to you with solid ideas or did they let you run with it?

Ari: Actually, the basic idea was that they had the Angry Birds game, and they said “We need some good sounds and music for the game”. I was like “Okay, let’s look around”. Basically I did all the sound design and music design for the game. Actually, the Angry Birds theme song that was the first version I did for the game. The designer Jaakko Iisalo thought it was too annoying.

He wanted to have this cool. jazz style, hip and hop, easy listening style of the song. You can hear the song in the first trailer. In the last ten seconds of the trailer, that’s the theme song the Jaako wanted. We decided to go with the version I did at first - because even if it’s annoying, it’s setting up the theme song and everybody remembers the music. ♫ ♪ ♪ ♫ ♪ ♪

Ben: It’s very energetic. What’s a brief rundown of your work flow there?

Ari: Actually for the music I have lots of virtual instruments. I was actually in my summer cabin that summer when I did that song. I could show you the picture. I was taking it quite relaxed.

My summer cabin is near the biggest lake “Saimaa” and it was a hot summer and took some beers and you go with the flow. I was having a beer and there were like ten ducks with me I nthe water and I was feeding those ducks and after this short break I went back and continued with the theme song.

Ben: So you heard it in your head first?

Ari: Actually, you know, those Angry Birds, they are kind of silly and their angriness levels are (low)…actually I had this idea of a song that had some interaction. [♫ ♪ ♪ ♫ ♪ ♪] That’s the first phase. And then there’s this confusion phase where the birds realize that their eggs have been stolen. That’s the part – I had writers block one night, with the confusion part. It’s just silly sounding. My wife told me that I should relax and go somewhere else for a moment and come back – had a beer. [Laughter] Just don’t think about it. Last phase was the original Angry Birds song ending. I think it underlines that birds and of course the instruments are kind of, I though that the birds should sing it actually.

Ben :Out of curiosity, what’s your workstation?

Ari: I use Cubase 5. When I was doing stuff when I was younger, I did some tracker music. I from the MOD scene Ben: Yea, I saw that on your bio, that’s like making music on your typewriter.

Ari: Yea, that’s hardcore man. I enjoyed it because that’s a different approach to music actually because you didn’t have much options to do the music. I actually learned how to make memorable themes use the crappy instruments that I have in that time and make it something spectacular. I enjoy Amiga music and you know C64 music.

Ben: That’s a great skill to have, because FMOD supports all the MOD tracker formats. With all this technology, you still have to keep everything really small.

Ari: Yea, I have made stuff for NintendoDS and I did four channel MODs for one game and tracker music background was just really great with that project, you know it’s a different approach to music. You know the limitations. And four channels seems to – composers nowadays, four channels seems to be “No way man”. It cannot sound good but you know it’s all about the melodies after that.

Ben: As far as the sound effects, how do you approach all that?

Ari: I had this main design of the sounds that birds and pigs should be underlined. They should be funny. They should be the main focus of the sound design. Everybody should recognize the birds and the pigs from the sound design. Different birds have different voiceover sound groups Special sounds are uplifting, noticeable from the other sound design and then the other sounds are quite realistic. People actually focus more on the pig and bird sounds. Physics sounds are quite realistic because of that.

Ben: I noticed when you run into different objects it very realistic. How did you approach that as far as hitting wood or hitting stone?

Ari: Physics sounds are one of the basic sounds in audio game design that I have worked many places before and so it’s like collision sounds you can have many different groups that have a different velocity and different weight classes. But for Angry Birds I decided to go with just one collision sound because it had limitations for memory and stuff. I decided that velocities are almost always the same with Angry Birds physics. They have only these random groups that we

are using. So there are collision sounds for about every object in the game that has physics and those crashing sounds for objects that crash.

Ben: For the pigs and the birds, did you bring in voice talent? Or did you layer sound effects.

Ari: Actually I said to Rovio, you know guys I’m going to record these sounds with you. And they were like “What?” With us? we are not voice actors! I said that doesn’t matter, I can direct you guys. I went to the Rovio offices and I said to them about two days earlier that you should have beer available and just have a good time. For the first hour of recording sessions, we didn’t record anything. We just watched some silly YouTube videos and drank some beer and after the first hour I said, “Ok, I’ll show you how it’s done”. I directed myself and after that the guys were like “Oh yea, we know what you mean”. I directed guys from Rovio like Jaakko Iisalo and the CEO of company “Mikael Hed” was there and a couple other guys also. So I just directed the guys and there were the sounds, made by the team.

Ben: That’s a great story. You know that’s funny, I do the same things. A lot of times I do it myself on these games I work on.

Ari: I actually did that Zombi game myself: PS3 Dead Nation. We had these crazy sound sessions with the Zombies. You know it’s quite hard to make realistic Zombies. For Dead Nation we had these – it’s a rated game, so it’s not like Zombies are cute or anything - They’re really angry and stuff. The hardest part was when the Zombies get really annoyed and they’re chasing the player. Sessions for Zombies were quite exhausting. Lots of sweat and I was the first one that made the sounds and I was really sweaty after the sessions. And the other guys were weren’t in the studio at the time said “What the hell did you do in that studio?” I was like “Come on guys, I just recorded the most amazing Zombie sounds ever. You should try it too”.

You know every other guy after me came out sweaty. They were like running a marathon or something. It was really hard. Try to be an angry, really hungry Zombie that is chasing someone. It’s hard.

Ben: Yeah, Yeah. Tell me about Angry Birds seasons.

Ari: Actually we have the same basic sounds with the pigs and birds, but we have additional sounds. Also but the main thing about seasons is the ambient sounds in the background and the themes - theme songs. The basic idea of the seasons is that you have the idea of when it’s Halloween it feels like Halloween. And when it’s Christmas, it feels like Christmas, and so on.

So we don’t change the bird and pig sounds, we’re just setting the atmosphere of the game.

First we had a design where we had music in the background of the game. The first time I thought about it, I was like umm, if we have three minute tracks behind the game, it’s not going to be enough. You’re going to have like a thousand maps in the game and if there’s going to be in-game music, it’s not going to work. It’s going to repeat itself. No amount of music is enough for Angry Birds.

Ben: What are your tips in general about working with audio in mobile games?

Ari: My first tip is the first tip of the game design: You cannot have annoying music or sound in the mobile games - ever. Because if you have any annoying music or sounds, people are going to turn it off or people are going to turn the game off. You just have to make sound design and music for many hours of playing. That’s the first rule I have. I’ve had that rule since I’m began this work. Like Angry Birds, it has these sounds that are lasting pretty great and ambient sounds work in game so it doesn’t need music in the game. That’s my first rule actually.

Ben: Any other rules? Any other tips?

Ari: Well I have lots of those rules. I like to have theme songs in any game. You know, you have to have audio branding. For audio branding you have to have memorable themes in the menu title. That’s the thing that separates the games from another.

Ben: It’s got to feel right and people are getting more and more picky about their games.

There’s some many games out there you have to really make an impact right off the bat.

Ari: I know. And you know what the hardest part of making mobile games – games usually, I think it’s the GUI sounds, graphical user interface. Usually you don’t have any graphical style graphics or animation with those. You just have to design the sounds the way that works in the game. And you have to have happy sounding user interface effects, you have to have sad interface. You know for my part, I think the hardest sounds are user interface sounds. That’s the one thing that many designers just – they just put some clicks in the game and stuff. But for premium games, you really have to have thought about those games also.

Ben: Exactly. My personal trick is putting them on loop. Now this might drive you nuts, but if you loop even a short musical piece, but mainly sound effects, it keeps playing over and over and you kind of sculpt it like clay while it’s looping. You basically try to get it where it’s not driving you insane.

Ari: Yeah, I have these customers who think that UI sounds are easy to make, but they are always the hardest to make.

Ben: So tell me what is your favorite mobile game?

Ari: I actually enjoyed Plants vs. Zombies. I think that’s the best game for me for iPhone.

Ben: That’s interesting because it one of the few games that has an actual song, with a singer.

I’d like to see more singing in games for sure, for sound tracks and a lot more user interface with voice.

Ari: Yes, singing makes it more down-to-earth like. It’s classical stuff there. Movies have always had singing voices, but games are lacking that. I making, trying to now, have singing voice in the main theme. Also in the end game music I have a female singer in there. It’s this high fantasy media-style music.

Ben: Which game?

Ari: Trine 2 team is awesome. They have this awesome artist making that game. The code is new and they this awesome new feature. It’s going to look great.

Ben: Cool. Thanks again.

Ari: No problem, because I’m just having fun here.

DRUMKIT - Conversation with Shane Vitarana of Crimson Jet: DrumKit is one of the top products on the app store.It’s been featured in iPod Touch TV commcerials and Apple billboards all over the world.

Shane Vitarana is the man behind “Crimson Jet”, the company responsible for developing Drum Kit. Let’ see how it all came together and where it’s headed: Ben: So how do you implement your audio in Drum Kit? Shane: So Apple has a bunch of audio frameworks, right, so the easiest one and the highest level one to use is probably the sound effects API that they have. I think it’s just one line of code and you can say “audio file”. I believe the first version of Drum Kit used that. And, it was just way too slow. The latency was just too much. Then after that I switched that to Audio Queues. I never released that version because it was also too slow. Then what I ended up using and I still use is a version of sound engine that came

with one of the sample apps – I believe it came with the Touch Fighter game.

Ben: So have you been using the same file formats for the audio files?

Shane: Yes, the audio files all .caf files, I did upgrade them. Earlier they were 22 kHz and I just upgraded them to 44.1 a few months ago.

Ben: So calf is an Apple format. Obviously to go to .caf you start with uncompressed wav or AIFF file?

Shane: So I start with an uncompressed AIFF file, right. Then I just used the convertor that comes with Macs, I think it’s an AIFF converter and converted them to calf files.

They’re a little smaller and I believer the i-Devices decode them better.

Ben: So you started at 22 kHz sound effects and then you went up?

Shane: Yes, the devises now are a lot faster: iPhone 4s, iPod Touches and stuff. So I don’t have to worry about performance as much because they are using newer devises and it’s a lot more responsive.

Ben: So you actually went into a studio and recorded the cymbals, hits and snares.

Shane: Well the standard kit actually has off-the-shelf sounds, like stock sounds. But I’m working on some kits that you can buy within a purchase and those I actually went into a studio and got photographers to come in and take overhead shots of the drum kits that they have there. So they had one kit that looks like John Bonham’s kit which is a Ludwig Vista-lite Kit, amber-colored kit. And I got overhead shots of them taken. Right now I’m working on creating a montage to create the image of that kit and also recorded all the sounds with the audio engineer there, who was really good at trying to recreate the sounds on Bonham’s kit. We also worked on a Beatle’s kit too. So both of these are going to be in-app purchases that you can find inside the app. After those are done, I’ll add them to the app. I’m pretty excited about that because then they’ll be all my own graphics and all my own sounds.

Ben: Any challenges you ran into implementing the audio as far as overlap when you’re hitting a lot of them at the same time?

Shane: Yeah, that’s a good question. So, I think Drum Kit may be the only app in the App Store that actually blends in sounds. So if you repeatedly tap on a symbol, like the cymbal or a crash, it kind of like blends in the cymbal, so you kind of like hear that Van Halen cymbal sound if you tap really fast. I haven’t looked at some of the newer drum apps, but I believe Drum Kit is still the only one that does that. I kind of create this round robin thing for each drum. So, each drum head has five sounds in kind of like a queue. And every time you play one if you tap one it plays the first one, if you tap it again it plays the second one. And if you keep doing that five times, then when you reach the fifth one, it will stop playing the first one and go back in the loop. So, that the way it creates this kind of blended cymbals Ben:

Now when you recorded those, did you do them completely raw or did you have effects on them?

Shane: The files themselves, at least for the newer sounds I’m working on, they were all initially done raw, but the sound engineer processed them a little bit to make them a little smaller because file size it kind of important and I don’t want the app to be over 20 MB. So, he did process them a little. And also for the Bonham kit, and the Ringo Starr Kit, he did add effects to them to make them sound as close as possible to the sounds on the records.

Ben:

John Bonham – a lot of those tracks that really make him so unique was a total experiment by the engineer. They were recording that in a church and they put him in a huge hallway, and they just – it was very narrow and they puts the mics up high. That’s how they got that sound. Of course he’s a phenomenal player.

Shane: Yeah, but it’s a very big kit -the bass drum is huge. And it has a very, very big loud sound. Yeah, makes sense that it was done in a church. It totally makes sense. We had to process them a little bit to make them sound bigger. Obviously a kit like that sounds better with headphones on. So, it’s hard to get a nice loud sound out of a tiny iPod Touch speaker.

Ben: Ringo’s kit is traditionally mic’ed. It seemed he was close mic’ed – least on the snare.

Shane: Yeah, the studio engineer, actually he was really good, he actually tried to mimic how they were mic’ed in the studio when they were recording, so like he knew all the details about that and he ended up measuring everything to make sure the mike was in the right place and stuff like that.

Ben: So you have the record function on there as well. What’s the file format that you have set up?

Shane: Right now that’s just saving as an internal format. It’s just an array with a value for the drum sound and the time. So, right now it really primitive. I’m thinking of maybe using MIDI or something like that.

Ben: Yeah, that would definitely take up less space.

Shane: Yeah, it could get pretty big if it’s saved as an mp3 or something like that. I’m thinking about it a bit more because people want to share their songs and stuff. But, there’s also some legal issues involved and stuff like - a lot of people play along to songs on their iPod, so obviously I can’t save the song that they are playing over.

Ben: Well you could have a queue of built-in songs of your own that they play along with and they could be different genres and they build to play along with them.

Shane: Yeah, I guess that’s one thing I can do. Along those lines, though one thing I’m adding is more App purchase kits and also in App loops and grooves and more sample stuff.

Ben: Oh, so there are grooves they can play along with?

Shane: Yeah, for example they’re if you‘re a guitarist or bassist or something and you want some drum beats to play along, it’s much easier to take off on the iPhone and play along with these grooves and stuff or you know make your own or add fills to them and stuff like that.

Ben: I guess you could have a drum-less track to play along with and have some kind of point structure?

Shane: Yeah, so I actually kind of have something like in the works but I can’t talk to you much about it.

Ben: [Laughter] Of course.

Shane: I’m kind of excited about the iPad too, so I do have an iPad version of Drum Kit called Drum Kit Excel. But right now its super basic, like it doesn’t have recording or playing along with your iPod. But I am adding those features; it’s just that I’m waiting on the interface. I want the interface to be awesome. It’s not just going to be a blown up version of the iPhone version. It’s going to be completely new. I have designers who are working on that right now and kind of excited about working on that soon.

Ben: Alright man, well keep me in the loop and take it easy.

Shane: Alright cool Ben. I’ll talk to you later.

Ben: Alright, thanks.

Sector 3 – Everything Else!

Windows Phone 7: The New Kid on the Block With XBLA support, the WP7 is bringing a new dimension to the mobile gaming realm. There are 2 games that really stand out and the developers were nice enough to reveal their development process. Let’s start with “The Harvest” which is quite possibly the best looking mobile game to date.

The Harvest: Conversation with Matt Benic and Dale Best of LumaArcade, in Johannesburg, South Africa: Ben: I basically wanted to talk to you guys about “The Harvest”, which is an incredible game.

Matt Benic: Thank you. Basically Ben, a couple of years ago, I started just putting together some lose ideas, some various ideas that we had going forward, and I kind of just started putting together the pitch docs. - building up a library of pitch docs. And one of the ideas we had was of creating a Diablo style game, but not basing it on anything magical or anything like that. So bringing it into a sci-fi scenario. And then we kind of started playing with the idea of adding a horror edge to it. And the ideas sort of evolved from that point. Always sort of an isometric, 3D, top down view. And the original idea was to create a web game, where you’d have a full-on editor and that thing. It seemed a pretty good fit for Windows Phone 7 and having the graphic fidelity that it was capable of producing. So it evolved into a mobile game.

Ben: Is Windows Phone 7 a new thing for you guys? What’s your native platform?

Matt Benic: Well, in developing for PC mobile, it’s our first Windows Phone 7. We actually developed on prototype hardware and software. We were quite heavily involved directly with Microsoft Games Studios as well as their software team. Basically, Windows Phone 7 is a new platform for pretty much everybody. We did a demo of the early concepts of “The Harvest” for Microsoft at the GDC last year where they were sort of showing off what the phone could do and that has evolved into a let’s do this as a full game. But from a technology point of view, being XNA. Luma does have a bit of background in XNA as well on a PC platform as opposed to mobile. It’s very similar. Throughout development, we had a PC build running. There’s quite a strong i-Phone background at Luma as well, so the competence was there on both sides – on the mobile and the actual tech that was used..

Dale Best: We’re pretty used to getting stuff running on hardware that’s not very powerful.

Ben: Overall do you think that Window Phone 7 is pretty strong?

Matt Benic: It’s pretty much on par with contemporaries at the moment.

Ben: It looks like you guys have pushed the limits on the hardware.

Matt Benic: What it’s going up against at the moment? It’s going up against iPhone. It’s going up against Android devices. And you know, to a lesser degree the smaller players like Bada.

The two big competitors at the moment, from a games point of view, I would think are iPhone and Android. Windows actually for a change, has taken a page out of the Apple book. And while they are not manufacturing the device themselves, they got a very, very strict list of device specifications.

Ben: Yes, I noticed that.

Luma: And compared to Android, it’s a pleasure. You know what you’re aiming for.

Ben: You guys obvious just went for it, making this the best looking Windows mobile game out there. How did audio affect that?

Matthew Benic: We weren’t sure what our limits would be, so we sort of didn’t place too many limits on ourselves. From an early point on, our design doc already had a good idea of what sort of resources we would going to be looking for. We tried to aim as high as we could, with the idea that we could knock down if we found we hit limits on the device. With mobile games, traditionally audio has been a bit of a rough patch to deal with. There’s usually a lot of limitations. It’s difficult to get something atmospheric going. It played well into the genre and the experience we were going for. Unlike some mobile games where you’re looking at a couple of minutes of game play, with an extended period of play, you want to kind of engage the player from all areas.

Dale Best: Yeah, and just to add to that Ben, I think it’s safe to say that our approach through the whole development processes of The Harvest, wasn’t “Okay we’re building a mobile game”.

It was, “We’re building a game”. And whether we were being stupid approaching it that way or not, because we didn’t know the final specs of the hardware, we didn’t really know what it could handle until we really started getting our teeth into it. But essentially, it was, “Okay, if we were building an XBLA title, how would we approach it”. And that was our approach. And it was aim high and start optimizing. Aim high, optimize. One of the big limitations, I would say in terms of audio, is localization. We knew that the game would be localized to different languages. We got a list of languages. It’s a pretty long list. You know the basic ones. When it came to cut scenes and all that kind of stuff, we couldn’t have our character – we couldn’t have any voiceovers. It just bloated the download size. You’re always cognizant of download size, and memory issues. So we had to design our cat scenes and our story telling through pictures, and just having sound just sort of back up and add mood to those scenarios. So, just in terms of sound effects, we didn’t know what our target was so we aimed high. We initially tried – we actually went through a few alterations. We got some feedback from our publishers and the feel was, on an early version, that it wasn’t enough variation in the audio. So we went the route of trying to have multiple versions of sounds, things you hear frequently, like the footsteps and guns and stuff that could become horribly repetitive. (vocal imitations of footsteps). You know when you’re walking, you don’t want to hear that all the time.

Ben: Absolutely.

Matthew Benic: So we went through a part when we actually tried to have quite a few versions of those. You know, have the audio engineers put together a couple of those different versions and mix them up a little bit. And unfortunately that’s one of those areas where we had to have – we ended up having to cut back. The audio was just too big. It was taking up too much memory, so we rather went for a dynamic approach and said “Oh well, for things like that we randomized a core sound but we’ll mix it with, say the volume and the pitch a little bit. We dynamically changed sounds randomly, so that you’ve got a single sound asset that you’re just varying dynamically on the fly.

Ben: So you did it in real time?

Dale Best: Yes. It was the less than ideal choice we had to make, but it was a pretty smart choice in the end. I think it was pretty good. I think we actually had more variation as a result, than we would have had maybe having two or three samples per event sort of thing. The end result was pretty good.

Ben: Sure.

Dale Best: And just in turns of the music composition, the guy that we work with, by day they are sound engineers, by night they’re all musicians - releasing albums. These guys do gigs.

These guys are musicians. And the two guys are in different bands. They both like heavy, dark, that kind of thing. That’s an entrance. I think Greg and Paul’s dark side kind of came through in a way and allowed us to set an interesting tone for the game, which was completely appropriate.

Ben: Yeah.

Matt Benic: The Harvest often gets compared to Halo in the sense that its a flagship title for the platform. One of the core aspects of Halo is so much work went into the audio engineering and through the entire series you’ve got that immediately recognizable soundtrack coming through. I think we pulled that off very well with Harvest as well. Our tracks change throughout the game, but we’ve got a really nice consistent, appropriate riff that comes through all the time.

When you hear it coming up, you know that’s the Harvest coming up.

Dale Best: And also on that note, Ben you know that being a musician yourself. You’ll know that when you’re creating a loop, it’s got to be as short as possible. There’s a certain amount of skill that taken to make not sound completely repetitive if that loop is short. 

Ben: Absolutely. That’s one thing I wanted to ask you about – the technical requirements as far as when you worked with the contractors, what specs did you give them as far as audio file size, format, things like that?

Matt Benic: We didn’t know what the limits were. We started with on this when there was not peer specs for the device. So, we used wavs for the effects and mp3 for the music. The devices have hardware decoders so it makes sense for the music. We got the resources from the guys in the same format we would ultimately have them. We tried doing a bit of the conversion work ourselves and you know you say you are not a programmer. We are not an audio engineers.

Dale Best: And then later on in the process we actually had devices to test them on. Both Matt and myself had a good couple of hours in the studio where we basically they played the game and we listened to how these sounds were coming through on the devices’ speaker and its earphone And obviously you know what it’s like, it sounds fantastic when you got all these amazing speakers in the studio and then you put it through a little mobile tiny speaker and you lose ninety percent of that sound. But the guys then went and basically just reworked their sounds with the device and got the sounds sort of reflecting more what they had in mind initially.

Those sound tracks through proper speakers, it sounded like you were in an epic movie.

They’re beautifully composed. You really got transported into another place. And through the little speaker, it’s like – Oh, there’s a little jingle in the background.  Ben: Right.

Dale Best: Yes. Exactly. It’s a process of making it appropriate for the device and the hardware that you’re working on.

Ben: And it’s tricky too because you have to keep in mind that the gamer is going to put on the headphones eventually and they need to sound great in there as well.

Matt Benic: Exactly. We needed to basically compensate for both of those.

Ben: You guys mentioned earlier the real time effects processing. Was that in the run time or could that cause any kind of frame drop?

Matt Benic: It was very limited. Basically, the variation that you’re given on XNA is pretty limited compared to say what you’d have on the Xbox. Literally just you can set your volume, and you can set your pitch.

Then obviously the other thing that we were able to do was fade some tracks in and out. So a good example would be sort of when you get hurt in the game this was actually one that came out late in development and was a suggestion from our publisher that turned out to work really really well. You know when your character’s hurt and the screen is flashing, we bring out a bit of a heartbeat. And it sort of ramps up based on how bad he hurt you on and that sort of thing.

It’s a single very simple loop.

Dale Best: It works really well on an emotional level.

Matt Benic: And it also gives you game feedback.

Dale Best: Absolutely.

Matt Benic: So that was a really fantastic design section - And it was one of those things that it was not in the original design. And we sort of felt we needed something, like Dale says, something to connect with the player.

Ben: Any tips in general with XNA Games Studio 4 or things that you ran into?

Matt Benic: Well one of the things that we did run into was probably a bit unfortunate in that, when using Game Studio 4 for Xbox or PC, you can use 3D sounds, fairly effectively, there’s a chance your players are going to have some decent 3D hardware that’s going to bring that across quite nicely. But the effect that we got out of it was basically horrible. It was terrible. It just basically ended up messing with the levels of our sound. I think translating it to a mono or stereo output, it just wasn’t worth it. So we ended up pretty early in development going off and saying, “let’s just stick to 2D “ and getting it sounding as good as we can.

Ben: Gotcha Matt Benic: It’s a case of playing with what you have available to you. Another one that’s available out of phone is XACT. XACT is quite a comprehensible programming toolkit that’s available. Programming and audio technician tool kit that basically let’s you package up your resources nicely, and provide variations for sounds and do sound events and things like that.

Ben: Sure, I’ve worked with XACT. Matt Benic: It was not available to use on the phone unfortunately. Other than that, we ran into one or two small quirks with the API, but I think it’s par for the course on any platform and in

particularly a platform that’s new. Things where we didn’t expect things to hit us from a frame rate point of view and they did. Things like – what was it, checking what the current volume of something is something along those lines.

And the other big thing to take into account on the phone, that like any other mobile devices, there are some pretty strict restriction and to what you’re allowed to do with media. So you’re not allowed to take over if your player has decided that there’s playing their own music, you’re not allowed to interfere with that process. Things like that, so actually becomes a very large part of audio coders work as well - is making sure meet up to all the requirements from certification point of view in terms of “Does your game behave itself properly in audio terms”?

Ben: Right. Is there a size limit currently on the Windows Phone 7, on their app stores, or the Windows marketplace?

Matt Benic: Overall size limit, very much like the iPhone, exactly like the i-phone. Anything over 20 MB, and you can’t download over 3G if it comes down over Wi-Fi. Ben: Right Matt Benic: You immediately have to look at it and say how’s this going to affect our sales numbers. That was actually quite a challenge for us because I think our publisher was quite surprised with the size of our game in the end, which we were actually quite chucked about it. I think we ended up putting in under 90 MB, which for a game of that complexity is quite something.

Ben: Absolutely, that’s a whole art in itself, is squeezing these games down. Thanks a lot and hopefully we see you guys soon.

Matt and Dale: Cheers Ben, Thank you.

Ben: Take care.

Retrospaced:

Retrospaced is a 3D arial shooter and another top ranked WP7 game. Jonathan Yong from PewPew Gaming shares the audio side of his development: Being a small indie dev team consisting of my girlfriend and me, I licensed existing music and sound effects from various online sites. It was very convenient to be able to browse a large library of material without having to worry about hiring an artist who may or may not compose something which I liked.

The criteria for my music was simple - medium to fast paced dance music that's fun but not too generic. However, the vast array of material meant that the selection process took much longer than I expected. I think I spent two weeks listening to over a thousand music tracks and sound effects before I finally settled on my choices.

Importing the sound effects was very simple. I used Audacity to edit the tempo and duration of clips and imported the wave files directly into the XNA content project. I then created a simple SoundEffect helper class to help with managing multiple instances of SoundEffects. All of the sound effects are played without any special effects such as reverb.

The music was a little more problematic. The XNA MediaPlayer class, which is used to play and loop music, had a problem on my Omnia7 phone where looping music will cause the game to momentarily freeze at the point where the track should loop. I'm not sure if the problem exists across all devices, but I didn't have any others to test with.

This freeze is small - perhaps 100ms - which may not pose much of a problem for slower paced games like puzzles or other casual games, but for a fast paced game like Retrospaced, it was unacceptable.

My solution (and compromise), was to import the music as a SoundEffect object. This has the disadvantage of having to load the entire music track into memory, thus forcing me to reduce the quality and filesize of the track. The freezing was gone and the music still sounded good, so I was happy with the results.

I believe that audio (especially music) is one of the most important components of a game. An evocative music track creates the atmosphere which is so important in giving the game a personality. My favourite game muscian is Alexander Brandon, who did the music for Tyrian, Unreal and Deus Ex, and I enjoy listening to his tracks on my mp3 player just as much as my "normal" music.

The #1 Secret to mixing audio for mobile apps Are you still there? I hope so because this is the best part! In the course of working on all the mobile games, I began to stumble across some cool tricks. When listening to your work on nice studio monitors, it’s common to grow very fond of the sound, but listening back on a phone can be an unpleasant surprise. In many cases, the content ends up sounding less then desirable.

I’m sure you’ve been wondering how to make vast improvements in the sound of your mobile audio content, right? Well, you may be surprised at how simple this is. Here is my current method for mixing content on mobile games…

Just open your headphones and lay them on the desk in front of you, with the earpads facing out..

When I stumbled upon this, mixing suddenly got a lot easier! This will bring you much closer to the sound of a mobile device. Just be sure to turn those big studio monitors off.

Of course, you’ll want to turn them back on when you are finished, if just for a final runthrough. This way you can get a final mix before rendering the audio file. My favorite headphones are the Sony MDR-7506 which are amazing, inexpensive and last forever.

Keep in mind that you’ll need put them on your head too! Although most mobile gamers play without headphones, you still need a nice mix to be heard when they clamp them on their heads. Going back and forth usually reveals any sonic warts and helps you get closer to the full potential of the content.

Another technique for testing your content for the iPhone speakers is to use NiceCast from Rogue Amoeba. This program allows you to broadcast your audio to an M3U playlist. You simply navigate to the URL in your device browser. It’s important to test audio on the iPhone as well as the iPod. The iPod is slightly quieter in volume.

UPDATE: That was written 7 years ago. Please read this article about Mixing on Headphones which will help you navigate this niche of audio production.

UPDATE: There is now a specialized tool called M-Ref which allows you to monitor directly from a an iPhone speaker. Thank Jim Welch for creating this thing!

Android Dev Tips Android is rapidly taking over the world of mobile devices. They have quite a ways to go before catching up with Apple but developers are starting to port their iPhone games to Android more and more. The big challenge with Android is the vast number of different devices running the Android OS. Having so many different devices from different manufacturers causes fragmentation. This is when a game works perfectly on one device, but glitchy on another.

As far as audio is concerned you can avoid many problems by simply using .OGG format for all of the audio content. Again, .OGG is natively supported in Android and

superior to MP3 in sound quality, file size and looping ease. Don’t even consider using .WAV or MP3 in your Android game, because it may not work on certain handsets. This info comes directly from surveying the top developers on the app store, who were nice enough to share this info. Several had to spend a lot of money on 3 rd party testing companies to learn this.

The best audio file format for Andriod is .OGG – This format is superior to MP3 in sound quality as well as file size. It’s much smaller in comparison and doesn’t have the notorious looping issues of MP3….. Kudos to Android for adding native OGG support!

Landing Work The video game industry is notorious for being tough to break into, especially for composers and sound designers.

Traditionally, if you didn’t live in L.A., New York, Austin or Seattle, it was futile to even attempt this career path. I live in Denver and have experienced this firsthand, however, my path into this industry was through a ‘side-door’ and I’m about to share it.

Your job is to “be known”, but this secret to this is to get out and “know others” Think about it, a game takes a massive amount of code and art assets. Your audio is there to support the gameplay, so you should be humble and helpful. If they don’t like the masterpiece you spent a whole week working on, then too bad! It doesn’t fit the game so try again. The two main career directions are full-time and freelance, with each one having its perks and pitfalls. Either way, networking skills play a huge role in your success as people tend to work with people they like and actually know. If you can deliver quality content fast and with minimum headache, then there is a good chance they will use you again.

Print up some nice cards and make an effort to listen and hold a conversation with people. Nobody wants a business card forced upon them. The rules of networking should be studied and applied. Make sure your business cards look amazing. Nobody will take you seriously if you hand out pathetic looking business cards or something scrawled on a napkin.

If you are active on forums, then that can be another great way to get work. Of course, the IGDA has a chapter in every town and every country, so why not head to your local chapter meeting and meet some developers in person? My first gig came from attending IGDA meetings. Visit the job board: https://www.gameaudio101.com/jobs.php Ok, so maybe you knew all of that! Here’s my secret of breaking into the industry: Three sure-fire ways to break in to Game Audio 1. Attend smaller conferences: Although GDC is a required experience, don’t

overlook the lesser known conferences. The smaller crowds can lead to better relationships and more gigs. The first one I attended was “CGDC” in Portland, Oregon. I was the ONLY composer/sound designer out of all attendees. It turned out to be a great event which led to my second gig! This is just one of the many smaller conferences out there. Seek them out and attend.

2. The Leisure Industry: This is probably the most powerful tip. There is an entire sector of the interactive entertainment industry that falls under “leisure and gambling” This means everything from coin-op arcade games to pinball to video poker machines for casinos. These companies have a constant need for sound and may give you a chance. One of these events had around 5000 attendees and I was the ONLY composer/sound designer there! – this is your golden opportunity to get involved with some very cool projects. Just go to highwaygames.com and check out the list of conferences held all over the world.

They keep the list updated regularly so enjoy this gold mine  3. Stay at home: Not very social? Some folks cringe at the idea of meeting strangers at some event. Luckily, you can still land gigs from home. You’ll need to scour the web for any and every game developer leaving no stone unturned! I have used search engines to find some interesting projects. Seek out the gaming industry blogs that focus on the business side. This is where you can find new companies forming and announcing their new studio. There is a very good chance that they haven’t even thought about audio. This technique works like a charm and I keep an eye on https://www.techcrunch.com UPDATE: Much has changed since 2010 and this industry is more competitive than ever. It also now has a robust #gameaudio community that helps each other out.

These game audio career tips still apply so be sure and also read all the great blogs out there, most of which can be found tweeting.

Payments, Contracts and Negotiations The great thing about the mobile development community is that teams are usually small and this means flexibility. I have seen almost every possible type of financial arrangement, but most will fall into one of these categories: 1. Straight contract – In this situation, the composer/sound designer is paid a fixed amount for the work and never sees another dime. These deals range anywhere from $3000 to $1,000,000 depending on the game and the caliber of the company. Your credits should be in the game as well. Start small and build cred.

2. Royalties only – In this situation, the composer/sound designer delivers the work in hopes of making money off sales of the game. No money is paid upfront.

It’s common for a small group to split all revenue equally which means you can make anywhere from $0 to $1,000,000 and upward. The app stores are hard to predict so it’s tough to forecast the success of a game. It’s important to understand that most indie games will never make a profit.

3. Half and Half – This is a combination of 1 & 2. It’s common for the composer/sound designer to be paid cash upfront plus a percentage of all revenue generated from the game. This is my preferred method for working and brings the most satisfaction from the entire team.

If you haven’t read ‘The Complete Guide to Game Audio” by Aaron Marks, then get it now. This was the first book on the subject and covers alot of ground - I highly

recommend reading it and keeping it close by. Also get “Game Development Essentials: Game Audio Development” by Jeannie Novak and Aaron Marks (I contributed to this one).

Bands and recording artists: Licensing, royalties and other income sources: Are you a recording artist just looking for new revenue streams? There are so many ways to get your music into a mobile game today. In the past, music licensing was focused solely on Film, TV & Advertising, but now we are seeing Mobile Games added to the list. You may just have some pre-existing music tracks gathering dust. Think a country song would never work in a game? Think again! Look at fishing/hunting games… They are directly targeting the country music demographic and could use some country music or even hard rock. This goes for any other genre as well. Submit your best tracks to agencies, and they may just get used in Film, TV, Advertising or Games.

The recent explosion of guitar games, band games and music games has done an incredible thing…. They made people listen again. Seriously, all of a sudden, people are back into sound in a big way and the music games are driving record sales numbers. It’s mind-boggling to me how many opportunities are out there. Maybe you’re in a band and would simply like to get songs from your albums placed in games. There are many companies that will review and accept your music. They work to get your music placed in Film/TV and Video Games and split the money it generates.

Developer resources: The harsh reality is that developers are often unable to hire a composer or sound designer. I know many of you are too busy programming a game to even think about audio. Luckily, there are some free options for you. I have complied a list of free resources ranging from free music tracks to free sound effects.

Here's one I created on FreeSound, which has tons of casual sounds: http://freesound.org/people/gameaudio/ Also try these: Incompetech.com Musopen.com Jewelbeat.com Flashkit.com

For those who would like to create their own free, original music and SFX, there are two programs that are up to the task. These are very simple but will allow developers to create original audio content very quickly: SFXR – (Mac version here CFXR ) Thanks to Dr. Petter for developing this awesome tool. It allows you to create sounds from a few tweak-able properties and also has the ability to randomly generate sounds of a particular type (e.g. it can generate random examples of beeps). You can export these sound effects in WAV format (perfect for the iPhone). Best of all it has a free software license so you can use it for whatever you like.

Bitnotic Chill is a relatively basic application that generates random examples of soothing music. Like CFXR, it has properties that can be tweaked to create truly unique pieces of music. Music is a slightly different ball game since it needs to play constantly in the background without getting too annoying. Again, you can export the music in WAV,AIFF, and MIDI formats. This one’s not quite free but at $20 you can’t realy complain! Again, you are free to use the music for whatever you like.

Adding the last layer of polish In order to ensure the sound effects and music (created with the applications above) truly fit, you can download Audacity. It’s the perfect solution for adding that final layer of polish and best of all, it’s free. Audacity is an open source application that allows you to edit AIFF and WAV files by tweaking numerous attributes. You can splice, mix and edit speed and pitch (amongst other things).

Time to get busy.

In closing, I want to thank you for your time, your trust and most of all, your ongoing support of this fascinating art form. I want to add more to this ebook and expand it as time goes and would love to hear any of your feedback.

So if you have any questions – anything at all – please shoot an email via https://www.gameaudio101.com I will collect all the emails and use your feedback to keep improving this eBook. Don’t be shy, either – the faster I get a ton of questions, the faster I can deliver the goods!

The goal here is to improve the medium of mobile games via improvements in audio quality, creativity and ease of development. The goal is to keep improving!!

To your success, B. Long