Thesis by trista Busch

A Generative System: Audio Visualization

A Generative System: Audio Visualization Printed at: Graphic Arts

Abstract

There are various elements within design that act as a catalyst for creative inspiration. Sound is an important part of design that can be utilized more effectively. This project reflects on sound as a tool and generative system through focusing on patterns and visualizing movement within sound. Analyzing the various concepts behind sound design as a reference tool for designers opens up innovative ways of perceiving sound in design. Through method based experiments that analyze visual interpretations of sound, this project illustrates the relevance of sound in design and acts as a reference for other designers.

My first attempt at thesis was inspired by a vacation I took to the Caribbean. After that refreshing look at cross cultural communication I began to reflect on what I wanted to work on for thesis. I started with a basic concept and moved forward from there. Over time my thesis began to transition from it’s original topic an move towards a very broad and abstract look at generative systems within design, and more specifically, generative systems that translate audio into visual. I discovered this metamorphosis of my thesis at about midterm, and the project continued to evolve from that point. The inclusion of interactivity provided a system which I could not. So the visualization of sound was dependant on user input. The structure of installation based design became the basis for the construction of this project.

Dear Can, “I believe that the word thesis has connotations, which create a certain impression. I am positive that we have all sat around trying to think of a brilliant idea. I personally spent about three days solid locked in my room doing nothing but research on the concept of space. All denotations and connotations related or subject to space were my initial focus. This is an all-encompassing factor that pertains to ever design methodology, and is far too broad to tackle in three months.

Now realize that this is audio advertisement at it’s crudest, people yelling over megaphones to come to their shops or advertise for beach parties, cars with speakers on the hood blasting the latest sale or hot spot, and just about everything else in between. Instead of printing, which costs money, this culture found an even more powerful form of connecting. Audio.”

In the midst of dealing with this conundrum I took a much needed relaxing Jamaican vacation. I spent an entire week laptopless on the beach just soaking in the radiant sun. (After about a day and a half I was sun burnt and bored), so I decided to adventure into the fascinating culture and various cities of the Jamaicans. The noise was overwhelming! One major form of communication that Jamaicans have nailed is how to vocally advertise. 5

This leads me to my thesis topic that I would like to explore. I feel that audio currently plays a major roll in several cultures around the world and carries with it a sense of universal expression that can transcend the bounds of traditional communication. All cultures use some form of audio to express and/or advertise. With the inundation of visual media across the bored, populations seem less responsive. With the addition of alternate sensory stimulation, I believe that we can increase awareness and message comprehension to add to the stimulation of visual senses or even the lack of visuals all together. Auditory receptors form a strong connection to memory that visual senses can sometimes lack.

Also, I discovered that audio is received differently depending on the type of audio, and which ear processes the information. The right ear processes language better and the left processes music more effectively. I intend to explore this concept by experimenting with instillations as it pertains to audio and the space it exists. I am going to move through experimentation with audio from dealing just with auditory and then progress to the visual end of the spectrum in search of a blend of commonalities that will play off of each other to benefit and solidify the concept being communicated. I feel that this exploration is a valid and fascinating part of the design world that goes often unaddressed. With movement toward new various technological mediums the bounds for design are limitless and audio is just the stepping-stone.

After the first letter to my instructor I began to explore various translations of “traditional” graphic design. This was an area that I felt was important to translate, but after careful reflection as to what my thesis project was, I decided that these traditional methods and elements were irrelevant to my project and my process. Ultimately acting as a misuse of my time.

I did enjoy making graphic icons and playing with word puns, but I had to make some dramatic realizations as to what my processes and project really was. I feel that designers become pigeon holed into a certain stereo type which seems to be existing as a dieing art. This project acts as a null to that traditional method. Almost a point of zero. A none traditional look at what design could be. Also, this project blurs the line between art and design, which really needs no catchy name or clever pun. The simplicity and beauty of this project speaks enough. Any thing else just adds contextual layers which do not pertain to the concept of this particular thesis topic.

Research became an intricate portion of my project. This is not the first project to translate audio into a visual realm, and it is certainly not claiming to be. Through careful and extensive audio visual technical research I began to build my project and bring to life its own originality. In a sense everything in life and time is borrowed and just about everything you see and touch is exposure to new ways of experiencing. I owe everything to the plethora of knowledge sitting at my fingertips every day. God bless Google.

Project Description Often as designers we become callus to alternate methods of design. Everyone thinks they can be a designer, so they do. This leaves us with a mess of visual information that we as a group are bombarded with on a daily basis. We are unable to receive these messages due to the fact that our every day minds are not capable of filtering that visual information out. With this conundrum designers are faced with the challenges of captivating the attention of an audience that is not so easily reached. With the mounds of technological advances that flood the population, there are more ways to communicate now more than ever. This has increased the level of performance that designers to meet. Because of these advances there is too much information flow which is distracting. Our challenge as designers is to rise above the media noise and be heard Visual problems can often be solved with delving into an unsuspecting media. Although sound is not traditionally thought of as visual, it has

many basic visual principles and systems which merit further investigation. Analyzing visual patterns in sound as a means for graphic production is a way to communicate in a non-traditional form, that generates new ways of conceptualizing visuals. I will create a system for designers to reference through a series of examinations, which will be interpreted based on the various outcomes from this method based process. My first examination will explore sound on a collective level and visually illustrate the disruption of sound through individual interventions. These interventions will act as a catalyst to move the design from the natural patterns that sound is predisposed. This communicates the concept that although sound has indicative patterns, one input can alter the moment/design. Generating various graphic patterns enables the designer to use a method based approach in creation.

A constant frequency produces a constant pattern. This pattern can be interpreted through a single audio input that affects the entire design. The relationship between sound acting on a collective level yet still affected by the individual is a concept that adds a greater level of meaning to visual and interactive design. With our cultures inundation of visual images, it is our job as designers to filter through the mundane and capture the imaginations of our viewers. Adding extra sensory levels into a visual media will change the way design is perceived and push the bounds of traditional design methodology. Exploring new ways to generate visual components through sound as a generative system will open the possibilities for experimental creative methods. Interactive design is now moving to the front as media becomes more prominent in our society. My second examination will focus on the personalization and interactive experiences of the viewer in relationship with visual/audio media. The overall focus of this experiment is to keep the design experience visually stimulating on a uni-

versal level while maintaining intimacy with the audience. Simplicity within interactively will create a forum that opens a dialog between the viewer and design. Interactive audio design is moving to the for front now more than ever, and it is important that we are aware of these audiovisual advancements. There are several interactive audiovisual media experiences being moved to social settings such as museums and parks. I will investigate the relationship between the audience and design to enhance the viewer experience creating a reference tool for designers. These experiments will act as a guide and creative source for generation of work across multiple disciplines of communication and design. This project will act as a reference tool/interactive installation and a creative open source toolbox for designers to inspire new processes within their own works. It will demonstrate the importance of broadening the definition of traditional visual and graphic design, and illustrate new processes in which work can be viewed and created changing the concepts and conventions of design. 19

I began to further explore these ideas of translation through more concrete methods. My first series of experimentations were with sugar on a speaker. I laid down a black cloth on top of a speaker and started video taping. Originally I was more concerned with what types of audio would translate visually. I started researching binaural beats and how they effect mood and stimulate other feelings and sensations. Also, various sound frequencies became a fascination of mine. Each frequency tends to have it’s own unique pattern. When the pattern is disrupted it creates slight irregularities within the pattern that alter the visual state.

After video taping these experiments at different angles and views, I began to play around with the video footage in Adobe After Effects. These videos produced beautiful and compelling visuals that were a direct translation of audio.

I researched other ways in which audio has been translated technologically and visually. The oscilloscope was of particular interest because it was technology that had been translating audio in measurable visuals for quite some time. Also, the beautiful graphic patterns where of particular interest. I considered these elements in my process of this project, and decided to maintain focus on pure, simplistic visuals.

Beautiful graphs of sound from: Comparing Noise Figures in Linear Regulators, “H I F I D U I N O”

Various materials were also considered when I started designing the methodology of this project. I came across several experiments with cornstarch that were audio based and decided to add that to the list of materials to experiment with. Unfortunately my translation with cornstarch was a bit muddled and muted. Apparently consistency was a serious issue.

diggabyte, “What Sound Looks Like,” Youtube, May 30, 2006,

For the second presentation I ended up having to video tape my presentation and post it online for critique. This led me to an important revelation in the process and construction of my thesis project. The projected images of the experimental videos were beautifully defused. As a result of the projected images being more visually compelling, I decided that my thesis was a projection based installation.

Musical instruments have always been prominent in my life, and for that reason I gravitated toward music instead of random sound. I first tried disrupting the audio frequencies through guitar riffs, and then through keyboard and violin. Ultimately the most user friendly element was the microphone. Most people like to hear themselves, so I eliminated the instrumentation in my installation, so I did not alienate people that do not play instruments. Plus, Karaoke seems to be popular, and I wanted my project the feel universal and easily accessible to my audience. Design and art should exist without boundaries or limitations to who can understand and appreciate these instances. Life is not about living above understanding, but rather to communicate and share. After all, life is nothing but a series of shared moments that links us all together.

i <3 music I probably listen to music about 18 hours a day or more. This was the mejorative reason behind why I chose an audio based thesis. However I had trouble finding structure in my audio selection. In order for my thesis to be considered a system, it needed a level of input to follow. Since I am such an avid music lover I decided to use the patterns at which I remembered music as a selection process to my own experimentation outside of an installation. More specifically I selected the last song I could remember hearing right before I fell asleep. Sleep is such an intricate portion of good design, and a good designer. Lack of sleep can cause changes to thought processes and methods which can translate into a career devastation. Audio tracks that we listen to subconsciously influence our work. So there for I chose that as the basis for my personal experimental system.

Grooveshark is a better version of Pandora that allows you to create play lists as well as listen to pre-selected radio, or other friends play lists. I often fall asleep to this web site. Because my installation will be selecting tracks based on user interaction, I felt that Grooveshark would give the audience the most direct and user friendly interface to experience my project. Also, I feel that it is easier to understand a concept if you have a kinesthetic view of how the process works.

This was a system that was continuous throughout my presentations, and was the original construct for my process book but unfortunately did not make the cut. (tribute to designs never used)

As a basic printed visual this project lost it’s thunder, so it has to exist as a projected installation in order to maintain the quality and integrity of the image.

Moving To Water Water is a fantastic and wondrous miracle of earth. It takes up three fourths of our planet and is beautiful in every way. It also made for the best visual translation of audio. The water seemed to be so sensitive to every incremental change and movement. Not to discount the other materials that I used, but water was by far the most aesthetically pleasing.

After giving several presentations and coming to the realization that I needed to break free and embrace the abstraction of my thesis, I decided to go back and analyze my problem statement. I focused on the most basic concepts behind the structure of my thesis project and emphasized those elements which were crucial to the understanding of this topic.

Project Components: Equalizer Mixing Board Oversized Speaker Black Cloth HD Film Camera Tripod Microphone Laptop Bitchin Selection of Music AV cables Projector Audio Wire Lots of Audio Wire Internet Medium (ie. water/sugar) Lights Technical Savvy Patience Blank Tapes Food Coloring (optional) THIN Glass Container Electricity A User

In order to have the water translate to film effectively I had to add color. Food coloring is an excellent source of immediate coloration. Many colors translated differently at various angles and camera transitions. I find that cultural connotation adds context to the color without a need for further explanation. It can be or represent whatever you connote to the piece.

As an after thought I briefly revisited After Effects, prior to being terminally banned for the semester, and finally agreed that manipulation was hurting my process in the context of this particular project. The visuals were fun and interesting but failed to stand up to the raw visual effect of non-manipulated videography.

This was a project that I attempted by placing semisolid fluid inside of the speaker, but instead of getting this brilliant footage seen to the left, I electrocuted myself. This is an experiment that needs to be saved for braver days.

The only project experiment that will never be seen; my electrocution.

bendhoward, “Non-Newtonian Fluid on a Speaker Cone,” Youtube, November 26, 2008, 57

I really wanted the user interaction to be dependent on the users preferences. Music is a way to connect with people on an emotional and physical level while maintaining solidarity. So the user is universally connected but it is personalized to the specific user. I felt that allowing the option of choice and preference is a way to please everyone. Also, I thought about how people connect with audio as a sense of memory. Audio is one of the strongest ties to memory, and everyone can connect a particular song, sound or video of audio space with an individual reference to their lives.

On my third progress report I started addressing concerns that my peers and instructors felt I needed to address. I included this information as follows: As far as input is concerned, I have been selecting my songs based on what I last heard or remembered hearing before I fell asleep. This imputation method does not relate to the concept, but acts as a rule within a method based process. The reasoning behind this method of imputation is simply the fact that sleep patterns often coincide with a designers process and workflow. Because process is such an intricate portion of the success of a project I felt that whatever the last song I heard while falling to sleep was effecting my methods and processes already. So this imputation highlights that aspect of importance to design process as a whole. Naming is often a difficult task. I have proposed several ideas for naming but have not gotten a tone of feed back. As of right now I am still calling the project a visual system / generative system. I feel that naming is a focus that I have given quite a bit of consideration, and I would rather be generic than have the name take away from the project and it’s ab-

stract nature. A generic name gives individuals the opportunity to access their own opinions, rather than forcing the viewer to relate to the installation in a specific context. The context in which this project will be placed is an interactive installation at the senior show. It will be set up in a similar fashion to the way I have been filming these studies. There will be a speaker with some medium for visualizing sound, i.e. sugar/water/ cornstarch/light, and a HD video camera will be stationary filming the activity. A projector will be routed into the camera to project down the hallway in real time, and a microphone and computer will be set up as a station for the viewer to interact with the installation. The viewer will be able to select what song or sound they would like to visualize and/or sing/speak into the microphone to simulate the disruptive element explored in experiment one. This will create a clear understanding and transparency to this generative system.

After addressing those concerns I began to revise even further. Simplifying my project to it’s most basic form was the goal and intention at this point in my process.

I revisited the same projects several times meticulously scrutinizing the various camera angles and methods. Reflection was a particularly interesting point in my videography. Through reflection, images could be projected onto the surface and then broken by the sound and movement of the audio. This created very etherial footage that was conceptually interesting, but added a layer of complication that would not benefit my thesis.

I tried several videos with acrylic ink, and other substances. The video footage at first was amazing but after a while the water became muddled, and ultimately would not work well in an installation environment.

The hot pink on black ink was the most compelling at first glance. The colors managed to stay separated for quiet some time. Overall I could see these video instances as something non-installation based with the possibility for individual cut and edited footage.

graphing the most basic principles

Karaoke is a favorite past time amongst many cultures. Song in general is a form of oral communication that is displayed in almost every culture around the world. As an instillation, having a microphone for personal audio input was important to the user experience. I wanted my project to maintain a playful and inviting nature that would be simplistic in concept but complex on a visual experiential level.

Reflection

Overall my thesis evolved into something that I could have never predicted. I went from a complex look at audio systems and simplified that concept into a simplistic visual form that stands alone as an instillation. I am comfortable with where this process has taken me, and how it fits into my portfolio. My generative system acts as a wild card, demonstrating my ability to break out of the mundane and reach for an abstract and unusual approach.

Over time this installation could be an exhibit that travels to small intimate gallery settings that introduce design and production into an accessible space for the public. If I have inspired just one creative thinker to try outside of the box then I have succeeded. Design is not about a label or medium, It is about forward thinking, and creative solutions. Design is about wanting to make the world a more beautiful place. This project reflects that concept by maintaining simplicity in a world that is inundated with constant media. It just is. And that is all I will ever ask it to be. 81

Other directions that this project can possibly go in could be a sharing system where one designer pass a CD around to other designers and the visuals would be shared in a communal sense. I see the installation aspect as a public piece and the files that are the result as the personal piece that is shared in design communities as an open source material.

Using experimental process is an essential part of design, and something I felt that I was getting detached from. This project has revitalized my view on what design could be, and how to stay fresh and break from tradition. I am not discrediting traditional concepts of what graphic design is, I am simply expressing what I think about the conventions of “what is” graphic design. This is the answer to the question that was first proposed to me in 2008. This is graphic design.

Lighting was key to my success in these video experiments. For the installation, the lighting is controlled in a separate area or masked and out of sight. The only thing the user and viewers will be able to see is the projection of the video that is translating their audio selection/input.

The installation is a complex set up where there are two lights angled and shaded to provide optimum about of light into the camera lens. An HD camera is mounted on a tripod and stabilized at a certain height and position. The speaker is then running into a mixing board which is hooked up next to a laptop. The laptop will have open tabs from Grooveshark and YOUtube for audio selection purposes. The video camera is hooked to AV connectors in the back of the projector that will project on the wall in real time. 93

Simplified breakdown of how this system works

The image to the left was a very Meta moment in my process. This projection that was being video taped and re-projected, and video taped again. The idea of diffusion through projection would be a concept I would move toward in the future.

I have learned quite a bit about audio in the last three months, and have developed a close relationship with audio wiring, and audio input strategy. My technical knowledge has expanded, as well as my love of audio in general. I believe that audio will be the next revolution in the conventional practice of design and interaction. Audio has already started to build it’s prevalence in the design world. Everything is moving to a level of interactivity that is heavily audio and visual based. As designers it is important to stay on the curve of trends always pushing to new levels of convention. 99

Special Thank You to Mom and Dad for putting up with hours of endless basement racket.

101

A Generative System: Audio Visualization

103

INDEX JezebelDecibel,” 528 Hz Alpha Binaural Beat (listen with stereo headphones),”Youtube,2011, http://www.youtube.com/ watch?v=vQK_uZEPRaM. This video inspired the concept of using various frequencies to affect the visual output. I also found this an interesting concept that sound can alter the physical state. Sound is such a powerful tool, and using non-verbal audio is interesting when discussing communicability cross culturally. Android Bluetooth Oscilloscope” ProjectProto, Thursday, September 23, 2010, http://projectproto.blogspot.com/2010/09/androidbluetooth-oscilloscope.html. This was an interesting concept to use an Oscilloscope, which visualizes sound by measuring frequencies, as a portable method. Being able to probably measure sound outside of a controlled environment seems to be a more effective tool for visual design in non conventional settings. 1individeo, “Ondas Sonoras ( Sound waves),” Youtube, 2010, http:// www.youtube.com/watch?v=hIWova0DG48. I found this video to be inspiring, as a different way to visualize sound. Using different frequencies, sounds, and tools produces very different results. drdeak, “Acoustic Levitation Chamber,” Youtube, July 2, 2006, http://www.youtube.com/watch?v=94KzmB2bI7s. The Acoustic Levitation Chamber was a surprising video to watch. Sound is so powerful that it can levitate light weight objects!

“Comparing Noise Figures in Linear Regulators,” H I F I D U I N O, Monday, March 15, 2010, http://hifiduino.blogspot.com/2010/03/ comparing-noise-figures-in-linear.html. Hinduino is a blogspot set up by Arduino users that discusses various Arduino projects. I found this particular article of interest because it addresses regulators within noise that are typically undesirable. This particular graph was a beautiful representation of noise and regulations between various instances. lorne69, “Colour of Sound,” Youtube, July 09, 2007, http://www. youtube.com/watch?v=w1dDFjuu_To. I found the computer generated graphics that represented sound and the biological process to be very innovative. This was more of an informational video, but still was worth referencing. bendhoward, “Non-Newtonian Fluid on a Speaker Cone,” Youtube, November 26, 2008, http://www.youtube.com/ watch?v=3zoTKXXNQIU. diggabyte, “What Sound Looks Like,” Youtube, May 30, 2006, http:// www.youtube.com/watch?v=4shodbQMcmM Cool DIY analog experiment with cornstarch. buraqh, “Glass Destroyed only by Sound,” Youtube, May 13, 2007, http://www.youtube.com/watch?v=4EnSTLH492U. This made me realize the true power of sound and it’s existence. It is quite hard to visualize something that is not conventionally seen. This may be cliche` but it is an honest representation of sound. phineus [at] phineusmusic [dot] com, soundwidgets.com, http:// soundwidgets.blogspot.com/2008/12/arduino-audio-projects.html Wikipedia, “Sound art,” Wikipedia, 29 January 2011, http:// en.wikipedia.org/wiki/Sound_art.

The idea of sound art seems to be perceived as only a fine art medium. I would like to take the time to redefine this as an all encompassing medium that branches to various creative disciplines. sonusphere, “EARS,” Sonusphere, Wednesday, 26 May 2010, http:// sonusphere.blogspot.com/. Sonusphere is a blogspot that is devoted to the visualization, research and experimentation of sound. EARS is an organization focused on electroacoustics and various relationships within the audio realm. John Lambert, “Relay,” RelayProject, http://www.relayproject.com/. This is an awesome sound project that is a continuous stream of various projects by different sound artists! So awesome! I think that this format could inspire works within my own process. The documentation, design and fluidity of the project is beautiful. GENERATOR SOUND ART GALLERY, Generator, http://www.generatorsoundart.org/. abe@sleeperindustries.com, Listening Room, http://listeningroom. fettig.net/. This was an interesting approach to interactive sound. The creator is just a music lover that wanted to create a social setting similar to Pandora yet different. Everyone listens to the same thing at once, but you can veto anyone song. This emphasized the social element of audio, demonstrating it’s innate social nature. “Modified Toy Orchestra ,” Future of Sound, 2007, http://www.futureofsound.org/13.shtml. Modified Toy Orchestra is a group that makes innovative instruments by modifying children’s toys. The visuals are quite strange, and the idea is an original approach to sound design.