Contemporary Soundscapes

1

Contemporary Soundscapes Designing for Untapped Aural Potential in the Visual World

Antriksh Nangia 3

Cover Art by Aman Vig Copyright 2018

by Antriksh Nangia All rights reserved. No part of this publication may be reproduced, distributed or transmitted in any form or by any means, including photocopying, recording, or other electronic or mechanical methods, without the prior written permission of the publisher, except in the case of brief quotations embodied in critical reviews and certain other noncommercial uses permitted by copyright law.

For more, visit www.antrikshnangia.com School of Visual Arts MFA Products of Design 136 West 21st Street 7th Floor New York, Ny 10011

Antriksh Nangia Author & Designer

Hilary Gunning Editor

Allan Chochinov

Chair, MFA Products of Design Thesis Advisor

Andrew Schloss Thesis Advisor

5

CONTENTS I II III IV V

VI

VII

VIII

IX X XI

INTRODUCTION LEXICON GOALS & OBJECTIVES AUDIENCE & MARKET

9

Audience Market

26 34

PRELIMINARY RESEARCH

37

Aural & Visual Auditory Icon & Earcon Sound Now A Co Creation Session Results

39 43 45 47 52

DESIGN FOR MUSIC EDUCATION

57

Schoodio Synic Clairaudience for Kids

58 60 64

DESIGN FOR SPECULATION

73

The Chanter The Equalizer Noise NYC Amuze

76 84 86 90

DESIGN FOR SOUND

93

What The Sound Sonify Steptone Decibel Design

94 114 130 142 153 159 161

BIBLIOGRAPHY GRATITUDE MOVING FORWARD

17 21 25

7

INTRODUCTION

A child sits in the corner of a brightly lit room. It has been more than an hour without much movement. Surrounded by an opulence of stimuli, he chooses to be an observer, a receiver. I played this role for most of my childhood. We have a large family and attended many family functions. I have heard stories of us traveling to all different places and me being very present yet lost at the same time. The same thing happened in my other social circles.

Being an introvert has played a major role in shaping the way I navigate an environment. I starkly remember situations, including both what I saw and what I heard. Not getting engaged in these activities empowered me to become a keen observer of the people, places and objects that surround us. This requires a heightened focus on one’s senses where each is addressed consciously or subconsciously to pay attention to stimulus. The aural sense has played a significant role in my life. Spending so much of my time in silence—which, in one definition, is the absence of sound—I have found meaning in what our ears have to offer. I remember late night vocal practice sessions with the lights switched off to avoid disturbing my sister in her slumber

9

“In those days, men heard sounds whose angelic purity cannot be conjured up again by any amount of science or magic.�

- Herman Hesse, The Glass Bead Game

or identifying who was coming upstairs through the pitch their footsteps to know if they would check on me while playing video games late at night. More recently, I remember knowing the right moment to shift gears or how deep a tyre was stuck in the mud while racing cars. This can also be translated to situations in which I have isolated the visual sense to take in and experience my environment sonically.

I am still astonished by the things that we can discover by listening closely. So much is missed in a visual interaction due to limitations of both physical and cognitive capacities. Lately, I have begun to take the role of a creator rather than a spectator. In transitioning to being a designer, I have come to understand the power of expression and its ability to intercept situations and, thus, lives.

Audio is the most prominent of our senses. When we go to sleep, our perception of sound is the last to close and first to open when we awaken. It has been the primary input organ since humanity began, with God first conceived as a sound or vibration rather than an image. Before the advent of writing, receiving through listening was given much more importance than sight. The world has also been divided with Europeans being conduits for the idea that “seeing is believing” while Africans have commonly believed otherwise. In our global, contemporary times, the new soundscape has been an intersec-

tion of industrialization, cross-pollinating cultures and evolving nature.

I come from a culture that has deep roots in the way we speak and listen, which can be heard in the phonetics of our languages and in exchanges that are purely aural (especially religious, though I am not religious but spiritual).

Thus, I have seen the wonders and potential of sound in different ways.

Currently, we live in a world that is a consequence of long ignoring sound.

We have reached peaks of noise pollution in our physical environments, especially urban landscapes. This has occurred because the visual has been given priority over the aural, and continuous ignorance has lead us to create and use noise abatement measures to reduce the effects of unintentionally designed soundscapes. As in most other fields, this deprioritization is evident in design since the post-industrial era where Bauhaus brought aesthetics to machinery and mass production. A similar revolution is needed for the ‘aesthetics’ of sound. This has gained some traction with the advent of Acoustic Design, but the scope is very limited.

Sound Design has been around for some time now. It deals with music, theatre and many other fronts. However, one field that is still in its infancy is product sound design. As objects and artefacts that surround us evolve from mere functional products to experiences, there is a need to design for

Introduction

11

Contemporary Soundscapespes

senses in integration and not isolation.

Most product design is focussed on the visual aesthetic unless the function states otherwise. This is not limited to analogue objects; our digital lives are mostly based on screens, which have engulfed us in ways we do not yet have a measure.The practice of designing around the visual is evident in today’s VR interactions; the developers prioritized making visual technology better only to realize that immersive experiences are impossible without giving equal heed to designing for audio. The industry has been forced to invest in and boost audio technology.

a generation obsessed with screens.

This was an appropriate entry point

This thesis is an opportunistic exploration into using untapped audio potential as a medium of experience and product design through which I can engage people with the meaning of sound in its various forms.

Under the umbrella of Schoodio, there are four projects that tackle music education in different ways. A Co-Creation workshop included designers, musicians and music educators to brainstorm on problem identification and investigation in the current system and then some ideation exercises on how to begin addressing them. The Service is an alternate curriculum in the form of small workshops to introduce high school students to an experimental mindset while playing with sound objects and noise instruments to broaden their lens on what a musical instrument is and can do through electronic music.

The mobile-based application uses the current obsession with smartphones and provides a low barrier of entry to making music and changes the role of students from spectator to creator.

Clairaudience for kids is an exercise in the joy of listening, a workshop to get

As I started this journey, meandering through the fascinating world of sound, I came across various avenues that my intervention could address. With the overarching concept of elevating the medium of sound in design and our lives, I took on the premise of creating products and services to tackle the stagnating state of music education in schools. With a grave problem to investigate in a specific demographic, I dug deep into designing for sound. Tackling the current state of music education in schools seemed like an appropriate beginning towards re-establishing the importance of the aural sense among

in beginning to think about sound from the ground up, to understand it’s capacities and then use it to intervene in situations.

This initiation has resulted in producing a variety of offerings that take on different challenges faced by music education in American schools today from binarily structured curriculums to a focus on classical music theory and transition to an interleaved curriculum and experimental music theory, among others.

Introduction

13

children interested in what their aural sense has to offer.

As I got accustomed to working with the pitch and timbre of designing for sound, I engaged all my focus towards product sound design from a more solid perspective. Removing myself from music education, I pursued using sound as a medium equal to the visual to a more general, ground-up context.

Informed by all my past projects, the next set of offerings takes on product sound design through various lenses.

A staircase is reimagined to add sonic cues in the construction of an integrated experience so that a participant knows his or her location and can climb up or down without fear of tripping and constantly looking down. What the Sound is an experience to make the public aware of the product sounds that surround us to leave them with a sense of discovery as these sounds are exposed in a considered, interactive experience. Decibel Design is a service in the form of a product sound library that consults clients on product sound design to democratize it’s use and nature. Beyond Screens uniquely captures sound in a conversational user interface.

With endless sketches and prototypes, fascinating discoveries through research, and a continuous influx of inspiration through sounds around me, this collective is a start to my pursuit of sound and design, a begin-

Contemporary Soundscapespes

ning that has left me on top of the world and humbled to pursue in some form or another throughout my life. I thank you for coming on this journey with me.

“Now I will do nothing but Listen…

I hear all sound running together, combined, fused or following, Sounds of the city and sounds of the city, sounds of the day and night.”

- Walt Whitman, Song of Myself

15

LEXICON

A lexicon, word-hoard, or word-stock is the vocabulary of a person, language, or branch of knowledge. In this case, we are dealing with my explorations in contemporary soundscapes.

17

Soundscape:

Sonic:

Aural:

The soundscape is the component of the acoustic environment that can be perceived by humans. There is a varied history of the use of soundscape depending on discipline, ranging from urban design to wildlife ecology. We are dealing with the sounds of objects that surround us in the context of my work.

Denoting, relating to or of the nature of sound or sound waves.

Of or relating to the ear or to the sense of hearing.

Acoustics:

Medium:

Immersive:

The science that deals with the production, control, transmission, reception and effects of sound. We are dealing with acoustic engineering and psychoacoustics in my work.

An agency or means of doing something.

In the context of product and experience design, this refers to the medium of sound.

Providing, involving or characterized by deep absorption or immersion in something (such as an activity or a real or artificial environment). Used in the context of designing immersive experiences and products in the context of my work.

Audio Engineer:

Acoustic Engineering:

Sound Design:

An audio engineer (also referred as a recording engineer or a vocal engineer) helps to produce a recording or a performance. Editing and adjusting sound tracks using equalization and audio effects, mixing, reproduction and reinforcement of sound.

The branch of engineering dealing with sound and vibration. It is the application of acoustics, the science of sound and vibration, in technology. Acoustical engineers are typically concerned with the design, analysis and control of sound.

The art and practice of creating sound tracks for various needs. It involves specifying, acquiring or creating auditory elements using audio production techniques and tools. It is employed in various disciplines including filmmaking, television production, video game development, theatre, sound recording and reproduction, live performance, sound art, post-production, radio, and musical instrument development.

Universal Design:

Product Sound Design:

Consequential Sounds:

The design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design.

Inclusion of sound in the design process and its use as an essential aspect of controlling the quality of design. The sound involved in designing products.

Sounds generated by the operating of the product itself. Consequential sounds are the sounds generated due to the construction of an object and the medium of transmission of the sound around that construction.

Contemporary Soundscapespes

Intentional Sounds:

Augmentation:

Data Sonification:

Sounds that are intentionally added to a product. Intentional sounds are mostly composed and may be experienced as musical sounds. One could state that alarm sounds are, in fact, small musical compositions (i.e., musical motifs).

The action or process of making or becoming greater in size or amount.

Data sonification means representing data as non-speech sound. The basic principles are similar to visualization, but, where visualizations use elements such as lines, shapes and colours, sonification relies on sound properties such as volume, pitch and rhythm.

Psychoacoustics:

Ethnomusicology:

Sonic Branding:

The branch of psychology concerned with the perception of sound and its physiological effects.

The study of music in its cultural context.

Ethnomusicologists approach music as a social process to understand not only what music is but why it is: what music means to its practitioners and audiences and how those meanings are conveyed.

Sonic branding is about building a relationship between the product and its target market through the latter’s ears, fulfilling the role that a national anthem plays to a country or a hymn plays to a religion.

3D Audio Effect:

Skeuomorphism:

Sensory Design:

3D audio effects are a group of sound effects that manipulate the sound produced by stereo speakers, surroundsound speakers, speaker-arrays, or headphones. This frequently involves the virtual placement of sound sources anywhere in three-dimensional space, including behind, above or below the listener.

The design concept of making represented items resemble their realworld counterparts. Skeuomorphism is commonly used in many design fields, including user interface and Web design, architecture, ceramics and interior design.

Sensory design aims to establish an overall diagnosis of the sensory perceptions of a product and define appropriate means to design or redesign it on that basis. It involves an observation of the diverse and varying situations in which a given product or object is used to measure the user’s overall opinion of the product and its positive and negative aspects in terms of tactility, appearance, sound and so on.

Lexicon

19

GOALS & OBJECTIVES

The sense of hearing has agency that we do not address in our day to day lives. There is a constant widening gap between the two senses, with the visual given priority in the world that surrounds us. My work wants to be another push in exploring the capacity of this medium called sound. Specifically as a medium of design.

21

Design is limited to creating for senses in isolation and not integration

In our daily lives, we are immersed in sounds generated by products.

When asked to name sounds produced by products, people often mention those that alarm or inform us (e.g., microwave oven beeps, telephone rings). These are the sounds of which we are consciously aware. However, many sounds subconsciously play an important role in our interaction with a product. We can hear when the battery of a toothbrush runs out of power.

We can hear the power of a vacuum cleaner, and we can hear if the bag is full. And these are only the functional aspects; sound also plays a role in our aesthetic, quality, and emotional experience of products. For example, we can hear quality in the sound of a car door; car manufacturers have acoustical Contemporary Soundscapespes

engineers to make sure that a slammed door will evoke this sense of quality. To distinguish between sounds generated by the operating of the product itself and sounds that we intentionally add to a product, there are two categories in the field of product sounds. The first refers to consequential sounds, and the second refers to intentional sounds. This distinction is essential; both categories of sounds will require different design methods and the use of knowledge of different disciplines.

Intentional sounds are mostly composed and may be experienced as musical sounds. One could state that intentional sounds used as alarm feedback are, in fact, small musical compositions (musical motifs). There-

fore, these sounds can also be used to convey brand values of companies.

Consequential product sounds are experienced as ‘noisy’. It is very difficult for users, designers, and acoustical engineers to verbally express how they experience a sound. Several problems exist. In general, users lack the vocabulary to express themselves to explain what is wrong or right with a sound.

They normally will say the product makes an ‘unpleasant’ sound or noise.

Designers also lack the vocabulary to express design concepts that may be used in the design of a sound.

Acoustical engineers have a very technical vocabulary gleaned from the disciplines of physics and sometimes psychoacoustics, which does not com-

Illustrating goals within the context of the thesis timeframe. The current state, during the middle of the thesis(defence) and the intended state

municate very well to designers and users. In addition, understanding the aesthetic and emotional experience of product sounds requires knowledge from the field of psychology (auditory perception, cognition and emotion theories).

As stated before, product sounds are loud and noisy. This inherent property makes it difficult to describe the sound in a structural manner. The reason for this is, of course, that noise by itself is random and lacks structure. However, product sounds do not produce completely random noise due to the resonance and engine/boiler properties of products.

My exploration attempts to make product sound design more prominent in the design of objects as experiences. In tangible product design, intentional functions are integrated into the construction of the object, considered from the ground up so that sound and its forms can be used for more than a feedback mechanism or another augmentation to enhance the product experience. Taking the case of Alessi’s whistling bird tea kettle, the designer augmented the ritualistic tea drinking experience by adding an object sound (the bird) to enhance the experience of using the product. My work aims to use sound as a function in the tea-drinking process; for instance, a teapot could be redesigned to be used by the blind

with the sonic sense as its primary medium. We can build a teapot that can be used with only our aural and tactile senses, perhaps using a sonic cue to understand that the pot is filled and knowing where to pour using a frequency that resonates only when the neck of the pot is over the cup.

As we have seen in the past, most of universal design starts by designing for a niche, which then results in a better design for everyone. So, if we start to consider sound (and its other forms) as the primary medium, it might have disruptive consequences.

Goals & Objectives

23

AUDIENCE & MARKET

Since my exploration into sound began with my thesis last semester, I have had fascinating conversations with a wide spectrum of people working directly and indirectly in the realm of sound and/or design, all of whom have an inherent appreciation of the sonic medium.

This audience included musicians, interaction designers, creative technologists, music educators, sound artists, sound designers and an ethnomusicologist. This diversity enabled me to delve in the various facets of what sound consists of and can be. Articulating these multiple lenses helped inform me of the disruption sound is capable of as a medium in design and the current shortcomings it faces in the field. To get started, I focused on music education in American schools.

25

To design for music education, I chose middle and high school students as my primary users and music instructors at these schools as my secondary users.

Interviewing relevant stakeholders in this context helped me converge on specific problems faced by music education in its stagnated state in schools today and then design interventions to start tackling some of those problems.

This exploration proved to be a guidepost, fortifying my belief in sound as a powerful medium with much untapped potential. Armed with some knowledge and experience garnered in the past semester, I have begun my investigation into re-establishing the importance of the sonic medium in a visual world, prioritizing the aural as equal to the visual. This is done through design interventions that reimagine experiences and products to make people more aligned with their hearing sense.Designing these products will provide an overall integrated experience for users. This experience can be generalized into any product category and, thus, can serve all kinds of users.

However, it is best to have a starting demographic to allow for clarity on the problems being investigated and the corresponding needs to be fulfilled.

The audience that I am designing for spans a relatively wide spectrum from early adopters excited to try out a new way of experiencing things to people that will directly benefit from the function of the aural sense integrated into products.

Contemporary Soundscapespes

Music Education in Schools

Audience & Market

27

The Visually Impaired or Blind I consider this audience to be the benchmark or ideal threshold for my design interventions. Ideally, if my design is successful, people who have lost their visual capacities or are visually impaired will be able to ‘successfully’ experience a product using only their aural sense.

A quick search shows the limited number of objects designed purely for the blind. These are restricted to a few domestic appliances and some other crucial items. It is also common to use the sense of touch provide a tactile experience rather than using the aural sense. Highlighting the aural sense can prove ground-breaking for how these objects are designed. Another advantage of taking up an audience as challenging as this is the slight chance of the intervention becoming universal. An initial product may be designed to meet a very niche need but then become reflected into an intervention that benefits everyone at the macro level. A clear example of this is how OXO was founded based on a ‘good grip’ design for a potato peeler. Initially designed for a woman with arthritis who had difficulty peeling potatoes, the modification to the handle eventually revolutionized the handheld tool industry. Since then, every hand tool that we use today has been influenced by that one intervention. Inclusion of a specific group can feed into the way various products are designed depend-

Contemporary Soundscapespes

ing on the change in behavior caused by the initial design and the problem it solved. Similarly, designing for the aural sense will be most successful if it can serve people who are heavily dependent on capacities other than the visual.

Further investigation into product sound design for the visually impaired might yield some useful insights into how these problems can be approached.

OXO as an example of Universal Design

Audience & Market

29

Eargo Max Hearing Aids

Left : Sign for Deaf and Hard of Hearing Accessibility Right: Sign of Deafness

Contemporary Soundscapespes

The Sonically Impaired This is another niche audience that shares similar characteristics as an ideal threshold for design interventions. The difference in aural capacity from the previous audience has implications for entry into universal design but with a completely different approach. In this case, tactile design can be used in tandem with the sonic to design for alternate mediums of sound such as vibrations. The sense I am hoping to intervene with is affected differently, which makes problem investigation and corresponding interventions particularly challenging.

The products in the current market can serve as inspirational guiding posts for how I can approach designing these products. For instance, the hearing aids available in the market today are pieces of phenomenal industrial design due to advancement in technology and prioritizing the need to design for the differently abled. Thus, the design changes brought about by a specific need like this can be translated in how the aural can be prioritized in the design of everyday objects.

Audience & Market

31

Technology Enthusiasts

Audiophiles

Makers/DIY Enthusiasts

Some people just like technology for its own sake. They enjoy playing with all kinds of technologies. Most early adopters are technology enthusiasts in one respect or another—that is, enthusiasts often become early adopters when they want to put technology to use in some grand vision, making their first purchase when the technology is still quite young and not yet mainstreamed. However, the definition encompasses more than just that behavior. In short, most early adopters are enthusiasts, but not every enthusiast is an early adopter. Good marketing executives understand that.

“A person in pursuit of audio nirvana.”

The maker culture is a contemporary culture or subculture representing a technology-based extension of DIY culture [citation needed]. It intersects with hacker culture (which is less concerned with physical objects as it focuses on software) and revels in the creation of new devices as well as tinkering with existing ones. The maker culture in general supports opensource hardware. Typical interests enjoyed by the maker culture include engineering-oriented pursuits such as electronics, robotics, 3-D printing, and the use of Computer Numeric Control tools as well as more traditional activities such as metalworking, woodworking, and, mainly, its predecessor, traditional arts and crafts. The subculture stresses a cut-and-paste approach to standardized hobbyist technologies and encourages cookbook reuse of designs published on websites and maker-oriented publications. There is a strong focus on using and learning practical skills and applying them to reference designs.

Designing interventions for an audience that is curious and excited to learn about and use a ‘new’ product is an exciting avenue to explore. It is not as heavy a challenge as for the previous two audience types, much more general in context, and, thus, much more open-ended in terms of constraints.

There is also more room to design in terms of tangible and intangible objects with digital or non-digital interfaces. I am interested in designing for two particular subsets of enthusiasts: Audiophiles and Makers.

Contemporary Soundscapespes

This is one of the better definitions of the overtly ambiguous and exploited term for people who appreciate music or listening in general.

Consumption for this category is limited to Hi-Fi equipment to either record or play music; it does not cater to the overall need for living a more ‘sound’ life. I consider this category to be one of the early adopters of the integrated sensory experiences that I intend to design; product sound design is a relatively new field that attempts to bring sound to everyday objects whether it be constructional or intentional.

Another fascinating facet of this audience is their appreciation and ability to ‘listen’—something that I have been trying to push with the general population using a toolkit, among other interventions. The practice of clairaudience is only developed once we are trained for it as we live in a world that has only recently started heeding acoustically designed spaces and making our environments (particularly urban) easier on our ears. This audience consists of people who understand this importance and are willing to go the extra mile when it comes to choosing a product or experience for themselves.

Brooklyn Synth Expo : A mix of technology enthusiasts, Audiophiles & Makers

Audience & Market

33

MARKET

The field of sound design is massive.

For my thesis, I have focused on work being done in limited fields—namely, product sound design, sonic interaction design and sonic branding. It has been a riveting exploration as we begin to venture into giving sound a function in avenues that previously did not include the sonic sense.

The sound that a Sharpie makes when it rushes along paper is a very significant part of the user experience. It is intentionally designed into the product.

Various brands are getting into sonic branding as products evolve from mere objects to integrated experiences; the sound that a MacBook makes on startup is one of the oldest examples of this integration. Further, we are constantly surrounded by digital devices and are engulfed in the services they offer.

Consider the ‘post’ sound in the Facebook mobile app. Sound designers at Facebook designed this sound to create a simple yet satisfying moment when people post content, but they carefully considered its placement in the flow of the action. They saw an opportunity to make the sonic element more helpful by attaching it to the actual publication

Contemporary Soundscapespes

of the post rather than just the action of tapping the button. A few examples of firms working in these fields are the Lorelei Project and The Sound Agency, which are focused on sound branding.

Projects ranging from the sound of the Ducati logo to the acoustic sound for Heathrow airport demonstrate the significance of sound in enhancing a product or brand experience with disruptive statistics to back them up.

The effort to engage multiple senses within products and experiences is a field that is now gaining traction and booming like never before. I cannot wait to be a part of a world where sound impacts our designed environments.

Businesses know the importance of carefully designing the way they look, but very few remember to design the way they sound - The Sound Agency

35

PRELIMINARY RESEARCH

Exploring the fascinating world of sound has been one of the best things I have done in my life. I feel privileged to have talked with a variety of experts working in the field. Read numerous texts from the history of music to Wavefield Audio Synthesis. Learning about the golden ratio that forms our inner ear to experiencing Max/MSP in all its glory. Understanding ethnomusicology and the importance of psychoacoustics in sound design. This is just the beginning of a pursuit that will last my lifetime.

37

AURAL & VISUAL

Synesthesia is a neurological condition in which stimulation of one sensory or cognitive pathway (e.g., hearing) leads to automatic, involuntary experiences in a second sensory or cognitive pathway (e.g., vision). Simply put, when one sense is activated, another unrelated sense is activated at the same time. This may, for instance, take the form of hearing music and simultaneously sensing the sound as swirls or patterns of color.

The composer Olivier Messiaen, speaking of the union of color and tone in his music, explained to an interviewer: ‘’When I hear music, I see inwardly, in the mind’s eye, colors which move with the music. This is not imagination, nor is it a psychic phenomenon. It is an inward reality.’’

A 21-year-old woman, participating in an continuing synesthesia study at the National Institute of Mental Health, told researchers that when she ate buttered toast, ‘’it is rough, but not pointy; and if it has jelly on it the rough texture is rounded.’’

Preliminary Research

39

“I take a music class with four-year-olds. I get them crayons and a pad, and they colour and draw while listening to me play, helping them visualize the music.

There is a reflection of Synesthesia in this relationship between the visual and aural..”

- Daniel Tortoledo, Music Teacher

“Fundamental music theory has changed. To be a musician in the contemporary scene, a person needs knowledge of both classical music and sound design.

The current music curriculum does not account for the intersection or the entire journey.”

- Ritwik Deshpande, Interaction Designer

The relationship between Audio and Visual is much more than what we perceive on an everyday basis, which is why sound, even if input as a subtle detail has huge impact on the way we perceive an activity. With the popularization of using languages like processing to create art based on music, many more avenues of exploration have opened up where coding, conventionally a non creative task has creative output and has enabled expression in different ways.

Examples of such can be illustrated in websites like www.patatap.com, www.whalesynth.com and other experiments like Webgl Particle Audio Visualizer as a chrome extension.

Top : www.whalesynth.com Bottom : experiments.withgoogle. com/chrome/webgl-particleaudio-visualizer

Preliminary Research

41

AUDITORY ICON & EARCON

“Music-making using ubiquitous technology has enabled creativity in various ways, leading to a An auditory icon is a brief sound that is used to represent a specific event, object, function, or action. The sound takes advantage of the user’s prior knowledge and natural auditory associations between sounds and their results. In a sense, auditory icons are essentially caricatures of naturally occurring sounds. Natural sounds are preferred because they are related to events in a principled, systematic way (described by physics), and people learn this mapping from early childhood in their interactions with the world. In a sense, the sounds are meant to be the auditory equivalent of visual icons that are widely used in personal computing (where they represent objects or processes through graphical symbols), but in this case, represented by a specific, commonly-recognized sound related to the activity.

Direct relationships can be characterized as follows: a telephone ring can be used to represent the telephone, a car horn to represent a car; a train toot for a train, etc. The directness or auditory similarity between the auditory icon and the

Contemporary Soundscapespes

kind of democratization through objects like the Makey Makey.�

- Josh Corn, Designer

actual object can vary considerably.

But, as long as a sound evokes the associated sound of an object or action, it is classified as an auditory icon. The auditory icon need not be a realistic representation of the object/ function it portrays, but should capture the essential features, in which case it would be identified as a representational auditory icon.

Earcons are structured sounds, defined as a brief, nonverbal, distinctive audio sound/message to represent a specific event or convey other information and feedback to the user. Some examples of earcons (sounds) associated with personal computers are for example the arrival of a mail message, the system going offline in a few

minutes, the indication of a word that is misspelled in a user input or a sound equivalent of a file name or available tool. Earcons differ from Audtory Icons in that earcons are generally synthesized tones or sound patterns, and have no direct relationship to the event.

A learning process is involved for the indirect sound to eventually have a specific meaning. Although frequently associated with computer interfaces, earcons have a much longer history, such as the alert signal from an emergency braodcast system. The signature three tone melody used by NBC in radio and television brodcasts and audio alarms and signals of various types.

Repository of sounds designed by Facebook for their interactions

Preliminary Research

43

“Advancement in sound like Wavefield Audio Synthesis has enabled us to create soundscapes like never before. We finally have the capacity to play and control environments like never before.”

- Bobby McElver, Sound Designer

“My work is based on pure experimentation, recording sounds by activating materials like rice and glass.

Expanding our perception of sound through applying these recordings in a spatial context.”

- Wolfgang Gil, Sound Composer

SOUND NOW

with everyday technology and exploration around these ideas have both foreseen and unforeseen consequences to propagate sound as a medium of design in our lives.

Wavefield Audio Synthesis is a new spatial audio rendering technique, which has a very unique characteristic that is the localization of a virtual sound source does not chance with the listener’s position. In other words, it does not have a so called listening sweet spot. Developed in the late 80s, it has witnessed moderate popularity, it`s main drawback being the complex and expensive hardware necessary to operate. A typical Wavefield synthesis system can have anywhere between 16 and 500 speakers. Fortunately this aspect is overcame by the opportunities it provides, allowing a new dimension of audio content to be explored. There have been musical concerts developed for WFS, games, as well as a lot of scientific research. Now that Virtual Reality is becoming mainstream, WFS could provide the sound necessary for an amazing multisensory experience.

Progress in Sound Design has been incremental with technologies like WFS, Conversational User Interfaces and 3D Audio. Integration

Preliminary Research

45

A CO CREATION SESSION

One of the most wonderful things about music is its worldwide appeal and appreciation—the way it contains and represents our emotions both individually and as a collective. It has been the reason for people coming together to share experiences in all its forms and glory.

Designing for music education in schools is incomplete without these people becoming participants in the design process. Besides the general user research process and interviews, there is much to learn, understand and produce by changing our role from designer to facilitator and empowering users and experts to become designers.

This is a valuable exercise considering the scale and scope at which music operates as it resonates with everyone from a casual listener to practiced musicians and everyone in between.

The mission for this workshop was to get people interested in music and music teachers together in a room to identify problems in the current music education system in schools and then ideate and design around some solutions identified

Contemporary Soundscapespes

earlier and other prompts through my preliminary research.

Music education in schools has stagnated with a fixed method of instruction; a curriculum that has not been modified in the last fifty years; and participation limited to the choir, orchestra or band. Along with these issues comes the grave problem of no money being allocated by the board, which reduces the chances of any valuable resurrection even further. Considering the power and ability of music to harmonize people’s lives, it has been left to rot with sports and technology education taking first place after academics.

This workshop attempts to get people together to collaborate on this change.

The setup for the co-creation involved sending out invitations to target stakeholders, crafting an experience that keeps people enticed and engaged to participate, and reaching the desired outcome all while maintaining an atmosphere that is both empowering and constrained for direction. The carefully articulated prompts were crucial in the steering process.

The workshop started with smooth lo-fi electronic music playing in the background while everyone got comfortable. We started with an icebreaker exercise, which involved three short tasks of a minute each, to be done individually. The participants

were instructed to write words that came to mind when I said the term ‘music’; after that, they were asked to write the words that came to mind when I said the term ‘education’.

The last sprint involved making connections between these two entities while finding patterns and intersections between the two (music and education) standing in isolation ending with a casual prompt to write down their current favourite album.

Once finished, the participants were encouraged to head over to a stack of vinyl records and asked to pick their choice. This activity had a two-pronged motive where people would pair up into teams of two (based on their common selection of a record) and pick the prompts that they would work on (which were placed inside the record packets). This was followed by participants sitting together in teams to go through these prompts, which involved time for discussion, ideation and sketching out ideas.

47

Snapshots from the workshop in action

49

Paramount Importance of Music Education Funding for music education in schools is reduced every year, being secondary to sports and then turning into an afterthought. Arguments to re-establish significance are based on dependencies to help in performing better in academics and so on. How might we re-establish the importance of music and its education in schools as a significant part of education and life?

Dissonance There is dissonance between the music taught at school and popular music that kids are exposed to in the outside world. How might we address the tension of a child struggling with participation in traditional education not being appropriate for his/her self-image?

Contemporary Soundscapespes

Expression for All

Stagnation

Most students consume music at school instead of making it, setting them as spectators. How might we change that role to one of a participant?

The style of instruction and curriculum in schools has stagnated with limited options for participation (choir, band), and the course of learning is stringent, disciplined and strenuous. How might we increase participation and engagement among students?

Preliminary Research

51

RESULTS

These prompts contained three constraints, each operating as a starting point and guidepost to keep the flow for ideation alive and engaging. Once worked upon by the participants, each team was instructed to share some of their ideas and thoughts from the last 30 minutes. This enabled cross-pollination of ideas across teams and led to an insightful brainstorming session.

The workshop ended with a constraintfree, blue-sky individual exercise in which people were encouraged to sketch any ideas on what they would add to music education in schools if there was unlimited funding for music education.

All these exercises led to some fascinating insights and ideas ranging from product solutions to systemic changes in the current scenario. A workshop like this, where the users and experts became the designers, added much value to my pursuit.

An amalgamation of the results from the co creation workshop combined with research from subject matter experts and various other sources condensed to form a system map to be used as a guidepost in informing the forthcoming projects.

Contemporary Soundscapespes

53

55

DESIGN FOR MUSIC EDUCATION

Taking a fresh approach to the way music education is presented in schools. Defined by research, it stirs up the stagnated state of affairs using interventions interleaved within the current curriculum.

Output ranges from workshops to digital applications.

57

SCHOODIO

Music education in schools has been stagnating, suffering and losing relevance. The reasons for its slow death have been mentioned at various locations in this text.

This service attempts to be another push towards change.

Schoodio is a service that schools can enrol in that enables a group of musicians and professionals to carry out workshops in schools, in addition to and in tandem with the current music education curriculum. We give life back to music education in schools by offering a radical new approach to how students engage with and learn music and related practices.

The curriculum is based on interleaving the classical theory and method of instruction with a new-age experimental mindset that instils playfulness, enables creativity and keeps students engaged in the learning process. This is the first step towards change as the conventional method is strenuous, disciplined and direction-intensive. An experimental mindset is at the core of the new curriculum schoodio offers. The

Contemporary Soundscapespes

method of instruction also shifts from individualistic to more collaborative as

and Supercollider to generate their own sound using mathematics and

it enhances and changes how students interact with music and each other.

This changes the conventional role of an instructor to that of a facilitator.

physics with no prerequisite classical music theory. These new avenues can also entail the introduction and use of synthesizers and using purely electronic mediums to make music, which require different skillsets.

To rekindle interest and make music learnt in school relevant, schoodio takes an electronic approach to make students learn and create music regardless of whether they have prior knowledge or acumen in music. Tapping onto other skills and areas of interest, another offering that schoodio has is related to noise instruments or sound objects. This workshop is aimed at enabling students to learn how to make their own noise instruments using knowledge in science and math rather than any kind of music theory.

Students can discover wonder in making their own music from the get go and realize the potential of creation rather than just consumption of music.

Noise instruments can easily integrate with everyday technology with which students are now constantly surrounded. This provides a new lens for how music creation is seen.

Advanced workshops can also entail translating skills like coding to create music and sound. Creative Coding is an avenue with potential to be a form of expression like any other. With the increased focus on computer class and coding, students can translate those skills into using software like Max

Design For Music Education

59

SYNIC

Generation Z was born into screens.

The app stores are filled with a plethora of applications made for kids as young as three years of age. The toys are connected to the internet with complimentary apps, and building blocks consist of code elements rather than Lego. Exposed to extensive media since then, dexterity with technology is far greater than any time before such that pencils and crayons can be switched with keyboards.

My target demographic involves these students in schools where digital social media are as important as, if not more than, their real social lives and incredible amounts of content are created and consumed every day.

So much of their time is spent on these devices that not tapping into this medium as a purveyor of music education would be a great loss.

Flooded with visual interactions, most applications focus on the sense of sight unless intended otherwise. All the popular social media platforms deal in either words or the most ‘effective’ medium of communication—namely, images or short videos. Designing for

Contemporary Soundscapespes

such a demographic requires intense research on identifying points for low

of making music with the traditional way in balanced accounts. This

barrier of entry, enticing the user to start using the application and then keeping her/him engaged, and working through a user experience, which accounts for all the desired outputs especially when the underlying intent is music and learning.

technique has proved to be a better way to learn music while keeping participants engaged at the same time.

The primary aim of designing these screens is to get music education in the hands of this generation.

Everyone is consuming music constantly, setting them as spectators and not participants, and there is immense value in being a creator.

The benefits include personal expression and dealing with everything else that age has to offer.

Thus, getting kids to play music or create sounds in a highly accessible way can go a long way as a starting point to create music and get out of the traditional curriculum of band and choirs that schools offer. It also targets making music outside of the school schedule wherein students go home and engage with the application in a collaborative manner instead of the conventional individualistic approach. The application I design also considers that students operate on different levels of knowledge in music, thus keeping a low barrier of entry for the newbies while at the same time keeping seasoned students engaged with the application. The offerings rely on interleaving the experimental way

The process starts with imagining an accurate scenario in the daily life of one’s target demographic, going through step-by-step how a typical student spends her/his day, the things she/he feels and the way she/he spends time—things that are important for her/him and things that she/he might be interested in.

Personifying the user’s story is very useful to identify pain points and empathize with the user’s story, thus empowering the designer to accurately design for her/him. The storyline leads to a point where the user will first interact with the application and what that might look like.

61

To design for an audience that is focussed on visual stimulus, an ideal way to drive their attention towards the aural is to make an interface that sits at the intersection of the visual and aural. This integration allows the user to engage with the application by keeping a low barrier of entry and enough of a hook to get her/him started.

Another element this helps with is the reduced patience levels of the target generation; thus, the ‘hook’ has to be an invitation like no other.

For the first part of the design for this medium, we will focus on this part of the design process.

Making the ‘first’ page of the digital platform after the login was iterated and based on three different approaches.

The first approach was based on the theme of having an enticing graphic on the screen, which the user could interact with using swiping gestures.

This action would create sounds and music in different combinations, which were triggered based on the gesture it recorded. This was manifested by having a single-colour gradient distributed into layers on the screen. Any movement would create a beautiful ‘flow’ and mix of colours on the screen while creating music at the same time. The different layers constituted different complimentary tones, which would always sound

Contemporary Soundscapespes

pleasant to the participant.

The next step added a small pop-up tab on the screen, which introduced alterations such as a change in graphic and introductions to modifying pitch and tone of the layers on the screen.

This serves as an introduction to music terms and the basics of how changing these elements affects the sound created. An addition is the option to share with friends on the app wherein they can ‘create’ on the same screen and make it a collaborative effort.

The second approach was based on the theme of having a blank canvas on the screen with tabs open on either side (Landscape Mode). These tabs contain puzzle-like enticing graphic blocks, which, when set up on the screen in any arrangement, would create a harmony. Thus, placing visual elements creates tunes. This being the premise of this interaction, it would also contain elements from the last iteration to modify elements, sounds, and the sharing and collaborative feature.

For the generation obsessed with photo filters in the Snapchats of the world, the third iteration was based on creating music and sound by adding/ modifying filters on their photographs to share on their social media. Thus relying on current behaviours of modifying images, participants can create fun filters for their images, which create sounds at the same time.

Besides the main premise of the interaction, this application would also include elements from the previous iterations.After some user testing, the first approach was taken forward and developed with a few more screens on how the application might function and look like. This version is illustrated here.

Design For Music Education

63

CLAIRAUDIENCE FOR KIDS

Clairaudience is the ability to actually listen. Clairaudience for kids is an intervention, designed in the form of workshop to get school students engaged with the joy of listening.

A kit is provided to facilitators, it does not require any expertise in sound to use it for the workshop.

Contemporary Soundscapespes

Design For Music Education

65

Contemporary Soundscapespes

Design For Music Education

67

Contemporary Soundscapespes

Design For Music Education

69

Contemporary Soundscapespes

Design For Music Education

71

DESIGN FOR SPECULATION

The thesis exploration is a journey. And like with any journey, it has off beaten tracks that help navigate back to the primary path. These off beaten paths are design experiments that informed the projects around it. These necessary deviations are in the form campaigns against school shootings to awareness around noise pollution.

73

Constant speculation, experimentation and iteration have shaped sound and music since we have had the ability to hear. Throughout history, the meaning, definitions and explanations of what sound and music mean have constantly developed, and a few schools of thought have developed structures, words and instruments to make sense of the many sounds in nature, our vocal chords, various instruments and other agencies.

However, it has been impossible to define the magnanimity of sound and music, and our theories constantly evolve based on what we have heard and what we are hearing and speculating on what else we might discover or create.

It would be a shame to introduce speculative objects and not start with Pierre Schaeffer. Pierre Schaeffer is considered to be the founder of musique concrete, the significant transition point between where classical music and sound were and where sound is now. Trained as a radio engineer rather than a composer, he saw in the invention of radio, tape recorders and phonographs the potential for a new experience of sound, separated from its source, allowing sounds to have their own existence.

He coined the term ‘Sound Object’, which paved the way for a new kind of perception, ‘Acousmatic Listening.’

As explained by Andreas Bick in his

Contemporary Soundscapespes

blog Silent Listening, for Schaeffer, the ‘sound object’ is an intentional representation of a sound. With the rise of new audio technologies, the ‘sound object’ recorded on magnetic tape or phonograph (or any other medium that follows) does not indicate a sound source, such as ‘this is the sound of a violin or of a guitar’; rather, the ‘acousmatic experience’ refers to sounds that one hears without seeing the causes behind them.

Schaeffer’s insights were inspired by the progress of recording technology. He claims that the playback of a sound recording through a loudspeaker immediately performs the acousmatic reduction; the recorded sound is without its original cause.

Following a similar theme, by removing sounds from the flux of causality, Schaeffer’s sound object is irreducible to a physical core and, therefore, can be studied in a speciality until then unimagined. The new mode of listening enables the listener to construct a taxonomy of sounds capable of organizing and classifying the entire sound universe. Hearing itself becomes the origin of the phenomenon to study.

Schaeffer also suggested that there is an inherent timbre related to every sound being produced by a sound object. This sound is inherent to the object irrespective of change in variables like frequency, amplitude and time. Enough change in these three variables can indicate that

the same instrument is another with the drastic change in sound; however, the timbre is the property that identifies the instrument as the same. It is the property that gives colour to the pitch of a sound.

The concept and invention of these sound objects shows that music and sound have grown as a product of constant experimentation and iteration.

On that same sentiment, I attempt to create speculative sound objects as conduits or instruments to assist in solving for human interaction.

75

THE CHANTER

An instrument that opposes one of the biggest ills in our society, gun violence. In this instance, a wind instrument (the Sheng, an ancient Chinese wind instrument that creates beautiful, soft sounds when played) was made to look like a weapon of mass destruction. Thus, the sound object looked like a gun but was a purveyor of creation and peace rather than destruction. Following through with the idea of make music, not war, this attempt to recreate this instrument in a completely different concept acts as an important conversation piece to instigate difficult conversations and induce a state of emergency in the matter at hand .I imagine it to be effective as a part of a campaign against gun violence that invites spectators to something intriguing and then calls for action against guns.

Contemporary Soundscapespes

Design For Speculation

77

Contemporary Soundscapespes

Design For Speculation

79

Contemporary Soundscapespes

Design For Speculation

81

Campaigns were imagined in a variety of formats, I finally settled on making an audio ad campaign for Spotify. It started with an alarming sound of a machine gun firing followed by an emergency alarm and a call to action for support towards enforcing new gun laws.

Contemporary Soundscapespes

Design For Speculation

83

THE EQUALIZER

The second speculation involves enabling interactions. There are various platforms that enable human connection like never before. We are in constant contact with people far and near, sharing diverse media on many platforms, engaging users in different ways. It has bought a revolution in the way we interact and exchange ideas, thoughts and data.

But there is an entire population of humans bereft of this connectivity due to their disability: blind people.

Extensive study has shown how blind people feel disenfranchised with the current scenario of social media.

An attempt for inclusion of this population is made in the form of a device that enables the visibly challenged to use social media. Based on the primary sensations of voice and tactility, the device features a ‘screen’ and bold tactile buttons, which are easy to locate and interact with. Online data are converted into ‘images’ on the screen using the pin art method, which creates a relatively accurate ‘picture’ of the person or scenery. All these data are converted in real time. The ‘screen’ is limited to these pins, which are

Contemporary Soundscapespes

taken in as an accurate input by touch.

In an attempt to make all interactions fair, a voice modulator might

The other primary method for communication is voice and sound; a person’s profile will feature short messages, which they have recorded and choose to share with the world.

The voice is analogous for the elements that are usually written on conventional social media with elements like comments, posts, and calls, all made using voice. This enables blind people to discover and hear from new people.

control the loudness and tonality of the spoken word. For instance, if a person starts speaking too loudly, the modulator can modify the loudness of the voice to bring it down and, thus, maintain fair conversation. On the other hand, if one is too soft-spoken, the modulator can modify the voice to make it louder and present with more clarity. A conversation wherein the involved parties are literally speaking on the same level will prove to be a game changer in all interactions.

The third speculation involves enabling fair interactions. People come in various shapes, sizes and forms. We are a diverse species who differ based on gender, culture, inherent nature, the way we speak, how loudly we speak, and the way we communicate when we are in distress, angry and feeling numerous other emotions.

This can create much tension and disparity between two or more communicating parties in an interaction, which might be either personal or professional. There are many who raise our voices to prove our points and become more dominating to force our opinions, which kills confidence in the other party, making them susceptible to give in. Others talk softer to manipulate another and gain sympathy. Another common issue is that of extroverts and introverts. Extroverts tend to take over a conversation from introverts simply by their nature, regardless of who makes sense.

Design For Speculation

85

NOISE NYC

We have been losing our sense of listening. As humanity has progressed, we have been constantly modifying the natural soundscape for the worse.

There have been dire consequences since industrialization wherein we have created massive and destructive sounds and then tried to abate those through preventive measures.

This irreversible loop has reached a point where clairaudience remains a theoretical concept as our ears lose their capacities. Try turning the HVAC system off, and notice that odd difference. We are making white noise generators to help us sleep at night.

Thus, the choice for social enterprise seems obvious. The first step in reimagining a city soundscape is to arouse awareness around the grave subject of omnipresent noise pollution, especially in a city like New York. The noise code was extensively revised in 2007, for the first time in 30 years, to reflect the changing landscape and advances in acoustic technology. This covers an introduction to sound and loudness levels and various forms of noise sources such as construction noise, all

Contemporary Soundscapespes

“It would seem that the world soundscape has reached an apex of vulgarity in our time, and many experts have predicted universal deafness as the ultimate consequence unless the problem can be brought quickly under control.�

- The Soundscape

kinds of transport, animal noise, air conditioners, and music from bars.

However, most of this information is distant from citizens living their daily lives in NYC, be it in their workplaces or casual experiences like commuting through the subway. This organization aims to reduce this gap between noise pollution and awareness-driven action through a platform that uses current resources and technologies.

We start with a campaign to make citizens aware of the harms of noise pollution, democratizing language around sound levels, and understanding commonplace consequences of prolonged exposure with the introduction of a user-friendly online platform to personally understand and communicate these issues.

The phone-based application contains a few key features, the most important being the integration of a sound level meter. Approved and made by The National Institute for Occupational Safety and Health, this feature empowers regular citizens to check and record sound levels in their environments be it at home, on a commute or in the workplace. With an easy-to-understand user interface, it alerts the user if sound levels are over the safe limit and encourages action by offering solutions like hooking up earbuds, alerting the authority (NYC311 or NYPD) or avoiding exposure in the first place. With geo-tracking enabled, the application can also

estimate and share details like noise hotspots that one might be traveling through today or encountering at the workplace and suggest abatement measures. Enabling automatic mic detection could allow one’s device to suggest alerts and actions in real time.

noise absorbers. These small actions will ignite a ripple effect in making NYC even more safe and sound.

There are various instances wherein the general public is unaware of getting exposed to noise pollution and realizes too late with dire consequences like tinnitus, partial hearing loss and hypertension. For example, an MRI technologist might experience negative effects on her/ his hearing capabilities due to the loud, repetitive and disruptive sounds that the machine makes.

The mobile application also contains user-friendly quick links to the NYC Noise code (Established by the NYC Environment Protection Organization) and everyday abatement measures that a person can take up to mitigate noise pollution in her/his respective environments.

A collaborative effort with respect to using these resources will lead to the concerned stakeholders responding with action to assist citizens and actively mitigate noise pollution. For instance, many noise alerts coming from the union square subway terminal will force the New York City Transit Authority to offer earbuds on demand or apply permanent abatement measures in the form of

Design For Speculation

87

Contemporary Soundscapespes

Design For Speculation

89

AMUZE

This service is aimed at improving the logistics involved in keeping up with and attending live music events in a city like New York. There are hundreds of live music events happening every week, catering to an extensively diverse audience. The offerings include variety in genres, types of events, and scale in terms of number of people and venue size. The sheer number of relevant events is impossible to follow presented in randomized listings with multiple sources (Artists and Venues) on multiple platforms like Facebook or Resident Advisor.

Once noted, the paradox of choice is unnerving with multiple events happening on the same date or a better one happening somewhere else. This search can quickly turn into a timeintensive activity with major hassle across various touch points. Another turnoff is following multiple ticket vendors and keeping tabs on scheduling these events.Amuze enables people to discover, explore and engage in the live music scene in their cities by offering a highly curated, hassle-free experience. A typical user journey can be mapped as follows. Once a user has

Contemporary Soundscapespes

signed up on the mobile application or webpage, she/he is prompted to

interactions, heavily targeting new entrants to the city with the

enrol in a membership model based on time period (One to Six Months) and the amount of money she/he is willing to spend on live music events for that period. Like a typical music subscription service, she/he chooses preferences and curates her/his music tastes in terms of genres and venue types among other preferences.

That is all the user has to do. This is followed by her/him receiving a package with band swag and other objects to augment her/his experience.

age restricted to 21 and above.

The application now suggests expertcurated recommendations on the live events the user can attend. The tickets for these events are included in the subscription with the option to add guests and automatically sync with the user’s calendar. This enables her/him to experience a wide variety of the best gigs her/his city has to offer without any hassle. Some other features that are included are special occasion multivenue crawls, exploration of randomized events, community building, party hampers and other customizable options.

Amuze makes money through membership fees, artist and venue promotion, and event partners. We have expertise in interaction design and music and event curation to enable us to launch this service. We plan to get our first customers via social media and word-of-mouth

Design For Speculation

91

DESIGN FOR SOUND

These interventions aim to elevate sound as a medium of design, using Sound as an active medium than a passive one. Considering the vast amount of expertise in the field of Acoustic Engineering, Sound Design, Psychoacoustics and more, it attempts to democratize and empower its use through a variety of lenses ranging from a more experiential standpoint to adding sound to conventional industrial design.

93

WHAT THE SOUND

The intent was to move object sounds from the subconscious to the conscious level.Homing in on product sounds informed by continuous research led to a personal discovery in the fascinating world of Foley art.

The most important part of building an experience is to design for a feeling—to evoke that feeling in the experience. The entire experience from the get go must have that feeling as its premise. Starting from initial ideation to the assets to outreach channels to setting up a location and the actual experience, the desired feeling to evoke through What the Sound was discovery.

The purpose was to get participants to ‘discover’ the sounds that surround them. But how can one make a person listen better, pay attention more to the aural than to the visual, in a way that is interesting, engaging and enlightening at the same time?

Initial explorations included more abstract ideas, such as an array of speakers spatializing sounds of nature or a dimly lit room with a tree facade made using LED lights and speakers that create a distorted progressive narrative of, say, a bird chirping to more and more sounds. However interesting these avenues seemed, they did not have clarity in their intent.

Contemporary Soundscapespes

Foley (named after sound-effects artist Jack Foley) is the reproduction of everyday sound effects that are added to film, video and other media in post-production to enhance audio quality. These reproduced sounds can be anything from the swishing of clothing and footsteps to squeaky doors and breaking glass. The best Foley art is so well integrated into a film that it goes unnoticed by the audience. It helps to create a sense of reality within a scene. Without these crucial background noises, movies feel unnaturally quiet and uncomfortable.

Foley artists recreate the realistic ambient sounds that the film portrays.

The props and sets of a film often do not react the same way acoustically as their real-life counterparts. Foley sounds are used to enhance the auditory experience of the movie.

Foley can also be used to cover up unwanted sounds captured on the set of a movie during filming, such as overflying airplanes or passing traffic.

The term “Foley” also refers to a place, such as Foley-stage or Foley-studio, where the Foley process takes place.

95

Contemporary Soundscapespes

Design For Sound

97

The primary feeling to be evoked was a sense of discovery. Secondary emotions included awe and delight.

The exercise aimed to get people listening and interacting with products using the aural sense, which is the first step in thinking about consequences that sound design can have in our daily lives. The idea was to question, if we aren’t exposed to something, how might we notice and elucidate it in the first place?

The assets used for outreach were channels like Facebook and Eventbrite as well as posters placed in popular physical spaces around campus. The graphical assets were the start of an elaborate logistical hurdle. Foley is a very technology Contemporary Soundscapespes

and prop intensive activity.

The technology included speakers, mics, soundboard, pre-amplifiers, sound blankets and C Stands, among other things. Extensive research provided a plethora of objects of Foley.

“I like films where the music and the sound design, at times, are almost indistinguishable.�

- Christopher Edward Nolan, Film Director

99

101

103

Demonstration

Re-Enactment

Free Play

Participants enter the space in groups of three. The first interaction is performed by the facilitator. She/he gives the participants an introduction to Foley Art through an engaging activity. She/he asks the participants to choose an object and then has them shut their eyes to enable visual isolation and makes them guess the sound that the object is making.

After this, the sound is shown in the context of a small video clip that provides clarity on how the object is used to make sounds in films.

In this interaction, the participants become the Foley artists. The iconic egg hatching scene from Jurassic Park is recreated using specific objects used to make sound in the film. The egg hatching entails the egg breaking apart and a dinosaur baby coming out of its shell. These sounds are recreated by crumbling waffle cones, which makes the sound of an egg cracking, while another participant mulches the inside of a cantaloupe to produce gooey sounds followed by the third participant rubbing the squashed cantaloupe over a pineapple, which produces the sound of the doctor in the movie touching dinosaur scales for the first time.

This stage is just all-out free play.

Participants activate various objects of Foley through prompts provided on cards to create interesting sounds and knit narratives around using multiple objects in tandem or some other arrangement.

Contemporary Soundscapespes

105

Design For Sound

107

Contemporary Soundscapespes

109

Design For Sound

111

113

SONIFY

Sonic Interaction Design (SID) is the study and exploitation of sound as one of the principal channels conveying information, meaning, and aesthetic/ emotional qualities in interactive contexts. Sonic interaction design lies at the intersection of interaction design and sound and music computing.

Multimodality, especially the tight connection among audition, touch, and movement, is examined in an ecological framework to develop new design principles and apply these to novel interfaces. Confronting the challenge of creating adaptive systems that sonically respond to the physical actions of the user, SID explores questions of embodiment and performance and investigates how sonic gestures can convey emotions and promote experiences.

In this sense, SID follows the trends of the so-called third wave of human–computer interaction, where culture, emotion and experience, rather than solely function and efficiency, are the scope of interaction between human and machines.

Contemporary Soundscapespes

From the methodological point of view, this requires novel perspectives

within a real-world context.

that move away from the rigid guidelines and techniques that have traditionally been adopted in the sound and music computing communities.

Strictly technical recommendations and formal listening tests are moved aside in SID and replaced by design and evaluation principles that are more exploratory in nature. The design challenges are no longer of a technical nature because, nowadays, the wide availability of physical computing resources allows practitioners who are not technically trained to easily produce interactive sound artefacts.

To come closer to reaching such a complex goal, the field of SID, which is in its infancy, must engage with a wide range of research topics including the perceptual, cognitive, and emotional study of sonic interactions, improved models for the reception of sound and its role in performance of actions, adapted design methodologies, sound synthesis technologies and their use, and, finally, design and evaluation methods addressing the individual and social experience with sounding objects. For a new generations of sound designers to be capable of addressing the interdisciplinary problems the field raises, a more solid foundation needs to be developed that can draw on such bodies of knowledge.

Instead, the challenges lie in the ways in which designers may successfully create socially meaningful, physically engaging, and aesthetically pleasing sonic interactions. Thus, new pedagogical and design methods are needed to bring SID to professionals and students in a creative and ‘designerly’ manner.

For example, explorative studies of the connections between actions and sound in the real world help designers learn how to transfer the natural and ecological relationships to the development of intuitive sonic interfaces. The multisensory aspects of interactive sonic experience that SID is concerned with must be designed with consideration of the orchestration of the auditory, tactile, visual, and kinaesthetic senses

The emerging field of SID establishes sound as a force to reckon with as a medium of interaction design, going beyond the nominal use of sounds as a secondary consideration to a primary from the get go.

However, the motive behind the intent for the forthcoming design intervention was to democratize the use of sound for a large audience.

The first step to design is the identification and investigation of the target personas of the intervention. With the intent to go broad, the persona mapping exercise fleshed out an efficient demographic age and articulation that informed the initial ideation phase.

115

Contemporary Soundscapespes

Design For Sound

117

Initial ideation started with a blunt output of ideas, a recurring theme that would connect a demographic this broad was some sort of social media application as most of the current platforms are heavily visual. (Snapchat has a dedicated device in the form of spectacles to record footage and take pictures without using a smartphone!)

Entrenched with purely visual interactions, these social media applications have propagated a culture that gets people to edit every picture they take, adding numerous filters to modify their realities into a quest for likes.

Tapping on the ongoing addiction for these interactions, forthcoming prototypes shaped up accordingly.

The paper prototype talks of The Voice, an idea instigated by the thought of people being unable to look at their phones, like when they are driving or walking and running—any activity that occupies the hands and makes a person unable to hold the device.

Another premise was to pitch it against the social isolation bought on by constantly looking at a screen.

The user journey was as follows; After a vetted signup into the application, a typical newsfeed shows up. The only difference is that there are no visuals, only recordings. A participant can search sounds and stories using hashtags linking to an ever-growing archive of sounds from an event or trend. Taking the example of the #nycwomensmarch,

Contemporary Soundscapespes

participants tap into all the voice notes posted during the event.

Removing the visual, participants can listen to actual stories and hear significant voices like a podcast, thus, getting further insight into people’s personal stories and views on the event.

There is an option to listen to voices by location, so putting in a specific location brings out a vocal library of real stories and adventures that people experience daily. This location archive is also divided into categories that people choose to speak and post about.

The USP was to be able to navigate the application using aural cues with options to minimize interaction with the visual user interface and switch to a conversational or tonal user interface.

However, upon some user testing, it became clear that such an extensive application would prove to have too high a barrier of entry for participants engaging with a sound-based application for the first time, especially as it did not have something as enticing and easy to engage with as Instagram filters. The additional complications brought on by categories and voice-only notes did not add to the argument.

119

Contemporary Soundscapespes

Design For Sound

121

Imagine Instagram for sound—a very intuitive, easy-to-use user interface combined with a delightful, low-barrier-of-entry user experience that gets the participant directly into recording videos and adding sonic filters to modify the sounds in them. The application only deals in videos as still images have no movement and, thus, no sound.

A simple scenario for illustration might be a bus passing. As you record a video of it, you can apply a filter of a supercar sound. The application matches the frequency of the incoming signal and superimposes the sound over it, so it feels like a supercar went by, sonically speaking. This can then be shared with people on your contact list.

An advantage that sonic filters have over visual filters is the ability to create narratives. For instance, there is a scenario in which a person is angry at someone else and starts screaming at her/him. You can superimpose the sound of the person’s voice with the sound of glass breaking, so, whenever she/he opens her/his mouth, glass cracks are superimposed. In the same scenario, when the person being screamed at moves towards the door and slams it, you can superimpose the sound of a thunder strike. And, while the screen pans from the two to the door, you can still hear the glass breaking in the background as you would be able to in real life. This

Contemporary Soundscapespes

option is lost in visual filters as, when the camera pans over, the filter applied on an object in the previous screen is not visible as with real eyesight.

The application has a feature in which a participant can capture interesting sounds using the 3D touch feature on their phones and add them to the application to be used as filters while using the application.

123

125

Contemporary Soundscapespes

Video with audio filter superimposed over the sound of the bus, with the sound of a car

Design For Sound

127

Recorded sounds can be used as audio filters

Contemporary Soundscapespes

Design For Sound

129

STEPTONE

After delving into product sound design and understanding intentional and consequential sounds involving products. I aimed to add intention to the construction of an object, by adding analogue sounds to a conventionally tactile interaction.

I tried to imagine consequences by using sound as a medium of design. Steptone is the step ladder re-imagined. By adding sonic cues to the extreme treads, it augments the user experience of climbing up/down the step ladder by notifying the user of their position on the ladder without adhering to visual caution or attention.

This is primarily a safety feature that starts with assisting visually challenged folks to an older demographic to everyone. Designed in consideration with the variables that can be involved in affecting the sound in a step ladder interaction, like the kind of shoes one is wearing and whether the sound is loud enough to communicate it’s intended purpose.

Contemporary Soundscapespes

131

133

The process started by laying out most analogue sounds that one can listen to in a typical home. This experimental investigation helped inform various impact points in the user journey of navigating a household. Interesting concepts cropped up, for instance, if it is raining and one is standing on the balcony, we can hear the sound of rain crashing around us as an ambient collective sound. In reality, the droplets are forming different sounds as it goes in contact with the planter, the mud in the planter, the floor, the railing and so on. This is an example of how we perceive sound and the psychoacoustics involved in that interaction.

Design For Sound

135

Contemporary Soundscapespes

This initial prototype ladder was made using three materials namely MDF, Soft Wood and Hard Wood. With separate physical properties, they created different sounds when tapped onto. The idea was to attempt to create different sounds with the tap of the foot, further variation in sound was added by varying the density of the material by drilling variables of holes within each. This enables a non-digital solution to add sound to the ladder.

Next I head to the fabrication lab to build prototypes, and finally came out with a proof of concept that works installed on a readymade step ladder.

Continuous user testing informed various iterations involved through the entire process. The material used to make the sound was spring steel.

Design For Sound

137

139

141

DECIBEL DESIGN

4) Advertising sound: Sound in advertising on TV, radio

of the new Clinique Mascaras, product sound design is limited to either

and online platforms

luxurious objects, niche markets, or absolute necessity (for instance, feedback signals from a microwave).

Delving into product sound design and sonic branding through this thesis paved way for me to imagine democratized access to product sounds.

5) Branded audio: Highvalue podcasts, branded playlists and audio downloads

Bringing back the omnipresent ignorance of sound as a medium of design, you can take a look around you right now and notice the disparity indicated in this quote. The difference between time and effort spent in building landscapes and investment in building intentional soundscapes is disappointing to say the least.

Organizations like the Sound Agency are revolutionizing brand experiences through the use of sound. This is not limited to adding a sound to your logo, like the sound that shows up with the apple logo when you turn on a MacBook. Sound can create an entire sonic experience for the brand in more ways than you can imagine.

1) Brand voice: What you say and the way that every human voice representing your brand says it 2) Sonic logo: A short, powerful audio mnemonic that represents your brand 3) Brand music: Custom composed and/or licensed music used in any branded context

Contemporary Soundscapespes

6) Telephone sound: On-hold sound, ringtones, call centres and interactive voice response systems 7) Soundscapes: The sound of commercial spaces from malls and airports to shops, reception areas, offices and warehouses 8) Product sound: From the fizz of a can of soda to the roar of a motorbike Recognition of the paramount importance of intentional sound design is still growing, but the work of firms like The Sound Agency is a start in making our soundscapes sonically engaging. All this work is based on desired behavior change by sound design. These psychological changes are quantifiable, and analytics proves their effectiveness.

There are extensive sound libraries available on the web ranging from music clips to door creaks, from alien feedback signals to sounds of nature and many other obscure sounds one can imagine. All these sounds are readily available for download and use. However, like the signature thud of a BMW door to the satisfying click

“Take a look around you. Most of what you see was consciously designed by someone—from the wall colour to the door handles. Now think about the sound that you’ve experienced today. Chances are, most of it has been accidental, unpleasant, and counterproductive..”

- Julian Treasure, The Sound Agency

143

The democratized access I envision would be enabled through a service called Decibel Design. This service provides an extensive database of product sounds through a digital library and consults with brands and individuals to construct and integrate sounds for their product offerings, covering all their sonic needs for a variety of interactions taking place in your object.

User Journey can be illustrated as; a new stationary brand is being established called Sense. They plan to offer a variety of sophisticated stationary, starting with pens. Looking to enter the high-end market, they are at the initial Industrial Design stage.

Understanding the importance of products as experiences, they are looking to go beyond the visual aesthetic to create a pen that offers a multisensory experience.

They start by signing up on the web platform, which encourages the user to wear headphones while navigating the website. The homepage consists of various useful links, including a featured sound of the day. Once signed up, they can access the sound library.

The library presents a variety of products and their sounds from domestic appliances to toys and tools. The user selects pen sounds with the option to choose the kind of pen and access to all the sounds corresponding to the interactions happening in the object—

for instance, the sound of opening/ closing the pen or the sound the nib of the pen makes on a particular kind of paper. The website provides intricate detail in the sound selection process.

Once satisfied with a kind of sound, they can consult us regarding the construction and mechanics of the pen. For instance, a certain kind of threading might be needed on the neck of the pen so that it sounds a specific way when the cap is pressed back on.

For maximum benefit, participants are encouraged to include us in the beginning of the design process.

nizations that would not approach multisensory design due to lack of sound design teams at their facilities.

The expertise required to build such a platform would be inclusive of but not limited to Sound Designers, Industrial Designers, Interaction Designers, Mechanical Engineers and experts in psychoacoustics.

As a result, they obtain a unique, satisfying sound for their object interactions, which can become a ‘signature’ sound for their brand.

Anyone in the world would be able to identify this pen by the sound it makes even without looking at it.

To popularize product sound design and get people invested in its consequences, Decibel Design offers membership to schools, universities and other educational institutes and individual audio contributions. For instance, if a new sound is developed by an individual, she/he can post it on the library after it gets approved. If a firm decides to buy that tone, that individual gets royalties for the transaction.

Thus, Decibel Design provides accessibility through service by enabling opportunities for orga-

Design For Sound

145

147

149

151

BIBLIOGRAPHY

The works or a list of the works referred to in a text or consulted by the author in its production. In this case, references to the wealth of knowledge that informed my work for sound.

153

Oellers, Helmut. “3. Wave Field Synthesis.” Wave Field Synthesis, holophony.net/Wavefieldsynthesis.htm.

“The Rise Of Intelligent Conversational UI.” Smashing Magazine, 5 Apr. 2018, www. smashingmagazine.com/2018/04/rise-intelligent-conversational-ui/.

Techlabs, Maruti. “What Is a Conversational UI and Why Does It Matter?” Chatbots Magazine, Chatbots Magazine, 13 Oct. 2017, chatbotsmagazine.com/what-is-a-conversational-ui-and-why-it-matters-de358507b9a2.

Eckler, Daniel. “Conversational User Interfaces – The Mission – Medium.” Medium, Augmenting Humanity, 6 Apr. 2016, medium.com/the-mission/the-future-of-cui-isn-t-conversational-fa3d9458c2b5.

“Http://Ljournal.ru/Wp-Content/Uploads/2016/08/d-2016-154.Pdf.” 2016, doi:10.18411/d-2016-154.

“Kahn Project: Composing a Space.” The Infinite Fractal Universe, www3.amherst.edu/~kahn/kahn_info.htm.

ABA. “( ( ( ABA ) ) ) Audio Branding Academy.” ( ( ( ABA ) ) ) Audio Branding Academy, audio-branding-academy.org/.

Kraemer, Brandy. “Learn About Musical Symbols in Piano Music.” ThoughtCo, ThoughtCo, www.thoughtco.com/musical-symbols-in-piano-music-3893554.

“Indian Music Glossary.” Agra India – History, Architecture, Facts, Myths, Visit Timing & Entry Fee, www.culturalindia.net/indian-music/music-glossary.html.

ContinueShow. “Continue?” YouTube, YouTube, www.youtube.com/user/ContinueShow.

Nypl.org, nypl.org/locations/heiskell.

“How to Build a Quiet Restaurant.” Food & Wine, www.foodandwine.com/blogs/how-build-quiet-restaurant.

MCLbanaan. “Andre Rieu - Ave Maria (Maastricht 2008) DIGITAL TV.” YouTube, YouTube, 24 Aug. 2008, youtube.com/watch?v=TkY9HtwXNU8. aaag25. “Simon & Garfunkel - The Sound of Silence.” YouTube, YouTube, 10 May 2010, youtube.com/watch?v=dTCNwgzM2rQ. earthtouch. “Holey Foley | Sound Design at Earth Touch.” YouTube, YouTube, 19 Dec. 2012, www.youtube.com/watch?v=Li6TSwybqjU. “The Magic of Making Sound | That’s Amazing.” YouTube, YouTube, 12 Jan. 2017, www.youtube.com/watch?v=UO3N_PRIgX0. getintofilm. “How To Make Your Own Foley Sound Effects with Peter Burgis (Interactive On Web

Contemporary Soundscapespes

Browser Only).” YouTube, YouTube, 21 Dec. 2016, www.youtube.com/watch?v=cg_daxoXMaQ. “Oddio Studio Inc.” YouTube, YouTube, www.youtube.com/channel/UCQiUejWq__8JWqvbLRZ8NQw/videos.

“Home - PubMed - NCBI.” Advances in Pediatrics., U.S. National Library of Medicine, www.ncbi.nlm.nih.gov/pubmed.

Day, Aaron. “Why Strong Sound Design Is Critical to Successful Products.” O’Reilly Media, Inc., O’Reilly Media, Inc., 8 Sept. 2017, www.oreilly.com/ideas/why-strong-sound-design-is-critical-to-successful-products.

“Product Sound Design.” Product Sound Design, productsound.wordpress.com/.

Staab, Wayne. “Auditory Icons, Earcons, and Speech | Wayne Staab, PhD |Hearinghealthmatters.org/Waynesworld/.”

Hearing International, 24 Sept. 2014, hearinghealthmatters.org/waynesworld/2014/auditory-icons-earcons-speech/.

“Http://Ljournal.ru/Wp-Content/Uploads/2016/08/d-2016-154.Pdf.” 2016, doi:10.18411/d-2016-154.

“Sonic Interaction Design.” Exhibited Works : Sonic Interaction Design, sid.bek.no/Works?filter=exhibition.

“Bodystorming.” Wikipedia, Wikimedia Foundation, 7 Apr. 2018, en.wikipedia.org/wiki/Bodystorming.

The MIT Press. “Sonic Interaction Design.” The MIT Press, The MIT Press, mitpress.mit.edu/books/sonic-interaction-design.

“For BrandSound Optimisation, Soundscapes, Sonic Logos, Brand Sound, Audio Marketing.” The Sound Agency, www.thesoundagency.com/.

“Free Soundless Clips.” Sound Design Stack Exchange, sound.stackexchange.com/questions/9632/free-soundless-clips.

“Patatap.” Patatap, www.patatap.com/.

Wellington, Ben. “Mapping New York’s Noisiest Neighborhoods.” The New Yorker, The New Yorker, 20 July 2017, www.newyorker.com/tech/elements/mapping-new-york-noise-complaints.

“IRCAM.” Accueil, www.ircam.fr/.

“How to Bring Music Into the Classroom (Even If You’re Not a Music Teacher).” Rubicon, 23 Feb. 2018, www.rubicon.com/bring-music-classroom/.

“Cycling ‘74.” Cycling ‘74, cycling74.com/.

Walls, Seth Colter. “Surround Sound? You Ain’t Heard Nothing Yet.” The New York Times, The New York Times, 14

Bibliography

155

July 2017, www.nytimes.com/2017/07/14/arts/music/surround-sound-you-aint-heard-nothing-yet.html.

“Mesh2HRTF.” mesh2hrtf.Sourceforge.net, mesh2hrtf.sourceforge.net/.

“The Future Of Game Audio - with Ben Minto.” A Sound Effect, 5 Apr. 2017, www. asoundeffect.com/game-audio-future-ben-minto/.

“Envelop.” Envelop, www.envelop.us/.

Ecomusicology, www.ecomusicology.info/category/er/.

DrG. “Musication: Interleaving Methods and Curriculum for Well-Rounded Music.” Alan J Gumm, 17 Aug. 2017, alanjgumm. wordpress.com/2016/10/20/toward-a-well-rounded-music-program-interleaving-a-broader-methodology-and-curriculum/.

“Week 2: Acoustic Spaces: Resonance and Amplification.” The Art of Sound and Noise, 23 Mar. 2018, artsoundnoise2018.wordpress.com/week-2-acoustic-spaces-resonance-and-amplification/.

“Math 5: The Mathematics of Music and Sound - FALL 2011.” Does Capital Punishment Deter Murder, math.dartmouth.edu/archive/m5f11/public_html/.

“Math 5: The Mathematics of Music and Sound - FALL 2011.” Does Capital Punishment Deter Murder, math.dartmouth.edu/archive/m5f11/public_html/.

Metz, Cade. “Google’s AI Invents Sounds Humans Have Never Heard Before.” Wired, Conde Nast, 3 June 2017, www.wired.com/2017/05/google-uses-ai-create-1000s-new-musical-instruments/.

“British Library Sounds.” World and Traditional Music | British Library - Sounds, sounds.bl.uk/.

“Arup.” ‘s Cork Office Becomes First in Ireland to Achieve Leading-Edge WELL Certification, www.arup.com/da/perspectives/themes/Transport/Soundlab.

Langeveld, Lau, et al. “Product Sound Design: Intentional and Consequential Sounds.” Advances in Industrial Design Engineering, 2013, doi:10.5772/55274.WW

Contemporary Soundscapespes

Bibliography

157

GRATITUDE

Anmol, Anita, Sanjay, Shitzi, Pari, Aman, Deepanshu, Kusumaker, Ajitesh, Mansi, Heena, Utkarsh,Rajesh, Shiv, Shivani, Meghal, Meenu, Sunita, Piyush, Surabhi, Tanu, Bela, Parul, Prashant, Allan, Andrew, Krithi, Marko, Alisha, PoD Class of 2018, PoD Class of 2017, PoD Class of 2019, Sinclair, Brent, Steven, Emilie, Bill, Hannah, KT, Abby, Rest of the amazing faculty @ PoD, John, Maya, Tak, Anne, Oya, Wolfgang, Claire, Juan, Avinash, Renu, Neha, Hitesh and all the names I have forgotten to mention that have been a part of my journey. I feel immense gratitude, thank you for your support, inspiration and energy.

159

MOVING FORWARD

The past year has been a privilege. Having the opportunity to immerse myself in a world I have been curious to explore for the longest time, through a variety of design lenses. Throughout this journey, I have discovered, learned and lived the massive world of sound. And after a year of work, even though I haven’t been able to scratch the surface, I am reaffirmed in its power as a medium more. This is just a start in what is a pursuit of a lifetime. There are so many things that I am looking forward to work on and experience, I will get on with it and leave those stories for another book. I thank you for your attention and time.

161