My primary thesis exploration is inspired by my experience playing instruments within a Gamelan ensemble. Gamelan music has an astonishing calming effect on most audiences. In my own experience of playing in a Gamelan ensemble, I found the progression of tones and resonating sounds induced feelings of peace and gives me a sharper focus. The beautiful and resonating sounds are like spring water that washes away all my worries and uneasiness.
My thesis project gameLAN 2.0 suggests a new way of playing the traditional Gamelan . This project is a new iteration of my gameLAN project. The gameLAN was a website interface that engaged the user in playing the Gamelan instruments by pressing the keyboard. I created an audio-visual installation with the guidance of my thesis advisor, Jan Kubasiewicz. I had help with programming and prototyping from an electrical engineer, Fred Wolflink. Using the Gamelan physical interfaces we have built, the participant can engage a physical action of striking with a mallet. It does not simply replicate the classic gamelan ensemble but does retain the striking action used in traditional Gamelan performance.
Gamelan refers not to a specific instrument, but a traditional musical ensemble from Indonesia that features gongs, drums, metallophones, xylophones and bowed strings instruments . In building the Gamelan, the metal smith first forges the gong. The rest of the instruments are tuned to the gong in order to make a unified, orchestral family. When we were in the ensemble, we used to joke about it by saying it is the “original heavy metal.” The uniqueness of the Gamelan is that it is an ensemble of instruments played by a community of twenty to thirty musicians. Each musician’s instrument sounds in only one or two octaves. When the ensemble plays together, it has a range of five or six octaves.
When I was playing with the group, I felt that we were not only communicating with each other beyond language, but also communicating with the ancient culture of ancestors through these traditional instruments. In gamelan music, we focus on a group effort rather than individual artistic expression. Before we got on the stage, every time we had a performance, our mentor I.M.Harjito would gather the members and encourage us and thanks us for a wonderful semester. Then everyone would put our hands together saying “Gamelan for life” to pass on positive energy.
By introducing gamelan into the digital domain, I have created a new platform that will allow me to simultaneously translate the tones to visual output, such as light, shapes or both. I realized that Gamelan could be a tool for others seeking meditative focus, and by connecting the sound with its visual representation.
As part of my research, I interviewed my colleagues in the ensemble about how playing the music affects them physically, mentally and spiritually. Here’s what they had to say.
In these quotes, we can find the keywords of my thesis research: “being present through the practice of sound.” It has also strengthened my belief that the sounds have a meditative quality and bring us to a state of focus. Traditionally, Javanese Gamelan can accompany dance or shadow puppetry. Thus, I wanted to experiment with abstract visuals which could be synced with the instrumental output to have a similar claiming effect on people. I am experimenting to see what kind of visuals could be synced with the instrumental output to have a similar calming effect on people.
Royal Hartigan Percussionist / Pianist / Tap Dancer
When I play Gamelan, I feel I am in another place, beyond the work-aday world, beyond society, beyond culture, even beyond time and place, a truly spiritual path. It forces me to see things in a real and deep way, something that is not common in a materialist, self centered society full of technology and no heart or soul.
—
Rik Llyod
Guitarist / Vocalist / Composer
When I play Gamelan, I feel pensive. I almost feel like I’m meditating-the repetition really makes me concentrate a lot and the bells are soothing. I like the idea that it is so culturally different from the U.S. aesthetic and the mysticism of it intrigues me a lot. The scales are totally different than western music and the way they write their music is fascinating to me. The specific sound of the bells is really interesting too, and the arrangement of the instruments is unique to Javanese culture as far as I have seen.
Logan AmaralGuitarist / Drummer
I like Gamelan because of how relaxing it is. I also like the fact that each instrument on its own is relatively simple (minus a few), but creates a complex overarching dialogue with the rest of the ensemble. Finally, I find it fascinating that Gamelan music blurs the lines of what Westerners feel as ‘downbeats.’ There are multiple ways to distinguish starting points within the music.
Javanese Gamelan
Ensemble at the University of Massachusetts, Dartmouth.
I researched and learned more about the structure and terminology of Gamelan music. The beginning of the melody in Gamelan music is marked by the gong. The rest of the instruments then come together producing a rich layered rhythmic pattern. The notes in a Gamelan scale have philosophical significance. They are from low to high as can be seen in the chart below:
head (forming the neck torso, basic chest physical structure)
five the five senses; sight, smell, touch, taste, hearing
six meaning feeling/ understanding, a spiritual sense
(Lindsay 27)
When I read about the relationship and meaning of each notes and how they resonate with our bodies, it makes sense to me when I recall how the energy created by playing these beautiful sounds had tried to communicate with me. I finally realize why when playing the Gamelan I feel that it is trying to communicate with me spiritually.
As Lindsay writes:
For the musician, playing Gamelan helps him to limit his own emotions and feelings and further reach the state of detachment, the concept of detachment expressed the need to escape from the unending cycle of happiness and unhappiness.
(Lindsay 35)
The name “Gamelan” means “striking them with a mallet.” So the next step in my project gameLAN 2.0 was to find a way of engaging the audience by physically striking the interface. In the Design Symposium class held in Spring 2014, we read Bret Victor’s A Brief Rant on the Future of Interaction Design. Victor invents interactive tools that enable people to understand and create. He pointed out in his essay that the current state of interactive tools limited our sense of touch and the manipulations that a hand can do to interact with objects. He urges designers to break out of the “moving pictures under glass” paradigm and realize how sensitive and powerful our hands are. He believes that we can do so much more than sliding our fingers on a cold, flat surface. (Victor)
At the same time, I am interested in how the Gamelan sound can be translated as a visual representation in such a way that the visual would have a similar calming effect as the sound itself. So I began to research the term “audiovisual” and terms that relate music to the visual. I also found myself collecting phrases that join our sense of hearing and seeing, such as ”music you can see, art you can hear,” and “see this sound.” (Daniels)
In my own work, I often find inspiration from traditional practices and bringing the ancient into modern life. Since my gameLAN 2.0 one such project , I began to pay attention to the projects that use Gamelan instruments in a new way.
One of the many projects I discovered is the Gamelatron. It was created by the artist Aaron Taylor Kuffner. The project consists of kinetic sculptures that play the traditional Indonesian Gamelan using mechanical mallets. The artist hoped to create harmony by combining emerging technology with the “thousand-year-old sonic tradition of Gamelan.” The kinetic sculptures play a composition written by the artist. In my own project I aim to provide an interactive platform that encourages participants to produce their own musical pieces.
The work shown here is another project that uses Gongs called Water Gongs It was created by David Birchfield, Kelly Phillips and David Lorig. It is a sound installation made of seven autonomous gongs partially dipped in water. The very small waves generated from the vibration are projected on the ceiling through a lighting system located at the bottom of each tank. The reflection on the ceiling creates light shows when a gong strikes a tone. In John Cage’s documentary I Have Nothing to say and I am Saying It, he placed the gong under water and also experimented by filling a shell with water to produce unpredictable gobble sounds. Water can take the shape of the container it holds. It inspired me to consider water as an element in my own work.
Gamelan Wizard is a pinball game which incorporates Gamelan instruments into its play field. The artist Lucas Abela took advantage of the Gamelan instruments for their beautiful acoustic sound. When visitors saw this installation, they intuitive knew how to play the game and started playing without needing any instruction. By putting together the Pinball game and Gamelan, this artist created a unique participatory installation that meshes together arcade culture and eastern traditional musical elements.
Interactive Augmented Gamelan is an installation created by Charles Matthews/ Ardisson. It explore the combination of Gamelan and electroacoustic music. Sensors are attached to the instruments and the sound is extended and altered through custom software. The lights come on when the musicians play the instruments.
Steve Reich, a minimal composer, in his book Writings on Music he compared Gamelan music with Western music. Reich wrote about the repeating patterns and rhythmic structure of non-western music. He was fascinated with Balinese scales and particularly the timbres of the instruments. Reich would very much like to expand this idea into the making of his music. As he said:”I didn’t want to sound Balinese or African, I wanted to think Balinese or African.”(Reich 148) In my thesis project, I kept this idea in mind that “I wanted to think like Javanese,” too.
The design process consisted of two parts. First, I needed to build the physical Gamelan interface that triggers the sound samples. Next, I had to experiment with the visual representation of the Gamelan sound. These two parts of the process connecting coherently create the whole experience.
The Gamelan Interface
At first, I was building prototypes that used piezo sensors to detect when the knock was performed. When the participant knocked the instrument with a mallet, the vibration was picked up by the sensor, and the signal was sent to the computer, which played the corresponding sound sample. Sound samples from the project may be downloaded at marsudiraras.org (under creative common license). In the future, I would like to collaborate with a sound specialist to record my own samples of Gamelan.
The embedded sensors in the early prototype were very fragile. When I was testing them with users, one of my instruments broke when the participant hit it too hard. So I started to build a stronger version and at the same time thought about alternatives. I brought the first prototype to our Javanese Gamelan Master, I.M. Harjito. He was delighted to learned that I was making Gamelan accessible in a digital domain. He even smiled and told me that now I can practice Gamelan at home and play instruments in the real Gamelan rehearsals.
Fred suggested using Korg nanoPad2 as the hardware instead of building a new instrument using Arduino and piezo sensors. The nanoPAD2 is a MIDI controller that senses the key velocity, so that a soft approach to the key pad will result in a soft sound sample. The mechanism of the existing nanoPad is a lot more reliable.
Using the laser cutter, I built a prototype box to house the nanoPad2. I experimented with different materials such as plywood, museum board and cardboard and settled on wood as a base material, because the natural wood color seemed more welcoming than the other material. This can contributes to the entire experience. My decision was not based simply on the color. Wood is used in the traditional gamelan instruments, and I had felt that wood could communicate a spiritual tone.
The Visual Output that Responds to Gamelan Sound
I discussed the possibility of collaboration with my classmate, Katie, who is trained in yoga. Katie wanted to incorporate the yoga philosophy with modern technology to let more people experience the benefit of balancing and being present. We had the idea of syncing up gamelan instruments with an ever-evolving mandala computer generated sketch based on the instrument input. Another idea we came up with was playing the Gamelan tangible interface trigger the Om meter to light up. The seven notes in Gamelan scale can map to the seven color of Chakra. The light will gradually fade out at the same amount of the time as the sound. Both of the idea didn’t worked out because simply the color lighting up responding to sound is not interesting enough visually.
One ordinary Sunday afternoon, I noticed an interesting reflection on my wooden wardrobe. Tracing where it came from, I discovered it was my plastic folder sitting on the edge of my desk casting organic and complex forms. It looked like multiple representations of the sound waves weaving together. The shapes suggested dimensions to the space as well. The light and shadow created by this simple mechanism was very appealing to me. The mundane everyday objects in our lives create magic when we’re not looking. I decided to manually moving the plastic materiel, and recorded video footage of the light reflections. I then crop the videos into a few seconds to map to the Gamelan sound. I have built an early prototype using the program MAX. The user hits the keyboard to trigger the Gamelan sound samples as well as the light reflection video clips cross fading in and out.
Fred introduced a few MAX audio-visual jitter recipes and helped with me using the nanoPAD2 to use the sound samples as input and triggers visual output. In its current iteration, the visual representation of the notes created a deep purple, silk-like texture as background with ocean blue swirling lines on the top. When a tone is sounded, color first appears in the center of the screen then spreads to the edges as the tone resonates. The intensity of the strike determines the brightness of the color. I imagine it as a peaceful lake. A strong strike is like a fish jumping, causing dramatic swirls out from the center to the corners of the projection. A softer strike, creates less dramatic flow of color and a projection of less apparent depth.
Light Reflection Studies
After my unintentional discovery of the interplay of light and shadow created by an ordinary plastic folder, I have paid more attention to the beauty in mundane objects. For example, when I was sitting in a coffee shop with the sun coming through the glass door, the adjacent wall became a playground of light and shadow as people walked and cars sped by. When I am on the road, I see the sunlight bounce off the metallic surfaces of vehicles, creating light shows everywhere. I can look down at the asphalt and see the reflections projected by the wheel rims as cars speed by. This delights me.
I began to experiment with light projection using different filters such as reflective mylar sheet, dichroic film, lens and magnifying sheet to create different projections of light. Using this method, I was able to abstract the visual output. The visual elements were similar to a sound-wave form with some kind of dimension suggested. I was also inspired by the aurora, also known as the northern lights.
In Audery Yen-Ning Fu’s DMI thesis book Perceiving Interaction, she walks us through her experiments. One of the valuable insights I took away was that the system contains three layers: the light source, the physical filter and the spatial surface. The artist can alter any of these elements to create an infinite number of light effects. Inspired by Audery’s work, I also created a list of the materials I experimented.
Light point source light projection (image/ video)
Filters
plastic folder
mylar sheet water
mirror acrylic dichroic film
magnifying sheet
Surface wall hand-made paper
In the Language of Motion class taught by Professor Jan Kubasiewicz, I experimented with real light reflections as the visuals. I created a short film in homage to Oskar Fischinger, a great abstract filmmaker. I projected the cover of the book, Oskar Fischinger 1900–1967: Experiments in Cinematic Abstraction, which features stills from Fischinger’s film Kreise Then I printed out the book cover on a clear film and, using it as a filter, superimposed the moving cover on the still cover. The colors and pattern were layered creating complex visuals. I then rolled up a sheet of reflective mylar to create a tunnel in front of the projection. I also took advantage of the dynamic characteristic of water by filling a glass jar of it and gently shaking it, resulting in a dynamic filter.
These experiments opened up a lot of new possibilities in my use of colors and moving images. Since I moved the filters by hand, it was not as easy for me to come up with mechanical moving parts to control the visuals in an organic way as I had imagined, but I achieved my goal of obtaining analog visual effects and changing colors.
