Metaphor Memory and Story
2012 Researched, written, and designed in partial fulfillment of the requirements for the Master of Fine Arts degree in Graphic Design at the School of Art and Design, College of Architecture and the Arts at the University of Illinois at Chicago. All statements are the responsibility of Matthew Wizinsky. — On the cover Electron microscopic image of fresh snow from the Dartmouth College Electron Microscopy Facility. Collected and imaged by Si Chen, 2008. — Type set in Utopia, designed by Robert Slimbach for Adobe Systems in 1989. Akzidenz Grotesk, originally released by the Berthold Type Foundry in 1896.
Metaphor Memory and Story
Evocative Communication in Experiential Learning
by Matthew Wizinsky
And the story goes...
0.0
Prelude
1.0
Introduction
1.1
Context
1.2
Body and Environment
1.3
Experience and Memory
1.4
Narrative Experience
1.5
Evocative Visual Communication
2.0
Embodied Cognition
2.1
Thinking Starts with the Body
2.2
From Physical to Abstract via Metaphor
2.3
Aesthetic Experience
012
3.0
Perception Memory and Learning
3.1
Perception
3.2
Form and Function of Memory
3.3
Negotiating Memory and New Experience
4.0
Interlude
5.0
Narrative
6.0
Conclusion
5.1
Narrative as Communication
6.1
Prelude to a Summary
5.2
An Embodied View of Narrative
6.2
Summary
6.3
Context
Narrative as Learning Experience
6.4
Opportunity
5.3
3456
0
0.0
Prelude
Âť I hear and I forget. I see and I remember.
I do and I understand. ÂŤ Confucius
1
When I was six years old, my first grade class took a field trip to the Detroit Institute of Arts. Walking slowly up those broad, long stairs toward the imposing neoclassical building proved a dramatic introduction to this hulking repository of mysteries. Passing through the front lobby, I encountered the smell of cheap floor polish mixed with that of cold, aged stone. A little further in, past the suits of armor frozen in time, I marched single file into the central courtyard of the building—home of Diego Rivera’s Detroit Industry Murals. I cannot recall what I was thinking when I entered that large sun washed room. But, I am sure that I had no idea what an impact the experience about to unfold would have on my perception of the world. What happened was this: in that room, for the first time in my life, I saw and I felt the meaning of an image. Rivera’s dramatic portrayals of the production line at the Rouge River Ford Plant, the seeds and harvests of science, medicine, and technology, all races of humanity embraced by Mother earth and the lifegiving sustenance of nature—all drenched in Rivera’s own political and spiritual points of view. All of this was brought to life in a dazzling visual performance whose scale and presence immersed me in a new world.
immersive visual experience
1 The courtyard of the Detroit Institute of Arts is home to the Detroit Industry murals, a fascinating and immersive visual experience.
2
3 Of course, these aren’t the words that shaped my thoughts at the time, I was six years old and barely able to read. And, indeed, words alone cannot fully express that moment as I experienced it nor as I remember it today. But, for the first time, I was able to see an image for its communicative power—a power to evoke meaning much deeper than the specific characters and actions portrayed on its surface. I could simultaneously see, feel, and think in a way that allowed me to interpret and learn from an iconic work of visual communication.
Decades later—in what I hope is still the early stage of a design career work» In its entirety, probably, it follows us at
every instant; all that we have felt, thought
ing across multiple dimensions and media—I find myself mining my own memory of this and so many similar yet unique situations to question what
and willed from our earliest infancy
is behind this power that visual communication wields in shaping our
is there, leaning over the present which is
thoughts, feelings, and sense of reality. For almost ten years, I have been
about to join it, pressing against the portals of consciousness that would fain
working with the tools of image, language, and experience in attempts to shape compelling narrative experiences across a wide range of topics and
leave it outside. « Henri Bergson » Nothing revives the past so completely as
a smell that was once associated with it. «
media. I have been doing so in a manner largely guided by intuition—that magical indescribable capacity designers use to neatly tie together disparate ideas, themes, and perspectives into relatable and understandable forms of
Vladimir Nabokov
narrative space
macro to micro
4
2–4 With floor to ceiling murals, Rivera Court immerses the viewer in a larger-than-life visual
communication. I cannot deny the existence of intuition—whatever shadowy thing that might be. But, what might be gained if I crack that door open
narrative experience. Twenty seven panels cover all four walls with Diego Rivera’s images of the assembly line at the Rouge River Ford
to see what is behind it? What can research into the functions of human cog-
plant, depicting technology and industry as the
nition tell us about how we perceive, relate, and apply images and language
indigenous culture of Detroit . The floor too
from visual communication into some understanding of the world? And,
is inlaid with related designs, and all is awash with natural light beaming in from the sky-
how might a clearer picture of the processes involved in perceiving and un-
lights above. This masterful expression of a
derstanding visual communication transform intuition from something in-
particular society’s combined cultural-social-
describable into a more refined, directable and functional tool in creating compelling and meaningful learning experiences?
political ambitions and failures reveals both a particular historic vista and the artist’s own unique point of view.
point of view
1
1.0
Introduction Evocative Communication in Experiential Learning
1.1
Context
1.2
Body and Environment
1.3
Experience and Memory
1.4
Narrative Experience Imag(in)ing the Universe Evocative Communication
1.5
Evocative Visual Communication Experiential Learning and Graphic Design
This research investigates the processes involved in the perception and understanding of visual communications—the confluence of image and written
1.1
language. My objective is to foster an awareness of these cognitive processes to better understand how meaning is made from such experiences. I do
1.2
1.3
so for the explicit purpose of improving capacities for creating compelling visual and narrative
1.4
experiences that translate communicative intentions into evocative learning potential.
1.5
1.1
Context
Image and language are the primary tools employed in conveying information, expression, and influence by means of visual design. The inextricable relationship between image and language in both cognition and memory— as demonstrated by much recent research in cognitive psychology and neuroscience—is precisely why this confluence is of interest to design and why there is a need for designers to better understand their unique functions. While language is frequently the focus of much research into meaning, understanding, and learning, I believe the role of the image is still unclear and often misunderstood. Research by and for design is in a unique position to expand and improve upon iconic, or image-based, theoretical possibilities because of its tightly coupled application of images and words toward communicative and evocative goals. As Vladimir Nabokov demonstrated through his literature and also stated explicitly: »Man thinks not always in words but also in images.« (Nabokov Lectures 289) Throughout this investi-
gation, special attention will be given to those situations that give rise to » Learning is the process of making
knowledge. « David Kolb
clarifying the distinction in cognitive function between image and language—wherever such distinctions are evident.
The goal is to trace the making of meaning in communicative experiences. We cannot define what that meaning might be in any given situation because the context and interpretation of the audience will be the prevailing and deciding factors. However, we need to start by understanding what it is we are looking for when seeking to understand how meaning is made.
Defining meaning is, of course, a slippery and complicated task. In the context of this research, we will see meaning as an understanding of objects, concepts, and relationships with a view toward their use or application in one’s interactions with the world. By positioning meaning in such a way,
» One of the polemical claims of Picture Theory is that the interaction of pictures and
texts is constitutive of representation as such: all media are mixed media, and all
I am aligning it with a philosophy of purpose. Ludwig Wittgenstein tells us in
representations are heterogeneous;
his later philosophy that »the meaning of a word is its use in language.« Be-
there are no »purely« visual or verbal arts,
cause I view design as a goal-oriented activity, purpose and use are central to my concerns.
though the impulse to purify media is one of the central utopian gestures of modernism. « W.J.T. Mitchell
5
6
The term evocative communication is right there in the title, and it too demands some parameters. In the tradition of John Dewey, I am writing from the viewpoint that significant learning only takes place through meaningful experience. In order for the learning experience to be meaningful, work is required on the part of the learner to distinguish, categorize, and conceptually assemble the discrete ideas and relationships at play. Therefore, to evoke interaction—by appealing to the memory, stimulating perception, or engaging the learner in narrative—offers greater opportunity for learning than to simply explain material in a static manner. Evocative communication creates an experience that demands audience participation through cognitive and physical processes of engaging with the material.
persistency of language
language and narrative space
7
8
5–8 The print Water, Fire, Air, Earth demonstrates the durable legibility of language forms via visual cognition. With just a few hints, we can assemble and identify full letters—maybe even the words and subsequent concepts they represent. Work-Shift is a public exhibition display employing language as a spatial expression of the roles and qualities self-identified by the residents of a south Chicago neighborhood. The information visualization Wachstum einer Avocado makes metaphor of image by linking
Through the practice of designing word-image constellations, I have come
the visual and physical similarities between an expanding balloon and the growth of an
to my own understanding—as difficult or impossible as it may be to articu-
avocado. At the National Holocaust Museum,
late—that words and images can and do work in different ways. It is what
the aggregation of hundreds of individual
they do when they reach the eyes, minds, and bodies of my audience that is of concern to me here.
images forms a visual and physical manifestation of love and loss through the evocation of so many personal stories.
image and metaphor
image and narrative space
1.2
Body and Environment
Most philosophies of meaning in Western thought stem from the notion that reason exists independent from human concerns. A philosophy built from recent empirical studies of embodied cognition would state just the opposite. Embodied cognition is an area of much recent research across various sub-fields comprising cognitive science. This work emphasizes the formative role of environment and direct physical interaction in the development of cognitive processes. Beyond the lab, many thinkers and writers are making use of this research to posit that the nature of the human mind is largely determined by the form and unique motor and perceptual systems of the human body. Further research in linguistics, cognitive psychology, and artificial intelligence is expanding the thesis to include new cognitive tools and processes such as conceptual metaphors and image schemas.
Following this position that all abstract thinking is linked to experience, pro‌our most refined thoughts and best
actions, our greatest joys and deepest sorrows, use the body as a yardstick. ÂŤ Antonio Damasio
ponents argue that an embodied philosophy can demonstrate that reason and logic are built from metaphorical constructions and are not attributed to a purely objective reality. While reason and logic are certainly structured,
as opposed to purely subjective, they originate in our human bodily interactions with our environment. For example, our understanding of such a basic concept as time is based fundamentally on metaphor. When we talk about events as though they have a place in time, we are transferring a spatial understanding of movement in space to something we cannot observe or grasp independently. In fact, if we attempt to conceptualize time without metaphor, we cannot get too far at all (getting too far being, of course, a spatial metaphor of its own!).
What this means for the designer is that meaningful image and language communications can utilize metaphor to trace physical activation in the environment, which poses the potential for making visual experiences also inherently visceral.
9
Experience and 1.3 Memory
What is it that we gain from an experience? What is it that sticks in our mind in a way that saves that moment, sets it aside, and allows us to return to it later? Is it a sense of revelation, of understanding, of simply experiencing the new? Is it to see something known in a new light, to make a connection between two seemingly very different things? Is it a full body-mind sensation? Âť To learn from experience, we must remem-
ber it, and, for a variety of reasons, memory is a faithless friend. ÂŤ Daniel T. Gilbert
Or can it be a heightened sense of stimulation of just one of the senses? Throughout our lives, we experience millions of sensations. Over time, these activations stimulated by our environment begin to form recognizable
visualizing the imperceptible
10 patterns—or experiential gestalts. Our memories of these patterns take hold in our minds shaping an awareness and understanding of the world about us. We remember these patterns and patterns of patterns to create mental categories spanning image, language, and physical experience. When I see a chair—whether it is a rocking chair or a three-legged stool— I grasp it not only as a unique object whose physical function I understand visually, but I can also place it within hierarchical categories: rocking chair, chair, things to sit on, furniture, and so on. It has a size and shape of certain relationship to my body; I can sit on it, maybe it rocks or glides, or maybe it is steady. All of these functional and physical traits I take in visually through their relationships to remembered experiences.
9–10 Aaron Koblin’s Flight Patterns from the larger project Celestial Mechanics demonstrates visualizing systems imperceptible from a fixed, human perspective. Tracing air traffic over North America creates a visual memory of each
As we move through the world and confront new experiences via new sensa-
plane’s route forming an image of the earthly terrain. Blue Velvet , the surrealistic noir-drama
tions, we carry with us always the full, yet continually shifting presence of
by David Lynch, employs a dreamlike quality
our past experiences and the knowledge and expectations established by
of shifting focus, overlapping story lines and
their memory. Not only does this shape each of our individual understandings of the world, but these experiences also coalesce toward a unique sense of identity. A memory is an expression of experience; how I form this expression and what I do with it go a long way toward defining who I am.
unresolved narratives. Lynch himself called it a film about things that are hidden—within » a small city and within people.« In memory, as in dreams, we often examine and evaluate the nature of our own reflections as much as the stories themselves.
as in dreams
American pragmatist philosopher and educational thinker, John Dewey, was the first to pose an experiential theory of education. His theory of experience is built on the two central tenets of continuity and interaction. Continuity refers to the notion that we store and carry our individual experiences into the future (whether we like it or not). Interaction refers to the past experiences interacting with the present situation, to create one’s present experience. It is clear that both continuity and interaction depend heavily on the functions of memory—both to build upon existing knowledge through ex» The creative act is not performed by the
perience and to expand our capacity for learning.
artist alone; the spectator brings the work in contact with the external world by deciphering and interpreting its inner qualifications and thus adds his contribution to the creative act. « Marcel Duchamp
In 2007, a survey was taken of fifty visitors to the Montreal Expo ’67—forty years after the event occurred. From the research report: »Factors affecting memory vividness can be seen at work: affect in the form of recalled
excitement, surprise, frustration and rehearsal of the experience in the form of subsequent discussions with friends and relatives, or reminiscing the experience from personal photographs. ...they are overtly saturated with personal details, feelings and judgments making the event and self remembered simultaneouslyÂŤ (Anderson and Gosselin 6). This research indicates that re-
gardless of the content of the experience, the memory is absorbed into selfidentification—it is made to be personal, one’s own, and the affect of the experience stays with it in memory.
11
1.4
Narrative Experience
Imag(in)ing the Universe Art and science provide two different but often similar and even overlapping approaches to showing us those operations, relationships, essences, and structures that we cannot perceive or simply do not acknowledge or comprehend without iconic cognitive aides. Both art and science attempt to reduce the complex into relatable, understandable forms. What these new » Science is no more than an investigation
forms represent are metaphorical links connecting that which we currently
of a miracle we can never explain, and
understand with that which was previously imperceptible. This metaphori-
art is an interpretation of that miracle. «
cal linking typically involves images or image and language combinations
Ray Bradbury
taking the form of narrative to connect the known and the unknown.
» Better than ninety-nine percent of
modern technology occurs in the realm of physical phenomena that are sub or ultra to the range of human visibility. « R. Buckminster Fuller
By taming or harnessing complexities and unknowns into iconic narrative forms, we are able to think beyond the means of our immediate perception. We are able to see, and thus consider and interpret, the macro and micro
cognitive harmony
12
functions and relationships of our universe. We weigh the viability of new narratives against our existing knowledge—informed by memory—to determine its validity.
For example, when nasa released the first photograph of Earth taken from
11–12
space, this image of a tiny blue marble adrift in the silent abyss provided an
This image pair contrasts the potential of
awesome moment for mankind to reflect on its own unfathomable exis-
image and language to work in meaningful harmony or cognitive dissonance. Copernican
tence and fragility—to consider both how tiny and how precious Earthly life
Planisphere is a hand-colored engraving
really is.
beautifully depicting Copernicus’s radical heliocentric theory. As with all maps, both images and names are required to create
Evocative Communication
a meaningful expression of the imperceptible.
Visual communication is the process of weaving symbolic and verbal mes-
Art and science are working together to
sages into a cohesive form for a communicative purpose. Because sight is
image and imagine the universe. Alternatively, in René Magritte’s painting The Treachery
the primary sensory input for humans, visual messages—including both
of Images an apparent contradiction creates
iconic and written language—have the greatest power to inform, educate,
cognitive dissonance precisely because the
and influence. Therefore, graphic design has much to do with how effective, engaging, and meaningful experiential learning encounters can be.
cognitive dissonance
word and image are so tightly coupled in our minds. Through this dissonance, the artist asks us to question the validity of an image.
13
14
shifting social perspective
shifting scale perspective
15
13–14 Photographic and cinematic images offer new perspectives on the self and others. In Jacques Tati’s Playtime , we see the modern city and its social structures in a new light through the viewpoint of an outsider. Images taken by NASA astronauts in lunar orbit offered the first
glimpse of the Earth as a whole, and thus a huge shift in perspective through scale.
15 –16 Accumulating views in a single frame, via techniques such as montage, can provide a multiplicity of temporal or visual presences. Peter Tscherkaskky composites several narrative tracks and perspectives in the experimental film Outer Space . In his video series Invisible Mending , artist William Kentridge
confronts various media products of himself with one another.
layers of time
16
layers of self
17 In works of pure abstraction, the artist may intend to present a sensory experience for its own sake. However, the power of these works lie in thew viewer’s insistence at applying his or her own living context toward their sensory qualities; projecting narrative onto the purely aesthetic experience. The films of Jordan Belson are a great example. Experiments in motion and visual techniques create self-contained worlds of struc-
» People call our films › abstract,‹ but they’re not. They’re › concrete‹ films. › Abstract ‹ means to make an abstraction from something concrete, but our films are concrete. « Michael Whitney (Youngblood 229)
ture and form. Nonetheless, as viewers, we bring knowledge of our own world and begin to see similarities between Belson’s world and the cosmic or sub-atomic structures of our own.
18
Evocative Visual 1.5 Communication
Experiential Learning and Graphic Design Learning a written language, understanding symbolic representations and operations in mathematics, exploring scientific theories through visual representations of data, or formulating spatial relationships via modeling; these are all cognitive tasks that operate at the intersection of visual communication and embodied cognition. The student who draws a diagram to solve a mathematical problem is creating a design exercise to augment cognitive abilities in order to understand physical or metaphorically representational relationships.
Visual communication serves both communicative and evocative functions. The possible ranks higher than the actual. ÂŤ
However, a communicative experience or object cannot be divided neatly
Martin Heidegger
into content and form. The two are always conjoined in the communicative
Âť There are no facts, only interpretations. ÂŤ Friedrich Nietzsche
material, which is an expression of the purpose. The purpose is manifest in the material.
hearing visually
19
20
18–20 Visual artists and designers employ a variety of methods to translate auditory, tactile, or other non-visible sensations into compelling visual statements. Visually activating other sense perceptions engages the multi-modal methods by which we understand the world. In a magazine spread by Bradbury Thompson, the single image of a musician is multiplied in colorful layers, suggesting rhythm, melody, and the sounds and actions of playing rock n’ roll music. French artist duo HeHe bring attention to aspects of the shared environment with their Nuage Vert (green cloud) project, which visually traces the
An evocative narrative experience is an invitation for perception, interpretation—and, ultimately, what is of lasting value, it is an invitation to make meaning through creative participation with an interpretation of the communicative material. If an experience is truly evocative, your interpretation will shape your own reality, and that is sure to be unforgettable.
shifting size and shape of the smoke cloud emitting from a Helsinki electric plant. When we see the cloud, we too touch it with our eyes. Designer Cybu Richli manipulates the materials of his posters in a variety of techniques that elicit a tangible physical distortion resulting in a visceral visual experience.
touching visually
seeing physically
21
21 Alex Adai and Peter North create Internet Maps by spatially drawing all data accessed
on the web in a given day. By visualizing these connections across the world, not only can patterns of activity be investigated, but the nebulous imperceptability of the internet is given a tangible form. This form is simultaneously a map and an image for conceptualizing the nature of our global data connectivity. And, it’s quite beautiful.
seeing the connections
seeing the connections
2
2.0
Embodied Cognition
2.1
Thinking Starts with the Body Being Means Being Somewhere Embodied Cognition
2.2
Interaction, Metaphor, Abstraction, and Aesthetics
From Physical to Abstract via Metaphor Conceptual Metaphors Metaphor At Work Image Schemas and Categories What Does This Mean for Visual Learning Experiences?
2.3
Aesthetic Experience An Expanded Definition of Aesthetics What Does This Mean for Design?
Embodied cognition is a field of research spanning
2.1
neuroscience, cognitive psychology, linguistics, and philosophy contending that all aspects of cognition, including abstraction, logic, and reason, are shaped by aspects of the human body. All that we know of the world is the result of interactions with our environment. Environmental input is perceived through our sensory and motor systems, processed by the brain—both consciously and subconsciously— then translated via metaphor into increasingly abstract yet aesthetically grounded conceptions of the world and our relationship to it. Before we are thinking beings, we are first acting beings.
2.2
2.3
Thinking Starts with 2.1 the Body
When faced with complex situations, what influences our decisions? Is it purely a matter of logic and reason? Or is it based on our physical condition within the environment? Of course, it depends mostly on the context of the situation: Am I under immediate physical threat? Am I safe with time to consider my options, to imagine potential outcomes? Can I relate this situation to something I have faced before? Regardless of the variability of such a situation, the answer is: both. The mind that makes the deliberate reasoned choices when circumstance allows is the same mind that also reacts swiftly almost instinctually when it does not. After all, it is also this same mind
Âť To say that cognition is embodied means
which regulates breathing, heart rate, and other physical systems at all times,
that it arises from bodily interactions
whether calm or frantic. What becomes clear along this line of questioning is
with the world. From this point of view, cog-
that thinking and acting are not entirely separate activities. Rather, they are
nition depends on the kinds of experiences that come from having a body with particular perceptual and motor capac-
integral parts of the same spectrum of human experience that starts with the environment in which one exists.
ities that are inseparably linked and that together form the matrix within which memory, emotion, language, and all other aspects of life are meshed.ÂŤ Esther Thelen
Embodied cognition emphasizes the role of the environment, and an organism’s physical relation to it, in the formation and development of cognitive
processes. As a relatively new and quickly expanding multidisciplinary field of research, there are naturally a variety of different opinions and agendas. However, those involved tend to agree on the general theory that »cognitive processes develop when a tightly coupled system emerges from real-time, goal-directed interactions between organisms and their environment; the nature of these interactions influences the formation and further specifies the nature of the developing cognitive capacities.« (Fieser and Dowden). What
does this basic biological principle mean for the bigger picture of human consciousness and understanding?
Being Means Being Somewhere Early philosophical roots of embodied cognition can be found in the work of Martin Heidegger. One of the central themes in Heidegger’s work is to seek an explanation of what being itself means, and how it is possible for us to question it in the first place. In Being and Time, still much debated today, he determines that the context allowing our understanding of being is time. Our own sense of temporality—past, present, and future— gives a perspective space to consider being as a concept. Supporting his thesis, he investigates what it means to be human— a being, acting agent—by noting that
» The kind of dealing which is closest to
us is as we have shown, not a bare perceptual cognition, but rather that kind of concern which manipulates things and puts them to use. « Martin Heidegger
our being is conditional on inhabiting a world. He calls this Dasein—a German word that translates to being-there, with the there being our physical environment. What Heidegger’s statement does to the world of philosophy is undercut the influential view of Dualism, as defined by René Descartes, which separates mind and body into two distinct entities. Heidegger states that instead of being stemming from the position I think, the starting point is one acts.
The further implication is that thought, reason, and logic do not exist on some neutral neural plane independent from human actions performed by human bodies. Rather, all thinking comes from our interactions with the world. This means that the qualities of our worldly experiences—including emotion and feeling—are an inextricable part of our thinking, understand» Meaningful form comes from the nature of
our bodies and the patterns of interaction we have with our environment, and
ing, and reasoning. This can be understood in its simplest terms by the fact that we tend to repeat those actions and decisions that have previously lead
it is therefore shaped by our values,
to enjoyable results, which cannot be logically classified or quantified. This
interests, and purposes as active agents.
applies not just to the scale of survival but also to those acts of individual
…thought is never wholly divorced from feeling, value, and the aesthetics of
enjoyment—from the foods we prefer to the kind of company we keep.
our embodied experience. « Mark Johnson
Embodied Cognition The last few decades have seen a surge of interest and activity in the field of embodied cognition, resulting from new research and developments in neuroscience and cognitive linguistics. Collectively, this research outlines a new comprehensive approach toward explaining human intelligence, taking the bodily grounding of the mind as its central focus. This project is involved in detailing the myriad ways cognition depends upon the physical characteristics, inherited abilities, practical activities, and environments of thinking beings—that is, humans.
The significance of this approach to understanding the mind is that it requires all learning be made directly from physical interactions with our
world. Starting with primary interactions, based on direct physical experience, and ascending to increasingly abstract higher order conceptions, proponents suggest that we can trace all cognitive capacities to bodily interactions within our environment. By combining cognitive tools such as conceptual metaphors, image schemas, and linguistic patterns, we construct the basis for cognitive capacities that are well beyond our direct physical experience, and, in fact, may not even possibly exist in such a world. For example, we may not be able to relate extremely complicated mathematical processes to our daily lives, but the basis of the mathematical system—
» Our own bodily position, attitude, cond-
numbers—comes from our ability to distinguish, sort, and count real physi-
ition, is one of the things which some
cal objects.
awareness, however inattentive, invariably accompanies the knowledge of whatever else we know. We think; and as we think
This is our basis for understanding how concepts and objects relate to us
we feel our bodily selves as the seat of the
and to other concepts and objects in our world. This transition in our think-
thinking. If the thinking be our thinking,
ing from the tangible to the intangible—from sensory and motor response to abstraction, logic, and reason—starts with metaphor.
it must be suffused through all its parts with that peculiar warmth and intimacy that make it come as ours. « William James
22
From Physical to Abstract 2.2 via Metaphor Conceptual Metaphors Conceptual metaphor refers to understanding one concept in terms of another. A concept, or conceptual domain, can be any coherent organization of human experience, or an experiential gestalt. Typically, conceptual metaphors are used to understand or relate a non-physical or difficult to delineate concept (such as time or emotions) in terms of one that is more directly related to our physical engagement with the world. » Without metaphor, there would be no
philosophy, nor any other mode of reflective understanding of our world. « Mark Johnson »…we do not have two kinds of logic, one
for spatial-bodily concepts and a wholly
For example, the conceptual metaphor life is a journey gives the difficult to delineate domain of life the characteristics of an object with a front-andback orientation (like our bodies) moving along a defined path, which also has directional characteristics (Lakoff and Johnson Philosophy in the Flesh
different one for abstract concepts. There
32–35). This translation results in phrases such as things are moving for-
is not disembodied logic at all. Instead,
ward or you have come a long way. Conceptual metaphors give us a handle
we recruit body-based, image-schematic logic to perform abstract reasoning. « Mark Johnson
on those conceptions not directly grounded in our experience. This is our basis for abstract reasoning.
metaphor in drawing
23 Metaphor At Work Mark Johnson is the Knight Professor of Liberal Arts and Sciences in the Department of Philosophy at the University of Oregon. He and his colleague Dr. George Lakoff, professor of linguistics at the University of California, Berkeley, have written extensively on what it means to consider the embodied mind as a source for language, logic, and creating or interpreting symbols that provide the basis for abstract reasoning. In their book Metaphors We Live By, Lakoff and Johnson make the first attempt to trace the embodied origins of many primary conceptual metaphors. They rightly acknowledge that this is only scratching the surface of all the rich, layered metaphors we use in
22–23
speaking and thinking everyday. However, they are convinced that the same
Metaphors often draw from natural physical
systematic domain-mapping approach could be used to reveal the physical
phenomena, which can be translated to abstract ideas. In a series titled Visual Investi-
body-environment root for any conceptual metaphor, and thus any form of
gations by designer Cybu Richli, everyday
abstraction.
objects become metaphors for scientific phenomena. For example, the flight pattern of a sea gull is charted and compared to the
Conceptual metaphors start with direct interactional metaphors, such as
unfolding of an umbrella. Architect Santiago
orientational relations: based on the ways our bodies move, there is a signifi-
Calatrava also found a bird metaphor useful
cance to up-down and front-back orientations. These direct interactional metaphors can then lead to increasingly complex systems of relational
metaphor in building
in his design of the Milwaukee Art Museum whose animated wing-like structures provide both shade and personality.
24
25
embodiment of image
image and physical interaction
26
24–26 How can images convey embodied experience? Fragments of RGB by Onformative Studio
turns video footage into interactive engagement in embodiment. Playing on our experiences with LED images, the projected video of a woman's
face distorts in relation to physical movements and gestures by the viewers. Almost one hundred years prior, avant-garde Russian designers experimented with photo-montage to express the dramatic and sometimes distorting physical experience of living in a new and fast-moving technological age.
metaphor and movement
understanding that at first glance seem far removed from the day-to-day world of physical activities. However, by dissecting conceptual metaphors into their parts and functions, even the most high order abstract relations can be seen as stemming from a lived or perceived physical experience. What most commonly holds these distant concepts together is the use of language or linguistic patterns.
To demonstrate how a higher-order concept can be traced back to its bodily and physical manifestation, let us consider the metaphor purposes are
destinations . This conceptual metaphor proposes that a goal is some place ahead of us, and we must employ strategies for attaining this goal as analogous to the destination in a journey. In order to reach our destinationgoal, we must plan a route that accounts for potential obstacles, recognize » Our capacity to abstract farther and far-
ther away from the concrete richness of felt experience is still always and only
and spatially orient landmarks to track our progress, and consider how we chart forward or backward progress along our path. By doing so, our consid-
abstraction and selection from the flow
eration of pur pose —our goal, along with the time, states, and changes
of perception. The more we abstract, the
necessary to reach it—is rooted in how we think about moving through
more we are left only with perceived relations among qualities or shapes or internal structures of things. « Mark Johnson
space. In the process of transferring the physical sense of a journey to the concept of moving toward a goal, we subsequently bring with us the related
concepts of space. Notions of up, down, forward, backward, on and in all bring along a bodily orientation to our physical motion. Consider terms such as facing the future, making forward progress, being on top of things— these all inherit a reasoning about space, movement, and continuity that comes directly from what we know about moving our body within an environment (Lakoff and Johnson Metaphors We Live By 15–17). For the designer, it is interesting to consider if and how the richness of meaning metaphors like this one bring to language might also carry over to images.
Image Schemas and Categories An image schema is a regularly recurring embodied pattern of experience. Through the use of image schemas, many everyday situations do not require our focused and deliberate cognitive attention. Rather, we use automatic processing that places the task at hand within a known pattern. Such schemas are image-like in that they represent an analogic pattern of neural activations which shape the contours of the experience into a recognizable and cohesive whole. This allows us to undertake physically and cognitively complex tasks without needing to be directly aware of doing so. For example, walking up a flight of stairs requires a complex network of perceptual processing and motor movements. However, through some hard-learned lessons as children and daily practice, most of us have developed a stairway schema that activates the necessary processing and motion. When applied, this schema allows for climbing a flight of stairs we have never seen before so effortlessly that we can do it without even looking (and maybe even while talking on the cell phone). The evolutionary advantage of such a capacity should be obvious.
In conjunction with image schemas, we build categories to extend our understanding of specific objects to those that are similar. This notion of categorization in cognition is clearly manifested in our language and iconic symbols or images. There is no object so general as a leaf in nature. There
» What I find particularly interesting are the
ways in which patterns of our sensorymotor experience play a crucial role in what we can think, how we think, and the nature of our sym-bolic expression and communication. « Mark Johnson
are only specific instances of a leaf, and these are as diverse as a maple leaf, a palm leaf, and a poplar leaf. But, the cognitive category of leaf is manifested in the word leaf and abstracted images or icons of a leaf. All of these categorizations and their representations apply to a wide range of real objects, allowing us to consider those objects’ similarities in function and physical qualities.
Taking the cognitive potential of image schemas and categories one step further, we can translate these cognitive structures for generalizing real physical objects and experiences into those of abstract relationships.
Based on physical interactions, linguistic patterns, and social-cultural con» All perceiving is also thinking, all reasoning
is also intuition, all observation is also invention. « Rudolf Arnheim » Any line drawn on a sheet of paper, the
simplest form modeled from a piece of clay, is like a rock thrown into a pond. It upsets repose, it mobilizes space. Seeing is the perception of action.« Rudolf Arnheim
text, we can use the frameworks of image schemas and categories to establish patterns of understanding and reasoning. We are able to translate a physical relationship—body to environment—to relationships amongst ideas, concepts, or abstractions. Image schemas are dynamic embodied patterns taking place in and through time much as our understanding of abstractions take place in and through metaphors. Our lived experiences of patterns and relationships are how we hold together abstract or logical
thoughts without having physical objects to connect them directly. Because they rely on both visual and linguistic capacities, image schemas are considered multi-modal patterns of experience, in the same way that all visual communication can be considered multi-modal for its capacity to activate both sensory and motor cognition via optical information alone.
To illustrate how an image schema functions, let’s take a look at the containment schema and its translation to various spatial senses of the word out as outlined by Mark Johnson in his book The Body in the Mind (32–33). Out is used in situations where a trajector, a discrete object in motion along a path, leaves a spatially bounded landmark (fig. 28a): John walked out of the room. Mary got out of the car.
While these prototypical cases clearly define the landmark as a spatial container, out may also be used to indicate those cases where the trajector is a
27
28 mass that spreads out. This conceptually expands the area of the containing landmark to something less physically defined (fig. 28b): She poured out the beans. Roll out the red carpet.
We can also use out to describe motion with an implied landmark not at all physically defined, yet the trajector may have a destination (fig. 28c): The train started out for Chicago. Let’s go out tonight.
Furthermore, experientially-based spatial image schemas—such as the containment schema and its derivatives in the out schemas —can lend their logic to non-spatial situations in which neither landmark nor trajector is physically defined. Here, the physical relationship is translated entirely metaphorically (fig. 28d): I don’t want to leave any data out of my argument. (argument as container) Tell the story again, and don’t leave out any details. (story as container) She finally came out of her depression. (emotional state as container)
Johnson states that further and further abstract reasoning can be shaped by these underlying spatial patterns, experienced initially in our bodily
LANDMARK
TR A JECTOR
A
TR A JECTOR
LANDMARK
TR A JECTOR
B
LANDMARK
C
containment schema
LANDMARK
out schemas
TR A JECTOR
D
29 knowledge of the world. This logical conception of containment is not just a matter of being in or out of the container. For example, we say that a person is in deep depression if his or her condition is severe, and it will require a long period of time to recover. The further, or deeper, the trajector is contained within the container—physical or abstract—the longer it will take to get out.
What Does This Mean for Visual Learning Experiences? As we have seen, language typically plays an important role in uniting disparate concepts through conceptual metaphors. However, this is not just a matter of language alone. The use of metaphors in conjunction with the image schemas they activate create a conceptual system that is very real for us and affects how we perceive and act in the world. Therefore, a metaphor
27–29 The Containment image schema demonstrates a progression from the physical to abstract via metaphor and language. Considering the word out, the progression goes like this: An entity
serves as an expression for the conjoined functions of image and language—
X is contained within some bounded landmark.
each providing its own methods for understanding relationships, concrete
A trajector leaves a spatially bounded landmark.
or abstract, to shape our reality and how we respond to it.
ingly abstract and, eventually, cease to represent
Both trajector and landmark become increasphysically bounded relationships. However, the
What all of this relational understanding amounts to is a method for learning through the ongoing translation from physical experience to abstract
out image
physical action carries metaphoric significance. This same concept can be understood from the Way Out (or Exit) sign.
30
physical dissonance
and conceptual thinking. What much of the research in embodied cognition suggests is that human intelligence lies less in our brains themselves than from the dynamic interaction of our brains—embodied as they are—with our environment. Through the combined engagement of our sensory and motor systems, we dynamically interpret experiences by linking our physical activities and their conceptual metaphors.
If the designer of communicative experiences can tap into the embodied roots of metaphor, it will be possible to transcend the abstract symbolic interfaces of visual communication—be they linguistic, iconic, or both—and activate the thinking and feeling dimensions of an experience simultaneously. This can be done either to reinforce and correlate the abstract to the
30
physical or to challenge these relationships, thus making their connections
The playful Book Shelf installation by artist
visible and palpable for the viewer. This opens a rich, diverse world of poten-
Daniel Eatock captures the imagination because
tial for physical and sensory learning experiences geared toward expanded cognitive possibilities.
of a perceived inversion of physical properties. The initial physical dissonance is resolved when we can both see and feel what’s going on.
31
32
abstract space in three dimensions
dimensional metaphor
33
31–33 This image series demonstrates the potential for translating conceptualizations of space from a physical-bodily experience to that of pure metaphor in visual design. The structure in Traast+Gruson’s Playhouse traces the boundaries of architecture to suggest an enclosed environment within physical space. In Droog Design’s exhibition of domestic products, objects
are placed on a gallery floor marked with the rooms, passageways, doors, and other fixtures of a house that is not physically present. Shifting from three dimensions to two, Karel Martens’s cover for OASE 71, a journal on collective urban spaces, turns typography into a purely conceptual metaphor for physical space.
abstract space in two dimensions
34
2.3
Aesthetic Experience
An Expanded Definition of Aesthetics » One of the greatest impediments to an appreciation of the full scope of embodied meaning is the way philosophers of language focus almost exclusively on language (i.e., spoken and written words and sentences) as the bearer of meaning. Anything that doesn’t conform to this linguistic model is defined, by fiat, as not part of meaning proper. This language-centered prejudice leads many philosophers to overlook the deepest roots of meaning. The best way to avoid this blindness is to look beyond linguistic meaning and into the processes of meaning in the arts, where immanent bodily meaning is paramount« (Johnson Meaning of the Body 209).
Thus sets the stage for Mark Johnson’s plea for a revised, expanded view of the meaning and import of aesthetics in The Meaning of the Body. Johnson pays continual homage to the pioneering efforts of John Dewey to see art— specifically, our experience of art—as »a condition of life and meaning« (212)
abstraction and interaction
35 In this research I continue the tradition set out by Dewey and persisted by Johnson, establishing that aesthetics is much more than the confluence of formal qualities in a work of art. Rather, aesthetics define the contours that give all our daily experiences their distinctive character and significance. The precise combination of physical, perceptual, and cultural forces that converge in any given experience, or more broadly, in patterns of experi-
34–35 In the media installation Floating Numbers
ence—these are the qualities that affect us directly both consciously and
at the Jewish Museum, the myriad cultural
subconsciously. A subtle change in these qualities and the balance has
meanings of numerology are explored. Num-
shifted, and surely the experience itself will take on a different flavor, no
bers are projected and animated across a tabletop, and as visitors touch the numbers,
matter how nuanced. Aesthetics cannot be seen as a surface condition, but
information revealing their significance is
as the total quality imbuing the entirety of the experience. There cannot be
presented. This direct physical contact with
a distinction drawn between content and form; there is only the fullness of the experience—the experiential gestalt. Therefore, the meaning we make from an experience is inherently tied to those aesthetic dimensions of the experience itself.
abstract conceptualizations of meaning—pursuing, then capturing the meaning—creates a powerful physical link between participants and concepts. In his book Type in Motion II , designer Benjamin Dennel transforms letter forms into space. The images of typographic forms imply motion through blurred and
What Does This Mean for Design? Specifically in the realm of images, this means that it is exactly the aesthetic qualities that give meaning to the experience of viewing and perceiving an
abstraction and motion
layered instances, similar to our visual experience of quick movement. Motion implies space, so the image triggers a feeling of movement and space via visual metaphor.
36
physical eclipse
37
36–37 Although a rare event, the blotting out of the sun in a solar eclipse is a sensational phenomenon. The seemingly limitless power and magnitude of the sun is suddenly eradicated. This sensation is translated metaphorically to express emotional life in the aptly titled film L’Eclisse by director Michelangelo Antonioni.
emotional eclipse
38
image. Any shift in the formal qualities of the image—even if it be ever so slight—will result in a corresponding shift in the kind of meaning we take from it. »…a difference in form indicates a subtle difference in meaning « (Lakoff and Johnson Metaphors We Live By 131). We know how true this is from our own experiences with reading facial gestures. We are so attuned to the slightest nuances of facial gesture—probably because these gestures are so purely embodied—that we can translate the slightest arc of an eyebrow or curve of the lip into a world of meaning.
If all learning is based on our bodily experiences of the world, and those experiences are defined by aesthetics, then we can deduce that aesthetics largely define the nature of all learning experiences. The aesthetic nature of our interaction with the objects, environments, and materiality that
subject as material
39
constitute an experience will necessarily link the abstract ideas or concepts we take away with the concrete reality we have encountered. 38–39
When applied to the field of visual communication, the formal qualities of the communicative material—typically, the confluence of image and language—has everything to do with what and how that material is perceived, understood, remembered, and applied to existing knowledge. In short, the aesthetic nature goes a long way in determining the meaningful outcome.
language and material
At the American Museum of Natural History’s exhibition Water: H2O = Life , visitors enter the exhibition space through a wall of mist upon which a multi-language typographic message is projected. In this scenario, water becomes simultaneously subject, image, language, and material in the learning experience.
3
3.0
Perception, Memory, and Learning
3.1
Perception Visual Perception Visual Cognition
3.2
Form and Function of Memory Picturing Memory: a Historical Perspective Image & Language Image & Language Processing During Recall Memory at Work and Working Capacity Color Memory Expressive Memory
3.3
Negotiating Memory and New Experience Stability and Change I Am What I Remember: Identity and Memory
The capacity to perceive, store, and later retrieve
3.1
information in the form of images, language, or other abstract concepts is one of the most studied, debated, and written about aspects of the human condition. Memory aids our understanding
3.2
of self and the world by allowing us to continually evaluate new perceptions in light of previous experiences. In fact, consciousness itself seems to be comprised of two parts: 1– the present as defined by our senses of the immediate moment, and 2– memory, which supplies everything else we know.
3.3
40
3.1
41
Perception It is a well-known phenomenon that if a crowd of people all witness the same event, each individual’s report of what happened will vary greatly. This is manifested in the lack of dependability of eye witnesses in criminal cases— a trope explored by the fascinating portrayal of such an event in the film Rashomon by Akita Kurosawa. In the film, as in the original short story of the same name by Akutogawa, several people witness a murder, and each of their stories is re-enacted in conflicting episodes. Ultimately, the viewer is left to untangle the variety of perspectives to determine what judgment, if any, is to be made of the situation. » To perceive means to immobilize … we
seize, in the act of perception, something which outruns perception itself.«
What this story reveals through its open-ended telling is that we each perceive the world uniquely, and these perceptions are informed by our unique
Henri Bergson
accumulations of lived experience. What we know of our relationship with » Once my hand has drawn something my eye
has observed, I know it by heart, and I can draw it again without a model. «
the external world—including other people—is predicated on our own unique lived experiences, the memories we make from them, and the expectations these experiences produce.
David Hockney
points of vision
the whole and its parts
42
43
Visual Perception It would seem that memory and visual perception have a paradoxical relationship. Our perception of the world requires memory for understanding our relationship to other objects and the attributes we attach to them via images and language. However, we can only remember these relationships through the perceived attributes we attach to them. Much has been studied
40–43
and asserted about this complicated interwoven relationship. Regardless, it
Public art installations by artist Christian
is clear that whatever we perceive (visually or otherwise) in a situation is
Moeller often make use of visual perception
only useful as a learning tool when we can store and retrieve those perceptions and subsequent expectations for use at a later time.
and cognition phenomena to create arresting images in public space. At this bus stop in Seattle, hundreds of small plastic pucks attached to a chain link fence create an image whose scale and visual depth belie the simp-
Visual Cognition
licity of materials used. Our capacity for visual
Visual cognition is the method by which we see and make sense of visual
cognition connects these dots to create a fully
input. Humans simultaneously see complete images and individual compo-
shaded dimensional image of a human face,
nents of the full image. This parts-and-the-whole method of processing visual information is central to much of Gestalt psychology, which has been very informative to the field of visual communication. Our eyes do not fix on a single point to see an entire image. Rather, the eye reads information via fixations and saccades: fixations are the brief stops when looking directly at
similar to the way we read half-toned or piexelated images, in which our eyes take in the image as a whole-and-its-parts simultaneously. Individual fixations of the eye identify points to both distinguish and correlate in relation to the rest of the image as demonstrated by images for testing color blindness.
SACCADE
F I X AT I O N
F I X AT I O N
AREA OF FOVE AL VISION
reading an image
distinction and relation
44
45
46
an area within an image and the fovea (the part of the eye that gives us the most sharp and colorful image of the world) is fixed upon it; saccades are
44–47
quick movements of the eye from one area of an image to the next. During
These posters for musical concerts by Swiss designer Josef Müller Brockman demonstrate
saccades we experience temporary blindness as the eye moves too quickly
a number of visual perception and cognition
to receive any fixed image from what it is seeing. Despite this very rapid jerky
phenomena. In Gestalt psychology, isomorphism refers to a correspondence between an
method of seeing thousands of small points individually, we experience
array of visual stimulus and the mental model
viewing an image as one steady flowing input of visual information—we
created in the brain by that stimulus. For
perceive it all as one. Not surprisingly, this same process of fixations and
example, a pair of alternating and spatially
saccades also holds true for reading written language. One primary distinc-
separated patches of light will create the illusion of motion between two locations.
tion lies in directionality and linearity. For example, in Western languages,
Varying the scale of a repeated figure creates
the words must be read in a linear sequence from left to right, top to bottom.
the illusion of depth and movement within three-dimensional space. Optically mixing two
How the eyes follow and read an image may be informed partially by cultural
distinct colors at the intersection of their
influence but has mostly to do with the qualities of the image itself: the com-
shapes creates the illusion of transparency.
position, size, format, and so on. Of course, to build upon the image or text
Brockman’s compositions create these illusions of movement, depth, and layering through a simple but effective geometric visual language.
from one glance to the next, we need to remember where we have been to make sense of where we are going.
transparency and depth
depth and motion
scale depth and motion
47
abstraction and metaphor
48
Form and Function 3.2 of Memory
Picturing Memory: a Historical Perspective As we have seen previously, metaphor is a powerful tool for understanding » Memory is the mother of all wisdom. « Aeschylus »…the most complete pictures are formed in
those concepts which are difficult to delineate physically. So, it is no surprise that throughout history, philosophers, scientists, and artists have made numerous and sometimes fanciful metaphors to understand the mysterious
our minds of the things that have been
phenomenon of memory. Plato’s depicted memory as a tablet of soft wax
conveyed to them and imprinted on them by
that is impressed upon by the senses and even language. In De Oratore, Ci-
the senses, but … the keenest of all our senses is the sense of sight, and that
cero describes a Memory Palace, an architectural model in which »the order
consequently perceptions received by the
of the places will preserve the order of the things« (qtd in McConkey 12). The
ears or by reflexion can be most easily
16th century German monk Gregor Reisch introduced the idea of memory as
retained if they are also conveyed to our minds by the meditation of the eyes. « Cicero
a storage space in the brain tucked behind the ears, as if a kind of cellar— and, this concept still lingers in the expression to keep something in the
architecture of memory
49 back of one’s mind. In the 17th century, British scientist and alchemist Robert Hook’s related memory to phosphorescent minerals to describe the brain’s capacity to retain and emit light. This light-emitting metaphor subsequently influenced photographic then later video metaphors for conceiving the mind’s ability to see and play still and moving images.
While all of these metaphors may offer assistance in visualizing how we consider the storage and retrieval processes of memory and the shapes of these actions, none of them can fully encompass what it means to live each day
48–49 Giulio Camillo describes memory as a Vitruvian amphitheater in his treatise L’Idea
with an identity and knowledge of the world as built by the memory of our
del Theatro. The speaker on stage gazes into
previous experiences.
the auditorium whose tiered, semicircular construction is designed for housing memories in a clearly laid-out fashion. Seven sections,
Recently, the computer has become a popular metaphor for memory and
each with seven arches, span seven tiers.
mental processing, including its terms hardware, software, and, yes, mem-
On each of these stand emblematic images and
ory. Regarding the computer metaphor, Nobel prize-winning American neuroscientist Gerald Edelman has this to say: »The present is not pregnant with a fixed programmed future, and the program is not in our heads. The theories of modern physics and the findings of neuroscience rule out not only a machine model of the world but also such a model of the brain« (Edelman 205).
map of memory
signs to create associative combinations for encoding information to coordinate and spatialize micro and macro cosmic relationships in one’s own memory. Gregor Reisch’s depiction of the human brain and its connectivity to the bodily senses places Memoriana in the »back of the mind.«
50
51
Image & Language Have you ever found yourself closing your eyes when trying to remember something? It’s as though we must be removed from the present in order to search for that which we wish to recall. Because the present is primarily informed by the visual information of our environment, we close our eyes to turn off the visual input. As a visual designer, one of the most fascinating aspects of the function of memory is its tightly interwoven connection with images and language —the essential ingredients of visual communication. In a sense, every act of visual communication is an act of memory touching the present in hopes of enriching both. This tight connection of image and language in memory is evident both from recent studies in neuroscience and as we each know it from personal experience. Figures of speech like, I can’t picture it, or It’s on the tip of my tongue, reflect the commonality of this experience when we face difficulty in remembering. The distinct roles of image and language in forming and later recalling memories are difficult to untangle, but clearly their combined communicative and evocative qualities give form to our personal histories.
whom to believe
language image language
52 Image & Language Processing During Recall Steven Rose is a neuroscientist and director of the Brain and Behaviour Research Group at the Open University in London. He has researched and written extensively on memory and learning. His writings are typically geared toward explaining what new ideas and discoveries in neuroscience might mean to a lay-public, thus making this very complicated topic area accessible to a wider audience.
50–52 The Key to Dreams is one by René Magritte’s
many experiments from the 1920s, compar-
Among his many methodologies, Dr. Rose and his team have made use of an image processing technique called meg (magnetoencephalography), which measures changes in electrical current flow within the brain and charts temporal dynamics, making it possible to monitor tiny changes in brain activity by the millisecond. Because signaling within the brain is primarily electrical, it is believed that increased electrical activity means increased signaling—in other words, communication within different areas of the brain. In a series of experiments his team took research subjects on a virtual supermarket shopping trip and asked them to make simple decisions, such as which brand of coffee to purchase. Central to this study is the relationship between immediate decision-making processes in the present and memory of prior
word image
ing words and images as means of representation. By labeling the images with seemingly mismatched words, Magritte brings our attention to the processes by which we see and read the world. In Design Writing Research , an experiment in writing, design, and publishing, Ellen Lupton and J. Abbott Miller blend visual and verbal communication by integrating pictograms directly into the lines of type. Images serve as language and vice versa. Vince Frost created the 100 Font typeface for the Mitchell Library by forming legible typography from photographs of historic objects. We are confronted with the task of both seeing and reading simultaneously.
related experiences, outcomes, and their subsequent expectations. Faced with deciding between three brands of coffee, the subjects responded with decisions within two seconds. »But in those two seconds, there was a flurry of brain activity. Within 80 milliseconds, the visual cortex became active; by 300 milliseconds, the left inferotemporal cortex, assumed to be a site of memory storage. At 500 milliseconds, the Broca’s area, a region associated with speech, was engaged as the subjects silently vocalized the range of choice items—and at 800 milliseconds, as they made their final decisions as to which item they preferred—assuming they preferred any—the right parietal cortex, associated with affect-laden decisions, was active. These dynamics reveal the many regions of the brain involved in even a simple act of episodic » What does much more to distinguish our
and semantic memory; even primary sensory regions like the visual cortex
specific human from non-human memory
are more active when people are performing a memory-related task than
is our social existence, and the technolo-
when they see the same images but are asked simply to make a cognitive
gical facility which has created a world in which memories are transcribed onto
choice.« (Rose in Radstone & Schwarz 201)
papyrus, wax tablets, paper or electronic screens; that is a world of artificial memory. It is artificial memory which means that whereas all living species have a past, only humans have a history. « Steven Rose
The specifics of how these mental models for associating and organizing information in the brain may not be certain, but language and image seem inextricable when it comes to how we remember objects and experiences. This suggests that the visual and verbal forms we use in recall and cognition each provide unique methods for making meaningful distinctions, and that both are necessary in cognitive processes of decision-making that relies on both past and present information.
Memory at Work and Working Capacity In order for the mind to undertake complex tasks such as reasoning or comprehension, we rely on the capacity to actively hold information in mind while other tasks are being resolved. This is called working memory. Working memory tasks are those that require goal-oriented active cognitive processing in the face of interfering processes or distractions.
53
54
If you’ve ever lost track of a sum while trying to add a list of numbers in your head, you know that working memory has a limited capacity. This capacity varies and is dependent on many factors, but certain quantifiable generalizations have been made through a variety of experimental research. Age is one of the primary factors in determining an individual’s working memory capacity—young adults tend to have a greater capacity than either children or older adults.
George Miller provided the earliest attempt at a quantification of capacity
53–54
and described what he called the magical number seven in 1956. His re-
Recent developments in medical imaging
search indicated that the amount of information that can be remembered
technology offer myriad new outlooks on examining the internal workings of our own
from a single exposure to an array of sensory inputs is between five and nine
bodies, and specifically our own brains.
items, depending on the information. Applying a range of + /-2, the number
Magnetoencephalography is a technique that
7 became known as Miller’s Magic Number. He considered this to be the number of items which can be held simultaneously in short-term memory at any given time. Miller’s initial studies had considered subjects’ capacity to remember a series of auditory tones, but he found this number to hold relatively true for a variety of other tasks.
measures and visualizes pulses of electrical activity in different parts of the brain over tiny fragments of time. Scientists presume the patterns in these visualizations reveal correlations between structures and functions of various areas of the brain during various cognitive activities.
mapping the brain
internal imaging
55
56
Later research revealed that this capacity depends more specifically on the kind of information to be remembered. For example, this capacity of about seven units holds true for numerical digits, but it is about six for letters and about five for words. Capacity further depends on the features of the information within a given category. For instance, longer words would reduce the capacity. Memory span for verbal contents (digits, letters, words, etc.) also depends on the time it takes to speak the contents either aloud or internally and whether or not the particular words are known by the subject. Here again, we see language and image intersecting at the formation of memory. Because of the range of factors at play, it is difficult to specifically quantify the capacity of short-term or working memory to a number of chunks of information. However, Nelson Cowan, the Curators’ Professor of Psychology at  All colors made me happy: even gray.
My eyes were such that literally they took photographs.ÂŤ Vladimir Nabokov
the University of Missouri who specializes in working memory, has proposed that working memory generally has a maximum capacity of about four chunks in young adults, fewer for children and older adults.
color code numbers
color code language
This concept of chunking is recurrent in much of the research on working memory capacity and its application is seen in a number of memory tasks. For example, when remembering a seven-digit phone number, we break the string of digits into two or more groupings. Rather than remembering seven numbers, we remember two chunks of numbers: the first chunk of three and the second chunk of four. If it is long distance, then we add an area code. We actually remember 10 numbers by breaking it into groups of three. In this manner, we are able to remember large bodies of data through chunking of elements that work within our working memory’s capacity: four to seven chunks with four to seven elements each. By segmenting and organizing the information, the ability to remember and learn increases.
Color Memory When environmental stimuli are perceived and begin to activate cognitive processing, this activity in the brain is referred to as arousal. Many experiments have been conducted on the effect that color has on perception and stimulation, and it has been found that color can indeed increase a person’s arousal. It has been further claimed that warm colors (red, orange, yellow) increase arousal more than cool colors. It is proposed that because warm colors are more vivid in nature and tend to advance visually in space— as opposed to the recessive visual quality of cool colors —they cause higher activation of visual perception, thus making us more alert (Huchendorf 1–4).
Arousing events are generally considered to have the ability to increase memory. Research by Wolters and Goudsmit has shown that exposure to an arousing event causes hormonal changes in the brain which enhance memory. Regardless of age or time passed after an event considered significantly arousing occurred, many participants in testing of this phenomenon described their memories as »very vivid,« (Huchendorf 2) and displayed a high level of recall for details of the event.
55–56 Dutch designer Karel Martens makes use of color relationships to establish correlations across his compositions. By layering visual information, he forms dense visual fields from which the eye utilizes color to relate patterns shaping letters, words, or numbers.
57 As difficult as it may be to conceive, one of the most dominant qualities of our daily lives is not physically there at all. Color is a concept
» What we were interested in,« says Edith, »was the heightened visual effect resulting from the juxtaposition of complementary colors — in this case the coldness of blue, meticulously selected for its complementary properties, and the warmth of the richly colored, varnished canvases.« » The blue is so prevalent, Ewoud continues, » it’s disorienting. The dimensions of the space are neutralized. Everything appears flattened. The only thing that remains,« he says, eyes widening in remembrance, » are the paintings.« Edith Gruson and Ewoud Traast (Ferrill 64–68)
that depends on the reflective properties of objects, our perceptual systems, and the way our brains put them all together. In the end, color is neither purely subjective nor objective, but a qualitative, sensory interaction with the world. Yet because it is a cornerstone of so much human experience, color takes on a tremendous potential for meaning. So much so, that the significance of any particular color varies widely across different cultures.
If color has the capacity to increase arousal, and arousal increases memory retention, it would seem that color can play a role in increasing memory. In 2006, researchers Spence, Wong, Rusan, and Rastegar found just that. They presented test participants a sequence of images of natural scenes on a computer monitor in either full color or gray scale scenes. After a second showing of the same scenes in either color or gray scale, the participants were asked which scenes were new. »Spence et al. (2006) found that color
increased the recognition of the natural scenes by approximately 5% « (Huchendorf 2).
Testing the effects of color temperature, in a 1999 experiment Bruce McConnohie presented a classroom of middle schoolers a slide show with alphanumeric characters and asked them to recall as many characters as they could immediately after the presentation and one hour later. This process was repeated with the same characters three times, each with a different back» There’s a lot of landscape I never would
have described if I hadn’t been homesick. The impulse was nostalgia. « Joan Didion
ground color (white, blue, and green), but with the characters consistently in black. He found that the slide show with a white background yielded higher retention rates both immediately and one hour after viewing the slide show. Since blue and green are both cool recessive colors, this test indicates that cooler colors may not only have less of an arousing effect than warmer ones, but also less so than stark black on white.
While this and similar tests are not conclusive on the effects color can have on memory—and many similar tests are ongoing—it would appear that the use of color in learning tools can play a role in memory retention by creating a more or less vivid impression. What this indicates is that the vividness of the impression, toward which color contributes, applies itself as a component of the emotional response in memory. Put differently, the affectivity of color stays with the memory of an experience while simultaneously contributing to the experience’s memorability.
In summary, these tests indicate the following role of color in relation to memory retention: an image in natural color is more memorable than one in black and white; warm colors and objects associated with them are more memorable than cooler ones; stark black and white figure-ground relationships are more memorable than those utilizing cool colors.
Expressive Memory If we consider learning to be the understanding we make about objects, concepts and their relationships based on our own unique experiences, then memory can be seen as an expression of that understanding. Much as metaphor is an expression linking the abstract with the physical, memory is an expression linking the past with the present. Thinking back to a fond moment in childhood, the relations of events matter more than the details. However, the details too are embedded in evocative affect. The distinct smell of that moment, the quality of light lingering with the associated images— these all add up to an expressive encapsulation of that experience. Memories are expressions laden with all the emotional, self-identifying, and physically felt affectivity of their source experiences.
Negotiating Memory and 3.3 New Experience
Everyday, we rely on memory for a range of functions—from the most repetitive subconscious expectations, such as the sun rising in the morning, to knowing how to respond to complicated decisions. We are constantly weighing new experiences against memories of past ones. In doing so, our understanding of self and world shifts with each new experience that contributes to some form of learning.
Stability and Change Learning requires both stability in the environment as well as a capacity to change or adapt to new information. Stability provides a basis for expectations in future encounters. Without stability, anything would be possible and learning would serve no purpose. We can define knowledge as that which we have learned by experience and proves to be stable across repeated or similar experiences. Before we have the means to explain it, we learn fairly quickly that the Earth has a gravitational pull, and this has implications for what up and down might mean. Our memory of past experiences Âť Remembering is a bodily activity, taking
place in the brain, and also in the connections between the brain and the nervous system. ÂŤ A.S. Byatt
creates this knowledge through the formation of expectations met with stable and expected outcomes. However, if we are entirely resistant to change or adaptation, nothing can ever be learned. In evolutionary terms, this
would certainly lead to extinction. Again, we have a seemingly paradoxical relationship: learning requires both stability and change. It is specifically through this process of negotiation between old and new information that learning occurs.
At the Center for Cognitive Science at Ohio State University, researchers found that learning for adults is impinged upon by the fact that they know more than children. »The memory accuracy of adults is hurt by the fact that they know more than children and tend to apply this knowledge when learning new information,« thus making the acquisition of new information more
difficult (Grabmeier). This provides a little empirical evidence for the old adage that you can’t teach an old dog new tricks. While we build a body of knowledge through the addition of breadth, depth, and nuance to what exists, there is a conflict that must be worked out between new information and expectations. It is during these cognitive processes that take place when new information is being validated and interpreted with old, that an understanding is modeled that gives the new information a place in memory— a place in our collective experience. There is an apt metaphor for this situation: keeping an open mind.
I Am What I Remember: Identity and Memory Because it maintains all we know of ourselves and the world we inhabit, memory serves a significant role in shaping consciousness. In turn, this means that memory also serves a critical role in shaping an individual’s perception of self. The memory of distinct experiences, sensations, emotions, and all those complicated relations between the individual and the external world (including other people)— both what is remembered and how it is remembered— shapes our individual senses of identity. Despite our endless and ongoing interactions with new sensations, new information, and new challenges, our lives maintain a sense of coherency through the constantly shifting presence of our memories. This presence provides a context for all interactions that is unique to each individual.
If I ask myself, Who am I? I would have to base my answer substantially on » Life can only be understood backwards; but
it must be lived forwards. « Søren Kierkegaard
the past: experiences, accomplishments, friendships, those places I’ve visited, the things I have learned, thought, spoken and written about. If I wake
up tomorrow and cannot remember any of these things, I would certainly not be my same self. Each new day can only be new when we remember the day and days prior.
» You have to begin to lose your memory,
if only in bits and pieces, to realize
In this way, memory is not only very subjective but also highly personal. Each memory carries with it all of the affectivity— all of the emotion and
that memory is what makes our lives. Life without memory is no life at all… Our memory is our coherence, our reason,
feeling —that shaped the formative experience for us. Once again, this
our feeling, even our action. Without it,
points to the pervasive role of aesthetics in shaping the contours (Dewey) of
we are nothing. « Luis Buñuel
our experiences and definitively affecting our understanding of the world through learning experiences.
» It is better to paint from memory, for thus
your work will be your own... « Paul Gauguin
4
4.0
Interlude An Astonishing Exchange
The story I am about to tell could be called a fictional reality. It is both as real and as imaginary as all of the ideas expressed by its impressive cast of characters. I cannot vouch for any amount of truth in what you are about to read. I can only share the story as I know it. Of course, that means it must be from my own point of view.
It is a stark, cold morning, the thick fog of sky smothering the city with its steely weight. One of those skies, with an indefinably cold smell, that recalls a distant childhood morning—bundling layers of sweaters and coats before trudging off to the bus stop. In fact, that memory may not be so far away, as this morning I am again trekking to school—this time to graduate studies. Despite the cold, my mind is abuzz, swallowed in thought that is equal parts memory and anticipation. I am looking for a thread. A fluid, cohesive form to pull together my research and interests in the vast complicated worlds of visual art, language, science, psychology, mythology, history, cinema, and image-making of all varieties.
Drifting through this jumble of thoughts, I am suddenly smacked back into the present by a small book lying on the sidewalk. Picking it up, I am reminded of Duchamp’s Nude Descending a Staircase because its form is a multi-faceted labyrinth of geometry. The book mesmerizes me with its ability to distort my perception of space. I can’t tell what’s inside or outside these intersecting planes of geometry and typography. I pick up the book. Since there is no discernible front or back, I just start reading anywhere.
…the discussion is lively, but clearly Dr. Saussure is becoming frustrated with the interruptions. He gathers himself and attempts to demonstrate to the small group of Hopi Indians that indeed time, as all signs do, must have at least two dimensions. With a small diagram, he indicates the denotative and connotative planes of the sign: time. Dr. Saussure is considered one of the fathers of 20th century linguistics, and his concept of the sign/ signifier/signified forms the core of the field of semiotics. Nonetheless, he faces difficulty in discussing time, as the Hopi can not formalize the idea of »past« or »future.« Their language does not allow it. As caretakers of the planet, they speak only in the present tense. The past is indicated by »present manifested,« and the future is simply the »present manifesting.« Thus, time seems rather irrelevant. The previously quiet and pensive St. Augustine interjects with his own point of view, which only complicates matters further. According to him, time is a threefold present: the present as we experience it; the past as present memory; the future as present expectation. At that moment, a warm breeze carries in the smell of linden trees and a small yellow butterfly swoops past the window. Always of strong opinions, Vladimir Nabokov stands up to make a statement, »I confess, I do not believe in time.«
01
02
03
04
Whoa…
Wait a minute! Am I reading this right? This is an astonishing exchange going on between writers and thinkers from across history. Information and ideas f loating in atemporal space. I am so engrossed in the story that I didn’t even notice I’ve arrived at the train station. Where did this book come from? Time, perception, memory, cognition, and narrative communication—these are exactly the topics I’ve been researching and thinking about. It seems greater than serendipity to have stumbled across this gem, so I flip through a few more pages and keep reading:
…the door opens, and in walks Mikhail Bulgakov, who has just returned from some time away, only to find that the world has changed dramatically. But, he questions whether the struggles remain the same. Has humanity been further enslaved or liberated by science and technology? Has the role of art been made extinct? Can they live simultaneously and harmoniously? Of course, everyone in the room has a different take on it. Dr. George Lakoff asserts the central role of metaphor as a means for creating and understanding abstract concepts—those essences, structures, and relationships beyond our immediate senses. To augment this assertion, Buckminster Fuller points out that over 99% of modern technology takes place in realms of physical phenomena either sub or ultra to human visibility.
05
06
07
Metaphors—both visual and narrative—greatly expand potential understanding of our world by allowing investigation of what happens at unseen micro and macro scales and into the realms of logic and philosophy. Science, philosophy, and art have provided stories depicting the »beyond visible,« fueling our imaginations to ponder the universe more fully. As might be expected, Marshall McLuhan can’t bite his tongue for too long. He jumps up to intercede on behalf of art. Obviously, he has thought this over as he states, »The artist is the person who invents the means to bridge between biological inheritance and the environments created by technological innovation.«
08
Negotiations continue on both sides, and the conversation shifts toward the relationship between science and fiction. Sensing his stage, Ray Bradbury attempts to de-bunk the term »science fiction« to say that all fiction is science and all science is fiction. He then claims, »Science is no more than an investigation of a miracle we can never explain, and art is an interpretation of that miracle.« Aha! So, art is interpretation. But then what can be said about the interpretation of art?
09
This is both fascinating and bizarre. I became instantly obsessed with Bulgakov’s Master and Margarita when I first read it a few years ago. His ability to carry multiple simultaneous narratives that are at once compelling stories and powerfully discursive allegories is really remarkable. When I read his earlier short novel Heart of a Dog, I sensed his concern about the possibility of an over-scienced world, and he’s certainly not alone. Once again, I have lost track of present time while mentally living in and through this conversation. The train pulls up to my destination, and, as I hop off, I see the first scant rays of light as the sun tries to break through the heavy clouds. As I continue on my way, I dive back into the book.
…they seem to have been arguing for a very long time, but this is not a disagreement. In his humble but squeaky assertive way, David Lynch agrees with Fellini — only they are using different words. Of course, realism is a bad word. In fact, realism can’t even exist because it would require drawing a line between the imaginary and the real. Who could do this? Fellini ardently repeats his perspective that »a different language is a different view of life.« Mr. Lynch only understands fragments of Italian, but, fortunately, cool and serene Monica Vitti is there to help translate (he always travels with beautiful women). Her face is neither an Idea nor an Event —it is pure Zeitgeist. David Lynch pauses to reflect then responds that indeed, the artist must have a sense of what he says, but all viewers are different, and we each see something that takes us to another place guided by intuition. He calls this an »inner-knowingness.« So, as we start to form our own understanding of the abstractions carried in a story, we begin to know it for ourselves. And, what we know for ourselves is always valid. As if in a dream of his own—maybe way out in space — Andrei Tarkovsky crouches on the ground and nods in silent agreement.
10 11
12 13
14
This book was already sounding incredibly close to my own thoughts, but now I can’t believe what I’m reading, or I mean what I’m seeing… well, it’s both. As I continue, the words themselves are morphing from letter forms into distinct images. I am simultaneously reading verbal and visual language—both reading and seeing—the thing that Gerard Unger 15 has said is not possible.
Finally, it strikes me! However it has occurred, this book must be my own creation. It is my own confluence of thoughts as conversation with my influences and also with myself. I continue to read, or speak, or maybe write—regardless, I continue to think:
We are fully immersed in what could be called the culture-media ether: the ephemeral discursive space of floating ideas, personalities, abstractions, and interpretations. We have broken free of any bonds toward fact and openly accepted the role of evocative narrative and visual-experiential communication. To evoke—to call forth from the past or bring to the forefront of perception—is a powerful approach to sharing ideas. To communicate in this method means to accept that it is the context of the receiver that shapes meaning. And, to make meaning is to learn. If we can communicate in a way that allows others to creatively make their own unique meanings—to learn through participation—our ideas will surely resonate. We have achieved Henri Bergson’s definition of intelligence: making tools to help make tools. In this case our tools help make the tools of cognition. Of learning, and of understanding. Of making meaning from our living experiences.
Now, I am basking in the full light of day.
16
Notes for an Interlude
01
Ferdinand de Saussure (1857–1913) was a Swiss linguist, widely considered one of the fathers of 20th century linguistics. His concept of the sign–signifier–signified–referent forms the core of the field of semiotics.
02
» Hopi Indians, who thought of themselves as caretakers of the planet, used only the present tense in their language : past was indicated as present manifested, and the future was signified by present manifesting. … It seems that practically everyone but contemporary man has intuitively understood the space-time continuum.« from Expanded Cinema by Gene Youngblood
03
» Time, said St. Augustine, is a threefold present : the present as we experience it; the past as present memory; the future as present expectation.« G. Youngblood, Expanded Cinema
04
V ladimir V ladimirovich Nabokov (1899–1977) was a Russian-American novelist, short story writer, and literary critic and lecturer. Nabokov is noted for his complex plots, clever word play, and powerful ability to evoke scenes as though they are personal memories. He was a self-described synesthete, and concepts or aspects of synesthesia can be found in several of his works. In his memoir Speak, Memory, he notes that his wife also exhibited synesthesia and, like her husband, she associated colors with particular letters. They later discovered that their son Dmitri shared the trait, and even more, that some colors he associated with letters were blends of his parents’ hues: »as if genes were painting in aquarelle.«
05
Mikhail Bulgakov (1891–1940) was a Soviet Russian writer and playwright active in the first half of the 20th century. He is best known for his novel The Master and Margarita, which is a critique of Soviet society written within a framing narrative involving two characteristically related time periods and/or plot lines. In the 1920s, his work was considered increasingly unwelcome by the Soviet censors, and government censorship prevented publication of any of his work or the staging of any of his plays.
06
George P. Lakoff (b. 1941) is an American cognitive linguist and professor of linguistics at the University of California, Berkeley. Most well-known for his ideas about the centrality of metaphor to human thinking, political behavior and society, he is particularly famous for his concept of the »embodied mind.« He contends that almost all of human cognition, including the most abstract reasoning, depends on and makes use of such concrete facilities as the sensorimotor system and emotions.
07
Richard Buckminster Fuller (1895–1983) was an American engineer, author, designer, and inventor. Fuller published more than 30 books, and developed numerous inventions, mainly architectural designs, the best known of which is the geodesic dome. » Better than ninetynine percent of modern technology occurs in the realm of physical phenomena that are sub or ultra to the range of human visibility. .. Yet world society has throughout its millions of years on earth made its judgments on visible, tangible, sensorially demonstrable criteria.« R. Buckminster Fuller, Ideas and Integrities, p. 64
08
Herbert Marsha ll McLuhan (1911–1980) was a Canadian educator, philosopher, and scholar: professor of English literature, a literary critic, a rhetorician, and a communication theorist. McLuhan’s work is viewed as one of the cornerstones of the study of media theory. Quote citation: Marshall McLuhan, Laws of Media, p. 98
Ray Bradbury (b. 1920) is an American fantasy, horror, science fiction, and mystery author, best known for his dystopian novel Fahrenheit 451 and the stories gathered together as The Martian Chronicles. Quote citation: Ray Bradbury, The Martian Chronicles
09
David Lynch (b. 1946) is an American filmmaker, television director, visual artist and musician, known for surrealist films whose unique cinematic style is characterized by dream imagery. »I should know the meaning for me, but when things get abstract, it does me no good to say what it is. All viewers on the surface are all different. And we see something, and that’s another place where intuition kicks in: an inner-knowingness. And so, you see a thing, you think about it, and you feel it, and you go and you sort of know something inside. … For yourself. And what you know is valid.« David Lynch discussing his film Mulholland Drive, 2007.
10
Federico Fellini (1920–1993) was an Italian film director known for his distinct blend of fantasy with baroque images, often including aspects of autobiographical narrative.
11
Monica Vitti (b. 1931) is an Italian actress most widely recognized from her cool, detached, even icy expressiveness in starring roles by Michelangelo Antonioni’s 1960s film cycle L’Avventura, L’Eclisse, and La Notte.
12
In his 1957 Mythologies—a collection of essays examining contemporary social value systems as creators of myths—Roland Barthes includes the essay The Face of Garbo . »The face of Garbo is an Idea, that of Hepburn an Event.«
13
Andrei Arsenyevich Tarkovsky (1932–1986) was a Soviet and Russian filmmaker whose work is characterized by spirituality and metaphysical themes with a lack of conventional dramatic structure and plot.
14
Gerard Unger (b. 1942) is a Dutch designer, educator, practitioner, and writer specializing in typography. From While You’re Reading (2007): »...it is almost impossible to read and look at the same time: they are different actions.«
15
Henri-Louis Bergson (1859–1941), was a French philosopher who wrote extensively on his premise that immediate experience and intuition are more significant than rationalism and science for understanding reality. He was awarded the 1927 Nobel Prize in Literature.
16
5
5.0
Narrative
5.1
Narrative as Communication The Stories of Our Lives Narrative Studies & Theories
Communication and Interpretation
A New Theory of Narrative Narrative Across Media 5.2
5.3
An Embodied View of Narrative Narrative as Learning Experience Interpretation of Narrative What Can We Learn from Narrative?
Narrative communication is one of the most wide-
5.1
spread, culturally significant, and individually defining acts of human activity. Across many forms—oral, written, iconic, dramatic, and cinematic, to name just a few—narrative communication shapes our vision of the world and ourselves. It is narrative that
5.2
gives form to our personal histories, defines our present, and suggests new possibilities for the future.
5.3
5.1
Narrative as Communication
The Stories of Our Lives Narrative as a communicative form is so prevalent, that I would need to employ narration even to reflect on my own personal history. Through the reconstruction of my own lived events, actions, and attitudes, I am re-memoring (a term coined by novelist Toni Morrison) in order to tell the story of my life — either to myself or others—in a narrative form. »Evidence strongly suggests that humans in all cultures come to cast their own identity in some sort of narrative form. We are inveterate storytellers« (Flanagan 198).
» We tell ourselves stories in order to live…
The word narrative comes to us from both the Latin verb narrare, to recount,
We live entirely … by the imposition of
and the adjective gnarus, knowing or skilled. By its very definition then,
a narrative line upon disparate images, by
narrative involves itself in both what it tells and how it is told. Narrative is a
the ideas with which we have learned to freeze the shifting phantasmagoria which is our actual experience. « Joan Didion » Art always serves a function — it either fur-
thers and helps the master narrative or it tries to disrupt it. « Felix Gonzales-Torres
form of communication, and communication is a social act with an agenda. As such, a narrative always expresses something about the one telling the story, even if it is only to reveal the source’s particular perspective. A story is never just a story. This is a statement that need only be validated by considering personal experiences. Indeed, a story is never innocent but always motivated. As soon as a story is told it relates or contrasts the intentions, affections, and assertions of the narrator with those of the audience.
Narrative is both the purpose and material for our contextualizing and making meaning of that which we have experienced. »Stories do not just provide us with a sense of who we are. To a large extent the stories about our lives and ourselves are who we are. … In a very fundamental sense, we exist and live our lives in and through stories « (Goodson, et al. 1). This fundamental
process by which we live in and through stories echoes our creation of conceptual systems via metaphor that relate back to our physical actions in and through our environment, in and through time. While many specific metaphors can be drawn, this points to a broadly applicable conceptual
metaphor of narr ative as container . We acknowledge its boundaries and our potential to step out of it. But, when we are actively experiencing a narrative, we place ourselves into its enveloping container. We think, feel, and experience it as a distinctly mediated world.
Over the course of history, cultural and technological innovations have given rise to a wide variety of narrative platforms—from the Greek drama to the printed page to the cinema and right up to today’s vast world of up-tothe-second digital communications such as twitter, text messaging, blogs, and myriad other social media. Regardless of the variety of narrative forms, the need for and function of narrative seems to remain always present, always current. It is an essential component of human expression, and perhaps the most enduring form of personal and cultural identity.
» Those who tell the stories rule society. « Plato » Living in a story, being part of a narrative,
is much more satisfying than living without
Narrative Studies & Theories
one. I don’t always know what narrative it is, because I’m living my life and not always
Because of its ubiquity as a cultural form, narrative has long been a source of
reflecting on it, but as I edit these pages
study, and many theories have been posed. Most theories distinguish be-
I am aware that I have an urge to see my
tween the story and the discourse, which equates to a distinction between what is told (story) and how it is told (discourse). While some theorists have
sometimes random wandering as having a plot, a purpose guided by some underlying story. « David Byrne
opted for a very narrow meaning of the term narrative, such as only orally transmitted stories, many argue that anything telling a story in any genre or medium is a narrative.
» Among the vehicles of narrative are articulated language, whether oral or written, pictures, still or moving, gestures, and an ordered mixture of all those substances: narrative is present in myth, legend, fables, tales, short stories, epic history, tragedy, drame [suspense drama], comedy, pantomime, paintings… stained-glass windows, movies, local news, conversation. Moreover, in this infinite variety of forms, it is present at all times, in all places, in all societies; indeed narrative starts with the very history of mankind; there is not, there has never been anywhere, any people without narrative; all classes, all human groups, have their stories, and very often those stories are enjoyed » I have no way of knowing whether the
events that I am about to narrate are effects or causes. « Jorge Luis Borges » A well-thought-out story doesn’t need to
resemble real life. Life itself tries with all its might to resemble a well-crafted story.« Isaac Babel
by men of different and even opposite cultural backgrounds… «
(Barthes Structural Analysis of Narrative 237)
Aristotle’s Poetics famously describes an ideal narrative as consisting of a core set of elements. From this starting point, a core format has been revised over the centuries into what some studies have recognized to be a nearly
universal structure to narrative. Central to this structure is a form of storynow language distinct from the speech of the here-and-now. Analysis of this structure breaks down into four sections typical to narrative:
Abstract A summary of what the story is about. Orientation Familiarizing the audience with the characters, locations, events,
time frame, and props or instruments important to the telling of the story. This component defines the who, what, when, and where. Complicating Action The telling of what happened, which is typically further
delineated into: conflict, peak, and resolution of the action. This typically accounts for the majority of the story, and includes the significant “events,” through which we can see the characters changing states. Coda The reprise of the story, taking us from the telling of the story as it hap-
pened to the current situation (back to the here-and-now).
» A book is more than a verbal structure or
series of verbal structures; it is the dialogue it establishes with its reader and the intonation it imposes upon his voice
Theorists of this universal structure believe that this recurring structure is a
and the changing and durable images
result of » the speakers’ unconscious knowledge of how to guide the listener
it leaves in his memory. A book is not an
into, through, and out of a cognitive representation or › situational model ‹ of past or imagined events.« (Bucci). By doing so, the narrator is able to cogni-
tively transport the listener/viewer out of the here-and-now, and its associated speech patterns, into the story-now language of the narrative—to offer an experience in and through the story. According to Wilma Bucci of the Research Training Programme of the International Psychoanalytical Association: »…the distinct grammar of story-now speech also signals a distinct cognitive state in which speaker and listener jointly imagine that the retold events are occurring now. A similar cognitive state occurs spontaneously in children play-acting and adult theatergoers: the here-and-now becomes less salient than the › other time ‹ represented. This is also the state of mind people enter when recalling personal episodic memories: what Endel called
› time travel, ‹ noting that episodic memory is the only memory system that allows people to re-experience the past. « Here, we should recognize the
isolated being: it is a relationship, an axis of innumerable relationships. « Jorge Luis Borges
important temporal factor in our experience of narrative and its relationship to memory. We move in and through the story as if in a physical progression . We experience the narrative arc in and through space, in and through time.
A New Theory of Narrative Rick Altman is a professor in the Department of Cinema and Comparative Literature at the University of Iowa. In A Theory to Narrative, he proposes a theory to expand our current understandings of narrative by looking beyond the analysis of either structure alone or singular characteristics such as genre, action, or character. He identifies three basic narrative strategies: single-focus, dual-focus, and multiple-focus narration. Given his field of study, » Stories only happen to those who are able
to tell them. « Paul Auster
it is not surprising that his framework aligns with recent studies of complex narrative in contemporary cinema, which have led many film scholars to recognize these same three categories. Nonetheless, he traces the shifting
» What would a narrative of happiness
be like? All that can be described is what prepares it, and then what destroys it. « André Gide
predominance of these three strategies through literature, visual art, drama, and eventually cinema— aligning this shift to the arc of Western art from the sacred (dual) to the profane (single) to the analytical (multiple).
Altman starts with the critical concept of following, by which we follow a narrative through a given perspective (typically a character’s) from one interaction to the next. This approach to narrative analysis allows us to expand beyond looking solely at action or character, and to see both. Most narratives work from a predominantly single, dual or multiple focus narrative. However, modulations within the story are possible, and allow the narrator to shift our focus to different points of view. Generally speaking, the different approaches work like this:
» Because you have seen something doesn’t
mean you can explain it. Differing interpretations will always abound, even when good minds come to bear. The kernel of indisputable information is a dot in space;
Single Focus Consistent following of a single character » along with whom
interpretations grow out of the desire to make this point a line, to give it direction.
we discover the surrounding world.« (Altman 325) For example, in the novel
The directions in which it can be sent,
Catcher in the Rye, we are guided through the story by a character whose
the uses to which it can be put by a cultur-
voice, actions, and attitudes very much define his perspective, which also slants our view of his world.
ally, professionally, and geographically diverse society are almost without limit.« Barry Lopez
58
59
58–60 In Fellini’s 8 1/2 , we follow the single focus story of the protagonist to share his vision of life, which swings wildly from memories to present situations to fantasy. A poster by Katherine McCoy for the Graduate Program in Graphic Design at Cranbrook Academy creates a
dual-focus narrative confronting many dichotomies within design discourse. Encountering these dichotomies episodically across the composition, we are invited into further dialogue and consideration. Da Zi Baos by the artists of Group Material aggregates varying opinions on contemporary issues from many organized social groups as well as people interviewed on the street. This multiplefocus approach to social dialogue outlines the group’s vision of the de-centeredness of power: »From poetics comes conversation and then collective decentering and the search for resolution forms poetics .« (Ashford 5)
single focus
double focus
60
multiple focus
61
62
evocation of play
evocation of the season
Double Focus Roughly symmetrical following of two characters or two sides
of an opposition, typically corresponding to notions of Good and Evil as expressed by the culture surrounding the story. For example, most Medieval religious art makes use of dual-focus narrative with its contrasting elements of Good and Evil or Heaven and Hell.
Multiple Focus Following of multiple story lines shifts identification from
single or dual character identifications. Discontinuity across story lines encourages the reader/viewer to question the conceptual schemes from which
61–62 The painting Children's Games by Pieter
the larger narrative is arranged. Subsequently, this encourages consider-
Bruegel is great example of the artist's ability
ation and analysis of broader themes and concepts than those specific to the
to evoke a time, place, and story through a
narrative itself: How is this narrative and its distinct form a reflection of broader issues in the present culture and society? By stepping beyond the point of view of any one individual, it is possible to evoke an event that »in all its volume takes on a meaning and an existence of its own, one to which only the reader can claim full access.« (Altman 251).
multiple-focus narrative strategy layering of perspectives. The result is a complex web of events that give an overall, collective impression of children at play. In Summer, Bruegel evokes the season through activities and sensations that visually capture the totality of the experience.
Altman cites the work of 16th century Flemish artist Pieter Bruegel as an example of multiple focus narration within a static image. Altman says the figures and actions in the engraving Summer »taken together and understood not in their specificity but in their collective generality, make up the season known as Summer« (Altman 238). Bruegel evokes summer via the relations,
actions, and patterns of many individual characters.
What is particularly useful in Altman’s theory is its ability to transcend both a structural analysis and any singular distinction of character, action, and genre, in order to provide a consistent method for understanding narrative in its myriad forms across multiple media.
Narrative Across Media » I’m not interested in art that is not in the
How is it that we are able to perceive a story as the same across different
world. And it’s not just the narrative,
media? How does the medium directly influence its understanding? Cer-
it’s not just the story; it’s the language and
tainly, different media affect different modes of stimulation, and many, such
the structure and what’s going on behind it. Anybody can make up a story.« Toni Morrison
as cinema, are multi-modal because they invoke multiple senses. Certain stories have such cultural significance or are so culturally adaptable— meaningfully different across different cultures in different times —that they are re-told many times and re-framed in different media.
One often hears, »The movie was okay, but I liked the book better.« This brings into comparison the cognitive process of imaging a story that is read as compared to the visual experience of seeing it in a cinematic form. Can you feel one more directly? Or just differently? For example, when I first read The Great Gatsby, I didn’t necessarily picture Robert Redford in the role of Gatsby. However, on subsequent readings after seeing the film adaptation, I can’t help but place his face with this character. The cinematic telling of the narrative, by its very nature, has assigned certain visual forms to the story that I had otherwise uniquely constructed through my reading of the text. In
a sense, it has both added to and subtracted from my experience: by adding the specific, it has removed potential for creating unique images of this narrative.
In the spirit of evocative communication, the challenge is to find methods for creating iconic narratives that are evocative and revealing of the story without diminishing the viewer’s opportunity for unique interpretation. This means acknowledging any situation’s complexity while harnessing it into a narrative form that leaves enough cognitive space for the viewer to make connections between the communicative material and his or her own personal experiences.
63 By making functional and aesthetic meaning of space, architecture is itself a narrative form. The ephemeral nature of Diller, Scofidio, and Renfro’s Blur Building daringly questions
» Upon entering the fog mass, visual and acoustic references are erased, leaving only an optical › white-out ‹ and the ›white-noise ‹ of pulsing nozzles. Blur is an anti-spectacle. Contrary to immersive environments that strive for high-definition visual fidelity with ever-greater technical virtuosity, Blur is decidedly low-definition: there is nothing to see but our dependence on vision itself.« Diller, Scofidio, and Renfro
this conception of architecture through its challenge to our sensory perceptions. It is an architecture of pure atmosphere, in which the primary building material is continually and instantaneously drawn from its site upon Lake Neuchatel. That material is both physically and metaphorically tied to all of human experience. It is, of course, water.
An Embodied View 5.2 of Narrative
The investigation into embodied cognition tells us that all learning starts with our interactional experiences in the world. What is the relationship between following in a narrative and the embodied perceptions of the world that we arrange into our memories of events, places, actions, people, etc.? How does narrative relate to this body-in-environment experience?
I believe the continuity we create by forming personal narratives can itself be seen as a conceptual metaphor, or consistent system of metaphors, whose origin is in our sensory and motor experience. When we use metaphor to define one thing in terms of another, we are highlighting certain aspects to relate while hiding or down-playing others. For example, in the previously described purposes are destinations metaphor, only some of the qualities of destinations , as source material, are attributed to the target of pu r poses . We apply as much structure from the source as is needed to create a cohesive metaphor while not worrying about the rest. Similarly, the events, objects, and perceptions we retain in memory are highly selective and will also necessarily hide or downplay others. When Âť Where all is known, no narrative is possible. ÂŤ Cormac McCarthy
combining memories in recall, we piece together those elements that fit together coherently into a narrative form. Shaping a narrative is directly
analogous to the cognitive process of assembling meaningful experiential gestalts into a cohesive whole: discrete characters, actions, locations, and dialogue can be seen as parallel to those interactions and affective qualities, which our perception and memory pull from the perpetual flow of sensations.
» …it is half the art of storytelling to keep
a story free from explanation as one reproduces it… The most extra-ordinary things, marvelous things, are related
This seems to be the embodiment of our inherent understanding of narrative. Our experience of objects, concepts, and events in and through stories, as in metaphors, helps explain the pervasiveness of narrative as an enduring and powerful communication tool. It is all the more powerful when we can sense the lived relation between the narrative and our own lives.
with the greatest accuracy, but the psychological connection of the event is not forced on the reader. It is left up to him to interpret things the way he understands them, and thus the narrative achieves an amplitude that information lacks. « Walter Benjamin
Narrative as Learning 5.3 Experience
Interpretation of Narrative Like any work of art or act of communication, it is the viewer’s/reader’s/listener’s interpretation of a narrative that will define its individual meaning. There is nothing too interesting about some ink on paper or light flashing » I should know the meaning for me, but
when things get abstract, it does me no good to say what it is. All viewers on the
erratically on a big screen without the viewer assembling these symbols into a narrative—no matter how loosely defined in his or her own mind.
surface are all different… intuition kicks in: an inner-knowingness. And so,
Communication requires abstraction, and abstraction invites interpreta-
you see a thing, you think about it,
tion. Extracting meaning from abstraction always depends on the context.
and you feel it, and you go and you sort of know something inside, for yourself. And what you know is valid. « David Lynch
The viewer’s embodied mind provides the context that defines the narrative’s meaning—with meaning being defined by purpose. By extension, this
context will determine the meaning and meaningfulness of both the story and the discourse.
The narrative becomes a metaphor for our own experiences and our own lives when we insert ourselves (our own memory, identity, expectations) into the story. By imagining myself in the Western, in the battle scene, in the romance, or in the somber drama, I am physically, cognitively and emotionally experiencing the metaphor of this story as my life— in and through my own experiences. And, I will certainly think and feel the implications of that metaphor. The stronger this experience is, the more likely I am to remember its effects. Those effects are then absorbed, felt, and applied to later experiences.
How powerful are the first few lines of a beloved story, even if it has been a decade since you read it? Those first lines ignite the whole story instantly in your mind, not only as a series of characters, events, dialogue, locations, but as an all-at-once experience. This narrative in a spark is analogous to a very specific experiential gestalt, which is understood and felt as the sum-andits-parts simultaneously.
What Can We Learn from Narrative? Reading Charles Dickens in 21st Century America provides a very different context than would reading it in Dickens’s own time. Contemporary situations and circumstances provide a critical distance and distinct perspective. This allows for a reading of the story as historical account, but it also means the contemporary reader can domain-map (to borrow language from embodied cognition) that situation to his or her own. The cultural shifts brought upon by the Industrial Revolution will likely map to the cultural shifts brought on by the Digital Revolution. The politics of Imperialism will map to the politics of Global Capitalism.
By making such translations as the reader, I have created (wittingly or otherwise) an interpretation— a second reading of the narrative, and I have made it my own. By assimilating the narrative, it is now part of my understanding of the world. I have made it my own learning experience. The meaning I » Of course, a sign doesn’t mean anything
unless you know how to interpret it. « Arthur Golden
make from this experience shapes my perspective, informs my view of reality, affects my sense of identity, and will shape my perceptions and expectations for future experiences.
64
64 In the historic Little Tokyo neighborhood of Los Angeles, Sheila Levrant de Bretteville and Sonya Ishii inlaid the sidewalk with a multitiered time line as a work of public art. Through images, written memories of residents, and names of the businesses which used to be in this area, it tells the stories of multiple histories simultaneously. The result is a rich layering of times, places, and events in a multiple focus narrative, acknowledging the nonlinearity of memory and history.
layers of narrative
6
6.0
Conclusion Designing for Experiential Learning
6.1
Prelude to a Summary
6.2
Summary
6.3
Context
6.4
Opportunity
Image and language serve unique but interrelated roles in making meaning from experience. From direct
6.1
interactions in our environment through metaphors that make abstraction possible, word and image collaboratively form our potential for understanding
6.2
6.3
objects, concepts, the world we inhabit, and ourselves. These interactional experiences become meaningful when we find purpose in them — in resonance with
6.4
memories of past encounters, relevance to our present situation, or shaping expectations for the future. Our own embodiment is always present in the images and language we use to narrate and share that which we find meaningful.
Prelude to a 6.1 Summary
Let’s step back in time for a moment to that distant but still resonant afternoon when I had my profound experience in the Rivera Courtyard at the Detroit Institute of Arts. At some root level, this research has unwittingly sought to understand what was going on in my six-year-old mind that afternoon. I was confronted with the vivid and evocative imagery of Rivera’s murals as an immersive experience. These four massive walls with their rich and varied image constellations worked in concert to establish a powerful sensory experience encompassing so many aspects of specific and universal human conditions. Delving deeper, many smaller narratives were to be found within the image panels, lending a sense of both cosmic and microscopic scale to the experience.
Much was certainly mysterious to my young mind that day. However, this fact did not diminish—in fact, it probably heightened—the impact, the awe and wonder of this experience. What I learned in quantifiable information that day was certainly limited: this is what a factory line looks like, the guys in suits are the rich guys with power, here are some symbols for different big ideas about nature, science, and technology, and so on. The meaningful significance of this experience has been expanded and built upon over many
years by repeat visits to the courtyard, learning much more about the art and politics of Diego Rivera and the United States at that time, and developing my own world view on these issues. The aesthetic qualities of my experience on that day mixed with the powerful evocative qualities of memory and personal narrative have made this not just an important moment in my youth, but a significant story of my own. Additionally, in a much broader sense, I gained an expansion of perception —a grasping of the simultaneously evocative and communicative powers of an iconic experience. I witnessed the potential for an image constellation to create meaning sited both within itself and the outside world—within my world, within myself.
Every word, image, and action is laden with innumerable potential for meaning. All that we know of the world—and indeed ourselves—we learn 6.2
Summary
through interactions with our environment. These interactions affect our perception, trigger our memory, and offer the potential for making meaning from new situations and new information. Defined as they are by the nature of their formal, physical qualities, these interactions can be translated into conceptual metaphors of an increasingly abstract nature. Metaphors become expressions of abstract or relational concepts to help us sense the imperceptible. As expressions, these metaphors bring a richness and depth with which we can better grasp those concepts we call abstract—concepts that are based on relations rather than physical objective scenarios. This capacity for abstraction allows us to think, discuss, and communicate concepts not directly or immediately perceptible. It greatly expands what we can think about and how we can think about it.
Of course, once we have exciting new ideas, we are inclined to share them. Communicating concepts requires the use of further abstractions — such as language, numbers, images, icons, etc. These symbolic devices, in turn, open the door to a variety of interpretations. A word always has multiple
possible meanings, and this is exponentially true for images. This means a multiplicity of potential interpretations is always available within any single abstraction or event and the associated objects or environments in which it occurs. The more evocative and less singularly explanative a communicative object or event, the greater potential it has to offer the audience opportunities for making meaning in and through its experience. The more fully I can bring myself, my perspective, and my physical self into an experience, the more that experience is enriched for me and by me.
This presents a spectrum of great opportunity for the creation of objects, environments, and interactions with some form of learning as the desired purpose. To evoke perception and perceptive memory, suggest or invent conceptual metaphors, or to converge multiple perspective—these are all paths toward memorable, meaningful engagement. Through the use of evocative abstractions whose interpretation is itself an invitation to make meaning, these kinds of experiences become opportunities ripe for learning through one’s own creative assembly of perceptions, memories, interactions, and personal narratives.
6.3
Context
Visual communication operates at the intersection of sensory perception and cognition. Making use of humans’ dependence on vision as their most relied upon sensory source, the eyes are the key to understanding most work in visual design. However, there is more to it than that. The products of visual communication are objects, environments, and experiences themselves. The tactile quality of turning the pages in a book plays a significant role in the experience of reading it. The haptic feel of a digital interface connects our sense of touch and movement to the material. And, the poster, often considered the most revered product of graphic design, transforms vision into action by unfolding its meaning in stages as the viewer walks toward it; encouraging both distant and close readings. These examples all demonstrate methods by which visual communication engages its audiences through acts of embodied cognition. I argue that the tactility, physicality, haptic responsiveness, or other physical
manifestations that define the form of any piece of communicative design have an equally significant role in making meaningful connections as do the abstract symbols within them or on their surfaces. Verbal language, visual language, and embodied language operate simultaneously—in communicative design as in our embodied lives.
6.4
Opportunity
By working at the intersection of image and language, visual communication has the capacity to profoundly shape its audiences understanding of the world. This creates both opportunity and responsibility for the visual designer and communicator. This research for design has revealed the potential for a broad program of additional research conducted by design investigation. The questions that arise point to the limitation of investigating these topics with language alone: How can we begin to identify and distinguish unique roles of
image and language in cognitive processes? How might we maximize the potential for each in evocative communication? What can be learned from the simultaneously evocative and explanative nature of metaphor as applied toward images? Language? Physical environments? Could such learning experiences heighten sense perceptions, extend learning potential, enhance memorability? Could they help us to see the world in broader and more vivid impressions? These questions not only open the potential for more research by design, but lend themselves toward application in a broad field of communicative endeavors. In creating evocative communicative experiences, the designer creates a dialogue with the audience through an invitation for interpretation. This dialogue takes place in and through the confluence of image, language, and physical interaction. It takes place in and through interactions over varying periods of time, in and through different media, objects, or environments. This combined physical and cognitive interaction, this mental and physical play, becomes definitive of the experience. In and through experience, we learn something new about the world while also learning something new about ourselves.
65
imagine
66
explore
Image List of Illustrations
Detroit Industry murals by Diego Rivera. Fresco cycle in twenty-seven panels, 1932–33. Detroit Institute of Arts.
01–04
Water, Fire, Air, Earth by Akko Terasawa. Logo design for Genus Design, 2007. Tokyo.
05
Work-Shift by Krivanek+Breaux. Public installation, 2007. Chicago.
06
Wachstum einer Avocado from the series Visual Explanations by Cybu Richli. Book, 2007. Lucerne.
07
National Holocaust Museum designed by Ralph Appelbaum & Associates. Exhibition design, 1993. Washington, D.C.
08
Copernican Planisphere by Andreas Cellarius. Hand-colored engraving, 1660. University of Utrecht.
09
The Treachery of Images by René Magritte. Belgium, oil on canvas, 1928–29. Los Angeles County Museum of Art.
10
Flight Patterns from the series Celestial Mechanics by Aaron Koblin. Collaboration with Wired Magazine and FlightView Software, 2008. www.aaronkoblin.com.
11
Blue Velvet by David Lynch. Film, 1986. Starring Isabella Rossellini, Kyle MacLachlan and Dennis Hopper. United States.
12
Play Time by Jacques Tati. Film, 1967. Starring Jacques Tati, Barbara Dennek and Rita Maiden. France.
13
Earthrise by NASA astronauts on the Apollo 8 mission. Photograph, 1968. Lunar orbit.
14
Outer Space by Peter Tscherkaskky. Film, 1999. Austria.
15
Invisible Mending from 7 Fragments for Georges Méliès, Journey to the Moon, and Day for Night by William Kentridge. Film, 2003. Collection of the artist, South Africa.
16
Allures by Jordan Belson. Film, 1961. United States.
17
Rock n Roll by Bradbury Thompson. Print, 1958. Inside spread from Westvaco Inspirations for Printers. RIT Library, Rochester.
18
Nuage Vert by HeHe (Helen Evans & Heiko Hansen). Light projection on power plant cloud, 2008. Helsinki. Photo: Antti Aahonen.
19
Infotag 2005 by C2F (Cybu Richli & Fabienne Burri). Poster for the Hochschüle für Gestaltung und Kunst Lüzern, 2005. Lucerne.
20
Internet Map, Nov 23, 2003 from The Opte Project by Alex Adai and Peter North. Data visualization using Large Graph Layout and Graphviz graphic engines, 2003. www.opte.org.
21
22
Die Flügel der Seemöve from the series Visual Explanations by Cybu Richli. Book, 2007. Lucerne.
23
Milwaukee Art Museum by Santiago Calatrava, 2001. Milwaukee. Photo: Derek Counts.
24–25 26 27–28
Fragments of RGB by Onformative Studio. Interactive installation, 2010. Berlin. Chelovek’s Kino-Apparatom by Georgi and Vladimir Stenberg. Poster, 1929. Russia. Containment Schema and Out Schema illustrations by author, 2011.
29
Way Out, designer and date unknown.
30
Book Shelf by Daniel Eatock. Installation, 2010. Eastside Projects, Birmingham, UK .
31
Playhouse by Traast+Gruson. Exhibition for Bang & Olufsen, 1999. Salone Internazionale del Mobile, Milan.
32
Simply Droog by Droog Design. Exhibition, 2006. The Museum of Arts & Design, New York.
33
OASE 71. Journal cover design by Karel Martens. Print, 2007. NAI Publishers, Rotterdam.
34
Floating Numbers by Art+Com. Interactive installation, 2004. Jewish Museum, Berlin.
35
Type in Motion II by Benjamin Dennel. Book, 2007. France.
36
Totality by Lutfa Rahman Nirjhar. Photo of solar eclipse on July 22, 2009. Bangladesh.
37
Poster for the film L’Eclisse by Michelangelo Antonioni. Film, 1962. Starring Alain Delon and Monica Vitti. Italy.
38–39
Water: H2O = Life at the American Museum of Natural History. Exhibition, 2004. New York.
40–41
News Readers by Christian Moeller. Plastic, chain link fence, 2006. Seattle.
42
Reading an Image illustration by author, 2011.
43
Ishihara Color Test, 74 designed by Dr. Shinobu Ishihara. University of Tokyo, 1917.
44–47
Zürich Tonhalle poster series by Josef Müller Brockman. Posters, 1955–58. Zürich.
48
Memory Theater by Giulio Camillo from his treatise L’Idea del Theatro, 1550. Italy.
49
The Medieval and Renaissance Brain diagram by Gregor Reisch, 1503. Germany.
50
The Key to Dreams by René Magritte. Belgium, oil on canvas, 1930. Private Collection.
51
Design Writing Research by Ellen Lupton and J. Abbott Miller. Book, 1996.
100 Font by Vince Frost. Typeface for the Mitchell Library, Melbourne. 2010, Austuralia.
52
Magnetoencephalography images of the human brain using MEG 160 software by the Yokogawa Electric Corporation, Japan.
53–54
Posters by Karel Martens published in his monograph Printed matter/Drukwerk, 2010.
56–56
De Stijlkamers (The Theme Rooms) by Traast+Gruson. Exhibition design for Centraal Museum, 2000. Utrecht. Photo: Ernst Moritz.
57
Scene from 8 1/2 by Federico Fellini. Film, 1968. Italy.
58
Cranbrook Academy of Art by Katherine McCoy. Poster, 1989. Cranbrook, mi .
59
Da Zi Baos by Group Material. Hand-lettered posters on colored paper, wheat pasted on the abandoned S. Klein building in Union Square, 1982. New York.
60
Childrens’ Games by Pieter Bruegel the Elder. The Netherlands, oil on canvas, 1560. Kunsthistorisches Museum, Vienna.
61
Summer by Pieter van der Heyden after Pieter Bruegel the Elder. The Netherlands, engraving, 1570. The Metropolitan Museum of Art, New York.
62
Blur Building designed by Diller, Scofidio, and Renfro. Built for the Swiss Expo 2002, platform outfitted with thousands of nozzles emitting a cloud-like mist of water drawn from Lake Neuchatel. Yverdon-les-Bains, Switzerland.
63
Omoide No Shotokyo by Sheila Levrant de Bretteville with Sonya Ishii. Time line of businesses, residents, and events inlaid in the sidewalk, 1996. Historic Little Tokyo, Los Angeles.
64
Scenography of the Copernican World System by Andreas Cellarius. Hand-colored engraving, 1660. University of Utrecht.
65
Z is for Mars by Tunc Tezel. Photo of Mars in retrograde motion, 2005–2006. Turkey.a
66
Detroit Industry murals by Diego Rivera (detail). Fresco cycle in twenty-seven panels, 1932–33. Detroit Institute of Arts.
67
Photo of the author taken in the courtyard of the Detroit Institute of Arts, 2010. Photo: Amanda Wizinsky.
68
Language Selected Bibliography
Aicher, Otl. Analogous and Digital. Berlin: Ernst & Sohn, 1994. Aicher, Otl. The World as Design. Berlin: Ernst & Sohn,1994. Altman, Rick. A Theory of Narrative. New York: Columbia U.P., 2008. Akutagawa, Ryunosuke. Rashomon and Other Stories. New York: Liveright, 1999. Aristotle. Poetics. Trans. S. H. Butcher. The Internet Classics Archive. Web Atomic and Massachusetts Institute of Technology, 13 Sept. 2007. Web. 5 May 2011. Arnheim, Rudolf. Art and Visual Perception: A Psychology of the Creative Eye, The New Version. Berkeley: University of California Press, 1974. Barthes, Roland. Mythologies. New York: Hill and Wang, 1957. Berger, John. Ways of Seeing. London: Penguin Books, 1977. Bergson, Henri. Matter and Memory. London: George Allen & Unwin, 1919. Damasio, Antonio. The Feeling of What Happens: Body and Emotion in the Making of Consciousness. San Diego, New York, London: Harcourt, 1999. Dewey, John. Experience and Education. New York: Free Press, 1938. Elsaesser, Thomas and Hagener, Malte. Film Theory: An Introduction through the Senses. New York: Routledge, 2010. Ferrill, Meghan. Traast+Gruson Exposed. Amsterdam: Frame, 2001. Flanagan, Owen. Consciousness Reconsidered. Cambridge: The MIT Press, 1993. Foster, John Burt, Jr. Nabokov’s Art of Memory and European Modernism. Princeton: Princeton U.P., 1993. Goodson, Ivor F., Biesta, Gert J.J., Tedder, Michael, and Adair, Norma. Narrative Learning. New York: Routledge, 2010. Hofstadter, Douglas. GÜdel, Escher, Bach: An Eternal Golden Braid. New York: Basic Books, 1979. Jahn, Manfred. Narratology: A Guide to the Theory of Narrative. Cologne: University of Cologne, 2005.
Books
Books continued
Johnson, Mark. The Meaning of the Body: Aesthetics of Human Understanding. Chicago: University of Chicago Press, 2007. Johnson, Mark. The Body in the Mind: The Bodily Basis of Meaning, Imagination, and Reason. Chicago: University of Chicago Press, 1987. Kolb, D.A. Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, NJ : Prentice Hall, 1984. Lakoff, George and Johnson, Mark. Metaphors We Live By. Chicago & London: University of Chicago Press, 1980. Lakoff, George and Johnson, Mark. Philosophy in the Flesh. New York: Basic Books, 1999. McConkey, James, ed. The Anatomy of Memory. New York & Oxford: Oxford U.P ., 1996. Mitchell, W.J.T. Picture Theory. Chicago: University of Chicago Press, 1994. Nabokov, Vladimir. Lectures on Literature. New York: Mariner Books, 2002. Nabokov, Vladimir. Speak, Memory: An Autobiography Revisited. New York: Alfred A. Knopf, 1999. Polt, Richard, ed. Heidegger’s Being and Time: Critical Essays. Oxford: Rowman & Littlefield, 2005. Radstone, Susannah and Schwarz, Bill, eds. Memory: Histories, Theories, Debates. New York, Fordham U.P. , 2010. Varela, F. J., Thompson, E. and Rosch, E. The Embodied Mind: Cognitive Science and Human Experience. Cambridge: MIT Press, 1991. Wittgenstein, Ludwig. Philosophical Investigations, 4th ed. Trans. P.M.S. Hacker and Joachim Schulte. Oxford: Wiley-Blackwell, 2009. Wood, Harriet Harvey and Byatt, A.S., eds. Memory: An Anthology. London: Chatto & Windus, 2008. Youngblood, Gene. Expanded Cinema. New York: E.P. Dutton & Co, 1970.
Anderson, D. and Gosselin, V. »Private and Public Memories of Expo ‘67: A Case Study of Recollections of Montreal’s World’s Fair, 40 Years After the Event.« Museum and Society, Mar 2008: 1-21. PDF .
Articles
Anderson, Michael. Embodied Cognition: A field guide. Artificial Intelligence 149, 2003: 91–130 PDF . Ashford, Doug. »Notes For a Public Artist.« Public Art is Everywhere. Hamburg: Kunstverein in Hamburg and Kulturbeholdeof Hamburg, 1997. Print. Barthes, Roland. »An Introduction to the Structural Analysis of Narrative.« New Literary History 6, Winter 1975: 237-272. Print. Brean, Joseph. »Memory Metaphors.« National Post 21 Dec. 2009. Print. Cowan, Nelson. »The magical number 4 in short-term memory: A reconsideration of mental storage capacity.« Behavioral and Brain Sciences 24, 2001: 87–185. PDF . Edelman, Gerald. »Memory and the Individual Soul: Against Silly Reductionism.« Nature's Imagination 1995: 204–5. Print. Huchendorf , Lynnay. »The Effects of Color on Memory.« UW-L Journal of Undergraduate Research X, 2007: 1–4. PDF . Susi, Tarja; Lindblom, Jessica; and Ziemke, Tom. »Beyond the Bounds of Cognition.« Proceedings of the 25th Annual Conference of the Cognitive Science Society. Mahwah, NJ , 2005. PDF . Wolters, G. and Goudsmit, J. J. »Flashbulb and event memory of September 11, 2001: consistency, confidence and age effects.« Psychological Reports, 96, 2005: 605-619. PDF .
Bucci, Wilma. »Narrative and Referential Activity.« The Referential Process, thereferentialprocess.org. Web. 5 May, 2011. Diller Scofidio, and Renfro. »Blur Building.« dillerscofidio.com/blur, n.d. Web. 25 Nov, 2011. Fieser, James and Dowden, Bradley, eds. »Embodied Cognition« The Internet Encyclopedia of Philosophy, A Peer-Reviewed Academic Resource, iep.utm.edu, 21 Jan, 2004. Web. 5 May, 2011. Grabmeier, Jeff. » Children Can Have Better Memory than Adults.« OSU Research News, researchnews.osu.edu, 21 July 2004. Web. 5 May 2011.
Web Sites
Thank You Author’s Acknowledgements
My gratitude goes out to the graduate faculty in the Graphic Design program at the University of Illinois at Chicago: Jörg Becker, Matthew Gaynor, Marcia Lausen, and Sharon Oiga, your guidance and encouragement have made me a more confident explorer. Thanks also to my classmates, Raul Peña and Chantrell Williams, for reassurance, laughs, and two years of patiently listening as I wrangled my sprawling thoughts into this thing.
Special thanks to: Meghan Ferrill, I am indebted to you for your exhaustive support in my brazen attempt at being a writer. Just maybe, designers can (and even should) learn to use language, too. Michael Renner at the Hochschule für Gestaltung und Kunst Basel, your incisive and critical comments focused my efforts tremendously, for which I am very grateful. Philip Burton, your passionate guidance during my process of designing this book will be unforgettable. Yours is a special form of teaching that comes with a complicated—but always compelling and instructive—narrative of its own.
And, of course, my undying love and gratitude to my wife—and sometimes editor, sometimes collator, often-times therapist: Amanda Wizinsky. Without you, this whole endeavor would certainly not have been possible. Thank you for your endless and ongoing support in all that I do. You inspire me constantly to pursue my dreams (distant or unreasonable as they might be), while always keeping me grounded in a place I very much want to live.
67
the maker’s touch
68
67– 68 A detail of the Detroit Industry murals reveals Diego Rivera’s self-portrait amongst the world he created. The author of this book in the Rivera Courtyard at the Detroit Institute of Arts in December, 2010 : a few decades after that first visit and many more since Diego's time.
back to the beginning
And then‌
Every word, image, and action is laden with innumerable potential for meaning. To understand the world we inhabit—and ourselves—we translate our living experiences through both physical and cognitive acts of metaphor, memory, and narrative. This research investigates processes involved in the perception and understanding of visual communications; to discover what happens in those learning experiences at the intersection of image and language. The purpose is to identify the potential for evocative communication to convert visual information into meaningful and memorable learning experiences.