Renner lecture notes

Word and Image – An Eternal Theme from a Designer’s Perspective Lecture Notes 7th of October 2015, Taipei, Taiwan

Prof. Michael Renner, Visual Communication Institute

The Basel School of Design HGK FHNW Basel, Switzerland michael.renner@fhnw.ch Manuscript not for print publication

1.

2.1 2.2

2.3 2.4 2.5

2.6 3.

Prelude Introduction to Word and Image – The Missing Link A poster for the Swiss Architecture Museum (SAM)

by Jiri Oplatek Drawing Processes and Image-Schema Figuration and Image-Schema Diagrammatic Images: Variations of the relationship of Word and Image The Spoken Word and Typography Conclusion

1.

Prelude

In the first part of this lecture I will give a short overview of key experiences that I consider to be responsible for my interest in the theme of Word and image. My experience in the practice of visual communication can be summarized in 4 phases.

1.1

Exploring Digital Tools

After finishing my Graphic Design education at the Basel School of Design where I studied with Armin Hofmann, Wolfgang Weingart and many other exceptional personalities, I went to San Francisco in 1986.

It was the time when Apple Computer and Adobe worked on the first PostScript applications. This technological progress allowed the description of forms with Bezier Curves. I was hired by Apple Computer in Cupertino to experiment with an alpha version of what we know today as Illustrator, which was called Picasso at the time. It was my task to explore the visual possibilities that these new tools promised

Word and Image – An eternal Theme from a Designer’s Perspective

Figure 1, 2, 3: Pacifc Bell telephone books, 1987.

to evoke. The question of how digital tools influence the creation of visual messages has accompanied my work ever since. Is there a digital aesthetic or is the computer merely a tool which replaces the analog means of graphic design? Is the global dissemination of the identical tools leading to a leveling of the visual expression?

With my first applied project to design the Pacific Bell telephone books, I was able to experiment with the desktop computer on a large scale at Richard Saul Wurman’s “The Understanding Business” in San Francisco [1, 2, 3].

Since no one was aware of the capability of digital tools, we had a lot of time to spend on the design development. In retrospect we can say that all the elements making up the design – pictograms, large key image and typography – could have also been made without the computer. What seems typical though for the digital means is a closer and more interwoven relationship between typography and images.

As a beta tester for Adobe, I was able to experiment with the newest software versions. The following poster series announcing lectures on art history, which I designed from 1991 until 1995, are typical examples of the exploration of the emerging digital tools [4, 5]. One could say that the digital means are again facilitating a close relationship between text and image in this example. Furthermore, the origin of the images becomes unclear. Are these photographic images? are they rendered or drawn? The method of their creation is hidden.

After these explorations with pre-released software packages, I realized that the visual invention, purely based on the technical possibilities, was a short-term success. Just after the official release of a new software version everyone was able to achieve the same effect with the same digital tools. This realization made me concentrate more on design solutions that were derived from the content of the message and from the printing technology.

1.2

Figure 4, 5: Posters for a lecture series in art history, 1991 – 1995.

Figure 6: Lecture announcements MIT Media Lab, Aesthetics and Computation Group, 2001/2002.

Beyond the Tools

These three posters [6], designed for a lecture series given by the members of the MIT Aesthetics and Computation Group, are examples that exemplify digital tools. I became interested in the questions concerning what it needs to break the stereotypical use of digital tools that are offered in a software interface. Do we use a rectangular box just because we have a tool for it? What does it need to avoid the reference to this simple application the software suggests? Do we use a font as it exists or do we invent letterforms to reflect the content?

In addition to exploring these possibilities, the posters were printed in offset with a small addition of silver to the ink. This created a material quality that was beyond the stereotypical digital leveling of visual expression. (This material quality can’t be reproduced in the digital presentation. You need to look at the original prints afterwards).

2

Michael Renner

Taipei Lecture

The following projects show an ironic approach in a context that is famous for its differentiated typographic culture. This poster for the thesis exhibition of the Academy of Art and Design 2004 does not reflect a typical typographic sophistication but instead exemplifies a sensational style by referring to cheap product advertisement [7].

Also the logo and covers of the yearly publication for the Academy of Art and Design (2004 – 2007) counteract the classic aim of presenting information with clarity [8, 9]. The logo contains two “mistakes” with the goal to attract attention. The capital letter “R” in the word “KURT” (a common German first name) has a filled in counter form, and the raised number 1 at the beginning of the word mocks its true position that would normally be at the end as a footnote. Both “mistakes” evoke an ironic and irritating reading and are surprising.

The series of posters for a Science Slam (2011 – 2013) at the Municipal Theater in Basel also request an ironic reading in their exaggerated and brash portrayal of the message [10].

1.3

Figure 7: Poster for the Thesis-Exhibition FHNW 2004.

Figure 8, 9: Yearbook of the FHNW Academy of Art and Design, 2004 – 2007.

Exploring the Computer as a Medium

Parallel to my exploration of digital tools as a means to design and to produce print products, I have been working on many projects that employ the computer as an interactive medium. With the suspicion that the computer as an interactive medium also preconditions our design of interaction, I have explored the possibilities to deviate from the expected (1995). The following screen recordings show an interactive experiment where the user can experience his or her preconceptions by alternately selecting simple yes and no buttons [11].

The interface of information kiosks, for example in museums, is a medium which can be designed with more liberty than interfaces for standardized software applications. In 2000 I designed interfaces for information kiosks at the vitra design museum. In the Luis Barragán exhibition interactive kiosks were used to make large amounts of information accessible to the visitors (2000) [12].

This telephone prototype for a Swiss telecommunications company was developed before the Apple i-phone [13]. It shows a phone that still has a receiver and a touchscreen to interact with the services.

One could place phone calls and have access to the Internet. It was never produced because the cost of almost $ 1,000 was evaluated as too high for a consumer market.

Out of the frustration of standardization in the context of interface design I have explored generative design methodologies [14]. Writing code to generate images is a peculiar way to explore how the relationship between words (code) and image (visual output) can occur. It seems to me that when we write in computer code the outcome rarely exceeds our expectations. Only if we are poor programmers we may surprise ourselves with mistakes which lead to unexpected results.

3

Figure 10: Posters for a Science Slam, 2011 – 2014.

Figure 11: Interactive explorations, 1995.

Figure 12: Information kiosk, Luis Barragán exhibition at the vitra design museum, 1995.

Figure 13: Ascom Touch Phone, 2000.

Word and Image – An eternal Theme from a Designer’s Perspective

Michael Renner

In 2007 I was invited to participate in a competition to design the 9th series of banknotes for the Swiss National Bank. Given the need of complexity in order to prevent banknotes from forgery, we explored generative methodologies to develop a script that translated continuous tone images into screens with customized dot shapes [15, 16, 17].

With this method the digital technology allowed us to create a new design tool with the computer language. It seems obsolete at this point to ask if a visual message is designed with digital or analogue tools. What counts today 1is the meaning it generates for the beholder in a given context.

2.1

1.4

Figure 14: Generative Drawing, 2001.

Figure 15, 16, 17: Idea sketches for the 9th Series of Swiss Banknotes, 2001.

From Digital Turn to Iconic Turn

Since its foundation in 2004, I have been a member of the research cluster eikones Swiss National Center of Competence in Iconic Research [18, 19, 20]. In the context of Art History the term “Iconic Turn” was coined in the mid 1990’s (Boehm 1994, Mitchell 1995). This term describes a major shift caused by digital technology from verbal communication to communication with images. The shift is problematic for democratic societies since the awareness of the power of images is not established until today. Since antiquity we have reflected upon how language creates meaning for an audience, but at a scientific level we don’t know what effect images cause. From this observation, interdisciplinary research clusters such as eikones have been founded in Europe and abroad.

In the collaboration with philosophers, art historians, psychologists, sociologists and historians of science, designers make a unique contribution through their ability to create images. Today, at the Basel School HGK FHNW, we call this research method “Practice-led Iconic Research” (Renner 2010, Renner 2011). The methodology can be described in short as: creating images in order to critically differentiate their meaning [21].The following reflection upon the relationship between words and images is an example of this research approach.

Figure 18, 19, 20: eikones logo and publications, since 2005.

4

Taipei Lecture

Introduction to Word and Image – The Missing Link „The Iconic Turn“ Gottfried Boehm, 1994 „The Pictorial Turn“ Thomas W. Mitchell, 1995 _ Digital technology causes a major shift from communication in language to communication with images.

The relationship of word and image, understood as the relationship between generalized idea and sensuous experience, has been elaborated on ever since we have written proof of philosophical reflections (Plato, 360 B.C.E.). What are the abilities of our senses to understand the world in which we live and what is the role of language in this process? We could enter this field of inquiry by recapitulating Friedrich Nietzsche’s (1844–1900) short sketch of Western Philosophy entitled How the ‘True World’ Became a Fable: The History of an Error (Nietzsche 1888) as a starting point and draw from Martin Heidegger’s (1889 – 1976) interpretation of Nietzsche’s cryptic notes (Heidegger 1961). This would help us to understand historical philosophical phases (Plato, Platonism, Kant and Idealism) where the hierarchy between the sensuous and conceptual thought in the perception of the true world [22] had been set. Plato believed that the “truth” can only be assessed through the idea and that the senses were inferior. In opposition, Nietzsche criticized this deeply rooted aversion to the sensuous, which was against images and their capability to truly asses the world [23]. He concludes in his philosophical contribution that we have to continuously renegotiate the relationship of the sensuous and the “super-sensuous” and that there is not a fixed hierarchy between the two entities.

We could also call upon the work of Lessing’s (1729 – 1781) Laokoon, in which he evaluates the capacity of literature versus the pictorial arts (Lessing 1766). We could also refer to more recent positions such as Thomas Mitchell or Mieke Bal. Or we can consider Hans-Georg Gadamer’s (1900 – 2002) essay Word and Image, following his claim that art is equivalent to the humanities and the sciences in its aim to understand the true world (Gadamer 1992) [24]. Continuing this line of thought, I propose the practical creation of images as a means to evaluate the world. This means: to interpret objects of design practice, to observe the processes of image creation, and to create image series, in order to differentiate how the two entities – word and image – relate, differ, or complement each other.

In opposition to this practical approach, the following section attempts first to look for a connection between words and images in the humanities which, in the above-mentioned references are mainly considered as opposed and unrelated entities [25].

If we are looking for a connection between conceptual thought in language and the sensuous experience conveyed by images, the philosophical debate on schematism will provide an entry into the topic. Heidegger (Heidegger 1929) explained his idea of the connection between a full-fleshed experience and language based on Kant’s (1724 – 1804) concept of metaphysics worked out in his Critique of Pure Reason (Kant 1881/1787). Heidegger emphasized that one always sees a specific object or a specific event [26]. The ability of our 5

_ In comparison to language, we have little scientific knowledge of how images generate meaning. _ A broad and critical understanding of images is crucial for a free and democratic society. _ Interdisciplinary research clusters such as eikones, Swiss National Center of Competence in Iconic Research, have been founded in Europe and abroad in order to advance the knowledge about how images create meaning. _ The contribution of design in this context of research is the knowledge about the generation of images.

– Pratice-led Iconic Research means: The generation of images in order to understand their meaning.

Prof. Michael Renner

University of Applied Sciences and Arts Northwestern Switzerland Academy of Art and Design HGK FHNW

Visual Communication Institute The Basel School of Design

9th of October 2015 michael.renner@fhnw,ch

Figure 21: Textslide, Iconic Turn.

How the ‘True World’ Became a Fable 1. Plato 2. Platonism 3. Kant 4. Idealism 5. Early Nietzsche 6. Late Nietzsche

Friedrich Nietzsche: How the ‘True World’ Became a Fable: The History of an Error, in: Christopher Kul-Want (ed.), Philosophers on Art from Kant to the Postmodernists, A Critical Reader, New York 2010, pp. 63 – 64.

Martin Heidegger: Nietzsche’s Overturning of Platonism, in: Christopher Kul-Want (ed.), Philosophers on Art from Kant to the Postmodernists, A Critical Reader, New York 2010, pp. 121 – 131.

Prof. Michael Renner

University of Applied Sciences and Arts Northwestern Switzerland Academy of Art and Design HGK FHNW

Visual Communication Institute The Basel School of Design

9th of October 2015 michael.renner@fhnw,ch

Figure 22: Textslide, Nietzsche 1.

How the ‘True World’ Became a Fable 1. Plato 2. Platonism

If something allows us to understand the „True World“ it is the Super-Sensuous (Idea, Concept, Word, Language).

3. Kant 4. Idealism

The evaluation of the Sensuous experience as inferior to the Super–Sensuous is key to the description of images as misleading and seductive.

5. Early Nietzsche

The Sensuous Experience is above the Super-Sensuous.

6. Late Nietzsche

There is no hierarchy between the two entities.

Friedrich Nietzsche: How the ‘True World’ Became a Fable: The History of an Error, in: Christopher Kul-Want (ed.), Philosophers on Art from Kant to the Postmodernists, A Critical Reader, New York 2010, pp. 63 – 64.

Martin Heidegger: Nietzsche’s Overturning of Platonism, in: Christopher Kul-Want (ed.), Philosophers on Art from Kant to the Postmodernists, A Critical Reader, New York 2010, pp. 121 – 131.

Prof. Michael Renner

University of Applied Sciences and Arts Northwestern Switzerland Academy of Art and Design HGK FHNW

Visual Communication Institute The Basel School of Design

9th of October 2015 michael.renner@fhnw,ch

Figure 23: Textslide, Nietzsche 2.

Following Nietzsche’s claim of an un-hierarchical relationship between the sensuous experience and conceptual thought Hans-Georg Gadamer describes the arts as a third field – equal to the humanities and the sciences – with its own right and methodology to asses the „True World“.

Hans-Georg Gadamer: Wort und Bild – “so wahr, so seiend”, in: Hans-Georg Gadamer, Lesebuch, Munich 1997, pp. 172 – 198.

Prof. Michael Renner

University of Applied Sciences and Arts Northwestern Switzerland Academy of Art and Design HGK FHNW

Visual Communication Institute The Basel School of Design

9th of October 2015 michael.renner@fhnw,ch

Figure 24: Textslide, Gadamer.

Do we have to describe Words and Images as two opposed positions without a relationship?

Is there a missing link between Word and Images?

Prof. Michael Renner

University of Applied Sciences and Arts Northwestern Switzerland Academy of Art and Design HGK FHNW

Figure 25: Textslide, Word and Image, opposite or continuum?

Visual Communication Institute The Basel School of Design

9th of October 2015 michael.renner@fhnw,ch

Schematism Heidegger/Kant

_ We are always experiencing a specific situation. _ We have the ability to generalize from a series of recurring experiences. _ In a photograph of a death mask we see an individual face, but also how faces, death masks and photographs look in general. _ The ability to generalize is key to understand a continuum between Word and Image. _ On one side of the spectrum we have the original experience on the other side we have the most generalized form of the experience – the linguistic term. _ Hearing a word activates a schema, which is formed by the generalized experiences of an individual’s past. This schema is neither an image as an immediate perception, nor an image as a depiction. It is connected to the image through the image-schema.

Martin Heidegger: Kant und das Problem der Metaphysik, in: Vittorio Klostermann: Martin Heidegger, Gesamtausgabe Band 5, Frankfurt am Main 2010, § 19 – 21, pp. 90 – 101.

Prof. Michael Renner

University of Applied Sciences and Arts Northwestern Switzerland Academy of Art and Design HGK FHNW

Visual Communication Institute The Basel School of Design

9th of October 2015 michael.renner@fhnw,ch

Figure 26: Textslide, Heidegger, Image-Schema.

Schematism Lakoff/Johnson

_ Mark Johnson defines the image-schema as a result of recurring structures and patterns of our sensorimotor experience, which are mental and corporeal at the same time. They follow a reliable logic and constitute an unconscious level of meaning. They are described as “image”-like, consisting of a topological structure of the perceptual whole realized in activation patterns in the form of neural maps.

Schematism Damasio

_ In other contexts, these records are called dispositions and described as condensed memories of our experiences stored below the threshold of consciousness.

According to Damasio’s model, these records are forming the dispositional space, which interacts with the image space.

Mark Johnson: The Meaning of the Body. Aesthetics of the Human Mind, Chicago 2007, pp. 137 – 144.

Antonio Damasio: The Feeling of What Happens. Body and Emotion in the Making of Consciousness, Orlando 1999, pp. 331. .

.

Prof. Michael Renner

University of Applied Sciences and Arts Northwestern Switzerland Academy of Art and Design HGK FHNW

Visual Communication Institute The Basel School of Design

9th of October 2015 michael.renner@fhnw,ch

Figure 27: Textslide, Johnson, Damasio, Image-Schema.

Word and Image – An eternal Theme from a Designer’s Perspective

Michael Renner

mind to go beyond the specific instance, to find common characteristics between repetitive experiences and, therefore, to generalize, is the key to understand any continuum between words and images.

Heidegger exemplifies the process of generalization with the photograph of a death mask. The death mask shows the face of an individual person. It makes a face viewable in its particularity but also as an example of how faces, a death mask, or a photograph can appear in particular or in general. Through the ability of generalization, our mind is able to develop a continuum starting with the experience leading to the most generalized schematic concept, attached to a linguistic term. The arbitrary word is referring to a schema, consisting of a rule defining the boundaries within which the object is recognized as such. Hearing a word activates a schema, which is formed by the generalized experiences of an individual’s past. This schema is neither an image as an immediate perception, nor an image as a depiction. It is connected to the image through the image-schema (“Schema-Bild”, Heidegger 1929).

The term image-schema appears in more recent positions found in philosophical elaborations versed in cognitive science (Lakoff/Johnson 1980, 1999). Mark Johnson defines the image-schema as a result of recurring structures and patterns of our sensorimotor experience, which are mental and corporeal at the same time (Johnson 2007) [27]. They follow a reliable logic and constitute an unconscious level of meaning1. They are described as “image”-like, consisting of a topological structure of the perceptual whole, realized in activation patterns in the form of neural maps. This topological structure can be applied to all sensuous experiences and is not limited to visual perception. We build such a structure for example if we hear music, taste food or walk in the street, and store them as condensed records in our memory.

In other contexts, these records are described as dispositions and they function as condensed memories of our experiences stored below the threshold of consciousness (Damasio 1999, 2010). According to Antonio Damasio’s model, these records are forming the dispositional space, which interacts with the image space, allowing us to explicitly address the sensuous percept [27]. These dispositions allow us to recognize a familiar face, formulate and understand a sentence in language, imagine something that we have seen or not yet seen before, and move our body in a certain way.2 We have now reached the point where we can say that an image-schema, conceived as a disposition underlying any human expression (be it in the form of gesture, language, or image) is not merely be-

tween word and image but rather forms a basis for both of them. As a function of the pre-verbal and the pre-iconic, the image-schema plays a central role in forming the unseen image in design and it is a critical entity in the process of perception in general. To create images can be described as working with potentialities and, in reference to Jean-François Lyotard (1924 – 1998), we can define the essence of communication design as the creation of an aesthetic experience by intriguing, confronting and surprising the beholder with something he or she has not yet seen (Lyotard 1997).

1 “Image schemas constitute a preverbal and mostly non conscious, emergent level of meaning. […] Although they are preverbal they play a major role in the syntax, semantics, and pragmatics of natural language. They lie at the heart of meaning, and they underlie language, abstract reasoning, and all forms of symbolic interaction.”

(Johnson 2007, 144) 2 We can quote Antonio Damasio at this point to avoid the misunderstanding that dispositions are understood as words: “Note that dispositions are not words. They are abstract records of potentialities. Words or signs, which can signify any entity or event or relationship, along with the rules with which we put words and signs together, also exist as dispositions and come to life as images and actions, as in speech or signing.” (Damasio 1999, 332)

6

Taipei Lecture

With an awareness of how word and image are portrayed from a philosophical viewpoint and using that as a backdrop, designers can take advantage of their ability to create images in order to evaluate their effect. In reference to the above described image schema we can create simple graphic compositions and evaluate what interpretation they provoke in the beholder. The simple examples here show four variations of a centered line [28]. ln the first composition from the left, we probably agree that the line divides the white background into two separate equal elements and that the black line continues outside the white square. ln the second image from the left, the line is a stick pointing up and going beyond the upper edge of the white square. This square can be read as a plane and the interpretation relates closely to a common experience of walking on a field. The third image from the left could be read as a stick reaching into the format. But the three dimensional reading is not as strong as in the image on the left. We could argue that the sight of a stick being held in front of us, where the stick comes from below, is much less experienced in daily life in comparison to the focus on the ground to detect possible obstacles. The last image with its vertically centered line makes it more likely to be read as a flat surface with a static line balanced in the middle.

The second series of images also relates directly to experiences of being in the world [29]. While the first image on the left is most likely read as a flat surface, the second image from the left is an image that is being divided, a river or a branch comes to mind. A thicker entity becomes two smaller ones and it would not be read as such if one of the branches would be thicker than the unified one.

The third image cannot be labeled with such schemas derived from nature. The strong triangle can be understood as an arrow. ln the fourth image the thick line is directed towards the beholder, a three dimensional representation is suggested. This series demonstrates with simple means that an abstract graphic composition can trigger image schemas formed by our experience with three dimensional space. It can also fit into a schema in a more generalized manner such as the principle of division or as the most generalized way to form a sign standing for direction of movement such as the arrow.

We can conclude from this short interpretation of graphic composi7

Figure 28: Graphic composition 1.

Figure 29: Graphic composition 2.

Word and Image – An eternal Theme from a Designer’s Perspective

Michael Renner

tions that there are many ways an image can address an image schema. With the spatial schemata described above it seems to be plausible that they derive from the human experience of moving in space and that they are implicit and not a result of learning, even though the experience of looking at representations of space are influencing the image schema as well. They belong to the iconic rather than being words in a language. The arrow on the other hand can be read as a visualization of movement and could therefore be understood as a graphic mark derived from our experience. On the other hand we have seen so many arrows, and are conditioned to read them as a convention, as a sign standing for the indication of a direction. ln this case we can say that the schema is relating to an experience but its generalization is formed by convention – by a language. The arrow could be a designers description of the middle ground between symbolic meaning in words or symbols and the representation of an experience in an image.

sight which sparked the typographic idea for the final series was the S AM comprehension that architecture is built from the ground up3. This 11.12.–11.01.07: S AM simple observation resulted in the designs shown here which inverted the conventional direction of reading from top to bottom, to a reading of the information from bottom to top [31]. With the arrangement of the S AM most important information at the lower edge of the format in large letters and a sequence of information blocks set in decreasing type Figure 31: Early sketches for the SAM poster series by Jiri Oplatek, 2006. sizes toward the top of the poster, an unexpected effect is evoked.

The typographic composition refers to the construction of the building represented in the photographic image and achieves with this reference a tight relationship between typography and photograph [31 right]. We can conclude that the typographic composition triggers the image-schema of building an object based on our experience of gravity. Furthermore the decreasing sequence of type sizes addresses an- Figure 32: Sketches leading to the deconstruction of the typography other image-schema. The dependence on the relative size of objects by Jiri Oplatek, 2007. from their distance to the beholder is a reason to read the typographic composition also as a spatial arrangement. Through the gradation of type sizes from bottom to top, a sequence of objects are created, which are arranged in increasing distance to the beholder and lead him/her towards the building. Both iconic messages address the beholder on the level of his/her bodily experiences in 3-dimensional space and not through the message encoded in the words represented by typography. Through the comparative observation of the sketches [32], which led to the final poster, the significance of deconstruction of the letter Figure 33: Final poster for the Pancho Guedes Exhibition at the SAM forms becomes apparent. The use of type without any alterations of by Jiri Oplatek, 2007. the letterforms leads to the effect, that the typography operates on a separate level which overlays the photographic image. The alteration of the letterforms through organic shapes was done with the intention to refer to the design process of the architect. But for the beholder they appear as a contradiction to the clear cut architecture and dissociate from the photographic image. What is especially striking is the deconstruction of the letterforms through clear cut lines, as long as they refer directly to the photographic image [33]. The deconstruction of the letterform in accordance with the photographic image creates two interfering descriptions of the three dimensional space. On one hand the letters SAM are overlaying the photograph and are therefore in the foreground. On the other hand the cut line of the letters suggests that the plane depicted in the photograph is covering up the letters. Through this conflicting message a close relationship is created between the two levels of communication. The mutually contradictory reference to space created between photography and typography is evoked by the iconic dimension of the typographic composition independent of the linguistic content of the words. We could also interpret the text and its relationship to schematic records. But this area of inquiry belongs to linguistic studies.

2.2

Figure 30: Poster on the outside of the Swiss Architecture Museum, SAM in Basel, 2010.

A poster for the Swiss Architecture Museum (SAM) by Jiri Oplatek

The history of posters comprises a wide spectrum of combinations of words and images and is therefore a good object for analysis. Despite the fact that we find an endless reconfiguration of combining word and image in poster design there is little reflection upon the subject in writing. Often projects of visual communication are explained referring to semiotic theory. Roland Barthes has demonstrated in his essay on the Panzani pasta advertisement (Barthes 1964) how far the analysis of a complex composition of text and photographic image can reach using a semiotic model as the starting point. He distinguishes the linguistic message, the symbolic or connoted message, as well as the literal or denoted message.

The denoted message is simply explained by Bathes as the photograph of a tomato referring to a tomato. But the relationship of the elements of the composition, the quality of light and color and the use of focus and other iconic qualities of typography and photography are out of reach in this analysis, which approaches the image from a model inferred from language. For the practice of visual communication, the semiotic model might be helpful to touch obvious aspects in a client’s presentation, but it’s not able to explain the affect a product of visual communication creates in the beholder.

The concrete object of analysis we focus on now is part of a poster series for the Swiss Architecture Museum (SAM) in Basel [30]. Since 2007 the series has been designed by Jiry Oplatek of the Design Office Claudia Basel. The poster for the exhibition of the Portuguese Architect Pancho Guedes (2007/08) was the first poster in the series.

During my interview with Jiri Oplatek, he explained that the crucial in8

Taipei Lecture

P ARKPL AT Z SHOPPING CENTER SPREITENBA CH ARGA U STEINENBERG 7 POSTFACH 911 CH-4001 BASEL TEL. +41 61 261 14 13 WWW.SAM-BASEL.CH

ÖFFNUNGSZEITEN DIENSTAG, M ITTWOCH UND FREITAG 1 1-18 H DONNERSTAG 11-20.30 H SAMSTAG / SONNTAG 1 1-17 H

3

Ö F F N U N G S Z E DIENSTAG, MITTWOCH UND FREITAG DONNERSTAG SAMSTAG / SONNTAG

Ö F F N U N G S Z E DIENSTAG, MITTWOCH UND FREITAG DONNERSTAG SAMSTAG / SONNTAG

I

T

E N : 11.00–18.00 H 11.00–20.30H 11.00–17.00H

Figure 34: Examples of of the poster series for the SAM by Jiri Oplatek, 2007-2014.

Unpublished interview with Jiri Oplatek by the author, recorded on December 4th, 2012.

9

I

T

E N : 11.00–18.00 H 11.00–20.30H 11.00–17.00H

11. 12. 06–11. 01. 07: SCHWEIZERISCHES ARCHITEKTURMUSEUM STEINENBERG 7 PARKPLATZ — CH-4001 BASEL : +41 (0)61 261 14 13 SHOPPING CENTER WTELEFON W W . S A M - B A S E L . O R G S P R E I T E N B A C H 11. 12. 06–11. 01. 07: AARGAU — PARKPLATZ — SCHWEIZERISCHES ARCHITEKTURMUSEUM STEINENBERG 7 SHOPPING CENTER CH-4001 BASEL TELEFON : +41 (0)61 261 14 13 SPREITENBACH W W W . S A M - B A S E L . O R G AARGAU — IM

Word and Image – An eternal Theme from a Designer’s Perspective

Figure 35: Visual questionnaire, Indre Grumbinaite, MA Thesis, 2011.

We can conclude that the poster of Jiri Oplatek achieves a close relationship between word and image, since the typographic composition addresses the beholder in iconic form as a representation of architecture and three dimensional space. Through the deconstruction of the typography, a paradoxical description of the spatial situation occurs, which leads to the flattening of the three dimensional effect of typography and photography. Furthermore an ornamental effect becomes apparent through the two dimensional accumulation of visual stimuli.

They capture the attention of the beholder because the contradictions of the spatial situation cannot be solvable by relating them to a bodily experience or a schema. The series of posters creates this relationship between typography and photography in different ways [34].

With this analysis of the poster we extend the possibilities of Barthes interpretation of the literal message to an iconic interpretation and abandon the preconceptiopn that images are comparable to language.

Furthermore, the example shows that the process of design results in artifacts, which bare an implicit knowledge about the relationship between word and image.

Obviously the designer of the poster would not explain the effect of the poster and the interaction between typography and photography like I described it above. Nor did he make the decisions in the design process on a level that can be replaced by rational thought. On the other hand, the result of the poster cannot be explained as an accidental process. Instead we can describe the design process, as an experimental methodology, leading to a series of variations in the form of sketches.

In view of them we can conduct a comparative analysis and evaluate the meaning of one composition against another. ln this form of critical differentiation the implicit knowledge of designing can be compared to the discourse in language.

Let us now leave the analysis of design objects, and focus instead on processes which are guided by implicit knowledge. We can compare drawing, which is considered as an intuitive approach in the context of images, with collaging which is described as a conceptual approach in the context of language (Mersch 2007).

Figure 36: 9 textural samples of the visual questionnaire, Indre Grumbinaite, MA Thesis, 2011

2.3

Figure 37: Collected results from the questionnaire, Indre Grumbinaite, MA Thesis, 2011

Drawing Processes and Image-Schema

In the following experiment, twenty participants were given nine samples of mostly textural images in a visual questionnaire [35]. After contemplating the visual stimuli, they were asked to draw their mental images directly onto the nine samples [36]. It is evident that the participants in the survey were considerig the contingencies based on their own dispositions. From the results, we can infer some general observations in a comparative analysis: with the photographic 10

Michael Renner

Taipei Lecture

image of gravel [36 bottom right] for example, it was more difficult for the participants to develop an image than with the drawn textures.

The ambiguous, clear-cut shape turned out to be as productive in triggering visual imagination as the textural images [37]. Faces and figures of people and animals appear more often in imagined images than in any other possible subjects.

Already these observations clearify how the processes of perception and image generation stimulate or inhibit the emergence of past experiences stored in the dispositions of the participants. In other words, we can see how the interaction between stimulus and image-schema is conducted in the diverse samples. The photographic image is recognized as a representation of a specific situation and thus the process of contemplation is soon terminated (Dewey 1934). On the other hand, the stimulus does not have to occur out of focus. Especially the “affirmed ambiguity” presented by the sample with a clear-cut form [37] produces a broad range of visual associations. The numerous appearance of faces lets us assume that image generation depends on the hierarchy in which a persons dispositions are organized.

Furthermore, the comparative analysis of all the resulting images enables us to distinguish the different processes employed by the participants who transformed their mental images into a material form.

A first group of drawn additions clearly shows how the image was developed with a repetitive manual gesture from the existing texture.

We can infer from this process that these images are the result of a procedure in which the definite form has developed in the process of image generation through the free gestural movement of hand and arm. This approach is often chosen when the stimulus is a dense textural quality [38].

A comparative analysis of the images shows a second category of results. In a relatively small number of final drawings, the stimulus is explained through the addition of drawn elements. The composition of freely arranged rounded lines for example, identifies the lines as randomly arranged threads through the addition of a sewing needle.

This approach to image creation does not rely on a step-by-step materialization of a concrete form through the gestures of drawing. The act of drawing follows an idea that is conceived before the first line is set on paper. In this process of materialization, the mental image follows a concept, which can be verbally described before the image actually emerges [39].

We can describe a third group of drawings consisting of single lines added to the texture. The lines extend, complement or outline the existing elements and integrate them as part of a figure or the ground.

This kind of drawing method is more often applied in the context of stimuli with a blotch-like appearance. This approach of linear drawing can be situated between the first group of gestural movement and the second group of a conceptually framed drawing gesture.

The third group develops form in a flexible manner as a progressive 11

Figure 38: Drawing as a result of a gestural repetitive movement, Indre Grumbinaite, MA Thesis, 2011

Figure 39: Drawing as a result of conceptual reflection, Indre Grumbinaite, MA Thesis, 2011.

Figure 40: Drawing as a result between conceptual reflection and gestural movement, Indre Grumbinaite, MA Thesis, 2011.

Word and Image – An eternal Theme from a Designer’s Perspective

Michael Renner

line, which alternates between premeditation and the evaluation of the previously set stroke [40]. .

A variety of possibilities employing a sensuous dimension and a conceptual dimension in the process of drawing can be inferred by the categorization of the drawings into three groups. In reference to the image-schema described above, we can classify the process of gestural drawing as playful and flexible approach that leads to matching contingencies between the material sample and the mental image arising from dispositions. In opposition to this extended process of search, the conceptual approach is a direct matching process even before the pencil touches the paper (Dewey 1934). Once the visual idea can be described with words, the movement of the arm is primarily an act of execution. In the process of becoming an image with gestural form finding, the negotiation is an ongoing aesthetic experience, whereas the recognition of the conceptual stops the fluidity of contemplation.

If we turn away from drawing and focus on the most figurative process, we can analyze the methodology of arranging existing images and we can once again ask how unexpected constellations appear in the process of image creation.

them. This leads to an orderly and structured impression, within which each object is placed according to a systematic evaluation.

We know this kind of arrangement from biological collections or archaeological surveys of artifacts. Already on the level of composition, we are referring to principles derived from our bodily experience in the environment. Image-schemas are triggered in the fluid process of creating these arrangements as well as in the process of perceiving them. The act of arranging the overall composition is guided by the interaction of the images and the dispositions of the designer.

Physical movements of the arm and hand are also important in this process to evaluate the optimal relationship between the single images as directly as possible.

On the level of the single object represented in the arrangements, we can highlight the doormat with the diagonal pattern. In fact the rug is neither belonging to the context of Art Nouveau nor the patterns of an African culture, the context of board games or beekeeping. The arrangements allow us to experience how flexible we are in adapting the core structure of an image-schema to a context. The specific diagonal pattern of the doormat is not part of the rule defining what a doormat is. It could be any kind of textile decoration. Therefore, we can bend the meaning of the diagonals in many directions supported by the overall reading of the objects thus arranged. The pattern lies within the boundaries of the generalized rules describing the essence of a doormat.

With the arrangements combining shoes, baby seats and baby carriages, we can experience the matching process within the image-schema [42, 43]. What makes us recognize a shoe and what does it need to relinquish the expectation of seeing an arrangement consisting only of shoes?

In the following arrangement selected from the same study, we recognize the arrangement of 16 similar shapes representing 15 vacuum cleaners and the Empire State Building [44]. With the formal similarity and the absence of scale, we can see all of the objects as vacuum cleaners or as high rises. Even though the photographic image is not as ambiguous as the one in the previous experiment of drawing, the ornamental repetition of the objects and their formal similarity let us change our point of view and attribute a new meaning in the process of perception. This example can be used to demonstrate a model-like situation in which the jump from a preconceived idea to a new association can happen in the beholder as well as in the designer who is creating the collage. In addition, the clash of meanings between a daily household item and a landmark of national importance triggers a whole series of associations. They are evoked by the perception of formal similarity and the experience of misinterpretation either of the vacuum cleaners or the landmark.

If we alter the photographic representations to high-contrast silhouettes and simplify them even further, the ambiguity of the interpreta-

2.4

Figure 41: A Matter of Arrangements, Pauline Guex, MA Thesis, 2013.

Figuration and Image-Schema

In the following image combinations [41], we can distinguish different levels emerging in the process of perceiving the compositions. At first, there is an overall evaluation of the plane. On a second level, one starts to recognize the single objects represented photographically and, in a last step, we are comparing the objects in search of the reason for their combination. Even though the examples selected show a small range of possible options, each of the compositions addresses a specific message through the placement of the objects within the format.

On the first level of interpretation, the composition on the left suggests that we compare the two objects [41]. The three objects in the second composition from the left are suggesting a spatial situation.

The objects seem to sit on the ground. Through the difference in the size of the objects, they are not asking for a one-on-one comparison of the objects. Perhaps they are understood as telling the story of an accidental arrangement encountered when setting up a gallery exhibition. The middle composition is giving each object a comparable space and, through the closeness of the objects, we are requested to draw conclusions from the similarities and differences of the objects depicted. It is hard to isolate one single object, and they are competing with each other for attention. Both compositions on the right present the objects with more space around 12

Taipei Lecture

13

Figure 42: Triggering similar image-schema 1, Pauline Guex, 2013.

Figure 43: Triggering similar image-schema 2, Pauline Guex, 2013.

Figure 44: Triggering similar image-schema 3, Pauline Guex, 2013.

Figure 45: Triggering similar image-schema with an ornamental approach, Pauline Guex, 2013.

Figure 46: Diagrammatic images, traces on a piece of paper of a felt pen blowing in the wind, Marie-Louise Greb, MA Thesis, 2013.

Word and Image – An eternal Theme from a Designer’s Perspective

Michael Renner

tion is even stronger [45]. By leaving the mimetic quality of the photograph behind, we are approaching a purely ornamental arrangement.

This is most effective in triggering a long process of matching elements of the material image with generalized memories.

Even though drawing is often considered as a process of imagination and arranging as a process used in the context of language, we can say that both processes can be described as a continuous evaluation of the relationship between the specific constellation of the material sketch and the image-schema in the designers memory.

3.

2.5

Figure 47: Description of the process with language, Marie-Louise Greb, MA Thesis, 2013.

Diagrammatic Images: Variations of the relationship of Word and Image

Diagrammatic images use a combination of words and images in order to provide an instant comprehension of complex data by the beholder. This experiment uses the marks [46] and the video recording of a felt pen moving in the wind as the basic data to explore the description of the process with language [47], with the reference to coordinates [48, 49] in form of a table [50] and different variations of diagrammatic interpretations. The variations allow the demonstration of an ideal combination [51] of the two entities to provide an insight at a glance to a beholder. They also demonstrate how the graded system of language is not transferrable to the ungraded system of images [52] (Goodman 1976).

Figure 48: Description of the recorded process with a coordinate system, Marie-Louise Greb, MA Thesis, 2013.

2.6

Figure 49: Description of the recorded process with text and coordinates, Marie-Louise Greb, MA Thesis, 2013.

Figure 50: Description of the process with a table with coordinates and time, Marie-Louise Greb, MA Thesis, 2013.

The Spoken Word and Typography

The transformation of spoken language to a written notation is bound to the loss of nuances expressed in sound. From the way a sentence is articulated one can understand intuitively the emotional state of the speaker. The same sentence written down on paper does not carry the full scope of the audible experience. On the other hand, an alphabetical letter is not just a sign that is referring to a sound in a spoken language.

Beyond the reference to a sound, a letter in a specific font always carries a message on an iconic level (Krämer et al. 2012). A letterform may be read as static, dynamic, reduced or ornate, or as belonging to a specific era or cultural context. The following experiment persues the goal to extend the iconic level of a letterform in order to visually refer to a specific sound quality in language [53]. The posters for a sound poetry festival combine visually extended letters addressing sound with letters from a standardized font in order to address the audible quality of poems [54].

14

Taipei Lecture

Conclusion

At the beginning of my discourse on word and image, we defined the image-schema as a dispositional element situated between conceptual thought (word) and sensuous perception (image). Afterwards we examined a series of images – basic compositions and posters – in order to show how the schematic record can be related to the interpretation of an object of visual communication through the beholder. I have described the arrow as a middle ground between the visual derived from experience and the sayable learned in a socio/cultural exchange. We have also found a middle ground in the typographic compositions employing the alteration of letterforms between the iconic and the conceptual.

Through the interpretation of the process of drawing and composing the intuitive (iconic) and the preconceived (conceptual) aspects of image generation have been addressed. Even though drawing is usually related to the idea of “creatio ex nihilo” – and thus to the iconic – and collaging to figuration of existing elements in language, both processes depend on the schematic record of the designer and the beholder. Both processes have the potential for creating the unseen if the intuitive and the conceptual complement each other.

The continuous visual innovation design strived toward is only possible in the fruitful combination of the two aspects of thinking in images and thinking with words.

Finally the relationship between word and image was addressed in the context of diagrammatic images. By complementing each other words and images can form a constellation that provides a unique insight into accumulated data. Another meeting point between words and images can be found if letters show for example a specific spoken sound.

After all these aspects you may ask, what does it help to reflect on this topic as demonstrated above? If we think of the design process as a sequence of phases consisting of (1) developing the conceptual framework of the design, (2) physically creating material variations, and (3) the analysis of the variations, we can summarize that the conceptual knowledge about the issues are not helping in the creating sketches in phase 2. It might even inhibit the process of image generation if we focus on conceptual thought in this phase. But in the phases 1 and 3 we can use slow thinking and direct the fast thinking in phase 2 to other results (Kahnemann 2012). We can conclude that these two modes of thinking and the awareness of their specific potential in the above distinguished design process phases, is key to discover innovative solutions at least in the context of visual communication.

With one last example of my recent work, I would like to emphasize the idea of a middle ground between word and image [55]. The poster shown here announces the thesis exhibition at the Basel School 15

Figure 51: Ideal combination of Word, and Image, the process can be reconstructed at a glance, numbers indicate length of position at indicated point, the starting point is 54.

Marie-Louise Greb, MA Thesis, 2013.

Figure 52: Transforming the continuous system to a disjunct system creates a result without meaning, Marie-Louise Greb, MA Thesis, 2013.

Figure 53: Sketches to Word and Sound, Ondrej Jelinek, MA Thesis, 2013.

Figure 54: Word and Sound, Posters for a Sound Poetry Festival, Ondrej Jelinek, MA Thesis, 2013.

Figure 55: Poster for theThesis Exhibition HGK FHNW, 2015.

Figure 56: Campus of the Arts Basel during the Thesis Exhibition of the HGK FHNW, 2015.

Word and Image – An eternal Theme from a Designer’s Perspective

Michael Renner

of Design HGK FHNW 2015. The circle is a design element which guides the viewers gaze to the activity of the people in the photograph and is also the letter “O” of the word “DIPLOM”. I might have done this variation which emerged from a broad spectrum of design directions without the above elaborated aspects. But I would not have recognized, without the analytical reflection described in this lecture, the fruitful power of addressing the reading of a letter and the image-schema of focus and perfection with the circle, among all the other possibilities. The broad acceptance of this basic idea has led to the use of the circle by the organizers of the exhibition. They mounted the circle on the new building of the HGK FHNW in which the exhibition took place [56].

The following poster series for the inauguration of the new campus of the HGK FHNW show other variations of Word and image relations which could be discussed in detail [57, 58].

References

Thank you for your attention…

Thank you Juo-Yu Chang and Joe Chang for organizing this lecture today and thank you Juo-Yu Chang and Evelin Tu for the translation into Chinese.

Taipei Lecture

Bal (1991). Mieke Bal, Reading “Rembrandt”: Beyond the Word-Image Opposition, Amsterdam 1991.

Barthes (1964). Roland Barthes, The rhetoric of the image, Oxford 2007 (1964).

Boehm (1994), Gottfried Boehm, Die Rückkehr der Bilder, in: Gottfried Boehm, Was ist ein Bild, Paterborn, 1995, p. 13.

Damasio (1999). Antonio Damasio, The Feeling of What Happens. Body and Emotion in the Making of Consciousness, Orlando 1999, pp. 331 – 335.

Damasio (2010). Antonio Damasio, Self Comes to Mind. Constructing the Conscious Brain, New York 2010, p. 151.

Dewey (1934). John Dewey, Art as Experience, New York 1934, pp. 35 – 56.

Gadamer (1992). Hans-Georg Gadamer, Wort und Bild – “so wahr, so seiend”, in: Hans-Georg Gadamer, Lesebuch, Munich 1997, pp. 172 – 198.

Goodman (1976). Nelson Goodman, The Languages of Art: An Approach to a Theory of Symbols, Indianapolis/ Cambridge 1976.

Heidegger (1961). Martin Heidegger, Nietzsche’s Overturning of Platonism, in: Christopher Kul-Want (ed.), Philosophers on Art from Kant to the Postmodernists, A Critical Reader, New York 2010, pp. 121 – 131.

Heidegger (1929). Martin Heidegger, Kant und das Problem der Metaphysik, in: Vittorio Klostermann: Martin Heidegger, Gesamtausgabe Band 5, Frankfurt am Main 2010, § 19 – 21, pp. 90 – 101.

Johnson (2007). Mark Johnson, The Meaning of the Body. Aesthetics of the Human Mind, Chicago 2007, pp. 137 – 144.

Kahnemann (2011). Daniel Kahnemann, Thinking, Fast and Slow, New York 2012.

Kant (1781/1787). Immanuel Kant, Kritik der reinen Vernunft, Stuttgart, 1986.

Krämer et al. (2012). Sybille Krämer, Eva Cancik-Kirschbaum, Rainer Totzke (eds.), Schriftbildlichkeit; Wahrnehmbarkeit, Materialität und Operativität von Notationen, Berlin 2012.

Lakoff/Johnson (1980). George Lakoff and Mark Johnson, Metaphors We Live By, Chicago 1980.

Lakoff/Johnson (1999). George Lakoff and Mark Johnson, Philosophy in the Flesh. The Embodied Mind and Its Challenge to Western Thought, New York 1999, pp. 508 – 509.

Lessing (1766). Gotthold Ephraim Lessing, Laokoon, Über die Grenzen der Malerei und Poesie, Stuttgart 1986.

Lyotard (1997). Jean-François Lyotard, The Paradox on the Graphic Artist, in: Jean François Lyotard, Postmodern Fables, Minnesota 1997, pp. 33 – 48.

Mersch (2006). Dieter Mersch, Medialität und Kreativität. Zur Frage der Künstlerischen Produktivität, in: Bernd Hüppauf/Christian Wulf (eds.), Bild und Einbildungskraft, Munich 2006, pp. 79 – 91.

Mitchell (1986). Thomas W. J. Mitchell, Iconology: Image, Text, Ideology, Chicago 1986.

Mitchell (1995). Thomas W. J. Mitchell, Picture Theory, Essays on Verbal and Visual Representation, Chicago 1995.

Nietzsche (1888). Friedrich Nietzsche, How the ‘True World’ Finally Became a Fable: The History of an Error, in: Christopher Kul-Want (ed.), Philosophers on Art from Kant to the Postmodernists, A Critical Reader, New York 2010, pp. 63 – 64.

Plato (360 B.C.E.). Plato, The Republic, Book VII, translated by Benjamin Jowett, http://classics.mit.edu/Plato/republic.8.vii.html (January 10th, 2015).

Renner (2011). Michael Renner, Die stumme Bildkritik des Entwurfs – The mute Iconic Criticism of Design, in: Rheinsprung 11. Zeitschrift für Bildkritik, Basel 2011, pp. 92 – 116.

Renner (2010). Michael Renner, Practice-Led Iconic Research, in: diid, disegno industriale industrial designRdesignpress, Rom 2010, Nummer 41, pp. 76 – 82.

Figure 57: Poster exhibition in the lobby of the HGK FHNW, 2015.

Figure 58: Posters series for the inauguration of the Campus of the Arts Basel, 2015.

16

17