CH28 - The Encyclopedia of Human-Computer Interaction

1955

Chapter

28

Phenomenology

by Shaun Gallagher.

P

henomenology begins as a rigorous and systematic study of consciousness in the work of Edmund Husserl at the start of the 20th century. Phenomenologi-

cal research has direct relevance to design since the artifacts, tools, and technologies that we make affect the way that we experience our surroundings, and this is what phenomenology studies. Emphasizing the important role that embodiment plays in perception and cognition, it investigates, among other things, affective, aesthetic, and action-oriented experience as it is informed by environmental factors and by actual and potential bodily movement. Phenomenology explores the ways that our physical and social environments, including the things and instruments in such environments matter for experience, cognition, problem solving, and for shaping our intersubjective and social interactions.

1956

Encyclopedia of Human-Computer Interaction

Figure 28.1: Edmund Husserl.

Copyright: pd (Public Domain (information that is common property and contains no original authorship)).

In Husserl’s view phenomenology is a project aimed at describing the basic structures of consciousness, that is, those features of consciousness that shape the way that various objects in the world appear to us. Towards the end of the 1920s Martin Heidegger, who had worked with Husserl, offered his own somewhat different characterization of phenomenology. For Heidegger the aim of phenomenology was to provide a basic ontological analysis of human existence as ‘being-in-the-world’. An important part of his analysis showed that our primary way of existing involves a pragmatic action-oriented way of being related to our environment. A third development in phenomenology can be seen in the emphasis placed on embodiment in the work of the French philosopher Maurice Merleau-Ponty. Merleau-Ponty pursued several insights offered by Husserl and

Phenomenology

1957

furthered the analysis of embodied perception by integrating psychology and neuroscience with phenomenology and laying the groundwork for some contemporary approaches to embodied cognition. By no means do these three aspects or developments in phenomenology capture the entire scope in this field, but they do signal three areas that are directly relevant to design. Accordingly, I’ll organize my account of phenomenology by focusing on these areas (See, Gallagher 2012a for a more extensive account).

28.1 Consciousness is about something It is obvious that the design of objects in the built environment will have some aesthetic effects on the way we experience the surrounding world. Such aesthetic effects may range from vague and perhaps mostly unnoticed feelings that are difficult to express, to a clearly delineated pleasure or pain tied to the way that a certain object or configuration of objects appears. For Husserl this experience can be described as involving a variety of factors.

One of the first phenomenological principles is that all consciousness is characterized by intentionality, which simply means that one’s consciousness is directed at something, or is about something. Perceptual consciousness, for example, always has an intentional object. Consciousness is always consciousnessof-something. When I see, I see something, some state of affairs, event, or entity.

The intentional structure of consciousness can be analyzed into two aspects, which Husserl calls, referencing Greek philosophical terms, the noesis and the noema.

The noesis includes the particular character of the mental act, that is, whether the conscious act is perceptual, or is a case of memory, judgment, desire, etc., or involves some combination of these. I can see the apple on the table in front of me; or I can remember it being there; or I can judge that the apple will taste good, or I can desire to eat the apple, etc. In each case the intentional object, the apple, remains the same, but my consciousness is different. The noesis can be complex. I may simulta-

1958

Encyclopedia of Human-Computer Interaction

neously see, judge and desire, and each element in this complex act may modulate the other. The way I see something may lead me to judge it in a certain way, and this may lead me to desire it or not. It may also be the case that my desiring of something makes me see it in a certain way. In some cases my complex noetic state may be fulfilled when I pick up the apple and take a bite; or it may be disappointed if the apple doesn’t taste right. Likewise, I may imagine that something has a certain appearance, and that intuition may be fulfilled (or unfulfilled) by my subsequent perceptual experience of it. Various cognitive (noetic) processes may shape my perception or vice versa. One can thus develop a rich description on this noetic side of consciousness.

The way the object appears to conscious perception (judgment, memory, etc.) is what Husserl calls the noema. This is the ‘something’ as it appears in my experience. I see the apple, for example, from a certain angle. The apple is presented in a certain profile. I can pick it up and manipulate it in my hand, turning it, seeing one side and then another. All of these manipulations do not change the apple itself; rather, they result in noematic modulations. There is a noematic nucleus — in each case I am seeing the apple, and that has a certain meaning or sense; but the appearances of the apple change as I manipulate it. The full noema includes these actual changes and any other possible changes that might happen, for example, if I took a bite of the apple. Changes in what I am tasting, how the apple feels to touch, and the visual appearance of the apple all enter into perceptual noematic modulations. The noema may also be conceptual, if, for example, I am making a judgment about something.

Husserl further distinguishes between the inner and outer horizons of the object — all such aspects showing up in noematic variations. The inner horizon of the object includes the way the object can change appearance, for example, as I manipulate it, or move around it, or as the light hits it differently. The outer horizon includes its relations to other things in the nearby environment, those things that constitute the spatial (or Gestalt) background of the object, its spatial context.

Phenomenology

1959

All such noematic aspects are in correlation with noetic aspects. Certain noematic correlates appear only when I taste as opposed to see something; or only when I see something from a certain angle and in a certain profile. Moreover, although much of what I have just said pertains to perception, it also pertains to other noetic act qualities. For example, if I am thinking about (but not perceiving) something in order to make a judgment about it, what counts as its inner or outer horizon may in fact be certain conceptual rather than perceptual relations. Thus in thinking about an apple and judging its aesthetic appeal, I may be comparing it to an orange or a pear or a lemon. More abstractly, when thinking of the concept of justice, I may be doing so by contrasting a conservative/aristocratic concept with a democratic/liberal concept of justice. Here, the various inner or outer noematic horizons of the concept of justice are conceptual.

28.2 Consciousness is phenomenal Such considerations are part of an intentional analysis of consciousness. Consciousness also involves a certain affective feel. In contemporary discussions in the field of consciousness studies this is sometimes referred to as the phenomenal aspect of consciousness or the ‘what it is like’ aspect. There is not only an intentional difference, but also a phenomenal difference between tasting an apple and tasting a lemon. One has a different aesthetic experience in these two tastes. In the same way the color of an environment can be experienced as warm or cold; the surfaces as sharp or soft and what comes along with (and in) such experiences is a certain phenomenal feel — perhaps a feeling of comfort or discomfort. In his work Thing and Space, Husserl also develops the idea that our perceptual experiences of environmental objects have a kinaesthetic value. The artifacts around elicit certain kinds of movements by us, and we feel them in our bodily preparations for such movements. That is, they affect our bodily attunement. This is an aspect of Husserl’s phenomenology that Merleau-Ponty develops, and I’ll return to this point later.

1960

Encyclopedia of Human-Computer Interaction

28.3 Consciousness has a temporal structure Not only does Husserl have much to say about intentional, spatial, and aesthetic aspects of experience; he also has a dev0eloped analysis of the temporal nature of consciousness. Experience is structured in such a way that it is not simply in the present, but includes an anticipation of what is about to happen, and a retention of what has just happened. Consciousness, on Husserl’s account, has a three-fold structure of retention, primal impression, and protention. The retentional aspect at any moment of consciousness provides a sense of what I have just experienced; primal impression is directed at the present moment; and protention is oriented towards what may happen in the next moment.

Retention and protention involve a double intentional structure (see Figure 28.2). Retention, for example, is primarily directed at the previous phase of consciousness — the just past phase of consciousness that I have just lived through.

This just past experience contains in itself a retentional element directed at the just previous phase of consciousness which again includes a further retention, etc. etc. Thus my current awareness includes a sense of the flowing or fading past of my experience, built up in a train of retentions, and this provides me with a sense of self-connectedness. I don’t spring into existence anew in every moment, but I experience myself as extended over time. If, for example, I am listening to a melody, I have an immediate and lived sense or feeling (which is not yet a recollective memory) that I have been engaged in this listening process over the course of the last several seconds or minutes. In addition, retention keeps open a second dimension. Since each past phase of consciousness that is retained also involves a primal impression of the object that is experienced I retain a sense of the continuity (or of something changing) in the object that I experience. If I am listening to a melody, then not only do I have a sense that I have been listening to it for some time, I also have a sense of how that melody has developed. Indeed, without this kind of retentional temporal structure, which is built into the perceptual act, we

Phenomenology

1961

would not be able to perceive melodies at all. We would perceive only the current moment of one note sounding, and then we would have to add something like a recollection of having heard the previous note. The noesis would have to be a complex combination of momentary perceptions supplemented by a succession of memories rather than an actual perception of a developing melody.

Figure 28.2:  A diagram of the temporal structure of consciousness. The horizontal line designates a series of sounded notes (C, D, E, F). The vertical lines indicates phases of consciousness, consisting of protentions (above the horizontal), primal impression (where horizontal and vertical meet), and retentions (below the horizontal).

The diagonal line (e.g. rC, rrC, rrrC) illustrates how a specific note (C) remains in consciousness in an ever-growing retentional train. (From Gallagher and Zahavi 2012).

Copyright status: Unknown (pending investigation). See section “Exceptions� in the copyright terms.

1962

Encyclopedia of Human-Computer Interaction

Although Husserl doesn’t apply this analysis to protention, it is clear that protention has a similar double intentionality. In protention I anticipate both (1) that I will continue to experience something, i.e., that it will be my continuing experience, and (2) that in such experience the object, e.g., the melody, will continue to appear in a similar or changing way. The protentional structure of consciousness provides an account of how I can be either unsurprised — if what I anticipate actually happens as anticipated — or surprised — if what I anticipate doesn’t happen in the way I expect it to. In listening to a melody my anticipation may thus be fulfilled as the familiar melody continues as expected, or disappointed if the musician suddenly hits a wrong note, or if the melody is suddenly interrupted by some other event.

This temporal structure of experience is what psychologists call “ecological,” which means that it delivers in an ongoing way both a coherent experience of the world, and simultaneously a self-experience of my own coherence as a subject.

With respect to objects in the environment, the coherence delivered by the temporal structure of experience is important if, for example, the sense of an object, including the aesthetic effect of the object, depends on experiencing it over time.

This pertains not only to temporally changing objects like melodies, but to temporally stable objects like physical things that are either stationary or moving in space. In many cases, for instance, I may have to manipulate or walk around the object, or wait for the object to move in a certain way to gain the full effect of its appearance. For any such object to appear in a coherent way over time, my experience of it must involve the retentional-protentional structure.

Phenomenology

1963

28.4 From Dasein to Design

Figure 28.3: Martin Heidegger.

Copyright: pd (Public Domain (information that is common property and contains no original authorship)).

One of Heidegger’s most important insights is that our primary stance towards the world is more like a pragmatic engagement with it than like a detached observation of it. We don’t simply open our eyes and look at the objects around us; rather, we are more inclined to grab things and use them. Even if we are just looking, we look at things in terms of their relevance for our pragmatic use. Our intentionality is shaped by this orientation to action. From this perspective the things around us appear as “ready-to-hand” (Zuhanden). Heidegger drives this home with the example of a hammer. For the carpenter who is proficient in her use of the hammer, the hammer is not something that she contemplates or theorizes about; it is rather something that she picks up and uses, and in doing so, the hammer is such that it becomes experientially transparent — that is, it is barely noticeable as it is assumed into a complex involvement in the project of the carpenter. The hammer as such has the status of an instrument, tool, or piece of equipment that serves to

1964

Encyclopedia of Human-Computer Interaction

support my action or project. More generally, I approach objects, I am in relation to the world, by way of my projects. Heidegger suggests that the hammer takes on a different status if for some reason it breaks, or if it is improperly designed and difficult to use. Suddenly the hammer is an object to be thought about, a problem to be fixed — it becomes something “present-at-hand” (Vorhanden) which gets in the way of my project, or prevents me from moving forward towards my goal.

According to Heidegger, philosophers and scientists have generally, and mistakenly, developed their explanations of the world by taking things as primarily present-at-hand — that is, by treating things as objects to be observed from a detached, objective perspective. Indeed, this sort of objective thinking gets applied not only to things in the environment, but to human agents as if their primary way of existing were to be present-at-hand things; and to Being itself, as if Being itself were an entity, or thing, or a particular being. The human being (or human existence, Dasein) is precisely not a thing. It has a different kind of existence altogether and is primarily an agent, action-oriented, and that, at least in part, defines its existence.

One can get a sense of how this ontological difference, not only between Being and particular entities, but also between objects or things and human agents, has been misconstrued in the history of philosophy and science by looking at Aristotle’s analysis of human existence. In the Physics, Aristotle is concerned to provide the proper ontological categories for understanding physical things. In this respect he offers his famous hylomorphic theory. That is, he suggests that every physical entity is composed of matter (hyle) and form (morphe). The latter in the first instance means shape. If we take a particular material and give it a certain shape, we also give it a certain function and make it what it is. The design of an automobile, for example, must meet certain requirements, both materially and morphologically, in order to function as an automobile. The form, the function, and the thing’s “whatness” (eidos, essence) define it as such. Aristotle explicates his analysis in terms of the four causes: the material and formal causes, as just defined, and the efficient and final causes. With regard to an automobile, for ex-

Phenomenology

1965

ample, an explanation of the material that goes into it, and its functional shape, plus an account of how these things came together (e.g., a manufacturing process) which is the efficient cause, plus an account of why anyone would want to build an automobile, or what its purpose is, which is the final cause, would provide its complete explanation.

All of this is well and good for explaining physical things and doing physics, at least of the sort that Aristotle envisioned. Aristotle, however, thought this kind of explanation was also useful for explaining other things. Thus, in his Metaphysics, he used precisely the same categories, specifically form and matter, to explain being qua being; and in his Psychology he used precisely these same categories, derived from his analysis of physical things, to define the human agent. This, from Heidegger’s perspective, was a mistake. To use explanatory categories from physics to explain Being itself is to reduce Being to a thing-like entity and to ignore the ontological difference between Being and particular entities. And to use explanatory categories from physics to explain human existence is to reduce the human being to an object and, at the very least, to ignore its peculiar kind of agency or its particular way of being-in-the-world.

Dasein, the human being, has the following kind of existence, which is different from any other kind of being. The human being is characterized as always finding itself already in existence, discovering itself already in specific circumstances and moods (an existential characteristic which Heidegger calls Befindlichkeit, disposition, or thrownness). Another way of saying this is that human agents always finds themselves situated. Furthermore, humans are always projecting meaning — they always finds themselves in a world that they are constantly trying to interpret (a characteristic which Heidegger calls Verstehen or understanding).

Human existence is also always intersubjective — humans are always influenced by their encounters with others, in such a way that part of their existential nature is to be-with others (Mitsein or being-with). In these various ways human existence is differentiated from the kind of being that things have. Setting aside the question of whether non-human animals can have this kind of existence, Hei-

1966

Encyclopedia of Human-Computer Interaction

degger clearly thinks that non-human physical things of the sort that we encounter in the environment do not have dispositions, are not attempting to understand, and are not interacting with others in an intersubjective way. The meaning of things, and specifically whether they have a mode of being that is ready-tohand, like the hammer-in-use, or present-at-hand, like the broken hammer, depend on the fact that humans give them meaning within the context of various projects, most of which arise through intersubjective and social processes.

28.5  From hammers to wine glasses

Figure 28.4:  Jean-Paul Sartre in 1950.

Copyright: pd (Public Domain (information that is common property and contains no original authorship)).

Jean-Paul Sartre, who was influenced by both Husserl and Heidegger, once remarked that with phenomenology one can philosophize about a wine glass. That is, one can begin to think about the purpose of a wine glass within the context of human projects. Why the wine glass is designed in the specific way that it is designed may have something to do with function or style, but it is clear that it is

Phenomenology

1967

designed for humans and by humans. Like Heidegger, Sartre suggested that we encounter not only the specifics of our own existence in our products and how we use them, but we encounter others in such products. We encounter them implicitly as producers and as organizers of the means of production, but we also encounter them in the utility provided by such products. We encounter others in the practices that employ such objects. In this respect, viewing an object from both a present-at-hand perspective and a ready-to-hand perspective is essential for design. That is, a designer must treat her product as something Vorhanden — as something that is shaped in a certain material that facilitates function — but also as something Zuhanden — something best suited to the use of human agents.

This leads us directly to considerations about the human body since it is normally with a human hand that we reach out to grasp a wine glass, and it is only with a human mouth that we take the sip.

28.6 Embodied and enactive design

Figure 28.5: Maurice Merleau-Ponty.

Courtesy of Pierre-Alain Gouanvic. Copyright: CC-Att-3 (Creative Commons Attribution 3.0 Unported).

1968

Encyclopedia of Human-Computer Interaction

In the work of Merleau-Ponty we find greater emphasis on the body than in either Husserl or Heidegger. In contrast to the traditional denigration of the body found in western philosophy from Plato to Descartes, and in certain forms of gnostic religions, where mind, or soul, or spirit dominates, Merleau-Ponty treats the body as the perceiver, the knower, the agent. Intentionality is primarily a motor intentionality. We make sense of the world through our bodily actions. Indeed, what remains implicit in Heidegger becomes explicit for Merleau-Ponty, namely, that our primary relation to the world, as pragmatic, and as ready-to-hand, is primarily an embodied relation where our hands and motor systems necessarily play an essential role.

Merleau-Ponty takes over the basic phenomenological distinction between lived body (Leib) and objective body (Körper) from Husserl. Recently this distinction has been phrased as the body-as-subject (or we might say as-agent) vs the body-as-object. We engage with the world, and we encounter others intersubjectivly, for the most part as embodied agents rather than as passive observers or detached minds. Our bodies are not primarily objects — either for ourselves or for others — but agentive bodies that express their subjectivity in their postures, movements, gestures, actions, and expressions. My body as perceiver and agent is in-the-world, engaged in projects, and participating with others in the making of meaning.

Merleau-Ponty has inspired recent work on enactive and extended conceptions of cognition. On the enactive view, our perception of the world is essentially linked to our movements and our motor possibilities (Varela et al, 1991). Perception is pragmatic in the way indicated by Heidegger, and it’s not just a matter of the brain processing information or representing objects in the environment on a kind of internal map. Our perception is based on specific sensory-motor contingencies (see O’Regan and Noë 2001). I see and understand an object in terms of what I can do with it. In this regard Merleau-Ponty exploits Husserl’s insights about the kinaesthetic dimension associated with perception. My perception is

Phenomenology

1969

informed by my ability (or lack of ability) to move. Husserl and Merleau-Ponty also talk about the “I can” intentionality of perception. The fact that an object appears in the reachable (peripersonal) part of the environment where I can reach it, for example, as opposed to an unreachable (extrapersonal) part of the environment has an influence on its relevance, its valence, and how I perceive it. This phenomenological insight has been confirmed by neuroscientific studies that show different activation patterns under these different conditions (see e.g.

Caggiano et al. 2009). Our pragmatic, action-oriented way of being-in-the-world is also reflected in the activation of “canonical” neurons that are activated both when we reach out to grab a tool or object, and simply when we perceive that tool or object (Grezès et al. 2003). We perceive the world in terms of the possibilities for action that it offers.

Accordingly, perception is guided by what J. J. Gibson has called ‘affordances’. For example, I see a chair as affording me the possibility of sitting. That clearly depends on the way it is designed and how that design is related to my body. An object that fails to provide a flat horizontal surface doesn’t afford sitting; nor does a chair that is three inches tall. A well-designed chair is not a sitting-affordance at all for a lion or for an animal or robot that does not have flexible joints. Affordances depend on specific relations or negotiations between objective shape and the shape and flexibility of the agentive body.

28.7 Extending the mind into things Like the enactive view, the extended mind hypothesis (Clark and Chalmers 1998; Clark 2008) argues that cognition is not simply “in the head”, but depends on extra-neural bodily and environmental elements. Cognition extends to processes in the environment, to processes involved in our use of technical instruments that support our cognitive activities. Our handy use of pencil and paper not only facilitates our calculations, it can, in part, constitute our cogni-

1970

Encyclopedia of Human-Computer Interaction

tive process. My memory can supervene on the use of a PDA as much as it can on my hippocampus; my solutions to navigational problems can incorporate the use of GPS; my correct spelling can be the result not only of my biological memory but also of my use of a spell check feature on my laptop. There are debates within the philosophical community about how far we can take such claims about the extended nature of cognition, but most theorists accept that tools and technologies can facilitate our cognitive processes. This also applies to the way that we design our environments. The way that we set out instruments in an operating room can facilitate the performance of the surgeon; the way we design a cockpit can facilitate the decisions of the pilot; the way we furnish a classroom can support learning. All such design details can also work the other way. That is, our cognitive tasks can be thwarted, our imagination stifled, our problem solving slowed, our judgments clouded when our environment is ill designed, just as much as when our tools and technologies start to interfere with our thinking.

One essential aspect of both enactive and extended conceptions of cognition is a specification of how precisely we are coupled with tools, technologies, and environments. That is, the cognitive process is in some cases constituted and in some cases simply facilitated depending on the nature of the body-environment coupling. Once again this will depend both on various bodily habits, the condition of the body, and the nature of the bodily practices engaged, and on the arrangement and condition of the environment. These aspects are measurable and explainable in terms of dynamic systems theory (see, e.g., Chiel and Beer 1997).

They can also be explored experimentally using methods of neurophenomenology (Varela 1996).

Let me mention that much of what I have just indicated about perception and cognition can be repeated in regard to intersubjective processes and social arrangements. For phenomenology following Merleau-Ponty my ability to understand another person involves an embodied set of processes that are sensory-mo-

Phenomenology

1971

tor and contextual rather than primarily inferential or simulational. Specifically, I am not primarily a third-person observer of others; rather, I engage and interact with them in socially defined contexts. My understanding of others is also facilitated by the kind of environments that we find ourselves in, the kind of interactions that such environments (including artifacts, tools and technologies) allow.

Environments can be built in ways that can either promote or prevent communication, and in this sense physical environments are also social environments and have direct relevance for intersubjective interactions.

28.8  The design of intelligent systems In the 1970s Hubert Dreyfus relied on phenomenological insights from Heidegger and Merleau-Ponty to develop a critique of strong AI — that is, the kind of artificial intelligence projects that aimed to construct machines with human-like intelligence. Dreyfus presented a general critique of computational models of cognition. Thinking of the brain as a computer that simply processes information in discrete operations not only misses a number of important aspects of human cognition; it entirely misconstrues the nature of cognition. For example, it misses the Heideggerian and enactive conception that cognition is for action and that the human is pragmatically involved in the world. Important for this view is the fact that the world offers highly complex and ambiguous situations that require a kind of interpretation that is not possible if one has only a set of rules to follow. If AI can create machines that function in extremely circumscribed contexts where a machine or robot can do what they do by following a set of formal rules, this kind of intelligence generally does not transfer to other contexts, and especially changing contexts where one needs to recognize without rules what kind of situation one is confronted with. These kinds of problems typically are related to the frame problem in AI. Dreyfus’s position continues to be debated in recent books and articles (see Wheeler 2005; McDowell 2007).

1972

Encyclopedia of Human-Computer Interaction

Dreyfus also appealed to Merleau-Ponty’s work on embodiment, and this has special reference to robotics. If in fact cognition is embodied from the bottom up, then the design of robots with a set of top-down executive controls will have limited success in performing pragmatic tasks in ambiguous situations or in interacting with human agents. This idea motivated Rodney Brooks (1991) to re-think AI and robotics to address basic sensory-motor problems, and to attempt to build intelligence from the bottom up. For Dreyfus, to design a robot with human-like intelligence would require the robotic body to be human-like, not just in appearance, or in its ability to pragmatically engage in the world, but to have bodies more or less like ours, with the possibility of social interaction and acculturation.

28.9  You and I, robot Researchers in advanced robotics are attempting to design and build autonomous social robots that will be able to seamlessly and reliably interact with humans in specific situations. A huge amount of research funding is currently being invested in human-robotic interaction (HRI). There has been some progress in building robots that can learn to respond to (and with) certain emotional expressions, and to respond to (and with) specific gestures. There are still serious limitations in designing robots that can operate in the kind of ambiguous situations that typify human projects and interactions (see Gallagher 2007; Gallagher 2012b). This motivates both philosophical and practical questions about what precisely is required in a robot if it is to be able to engage in something close to the kind of interaction that characterizes human-human relations, even if only on a pragmatic level.

Dreyfus is right to suggest that we need to have a correct understanding of human cognition if we want to design intelligent systems. We also need a good understanding of human-human interaction if we want to design a robot capable of interacting with humans in smooth and problem-free ways. This means that

Phenomenology

1973

we need the right theory of social cognition, and this is another area in which phenomenology has been challenging received doctrines and offering its own constructive account.

In contrast to the standard “theory-of-mind” (ToM) accounts of social cognition, which focus on “mindreading” construed as observation-based inference or the running of simulation routines, phenomenological approaches to social cognition emphasize the embodied and enactive aspects of our interactions with others. Our understanding of others is based on sensory-motor, perceptual and interactional abilities in rich and meaningful (and, in most cases, socially defined) contexts that are also defined by artifacts, tools, and technologies. In most everyday situations, I can get a sense of what you want to do in the way you are doing it. That is, I don’t have to attempt to infer or simulate your mental states. In many cases your intention is obvious to me because I am not simply observing your behavior in an off-line (third-person) mode; rather I am engaged with you in an on-line (second-person) interaction. Moreover, as research in developmental science and phenomenology suggests, in our interactions I understand the meanings of your actions (and gestures and expressions, etc.), enactively, that is, in terms of social affordances, in terms of my possible responses to your actions. In many cases, also, the particular situation (the physical setting, the social environment) does some of the work. The meaning of a certain gesture or a certain action is specified by the social or physical situation in which it is enacted; the meaning of the same set of movements may be specified differently in a different situation.

In regard to designing autonomous robots that are capable of genuine social interaction, standard ToM theories suggest that we need only create specific mechanisms inside the individual robot that will provide it with the ability to infer, simulate, or mindread. In contrast, phenomenologically inspired interaction theory claims that in many circumstances interaction itself (the dynamic, enactive engagement of two or more individuals which is not reducible to the actions of the

1974

Encyclopedia of Human-Computer Interaction

individuals qua individuals) constitutes social cognition (de Jaegher et al 2010).

Just as in dancing the tango, the emergent dynamic phenomenon, the tango itself, is something more than an addition of individual contributions. In social contexts, as in dance contexts, and as predicted by the phenomenological analysis of the temporality of experience, the timing specifics of interactions are essential.

This suggests that even if a workable social robot were designed on standard ToM principles to support inferential or simulative mindreading, for it to be capable of smooth and reliable interaction with humans it would already have to be a robot that behaves sufficiently like us so as to engage in embodied interaction.

If this is true, then at a minimum we need to design robots that have similar sensory-motor abilities and that can recognize the significance of the same kind of rich and meaningful contexts in which humans act, rather than machines capable of more precise inferential calculations or abilities to run more powerful simulations. In effect, social understanding has to be already implicit in the robot’s actions and in its abilities to see others in terms of how they can interact with them (i.e., as social affordances).

This approach places extraordinary demands on robotic design. Is it even possible to build such a robot? Not only should we not rule anything out apriori, ongoing research in evolutionary robotics provides minimal instances of such interactive machines. Di Paolo (2000), for example, evolved agents that were able to achieve coordination through interaction. Two robots, whose only task was to locate each other and remain close as they moved through a large space, used simple auditory signals and rotating motor behavior, to set up a specific sound pattern that differentiated between self and non-self, and simplified what would otherwise be a complex recognition problem.

Phenomenology

1975

Courtesy of Ezequiel Di Paolo. Copyright: CC-Att-SA-3 (Creative Commons Attribution-ShareAlike 3.0).

1976

Encyclopedia of Human-Computer Interaction

Courtesy of Ezequiel Di Paolo. Copyright: CC-Att-SA-3 (Creative Commons Attribution-ShareAlike 3.0).

Figure 28.6 A-B: Left: Trajectories of approaching agents. Right: Motion of agents during period of coordination. From Di Paolo 2000.

Importantly, when an individual agent was presented with a recording of its partner from the previous successful interaction, that is, when it was presented with precisely the same data that it had previously processed in an interactive way, it was unable to reproduce its own behavior due to the fact that the other robot’s recorded actions and “responses” were non-contingent and mis-timed. One-sided coordination is not achievable, which suggests the important contribution of interaction itself. Similar results have been found in other experiments involving artificial agents (Auvray et al 2009; Di Paolo et al 2008). Such experiments in

Phenomenology

1977

evolutionary robotics can supplement phenomenological methodology in order to identify the enactive principles involved in minimal social interaction (see Froese and Gallagher 2010; Froese and Gallagher 2012). The challenge is to see if these principles scale up to the kind of non-minimal behaviors that will characterize full-out human-robotic interactions in worldly environments.

28.10 Conclusion Although phenomenology originates as a philosophical enterprise, it has practical application in a number of fields, including design and human-robotic interaction. As a study of our intentional, temporal, and lived experience it is directly relevant to design issues. It suggests that the intentional structure of consciousness as informed by bodily movement and kinaesthetic sensation will shape our phenomenal and aesthetic experiences. It points to our action-oriented way of being-in-the-world as determining how we experience the things around us, and it emphasizes the important role that embodiment plays in perception and cognition. Phenomenology explores the ways that our physical and social environments, including the things and instruments in such environments matter for experience, cognition, problem solving, and for shaping our intersubjective and social interactions.

28.11 Acknowledgements My work on this chapter was s0upported by a Research Fellowship at the KollegForschergruppe Bildakt und Verkörperung at Humboldt University, Berlin (2012).

Also thanks to the ongoing support of the Marie Curie Initial Training Network, Towards an Embodied Science of Intersubjectivity (TESIS). Marie Curie Actions, European Commission Research for support of my research on intersubjectivity, and the Humboldt Foundation fellowship: The Anneliese Maier Research Award [Anneliese Maier-Forschungspreis] at the Ruhr-Universität Bochum (2012-17).

1978

Encyclopedia of Human-Computer Interaction

28.12 References Auvray, Malika, Lenay, Charles and Stewart, John (2009): Perceptual interactions in a minimalist virtual environment. In New Ideas in Psychology, 27 (1) pp. 32-47 Brooks, Rodney A. (1991): Intelligence without representation. In Artificial Intelligence, 47 (1811) pp. 139-159 Caggiano, Vittorio, Fogassi, Leonardo, Rizzolatti, Giacomo, Thier, Peter and Casile, Antonino (2009): Mirror neurons differentially encode the peripersonal and extrapersonal space of monkeys. In Science, 324 (5925) pp. 403-406 Chiel, Hillel J. and Beer, Randall D. (1997): The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment. In Trends in Neurosciences, 20 (12) pp. 553-557 Clark, Andy (2008): Supersizing the Mind: Embodiment, Action, and Cognitive Extension (Philosophy of Mind). Oxford University Press, USA Clark, Andy and Chalmers, David J. (1998): The Extended Mind. In Analysis, 58 (1) pp. 7-19 Dreyfus, Hubert L. (2007): Why Heideggerian AI failed and how fixing it would require making it more Heideggerian. In Artificial Intelligence, 171 (18) pp. 1137-1160 Dreyfus, Hubert L. (2002): “Intelligence without representation” Merleau-Ponty’s critique of mental representation The relevance of phenomenology to scientific explanation. In Phenomenology and the Cognitive Sciences, 1 (1962) pp. 367-383 Dreyfus, Hubert L. (1992): What Computers Still Can’t Do: A Critique of Artificial Reason.

The MIT Press Froese, Tom and Gallagher, Shaun (2010): Phenomenology and Artificial Life: Toward a Technological Supplementation of Phenomenological Methodology. In Husserl Studies, 26 (2) pp. 83-106 Gallagher, Shaun (2007): Social cognition and social robots. In Social Cognition, 3 (3) pp.

435-453 Gallagher, Shaun (2006): How the Body Shapes the Mind. Oxford University Press, USA Gallagher, Shaun and Zahavi, Dan (2012): The Phenomenological Mind: An Introduction to Philosophy of Mind and Cognitive Science, 2nd edition. Routledge Gibson, James J. (1979): The Ecological Approach to Visual Perception. New Jersey, USA,

Phenomenology

1979

Lawrence Erlbaum Associates Heidegger, Martin (ed.) (1982a): The Basic Problems of Phenomenology, Trans. Albert Hofstadter. Indiana University Press Heidegger, Martin (1989): Ideas Pertaining to a Pure Phenomenology and to a Phenomenological Philosophy: Studies in Phenomenology of the Constitution. Transl. R. Rojcewicz and A. Schuwer. Springer Heidegger, Martin (ed.) (1982b): Ideas Pertaining to a Pure Phenomenology and to a Phenomenological Philosophy: First Book: General Introduction to a Pure Phenomenology.

Transl. F. Kersten. Springer Heidegger, Martin (1996): Being and Time (translated by Joan Stambaugh and Dennis J. Schmidt). State University of New York Press Jaegher, Hanne De, Paolo, Ezequiel A. Di and Gallagher, Shaun (2010): Can social interaction constitute social cognition?. In Trends in Cognitive Sciences, 14 (10) pp. 441-447 Mcdowell, John (2007): What Myth?. In Inquiry, 50 (4) pp. 338-351 Merleau-Ponty, Maurice (1962): Phenomenology of Perception: An Introduction. Routledge O’Regan, J. Kevin and NoÍ, Alva (2001): A sensorimotor account of vision and visual consciousness. In Behavioral and Brain Sciences, 24 (5) Paolo, Ezequiel A. Di (2000): Behavioural coordination structural congruence and entrainment in a simulation of acoustically coupled agents. In Adaptive Behavior, 8 (1) pp. 25-46 Paolo, Ezequiel A. Di, Rohde, Marieke and Iizuka, Hiroyuke (2008): Sensitivity to social contingency or stability of interaction? Modelling the dynamics of perceptual crossing. In New Ideas in Psychology, 26 (2) pp. 278-294 Varela, Francisco J. (1996): Neurophenomenology: A methodological remedy for the hard problem. In Journal of Consciousness Studies, 3 (4) pp. 330-349 Varela, Francisco J., Thompson, Evan and Rosch, Eleanor (1991): The Embodied Mind: Cognitive Science and Human Experience. Cambridge, Massachusetts, MIT Press Wheeler, Michael (2007): Reconstructing the Cognitive World: The Next Step. A Bradford Book Wheeler, Michael (2005): Reconstructing the Cognitive World: The Next Step (Bradford Books). The MIT Press

1980

Encyclopedia of Human-Computer Interaction

About the author Shaun Gallagher

© Shaun Gallagher

Personal Homepage: http://pegasus.cc.ucf.edu/~gallaghr Current place of employment: Department of Philosophy, University of Central Florida Shaun Gallagher is the Lillian and Morrie Moss Professor of Excellence in Philosophy at the University of Memphis. He has a secondary appointment at the University of Hertfordshire (UK) and is Honorary Professor of Philosophy at the University of Copenhagen (Denmark). He’s held visiting positions at the Cognition and Brain Science MRC Unit at the University of Cambridge, the Ecole Normale Supériure in Lyon, and the Centre de Recherche en Epistémologie Appliquée (CREA), Paris, and he is currently Visiting Fellow at the Kolleg-Forschergruppe Bildakt und Verkörperung at Humboldt University, Berlin. He holds the Humboldt Foundation Anneliese Maier Research Fellowship (2012-17) and is PI on grants

Phenomenology

1981

to conduct research on intersubjectivity and institutions (Marie Curie Foundation) and the aesthetic and spiritual experiences of astronauts during space travel (Templeton Foundation). His publications include How the Body Shapes the Mind (Oxford, 2005); The Phenomenological Mind (with Dan Zahavi, Routledge, 2008), and as editor, the Oxford Handbook of the Self (Oxford, 2011). He’s editorin-chief of the journal Phenomenology and the Cognitive Sciences.

Your notes and thoughts on chapter 28 Record your notes and thoughts on this chapter. If you want to share these thoughts with others online, go to the bottom of the page at: http://www.interaction-design.org/encyclopedia/phenomenology.html NOTES:

_______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________

1982

Encyclopedia of Human-Computer Interaction

_______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________ _______________________________________________________