GESPIN – GESTURE & SPEECH IN INTERACTION – Poznań, 24-26 September 2009
In search of a ‘gesture instinct’: The evidence from signed communciation systems Dorothea Cogill-Koez University of New England(s) Armidale, NSW, Australia. dcogill@une.edu.au
Abstract Homo sapiens’ communicative abilities have evolved in the context of species-wide abilities to see and to hear. Two major subsystems within our communication systems, gesture and speech, build respectively on each of these sensory abilities. The two subsystems do not differ only in modality, however; the ‘classics’ of gesture literature suggest that gesture on the one hand and the contemporary linguist’s conception of speech on the other also differ reasonably systematically in the principles of representation that predominate in each, and in the level of structure typically reached in each. This would seem to reinforce the claim of each to be a good natural class of behaviour. It might seem reasonable to assume, therefore, that speech and gesture are also primary communicative categories, supported by (in each case) some distinctive cognitive or communicative abilities that contribute uniquely to each; that there may be a ‘gesture instinct’ as there is argued to be a ‘language instinct’! Signed communication systems are an alternative product of our species’ communicative potential. If gesture is indeed a primary communicative category— if there is any kind of ‘gesture instinct’— it should be seen to emerge in signed communication as well. Sign language linguists have indeed sought parallels with speaker-hearer’s gesture in signed systems, and have identified a number of different possible analogues. However, each of these potential parallel systems are analogous to speaker-hearer gesture on only one of the multiple parameters that unite gesture in the latter’s communication system. In signed systems, quite simply, gesture as a bounded class of communicative behaviour falls apart. Notably, the parameters which enter into it do not change: it is, rather, their configuration that changes. As such, it is arguably the parameters themselves which emerge as primary potentialities of human communication. The effect of the comparison with signed systems is to suggest that what might, from a hearer’s perspective, appear as secondary features of gesture, may deserve greater attention as the parameters which not only distinguish gesture but also unite it with other subsystems of human communication.
1
What is gesture in speaker-hearer communication systems?
In recent years there has been considerable interest, in sign language research, in seeking an analogue of gesture within signed communication. Surprisingly, though, the sign literature lacks much discussion of what this thing called ‘gesture’ is. Turning to the speech-based literature, it might appear that the tendency is to describe, rather than to define. However, classics such as McNeill (1992) point to a nexus of factors that typify gesture; parameters, importantly, on which gesture contrasts with its sibling system, speech.
GESPIN proceedings, vol. I
Most obviously, gesture in speech-based communciation systems is distinguished by channel, the way in which the representations are physically embodied. Gesture is communication on the hands (if not also the face and body); speech is confined to the oral-aural channel. Additionally though, while speech carries (at least) representations composed according to what will here be dubbed ‘linguistic principles’, as captured in abstract rules of phonology, mophohology and syntax, gestures are described, in the bulk of the literature, as being formed according to principles that relate form to meaning by strategies other than the linguistic; strategies that themselves form the basis of many a gestural classification in the early gesture literature. They include, among many others, depiction or visual representation, the process of mapping between a representational form and a mental visual image to code or decode meaning, pointing, by which an object or location is indicated by forming a trajectory which, if followed with the eyes or the mind, extends out to the object or location, as well as the symbolic principle, a strategy by which a form is linked to its meaning by a cognitively direct association, such as the English speakers’ gestural symbol of ‘thumbs up’ for ‘good’ or ‘all is well’. As the many classification systems of gesture show, hearer’s gesture may use any of these three principles, and more, to link form and meaning. What does not appear in extant classifications of gesture is a class based on the ‘linguistic representational system’ with which the oral channel, in contrast, is flooded. This presents a second way in which gesture, as a subsystem of speakerhearers’ communication, is internally unified—albeit in only a negative sense— and stands in contrast to the subsystem of speech. A third parameter by which gesture appears to be distinguished from speech (at least in the ‘classics’ of the gesture literature) is structure. At least for most of its history, the literature on speaker-hearer’s gesture states that it is relatively unstructured. It seems accepted that gestures only rarely reach the level of discrete, standardised form. Most speech-accompanying gestures have long been charactierised as analogue in form —varying in form continually with their referent. Even while more structure is being detected in speech-accompanying gesture, it does not appear yet to be suggested that the level and extent of such structure ranks on a level with that found in the linguistic representations carried in the speech channel, where discrete forms predominate, and display discrete-combinatorial substructure, as well as combining into further higher structures themselves. Finally, the classics of gesture literature as well as its more recent focus all highlight a fourth factor which unifies gesture. It is a process of development by which the gestural subsystem comes to be integrated in certain, at least partly culture-specific ways, with the speech stream. One reads in the gesture literature that in very early childhood gesture first emerges as a stand-alone alternative to speech, alternating with it. Over time, though, and with some developmental stages, it comes to be produced simultaneously with speech and smoothly integrated with it, in patterned and at least partly culture-specific ways. Much of hearers’ use of gesture is unified in this fourth way. In speaker-hearers’ communication systems, therefore, gesture is not a synonym for ‘manual communication’. Hearers’ gesture is united in sharing certain realisations of other major parameters; cognitive principle of representation, and level of structure. The communicative subsystem of gesture, as a whole, also shows evidence of being developmentally integrated, to work smoothly with speakers’ other major representational subsystem, speech. In many ways, gesture appears able to lay claim to being a well-bounded, distinct class of communicative behaviour in a mature speaker-hearer’s communication system. As a result of this— and as a result of the fact that we are almost all speaker-hearers— there might be an expectation that gesture is also a genuinely basic category of human communicative behaviour; that gesture as a communicative subsystem is already ‘in’ us at birth, coded genetically into our communicative systems as a single category, just waiting to flower as we grow – much like the nativist’s vision of linguistic representation. If this is true, then even when humans are born deaf, and grow up signing, we should see gesture revealing itself as a unified subsystem in signed communication.
2
Dorothea Cogill-Koez: In search of a ‘gesture instinct’: The evidence from signed communciation system
2
Is gesture a natural class in signed communication systems?
Signe d language research had its birth in the discovery that in spite of their manual channel and superficially iconic appearance, the bulk of signed communication is produced in a way that is ‘form blind’. At least in the flow of online conversation, signers do not decode signs by detecting iconic resemblances, but by the cognitive principle of immediate, direct association of form and meaning. More, signers compose these signs, and combine them into higher structures, in a way that is not merely analagous but apparently identical to the lingusitic representational system that dominates speech in speaker-hearer communication. This identity extends all the way from details of formal structure through to biological underpinnings, breaking down with damage to lefthemisphere areas that support ‘speech’ – or more accurately now, linguistic representation— in speaker-hearers, and that is unaffected, conversely, by right-hemisphere damage that affects the signer’s ability to see and produce pictures (Stokoe 1960, Klima & Bellugi 1979, Poizner, Klima & Bellugi 1987). In the decades since its birth, signed language linguistics has focused mostly on exploring how these linguistic representations are realized in signing. Recently, however, and with the resurgence of interest in gesture within speech-based communication, there has been more interest in seeking some parallel of gesture in signing, as well. How, though, is such gesture to be identified?
2.1
Channel
Chronologically, the first transfer of the concept of gesture onto sign-based communication systems exported just the fact of channel or modality. This was the predominant usage of the term by many sign language linguists up until, and into, the 1990s, with signed languages being spoken of as ‘visual-gestural’ communication codes, as opposed to the ‘oral-aural’ codes of those with intact hearing. The definition of gesture implicit in this is that gesture is communication on the hands. However, the bulk of signed communication is carried on the hands. Manually-created forms do not stand in contrast to any other major communicative subsystem, in the same way that gesture does in speech-based systems. In speaking of sign as ‘gesture’, then, one is simply renaming almost the whole communication system. That re-naming does amount to a reconceptualisation which itself is not without value. Labelling signed communication as gestural potentially directs researchers, for example, to investigate the ways in which the manual modality shapes communication, across both signed and speech-based systems. Nevertheless, it is a re-naming which makes a poor choice of terms. It implicitly conflates gesture with just one of its distinctive parameters; channel. In its origins as a descriptor of a subsystem of speaker-hearer communication, though, ‘gesture’ refers to a complex of factors, not simply to channel alone. Here as later, it is argued that to choose the label for the entire subsystem, as a substitute for the label for a single parameter of that system — to use the term ‘gestural’ here, rather than the more specific term ‘manual’, when ‘manual’ is what is meant— is a poor strategy in theory-building. Connotations are inevitably imported that are unintended in the original terminological borrowing, and unjustified by the evidence on which the original borrowing was based. In short, if ‘manual’ is what is meant, better by far to use the term ‘manual’.
2.2
Principle of representation
What, then, of principle of representation? In searching for a product of a ‘gesture instinct’ in signed communication systems, are we looking for a communicative subsystem that is distinguished, at least negatively, by the cognitive strategies which link form and meaning: “in signing, gesture is all representation that codes meaning by other than linguistic principles”? This is another definition of gesture that has been tacitly assumed in more recent sign language literature. It seems likely to become a increasingly common one, as more researchers are exploring the possibility that signers, like speakers, use more than merely the linguistic system to actively encode meaning. For example, the author has argued that one distinct and problematic subclass of signing, the signed classifier predicate systems, are almost entirely generated by genuine use of
3
GESPIN proceedings, vol. I
manual depiction as an active principle for forming and decoding signs (Cogill-Koez 2003). Liddell (2000) has convincingly argued that signed deictics and pronouns contain a genuine use the trajectory — that cognitively, they are not full analogues of spoken pronouns and deictics, but include aspects of form coded and decoded as ‘pointing’. However, even if sign researchers allow the presence of other principles of representation into signed communication, is there anything that additionally justifies placing all such ‘nonlinguistic’ representations into one class in signed communication? In speech-based communication, this purely negative feature of gesture (“gesture is nonlinguistic representation”) gains gravitas from the fact that it maps onto other distinctive features that make up the positive face of gesture. In signing, points and manual depictions must similarly have something more in common than simply ‘not being linguistic’, if we are to group them as a class of any sort, let alone a class analogous to gesture. Channel can no longer perform this function. Can the third parameter provide it — are signed manual depictions and signed points similar in a having relatively little in the way of structure compared to linguistic representations?
2.3
Aspects of structure
A conflation of gesture with lack of discrete structure—“in signing, gesture is analogue representational form”— is the third definition of gesture implicit in the sign research literature to date. And here, a potential merging of two parameters, at least, might seem within reach, since where analogue form is allowed to exist, a lack of linguistic representational principle is acknowledged. Liddell (2003), for example, identifies only limited aspects of classifier predicate construction as ‘depictive’, and of signed deictics as ‘pointing’, on the grounds that just these parts of classifier predicate structure are analogue in form. However, it is simply a mistake to assume that uses of the trajectory can only be analogue in form, and that depiction is similarly always free-form. As regards the former, in Auslan (Australian Sign Language) Johnston (1987) states that one can use points to locations or objects at locations using four contrastive angles of the hand. (The pointing trajectory must be imagined as an arc, here.) The contrastive signficance of these angles are translated into English as ‘this’, ‘that’, ‘that over there’ and ‘yon’. An analogue reality, distance from the speaker, could be mapped onto an analogue variation in the angle of the signer’s pointing hand— but it is not. Instead, the representation of distance by pointing is ‘discrete-ised’ into four contrastive standard forms in this language. As regards manual depiction, for decades, sign researchers have been arguing that a class of bafflingly depictive-like signings long called classifier predicates must be linguistic representations simply because they are so rich in rule-governed discrete-combinatorial structure; standard handshapes and movements, at least, that are selected and combined in conventionalised ways, as when the upright G hand is used to show one person approaching another, then turning and going away at right angles. Speaker-hearer gesture includes manual depiction, but these manual depctions are largely described in the literature as analogue and nonstandardised in form. On the other hand, from its earliest investigations in the late 19th centruy to the most recent decades, literature on the acquisition of drawing repeatedly insists that discrete-combinatorial form and convention-bound construction is typical of depiction in even the youngest children (Goodnow 1977, Cox 1992; see Cogill-Koez 2003). The fact of the matter is that while realistic depictions are easy for humans to decode, their construction is far from natural; rather, discrete-combinatorial structure is how human beings, speaker or signer, most easily create visual representations, whether on the channel of the hands or the channel of pencil and paper. Returning, then, to the issue of gesture in signed systems, the incipient combination of interesting parameters (where ‘nonlinguistic’ or at least depictive and pointing representational principles might seem about to map onto analogue structure) now collapses. Discrete and discretecombinatorial structure can and do emerge naturally in nonlinguistic principles of representation. They may not do so very often in the points and the iconic gestures of most speech-based gesture systems, but this more sophisticated structure actually typifies much of the manual depictions and even the points of signed communication systems.
4
Dorothea Cogill-Koez: In search of a ‘gesture instinct’: The evidence from signed communciation system
2.4
It is not useful to speak of gesture in signed communication systems
In signed systems, then, the bundle of features that typify gesture in speech-centred systems appear to go their separate ways.They enter into different communicative subsystems, some of which reach levels of structural sophistication not at all a normal feature of anything like their nearest equivalents in speech-based gesture. There is no obvious collocation of parameters or subsystems of signed communication that stand forward as candidates for the signers’ realisation of a ‘gesture instinct’. Much appears tantalisingly familiar, but what is familiar is, on closer inspection, simply the individual parameters of gesture. Gesture itself breaks apart, and in doing so it appears to lose strength as a candidate for a primary communicative subsystem.
3
Lower-level shared parameters of human communicative architecture
More positively, though, what can be seen on looking across speech-centred and sign-centred communication systems? The lower-level parameters themselves, and their range of realisations, do remain a constant in the architecture of each communicative system. The comparison of signed and speech-based systems does reveal an ability to draw on a variety of channels, a capacity to expoit a variety of representational principles, and a striking and perhaps unexpected ability to take any of the resulting forms to varying levels of structure. The comparison also highlights the role of development, in our species, in integrating selected parameters into useable communicative subsystems — and this in a ‘mix-and-match’ way that seems almost as free in the possible combinations as it is logically possible, or practically necessary, to be. In place of a ‘gesture instinct’, the comparison has pointed to a range of more fundamental communicative instincts, that seem to be more robustly present in human communication systems.
4
A useful reconceptualisation of gesture?
Might it then be useful to give more prominence to these lower-level parameters in constructing, equally, models of gesture in speech-based systems and of diverse subsystems in signed communciation? While one is obviously extremely hesitant to make suggestions within a field outside one’s own, one argument stands in favour of the approach: we instantly have a common ground on which researchers in gesture and researchers in signed systems can communicate! Reconceptualising gesture as being just one of many possible configurations of more basic human ‘communicative instincts’, and conceiving of the major signed communicative subsytems in the same way, gives us a means of making reasonably detailed comparisons of how humans represent meaning across very diverse communicative subsystems, and in what otherwise might appear to be very divergent total systems. Secondly, it cannot escape readers of the gesture literature that, after an initial phase in which identification of gestural subclasses was a major focus, the field of gesture has rather turned its back on issues of categorization. This is natural when such categorization leads nowhere in particular. In the light of what is seen to be possible in signed systems, though, would it not be possible to ask a number of questions of such categories now? These are all questions that, mutatis mutandis, also spring to mind when reflecting on signed communciation in the light of this model. They include empirical and theoretical questions such as the following: When does structure appears in different classes of gesture, under what communicative pressures, how quickly, and to what degree? Can the same set of parameters conceivably be extended to other channels, such as the face (rather the ‘poor cousin’ of research in gesture as in sign)? What realizations of which parameters can be simultaneously combined, and are there are limitations on the degree of structure that can be produced in simultaneously-active channels? These and many other questions are arguably highlighted by a keener focus on the parameters within gesture that it shares with signed subsystems. They could, perhaps, prove profitable to investigate, in the search for what evolution has given us in the way of ‘instincts’ for the construction of human communication systems.
5
GESPIN proceedings, vol. I
Bibliography Cogill-Koez, D. 2002. Signed language classifier predicates: linguistic structures or schematic visual representation? Sign Language and Linguistics 3 (2); 153-208. Cox. M. 1992. Children’s drawings. London: Penguin. Goodnow, J. 1977. Children’s drawing. Cambridge, Mass: Harvard University Press. Johnston, T. A. 1987. A general introduction to Australian Sign Language (AUSLAN). Payneham, South Australia: TAFE National Centre for Research and Development. Klima, E. and Bellugi, U. 1979. The Signs of Language. Cambridge, Mass.: Harvard University Press. Liddell, S. K. (2000) Indicating verbs and pronouns: pointing away from agreement. In Emmorey, K and Lane, H (eds), The signs of language revisited: An anthology to honor Ursula Bellugi and Edward Klima; 303-319. Mahwah: Lawrence Erlbaum Associates. Liddell, S. K. 2003. Grammar, gesture and meaning in American Sign Language. Cambridge: Cambridge University Press. McNeill, D. 1992. Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. Poizner, E., Klima, E. and Bellugi, U. 1987. What the hands reveal about the brain. Cambridge, Mass.: MIT Press. Stokoe, W. C. 1960. Sign language structure; an outline of the visual communication system of the American deaf. Studies in Linguistics: Occasional Papers, 8. Dept. Anthropology and Linguistics, University of Buffalo, New York.
6