COMPARATIVE JUDGMENTS AS A FUNCTION OF THE by Juan Pardo

COMPARATIVE JUDGMENTS AS A FUNCTION OF THE DIRECTION OF COMPARISON VERSUS WORD ORDER

Previous findings showed that comparative judgments in survey questions are largely affected by the direction of comparison, that is, whether the question asks respondents to compare A (subject) to B (referent) or B to A. These asymmetry effects were attributed to the dynamics that respondents attend differently to the features of an object depending on whether it functions as the subject or the referent of a comparison. Most research on direction-of-comparison effects, however, confounded direction of comparison and word order so that the subject is typically presented first and the referent second. This article disentangles this confound and investigates the separate impact of the direction o f comparison and word order. The results replicate earlier findings regarding direction-of-comparison effects and found no evidence for a systematic impact of word order. Even when the referent of the comparison is presented first and the subject is presented last, direction-of-comparison effects are observed. The findings are discussed in terms of applied and theoretical significance. Abstract

Previous findings demonstrated that asking respondents to compare A to B resulted in different comparison judgments than questions that asked respondents to compare B to A (Wanke, Schwarz, and NoelleNeumann 1995). For example, in the study by Wanke, Schwarz, and Noelle-Neumann (1995), 59 percent of the respondents indicated they had had more luck in life than other people when asked to compare themselves with other people, while only 29 percent thought themselves luckier when asked to compare other people to themselves. is a research associate at the Universitat Heidelberg. Address correspondence to Michaela Wanke, Psychologisches Institut, Universitat Heidelberg, Hauptstr. 47-51, D-69117 Heidelberg, Germany; E-mail ziv7@psi-svl.psi.uni-heidelberg.de. The reported research was supported by grant 28915-1 from the Deutsche Forschungsgemeinschaft to H. Bless, N. Schwarz, and M. Wanke. The author would like to thank Eric Igou, Eva Ischen, Daniel Kumpf, Claudia Scharwachter, Markus Schuster, and Kornelia Stark for collecting the data, and Norbert Schwarz for comments on a first draft. MICHAELA WANKE

Comparative Judgment Wording Effects

40 1

These asymmetries were predicted on the basis of models regarding the cognitive dynamics underlying comparison judgments (Tversky 1977; Tversky and Gati 1978). It is argued that when respondents compare A to B they focus on the features of A (the subject of comparison or the to-be-compared stimulus) and check to which extent these features are present in B (the referent of the comparison or the stimulus to which the subject is compared). By doing so respondents neglect those features of the referent that are not shared with the subject. Thus, the comparison judgment is overly influenced by features of the stimulus that functions as the subject of comparison. When the direction of comparison is reversed, respondents focus on the features of the previous referent (now the subject) but neglect the unique features of the former subject (now the referent). Thus, reversing the direction of comparison, for example, by the wording of the question, will result in different feature accessibility and different mental representations of the compared objects (see also Wanke 1993, 1996). However, it may be argued that rather than focusing on the subject of comparison respondents focus on the Jirst object presented in the question. In fact, typically studies that varied the direction of comparison also presented the subject first (see, e.g., Dunning and Parpal 1989; Holyoak and Gordon 1983; Karylowski 1990; Srull and Gaelick 1984; Wanke 1993; Wanke, Schwarz, and Noelle-Neumann 1995). Thus, we cannot assume beyond doubt that the observed effects are elicited by the direction of comparison rather than by the order in which the two stimuli are presented in the question (word order). Note that this alternative explanation assumes the same dynamics of respondents focusing differentially on the two objects of comparison. Correspondingly, focusing on the first object presented would also predict different mental representations resulting from rephrasing the question. Nevertheless, for theoretical and applied reasons it is interesting to investigate to what extent asymmetric comparison judgments are the result of the varying direction of comparison and to what extent they are the result of word-order effects. My experiment addressed this confound and varied the direction of comparison independent of the word order. More specifically, direction of comparison was orthogonally crossed with word order. As reflected in figure la, previous research had only tested the cells of the main diagonal and found different judgments (asymmetry), indicated in the figure by the different patterns in these two cells. The conditions of the off-diagonal remained uninvestigated, indicated by the blank cells. If this asymmetry is indeed due to the direction of comparison this should be reflected in a main effect for this factor (fig. lb). If this asymmetry is, however, due to word order, no direction-of-comparison effect should emerge but, rather, a main

l a . Previous Findings of Assymetry Stimulus presented first

Direction of Comparison

A-8

1b. Direction of Comparison Effect Stimuius presented first

Direction of Comparison

A-8

BEl

1c. Word Order Effect Stimuius presented first

Direction of Comparison

A-B

Hull

I d . Direction of Comoarison and Word Order Effect

Stimuius presented first A

Direction of Comparison

Figure I. Possible accounts for asymmetry effects

Comparative Judgment Wording Effects

403

effect for word order (fig. lc). Of course, both effects are not incompatible. If respondents focus on the subject of comparison and on the object that is presented first, as would be reflected by two main effects, asymmetric comparisons should be strongest when both effects coincide, that is, when the subject is also presented first (fig. Id). This scenario would also imply that both effects work against each other and diminish asymmetries when the referent is presented first, as indicated by the same patterns in the cells of the off-diagonal. Respondents may focus on the subject, but they may in addition attend to the features of the referent when it is presented first rather than neglect its features, resulting in reduced asymmetry effects. The latter scenario especially makes it worthwhile to tease apart the two effects of direction of comparison and word order. These assumptions were tested by using the questions asked in study 4 reported by Wanke, Schwarz, and Noelle-Neumann (1995) and adding word order as a factor. According to the original study, more respondents should decide for a particular object when it was presented as the subject rather than the referent of comparison. According to the same dynamics (for a detailed account, see Wanke, Schwarz, and Noelle-Neumann 1995), one would expect the same effect for the object presented first rather than second if respondents indeed focus on the first mentioned object rather than the subject of comparison.

Method German adults (18 years of age or over) were approached in the downtown pedestrian mall of a German city. About 40 percent of the approached individuals agreed to participate in the study, resulting in a heterogeneous convenience sample of 112 males and 111 females, for a total N of 223. All interviews were conducted face-to-face. To assure random assignment of the respondents to conditions, the questionnaires were brought into a random order. The interviewers were unfamiliar with the hypotheses and were blind to the condition prior to the beginning of the interview, thus guarding against selection bias. The questions replicating the original conditions follow.

According to your opinion, does industry contribute more or less to

air pollution than traffic?

According to your opinion, is the news reporting on television better

or worse than the news reporting in your newspaper?

404

Michaela Wanke

Do you believe that other people had more or less luck in life than

yourself?

According to your opinion, does traffic contribute more or less to air

pollution than industry?

According to your opinion, is the news reporting in your newspaper

better or worse than the news reporting on television?

Do you believe that you had more or less luck in life than other people?

In addition, two versions were used with reversed word order. VERSION A 2 (DIRECTION O F

COMPARISON: A-B, B FIRST)

Compared to traffic, does, according to your opinion, industry contrib-

ute more or less to air pollution?

Compared to your newspaper, is, according to your opinion, the news

reporting on television better or worse?

Compared to yourself, do you believe that other people had more or

less luck in life?

VERSION B2 (DIRECTION OF

COMPARISON: B-A,

FIRST)

Compared to industry, does, according to your opinion, traffic contrib-

ute more or less to air pollution?

Compared to the news reporting on television, is, according to your

opinion, the news reporting in your newspaper better or worse?

Compared to other people, do you believe that you had more or less

luck in life?

Thus, the newly introduced wordings varied the direction of comparison but presented the referent first rather than the subject.' Each version was read to one-quarter of the respondents. Moreover, for half of the respondents in each version the order of the response alternatives was reversed to control for primacy or recency effects, resulting in a 2 (direction of comparison) x 2 (word order) x 2 (response order) factorial between-subjects design. The interviewers accepted "both are equal" and "don't know" as responses if volunteered but did not offer these alternatives. 1. As one reviewer noticed, reversing the word order may have changed the comprehensibility of the sentence. Although this may be so, certainly these questions are not incomprehensible and would not elicit mere chance responses. Moreover, respondents could ask the interviewers to repeat the question. Finally, the number of qualified responses, which may serve as an indicator of question comprehension, did not vary between old and new versions.

Comparative Judgment Wording Effects

405

Results Table 1 reveals that, as expected, the percentage of respondents choosing alternative A was higher when A was compared to B rather than vice versa and preferences for B were higher when B was compared to A rather than vice versa. In contrast, the word order did not show any systematic impact. This overall pattern is reflected in a significant direction-of-comparison effect (F(1,214) = 5.58, p < .02) and no effect for word order (F < I), according to an analysis of variance (ANOVA) that treated direction of comparison, word order, and response order as between-subject factors and question as a within-subject factor and the preference for A or B as the dependent variable (Rosenthal and Rosnow 1985). Univariate ANOVAs and conceptually analogous loglinear analyses with the same variables were performed on each item level. The latter are reported in the following ~ a r a g r a p h . ~ The direction-of-comparison effect, in other words, the higher preference for an alternative when it was presented as the subject of comparison, was significant at the 5 percent level for air pollution (z = 1.65) and for news reporting (z = 1.69), and at the 10 percent level for the luck in life question (z = 1.33). Thus, it is indeed the direction of comparison that is responsible for the previously observed asymmetries. It was, however, also speculated that word order may have an independent effect. The evidence for a word-order main effect is inconsistent and inconclusive. For air pollution, the effect is opposite what had been predicted; preferences were higher for the alternative presented last, although the effect was not significant (z = 0.68). For news reporting, primacy effects occurred but were not significant (z = - 1.10). Only for the luck in life question did a marginally significant primacy effect emerge (z = - 1.43, p < .08). However, this effect was not strong enough to wipe out the asymmetry due to direction of comparison completely, as the different percentages for "self' in columns 2 and 3 reveal. Altogether, there is no evidence that presenting the subject after the referent in comparison questions can diminish direction-of-comparison effects. In fact, it may also augment such effects, as the pattern for air pollution shows.

Discussion The data clearly replicate the earlier findings of Wanke, Schwarz, and Noelle-Neumann (1995). More respondents decided for an alternative 2. Effects of higher order than the ones reported were not significant.

Table I . Percentages of Preferences as a Function of the Direction of Comparison and Word Order Marginals Word Order A First

Primary contributor to air pollution: A. Industry B. Traffic Preferred news reporting: A. TV B. Newspaper More luck in life: A. Others B. Self N

Direction of Comparison

B First

Word Order

A-B

B-A

A First

B First

36 45

47 40

37 49

42 48

45 41

30 52

16 63

31 53

20 61

31 57

23 57

5 63 55

4 70 56

10 58 112

5 73 111

11 65 111

4 66 112

A-B

B-A

A-B

B-A

46 43

38 53

54 38

32 54

29 60

14 52 56

7 78 56

No~~.--Percentagesdo not add up to 100 percent because "don't know" responses and "both equal" are not shown.

Comparative Judgment Wording Effects

407

when it was presented as the subject rather than the referent of the c ~ m p a r i s o nThe . ~ data presented here eliminated the potential confusion between the role of the subject of comparison versus the object mentioned first in the comparison question. Independent of its position, an object was assigned more importance when it was presented as the subject rather than the referent. These findings are illuminating regarding the causes of direction-ofcomparison effects. Despite the numerous studies that have found asymmetry effects (see, e.g., Dunning and Parpal 1989; Holyoak and Gordon 1983; Houston, Sherman, and Baker 1989; Sanbonmatsu, Kardes, and Gibson 1991; Srull and Gaelick 1983; Tversky 1977; Wanke 1993; Wanke, Schwarz, and Noelle-Neumann 1995), it is not clear why individuals attend more to the features of the subject of comparison. In most studies, the subject was also mentioned first, which suggests that individuals simply focus on the object mentioned first, because it will catch more attention. The data presented here, however, suggest that primacy cannot account for asymmetry effects. Rather, the inherent structure of subject and referent seems to drive the effects. It is proposed here that respondents use the direction of comparison to determine which of the presented objects is the target of judgment. A comparison of A to B primarily asks for a judgment on A. Consequently, it makes sense to focus on A, at least initially. Given moderate involvement, respondents may then neglect B rather than attend to B's features in turn. Sufficiently high involvement can increase attendance to the referent and subsequently reduce direction-of-comparison effects (Wanke 1993). In sum, for the respondent, the direction of comparison represents a meaningful function as it draws attention to the supposed target of the question and helps the pragmatic understanding of what the interviewer wants to know (for a review on question comprehension, see Strack 1992). As such, its function cannot be substituted by merely formal aspects such as word order. However, future research has to explore to what extent other variables may assume the same function and interfere with direction-of-comparison effects.

Appendix German Question Wording

Tragt, Ihrer Meinung nach, die Industrie eher mehr oder eher weniger zur Luftverschmutzung bei als der Verkehr? 3. See Wanke, Schwarz, and Noelle-Neumann (1995) for a detailed account of when to expect the opposite effect.

408

Michaela Wanke

Ist, Ihrer Meinung nach, die Nachrichtenberichterstattung des Fernsehens eher besser oder eher schlechter als die Nachrichtenberichterstattung der Tageszeitung? Glauben Sie, dalj andere eher mehr oder eher weniger Gluck im Leben hatten als Sie selbst?

Tragt, Ihrer Meinung nach, der Verkehr eher mehr oder eher weniger zur Luftverschmutzung bei als die Industrie? Ist, Ihrer Meinung nach, die Nachrichtenberichterstattung der Tageszeitung eher besser oder eher schlechter als die Nachrichtenberichterstattung des Fernsehens? Glauben Sie, dalj Sie selbst eher mehr oder eher weniger Gluck im Leben hatten als andere? VERSION A2 (DIRECTION OF COMPARISON: A-B, B FIRST)

Verglichen mit dem Verkehr, tragt, Ihrer Meinung nach, die Industrie eher mehr oder eher weniger zur Luftverschmutzung bei? Verglichen mit der Nachrichtenberichterstattung der Tageszeitung, ist, Ihrer Meinung nach, die Nachrichtenberichterstattung des Fernsehens eher besser oder eher schlechter? Verglichen mit sich selbst, glauben Sie, dalj andere eher mehr oder eher weniger Gluck im Leben hatten? VERSION B 2 (DIRECTION OF COMPARISON: B-A, A FIRST)

Verglichen mit der Industrie, tragt Ihrer Meinung nach der Verkehr eher mehr oder eher weniger zur Luftverschmutzung bei? Verglichen mit der Nachrichtenberichterstattung des Fernsehens, ist, Ihrer Meinung nach, die Nachrichtenberichterstattung der Tageszeitung eher besser oder eher schlechter? Verglichen mit anderen, glauben Sie, dalj Sie selbst eher mehr oder eher weniger Gluck im Leben hatten?

References Dunning, D. and M. Parpal. 1989. "Mental Addition versus Subtraction in Counterfactual Reasoning: On Assessing the Impact of Perceptual Actions and Life Events." Journal of Personality and Social Psychology 575-15. Holyoak, K. J. and P. C. Gordon. 1983. "Social Reference Points." Journal of Personality and Social Psychology 44:881-87. Houston, D. A., S. J. Sherman, and S. M. Baker. 1989. "The Influence of Unique Features and the Direction of Comparison on Preferences." Journal of Experimental Social Psychology 25: 121-41. Karylowski, J. J. 1990. "Social Reference Points and Accessibility of Trait-Related

Comparative Judgment Wording Effects

409

Information in Self-other Similarity Judgments." Journal of Personality and Social Psychology 58:975-83. Rosenthal, R., and R. L. Rosnow. 1985. Contrast Analysis. Cambridge: Cambridge University Press. Sanbonmatsu, D. M., F. R. Kardes, and B. D. Gibson. 1991. "The Role of Attribute Knowledge and Overall Evaluations in Comparative Judgment." Organizational Behavior and Human Decision Processes 48:131-46. Srull, T. K., and L. Gaelick. 1983. "General Principles and Individual Differences in the Self as a Habitual Reference Point: An Examination of Self-other Judgments of Similarity." Social Cognition 2: 108-21. Strack, F. 1992. "Order Effects in Survey Research: Activating and Informative Functions of Preceding Questions." In Context Effects in Social and Psychological Research, ed. N. Schwarz and S. Sudman, pp. 23-34. New York: Springer. Tversky, A. 1977. "Features of Similarity." Psychological Review 84:327-53. Tversky, A., and I. Gati. 1978. "Studies of Similarity." In Cognition and Categorization, ed. E. Rosch and B. Lloyd, pp. 81-98. Hillsdale, NJ: Erlbaum. Wanke, M. 1993. Vergleichsprozesse bei evaluativen Urteilen: Der EinjuJ der in der Frage vorgegebenen Vergleichsrichtung. New York: Mellen. . 1996. "Contrast and Assimilation as a Function of the Direction of Comparison." Unpublished manuscript. Wanke, M., N. Schwarz, and E. Noelle-Neumann. 1995. "Asking Comparative Questions: The Impact of the Direction of Comparison." Public Opinion Quarterly 59~347-72.

http://www.jstor.org

LINKED CITATIONS - Page 1 of 1 -

You have printed the following article: Comparative Judgments as a Function of the Direction of Comparison Versus Word Order Michaela W채nke The Public Opinion Quarterly, Vol. 60, No. 3. (Autumn, 1996), pp. 400-409. Stable URL: http://links.jstor.org/sici?sici=0033-362X%28199623%2960%3A3%3C400%3ACJAAFO%3E2.0.CO%3B2-9

This article references the following linked citations. If you are trying to access articles from an off-campus location, you may be required to first logon via your library web site to access JSTOR. Please visit your library's website or contact a librarian to learn about options for remote access to JSTOR.

[Footnotes] 3

Asking Comparative Questions: The Impact of the Direction of Comparison Michaela Wanke; Norbert Schwarz; Elisabeth Noelle-Neumann The Public Opinion Quarterly, Vol. 59, No. 3. (Autumn, 1995), pp. 347-372. Stable URL: http://links.jstor.org/sici?sici=0033-362X%28199523%2959%3A3%3C347%3AACQTIO%3E2.0.CO%3B2-9

References Asking Comparative Questions: The Impact of the Direction of Comparison Michaela Wanke; Norbert Schwarz; Elisabeth Noelle-Neumann The Public Opinion Quarterly, Vol. 59, No. 3. (Autumn, 1995), pp. 347-372. Stable URL: http://links.jstor.org/sici?sici=0033-362X%28199523%2959%3A3%3C347%3AACQTIO%3E2.0.CO%3B2-9

NOTE: The reference numbering from the original has been maintained in this citation list.