QUESTION GRAMMAR EFFECTS PG. 1 by Juan Pardo

QUESTION GRAMMAR EFFECTS PG. 1

Question Grammar Effects and Survey Response Quality Adam F. Simon Department of Political Science, Yale University

June, 2007

Running Head: QUESTION GRAMMAR EFFECTS

QUESTION GRAMMAR EFFECTS PG. 2 Abstract Using data derived from 207 survey questions asked of national telephone samples by leading organizations, between 1990 and 2000, concerning Medicare or health insurance polices, this paper charts the effect of altering a questionâ&#x20AC;&#x2122;s grammatical construction (putatively absent other changes). For four positive concepts aggregate approval increases while for five negative concepts approval decreases when that concept appears as a cause as opposed to an effect. We then attempt to discredit rival hypotheses including primacy effects as well as non-random distribution of question components and dates. These findings suggest that in behaving as we would expect of ordinary language users, survey respondents and, therefore, poll results are of a higher quality than some believe.

QUESTION GRAMMAR EFFECTS PG. 3 Question Grammar Effects and Survey Response Quality This paper investigates two related concerns. A narrow concern comes first; does grammatical construction affect responses to poll questions? Like question wording and analogous effects discussed below, observers of polls are well aware of how seemingly innocuous changes can influence respondents. Grammar, being such a major part of language, may produce parallel results. Here, we examine the way the grammar of cause and effect influences approval to otherwise similar questions. We also wish to make a larger statement. By organizing the data to support an inference about the underlying mechanism, we hope to say something positive about survey response quality and the enterprise of polling more generally. Specifically, we believe that these results are tied to ordinary language comprehension and that should similar mechanics underlie other classes of response effects, then the public opinion surveys may be much more valuable than some may admit. Scanning the survey methodology literature can leave the uninitiated reader with a sense of horor. Many standard texts do little more than catalog what can go wrong with opinion polling, often implying (if not directly stating) that sampling error represents the minimum of uncertainty within a given question's marginal responses. This uncertainty stems mainly from potential artifacts that arise in survey construction. Payne (1951), for instance, lists a hundred considerations at the end of The Art of Asking Questions. The litany of difficulties includes: non attitudes, contextual effects, question order, question wording and response effects. More importantly, this list leads many people to air doubts regarding the polling enterprise in general, reflect on the title of Bishop's (2005) recent workâ&#x20AC;&#x201D;The Illusion on of Public Opinion, for example.

QUESTION GRAMMAR EFFECTS PG. 4 The empirical analysis presented below, first, suggests that this list requires another entry covering question grammar. An examination of nine health insurance concepts in 207 national survey questions asked by leading organizations, finds that altering a questionâ&#x20AC;&#x2122;s grammatical construction (putatively absent other changes) regularly produces statistically significant differences in aggregate response. However, this finding may represent more than another hazard because there seems to be a consistent mechanism underlying the effect, namely the way survey respondents use natural language. The study concludes with a discussion that reasons if similar mechanisms underlie other classes of effect then the survey enterprise is healthier than some will admit. Of polling's two steps, sampling may be less controversial. Aside from issues with response rate and perhaps sampling frame, probability theory generally makes deciding who to poll relatively uncontroversial. Thus, as Torangeau (2000) emphasizes, a surveyâ&#x20AC;&#x2122;s veracity depends almost completely on the accuracy of measuring and representing peopleâ&#x20AC;&#x2122;s answers. This takes a certain degree of art, especially given the aforementioned hazards. Of these dangers, question wording effects may be the starkest. One canonical survey research text reaches the following conclusion about this effect.

Absolute marginal proportions are to be greatly distrusted, for relatively innocuous changes in wording or context can result in considerable change in proportions. Yet in many cases the marginal change in response to such a question variation is slight or nonexistent. The basic problem is not that every wording

QUESTION GRAMMAR EFFECTS PG. 5 change shifts proportions—far from it—but that is extraordinarily difficult to know in advance which changes will alter marginals (Schuman and Presser 1996; pg 300).

Schuman and Presser (1996) go on to suggest that researchers rely on the associations between questions, rather than questions themselves. While useful to the social science of attitudes and behavior, this approach suffers in the eyes of democratic theory. Reliable marginals seem essential to majoritarian governance. Consider the problems in interpreting the results produced by split-half experimentation, the most robust way to show the impact of particular word changes. Given a substantial difference in response across two versions of ostensibly the same question, which one has normative standing? In other words, which should guide public action? This question has no answer absent some theoretical guide. Best practice calls for examining the versions to determine which seems more reasonable in light of public understandings. This raises a larger question as to how observers can best interpret public response. One answer may be to take the public’s language use seriously, using contemporary linguistics as a source of enlightenment. This paper, though, does not deal with question wording effects; rather it tackles a less complex effect related to question grammar. This research opens with a demonstration of the role grammar plays in altering answers between otherwise similar survey questions. At the same time, it seeks to organize the effects in order to uncover the mechanism behind them. Knowledge of the underlying mechanism, then, makes the effect comprehensible and predictable. By inference, predictable effects can be avoided;

QUESTION GRAMMAR EFFECTS PG. 6 thus diminishing the effect's normative import once it has been taken into account. Finally, by analogy, future research, perhaps in the linguistic vein, can be applied to other effects that also impair interpretation.

Literature Review Important lines of linguistic research background the project. Although these lines' perspective and conclusions may not be wholeheartedly endorsed by all linguists, they certainly do not reject them. Most of the works cited here are motivated by the view that language is an active tool in the communication process. As Clark (1996, p 3) states:

[L]anguage use is really a form of joint action. A joint action is one that is carried out by an ensemble of people acting in coordination with each other. â&#x20AC;Ś When Alan produces a signal for Barbara to identify, he means something by it: He has certain intentions she is to recognize. In coordination with him, Barbara identifies the signal and understands what he means by it. Much of what we think of as language use deals with the mechanics of doing this effectively. (Clark 1996, p. 23; see also Halliday 1985).

This view nicely fits the survey interview; pollsters or their confederates use language in an attempt to acquire information. More precisely, in Searle's (2000) jargon, each interview entails a series of speech actsâ&#x20AC;&#x201D;the interviewer's interrogatives intermixed with the respondent's answers.

QUESTION GRAMMAR EFFECTS PG. 7 At its simplest level language in general and survey interrogatives in particular consist of words (representing concepts) and the rules that combine them into meaning; these rules are known as grammar (Pinker 1999). Grammar traditionally subdivides into morphology and syntax (Bloomfield 1933), which concern the way words adjust to usage—subject verb agreement—for instance (Spencer 2001) and how words combine to form phrases, sentences and longer constructs (Baker 2001), respectively. Given pollster's professionalism, morphology probably does not often affect response, but is the same true for syntax? Answering this question calls for a more precise notion of syntax's function. While formal grammars, like the generative grammar associated with Chomsky (1980), focus on idealizations, functional analyses set out to describe and explain how ordinary language, including syntax, works (Foley and Van Valin 1984). Halliday's (1985) ideas about functional grammar suit the analyses of interrogatives like survey questions particularly well. Halliday (1985) develops a theory explaining how the interplay of phrases communicates meaning. Here, sentences don’t contain subjects and objects but participants that take on functional characteristics, such as actor or goal. This study revolves around the psychological notion of cause and effect (Edwards and Potter 1993), which two components of functional grammar—action and consequence—chart. As Hansen (1998) observes, assessing attitudes toward public policy tradeoffs constitutes one of polling's essentials. Take a paradigmatic survey question (studied in more detail below): "would you support a system of national health insurance, if it meant raising taxes?" Causal reasoning captures the inherent policy tradeoff. Reformulating the question into "would you support raising taxes in order to finance a system of national health insurance?" does not obviously change the meaning.

QUESTION GRAMMAR EFFECTS PG. 8 However, Cruse (1986) alerts us to the possibility that grammar can be semantically arbitrary, that it is independent of word usage. Thus, this and similar reformulations may not be inert; the empirical aspect of this study focuses on investigating these potential impacts. The psychology of cause and effect also allows us to make a prediction as to grammar's effect. According to Edwards and Potter (1993), humans engaged in causal reasoning generally pay more attention to the cause rather than the effect. Further, given the role attention plays in decision making (see Nisbett & Ross 1980), one could expect more weight to be put on causes. This reasoning leads to a prediction of a polarization effect, such that agreeable actions will produce even greater approval while disagreeable ones will produce less approval relative to their equivalents constructed as consequences. To isolate the effects of grammar, one must hold semantic effects constant. This requirement calls for another import from linguistics, the idea of a concept. Concepts are mental representations that denote all the objects in a given category or class of entities (Murphy 2002). The simplest theory of concepts that seems sufficiently uncontroversial for our application centers on the idea of a prototype. Whereas classical theories of mental representation relied on dictionary like definitions in an attempt to strictly demarcate the represented, prototype theory grows from Wittgensteinâ&#x20AC;&#x2122;s (1953) view that languageâ&#x20AC;&#x2122;s inherent ambiguity defies exhaustive categorization schemes. Accordingly prototype theory holds that the representation process is open-ended; prototypes embody the idea that mental representations move outward from a best example. In comprehension, then, humans use prototypes that allow them to recognize new instances of more familiar things, and they can also create new prototypes to deal with

QUESTION GRAMMAR EFFECTS PG. 9 novel inputs (see Murphy 2002 for more discussion). Here, we deploy this idea to eliminate minor semantic variation across question wordings. By essentially ignoring this minor variation, we can examine a larger set of identifiable concepts across grammatical roles in order to see if these roles influence survey response. Finally, the design discussed in the next section calls for additional tests in order to quash rival hypotheses. In brief, there are three that will be discussed after presentation of the initial resultsâ&#x20AC;&#x201D;first is the idea of primacy (and its companion recency), second is a biased distribution of question timing, and third is a biased distribution of question concepts.

Design, Data and Procedures The findings presented below depend on correlation analyses of Medicare and other health insurance poll questions asked during the period between 1990 and 2000. The empirical strategy, to reiterate, is simple; isolate a set of concepts that appear in question actions and consequences, assess the effect of this variation given the expectation of polarization and then attempt to discredit rival hypotheses. Ideally, following research could employ experimentation's known strength in identifying causation to uncover converging findings. Careful examination of Gallup Polling Report provided questions asked of national telephone samples by well regarded organizations concerning Medicare or health insurance. Of these, attitudinal questions with responses readily convertible into percentages approving and disapproving (for example, "support," "favor" and "yes" as

QUESTION GRAMMAR EFFECTS PG. 10 well as answers using scales containing words like "strongly") formed the initial dataset of roughly 600 questions. These questions were parsed by a team of undergraduates into the grammatical categories defined above. By definition, all questions had actions, that is programs or policies that the response specifically endorsed or rejected. Not all questions proffered a consequence, where this segment was present it was usually identified by a grammatical signal such as "in order to" and "even if." In order to make the parsing more complete, the category of background was added. Segments beginning with "as you may know" that contained factual statements were placed there. Less than two percent of the questions failed to parse well, usually due to grammatical ambiguity, and these were discarded. The remaining questions were then further examined to identify concepts that appeared three times or more in the action segment and three times or more in the consequence or background. Table 1 presents this list of conceptual prototypes and the variations they represent.

Insert Table 1 about here.

The coding and subsequent examination yielded nine concepts, which Table 1 lists and divides into agreeable positives and disagreeable negatives. The four positives are: Access health insurance this concept centers on the word access and pertains to policies that attempt to give presumably uninsured people the opportunity to acquire health insurance in an unspecified way.

QUESTION GRAMMAR EFFECTS PG. 11 Provide health insurance similar to access health insurance, this concept substitutes the word provides, indicating a more specific policy to give presumably uninsured people health insurance. Clinton health care plan this concept covers questions that refer to Bill Clinton by name in referencing his plan for health insurance in the 1993 to 1995 timeframe. Private health insurance this concept, similar to access and provide, covers policies designed to award the presumably uninsured with specifically private health insurance. The five negative concepts are: Higher premiums this concept specifically mentions increasing health insurance premiums for various reasons, usually to help fund Medicare or to give the presumably uninsured health insurance in some form Pay more, similar to higher premiums, these concept employ the phrase “pay more” without using the words premium or tax. Tax increase this concept covers all the uses of non speficif tax increases as well as one use of the phrase “income tax.” Payroll tax this concept covers only specific uses of the phrase “payroll tax.” Table 1 also lists the frequency of these concepts in their grammatical locations along with the actual wordings these prototypes represent. The nine concepts represent 207 total questions as some questions contained multiple concepts. Finally, the right hand column gives the keywords used to calculate the concept's position within the question— data used in later analyses.

Results

QUESTION GRAMMAR EFFECTS PG. 12 Given this set up it was straightforward to calculate the effect of grammatical placement on aggregate response. Figure 1 presents these comparisons graphically along with estimates of the statistical significance. All four positive concepts saw increased approval when serving as a question's action as opposed to its consequence or background. The greatest difference was for access to health insurance which sees a 19 percentage point increase and the least was for private health insuranceâ&#x20AC;&#x201D;a 1.9 percent increase. Of the four, two, access health insurance and provide health insurance, are statistically significant with provide health insurance, the concept featured in the most questions, reaching the .01 level. After the four concepts a fifth entry represents the effect of grammar pooled across all positive concepts; here, the average increase in approval is 12 percent. This increase is also significant at the .01 level (F = 19).

Insert Figure 1 about here.

The negative concepts evince a similar pattern of change. All the concepts show a decrease in approval when they play the role of action in a question. The greatest difference appears in payroll tax, which sees a 29 percent decrease as grammatical role changes. The smallest decrease, 10 percent, is for higher premiums. Three of the five reach conventional levels of statistical significance while pooling across the negatives produces a highly statistical significant difference of 16 percentage points (F = 21.6, p < .01). A regression specification estimates the overall affect of grammar across all nine concepts. To create the independent variable tapping grammar, an index charting the

QUESTION GRAMMAR EFFECTS PG. 13 position of concepts in each question was computed with negative items subtracted from positive items and then weighted by their appearance on either the action or the consequence Thus, to deal questions containing multiple concepts, a positive action and a negative consequence earned a two while a negative action and a positive consequence scored a negative two. Given this scoring, the distribution of 207 questions places 9, 101, 7, 82 and 8 at negative two, negative one, zero, one and two, respectively. The regression predicting the percent approving estimates the coefficient for grammar as 8.43 with a standard error of .45, statistically significant at the p < .001 level. Given these prima facie results, perhaps the most important task is to discount alternative explanations. Primacy will be the first that springs to most minds familiar with the psychology of memory. In this literature, primacy refers to the fact that when presented with a list of items, humans will best remember the first (Asch 1946). This finding has analogs in other areas. In persuasion, for example, the argument made first has an edge. On the other hand, there is the somewhat weaker recency effect, which refers to the fact that humans will also tend to remember the last, most recent, item on a list and may in persuasion give extra weight to the argument with the last word (Furnam 1986). Both primacy and recency predict that it is the placement of a concept in the question rather than its grammatical role that would explain the above effect. For example, under primacy the fact that an action comes first is more important than the questionâ&#x20AC;&#x2122;s grammar. Because grammar works independently of position, there is an opportunity to rule out these alternatives. To illustrate, the question â&#x20AC;&#x153;do you support providing health insurance to all Americans even if it increases taxesâ&#x20AC;? is grammatically identical to the

QUESTION GRAMMAR EFFECTS PG. 14 question â&#x20AC;&#x153;even if it meant increasing taxes, would you support providing health insurance to all Americans.â&#x20AC;? In this comparison, positions have changed without altering grammatical construction. The observations of concept position take the form of a count of the number of characters from the beginning of the question to the keywords listed in Table 1. Table 2 presents a series of 11 multivariate regression models that pit the estimated effect of position against the estimated effect of grammatical placement, tapped by a dichotomous variable, zero for action and one for consequence or background. A combined regression, using the technique described above, estimates the overall coefficient for grammar at 7.74 with a standard error of 1.7 while the coefficient for position is .01 with a standard error of .03, showing that grammar is far more statistcally significant than position.

Insert Table 2 about here.

The results are consistent across all nine concepts as well as the two aggregations. Of the nine individual regressions, grammatical segment reaches a statistically significant level of prediction seven times while position only attains this status twice. Further, in no case is position a better predictor of response than grammar. The two aggregate models are also telling. For positives, both coefficients reach levels of statistical significance, but the F statistic is far greater for grammatical role, that is F = 5.9 for grammar versus 4.0 for position. For the negatives, the coefficient on position does not reach statistical significance while the grammar coefficient is highly significant (F = 5.2, p < .01).

QUESTION GRAMMAR EFFECTS PG. 15 Another possible explanation of the initial effect may stem from a biased distribution of conceptual companions within each concept’s question. We employ the term conceptual companion to identify any other concept within a question besides the one under study. To illustrate, in the question “do you support providing health insurance to all Americans even if it increases taxes,” providing health insurance is the companion of increasing taxes. Hence, one possible alternative explanation holds that should positive actions be paired with less negative consequences than positive consequences are paired with negative actions, one would observe the demonstrated effect of grammar. It is difficult to completely eliminate this explanation in a non-orthogonal corelational design because it is necessary to create and compare equivalent conceptual companions. One can, nonetheless, investigate the distribution of companions across the nine concepts.

Insert Table 3 about here.

Table 3 charts the distributions of companions. To construct this table questions with no companions were noted and the rest of the companions were compiled into a single large list. Next duplicates were eliminated. A team of undergraduates examined the remainder in an attempt to consolidate them using the same prototyping strategy outlined above. Ultimately eight categories were constructed: 2 Sided Frame, Aid/Change Medicare, Budget/Deficit, Clinton Plan, Expand Coverage, Health Care Fact, Tax Cut and Tax Hike. Companions that did not fit into these eight were placed into an "other" category. As examination of Table 3 shows, there does not seem to be any patterning to the companion distributions that would cause the grammatical effect. A less complicated

QUESTION GRAMMAR EFFECTS PG. 16 analysis was performed to examine the interaction of question timing and grammar, which also seemed to be fairly random.

Discussion Hopefully these analyses have made a convincing case for potential grammatical effects in survey questions. While not all the examples here attain the conventional .05 level of significance, the exposed pattern varies little and in the aggregate the results are statistically and substantively as large as similar survey effects. The question arises as to their interpretation. One might merely report and leave the community to adjudicate over issues of import and remedy; however, we wish to use these results to underline a suggestion as to the meaning of this effect, similar effects as well as the polling enterprise more generally. While to our knowledge this is the first research into survey question grammar, the uncovered effect does not seem surprising given an understanding of the nature of language and language use. Humans depend on language to convey information, and each of its features participates in this task (Sperber and Wilson 1995). Being such a large feature of language, it would be surprising if grammatical changes did not have an effect. The specific form of cause and effect's influence on survey response may be less intuitive, although, as discussed above, psychologists know well the centrality of causal relationships and the role they play in mental attributions and judgments. So, it seems that an understanding of what can be called psycholinguistics sufficiently grounds the observed results.

QUESTION GRAMMAR EFFECTS PG. 17 The suggestion then is that a similar view may enlighten other effects. Take one of the paradigmatic cases in the question wording literature. Paraphrasing Schuman and Presser's (1996) report, in the context of the 1980s respondents asked "do you think we should aid the contras in Nicaragua," were far less approving than respondents asked "do you think we should aid the contras, in fighting communism, in Nicaragua." The most common interpretation of this finding holds that respondents were far more motivated to fight communismâ&#x20AC;&#x201D;a known threatâ&#x20AC;&#x201D;than aid the contras, who had a smaller footprint in the American psyche. Another non exclusive interpretation holds that the question offering the rationale of fighting communism mobilized more support than the one that did not. While this example usually is seen as a question wording effect, when seen as a policy plus a rationale, it seems to shade into grammatical territory. Observe that humans, generally being reasoning creatures, tend to support persuasive messages with rationales, all else equal (Cialdini 1998). Compare this vignette and explanation with the conclusions reached by Schuman and Presser (1996) and, more importantly, the way conclusions like this have entered the public mind. The social scientific explanation seems to hint that all the question wording effects uncovered, such as the Nicaraguan example, are readily explainable, but that one cannot predict a priori whether any specific change will have consequences. Yet the lessons drawn from this book seem to vary widely. Most practitioners, even going back before Schuman and Presser's (1996) research (cf. Gallup and Rae 1940) take this as a caution against using biased language. Correctly, pollsters understand that certain words are prejudicial in that they tend to lead responses in varying ways. Others,

QUESTION GRAMMAR EFFECTS PG. 18 usually outside the industry, take findings like this to mean that polling and the public it represents is capricious. Further, there is the belief that "manipulative" polling can be used to uncover support for any position. So, to conclude, consider how the linguistic view promulgated by the results presented in this paper addresses disparaging attitudes toward polling. A profession that expounds, and a public that believes, that citizens respond to polling questions as a speech act just as they respond to other speech acts in everyday situations inevitably supports the meaningfulness of polling results. This is not to say that polling is any less of an art, but the fulcrum of this art lay in creating sets of questions that accurately capture the polity's choices rather than in the interpretation of alternative results. Simply put, when respondents say they support X, they do indeed support X and, as a correctly selected sample, this support stands as representative of the entire body. Now, should support for X wane, when in a question it is accompanied by disagreeable consequences, that should not be surprising. Similarly, to restate the finding presented above, should X be a consequence of a disagreeable cause, its support should also wane though to a lesser degree. Most importantly, these changes are not the actions of a capricious public but rather one attuned to the potential of language in everyday use.

References Asch, Solomon E. 1946. Forming impressions of personality, Journal of Abnormal and Social Psychology, 41, 258-290 Baker, Mark C. 2001. The atoms of language. New York: Basic Books. Bishop, George F. 2005. The illusion on of public opinion. Lanham, MD: Rowman & Littlefield. Bloomfield, Leonard. 1933. Language. New York: H. Holt and Company.

QUESTION GRAMMAR EFFECTS PG. 19 Chomsky, Noam. 1980. Rules and representations. New York: Columbia University Press. Cialdini, Robert B. 1998. Influence: the psychology of persuasion New York: Collins Press. Clark, Herbert H. 1996. Using language. New York: Cambridge University Press. Cruse, David A. 1986. Lexical semantics. New York: Cambridge University Press. Edwards, Derek and Jonathan Potter. 1993. Language and causation: a discursive action model of description and attribution. Psychological Review 100: 23-41. Foley, William A. and Robert D. Van Valin, Jr. 1984. Functional syntax and universal grammar. New York: Cambridge University Press. Furnam, Adrian. 1986. The robustness of the recency effect: Studies using legal evidence. Journal of General Psychology, 113(4), 351-357 Halliday, M.A.K. 1985. An Introduction to Functional Grammar. Baltimore: University Park Press. Gallup, George and Saul F. Rae 1940. The Pulse of Democracy: The Public-Opinion Poll and How It Works. New York: Simon and Schuster. Hansen, John Mark. 1998. Individuals, institutions, and public preferences over public finance. American Political Science Review 92: 513-531. Payne, Stanley Le Baron. 1951. The art of asking questions. Princeton, NJ: Princeton University Press. Pinker, Steven. 1999. Words and rules. New York: Basic Books. Nisbett, Richard and Lee Ross. 1980. Human inference: Strategies and shortcomings of social judgment. Englewood Cliffs, N.J.: Prentice-Hall. Schuman, Howard and Stanley Presser. 1996. Questions and answers in attitude surveys: Experiments on question form, wording, and context. New York: Academic Press. Searle, John R. 2000. Mind, Language, and Society: Philosophy in the real world. New York: Basic Books. Spencer, Andrew and Arnold M. Zwicky, eds. 2001. The Handbook of morphology. Malden, MA: Blackwell. Sperber, Daniel and Dierdre. Wilson 1995. Relevance: Communication and Cognition Malden, MA: Blackwell.

QUESTION GRAMMAR EFFECTS PG. 20 Table 1. Concepts and Actual Wording Variations

Concepts

Consequences &

(Na, Nc or b)

Actions (N)

Backgrounds /B or C (N)

Keyword

Positives Access Health

Access to affordable

Access to health

Insurance (3,7)

health insurance

insurance /C (6)

Access

Access to health care /B (1) Provide Health Insurance (43,11)

Provide health insurance

Provide all Americans

(35)

with health insurance /C

Provide

(6)

Clinton Plan (2,8)

Provide their employees

Provide health insurance

with health insurance (8)

/C (5)

Clinton health care plan

Clinton

/C Private Health

Private health insurance

Insurance (10,5)

Private health insurance

Private

Negatives Higher

Higher premiums

Higher premiums /C

Premium

Reducing Medicare

Reduce the cost /C (8)

Cut

Premiums (15,3) Cut Medicare (18,14)

spending (11)

QUESTION GRAMMAR EFFECTS PG. 21 Cutting spending on

Reduce spending /B (3)

Reduce

Spending reductions in

Cutting spending on

Reduction

Medicare (2)

Medicare /C (2)

Medicare (5)

Cuts in Medicare /B (1) Pay More (34,5)

Pay more (21)

Pay more /C

Pay

Increase in taxes /C (22)

Tax

Pay a larger share (5) Pay an additional $50 per month (3)

Pay twenty percent more (2) Out of pocket payments (2) Pay one third more 1() Tax Increase

Increase in taxes (16)

(19,28) Increase in income tax (2) Tax increases /B (6) Additional taxes (1) Payroll Tax (11,3)

Payroll tax

Payroll tax /C

Payroll

QUESTION GRAMMAR EFFECTS PG. 22 Table 2. Assessing Primacy or Recency as an Alternative Explanation Gram. Concept:

Concept

Segment S.E.

Position

Adj. S.E.

Constant

R-sq

Access to 16.7

27.1

1.1

1.2

-40.6

.34

Health Ins. Provide Health

-13.1

4.7***

.03

.14

77.8

-12.9

10.3**

.24

.28

46.6

.01

4.1

.01

.02

75.6

.01

3.3***

.08

.02***

76.8

.31

9.4

13.1

.01

.04

36.2

.01

15.1

8.1*

-.04

-.17

19.1

.07

13.8

9.9+

.01

.03

28.3

.01

29.9

9.7*** -13.5

7.1*

33.5

.19

Payroll Tax

30.4

7.1***

-.07

.04*

22.9

.63

Negatives

18.3

3.5***

.01

.02

21.6

135

.16

Insurance Clinton Plan Private -1.6 Health Ins. Positives

-17.7

Higher Premiums Cut Medicare Pay More Tax Increases

QUESTION GRAMMAR EFFECTS PG. 23 Table 3. The Distribution of Concepts Accompanying Those Analyzed

Expand Coverage Fact Tax Cut Tax Hike Other

Budget/ Deficit Clinton Plan

Location:

Aid/Chang Medicare

Concept:

None 2 Side Frame

Accompanying Concepts

Access to Action

Health Ins. (3,7) Back/Con

Provide Health Action

1 9

Ins. (43,11) Back/Con Clinton Plan Action

(2,8) Back/Con Private Ins. Action

1 4

(10,5) Back/Con

Higher Prem. Action (15,3)

1 9

3 3

QUESTION GRAMMAR EFFECTS PG. 24 Back/Con

Action

Back/Con

1 18

Cut Medicare 5

(18,14)

Pay More (34,5)

Tax Inc. (19,28)

Payroll (11,2)

Action

Back/Con

Action

Back/Con

Action Back/Con

1 7

1 5

1 2

20 1 2

5 1

QUESTION GRAMMAR EFFECTS PG. 25 Figure 1. The Effect of Question Grammar on Aggregate Response

Action

Consequence or Background

Access to Health Ins. (3,7)

(F=4.3,p=.07)

60.4 67.2

Provide Health Ins. (43,11)

(F=6.8,p=.01)

55.4 46.5

Positives Clinton Health Care Plan (2,8)

(F=.9,p=.36)

37.9 73.7

Private Health Ins. (10,5)

(F=.5,p=.49)

71.8

Positives Overall 53.9

Concept (Na,Nb or c)

Higher Premiums (15,3)

(F=19.5,p<.01)

(F=.5,p=.48)

57 29.9

Negatives

Cut Medicare (18,14)

(F=3.2,p=.08)

41 43.6

Pay More (34,5)

(F=1.9,p<.18)

56.8 49.8

Tax Increase (19,28)

(F=8.3,p<.01)

63.3 37.1

Payroll Tax (11,3)

(F=15.9,p<.01)

41.8

Negatives Overall

57.6

Average Percent Supporting

(F=21.6,p<.01)