Election Polling Forecasts and by Juan Pardo

Election Polling Forecasts and

Public Images of Social Science*

A Case Study in the Shaping of Opinion among a Strategic Public BY ROBERT K. MERTON AND PAUL K. HATT NEWSPAPERMEN have a t once been among the chief consumers and severest critics of opinion polls. The following study, based on expressions by 107 editors and publishers, examines the impact which the failure of the 1948 election forecasts had on the attitudes toward polling held by this group of opinion leaders. It also explores the degree to which changed attitudes toward polls may be re-

flected in opinions regarding the specific field of market research and the broad area of social science. Dr. Merton is Professor of Sociology at Columbia University and Associate Director of the Bureau of Applied Social Research. Dr. Hatt is Assistant Professor of Sociology at Princeton University and Research Associate in the Office of Population Research.

I and development of science is in part dependent upon the climate of social opinion regarding its nature, past achievements, and future prospects. The prevailing images of science go far to restrain or to accelerate the realization of scientific potentialities. The higher the social repute of the discipline, the more likely it is to recruit talents and to gain widespread support, and the greater, therefore, its later achievements. And closing the circle, the greater its accomplishments, as these are culturally appraised, the higher, ordinarily, its social standing. This is an interlocking system in which the social status of science and its intellectual accomplishments endlessly interact.

T H E GROWTH

*This may be identified as Publication No. A-105 of the Bureau of Applied Social Research, Columbia University. This paper is part of a study of the utilization of social science now being conducted by the Bureau under a grant from the Carnegie Corporation. The Corporation does not, of course, assume any responsibility for any statements made or any point of view expressed in the article. Assistance in the early phases of this study provided by the Columbia University Council on Research in thr Social Sciences is gratefully acknowledged.

I 86

PUBLIC OPINION QUARTERLY, SUMMER 1949

The public images of a science, particularly those current in places of power, are of more than casual concern, then, to those seeking to advance its frontiers. It is with the social sciences as with any other intellectual resource. Prevailing images of social science may invite action which affects its basic support in society. A recent editorial comment in the New York Times on pending legislation for a National Science Foundation, for example, dramatically testifies to the close connection between the images of social science among those strategically placed in the framework of public decision and the immediate support given social science. Under this bill [the editorial states] the social sciences are not to be pursued. Dr. Vannevar Bush and his group, who played a conspicuous part in drafting all the science foundation bills that have been discussed since 1944, seem to feel that research in the social sciences spells political trouble. It does. We have an example in the controversy that has raged for years over compulsory health insurance [sic]. But even with the social sciences omitted the National Science Foundation bill in its present form deserves Presidential approval.

It is here implied that the image of social science as being politically troublesome, whatever the source of this image, at once left its mark. Public images evidently do not stand apart but enter intimately into the course of the development of a science. A HISTORICAL NOTE ON THE STATUS OF SCIENTISTS

This has long since been recognized. What is presently the case with public images of social science was approximately the case with the prevailing images of physical science a scant three centuries ago. There is a closer parallel than may meet the unhistorical eye of those physical scientists who now look askance at what they variously consider the ineffectual, or politically dangerous, or spurious character of social science. It was then that physical science was often greeted with mingled contempt and derision, with charges of obscurantism and unintelligible jargon, with ridicule for its absurd foci of interest (what more ridiculous than Swammerdam's interest in the anatomy of a louse!). It was against this background of mistrust and disesteem in the seventeenth century that Thomas Sprat had occasion to remark the consequences of making science "the subject of men's scorn." "It

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

187

is not to be wonder'd," h e wrote i n his History of the Royal Society (London, 1667, p. 27): "if men have not been very zealous about those studies, which have been so farr remov'd from present benefit, and from the applause of men. For what should incite them, to bestow their time, and Art, in revealing to mankind, those Mysteries; for which, it may be, they would be onely despis'd at last? How few must there needs be, who will be willing, to be impoverish'd for the common good? while they shall see, all the rewards, which might give life to their Industry, passing by them, and bestow'd on the deserts of easier studies?" A n d earlier, Francis Bacon h a d aphoristically generalized this awareness of the interdependence between the advance of science and the cultural appraisal of its worth:

". . . it is enough to check the growth of science, that efforts and labors in this field go unrewarded. For it does not rest with the same persons to cultivate sciences and to reward them. The growth of them comes from great wits, the prizes and rewards of them are in the hands of the people, or of great persons, who are but in very few cases even moderately learned. Moreover this kind of progress is not only unrewarded with prizes and substantial benefits; it has not even the advantage of popular applause. For it is a greater matter than the generality of men can take in, and is apt to be overwhelmed and extinguished by the gales of popular opinions. And it is nothing strange if a thing not held in honour does not prosper."' There is little systematic evidence concerning the sources of the prevailing images of science, either in the seventeenth century or today. But among the greater part of the lay public, it would seem that the authority of physical science rests rather heavily o n the technological achievements which it ultimately has made possible. T h e increasing comforts and conveniences deriving from technology invite the social support of scientific research and promote a generally favorable, though often grotesquely distorted, image of the physical scientist. These technological accomplishments may also testify to the integrity of the scientist and the power of his research, since abstract and difficult theories which cannot be understood or evaluated by the laity are presumably "proved" in a fashion which can be understood by allthrough their technological applications. I t may well be, therefore, that a ready acceptance of the authority of physical science stems Bacon, Novum Organum, Book I, XCI.

I 88

PUBLIC OPINION QUARTERLY, SUMMER 1949

largely from such daily demonstrations of achievement. Were it not for such utilitarian demonstrations, it is possible that the continued social support of science, incomprehensible to most people, would hardly be nourished on faith alone. The reputability of science and its social status in the estimate of the layman may in no small measure require highly visible applications. It is not implausible that the same criteria of evaluation are implicit in popular images of social science. Perhaps the current images of social science are in part shaped by conceptions of their success (or failure) in being put to "practical use." The periodic attacks on social science as having usurped the mantle of science, although it has not the stature to wear it gracefully, may in part stem from the belief that social science does not (and allegedly cannot) realize its claims through practical applications. THE INDISTINCT IMAGE OF THE SOCIAL SCIENTIST TODAY

Just as the seventeenth century physical scientists were driven by the attacks of a Stubbe or Crosse to consider their social status and the public opinions of their work, so under more or less comparable circumstances, present-day social scientists are coming to examine their social status, its causes and consequences. It was precisely when they had some basis in experience for the faith that their work would lead not only to the advancement of knowledge but also "to the well-being of mankind in societies" (as the contemporary phrase had it) that the seventeenth century scientists felt thwarted by insubstantial social recognition and by prevalent misconceptions of what they were about. When men regard their lowly status as unjustified, they come to consider the possible sources of this status. There is now a slowly emerging interest among social scientists in examining their place in society, the images of social science which prevail in the various strata of society, the sources of this public imagery, and the relation of all this to the advancement of that basic knowledge which may now and thereafter be applied to problems of the time. As a result of the earlier reluctance of social scientists systematically to inspect their own status in society,' after the fashion in which they have inspected the role of the hobo and the saleslady, the shaman, 2For a n extension of this observation, see R. K. Merton, "Role of the Intellectual in Public Bureaucracy," Soda2 Forces, May 1945, 23, 405-415.

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

189

foreman and thief, there is little systematic information on this subject. A few initial clues are provided, however, by a survey completed by the National Opinion Research Center in April 1947. The first question to be considered, of course, is the extent to which any image of social scientists is found among the general public. Table I indicates the percentage of ignorance3concerning a series of professions, avowed by a cross-section of the American population. This series includes the professional social scientists, physical scientists, and the free (or learned) professions. TABLE

PERCENTAGE SAYING "DON'T KNOW"

Profession

Sociologist Economist Psychologist Biologist Scientist Chemist Government Scientist Architect Civil Engineer City Welfare Worker College Professor Lawyer Physician Minister Total number of cases (=Ioo%)

Total

EDUCATION OF INFORMANT Attended Attended Attended High Grammar College School .Cchool

4% 3

3 3 3

23%

6 7 7 6

I I

17% I6

42% 40 34 37 I7 I6 I5 I4

3 3 3

489

1177

892

2920t

less than one per cent. 372 currently attending school.

t Includes

3This summarizes the frequency with which informants stated that they knew too little about an occupation to have any opinion of its social standing. *National Opinion Research Center Survey Number 244. This study was made possible by The President's Scientific Research Board, Ohio State University, and the College Study in Intergroup Relations. A detailed analysis of these materials is being made by Cecil C. North and Paul K. Hatt, under a grant-in-aid of the Social Science Research Council.

190

PUBLIC OPINION QUARTERLY, SUMMER 1949

This tabulation suggests that, irrespective of their own education, informants have a sufliciently clear image of the traditional free professions (professor, lawyer, physician, minister) to judge their standing in society. Not so concerning the social scientists. Public images of the applied professions (architect, civil engineer, and social worker) and of the physical scientists (though not the biologist) are more widely diffused through the population than are images of the social scientist^.^ Only the selected group who have attended college can identify social scientists about as readily as those who have attended grammar school can identify the free professions. Although the social scientists are not as familiar to the public as are other scientists and professionals, somewhat over three-quarters of the population affirm that they do have some kind of image of them. Other data from the same NORC study indicate the comparative prestige accorded these several occupations by those who feel capable of making some judgment. Prestige scores were obtained by asking the informant to rate each occupation upon a five-point "scale." These ratings were so combined as to produce a prestige score ranging from a possible minimum of 20 points (the garbage collector actually achieving a score of 35) to a possible maximum of IOO points (with a U.S. Supreme Court Justice reaching a score of 96). The comparative prestige of our select series of professions is shown in Table 2. TABLE

Profession

Physician ScientistX College ProfessorX Government Scientist* Minister

Prestige Score

89 89 88 87

6 This requires some qualification. Not all occupations in this table are of the same degree of specialization, and the social scientists are more closely delimited than such rubrics as "scientist" or "physician." Had the medical specialty of ophthalmologist been included, for example, there would probably have been an appreciably larger admission of ignorance (even if this were translated as "eye-doctor"). An apposite case is that of "nuclear physicist" who could not be identified adequately by 51 per cent of the sample (including 18 per cent of those who had attended college, 50 per cent of those who had benefited from high school training and 74 per cent of those who had never advanced beyond grammar school). With allowance for differences in the degree to which occupations were specified, a difference in the recognition of social scientists and physical scientists among a cross-section of the population seems to remain. 6 Adapted from National Opinion Research Center Survey Number 244.

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE Profession

Chemist Architect Lawyer Psychologist Sociologist Biologist Economist City Welfare Worker

191

Prestige Score

86 86 86 85 82 8I 79 73

*These categories may of course include any of the other academic or scientific specialties listed below. By being thus divested of their specialized reference, each specialty receives a higher prestige score. This is suggestive for the operation of diffuse stereotypes among a lay public. Would the converse be true among a public constituted by one's professional peers? Would the specialist reference be accorded a higher prestige than the generic professional label?

Both the free professions and the physical scientists are ranked materially higher in prestige than the social scientists. The exception of the biologist who is lodged among his social science brethren suggests the possibility that his status is not as honorific as that of other physical scientists because fewer people are acquainted with the esoteric applications of biological knowledge than with the highly visible technologies stemming from physical science. In summary, then, these data suggest that the popular images of the social sciences are neither as well-defined nor as prestigeful as those of other relevant professional fields. However, two salient limitations of these data should be noted. First, this cross-section of the population does not represent the strategic groups whose conceptions of social science significantly and directly affect the functional role of social science in this society; that is, they do not constitute a sample of the decision-makers or advisers who, as we have seen from the editorial on the National Science Foundation, most directly influence the fate of social science. And second, these comparative evaluations do not adequately indicate, in any detail, the content of the numerically prevailing images of the social scientist. At best, therefore, these materials can serve only as a point of departure for an investigation into the nature of public images of social sciences current among a strategic group of decision-makers and opinion-leaders.? "This paper is thus another in a series of inquiries concerning opinion-leaders conducted by the Columbia University Bureau of Applied Social Research. The first of these dealt with the role of local opinion-leaders in the presidential election of 1940,reported in Paul F. Lazarsfeld, Bernard Berelson, and Hazel Gaudet, T h e People's Choice, New York: Columbia University Press, 1948, second edition, Chapters V and XVI. (Footnote continued on following page.)

PUBLIC OPINION QUARTERLY, SUMMER 1949

THE CASE STUDY

This is a minor case study of the impact of a single dramatic failure of pre-election polling upon the images and evaluations of social science among a strategic public in our society. It represents a brief excursion around the edges of the larger and more important problem of searching out the determinants of the images of social science prevailing among diverse publics.' The specific case is the dramatic and abundantly publicized failure of polling organizations to forecast the victor in the 1948 presidential election; the strategic public is comprised of newspaper editors and publishers whose responses to this episode are here examined; the aim of the study is to identify and to analyze, so far as possible, the assumptions, attitudes and mechanisms involved in these responses, and the relation of these to their images of certain fields of social science. The selection of this case for study is premised on the view that crisis situations involving apparent shifts in public images present an optimum occasion for studying the dynamics of those images. Public reactions to the election polling forecasts were widespread, relatively prolonged, and intense. Nor, apparently, was the reaction directed wholly to the polls. There is the possibility that adverse attitudes developed toward election polling may have been generalized and applied to the neighboring field of market research and perhaps to as distant a region as the social sciences. In 1943, a pilot study dealt with types of opinion-leaders in a local influence structure; see R. K. Merton, "Patterns of Interpersonal Influence and of Communications Behavior in a Local Community," in Paul F. Lazarsfeld and Frank Stanton (eds.), Communications Research 1948-49, New York: Harper, 1949. Soon to be published by the Bureau is a study of opinionleaders in a middle-sized city. The study of the 1948 presidential election conducted jointly by Chicago, Columbia, and Cornell Universities in association with the Anti-Defamation League, Columbia Broadcasting System, Elmo Roper, Standard Oil Company of New Jersey, Elmira Star Gazette, and Time, Inc. includes considerable data on opinion-leadership. Most recently, Herbert Blumer has made a strong plea for the study of opinion-leaders; see his paper, "Public Opinion and Public Opinion Polling," American Sociological Review, October 1948, 13, 542554. T o the best of our knowledge, systematic study of decision-makers was first advocated in the writings of Harold D. Lasswell. 8 I t is thus an initial and limited part of a study of the utilization of social science to which reference was made in our first footnote. For a more detailed statement of the problems involved in this study, see R. K. Merton, "The Role of Applied Social Research in the Formation of Policy: A Research Memorandum," and the symposium devoted to a discussion of this memorandum in Philosophy of Science, July 1949.

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

193

The importance of stereotyped public images is of course well recognized, but the ways in which the boundaries of stereotypes expand or contract have been given much less attention. Under which conditions and through which mechanisms are socially stereotyped evaluations applied to successively larger or smaller sectors of experience? What, for example, promoted the radiation of the images of election polling forecasts to the adjacent area of market research or the more remote field of social science? And at which points do we find mechanisms of insulation curbing the radiation of these images? Pre-election polling forecasts have been widely identified with market research in at least two respects. First, the procedures of collecting data appear largely the same, involving the use of brief contact interviews with samples of the population. And second, the major pre-election polls have been largely conducted by commercial organizations whose more serious and abiding interest lies in market research. This twin association of election polling and market research might. be expected to promote the easy radiation of appraisals of the one to appraisals of the other. Attendant upon the failure of the election forecasts, changed attitudes toward the social sciences might also be expected, since some of these sciences have come increasingly to rely upon the techniques of interviewing population samples. If the election episode were taken as conclusive evidence of the intrinsic unpredictability of human behavior, it would presumably at once challenge the standing of social science. Studies of public images may deal with their frequency among the population at large, or among what are here called "strategic publics." These publics may be strategic because of their general position as opinion-leaders or because of their position in the social structure as decision-makers. (The roles of opinion-leaders and decision-makers need not, of course, coincide or overlap. Leaders of opinion, particularly in limited groups, need not occupy positions of decisional importance, nor do all decision-makers have a following whose opinions they shape.) In the case of the election polling forecasts, one such strategic public is comprised by newspaper editors and publishers.' In some measure, 9That the strategic position of this group has not gone unrecognized may be seen from the NORC study of attitudes of newspaper editors toward opinion polling in 1947, as reported in Paul B. Sheatsley, "The Public Relations of the Polls," International lournnl of Opinion and Attitude Research, 1948-49, Vol. z, No. 4.

I94

PUBLIC OPINION QUARTERLY, SUMMER 1949

they are both decision-makers and opinion-leaders. They are, first of all, the men who act both as consumers (through editorial use of polling results) and as middlemen of election polling (by providing the medium through which these polls reach the ultimate consumers, the reading audience). Secondly, they make decisions regarding the use of market research to establish the patterns of readership of their newspapers. And, finally, they are potentially powerful influences in shaping the public images of social science through their published accounts of "newsworthy" scientific developments. Accordingly, a sample of newspaper editors were requested to set down their reactions to the pre-election polls of 1948, and to report their conceptions of the relations between election polling, market research, and several of the social sciences. The questions as set forth in letters to editors are reproduced below.1째 As a guide to your candid judgments of polls and other social research, several points with which we hope you will deal are listed below.

(I) H ave you made any use, either privately or for publication, of opinion polls within the past year? If so, would you name the organizations making these polls? Do you plan any changes in your use of opinion polls? If so, what changes do you contemplate? (2) What was your personal judgment of the dependability of opinion polls, prior to the election results of this year? What did you consider to be the typical margin of error: I per cent, 5 per cent, or some larger percentage? On what basis did you arrive at this conclusion? Did you feel that the polls were so constructed as to take account of the significant factors which might determine the outcome of an election? (3) In view of the episode of November 2, what is your present judgment with regard to the dependability of future election polls? Of polls on opinion generally? Are there any types of problems for which the polling method can be expected to give reliable answers? If so, which? (4) Have the election polls modified your opinion of the social sciencesthat is, has it changed with respect to psychology, sociology, anthropology? If, so, how has your estimate of these fields changed? (5) Has your newspaper made any use of market research within the past year? Have your plans for its use been modified in the light of election poll results? lOThese questions were part of a letter making clear the purpose, auspices, and possible utility of the study. T o avoid the possibly damaging connotations of a questionnaire format at this time, editors were requested to discuss the several points raised by the questions rather than to respond to a predetermined check-list

ELECTION POLLING AND IMAGES O F SOCIAL SCIENCE

195

These letters were sent to a sample of 472 daily newspapers across the nation. The sample was derived from the random selection of communities after these had been stratified by size and geographical region. All newspapers in each community in the sample were included. From the composite Table 3, it can be seen that the replies came from a selected group of the people to whom letters were sent. Clearly over-represented are the North and East, the large cities and the large newspapers. TABLE 3

SIZE OF COMMUNITY

GEOGRAPHIC REGIONS

Region

North and East Midwest Far West South Total (=Ioo%)

Sample

Replies

28% 31 I4 27 472

33% 30 15 22

107

Sample

Replies

I,OOO,OOO or over 7% IOO,OOO-I,OOO,OOO 2 I 25,000-IOO,OOO 30 42 Under 25,000 Total (=Ioo%) 472

Size

13% 37 29 21

107

SIZE OF CIRCULATION

Circulation

250,000 or over IOO,OOO-250,000 50,000-IOO,OOO 25,000-50,000 10,000-25,000 Under ~o,ooo Unknown Total (=Ioo%)

Sample

Replies

7% 8

13% 16 18 I8 18 '7

13 23 38 I

472

107

In other words, those who made their views known to us include a disproportionately large number of the potentially more influential and powerful among this strategic public of editors and publishers generally. As editors of the large, urban newspapers, they are more often in a position to accept or reject the option of publishing public opinion polls, of utilizing market research and of occasionally reporting to their presumably more sophisticated readers newsworthy de-

196

PUBLIC OPINION QUARTERLY, SUMMER 1949

velopments in social science. They are, too, more fully concerned with the status of election polling and market research, as is suggested by their typically forceful and detailed replies, and also by the fact that fully 49 per cent had published election or public opinion polls, a percentage undoubtedly far higher than for newspapers at large. And, insofar as newspapers exert influence and power at all, it is clearly the larger urban newspapers which are more significant than the smaller rural newspapers. Since our major concern is with editors and publishers as a potentially strategic public, in the sense that they help shape public images and decisions regarding polls and social research, it is of some advantage that our informants constitute through selfselection the more articulate, interested, and influential core of this strategic public. Finally, since our objective in this paper is a qualitative analysis of the patterns of response to the election polling forecasts rather than an estimate of the frequency of these patterns of response, the self-selection of these informants does not represent a major liability. IMPACT ON UTILIZATION AND EVALUATION OF ELECTION POLLING

The problems, procedures, and sources of data having been described, it is necessary to establish a framework within which to scrutinize the responses of our informants. Because prior experience with election polls may be an important determinant of the character of images of these polls, it must be included in the framework. And, for the same reason, the editors' evaluations of the polls prior to November 2,1948 must also be considered. TABLE 4

PRIOR UTILIZATION OF

PRIOR EVALUATION OF

ELECTION POLLS

Favorable

Used polls previously Polls not used previously Total

44 15

Unfavorable

3 34

Total

96"

*This total will remain constant for the remainder of the analysis; eleven replies were not sufficiently specific for this purpose,

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

197

As can be seen from Table 4, our informants divide almost equally into those who had and had not published election polls in their newspapers before the November election. In this pre-election period about three in every five had had a generally favorable evaluation of the polls. There is, of course, a close relation between editors' previous judgments of the polls and their use. All but three of the 37 editors who regarded the polls with disfavor had not published them. The three deviate editors adopted a tongue-in-cheek policy of giving the readers what they presumably wanted, despite their own scepticism of the worth of the polls. Reader-interest took precedence over their private evaluations. An editor of a small city daily in the South expressed the policy in these terms: "Let me say at the outset that I have always considered them [the polls] akin to the comic page, or any other entertainment feature in the paper. They make interesting reading but I have never recommended the placing of any bets on their accuracy.'' Among the 59 who favored the polls previously, there was a group who did not utilize them in their papers, either because they did not consider them of sufficient reader-interest or because of budgetary reasons. But, in general, as the table shows, there was an appreciable connection between use and evaluation. How were the attitudes and behavior of editors affected by the episode of November second? Insofar as it led to any change at all, the impact of the erroneous forecasts upon the use of election polls of course resulted exclusively in abandoning their publication. One would scarcely expect the muchpublicized error to convert any editor to the use of the polls. And of the 47 who had published the polls previously, almost half now intended to discontinue their use.'' PRESENT USE

Used now Not used now Total

PRE-ELECTION USE OF POLLS

Used

Not Used

Total

o 49

26 70

The evaluations of the polls proved almost as vulnerable to the impact of the erroneous forecasts as did the editors' practices. Again, 11It cannot be too often repeated that these slight figures do not represent proportionate frequencies of the several patterns of reaction which occurred among news editors at large. We are here concerned only with discerning types of response, not estimating their relative frequency. This holds true for the entire ensuing discussion.

PUBLIC OPINION QUARTERLY, SUMMER 1949

198

not a single editor proved so perverse as to better his opinion of the election polls in response to their apparent failure. The negative judgPRESENT

PRE-ELECTION

EVALUATIONS

EVALUATIONS O F POLLS

-I-

37 22

Total

59 96

ments of the sceptics were uniformly confirmed, and over a third of the others reported having become disillusioned. Moreover, their previous practices with regard to the publication of polls apparently made little difference to this deterioration in favorable attitudes: approximately the same proportion of previous users (17 of 44) as nonusers (5 of 15) report a deterioration in their opinion of the election polls. The decision to discontinue the polls, at least for the immediate future, was largely but not entirely a result of editors developing an PRESENT EVALUATION

f Total

USEDPOLLS BEFORE PREVIOUS EVALUATION

-I27 =7

DIDNOTUSEPOLLS BEFORE PREVIOUS EVALUATION

Total

-I-

Total

unfavorable opinion of the polls. A handful of editors who retained their favorable estimate of the polls nevertheless discontinued their publication on the assumption that they no longer had high readerinterest. That this defection results from the assumed disillusionment of the public rather than their own disillusionment is indicated by one editor who writes, "I am now looking for some way to re-establish public confidence in the polls," and by another who says, "For the time being, we shall not publish any polls but shall wait and see what public reactions are at a later time." The obverse of this response is found among two of the three editors who had always used the polls with reservation and planned to continue them for the same reason as before, because "people are

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

199

interested in them." But the third discontinued their publication on the grounds that "this is a blow even more severe than that dealt by the Literary Digest poll." A comparison between the results found by an NORC sampling of newspapers in 1947 and 1948 with the statements made by the editors who replied to our inquiry is presented in Table 5. TABLE 5 EVALUATIONS OF POLLS BY EDITORS BEFORE AND AFTER THE ELECTIONS PERCENTAGE GIVING FAVORABLE EVALUATION

This sample NORC Sample12

Before November z, 1948

Ajter November z, 1948

61% 64

39%

These figures suggest that the replies secured in this study come from a group of editors whose pre-election attitudes toward polls were more or less representative of those prevailing among editors generally but who are now somewhat more negatively oriented toward the polls than is true of the NORC cross-section. More importantly, perhaps, a larger proportion of the editors whose opinion of the polls changed downward may have been motivated to reply. However, an over-supply of those for whom the election incident appeared as a crisis may be of real advantage in providing more cases for qualitative analysis of how these negative reactions developed. At any rate, it is sufficiently clear that both studies find a major shift from a favorable to an unfavorable evaluation among this strategic public of editors.

PATTERNS OF RESPONSE

The evidence presented to this point suggests that a substantial change in the reputation of election polling among newspaper editors has occurred as a result of the forecasts in 1948. But this finding is only incidental to the chief purpose of this paper which is, after all, concerned with the radiation of these changed attitudes and their impact upon the status of social science. A study of the changes that took l 2Sheatsley, op. cit. The NORC category is comprised by those who selected "good thing" as the appropriate answer to the question, "In general, do you think public opinion polls are a good thing for the country, or a bad thing, or don't they make any difference?"

200

PUBLIC OPINION QUARTERLY, SUMMER 1949

place moves our inquiry toward its objective, however, if the circumstances of the changes are analyzed together with the reasons advanced for the shifts of attitude. In the letters of the editors, these reasons are expressed as specific criticisms of the polls. The criticisms differ not only in substance, but also in depth. By this is meant that certain criticisms were directed at the assumptions of election polling which are so basic that were these criticisms to stand, much of social science would be put into question. Others were superficial from this point of view, pertaining only to limited aspects of election polling alone and providing little basis for radiating into the areas of market research or social science. I. T h e surface level: technical inadequacies. This category refers to the criticisms which were entirely concerned with technique. Although this is described as the superficial level, it does not follow that these criticisms were not cogent, but merely that even if these criticisms were sound, the result would not be destructive of the content or methods of social science. Essentially these are the criticisms which refer to such matters as inadequate sampling, failure to sample to the very day before election, and poorly trained and poorly selected interviewing staffs. On this level of criticism, there was no necessary implication that even election forecasts cozlld not be effective and accurate. The attack was aimed at pollsters and at current practices, not at the feasibility of election research itself. This pattern of response dealt with actual, rather than intrinsic, limitations of election forecasts. 2. The second level: the anpredictability of election behavior. This response-pattern aimed at establishing the impossibility of forecasting an election with sufficient accuracy to be useful. These criticisms stated or implied that the nature of elections was too complex to allow accurate prediction. Typical expressions of this view were to the effect that "people fear to express the way they actually intend to vote," "emotions are at too high a level to secure clear thinking by voters," and that "the effect of habit and last minute factors within the polling booth itself" inevitably makes for error. One editor even credited labor with deliberately misleading interviewers for the purpose of encouraging Republican complacency. Criticisms at this level may, of course, have been combined with objections at the more superficial level. They indicate a distrust of election polling without, however, necessarily doubting the eficacy

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

201

of the polling method for other problems. Some correspondents, for example, stated that while this research procedure could not be relied upon in such "private matters as elections" it could still yield accurate results in dealing with public issues. Another specific basis for excluding elections alone as proper subjects for accurate polling forecasts was the view that in election predictions, a small error was "all that it takes to be completely wrong while on other matters an estimate within five or ten per cent may be very useful." Other fields of research were thus insulated from the radiation of attitudes toward election polling by the conviction that election behavior involved peculiarities not found in other spheres of social behavior. 3. T h e third level: the fugacity of opinion. This level of criticism was reached when an editor asserted that election polling was inadequate or impossible because the attempt to secure reliable tabulations of any opinions was doomed to failure. Opinions were regarded as too fleeting, volatile, or delicate to be summarized in rows and columns of figures. Typical of this kind of conception are such statements as these: "opinions are simply too variable to place any trust in them," or "descriptions of what people actually have done, such things as their buying habits and so on are reliable and useful, but attempts to treat how they feel as statistics cannot be successful." This extends the area forever closed to research by the social scientist. These criticisms were sometimes combined with statements.at either or both of the preceding levels, but they invariably went on to assert that interview data were useful only in dealing with past performances, excluding any predictive value to opinion or attitude research. 4. Root-and-branch: the indeterminism of human behavior. Criticisms of this type were, in point of fact, the least supportable or cogent of all, but from the standpoint of the rationale and the status of social science they are potentially the most destructive. These affirm the unpredictability of human behavior in statements such as these: "the whole idea is impossible because it involves a kind of determinism which simply is not true of human beings," or again, "It is impossible, thank God, to apply any system of units to human beings." This response-pattern was found among those who consider the statistically analyzed uniformities of social behavior as an affront to the integrity of the individual, and who regarded the erroneous polling forecasts

PUBLIC OPINION QUARTERLY, SUMMER 1949

202

as further conclusive evidence of the freedom (unpredictability) of the human spirit. This conception of human behavior would of course argue for the impossibility of social science as well as of election and opinion polling. TARGETS OF CRITICISM

Identified by their targets, these four levels successively enlarge the scope of their criticism from specific techniques through election polling and opinion polling to social science. Using these as types of evaluation, it is possible to learn more about the responses of editors to the election forecasts. The materials are therefore classified in Table 6 in such fashion as to relate the target of criticism both to previous utilization of election polls and to changes in attitudes toward the polls. (It should be noted that each informant is classified in the deepest level of criticism reached in his discussion, e.g., those who questioned the predictability of human behavior are classified in the fourth, social science, category, though they may have raised other criticisms as well.) TABLE 6 TARGETS OF CRITICISM BY EDITORS RELATED TO THEIRPRIOR UTILIZATION AND THEIR EVALUATIONS OF ELECTION POLLING UTILIZATION AND EVALUATION

Use

Previous Evaluation

Present Evaluation

TARGETS OF CRITICISM (1)

(2)

Specific Techniques

Election Polling

(3) Opinion Polling

(4) Social Science

(The absence of the two remaining permutations in this table again registers the hard fact that there are no cases in this sample of editors who were newly converted to election polling after the election.) From the table it is apparent that the characteristics more significantly related to the targets of criticism are the patterns of present and past evaluations rather than their use. This fact, together with the notable consistency in the targets of criticism reached by the editors in the first four rows of the table, makes it possible to identify four types

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

203

of editors' orientation toward the polls, and their respective patterns of response to the erroneous forecasts of the pollsters. A. THE CONSTANT ADMIRERS

Type A, who never wavered in their positive opinion of the polls (although some did not actually publish them in their newspapers), may be called the "constant admirers" of the polls. From these editors, the November failure merely evoked criticisms calculated to tighten up the procedures used by the pollsters. In other words, they maintained their favorable attitudes intact by questioning, not the inherent a'dequacy of the techniques, but their execution by the technicians. An example of this response is provided-by the editor who writes,ls "I accepted them [the polls] along with everybody else. I did not think enough attention was paid in summing up, to the large 'don't know' vote. At any time during the campaign, the 'don't know' votes would have changed the result if a heavy last-minute shift to one candidate developed. But I was still surprised. I'm not trying to establish a reputation on hindsight. "With the changes in method which have been announced, I think election polls will be dependable in the future. . . ."

Another editor, experienced in the use of election polls, also emphasizes the purely extraneous character of errors in the last election forecasts. He writes, "My feeling is that the polls have sought to weigh the significant factors affecting the outcome of an election, and that in substantial part their errors are directly attributable to inadequate or faulty analysis of data, and not to the data itself." Since this restricted criticism of extrinsic rather than intrinsic defects of election polling is the largest category of response patterns in our series and since it is also a pattern which insulates against the radiation of effect into other social science divisions, it may be that the November episode was perhaps not as destructive in its impact upon the status of social science as some thought or feared. This strategic public of editors, at least, was in large part prepared to distinguish between the temporary, even adventitious, technical flaws of the past and the large potentialities of existing and improved techniques in the future. This was the exclusive response of these editors who constituted the constant admirers of the polls. Here, as throughout the ensuing quotations, the italics are inserted by the present authars.

PUBLIC OPINION QUARTERLY, SUMMER 1949 Upon review of all the criticisms aimed at the target of specific techniques, it was found that they covered problems of sampling, adequacy of interviewers, choice of interview questions, handling of the "don't know's," interpretation of data, and failure to sample in the very last stages of the campaign. Regardless of the relevance or adequacy of these as explanations of the polling failure, they do reflect considerable sophistication in matters of polling procedure.14 It is all the more significant, therefore, that none was sufficiently sophisticated in social science to make the appropriate connection between social science and polling. That is, if the capacity of sound social science theory1' to improve the results and practices of polling organizations now in the field were to have been seen by lay publics at all, we should have expected it to occur among this group of newspaper editors. But even those editors who have been sophisticated by considerable experience with polls, and who would not abandon a technique simply because it was honored in the breach rather than the observance, failed to raise this question of the extent to which election forecasts might be improved by drawing upon substantive social science. B. THE DISENCHANTED

If Type A, the constant admirers of the polls, confined themselves l4As may be seen by comparing this array of considerations with those taken into account by the committee established by the Social Science Research Council to track down the sources of error in the polling forecasts. See "Report on the Analysis of Pre-election Polls and Forecasts" in the Public Opinion Quarterly, Vol. 12,No. 4 (1948). 15 Reference is had here, for example, to the familiar distinction between empirical generalizations and analytical theory, between mere extrapolation of expressed intentions to act and consideration of the conditions under which the act overtakes the intent. For the general problem, see R. K. Merton, "Sociological Theory," American Iournal of Sociology, May 1945, 50, 462-73,esp. at 471: "By providing a rationale, the theory introduces a ground for prediction which is more secure than mere empirical extrapolation from previously observed trends. The atheoretic empiricist would have no alternative, however, but to predict on the basis of extrapolation." See also the discerning paper by John Dollard, "Under What Conditions Do Opinions Predict Behavior?", Public Opinion Quarterly, Val. 12, No. 4 (1948)~pp. 623-632. For the specific problems of studying election behavior see Lazarsfeld's preface to the second edition of T h e People's Choice, op. cit., esp. at page xxvi. "First of all we should like to repeat the present study under different political conditions. Are vote decisions arrived at through different processes when the election centers around important issues? In recent presidential campaigns, such as those studied in 1940 and 1944, there have been few issues on which the major parties were split. As a result, party tradition and machine politics have been potent factors in vote decisions. But there is growing evidence that the Republican and Democratic parties are now moving toward sharper conflict on such basic issues as labor legislation. Future presidential campaigns, then, should provide an opportunity to study how attitudes toward specific issues are crystallized, and how these attitudes are related to vote traditions and group influences."

...

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

205

t o criticisms of technique, it was otherwise with Type B, the "disenchanted" group of editors whose attitudes changed from confident approbation to censure a n d repudiation. T h e disenchanted group reacted i n m u c h more destructive fashion. W h a t the constant admirers considered a brief infirmity of the pollsters, the disenchanted regarded as a n abiding defect of the polls. For the first group the technician was temporarily at fault, for the second the technique itself was faulty. I n consequence of this one dramatic failure, they have become thoroughly disillusioned, not only with reference t o election polling, but, indeed, with reference to the general study of opinion a n d attitudes. It is among this group, then, that radiation of attitudes generated by the faulty election forecasts to the wider region of opinion research first becomes detectible. T h e progressive extension of the hostile attitude to wider areas of research can be illustrated by the following not-unrepresentative cases. Case

A Metropolitan Southern Daily.

At first this appears to be a case of slight radiation of effect; initially the election polls alone fail to survive that November day. The estimate of error deserves notice, since the expectation of extreme precision invites subsequent disillusionment. "I had thought the -Poll a very good one and that any margin of error of I per cent or perhaps even 2 per cent might be explained by innumerable factors entering into the question; however, it seems to us that any larger error margin makes them useless. The polls seemed to test the public pulse, and it was their business to so construct the polls that all significant factors were taken account of. As long as the polls were accurate and correct they were useful to us. Since they have proved inaccurate, we consider them no longer useful." But it soon becomes evident that these attitudes radiate, first to the entire realm of political research, and later to opinion research generally. "I do not know the reasons for my belief, but I do feel that polls can no longer accurately judge opinion on anything that has to do with politics. Polls of opinion on other subjects may likewise be just as inaccurate; I do not know any way to check them. W e do know they were proved wrong i n dramatic fashion on November 2, i n the only check possible." Thus, the prior belief that successful election polling forecasts have validated opinion polling results in the erroneous election forecast de-

PUBLIC OPINION QUARTERLY, SUMMER 1949

stroying confidence i n opinion polls generally. The ostensible bankruptcy of the one ensures the bankruptcy of the other. Case 2. A Midwest Metropolitan Daily. The editor begins with a report of his initial confidence in the polls. "You ask my personal judgment of the dependability of opinion polls prior to the 1948 election results. I imagine my opinion, based on personal laziness and an inclination to let somebody else do my job was favorable. I suppose I considered the typical margin of error whatever the poll people told m e it was. I accepted what I had been informed; viz. that after the unfortunate Literary Digest episode of 1936 all the big polls had been revamped and improved along the lines of scientific accuracy." This implication of excessive and misplaced confidence is followed by a rejection of election polling for the immediate future.

"I have no present judgment regarding the dependability of future election polls. I would say-and I am speaking selfishly-that I'd shy away from publication of them by this newspaper at least until the public has had time to forget their bad showing of last November. I don't know how long that will be-maybe four years, maybe eight, maybe twelve. I can say with virtual certainty that I wouldn't publish an opinion poll two years hence in the -municipal elections." Following the faint suggestion that his alienation from the election polls is all the greater as a result of an editor's public commitment to items published in his paper, he extends the range of repudiation to include polls of opinion. "I believe the polling method is more valuable i n factual instances such as finding out how many householders in a certain residence area use refrigerators, etc. I am afraid we have been hoodwinked as to its value i n gauging opinions. I am afraid that all of us have assumed a rigidity of opinion that does not exist. Maybe you, ,will fnd out that public opinion is almost as hard to nail down as mercury." His early faith and confidence in the election polls deceived, this editor abandons all hope for opinion research generally. Only the statistical report of items of possessions remains invulnerable to the impact of that traumatic November episode. Case 3. A Small Far West Daily. This brief excerpt clearly illustrates the outermost reverberations of the loss of original faith in election polls. For one thus disenchanted, the

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

207

fate of all polling was fixed in November 1948. Personal impressions are to validate polling data, rather than conversely. "Have lost faith in election polls and will decline to be guided by them in future unless they coincide with own observations. I am doubtful if the polling method can give reliable answers on any type of problems, unless the poll gets opinions from an extremely large fraction of the population-say, 25 per cent-which is probably impractical." Case 4. A Midwest Daily. This statement is of primary interest in showing forcefully how some editors reacted all the more sharply against the election polls precisely because they themselves had implicitly committed themselves publicly to the adequacy of the polls by publishing them in their newspapers. "We are still subscribing to his service although we are not publishing his articles as yet. We are waiting to see just what explanations there are for his error of the last election and whether they hold water before we publish any further polls of ,his. My personal judgment is that even if his explanations are entirely adequate the average newspaper reader would be inclined to look askance at the newspaper which published any further -polls. I think we have a lot to live down before we can publish the -poll without coming in for a lot of criticism and ridicule on the part of our readers, therefore although we are giving --- the benefit of the doubt by continuing our financial support of his organization, we are not running the risk as yet of offending our readers by publishing his weekly polls. Despite this continued contractual arrangement with the polls, the confidence of the editor in certain types of research appears to have been damaged. "In my position as a newspaper editor, I am acutely conscious of the fact that readers do not trust newspapers implicitly any more anyway, and if we were to proceed blithely in the next election with pre-election polls as in the past I think we would come in for a good deal of derision and I for one hope to avoid that. I think there is some place for polls on opinions generally and on questions where the element of prediction does not enter in, but, as I say, anything with the name of -or -attached to it is a poll to the average newspaper reader and a poll is in pretty low repute right now."

It is clear from these and similar cases that there is a considerable range in the degree of disillusionment which occurred. Piecing these together, the pattern of increasing scope of disenchantment with poll-

208

PUBLIC OPINION QUARTERLY, SUMMER 1949

ing seems to be something like the following. The major impact is seen in the now ebbing or vanished confidence in polls on election behavior -this, it appears, is because election forecasts require relative precision, and the tolerances or margins of error acceptable in the opinion field are too gross for the purpose. The fact that this was the first close presidential election in sixteen years perhaps accounts for failure to consider this before rather than after the event. At any rate, the reaction is one which would induce only weak radiation of effect into the area of social science. Another inference drawn from the forecasting episode by this disenchanted group of editors, and with far more destructive implications for social science, is the notion that elections entail peculiarly strong emotional involvement among the electorate. Presumably this has also been true in the past, but it was not considered as important before President Truman's surprising performance, since almost all agreed that there was no real enthusiasm for him in any quarter. The implication derived from the erroneous forecasts is that the interview method (certainly, in the form of the brief polling interview) is inadequate for dealing with situations of emotional involvement. A further conclusion derived by editors from this episode, challenging the very basis of public opinion research, holds that public opinion is either non-measurable by its nature or that its measurement is meaningless since it carries no necessary implications for subsequent action. As we have seen, this criticism confines the interview procedure of opinion research to studies, in the words of one editor, of "what has been done," or of "habits revealed by past action." Such conceptions either remove prediction from the field of opinion polling or severely limit the sphere in which it can achieve prediction. Another type of repudiation was manifested by those who concluded that not only is election polling inaccurate and opinion polling useless, but both must be considered harmful, inasmuch as they unwarrantably shape public attitudes. This judgment is by no means typical of this group of recently disenchanted editors and probably occurs within a different frame of reference than polling accuracy. It is, however, significant to note that such a shift in attitude was possible as a result of this single dramatic failure of forecasting. The differences between the response-patterns of this disenchanted group of editors, who so sharply changed their evaluation of polling

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

209

and related fields of research, and the response-pattern of the constant admirers, who largely retained their earlier favorable evaluation, are not explained by a study of the characteristics of the papers, at least so far as the data in hand are concerned. Each of the two types come from widely separated regions and from different types of communities. They represent newspapers of different size of circulation. Tentative interpretations must be sought within the response pattern itself. The disenchanted editors appear to.have been somewhat less versed in the assumptions and procedures of election polling than were the constant admirers. (Recall the assumption of one disenchanted editor that polls ordinarily operated with a "margin of error of I per cent or perhaps even 2 per cent," or the remark of another, "I accepted what I had been informed; that . all the big polls had been revamped and improved along the lines of scientific accuracy.") Common to all members of the disenchanted group was the explicit or implicit notion that polls would pick a winner, presumably under any and all circumstances, including elections so close that the proportions of votes for the chief candidates are interchangeable within the twilight zone bounded by the estimates of sampling error. There is, further, little doubt that the pollsters never adequately countered this misconception among this strategic clientele and took a presumably calculated risk when they made flat forecasts about the outcome of the election. Since the disenchanted editors were less sophisticated in the ways and problems of polling than the constant admirers, they were the less prudent in their expectations. And since their less instructed hopes before the event ran higher, their disappointment after the event proved deeper. They plainly had an incorrect image of scientific prediction.'" It was, presumably, the shock attendant upon the great contrast between these excessive expectations and the actuality which generated a reaction sufficiently intense as to permit radiation of adverse attitudes toward adjacent realms of social science.

C. THE SCEPTICAL USERS

We have previously met these three editors who personally distrusted the polls but published them nevertheless (because of readerinterest). Like Type D (the constant critics), the responses of this l a All this indicates the need, if applied social research is to have a sound status, for clearly instructing clients in the expectations which they may legitimately have concerning the kind of results which are possible from any given research,

PUBLIC OPINION QUARTERLY, SUMMER 1949

210

group exhibit no sense of disillusionment, simply because there was no previous confidence to be destroyed. In effect, these sceptics had always viewed the pre-election polls as a house of Hollerith cards ready to collapse under the gentlest breeze, to say nothing of the angry gusts which blew that November day. For the disenchanted editors, the polling misforecasts were proof of confidence misplaced; for the sceptical users, they simply represented a problem situation requiring review of the possibility that the polls no longer held reader-interest. The language of the disenchanted editors, consequently, is often agitated and full of affect, sometimes suggesting a sense of betrayal (recall the phrase, "I am afraid we have been hoodwinked . . ."). The language of the sceptics is cool, matter-of-fact, and calculating. It would be intriguing to learn if the impact of the November episode produced greater defection in the continued use of the polls among the disenchanted, with their sense of betrayal, than among the invulnerable constant admirers, or among the sceptics, attitudinal~~ with their more matter-of-fact perspective. Were there more than a trio of sceptics, this could, in some measure, be ascertained. The crosstabulation in Table 7 exhibits the pattern such an analysis would take, though in view of the paucity of cases it does not, of course, produce acceptable empirical findings. TABLE 7

DECISION O N

EDITORS W H O HAD PUBLISHED ELECTION POLLS

CONTINUED USE

BEFORE NOVEMBER

T h e Constant Admirers'

Will continue polls Will scrap polls

24 3

T h e Disenchanted+

T h e Scepticst

Total * T h e constant admirers, it will be remembered, are those editors whose personal evaluation of the polls remained favorable after the election. + T h e disenchanted are those editors whose personal evaluation was transformed from a favorable to an unfavorable one by the impact of the misforecasts. $ The sceptics are those editors whose personal evaluation was initially unfavorable and remained unfavorable after the election.

In view of the slight handful of cases, research prudence does not

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

211

permit one to call attention to the fact that there was indeed less defection in the use of polls among the sceptical users (33 per centI of 3 cases!) than among the disenchanted (100 per cent of 17 cases!). The constant admirers, whose attitudes were invulnerable to the impact of the episode, largely translated their personal judgment into newspaper practice with only 11 per cent (3 of 27 cases!) abandoning the palls in response to expected reader antagonism. Possibly those who saw in the forecasts the defeat of their expectations reacted more strongly through a change of policy than those who never identified themselves personally with the polls, and regarded them only as an interesting or entertaining "feature" for their readers. D. THE CONSTANT CRITICS

As can be seen from Table 6 (p. 202 above), a sizeable group of 34 editors had never used the polls nor regarded them with favor. And, as may also be seen from Table 6, their responses were polarized about two types of criticism: most of them had never trusted election polling because of objections to specific techniques adopted in these polls, and the lack of confidence among most of the others was based on a deep distrust of opinion polling generally. In a sense, these types of response are counterparts of the responses of the constant admirers and the disenchanted, with the important difference that these judgments were also present before, and not only after, the fact. This suggests that perhaps some of the disenchanted editors are on the way to becoming permanently hostile to the polls, after the fashion of the constant critics. And as for the constant critics themselves, the polling episode seems to have had the effect of providing them with fresh evidence serving to reinforce their earlier attitudes. The criticisms of specific techniques by the constant critics differ in one major respect from those by the constant admirers. They carry with them a heavier onus of blame heaped upon the pollsters for not having corrected defects of method (sampling, interviewing, questionwording, etc.) which they regard as previously known (at least to them). Most presented objections which they felt could have been met rather than posing difficulties which would prevent successful polling under any circumstances. The attachment of blame to the pollsters, a response which differentiated the constant critics from the constant admirers, may be illus-

212

PUBLIC OPINION QUARTERLY, SUMMER 1949

trated by several editors who charged the pollsters ,with "bias" and "prejudice." However, all of these make it clear that this criticism is intended to apply specifically to the instances at hand and not necessarily to the field of opinion research as a whole. Thus, one editor writes: "I have always feared the accuracy of political opinion polls by reason of the bias of many taking them. .. . I believe it is as hard for a pollster not to reflect political bias as for some political reporters, and some of them lean mightily toward their objectives." Another adds: "In our humble opinion, the pollsters will have rough sledding from here on out and should. This does not necessarily discredit intelligent investigation and analysis from facts but it does mean the racket boys will not have the easy pickings they enjoyed in the past." Manifestly, if this strategic public of editors were to have an image of opinion polling as an activity of "racket boys," this might ultimately infect their more or less related image of social research. Apart from these invidious judgments, the content of criticisms of specific techniques by the constant critics is much like that of the constant admirers. One editor states, for example, "Frankly I never had any faith in such polls pertaining to elections due to the fact that the poorer and less fortunate families of our population are never adequately represented in such samples." Another objects that "polls [are] too haphazard. I am not so concerned with their margin of error as with the fact the base was too small for the large conclusions that were projected." It would be instructive to know the frequency of this image of sampling problems among the strategic and relatively informed public of newspaper editors. Some editors drew upon their personal experience to illustrate the alleged sampling inadequacies of the polls. Two editors objected that they did not know of anyone who had ever been polled, and another took the opposite tack: "I happened to be interviewed three times by -poll interviewers, and they, themselves, did not impress me as persons who would be scrupulously careful in their reports." Another pointed to the timing inadequacy, maintaining that the polls should have been continued in the very last stages of the campaign, and adding that those polls which did so, such as the New York Daily News, actually did detect an emerging trend toward President Truman. Apparently, then, the constant critics feel that they have had con-

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

213

firmation of their belief in the technical weaknesses of election polling as this is currently conducted. They seem also to feel that the pollsters should have known of these weaknesses and were remiss in failing to guard against them. It appears not unlikely that this judgment might be found among a considerable proportion of those presently classified as constant admirers, if and when a comparable forecasting failure recurs. Added to these criticisms of specific techniques is the more destructive belief among those who had previously held polling in low esteem that accurate opinion polling is impossible. Although the editors holding this view are in a minority, they still represent a substantial segment of those who made their opinions known to us. The grounds of their disbelief in the possibility of adequate opinion polling vary widely, but they agree in one respect: polls have obviously never been good and this fact has now been made dramatically apparent. Presenting this point of view, one editor states that "we have no faith and never have had any faith in the dependability of election polls. That goes for polls on opinion generally. In our surveys of readers of our newspaper we have found people prone to try to give the answer which they think is pleasing to the questioner." Another expresses a vaguer reason for his lack of confidence: "I have always been skeptical of the dependability of opinion polls. I have never made a careful analysis of this skepticism, but I should say that it is based chiefly upon a doubt that any method of sampling opinion can give a fully reliable measurement of the whole. The concept of the population as units rather than individuals has, in my opinion, a very limited value." Those judgments by editors which do more than simply affirm the impossibility of successful opinion polling agree largely that the reason behind this is the essential "whimsicality" and fugacity of people's opinions, as well as the fact that verbal statements cannot be taken as valid indexes of consequent action. There is, in short, almost no difference in this respect between the constant critics and the disenchanted editors. The major difference between them is probably the fact that less affect is discernible in the responses of the constant critics than in those of the disenchanted. For the more that polls were taken "on faith," the less the nature of research was understood, and the greater was the subsequent reaction against the polling technique.

z14

PUBLIC OPINION QUARTERLY, SUMMER 1949

RADIATION OF EFFECrr INTO h4ARKET RESEARCH

Although the possibility of radiation of adverse attitudes into the areas of market research and academic social science has been suggested, we have yet to consider the evidence bearing on this point. Most significant, perhaps, is the fact that not a single paper of the forty-eight which had used market research had decided to abandon it as a result of the polls' failure to forecast the election. Six editors ctid indicate that their newspapers would modify their use of market research as a consequence of this event, but when compared with the other forty-two users of market research who planned no modification, this amount of radiation of attitude seems inconsiderable. The responses to the questions on market research yield some evidence of how this radiation was curbed. Although the editors were not specifically asked whether there was a relationship between election polling and market research, an opinion on this point was offered by twenty-seven of them. This becomes more meaningful when we note that of the twenty-seven volunteered replies, nineteen held that there was no connection between the two fields. Almost all of the nineteenseventeen of them, in fact-were found among the disenchanted editors and the constant critics. Thus, an appreciable number of those editors who have turned against the election polls, or who had never published them because of a fundamentally critical attitude toward polling, report successful use of market research. In these cases, then, attitudes toward market research had somehow to be insulated from the effects of a depreciation of the esteem in which election polling was held. The large newspapers, particularly, are often obliged by competition for advertising to obtain readership data (as distinct from circulation figures) which are available only through some form of market research. The pressure for insulating this field from repudiation is, then, considerable. From the materials thus far analyzed, at least three modes of insulation are evident. First, some of the criticisms of the disenchanted editors dealt only with election polling, that is, polling was either thought to be impossible due to strotzger than usual emotions involved in elections, or thought to be socially undesirable due to effects it might exert upon political outcomes. One editor enclosed an editorial from

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

215

his newspaper which included the following statement, "No harm can come from the possibility of error in reporting a public preference for white wrappers over green, or geranium soap in oval bars over nutmeg soap in square bars. But on matters of public policy, the consequences of mistakes are more serious." Second, a distinction between habits and opinions (or attitudes) was drawn by many editors as a device for insulating market research from opinion research. In these cases, the validity of opinion data for the prediction of other forms of behavior was challenged. This was expressed by one editor who held that interview data could indicate only what had happened, but not what was going to happen. Third, and in some ways most significant for the problem in hand, it appears that as users of the two services-market research and political polls-the newspapers were in decidedly different roles. As we have seen, several editors expressed concern over the fact that they had acted as "retailers" of the poll products and had thus implicitly vouched for the quality of the "goods" without having adequate knowledge of this quality. The same idea was stated most cogently by an editor who observed that the election polls were used by an ultimate consuming public (including, presumably, both newspapermen and newspaper readers) which was unaware of the strict meaning of the facts they were consuming. He went on to point out that this was not true of market research which was utilized by specialists, particularly by specialists who know how to interpret quantitative data of the type secured by the polling method. The more informed clientele would thus not develop the large and ill-founded expectations common among those lacking first-hand acquaintance with the research field. As a result, this would narrow or close the gap between the expectations of the client and the actual product of the researcher. Market research was not generally expected to do the equivalent of "picking the winner" in a close race but was relied upon for sounder purposes; the political pollsters apparently allowed themselves to be put into the dangerous business of forecasts in situations requiring precise discriminations. Of the twenty-seven editors who volunteered specific comments on the relation between election polling and market research, the remaining eight saw a direct connection between the two, primarily with regard to similar methods of collecting data. Seven of the eight re-

216

PUBLIC OPINION QUARTERLY, SUMMER 1949

sponses were by the constant admirers who planned to continue using the election polls. These were the editors somewhat more versed in the detailed techniques of polling, and consequently the ones who could perceive the points at which it coincided with the techniques of market research. In some measure, all of the foregoing may be considered exploratory evidence in support of the hypothesis that the defeat of the large expectations about the political polls held by some editors led them to abandon the polls, without impairing their evaluation of market research. Among this strategic public at least, market research was effectively insulated from the radiation of adverse attitudes generated by the great forecasting failure. The constant admirers, whose criticisms of polls were on technical rather than on deeper methodological levels, also show a continuing receptive attitude toward election polling. These are the editors who had a moderated expectation of the polls in the beginning, and who were apparently able to see, and willing to make, the connection between polling and market research. Not only do these constant admirers recognize this connection, but significantly enough, the majority of these were the only ones willing to modify their market research plans as a result of the election forecasting failure. In other words, market research was found "not guilty by association with election polling." But market research might be the beneficiary of more careful utilization by clients as a result of some slight radiation of attitudes among those who are knowledgeable in, and favorable to, both forms of research. It should be added, however, that although the possibility of this helpful outcome of the polling episode seems to be present in several responses, it was expressly formulated in only two. One editor writes that his newspaper ". . has carried out market surveys . present expectation is that similar surveys will be made in the future. Of course, lessons learned from the political surveys will be applied to those surveys." Another states, it is planned to continue such projects [market research]. Though the problems involved are different, the methods and practices employed are certain to take into account some of the lessons of the 1948 election." It can be seen, then, that among even those for whom the radiation of positive effects is possible, it is detectible for only a few. In general, market research is, in the minds of these newspapermen, a phenomenon

". . .

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

217

of an order different from election polling, and as such, is effectively insulated from radiation. Any effect of the November episode upon social science as such would be of a rather different character than would be an effect upon market research. Market research and election polling have more things in common, especially when viewed from the perspective of newspaper editors. Both are conducted in part by essentially the same people, both use the same techniques to a considerable extent, and often enough, the immediate consuming publics of both are the media of communication. They are, in a sense, parallel expressions of limited sectors of social science applications and are themselves closely related. RADIATION OF EFFECT INTO SOCIAL SCIENCE

Despite the seeming adjacency of election polling and market research, we found that radiation was slight and that insulation, though partially furnished by ignorance of technical details, was to a considerable extent the result of discriminating between the different roles which each of them played with respec< to newspapermen. But how did radiation operate with respect to social science? In the opening pages of this paper, we noted the relative vagueness of popular images of the major social science occupations. Although the highly selected public of newspaper editors are presumably far better informed than most other groups, the vagueness, and often the apparent absence, of an image of the social sciences remains the most striking fact. Responses to the question concerning attitudes toward the social sciences were more fragmentary, incomplete, and unrationalized and more often wholly omitted than were responses to any other question.'* Of the 107 replies, 26 completely ignored the question or availed themselves of the classic expression of newspaper taciturnity, "no answer." Of the 81 who replied, 72 asserted, in one form or another, that their opinion of the social sciences had not been modified by the November episode; 1 7 The question on this point was advisedly not closely delimited and specific and this may have contributed to the failure to reply. So, too, interest in the social sciences may be less often found among newspapermen than interest in the recent and dramatic failure of the polls or interest in the more immediately relevant field of market research. Many were thus not motivated to describe their opinions of social science. But this is all part of the story, and it is doubtful, moreover, that so great a difference as was observed can be attributed to this basis alone.

218

PUBLIC OPINION QUARTERLY, SUMMER 1949

3 reported a modified opinion and 6 expressed themselves as too ignorant of the field to be able to answer the question.

These figures are too slender to support any large statistically grounded conclusion, nor is that their purpose. The frequent absence of any reported image of social science, or its vagueness when reported, contrasts very sharply with the case of market research. There was little or no hesitation on the part of our informants to identify and to express an opinion with regard to the nature of market research and its relation to election polling. The attempt to study the nature of the images of social science among this strategic public is thus handicapped first by the absence of comment from so many of the editors, and second by its fragmentary character when it does occur. This is of course clearest among the six who stated bluntly that they felt unable to make any judgment. This feeling of incompetency concerning ,social science is well expressed by the editor of a large midwestern daily who writes, "I can't answer . . . intelligently because I don't know what other tools you utilize in your studies of psychology, sociology and anthropology." While professing ignorance, the writer is obviously not entirely unaware, since he has no trouble seeing that the interview procedure is at least one of the tools used in social science. Other responses of this type did not indicate even this much knowledge of the nature and methods of social science. However, the remaining materials do exhibit discernible patterns and do give rise to suggestive problems. First, it is clear that not all of the 72 who denied any such impact of the election forecasts were of a piece. Many, fully 46 of this group, answered with an abrupt and uninformative "no," and about these nothing more can be said. Another seven explained that they saw utterly no connection between the social sciences and election polling. For them, the two were worlds apart, remote and mutually irrelevant. This conception was perhaps most incisively expressed by the publishereditor who wrote, "The election polls have no more modified my opinion of the social sciences than they have modified my opinion of vegetarianism. Should they?" This complete segregation was again affirmed by a midwestern newspaperman: "My opinion of the social sciences has not been changed nor modified by the results of the election polls; nor indeed do I see how it could be." One might wish that the basis

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

219

for this segregation of polling and social science had been spelled out, but whatever its source, it of course militated against any inadvertent radiation of effect from one to the other. Another fraction of the 72 for whom social science remained invulnerable to the November episode-comprising eight editors in allperceived a connection between the two fields but reported that their attitudes toward social science nevertheless remained intact. For them, the forecasting error simply dramatized the need for more social science and for more practitioners to avail themselves of the knowledge presently available. This represents the most favorable image reported by any group of editors since it implies that the social sciences have arrived at the point where it is already possible for them to make some solid contributions to applied research. The election "fiasco" is thus made an occasion for requiring more social science, as in the case of the editor of a metropolitan newspaper: "The polls have not modified my opinion of the social sciences. Obviously we know too little and, therefore, we need further instruction in psychology, sociology, anthropology, etc." Again, it is stated that the pollsters failed to take advantage of sound scientific knowledge, but this in no way casts doubt upon the capacity of social science to produce adequate results if properly applied. (A Southern daily) "There seems to us no reason to change opinions of any of the social sciences. Nothing fundamental has changed. The pollsters did a poor job and need to learn how to do a good one. That seems to be all." ( A West Coast large newspaper) "I see no reason why the flop of the election polls necessarily should reflect on the social sciences. I presume the practitioners of the social sciences reminded themselves once more that the human mind in the mass is an extremely complex subject."

Others made a clear distinction between the social sciences and the arts which draw upon them, thus seeing no discredit for the first in the apparent ineptitudes of the second. (An Eastern daily) "Polls are not a part of social science. Rather they are an art of appraisal and measurement in the field of social science, not affecting the theory or practice of the sciences involved." (A midwest metropolitan newspaper) "But the value of the research in psychology and sociology is not, it seems to me, impaired by such differ-

220

PUBLIC OPINION QUARTERLY, SUMMER 1949

ences [between social scientists]; nor can the inexactness of an allied science-that of measurement of public opinion-which draws on both psychology and sociology, discredit them." Others adopt an historical perspective on the development of science, and among the four editors in this group there emerges an image of the social sciences which sees them as young and undeveloped but promising fields. In this view, there is no inherent impossibility for the growth and development of social science, but rather a feeling of sympathy for a young and struggling infant. If anything, their very immaturity only underscores the need for greater support and sustenance. "I have long felt that their aspiration to become sciences is a worthy one, but that they do not yet merit the name," is the verdict of one editor of a large Eastern daily. Perhaps the kindest variation on this theme is expressed by a Southern editor: "We believe, however, that there are great possibilities in the application of knowledge of the social sciences. Error necessarily must attend the exploratory and pioneering stage of any human effort." Finally, seven of those who report no modification of outlook exhibit the image of social science as somehow an inferior science, or perhaps not properly science at all. In general, this image seems to be closely related to the opinion, examined earlier in this paper, that "human nature" is tricky, unpredictable and unaccountable, and that consequently social science can never be very precise. In this view, social science is tolerated as being academic but not scientific. This position is perhaps best stated by a New England editor who writes, "I was taught that the social sciences are not exact sciences. Believing that, I have held to a healthy suspicion of them, and have applied them sparingly in the conduct of [his newspaper]." And another explains, in much the same terms but with exceptionally rigorous standards of comparison, that "the election polls have not modified my opinion of the social sciences in any way, since I have never thought of the. as exact sciences in the sense that mathematics is." The status of the social sciences as "second-class disciplines" is finally given an operational definition by another editor who states his feeling that no graduate degrees should be awarded in these fields. In the case of market research, it was shown that the possibility of radiation was great. That is, there were a large number of editors

ELECTION POLLING AND IMAGES OF SOCIAL SCIENCE

221

who had clear images of market research and whose reactions to the election polls were sufficiently intense to make an expectation of radiation not unreasonable. That this did not occur seems to be due to a motivated and often informed insulation, successfully keeping the two examples of applied social research separate. In the case of the social sciences, there is also little radiation perceivable, but in part for somewhat different reasons. With respect to social science, the newspapermen have little direct motivation for insulating it from the impact of what was widely felt to be a crucial failure of "the interview method applied to population samples." The fact appears that a substantial proportion of these editors have too ill-defined an image of social science to make the association with the election polls. And among the greater part of those who separated the two, we have only a cryptic monosyllable to indicate that this association was not made. For the rest, though they constitute only a fraction of the whole, there are those who regard the fields as intrinsically unrelated, others who do not wish to visit the alleged sins of the practitioners upon the heads of the scientists, and still others who adopt the long-run view and see social science as a promising youngster to be encouraged rather than chastised. Finally, there are the editors who insulated social science from the impact of the polls only in a pickwickian sense, for they implicitly regarded the erroneous forecasts as only further evidence of the assumed inexactness of these sciences. Since their opinion was confirmed by the event, this last group of seven actually approach fairly close to the three editors who report having reacted negatively against social science as a result of the polling episode. These additional three provide little explanation of their reaction. Instead, they present rather straightforward statements to the effect that the social sciences have, in their judgment, lost prestige. Representative of this view is the statement of a Southern editor, "The election polls to us have materially increased the suspicion of all polls, and of the sciences in which they move." In short, the polling episode seems to have had little effect upon the public images of the social sciences among this group of newspaper editors. This does not seem to be due, as in the case of market research, to the presumably successful demonstrations of utility, but rather to the absence of an image of the social sciences or, when present, to the

222

PUBLIC OPINION QUARTERLY, SUMMER 1949

fragmentary character of the image and to a prevailing lack of interest in the disciplines. In those few instances in which radiation of effect did take place, it operated to convince some of the need for more social science and to reinforce among others an already rather sour view of the social sciences, currently a dimly known and suspiciously viewed area, even to these highly selected members of a strategic public. CONCLUSION

Insofar as the students of public opinion are concerned with the prevailing images and the social status of their research, all this would seem to put the issue squarely before them. The targets of criticism are not infrequently the growing points of science. A continuing and major research program is evidently required to work out and to test the reliability of diverse types of opinion-interviewing; to seek out problems which permit the researcher to establish the orders of relation between opinions expressed in diverse settings and subsequent action; to work out researches which enable tests of social science predictions, rather than flat concrete forecasts based upon extrapolation of observed trends. This would in turn call for a linkage of systematic theory, research design, and empirical observation, rather than a congeries of periodic statistical distributions of answers to designated interview questions. It would call for a consolidation of rigorous methods and of social and psychological theory, much of which might be far more expensive of time, money, and effort than the resources of a commercial clientele could sustain or the budget-keepers in government would permit. It would require recognition by those who see the current promise of social science amply fulfilled in the distant future that this entails intensive, cumulative, and systematic research in the historical present. And as a prerequisite for this program of inquiry, it is needful that prevailing images of social science among the decision-makers in our society be thoroughly critical, moderately expectant, and slightly benevoient. If excessive expectations invite future disenchantment, excessive doubts invite present apathy. Or, in the words of the poet who so often perceived what Freud was later to systematize, Our doubts are traitors,

And make us lose the good we oft might win

By fearing to attempt,