COMMENT O N T H R E E "CLIMATE O F OPINION"

COMMENT O N T H R E E "CLIMATE O F OPINION" STUDIES

BY J A M E S S. C O L E M A N X HE short history of survey research has been marked perhaps less by purely technical change than by change in the uses to which it has been put. From a sociologist's viewpoint there has been one important milestone in the use of surveys, now being iollowed by a second that promises to be equally important.

Initially, surveys and polls were designed to measure a population characteristic through interviews of a sample of the population. T h e first important milestone was passed when, in the hands of analytically oriented researchers, these surveys came to be used to study correlations between characteristics of the population. A survey ceased to be solely a barometer (as it continues to be, for example, in Gallup Polls and some market surveys) and became in addition an analytical and explanatory tool. Used in this way, it has become the most important research instrument of the sociologist, which he uses for a host of purposes: to relate social class to child-rearing practices, to study the causes of voting changes, to explain variations in job satisfaction, etc.

Yet certain survey analysts, chief among them sociologists, felt some discomfort in carrying out such research. For the answers to interview questions, punched on IBM cards and cross-tabulated, gave rise to peculiarly individualistic studies. T h e typical survey analysis inferred causes and processes internal to the individual, simply because the variables being cross-tabulated were attributes of the same individual.

T h e sociologists' dismay with this sort of analysis arose because many of the problems that concerned them as sociologists were not of this kind. They were, instead, problems concerning the functioning of a social system, seen as a set of interrelated parts (not as an aggregate of individuals), or problems concerning the relations between people (not the relations between a single person's attitudes), or problems concerning the effect of a person's social context upon him (not the effects of his own background or attitudes). All these problems have been somewhat "unnatural" for survey research, for the processes they investigate are between people, rather than within a person.

But recently a second milestone has appeared. Under certain conditions of sampling, and with the appropriate problems in mind, some survey analysts have begun to study these three classes of problems.'

1

* T h e author is in the Department of Social Relations at The Johns Hopkins University.

1 The first two problems, the study of social systems and the study of interpersonal relations, are particularly appropriate to the ability of electronic computers to in-

608

PUBLIC OPINION QUARTERLY

I n particular, the last of these, the study of the effects of social contexts, has seen several methodological developments and has resulted in a number of fruitful investigations.2 One of the earliest of these (and one of the least known) was probably as close to the study of "climate of opinion" as any in this symposium. I t was a study by Philip Ennis, completed in 1952, of the 1950 Congressional election in two adjacent Congressional districts. I t showed, among other things, that in one district, where the dominant candidate (Judd, Congressman from Minnesota) made foreign policy a crucial issue in the campaign, the correlations between attitudes and voting diflered from those in the neighboring district.3 Although the papers in this symposium are labeled "climate of opinion" studies, it is probably more appropriate to think of them as falling under the broader heading mentioned above, studies of the effects of social context. For none of the three studies explicitly measures the "climate of opinion," and in only one (Levin's) does the underlying variable whose effect is being studied approximate a "climate of opinion."

T h e three studies reported in this symposium illustrate some of the possibilities, as well as some of the problems, that arise when survey analysts turn to the study of effects of social contexts. First, it is useful to note that this type of analysis is made possible by two departures from usual survey analysis: 1. T h e sample, by fortune or by design, includes interviews from all, or a large sample of, persons in each of several social groups. I n the Davis study, the groups are Great Books discussion groups. I n the Michael paper, they are high school senior classes. I n the Levin paper, they are high schools. Formally, the samples are merely two-stage samples, with two special precautions:4 (a) the units sampled in the first stage are actual social units, rather than purely statistical classes; and (b) the sampling ratio in the second stage is high enough so that the social unit can be reliably characterized. (In all three cases, every person in the units was interviewed, except for unintended losses. This need not be the case, however, as exemplified by the similar analysis in Lazarsfeld and Thielens, T h e Academic M i n d , which characterizes social science faculties on the basis of a sample from each college.)

T o a sampling statistician, this presents a novel aspect, for it introduces into the multistage sample a criterion that leads in the opposite direction from the one he would ordinarily take. He is ordinarily convestigate complex structures, and it ,is likely that these studies will flourish as analysts become proficient and ingenious in using computers.

2 A number of these investigations are referred to above in the papers of this symposium.

3 Philip H. Ennis, "Contextual Factors in Voting Decisions," New York, Bureau of Applied Social Research, 1952, mimeographed.

4 T h e set of ten schools studied in the Levin paper is not a sample of any definable population of schools, but in the other two papers the first stage consists of units sampled according to standard sampling procedures.

COMMENT

609

cerned to reduce clustering, in order to reduce the variance of the population estimate; but here he must increase clustering, to reduce the variance of the estimate of characteristics of the intermediate-level units. This action is necessary because of the second departure from ordinary sampling analysis.

2. T h e individual is characterized not only by attributes taken from his interview, but by attributes of his local unit. Thus half the items on an individual's IBM card may be characteristics aggregated from interviews of the others in his social unit. If these attributes have not been reliably estimated, then they cannot be used in the analy~is.~ Similarly, if the individual is not identified as to his particular social unit, it will be impossible to study its characteristics in relation to his own.

(The examples of otherwise sophisticated studies which neglect this are abundant. One of the best is the periodic management surveys carried out in all its operating companies by American Telephone and Telegraph Company. Although respondents were identified by level and type of job, originally they were not identified by the particular unit of the company in which they worked. Thus the effect of these organizational units upon morale and other attitudes could not be determined. I n A T & T's most recent surveys, such identification has been made, and this type of analysis is being carried out.)

I n short, we may think of this research in relation to the two earlier stages of survey research, in the following way: (1) Descriptive polling is concerned with reliable estimates of population characteristics. (2) Analytical surveys are concerned with characterizing individuals, so that these individual characteristics can be cross-tabulated, and causal inferences drawn. (3) Studies of social context, like those in this symposium, are concerned with characterizing (a) the individual and (b) the social unit in which he finds himself, so that these two kinds of characteristics can be cross-tabulated.

I t may appear from the content of the three studies reported here that the possibility of carrying out an analysis of this sort is quite limited, and restricted to esoteric samples (e.g. Great Books discussion group members). But this view results from a kind of self-imposed narrowness we have developed in conceptualizing study design. For example, in election studies such as the Ennis study cited earlier, Congressional districts can serve as one stage in the sample design, and as an intermediate unit to be characterized; or in surveys like Stouffer's Communism, Conformity, and Civil Liberties, communities can serve as the intermediate units, so that the respondents can be characterized 5 There are several examples in the literature that illustrate how the attention to reducing clustering prevented such analysis, by sampling too thinly in each social unit. Typical is a study in which the author participated, where printshops were sampled, and then men within printshops. T h e shops could not be reliably characterized by aggregating the interview response within them, and as a consequence i t was not possible to study the effects of many elements in the shops :on their members (see S. M. Lipset et al., Union Democracy, Glencoe, Ill., Free Press, 1956).

6 10

PUBLIC OPINION QUARTERLY

by the aggregate of attitudes (i.e. the climate of opinion) around them.

T h e methodological similarities in these papers end at about this point. One important difference among them lies in the number of "cases" in each study. These studies have two units of analysis, rather than merely one, so that it is necessary to examine two "numbers of cases." All three studies have a large number of individuals (1,909 being the minimum, in the Davis paper), but they differ sharply in their number of social units. Davis has 172 discussion groups, Michael 518 high school senior classes, and Levin only 10 high schools. Levin's situation, limited to only ten cases, prevents him from a certain kind of analysis possible in the other two cases: he cannot control for possible spurious relations at the level of the school (as, for example, Davis has done in examining simultaneously three group variables, in his Table 5). With only ten cases, variables at the level of the school are inextricably confounded. Michael, on the other hand, with 518 schools, can carry out a much richer analysis. H e can ask (as he does not do in this short paper) whether it is really the social-class level of the school that affects test performance and college plans, or another attribute of the school correlated with its social class (see, for example, the variables listed in his Table 1).

Another difference among these papers that illustrates typical variations in such analysis lies in the relation between the content of the variables at individual and group level. Michael's and Levin's is the most common pattern: the independent variables are alike in content at the individual and school level. At the individual level in Michael's study, for example, it is family socio-economic status; at the group level, it is proportion of families with high socio-economic status. T h e relative effects of this same variable at these two different levels constitute the analysis.

But Davis's analysis is somewhat more complex. I t includes the common pattern found in Michael's and Levin's analysis (see Table 3, in which the individual is characterized by presence or absence of outside contacts with the group members, and by the proportion in his group with outside contacts). But a large part of the analysis goes beyond this, into a multivariate examination of group variables, with individual independent variables absent altogether (see Table 5).

T h e future development and use of such dual-level (or multiplelevel) analyses as those exemplified here depend principally on the problem focus taken by survey analysts. T h e developments to date have not come from technical breakthroughs, but from asking different kinds of research questions from surveys than we have been used to asking. These papers show that when such questions are asked valuable new kinds of answers can be obtained from surveys. Whether surveys do broaden to obtain such answers depends upon whether survey analysts begin systematically to ask such questions.