A REVIEW OF SOME RESEARCH PARADIGMS AND PROBLEMS IN SOCIAL-SCIENCE RESEARCH OF INTEREST TO CONTEMPORARY GEOGRAPHY Richard J. Newcomb e
M r. Newcombe is a graduate studen t at Sou th ern Illinois Un iversi ty.
The purpose of this paper is to contrast research methods in use in the Biological and Physical Sciences with those in use in the Socia l Sciences, and to do so in a manner which emphasizes aspects of these methods of interest to Geographers. Attention is directed to the strengths and weak nesses of different research paradigms with respect to prob lems which are characteristic of social science subject matter. Many of these problems take a different form or are absent in the o ld er disciplines in which most of the research types reviewed have their origins. It is therefore appropriate to begin by briefly asserting some of the unities which bind all scientific research into a who le, and to explore some of the special problems which separate social sciences from the other major disci plinary gro upin gs. One can deduce from observation, and from di scussions such as that of Kaplan' that all four of the epistemo logies suggested by Royce2 are used in the construction of research. Without intuition, there would be but blind guides to the conception of resea rch problems, and, perhaps, little recogni tion of the signs frequently necessary to guide the researcher to the emotional fortitude required to complete his project. Without authoritarianism, the researcher would lack the definition s required to begin exa mining a probl em, and, at least initially, he 44
could have no cause for confidence in his intuition . Without empiricism, the researcher might both doubt and conclude, but could he communicate persuasively with other scientists? Without rationalism, his inferences would have the mystical opacity of the utterances of gurus and the sufi. Without logical thought, the scholar could only doubt the beginnings and ends of things, and the incremental growth of scientific knowledge might well be sacrificed. Without logic, empiricism could produce only primitive taxonomic endeavors, and intuition would lack the tools to test the strength of authority. Emphasis upon rationalism in science stems in part from the requirement of communicability in research results which , in turn, has come to require the safeguard of veritibility. The overall purpose of scientific research may be viewed as the improvement of communication by the reduction of uncertainty about the causes of events in the total human environment. A very similar view is held by Weinberg. 3 Special Problems of Social Science
In social sciences, confounding of research may result from sources of error which are either not present in other disciplines, or which are more subtle and important in social science problems and subject material. This is not to say that confounding is not a major problem in all scientific research, but that social science has sp ecial sources of confounding inherent in the material it studies. In addition to the ever-present danger of misjudgment or misreading of data due to unconsciously biased observation by the experimenter, there exists the danger of communication of experimenter bias to subjects, and the possibility of system atic errors produced in subject respon ses as a result of their interpre45
tation of the experimental milieu in relation to their own self images. 4 a nd 5 These effects are called demand characteristics in the literature of social psychology,6 and an extensive literature is devoted to demonstrations of thei I pervasiveness and strength. Texts on experimenta l techniques in psychology devote space to instructions on how such effects may be avoided: double blind conditions, and intricately deceptive experimental design are rapidly becoming the norm in experimental social psychology. Such effects are not limited to laboratory experiments, unfortunately. Conditioning effects between pre and post tests (and between survey and the behavior the survey sought to predict) are better documented than understood. The classic case is the self-fulfilling prophecy, but simi lar effects are demonstrated by pollsters, and again a literature on the role of discussion, counter-attitudinal advocacy, and the so-called " sleeper effect" on subsequent attitudes and behavior is currently expanding. Experimental manipulation can equal ly well contaminate subject material in the physical and biological sciences, but the resultant bias is, hopefully, more subject to detection and control. Certainly the corrective measures emp loyed by non-social science investigators do not result in serious debates concerning the ethical acceptability of further experimentation. Secondly, social scientists have a difficulty which is not unique to their field: the problems to which they address themselves are complex. Multivariate statistical design is the rule rather than the exception . This is frequently the case in biological and applied physical science, but it is perhaps significant that such problems were not attempted during the early period of physical and biological
science when formal theories were developed via atomistic analysis in laboratory experiments. Physical and bio-scientists thus meet complex applied problems with a wealth of wellvalidated and reliably interrelated con structs. Social scientists lack this advantage to a large degree. Finally, social scientists have problems of interpretation which do not arise in other areas. They must observe behavior, but before they can interpret the meaning of thi s behavior, they must test its meaning to the actorsubject, then interpret this ' behavioral intention' as variance around norms which have explanatory utility.7 A body of " theoretical " literature has arisen from these last two problems which represents many different frames of reference . Consistent explanation of a useful sort requires that time and attempted replication sort out the true hypotheses from the false and biased, and that patience demonstrate which explanations are the more useful. Ideally, communication would be facilitied if social scientists in different disciplines could learn each others' terms, and the limitations and intended domains of each others' theories. Much confusion seems to stem from the borrowing of constructs, and illdigested bits of models. The tragedy is that valid hypotheses frequently may be developed from careful reexamina tion of conflicting explanations, and from careful identification of the sources of confounding in differing explanation s of th e same phenomenon. Th e Exp erim ental Paradigm
When laymen think of science, they probably think of " the" method of science as the experiment. Some physical and bio-science method texts treat it as such, but even then a distinction is usually made between laboratory and field experiments. In
addition, one occasionally hears of " natural experiments" in which nature manipulates the subject material, and the experimenter requires merely the foresight to be on the spot with his measuring tools. In teaching experimental techniques, evaluation of statistical analysis is frequently glossed over in a cookbook manner and arbitrary treatments of statistical significance are promulgated. 8 The major stress is placed on the opportunities for anticipating and eliminating confounding in design, and upon the importance of replication. In the physical and biological sciences the laboratory experiment is usually preferred above the field or natural experiment due to economy, and its superior potential for precision and control. The existence of demand charactertistics in social experiments serves to reverse these priorities in behavioral scientists' approaches to experimentation. Field experiments can frequently be contrived so that demand characteristics are minimized (subjects are unaware of the purpose of the experiment) and good measures of typical behavior may be obtained . Often these measures must be obtained in situations where other relevant controls are sacrificed, and field experiments routinely pose difficulties in replication . Natural experiments are hard to come by (nowhere more so than in natural hazard research), and both types can pose ethical problems to the social scientist. Survey Research
As Diesing notes,9 survey methods employ the same statistical controls as do most modern experiments. In addition they provide access to problems involving multi - variate relationships which are not amenable to laboratory experiments. Broadly defined, survey methods are used as much by non -social scientists as by social scientists. To 46
all researchers, they provide several advantages. Careful sampling techniques can minimize control problems, and maximize precision. Election predictions based on carefully stratified samples have been accurate predictors of national behavior for some years. Survey methods can be readily combined with field experiment and case study or participant observer methods. They can provide a wealth of data for analysis and formal modelling with a minimum expenditure of money and effort. Surveys have great diagnostic utility. The results of shotgun surveys employing every conceivable relevant variable can conserve effort, and provide direction in preliminary research efforts. They are especially enticing to the neophyte since one can dream up a few questions in an afternoon, make five hundred copies of the questionnaire and mail them out with a minimum of foresight. A major limitation of their utility in many areas of behavioral research is their susceptibility to demand characteristics. The social scientist has a choice of tools: forced choice and open choice formats . Forced choice formats were developed in tests of maximal performance where the subject's motivation to score in a socially desirable manner is assumed. All that is required in maximal performance tests (such as the Graduate Record Examination or Scholastic Aptitude Test) is that the respondent do the best he can to achieve a high score defined by the rules of the test. In such tests criteria of achievement can be readily constructed from norm groups or arbitrary standards of proficiency. However, the forced choice format such tests employ, if used to measure typical performance, can become subject to several systematic biases. These are frequently referred to as response sets and styles. Response sets include positive and negative faking, and the 47
production of what the respondent perceives to be socially desirable responses rather than his true attitudes. Response styles include a tendency toward acquiescence on true-false type questions, and the deliberate or unconscious embracing or avoidance of extreme responses, for example, on Likert scales, where the respondent chooses a response from the range of five or seven supplied. These biases can be controlled on survey questionnaires if subtests are included to assess thei r strength, or if appropriate norms exist for the instrument in use. Since these procedures are expensive, they are frequently omitted. Open formats such as subjective tests and unstructured interviews reduce set and style biases, but the degree to which they reduce demand characteristics is a very open question. The utility of a projective test or an interview may vary inversely with the sophistication of the respondent. In addition, open choice formats provide unsolved problems of interpretation and evaluation. Many critics feel that such formats invite contamination by the unconscious biases of those who interpret them. The resolution of these prob lems in socia l science use of survey material may follow the same route as attempts to minimize demand characteristics in psychology experiments and result in ethical debates of a serious nature.
Participant Observer and Case Study Methodologies At first glance one might suppose that participant observer methods would be inappropriate in physical and bio-science, but recent work by animal ethologists, such as that of Jane von-Lawick Goodall, is in this category. This is probably the only method where researchers deal with the demand characiteristics of the research design by becoming part of them. The
fundamental similarity of case study and participant observer methods is in their common definition of the object of study as a system, and in an attempt to define their feedback mechanisms and structure by a researcher who becomes part of the system. Weaknesses of this method necessarily include sampling biases and lack of controls which limit external generalization. Much can be done to minimize these problems when variables and measurement techniques are sufficiently well established within the relevant discipline (as, perhaps, in cultural anthropology) so that repeated case studies can provide a mass of comparable data from many cases, and statistical analysis can commence. Lacking this degree of agreement (as do various theoreticians in clinical psychology and psychiatry) the participant observer method can consume much effort and yield unreliable results. These methods are not inherently well suited to the study of complex systems unless they are combined with statistical survey methods (as is becoming the case in animal ethology). Diesing exemplifies the limitations of this method by showing the failure of the participant observer studies of institutional economists in all but simple economies.'o In all cases it appears that successful inference is dependent on extremely thorough preplanning and training of research personnel. These methods are analagous to national espionage systems, and they share the same advantages and disadvantages.
Archival Research Social scientists and non-social scientists (excluding, of course, historians) seem to have a curiously hypocritical attitude toward archival research. In explanation amongst one's peers, archival research is the method
of last resort. Yet it continues to be a major tool for training students. Its disadvantages are well-known: it is time consuming and expensive, it poses sampling problems which are usually insurmountable, experimenter bias is produced in the researcher by such irrelevant sources as style of writing and length of exposition. Objective analysis of archival data can be achieved through content analysis, but this is infrequently employed by students. Samples are roughly representative at best, and cannot be random if the extent of relevant material, or the degree to which it represents the period or work in question is unknowable, as it usually is in practical cases. Its use by students may be justified by its convenience as a way of rapidly brainwashing the student to think about a given subject with a particular bias or biases, but it does little to train the skepticism so prized in mature researchers. We may conclude that archival research is to be avoided unless the sampling problems can be rendered unimportant by the urgency of the need for the information, or by a lack of alternative methods of inquiry.
Formal Methods-Modelling Traditionally, formal theory and models have been much more frequently employed in physical science and bio-science than in social science. Economics is of course major exception to this rule. In any case this generalization is being rapidly vitiated by social science interest in modelling. Definitions of the term model are of such a broad nature" that any universal hypothesis, when fully stated with the set of interpretative sentences which link it to empirical phenomena (this set is usually termed the text of the theory) can be termed a model. 12 A model proceeds from an observed 48
regularity with predictive value toward explanation. This progress occurs as the necessary and sufficient conditions are empirically identified for the model 's predictive capability. A model may equally well proceed from hypothetical statement and empirical identification (which yields explanatory power) toward predicition as the precise logical and mathematical processes are developed through simulation and deduction . Models can be based on descriptive primitives which purport to mirror reality, or they can be based on polar extremes derived from behavioral intentions in an attempt to describe maximal behavior. Many models in political science and economics are of the latter type, and derive their utility not so much from their descriptive power in individual cases as from the plausibility and general acceptance of the maximal behavior they portray, e.g., General Equi librium Analysis in economics. Models of both types can be used for a number of purposes, from the production of formal theory to diagnosis of complex systems. Gilbert White ' s well known model of choice in resource management is more normative than descriptive, but it provides a useful diagnostic tool for policy evaluation, as do many economic theories, provided only that one agrees with the implicit ethical premises. When survey methods can provide a wealth of reliable and comparable information about complex processes, computer sim ulation of the operation of social and economic systems can be usefully applied . A fully identified and testable model constitutes a theory of the system it models. Unfortunately, many of the models put forward by social scientists are of little value. This may be due to either of the following characteristics of their conception and construction. In the 49
first case a model may be testable and predictive, and it may thoroughly explain a system, but if the system is not of general interest at the time of its construction, or if the model can not be readily related to other knowledge about the system, it is premature. In the second case a model may attempt to portray a system in a plausible manner, but if its elements and processes cannot be provided with an explanatory text which demonstrates its domain and testability, it is largely useless. The production of useful models by social scientists seems largely dependent on the abandonment of such senile models. It takes more than a flow cha rt and a few paragraphs of neologisms to produce a productive model. Many of the models in social science literature are little more than arbitrary taxonomic devices which have only suggestive merit.
Summary and Conclusion From this overview, significant differences between social science research and physica l science research can be deduced . The social sciences deal with subject material which introduces subtle bias into research results. A major source of confounding has been the interaction of subjects' behavior with elements of the research procedure outside the control conditions of the research design . In physical and bio-science this type of interaction has been thought to be of less importance. Neither social scientists nor their subjects are as naive as they used to be. Considerable attention has been given to all plausible ways of reducing such systematic errors. Several research designs, both systemsoriented ones and the more formal analytical procedures derived from classical experimental and field survey methods have been created specifically to reduce contamination by demand
characteristics. In general special designs and statistical techniques for ran domizing such errors have vied with attempts at subject deception aimed to ensure measures of typical performances. Only formal methods escape such considerations, but they in turn are dependent on empirical mea sures for assessment of th eir reliability . In summary, attempts to reduce problem s stemming from demand characterictics co ntinue, but perhaps the greatest step forward will be a more general and explicit recognitio n of thei r existence. The other majo r problem area, the co nstruction of research designs capable of explicating multiple causation, is progressing as interest in formal modellin g of systems grows, but such progress is limited by the apparent lack of understanding of the degree to which formal mod els and theories mu st be dependent upon explicit statem ent of their domain, and defini tion of system elements and pro cesses in empirically ver ifiable term s. As these two problems (e.g. dealing with demand characteri stics, and formal mod ellin g of complex systems to deal with complex causation) receive added attention in future years, social scientists may build more lasting and useful theoretical structures comparable in utility to those in the physical and
biological sciences. Finally, as applied and ethologically or ecologically oriented work is produced by physical and bio-scientists, they will continue to face problem s analagous to those desc ribed above, and may be expected to borrow research techniques from soc ial scie nce to meet these problems. Kaplan, Abraham, The Conduct 01 Inquiry, Methodology For Behavioral Sciences , (San Francisco, Calif., Chandler Publishing Company, 1964) pp . 1-32. (2) Royce, Joseph R., " The Search For Meaning", American Scientist, v. 47, #4, December 1959, pp . 515523 . (l) Weinberg, Alvin M., Reffections o n Big Science, (Cambridge, Mass. : The M.I.T. Press, 1967) p. 20. (4 ) Brown, Robert, Explanation in the Social Sciences, (Chicago: Aldine Publish ing Co., 1963). (5) Weber, S. J. and T. D. Cook, "Subject Effects in Laboratory Research: an examinat ion of subject (I )
roles , demand characteristics, and valid inference,"
Psychological Bul/e(in, 1972, v. 77, pp . 273-295. Orne , M. T., "On the Social Psychology of the Psychological Experiment : with pa rt icular reference to de mand characteristics and their impl ications," American Psychologist, 1962, v. 17, pp. 776-783. (7 ) Kaplan, Abraham , Th e Conduct 01 Inqu iry, Methodology lor the Behavioral SCiences, (San Francisco, Calif. , Chandler Publishing Company, 1964) pp . 31-32 . (8 ) Anderson, Barry F., The Psychology Experiment, (Belmont, California : Brooks, Co le Publishing Company, 1971 ) p. 96. ( 0) Diesing, Paul , Patterns 01 Discovery in The Social Sciences, (N.Y.: Aldine, Athe rton , 1971 ) p. 5. (10) Diesing, Paul , Patterns 01 Discovery in the Social Sciences, (N.Y.: Aldine, Athe rton, 1971 ) p. 17. ( II ) Ha rvey, David, Explanation in Geography, (N.Y.: 51. Martin 's Press, 1969) pp . 144-145. ( 12) Brown , Robert , Explanation in the Social Sciences, (Chi cago: Aldine Publishing Co. , 1963) pp. 147-158. (0 )
50