JMMR291429 101..104

Journal of Mixed Methods Research http://mmr.sagepub.com

Media Review: Foundations of Multimethod Research: Synthesizing Styles (2nd ed) Manfred Max Bergman Journal of Mixed Methods Research 2007; 1; 101 DOI: 10.1177/2345678906291429 The online version of this article can be found at: http://mmr.sagepub.com

Published by: http://www.sagepublications.com

Additional services and information for Journal of Mixed Methods Research can be found at: Email Alerts: http://mmr.sagepub.com/cgi/alerts Subscriptions: http://mmr.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav

Downloaded from http://mmr.sagepub.com by on June 5, 2008 Š 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

Media Reviews Brewer, J., & Hunter, A. (2006). Foundations of Multimethod Research: Synthesizing Styles (2nd ed). Thousand Oaks, CA: Sage.

DOI: 10.1177/2345678906291429

Journal of Mixed Methods Research Volume 1 Number 1 January 2007 101-104 Ó 2007 Sage Publications 10.1177/2345678906290531 http://jmmr.sagepub.com hosted at http://online.sagepub.com

Multimethod Research and Mixed Methods Research: Old Wine in New Bottles?

One of the first things to notice about this book is that the terms mixing or mixed methods do not appear anywhere on the front or back cover, nor in the subject index. There are good reasons for favoring other terms. A number of authors do not like this terminology because research methods should not be ‘‘mixed’’ but rather should each make their own contribution to a particular research problem. ‘‘Blending’’ or ‘‘synthesizing’’ becomes thus the preferred metaphor. I understand part of the argument but am not sure if words like blending or synthesizing are a vast improvement over the term mixing. Nevertheless, I was interested in why the newly revised edition of Brewer and Hunter’s book on multimethod research from 1989 avoids the now well-established terminology.

On the first page of the preface, the reader is informed that multimethod measurement is equivalent to triangulation and that this is ‘‘the multimethod strategy’s most familiar application’’ (p. xi).

At first glance, it seems that triangulation is either particularly narrowly conceived or that measurement is rather loosely defined. But soon the reader realizes that the scope of multimethod research is quite limited. According to Brewer and Hunter, ‘‘multimethod research takes its name from Campbell and Fiske’s famous 1959 article on measurement validation. . . . The term ‘multimethod’ soon came to imply for many social scientists, including ourselves, both a critique of much social research and a possible strategy for improvement’’ (p. xiii). Groundbreaking as the article indeed was for psychological measurement theory and construct validation, it is rooted in a specific way of conceptualizing and doing research: At the heart of Campbell and Fiske’s multimethod approach is a concern for precise and valid measurement of (psychological) constructs, specifically in relation to convergent and discriminant validity based on a matrix of intercorrelations.

How do Brewer and Hunter extend a narrowly defined quantitatively-oriented measurement theory to multimethod research? The short answer: very cleverly, by, first, narrowly defining social science research; second, suggesting a division of labor among research methods; third, proposing four ‘‘principal research methods’’; and fourth, arguing that they all converge nicely—from initial assumptions to applications and results—within a multimethod research design. The basic premise is that ‘‘different research methods offer possible solutions for one another’s problems’’ (p. xi).

1. With regard to their narrowly defined focus, Brewer and Hunter state that ‘‘to qualify as a scientifically useful mode of inquiry, a research method must, at the minimum, be able to address questions of measurement and theoretical adequacy by providing the information needed to measure variables and test hypotheses’’ (p. 30). Accordingly, ‘‘the purpose of triangulation (as this multimethod approach is often called) is to ease validation which involves comparing various readings of the same or nearly identical social situations. From these comparisons, we infer the level of measurement validity that the measures have attained.’’ Although this is close to Campbell and Fiske’s ideas, it reduces the social science agenda to measurement and hypothesis testing, something that is either very narrow in focus or, if this indeed should define the social science project, rather old-fashioned.

2. To get at the complexity of social phenomena, Brewer and Hunter believe that it is necessary to study their structure, setting, and constituent social processes with different research methods, theories, and data sets. Implicit behind this idea is that the more data, theories, and analyses are 101 Downloaded from http://mmr.sagepub.com by on June 5, 2008 © 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

102

Journal of Mixed Methods Research

employed, the more ‘‘opportunities for comparison’’ toward one single, valid result are provided.

This seems reasonable in principle. But reflecting on the hundreds of data sets, and dozens of theories and ways of analyzing these in research fields such as the structures of social inequality in modern societies, the causes of ethnic prejudice, or the types of human personality, I suspect that more data, more theories, and more analyses are just as likely to yield more confusion than ‘‘valid research.’’ Additional considerations are necessary to enrich monomethod studies, which are not clearly enough presented in this book.

3. Brewer and Hunter state that there are four ‘‘principal methods’’ in the social sciences, which have their particular strengths and should therefore be combined such that weaknesses in one method are compensated by the strengths of another. These four methods are fieldwork, survey research, experimentation, and nonreactive research. There are a number of points that can be raised against this simplified mapping of the social science research landscape. First, it is questionable whether social science research can be divided into these four methods; second, whether the four methods have indeed the capacities attributed to them (e.g., that fieldwork ‘‘gives access to variables and hypotheses that pertain to relatively confined natural settings’’ [p. 30]); and third, whether they may be combined fruitfully across a wide range of topics, theories, data, or findings without further qualifications (e.g., that causal laws inferred from social science experiments can be generalized via surveys).

4. ‘‘The multimethod premise that no method is perfect,’’ so the authors write, ‘‘underscores the need to study sources of measurement error to determine precisely what it is that’s being measured’’ (p. 7). To assess ‘‘the validity of a theory’’ as the principal goal of multimethod research, all four research methods should ideally be used. Yet again, this idea seems logical in principle, but in practice, the fundamental goal of all social science research cannot and should not be reduced to measurement issues and hypothesis testing. Furthermore, it is unlikely that many different data sets, theories, and analyses converge into one valid result. This optimism is reminiscent of times when measurement theory dominated social science research. Brewer and Hunter’s book on multimethod research indeed belongs to an approach that stresses precise measurement and validation. This suspicion is supported by the chapter on causality, in which the main aim for the social sciences is implicit: to identify general causal laws. A review of the reference section confirms that the predominant tenor of this book is rooted in classical measurement theory. Well over 60% of the references used for this book were published before 1980, and only just over 10% of the cited publications relating directly to research methods were published since 1990.

Does this mean that this book is outdated and has no merit? Not at all. I liked its review of classical approaches and examples within multimethod research, which illustrate that a measurementfocused approach goes far beyond statistical modeling and experimentation. It also provides valuable ideas on how fieldwork, survey research, experiments, and nonobtrusive research could combine under certain circumstances and for certain purposes. As such, it is an excellent book and should be part of any university library.

Ultimately, the book fails to address contemporary issues in qualitative and mixed methods research. It is not a book about mixing methods, however, and this is why I was unable to find the terms mixing or mixed methods on its cover or in the index. For what it is—a text on multimethod research as inspired by Campbell and Fiske’s work—this book is an excellent source of information and research ideas.

Manfred Max Bergman Institut fu¨r Soziologie, Universita¨t Basel, Schweiz, Switzerland

Downloaded from http://mmr.sagepub.com by on June 5, 2008 © 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.

Bergman / Media Reviews

103

References Brewer, J., & Hunter, A. (1989). Multimethod research: A synthesis of styles. Thousand Oaks, CA: Sage.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105.

Atlas.ti Software to Assist With the Qualitative Analysis of Data. Berlin: Scientific Software Development. http://www.atlasti.com/ DOI: 10.1177/2345678906291490

Atlas.ti Version 5 is a program for Windows PC that assists with the analysis of qualitative data.

Two or three programs now dominate in this field, and Atlas.ti is one of them. Version 5 was released in June 2004 and while keeping the major aspects of the interface and functionality of the previous version, 4.2, it brought in some key revisions in usability and output to what was already a very powerful program.

The main way in which Atlas.ti assists in qualitative analysis is in its support for coding. In Atlas.ti, the text being worked on appears on the left of the main window, and in the margin area to the right of it, colored brackets can be displayed to indicate (by the displayed name) which lines of text have been coded to which codes. It is easy to see while reading the text what text has already been coded. Atlas.ti thus has a very visual and convivial way of showing both coding and its context.

Texts can be imported into Atlas.ti in .txt, .rtf, or MS Word .doc formats. Unlike most other programs, Atlas.ti also supports the coding of digital images, audio, and video, and a very wide range of different media formats can be used. In the case of images, rectangular areas can be selected and coded. With audio and video, portions of the time line can be selected and coded. At the moment, there is no way of selecting an area of a video image for coding.

One of the strongest aspects of Atlas.ti is its search facility. Searching can be done for text and combined with auto coding, whereby all the finds (and any surrounding text if required) are automatically coded. With careful formatting, this can be used to quickly code answers to open-ended questions in a survey. The other major form of searching is using codes. This enables, for example, the retrieval of text that is coded as X and also coded as Y. Such combinations can be Boolean (and, or, not, xor), semantic (associated with, causes, etc.), or proximity (within, overlap, etc.).

Complex searches can be constructed by combining these terms.

These are some of the significant ways Atlas.ti can be used in qualitative analysis. But how might it be used in mixed methods research? There are two main approaches that the program can support. First, research may be mixed at the level of the overall design; thus, a large-scale survey might identify certain respondents for additional qualitative interviews, or variables might be generated from the analysis of qualitative interviews and then combined with other quantitative data about the same cases. Atlas.ti provides some functions that enable the exchange of data with statistical packages like SPSS. Variable data in Atlas.ti is handled by using ‘‘families,’’ which are collections of documents, memos, or codes. A family can be thought of as a single value of a particular variable as in ‘‘town of origin: London.’’

In the case of a quantitative survey combined with interviews with selected respondents, you might want to import some categorical data from the survey to enable you to establish some basic information about the interviews. Variable data from the survey can be saved in the Comma Separated Values (CSV) format and then imported into Atlas.ti as document families. Several variables

Downloaded from http://mmr.sagepub.com by on June 5, 2008 © 2007 SAGE Publications. All rights reserved. Not for commercial use or unauthorized distribution.