Research Exposed HOW EMPIRICAL SOCIAL SCIENCE GETS DONE IN THE DIGITAL AGE
Edited by Eszter Hargittai
INTRODUCTION ESZTER HARGITTAI
Digital media bring social scientists many exciting opportunities, both methodologically and as an object of study.1 They offer the potential for processing data more quickly, for helping find patterns that are less obvious to the naked eye, for accessing information about people’s everyday behavior, for unearthing connections that require significant processing power—and the list goes on. Nonetheless, traditional methodological considerations such as sampling bias remain important aspects of research given that more data do not necessarily mean better data. More is sometimes more of the same bad information and can result in mistaken conclusions if the large data sets represent sampling biases.2 The mere existence of more data is also not helpful if much of the data are not visible to researchers, either because social interactions heretofore accessible to researchers conducting in-person observations on digital media are now restricted or because the data are in the hands of private companies. Indeed, opportunities often come hand in hand with challenges ranging from data quality to data access and ethical considerations. This book brings together essays from scholars doing cutting-edge research both using digital media to study social science questions and asking social science questions about digital media’s increasing importance in everyday life. Its aim is to offer researchers realistic examples (with solutions!) of how empirical social science gets done in the twenty-first century
2 I ntroduction
when so much of what people do concerns mediated communication through digital devices. Contributors explicitly reflect on the behind-thescenes realities of their experiences, sharing details that are rarely included in the methods sections of project write-ups even though they are essential for understanding how research actually gets done. Included here are vivid firsthand accounts of original empirical social science research as it is being done in the digital age. Using these innovative research projects, this volume presents a wide range of methods— some completely novel and others more traditional—in the digital context. Project methods range from data scraping (chapters 1, 2, and 10) to ethnographic observations (chapters 6, 7, and 8), from interviews (chapters 4, 5, and 10) to focus groups (chapters 9 and 12), and from survey recruitment (chapter 3) to ecological momentary assessment (chapter 11), with chapters 9, 10, 11, and 12 reporting on mixed-methods projects. Although books on research methods abound, few of them address the sometimes brutal and often undisclosed realities of collecting and analyzing empirical evidence, whether that concerns tracking down deleted Twitter data (chapter 1) or trekking in the Chilean mountains after a flood to connect with respondents in rural villages (chapter 9). By drawing on lessons learned from over a dozen scholars’ cutting-edge research, this volume addresses methodological challenges researchers face in our digital era. By focusing attention on the concrete details seldom discussed in final project write-ups or traditional research guides, it equips both junior scholars and seasoned academics with essential information that is all too often left on the cutting-room floor. Methods books rarely read like detective novels, but the chapters in this volume can resemble the genre. For example, the authors track down tweets no longer available on the platform (chapter 1), step inside the U.S. Senate Press Gallery to locate a press pass (chapter 6), familiarize themselves with the red-light districts of Zurich to find sex workers willing to talk to them (chapter 5), figure out the social media preferences of sexual and gender minority youth in order to reach them for a survey (chapter 3), knock on the doors of rural village residents to find participants for a focus group study (chapter 9), travel across the U.S. Midwest visiting arts and crafts shows to develop a defensible sampling method for a study of the economic benefits of selling creative goods online (chapter 10), and reach early adopters of technology even while living in a rural college town
3 I ntroduction
(chapter 4). These are just some of the puzzles the authors in this volume solve, showing how research can benefit from creativity, resilience, and patience in addition to thoughtful and careful planning. In some chapters, the biggest challenges concern identifying, approaching, and gaining the trust of potential study participants when simply turning to an existing list of possible respondents to contact en masse is not an option. Such is the case when wanting to interview sex workers to study their security and privacy practices (chapter 5), when attempting to reach sexual and gender minority youth to learn about their health practices (chapter 3), and when figuring out how to get to know homeless people on the streets of Chicago to understand the role of digital media in their lives (chapter 8). In other studies, the main challenge is making sense of large amounts of data (think millions of observations) when forces are working directly counter to your efforts (chapter 1) or determining what counts as a relevant case when there is no established consensus on the criteria (chapter 2). But small data can also offer plenty of challenges, as shown when a deep dive into a single Facebook post by a study participant supports the continued importance of in-person observational context when analyzing online content (chapter 7). Chapters 6, 7, and 8 use ethnographic methods to highlight what it means to observe people in person when many of their interactions with others happen on devices whose content may not be as readily accessible to the in-person observer as was traditionally the case. In chapter 8, in-person observations and conversations with people on the streets of Chicago supplement online interactions, while in chapter 6, unexpected data on sites that are extremely difficult for researchers to access, such as political venues in Washington, DC, are shown to be important. In chapter 7, the other side of digital media’s challenge to ethnographers—the still relevant contributions to a project from in-person observations, even if some of the focus is on people’s mediated communication—is highlighted; for example, the researcher interpreting a single social media post can benefit greatly from spending time with respondents in person to get to know their milieu and everyday interactions. Several chapters reflect on the complexities of relying on multiple methods within one study, which is an excellent way to triangulate data but can multiply the number of challenges researchers face. The authors use both surveys and interviews (chapter 9); rely on surveys, interviews, and
4 I ntroduction
automated social media data scraping (chapter 10); use surveys, automated data collection of participants’ media environment, and focus groups (chapter 11); and rely on observations, interviews, and content analysis (chapter 12). Many of the chapters reflect on how important it is to think about case selection carefully, with chapter 4 making this the central focus in its examination of qualitative methods when studying people’s use of mobile media. Because the writing of a chapter for such a volume is rather unusual compared to much of the academic writing scholars do, most of the chapters went through heavy editing. For their patience and openness to addressing my many questions and comments, I am deeply grateful to the contributors to this volume. I suspect some of them did not fully internalize what they were signing up for when they agreed to contribute, but all were great sports about engaging with my feedback. Thanks to this exceptional backand-forth, the volume offers a wealth of extremely accessible and engaging information about how to conduct numerous types of research studies from collecting automated bursts of media consumption (chapter 11) to interviewing homeless people (chapter 8) and sex workers (chapter 5), from thinking through sampling for qualitative studies more generally (chapter 4) and for the study of crafters in particular (chapter 10) to reaching marginalized teens on social media (chapter 3), from collecting in-person data in remote rural villages (chapter 9) to offering a computer and internet training program to low-income African American adults (chapter 12), and from solving the mysteries of Twitter bots (chapters 1 and 2) to learning about journalists’ practices (chapter 6) and Harlem youths’ daily lives (chapter 7) in the digital age. Whether directly related to the reader’s research topic or methods, the chapters offer important insights into how empirical social science research can be—indeed should be—both innovative and rigorous when dealing with the opportunities and challenges offered by digital media. The chapters are organized as follows. Chapters 1 and 2 concern largescale analyses of Twitter data, although the focus in both is more on identifying the relevant data than on describing the analyses themselves. Chapter 3 also focuses on social media but from the angle of using it as a tool for recruiting participants in a survey study. Chapter 4 is concerned with sampling and how traditional efforts to avoid biases must still be made in qualitative studies of digital phenomena. This is followed by chapter 5, which details efforts to recruit sex workers for a study about privacy and security
5 I ntroduction
concerns without biasing the selection against or toward participants who are particularly anxious about or, conversely, agnostic about their privacy and security. Chapters 6, 7, and 8 report on ethnographies, while chapters 9, 10, 11, and 12 present mixed-methods studies. While it is unlikely that any one researcher will use all of the methods covered in this volume, it is essential that all researchers understand how studies using different methods get done and what quality considerations must be kept in mind. If we are to be able to converse with our colleagues and evaluate papers using methods different from our own, we must appreciate both the opportunities and the challenges of the myriad of approaches available to scholars studying digital media. NOTES 1. Christian Sandvig and Eszter Hargittai, “How to Think About Digital Research,” in Digital Research Confidential: The Secrets of Studying Behavior Online, ed. Eszter Hargittai and Christian Sandvig (Cambridge, MA: MIT Press, 2015), 1–28. 2. Eszter Hargittai, “Potential Biases in Big Data: Omitted Voices on Social Media,” Social Science Computer Review 38, no. 1 (2018): 10–24, https://doi.org/10.1177 /0894439318788322;
“With candor and clarity, contributors to this invaluable collection describe the newest research pathways as well as the tangents and hurdles that arise along the way. This revelatory book should be required reading for all social researchers and for anyone seeking deeper insight into the trailblazing scholarship that is making headlines and illuminating contemporary society.” Alondra Nelson, Institute for Advanced Study, president of the Social Science Research Council
“Eszter Hargittai’s work places her without a doubt at the top of the field of internet research methods. For this edited volume, she has convened a group of experts who together bring alive the extraordinary opportunities and thorny challenges facing those using and studying digital media as serious researchers. Hargittai’s commitment to thorough work and exciting scholarship makes this study of methods a must-read for the internet-era researcher.”
Hargittai, ed. Research Exposed
Research Exposed offers in-depth, behind-the-scenes accounts of doing empirical social science in the era of digital communication. Through firsthand descriptions of innovative research projects, this volume shares lessons learned from over a dozen scholars’ cutting-edge work. These candid accounts describe what can go wrong when pioneering new genres of research and how such difficulties can be overcome, providing both big-picture reflection and actionable advice.
John Palfrey, president of the John D. and Catherine T. MacArthur Foundation
Claes de Vreese, University of Amsterdam, president of the International Communication Association
Eszter Hargittai is a professor and holds the Chair of Internet Use and Society at the Institute of Communication and Media Research at the University of Zurich. She is the editor of Research Confidential: Solutions to Problems Most Social Scientists Pretend They Never Have (2009) and coeditor of Digital Research Confidential: The Secrets of Studying Behavior Online (2015).
COLUMBIA UNIVERSITY PRESS / NEW YORK CUP.COLUMBIA.EDU
Printed in the U.S.A.
HOW EMPIRICAL SOCIAL SCIENCE GETS DONE IN THE DIGITAL AGE
COLUMBIA
ISBN: 978-0-231-18877-7 Cover design: Lisa Hamm
HOW EMPIRICAL SOCIAL SCIENCE GETS DONE IN THE DIGITAL AGE
“Research Exposed is a gem. It is a timely and important collection. It pushes our collective thinking about methods and designs, and, by implication, the substantive conclusions we can draw. It is inclusive and comprehensive and the contributions are strong, reflective, and honest, provided by a stellar lineup. Bravo!”
Research Exposed
Edited by Eszter Hargittai