This module serves as an introduction to the course and to the subject area of risk, crisis and disaster management, and it is also a conceptual tool box for the rest of the course. In particular, it introduces a range of theoretical perspectives on the concepts of risk and crisis such as how risk is assessed and managed. The overarching aim of the module is to identify different perspectives and examine the extent to which they inform practice and ultimately to lay a foundation upon which future modules will build.
MSc in Emergency Planning Management
Module 2 Managing Risk and Crisis
Module 3 Research Methods in Risk, Crisis and Disaster Management This Module aims to provide students with comprehensive knowledge and understanding of methodological issues in investigation studies research. The Module introduces students to research methodology on both a theoretical and practical level. Students are encouraged to analyse critically the process of social scientific enquiry and to examine the relationship between research problems, theoretical perspectives and methodological approaches.
Module 4 Case Studies of Crises and Disasters In this module a number of case studies of crises and disasters are examined. The case studies act as heuristics ‑ vehicles for exploring some of the issues and concepts introduced in modules one and two. Such issues include the impact of personality on crisis and disaster management, the influence of cultural factors and national preferences on crisis and disaster management techniques, and the impact on disaster investigations of paradigmatic interpretations of evidence. The rationale for the module is that important lessons can be learned from the detailed, objective analysis of past crises and disasters. The unit also provides an insight into the politics of the 1974 Health and Safety at Work Act, which set up the United Kingdom’s Health and Safety Executive, and into subsequent legislation on the regulation of developments close to hazardous complexes.
Module 5 Models of Risk and Crisis This module addresses the possibility that risks, crises and disasters may be understood in different ways by different people. An air crash, for example, may be understood primarily as a potential blow to profitability by an aircraft manufacturer, as a case for investigation by the relevant police service and national accident investigation bureau, as a destabilizing influence on the stock market by brokers and investors and as a human tragedy by the tabloid press (for whom disasters provide many column-inches of material) and relatives, partners and friends of the victims. Thus the same event may be ‘constructed’ or experienced differently by different parties. This module examines how parties with different ‘investments’ (reputational, financial, emotional etc.) in crises and disasters may experience them in quite different ways.
Module 6 Emergency Planning Management This module looks at the ‘front line’ management of risks, crises and disasters. The emphasis is on practical risk, crisis and disaster management, from risk assessments produced by Britain’s Health and Safety Executive to the factors that need to be considered by emergency planners when drafting an evacuation plan. The module aims to be as eclectic as possible, including, for example, a unit on the identification and management of post-traumatic stress disorder.
The course material is and remains the property of the University (and must be immediately returned to the University upon request at any time) and is either the copyright of the University or of third parties who have licensed the University to make use of it. The course material is for the private study of the student to whom it is sent and any unauthorised use, copying or resale is not permitted. Unauthorised use may result in the course being terminated. The course material was created in the academic year 2011/2012 Civil Safety and Security Unit • University of Leicester • 14 Salisbury Road • Leicester • LE1 7QR
RESEARCH METHODS IN RISK, CRISIS AND DISASTER MANAGEMENT
In this module some contemporary debates about security are explored. It brings together broad developments in theories of risk in the social sciences with risk issues of relevance to security managers. It also examines the relationship between these different perspectives on risk and a general theory of security. An attempt is made to highlight the relationship between the theory and practice of risk management and security.
MODULE 3
(updated August 2011)
Research Methods in Risk, Crisis and Disaster Management EMERGENCY
EMERGENCY
Module 1 Theories of Risk and Crisis
MODULE 3
MSc in Emergency Planning Management
PLANNING
MODULE 3 RESEARCH METHODS Copyright The course material is and remains the property of the University (and must be immediately returned to the University upon request at any time) and is either the copyright of the University or of third parties who have licensed the University to make use of it. The course material is for the private study of the student to whom it is sent and any unauthorised use, copying or resale is not permitted. Unauthorised use may result in the student’s registration being terminated.
This course material was originally created in the academic year 2001/2003 and an updating review was conducted in 2011.
crisis
PLANN NG
MSC IN EMERGENCY PLANNING MANAGEMENT
2
crisisdisaster RISK MANAGE
risk
EMERGENCY
disaster
riskcrisis MANAGE RISK
Table of Contents
2 Unit Two: Research Ethics............................................................................ 2-3 2.1 Aims and Objectives of this Unit................................................................................ 2-3 2.2 Introduction.............................................................................................................. 2-3 2.3 Values....................................................................................................................... 2-4 2.4 Ethics...................................................................................................................... 2-16 2.5 Main Points............................................................................................................. 2-25 2.6 Study Questions...................................................................................................... 2-27 2.7 Appendix – BSA: Statement of Ethical Practice (1993).............................................. 2-28 2.8 Bibliography............................................................................................................ 2-33 3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13
Unit Three: Reviewing the Literature......................................................... 3-3 Aims and objectives of this Unit................................................................................. 3-3 What is a Literature Review?...................................................................................... 3-3 Why Write a Literature Review?................................................................................ 3-4 What is Included in a Literature Review?.................................................................... 3-6 What is Left Out of a Literature Review?.................................................................... 3-8 What Issues Should a Review Address?...................................................................... 3-9 Narrative Review.................................................................................................... 3-11 Meta-Analysis.......................................................................................................... 3-12 Conclusion.............................................................................................................. 3-14 Main Points............................................................................................................. 3-15 Guide to Reading.................................................................................................... 3-15 Study Questions...................................................................................................... 3-16 Bibliography............................................................................................................ 3-16
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
1 Unit One: Getting Started: An Introduction to Research.......................... 1-3 1.1 Aims and Objectives of this Unit................................................................................ 1-3 1.2 What is Research?..................................................................................................... 1-3 1.3 What is Scientific Research?....................................................................................... 1-4 1.4 Science, Non-science and Nonsense......................................................................... 1-5 1.5 Empiricism and the Empirical Approach to Science..................................................... 1-7 1.6 Rationalism............................................................................................................... 1-7 1.7 Methods in Social Science Research........................................................................... 1-8 1.8 Triangulation............................................................................................................ 1-13 1.9 Threats to Valid Inference........................................................................................ 1-14 1.10 Concluding Thoughts.............................................................................................. 1-16 1.11 Main Points............................................................................................................. 1-17 1.12 The Following Course Units.................................................................................... 1-17 1.13 Study Questions...................................................................................................... 1-19 1.14 Bibliography............................................................................................................ 1-19
PLANN NG
DISASTER
3
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 4
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk 4 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9
Unit Four: The Social Context of Research................................................. 4-3 Aims and Objectives of this Unit................................................................................ 4-3 The Research Setting................................................................................................. 4-3 Human Diversity and Social Research........................................................................ 4-8 Political Issues in Social Research.............................................................................. 4-17 Conclusion.............................................................................................................. 4-19 Main Points............................................................................................................. 4-20 Guide to Reading.................................................................................................... 4-20 Study Questions...................................................................................................... 4-20 Bibliography............................................................................................................ 4-21
5 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12 5.13 5.14 5.15 5.16 5.17
Unit Five: Research Design.......................................................................... 5-3 Aims and Objectives of this Unit................................................................................ 5-3 What is Research Design?.......................................................................................... 5-3 Variables................................................................................................................... 5-4 Benchmarks of Quality.............................................................................................. 5-6 Types of Research Design........................................................................................ 5-11 Comparative Research............................................................................................ 5-16 Historical Approaches.............................................................................................. 5-17 Applied Research.................................................................................................... 5-18 The Integration of Theory....................................................................................... 5-24 Measurement.......................................................................................................... 5-27 Sampling Procedures............................................................................................... 5-28 Data Gathering Techniques...................................................................................... 5-32 Conclusion.............................................................................................................. 5-33 Main Points............................................................................................................. 5-33 Guide to Reading.................................................................................................... 5-34 Study Questions...................................................................................................... 5-34 Bibliography............................................................................................................ 5-35
6 Unit Six: Data Gathering Techniques.......................................................... 6-3 6.1 Aims and Objectives of this Unit................................................................................ 6-3 6.2 Introduction.............................................................................................................. 6-3 6.3 Observation and Ethnography................................................................................... 6-4 6.4 Interviews............................................................................................................... 6-10 6.5 Life History Research.............................................................................................. 6-18 6.6 Surveys................................................................................................................... 6-22 6.7 Documentary Research........................................................................................... 6-31 6.8 Secondary Analysis.................................................................................................. 6-34 6.9 Content Analysis..................................................................................................... 6-38 6.10 Conclusion.............................................................................................................. 6-42 6.11 Main Points............................................................................................................. 6-42 6.12 Guide to Reading.................................................................................................... 6-43 6.13 Study Questions...................................................................................................... 6-43 6.14 Bibliography............................................................................................................ 6-44
disaster
riskcrisis MANAGE RISK
7 Unit Seven: Qualitative Analysis and Presentation..................................... 7-3 7.1 Aims and Objectives of this Unit................................................................................ 7-3 7.2 Introduction to Qualitative Analysis............................................................................ 7-3 7.3 Basic Principles for the Analysis of Qualitative Data..................................................... 7-6 7.4 Analysing Transcripts................................................................................................ 7-11 7.5 Analysing Observational Data ................................................................................. 7-13 7.6 Pictorial Images....................................................................................................... 7-14 7.7 Content Analysis..................................................................................................... 7-14 7.8 Analysing Documents.............................................................................................. 7-14 7.9 Secondary Analysis of Qualitative Data Sources........................................................ 7-15 7.10 Looking for Meaning – Developing Theory.............................................................. 7-16 7.11 Presenting Your Theory........................................................................................... 7-17 7.12 Advantages and Disadvantages of Qualitative Research............................................. 7-18 7.13 Conclusion.............................................................................................................. 7-18 7.14 Main Points............................................................................................................. 7-19 7.15 Guide to Reading.................................................................................................... 7-20 7.16 Study Questions...................................................................................................... 7-20 7.17 Bibliography............................................................................................................ 7-20 Unit Eight: Statistical Tests.......................................................................... 8-3 Aims and Objectives of this Unit................................................................................ 8-3 Descriptive Statistics.................................................................................................. 8-3 Inferential Statistics.................................................................................................. 8-14 Factor Analysis......................................................................................................... 8-28 Linear Regression and Structural Equation Modelling................................................ 8-28 Main Points............................................................................................................. 8-30 Study Questions...................................................................................................... 8-31 Bibliography / Suggested Reading............................................................................. 8-31
9 Unit Nine: Writing a Research Proposal and Dissertation......................... 9-3 9.1 Aim and Objectives of this Unit.................................................................................. 9-3 9.2 General Issues........................................................................................................... 9-3 9.3 Research Proposals................................................................................................. 9-11 9.4 Dissertations........................................................................................................... 9-13 9.5 Empirical Research.................................................................................................. 9-15 9.6 Writing Up.............................................................................................................. 9-21 9.7 Common Problems................................................................................................ 9-27 9.8 Supervision............................................................................................................. 9-28 9.9 Dissertation Regulations and Requirements.............................................................. 9-28 9.10 Main Points............................................................................................................. 9-30 9.11 Study Questions...................................................................................................... 9-31 9.12 Bibliography............................................................................................................ 9-31
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
8 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8
PLANN NG
DISASTER
5
crisis
PLANN NG
MSC IN EMERGENCY PLANNING MANAGEMENT
6
crisisdisaster RISK MANAGE
risk
EMERGENCY
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 1 Getting Started: An Introduction to Research
disaster
riskcrisis MANAGE RISK
1 Unit One: Getting Started: An Introduction to Research 1.1 Aims and Objectives of this Unit The fundamental aim of this Unit is to offer an introduction to research and to introduce students to basic issues such as how to choose suitable research topics and the importance of issues such as ‘validity’ in research. After reading this Unit students should: • understand the basic notion of research as a means of answering appropriately formulated questions in a valid way; • appreciate that there are different types of data that researchers are interested in, and that there are a range of different methods for carrying out research; • understand why the perfect research investigation is never possible.
Research is easy. It is something that we all do all of the time. Suppose you decide that you need a new television. There are hundreds of different models available to you, but you do not make a random selection. Instead, you make a systematic decision about which is best for you. You decide how much you want to pay and what list of features you require, and then you examine the range of models available to you. You will make a whole series of complex decisions relating to the relative cost and benefits of each feature that is available, and if you are anything like me, at the end of the day you will select a TV that costs more than you meant to spend and has a host of features that you will never use. This process of investigating the options is clearly one of research. Some other examples of ‘everyday’ research might include the processes involved in making the following decisions: • deciding who to talk to and who to avoid at a party; • deciding which political party to vote for; • deciding how to get from place A to place B; • deciding what to prepare for dinner. In each case we will make this decision, not at random, but after considering and weighing up a
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
1.2 What is Research?
PLANN NG
DISASTER
series of observations. These observations are specific to the problem under consideration – it makes sense to examine the contents of your fridge when deciding what to have for dinner, but this observation is probably superfluous when deciding who to talk to at a party (I realise that I am making certain assumptions about your social life here!). Thus you research each question – you make observations and collect the data from these observations. These data are then weighed and a decision is made on the basis of them.
1-3
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 1-4
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
So, if we all do research all the time, then what is the big deal - why do you need to read this Unit? The answer lies in the distinction between general research and Scientific Research. Scientific Research is a systematic process designed to help us make accurate decisions. A more scientific approach to television purchases might even save me some money!
1.3 What is Scientific Research? The difference between the lay person and the scientific researcher is that the latter employs objective, systematic investigation with analysis of data in order to discern what actually is the case rather than a patchwork of likes and dislikes, rules of thumb, analogy and prejudice, half-truths and old wives’ tales. (Burns, 2000: 4) Before we consider what scientific research is, we should first consider what it is not. Research is not: • complicated • difficult • technical • boring • something done by people (specifically men) wearing white laboratory overalls, clutching clipboards and standing next to machines with flashing lights and those spools of tape that whir backwards and forwards. Scientific research is research that is conducted by following the principles of empirical verification. This is simply a description of a process of research that is based on direct observation where these observations are designed to establish the truth or otherwise of some proposition. For example if I want to establish if it is the case that smoking causes lung cancer, then I conduct a series of carefully planned observations designed to test this proposition. These observations will include careful examination of the levels of lung cancer in smokers and non-smokers. It is this process of careful and unbiased observation of the real world that is seen as the only route to a true understanding. You might find some aspect of some scientific research challenging, but that does not mean that the process is inherently difficult. Social science students typically dislike statistics – anything with numbers is regarded as gratuitously difficult. But not all scientific research involves complicated statistical analysis. Good scientific research can be very difficult or very simple. What distinguishes the good from the bad is not the complexity of the process, mathematics or machinery involved, but the quality of the thought behind it. Good research involves good thinking, and with a little training we are all capable of that.
disaster
riskcrisis MANAGE RISK
1.4 Science, Non-science and Nonsense If research is something we do all the time, then why do we need special scientific methods? The answer to this question lies in the fact that people make rather poor researchers because we tend to be affected by a series of biases and other effects that mean that we are likely to make inaccurate observations and draw invalid conclusions.
1.4.1 Relying on Common Sense
Perhaps we should make a virtue out of this and regard being told that what we have discovered is common sense as praise for a simple and clear exposition of our position. I once heard a social scientist say that he knew that he had a good theory when, on explaining it to his mother she replied ‘didn’t you already know that?’ (Unfortunately I remember the quotation but not the speaker, so am unable to attribute this quote.) So, although we shouldn’t be frightened of discovering what is apparently ‘common sense’, but we cannot regard ‘common sense’ as a research method.
1.4.2 Confirmation Bias There is another problem with relying on common sense or everyday research methods. We tend look for and remember events that confirm our beliefs and ignore events that contradict our beliefs. This is called the confirmation bias. For example, like many people you may feel that whichever queue you join at the checkout of a supermarket that queue will always be the slowest moving (a variation on this theme is the choice of lanes in a motorway traffic jam – should you switch lanes or not). The trouble is that you only remember the instances when your observations confirm this belief. On the days when your queue moves quickly you either fail to notice it, gratefully accept your good fortune and forget to record this falsification of your position, or declare that this is ‘the exception that proves the rule’. Thus, we only collect, record and take note of observations that confirm our beliefs. Let me give you another example. I have an (otherwise) intelligent colleague
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
You might think that we can rely on ‘common sense’ to make some useful inferences about the state of the world. However, we need to be wary of common sense. One reason for this is that there is very little agreement about what is ‘common sense’. For example, you will have heard of the expression ‘Out of sight is out of mind’. This is a bit of ‘common sense’ that tells us that when people are separated from other people or things, they tend to forget about the person(s) or object(s). Many people would tell you that this is clearly true, so should a social scientist regard this as data? The answer is no, and the reason is that there is also a piece of ‘common sense’ that holds that ‘Absence makes the heart grow fonder’. You will probably be able to think of lots of other examples of contradictory ‘common sense’ expressions. The point of this is not simply that we cannot trust what is seen as ‘common sense’. This muddled view about what is obviously true causes other problems for social scientists. There is a public perception that we spend lots of time and public money researching inane and simplistic questions which are either irrelevant or to which ‘everybody’ already knows the answer. Social scientists are frightened of the phrase ‘didn’t you already know that?’, but given that two completely contradictory positions about the consequences of separation can be held simultaneously, we clearly have a problem. In this case, no matter what we found about the relationship between separation and longing we would be told ‘didn’t you already know that’.
PLANN NG
DISASTER
1-5
EMERGENCY
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
who maintains that he can only get a parking space near the station each morning if as he drives along he mutters a ‘parking spell’ to clear other cars out of the way for him. On most mornings he finds a parking space near the station and arrives into work loudly proclaiming the success of his spell. When we ask him to put his spell to the test by not using it on some days he protests that he couldn’t do that because he can’t afford to be late for work! Thus his belief in the utility of the ‘parking spell’ is maintained.
1.4.3 The Availability Heuristic We tend to remember dramatic events and forget the ‘ordinary’. Thus we remember dramatic television reports of relatively rare aeroplane crashes and forget the daily reports of car accidents that we hear on local radio. Fatal road traffic accidents are far more common than fatal air crashes, and even taking into account total passenger miles travelled, cars are very much more dangerous than aircraft. Despite this knowledge I, like most other travellers, feel a little trepidation every time the aeroplane I am on takes off, and I always pay close attention to the pre-flight briefing presented
MSC IN EMERGENCY PLANNING MANAGEMENT
by the crew! The effect of the Availability Heuristic (probably coupled with the Confirmation Bias described above) is so strong that many rational people simply refuse to believe that air travel is safe.
1.4.4 We See What We Expect to See There is another problem that is likely to bias non-systematic observations. We have a tendency to see what we expect to see. That is, our expectations, our beliefs and our motivations influence our perception. This can be quite a profound effect and has been widely studied by psychologists. For example, read each of the following squares in Figure 1.1: Figure 1.1: Perception Squares
A 13 C
12 13 14
Now look again at the second item in each of these sequences. You read the same item as the letter ‘B’ in one context and the number ‘13’ in another. A second example is provided by the second word in the previous sentence (‘read’). You will have pronounced it as ‘red’, but in a different context the same string of letters would be pronounced as ‘reed’. A more applied example is the demonstration that racial stereotypes can bias our reporting of what we see. Boon and Davies (1987), for example, showed participants a picture of two men, one of whom held a knife, talking to each other. Under certain circumstances the participants tended to incorrectly report that they had seen the black man holding a knife when the picture clearly showed the knife in the hand of the white man.
1-6
disaster
riskcrisis MANAGE RISK
An even more extreme demonstration of our inability to accurately perceive our environment comes in the form of a recently reported effect known as ‘Change blindness’. In Simons and Levin (1998) demonstration of this effect an experimenter would engage a pedestrian in conversation, and while the pedestrian’s view of the experimenter was temporarily blocked by two people carrying a large screen, the experimenter was swapped for a different person. Once the screen had passed, only half of the pedestrians noticed that they were now talking to a different person, with the rest happily resuming their conversation with the new experimenter and apparently failing to notice this rather dramatic change. This is a very clear demonstration of a fact that psychologists have now established in a wide range of settings – people are very unreliable observers.
1.5 Empiricism and the Empirical Approach to Science The term ‘empiricism’ refers to the philosophical position that all knowledge derives from experience and that no knowledge is innate. We only know what we have seen. This should not be confused with an empirical approach to scientific research. The empirical approach simply emphasises the importance of observation as the basis of the research process. Empirical research involves collecting observations and drawing conclusions on the basis of these inferences. Empiricism as a philosophical position, by contrast, suggests that the process of observation is an end to it self and that all we need to do is to collect and report data – no interpretation will be necessary.
1.6 Rationalism Rationalism holds that it is possible to uncover truths about the nature of the universe by the process of logical reasoning alone and without the need to make observations about the nature of the universe. Loos (1995) suggests that a recent example of rationalism is to be found in the prediction of the existence of black holes. Astronomers predicted the existence of black holes purely on the basis of logical reasoning and without ever having observed one. It was not until many years later that the Hubble Telescope first recorded what are now thought to be black holes. Thus, logical reasoning alone, independent of observation, predicted the existence of these extraordinary phenomena. Observation has helped confirm our reasoning, but did not in itself form part of the process that led to the prediction. This is an interesting example, but ‘pure’ rationalism in modern science is very rare. It may have occurred to you that the theories that allowed astronomers to predict the existence of black holes were based on a mass of observational data collected by scientists over many centuries. In reality, modern science is largely an empirical process with just a little rationalism thrown into the mix.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Given these problems with common sense, and everyday observations, it is clear that we need some assistance if we are going to make reliable observations and draw appropriate inferences about the state of the world. This is why we need scientific research methods. We need a methodology that takes into account our poor observational skills and allows us to make accurate and systematic observations. However, before we look at some of the range of methodologies available to social scientists, it will be useful to explore just a little of the philosophy of the scientific method.
PLANN NG
DISASTER
1-7
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Loos (1995: 27) quotes Medawar (1984) as saying: I am a rationalist – something of a period piece nowadays, I admit – but I am usually reluctant to declare myself to be so because of the widespread misunderstanding or neglect of the distinction that must always be drawn in philosophic discussion between the sufficient and the necessary. I do not believe – indeed, I deem it a comic blunder to believe – that the exercise of reason is sufficient to explain our condition and where necessary remedy it, but I do believe that the exercise of reason is at all times unconditionally necessary and that we disregard it at our peril. This is an important point. The scientific method is based on observation and is thus empirical, but the interpretation of the data collected, and indeed the theory that should motivate the collection of data will (or should) be governed by rationalist reason. That is we should theorise about how we think the world might be, and then go out and collect the data to check our theory. Indeed, this is precisely the nature of the process involved in the discovery of black holes. The existence of black holes is a logical prediction of our understanding of various other structures and forces in the universe – thus we reason that they might exist. We then turn to empirical observation to check whether they do exist. Rationalism alone would suggest that
MSC IN EMERGENCY PLANNING MANAGEMENT
the prediction that black holes might exist is sufficient in itself and that there is no need to check. Empiricism would hold that we should not drive our observations by theory, but simply make observations – the process of scanning the skies will tell us all there is to know about the state of the universe. The modern scientific method combines these two approaches. We collect observations to allow us to test our theories.
1.7 Methods in Social Science Research The methods used to gather data in the social sciences will be covered in detail in Unit 6, but we will use this opportunity to give a brief introduction to the subject. Broadly we can divide the methods used in Social Science Research into Experimental Methods and Non-experimental Methods. We will start by looking at experimental methods
1.7.1 The Experimental Method The experimental method is a relatively recent invention, but is now regarded as having a special place in science. The experiment is so revered that we sometimes give the mistaken impression that other methods are not scientific. This is not the case, but the experiment does have one real and very special characteristic that marks it out from the other methods available to us. The experiment is the best method for establishing the existence of a causal relationship between two variables – allowing us to decide whether or not a change in one variable results in a change in another variable. The experimental method is a method of research that involves maximum input from the experimenter. The experimenter deliberately and actively manipulates one variable to establish what effect this change might have on the other variable. For example, we might be interested in the relationship between the size of a group of people and the extent to which members of that group offer assistance to someone who is apparently in distress. The two variables under examination are the size of the group and the amount of assistance offered. The experiment allows us to ask the question, does changing the size of the group result in a change in the amount
1-8
disaster
riskcrisis MANAGE RISK
of assistance the group members offer? (It just wouldn’t make much sense to think about it the other way round – that is to investigate whether the amount of assistance offered affects the size of the group.) In an experiment we seek to exercise as much control as possible. That is we seek to hold constant as many other variables as possible. Such variables might include: the gender of the participants and the person in distress, the nature of the distress, the expression of distress, the time of day, the level of lighting – an almost endless list. Control is achieved by conducting the experiment under controlled conditions where these variables can be fixed. This leaves us with the two variables we are interested in. We give the special name Dependent Variable to the variable that we think will be dependent on the other variable. Thus in this example, as we think the amount of assistance offered will depend on the size of the group, the Dependent Variable is the amount of assistance offered. The other variable is given the special name Independent Variable (these issues will be covered in more detail in Unit 5). The best definition of an experiment is that it is a study in which we control as many variables as possible while manipulating the Independent Variable (the IV for short) and observe the effect of this change on In a true experiment we must assign our participants randomly to our different levels of the IV. Thus each participant must have an equal chance of participating in the 3-person group condition and the 10-person group condition. This is important, because if this was not the case, we might introduce bias into the study. Suppose for a moment that you allowed your participants to choose which condition to participate in – you will immediately see that we have now confused (or confounded to use the technical term) the effect of group size with the sociability of the participants. That is, if we find that the bigger group offers less assistance than the smaller group then this might be because people who like big groups do not like helping others rather than anything to do with the group size per se. So in an experiment we manipulate the IV, measure the effect on the DV , control all other variables and randomly assign the participants to the different levels of the IV. If we do all this we can fairly safely conclude that any changes observed in the level of the DV are caused by the changes in the IV. Thus, we can infer a cause–effect relationship from the results of this experiment. If our participants offered less assistance when we arranged for them to be members of groups of 10 than when we arranged for them to be members of groups of 3, we could safely infer that changes in group size caused changes in the assistance offered. The experiment is really only a special type of observational study, but the fact that we observe the DV while both manipulating the IV and controlling all other variables makes it unique, and it is these characteristics that allow us to use the experimental
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
the Dependent Variable (the DV). But for a true experiment there is one other important requirement.
PLANN NG
DISASTER
method to establish the existence of a causal relationship between the IV and the DV. However, there are also distinct disadvantages associated with the experiment that make it an impractical and undesirable method in many fields of research. The major problem is that the level of control required is so high that we normally have to sacrifice a great deal of realism, and as a result our method has low ‘ecological validity’. Most experiments are conducted in laboratory settings to allow the experimenter to maintain control over a whole host of environmental and other variables. You may, justifiably, feel that the laboratory is not a good place to study natural behaviour.
1-9
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 1-10
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
There is one other warning that we should remember when using the experimental method. Just because we have conducted an experiment, this does not mean that our conclusions are ‘bulletproof’ – we can still make invalid inferences based on the results of our research if we fail to think very carefully about our research. We should always think very carefully about all the things that could go ‘wrong’ and that could threaten the validity of the inferences we draw. We shall return to this issue shortly.
1.7.2 The Quasi-experimental Method In social science research it is often simply not possible to conduct a true experiment. In particular it is often impossible to randomly assign participants to the levels of the IV. For example, suppose we are interested in the differences between the amount of assistance that men and women offer to a stranger in distress. An experimental design would require us to randomly assign our participants to either the ‘men’ or the ‘women’ condition. Even if we were allowed to undertake some rather radical surgery this would not be possible. Our participants arrive pre-assigned to our conditions. This is an example of a quasi-experimental study. Quasi-experimental studies are common in applied social science, and social scientists often blur the distinction between experimental and quasi-experimental studies. However, the difference is important because the quasi-experimental design does not allow us to make cause–effect inferences with the same degree of confidence as is possible with the experimental method. We always have to consider the possibility that it is some other variable related to the assignment of participants to the levels of the IV, that has a causal relationship with the DV. For example, it might not be the sex of the participants that determines their willingness to assist the stranger, but some other variable correlated with sex, for example physical size. Quasi-experiments are still valuable scientific studies, but are rather more limited than experiments and the results from these studies must be interpreted with some added caution.
1.7.3 Field Experiments Field experiments are experiments that are conducted in the field (!) – that is, away from the laboratory. Of course, the physical location isn’t in itself important. What is important is the extent to which it is possible to control other variables. Field experiments tend to be conducted in the ‘real world’ where the rigorous control of variables that can be achieved in the laboratory is not always possible. In fact many quasi-experiments (see above) are also field experiments, but field experiments like true experiments involve the random allocation of participants to conditions. Both field experiments and quasi-experiments are rather less reliable methods of determining cause– effect relationships than true experiments. The limiting factor for field experiments is that the lack of control may permit changes in other variables to influence the DV and so threaten the validity of conclusions about the cause–effect relationship between the IV and DV. For example if a researcher was conducting a field experiment to investigate the ability of participant ‘eyewitnesses’ to identify own- and other-race ‘suspects’, then s/he would need to ensure that the lighting and other conditions likely to influence the accuracy of the participants’ identifications did not vary systematically. It would be particularly damaging, for example, if the own-race identifications were tested on a day with better lighting than the other-race identifications. This disadvantage of field experiments has to be balanced against the increased realism or ecological validity afforded by this method.
disaster
riskcrisis MANAGE RISK
1.7.4 Non-Experimental Research Methods There are a variety of valuable research methods that are collectively termed non-experimental. It is important to realise that just because these are non-experimental methods does not mean that they are not valid and valuable research methods. However, each of these methods has particular limitations which must be remembered when they are used. A particular limitation that applies to all of these methods is that we cannot safely infer a cause–effect relationship between the variables studied using these designs.
1.7.5 Observational Methods Observational research is ideally suited to the study of natural behaviour as it occurs in ‘real’ settings – that is outside the laboratory. Observational studies are often undertaken informally to allow a researcher to obtain an initial impression of the variables involved in a situation. For example, a social scientist interested in violence outside pubs might spend a few sessions simply observing people coming and going outside pubs to
An obvious advantage of observational research lies in its high level of ecological validity – that is, it is far more realistic that the heavily controlled research that takes place in the laboratory. By comparison, however, observational research is severely lacking in control and thus we need to be very careful in drawing causal inferences from our observation. For example, even if after very careful observation we have established that it is always the case that 5 minutes before a fight there is a mix of men and women outside the pub, with the men being in the majority, we can still not conclude safely that it is this mix that causes the fight; for example it could be that this is the average composition of a crowd who like to come and watch a fight that they somehow know is about to occur.
1.7.6 Survey Research Surveys are mechanisms for collecting the self-reported observations of participants (or respondents as they are often called in survey research). Surveys can be administered either verbally, when they are called interviews, or in written form when they are called questionnaires. Surveys also vary along the dimension from Structured to Unstructured (with Semi-Structured somewhere between these extremes). Structured interviews or questionnaires have a rigid sequence of pre-scripted questions that are always posed in the same order. Unstructured survey methods are designed to allow the researcher to be responsive to leads provided by the respondent and so the questions and their order is not so rigidly pre determined. Surveys can involve either Closed or Open questions (or ‘items’). Closed items are those where the respondent is given only a set number of possible
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
get a feel for the variables that might be involved in determining when fights break out. Alternatively, observational methods may be used in a rather more systematic fashion once a researcher has a more focused notion of the variables involved. Here our researcher might spend 14 nights observing behaviour outside a selection of pubs and might particularly note the number of people outside the pub at any time and the ratio of men and women present. Finally, observational research might be even more systematic, involving careful planning by the researcher who has now identified that s/he wishes to record the number of men and women present outside the pub in the 5-minute period immediately preceding a fight. The researcher will have to decide how to classify who is present outside the pub (as opposed to walking past) and will probably need to check the reliability of his/her observations.
PLANN NG
DISASTER
1-11
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
responses (for example, yes/no or multiple-choice questions). Open questions allow the respondent to provide their own response. A major advantage of this method is that it allows for the rapid collection of large volumes of data – however, it can be difficult to design valid survey instruments. It is all too easy to bias an interview or a survey through the wording of the questions. For example I suspect that the following two questions on the same topic would generate very different responses: • Do you think dog owners should be allowed to walk their dogs in public parks where
EMERGENCY
they foul the grass where young children play and thus put the children at risk of blindness and other medical conditions? Yes/No Or • Do you think responsible pet owners should be allowed to exercise dogs in appropriate public places? Yes/No The researcher also has to design the survey items very carefully to avoid questions that are difficult
MSC IN EMERGENCY PLANNING MANAGEMENT
to understand (for example ‘Do you think that it is untrue that….’), make assumptions that might not be true or be offensive (such as ‘What is your husband’s occupation?’) or that have one answer that is obviously more socially desirable than the alternatives (e.g. ‘How often do you beat your children?’). Once a fair survey instrument has been designed, the next potential problem regards the selection of respondents – the sample. If the sample does not adequately reflect the population you wish to know about, then the results will not be applicable to that population. A particular problem here is that of low response rates – if only a proportion of people respond to your survey, there is a danger that the respondents have pre-selected themselves on the basis of some particular interest in the topic or other characteristic (for example, it has been shown that left-handed people are more likely than right-handers to return a questionnaire relating to handedness). Low response rates are likely to lead to very unrepresentative samples.
1.7.7 Archival Research and Other Unobtrusive Measures of Behaviour As we move through life we leave traces. For example, government and other archives will contain details of our birth, our health, our education, our income, our employment history and many other aspects of our lives. This information can provide a valuable source for unobtrusive research. For example, the massive growth in the use of email communication opens up a massive range of possible research topics to the social scientist. Many people send dozens of emails a day. A social scientist could use these messages to ‘reconstruct’ that person’s day and answer an enormous range of interesting questions. This type of research can be very time consuming – especially as it often involves ‘content analysis’. Content analysis is the name given to any technique designed to allow the researcher to make valid inferences by identifying specific characteristics of a message. Such characteristics might be particular phrases or ideas in written or spoken material. For example, a content analysis of emails might involve an analysis of the material in which the sender refers to the demands of their job. Content analysis is a very time consuming process and can be open to biases of the sort described earlier – unless researchers are very careful, they are likely to ‘find’ content that confirms their beliefs and to ‘miss’ content that disconfirms this belief. An additional problem is that the process of selective archiving or selective destruction of records can bias an archive.
1-12
disaster
riskcrisis MANAGE RISK
1.7.8 Case Studies A case study involves the collection of data from a single individual. This normally involves intensive investigation of one individual who is selected for this study on the basis of some special attribute that makes him or her particularly interesting to the researcher. This method can give very important insights. For example it was widely used by the famous psychologist Sigmund Freud as he worked to understand the human mind better, and today is adopted by cognitive neuropsychologists who study the precise pattern of cognitive impairments that follow brain injury. Detailed reports of the pattern of deficits suffered by a patient are closely examined by neuropsychologists who can use these data to both construct theories about the role of certain brain regions in particular cognitive functions, and ‘test’ existing theories against these new ‘data’.
using other approaches.
1.8 Triangulation We have seen that no one research method is ‘best’ in all situations, and each has flaws. Each method brings with it a chance of making a particular type of error. Fortunately these errors are not the same for all methods, and so if we can apply a variety of different methods, we should be able to get a more accurate view of the true picture. Allow me to draw an analogy with navigation at sea. For a sailor finding out exactly where you are is a constant problem (or at least it was until the advent of global satellite navigation systems). One method employed to work out their location is ‘dead reckoning’. This involves plotting the distance and direction travelled since your last known location. So, if you left port and sailed due west for 4 hours at a speed of 4 knots (4 nautical miles per hour), then you would estimate your position to be 16 nautical miles west of port. But a sailor knows that this will be a very inaccurate estimate of his or her real position as it does not take account of the currents, tides and winds, all of which will affect his or her true course over the sea bed. One way to correct for this is to look up in the appropriate reference sources the direction and strength of the tides and currents affecting the boat and to factor these together with an estimate of the wind strength and direction into the equation. But this will still lead to an imprecise estimate of true location. Another strategy will be to ‘sight’ landmarks and to take bearings from them – for example if our intrepid travellers can see a lighthouse bearing 240 degrees then they know that they are lying somewhere along a line drawn from that lighthouse. Perhaps the sailors now spot another landmark, they might be able to use this to get another bearing. In addition they may even use a sextant to estimate position using the sun. In this way the sailors are able to build up a series of estimates of their location. They know that each of these individual estimates will be inaccurate, but that their true location will very probably lie somewhere between these various estimated locations – this is triangulation.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
However, there are problems associated with this method and scientists adopting this approach need to be particularly careful before drawing any inferences about cause–effect relationships on the basis of such data. In addition, researchers need to remember that the data can easily be biased by inaccurate reporting or recording of the details of the single case. These criticisms aside, a major strength of the case study approach is that it sometimes affords us an insight into a unique and particularly interesting situation. The knowledge gained can often then lead to further research
PLANN NG
DISASTER
1-13
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
I hope that you can now see how the same method can be applied to scientific enquiry. If we know that each of our methods will give us an indication of the ‘true’ state of the universe, but that each will bring with it a variable and unpredictable error, then the obvious thing to do is to take as many measures as possible using as many different methodologies as possible. If they all indicate a broadly similar finding then we can be confident that we have achieved a fairly accurate view of the universe. Like the sailor we might not know the exact truth, but we will have a sufficiently accurate estimate to work with – so long as we remember the errors inherent in our methods. The sailors can use this estimated location to steer a new course, and so long as they make sensible allowance for possible errors they can steer clear of obstacles. The scientist, so long as s/he is constantly aware of the likely sources of error in his or her observations, can use them to make safe inferences. Thus, there is virtue in using a variety of research methodologies.
1.9 Threats to Valid Inference
MSC IN EMERGENCY PLANNING MANAGEMENT
As we have seen, each of the variety of research methods available to us has its limitations. Cook and Campbell (1997) considered a range of what they called ‘threats’ to the validity of a research project. These threats to validity are basically things that can go wrong and trip up the unwary researcher leading him or her to draw an invalid inference from his or her data. Although Cook and Campbell’s ‘taxonomy’ of threats to validity is based mainly on experimental and quasi-experimental methods, it still provides a very useful insight into the problems of valid inference in all forms of research. Cook and Campbell described four main types of threat to validity: • Statistical conclusion validity • Construct validity • External validity, and • Internal validity.
1.9.1 Statistical Conclusion Validity Statistical conclusion validity relates to the extent to which a researcher makes appropriate use of statistical techniques to draw inference from the data. Some particular problems, or threats to statistical conclusion validity identified by Cook and Campbell include: • Low power – some statistical procedures have very low power – that is they are virtually incapable of detecting anything but the very largest of effects. • Violation of assumptions of statistical tests – statistical tests make certain assumptions and if these are broken then there is a danger that an invalid inference will be drawn. • Data fishing – this is the tendency for researchers to collect lots of data and then go ‘fishing’ (searching) for an interesting result by dredging backwards and forwards through the data. The danger here is that working in this fashion increases the chances of making what is called a Type 1 error – that is of wrongly concluding that two variables co-vary when they do not.
1-14
disaster
riskcrisis MANAGE RISK
• Reliability of measures – if we do not have a reliable (that is an accurate or stable) measure of our variables, we are less likely to be able to detect real changes in these variables. This is because the real changes will be masked by the ‘noise’ in our data generated by the poor reliability of the measures. It is clear that even careful research planning can be let down by poor data analysis. The point about statistical conclusion validity is that we should carefully plan our data analysis when planning the original research. This will help minimise threats to statistical conclusion validity.
1.9.2 Construct Validity In Cook and Campbell’s taxonomy construct validity refers to the adequacy of the definitions of the variables under examination. For example, suppose we are interested in the effects of media portrayals of violence on children. We need to carefully define what we mean by ‘the effects of media portrayals of violence’ and ensure that we study the effects of a suitable range of materials. For example, if we only study the impact of conventional films on these children, then will we be able to generalise from our
1.9.3 External Validity External validity relates to the ability to generalise from our results to: • Other populations: For example, if our participants were university students can we say anything about ‘ordinary people’? • Other environments: If we conducted our research in a city centre, could we use the findings to make predictions about behaviour that might occur in rural settings? • Other times: If we conduct our research at a weekend, will the results generalise to other days of the week? A study that is very high in external validity will allow us to generalise to all these other situations. One that is low in external validity will be specific to the precise environment in which the research was carried out and will not allow us to say anything useful about other people, situations or times.
1.9.4 Internal Validity Internal validity relates to our ability to detect causality. This comes down to issues relating to the soundness of our methodology. Cook and Campbell list a series of possible threats to internal validity including:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
results to talk about the effects of the more graphic material they might have access to via the Internet?
PLANN NG
DISASTER
History: Something may happen to our participants during our research programme that could affect our data. For example, a change in the law relating to drinking might affect observations of fighting outside pubs. Maturation: Maturation is the effect of time – this can be problematic if our research occurs over a period of time. For example if we were studying children we would need to consider that during the course of the study the children would become older and we would therefore expect their behaviour to change.
1-15
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 1-16
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Testing: An apparent effect may be due to repeated testing – participants may remember responses made earlier. For example, if a survey is repeated too soon after its first administration participants are likely to recall their earlier response. Instrumentation: It is essential to use the same ‘instrument’ to measure behaviour at all times during the study. For example, if we have observers rating some behaviour, we need to ensure that the observers maintain the same system of rating over the course of the study. Statistical regression: This threat to internal validity relates to a fascinating statistical phenomenon known as ‘regression to the mean’. To put it simply, this describes the fact that if I get a high score when tested today (that is higher than the mean score) I am likely to get a lower score tomorrow. Similarly, if your parents were shorter than the mean height, then you are likely to be taller than them. Thus, scores regress towards the mean score. This is relevant here because it can affect our observations. Suppose for example we chose to observe a group of individuals who were particularly disruptive. Even if nothing was changed, we would expect that, as a result of regression to the mean, these individuals would be less disruptive next time they were observed. Failure to take this fact into account when interpreting your results could lead to invalid inferences about the effectiveness of some treatment or other intervention. Selection: This is the threat to validity caused by problems in participant selection. This is a particular problem when we are not able to allocate participants randomly (as we do in a true experiment). For example if we were interested in differences in the fear of crime experienced by young and old people, we would have to be careful that our participant recruitment procedures did not result in a particularly fearful or fearless group of participants being selected, for example, if the young people were all sampled from among recruits at an Army camp, while the old people were all sampled through an organisation which provides support for victims of crime! Mortality: Some participants might be more likely to drop- or opt-out of a study than others. This can also bias the research process. It is up to the researcher to carefully consider whether each of these threats could apply to their research and to attempt to rule each out as an explanation for their data. Although these threats are reduced when an experimental method is adopted, they still apply and need to be examined as possible sources of error.
1.10 Concluding Thoughts In this Unit we have considered the nature of research, and the special qualities of scientific research. We have seen that humans are not naturally accurate observers and that for this reason we need to adopt rigorous methods when conducting scientific research. We have then examined a range of methods available to the social scientist and have noted that each has it strengths and limitations. We have noted that wherever possible these techniques should be used in combination through the process of triangulation. Finally we have considered a variety of the ‘threats to validity’ identified by Cook and Campbell (1997) as being possible sources of invalid inference in research.
disaster
riskcrisis MANAGE RISK
1.11 Main Points • Science can be defined as a system of verifiable knowledge and as a method of investigation. • Research is concerned with identifying answers to questions that are as valid (i.e. useful and justifiable) as possible. • There are a number of different forms of validity and it is typically impossible to devise an investigation that will achieve high levels of all types of validity. • Social scientists are interested in a wide range of different types of data and use a varied range of methods to gather data. These methods have varying degrees of validity and all have their respective strengths and weaknesses.
1.12 The Following Course Units
• identify a research problem which you consider to be worthy of investigation; • choose an appropriate theoretical perspective (or perspectives) which informs the proposed research; • identify appropriate methods of social enquiry that will enable the collection of data to answer the research questions that you have set yourself; • locate potential problems that you would be likely to encounter while conducting the research. (Please note that you are not required to actually conduct this research – although you may choose to use the proposal as the basis of your dissertation research.) Some students will undoubtedly be apprehensive at the prospect of this task and we, therefore, offer the following two-part advice. First, the final Unit of this Module offers detailed information on how to write the research proposal – do not attempt the assignment, therefore, until after you have read the final course Unit. Second, we recommend that you read all the other course Units in this Module in order to identify the strengths of each method of enquiry (that is, the type and quality of data that you will obtain and the kind of research questions that it can answer) and their weaknesses (that is, limitations to the type of data you will obtain – issues of reliability of research instruments and the validity of data). The remaining course Units in this Module are as follows:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
In this section we will provide you with a brief introduction to the content of the remaining course Units in this Module. First, we offer a few words of caution. The written assignment you are required to undertake on completion of this Module demands a slightly different approach than previous assessments on this course. You will be asked to produce a written research proposal to investigate a particular area relevant to your course. This exercise will require you to:
PLANN NG
DISASTER
Unit 2 Research Ethics It is the aim of this Unit to introduce students to the important issue of ethics in social science research. Ethical considerations abound in all types of social research and students need to have a
1-17
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
good understanding of the way(s) in which ethical dilemmas can arise during the course of a research project. The objectives are to: (1) encourage students to think about some of the problems and dilemmas inherent in studying and drawing conclusions from human behaviour; (2) enable students to critically examine the findings of other research; and (3) prepare students for possible criticisms of their research in the future. Unit 3 Reviewing the Published Literature This Unit is designed to show students the role of the literature review in research. It also explains the process of conducting a literature review and describes the main types of review to be found in professional and academic use. Particular attention is given to the kind of issues a good review should address, what should be included, what should be left out, and how it should all be put together. Unit 4 The Social Context of Research
MSC IN EMERGENCY PLANNING MANAGEMENT
Social research is normally conducted in a non-laboratory setting. This Unit discusses the influence of external factors (i.e. the social environment) and internal factors (i.e. the internal characteristics of the researcher) on the research process. Research that is well designed has usually taken into account such issues as access, researcher and participant safety, the political nature of the research, the physical or organisational setting of the research, human diversity, and the individual characteristics of the researcher. Unit 5 Research Design Unit 5 introduces students to basic concepts in research design. Benchmarks of quality for good research are introduced. Types of research design such as correlational, experimental and case study designs are described. Comparative and historical approaches are outlined, applied research (evaluations, policy analyses and audits) is detailed as is collaborative or participatory research. The process of incorporating theory – either deductively or inductively – into research via conceptualisation, operationalisation and the creation of hypotheses is discussed, as well as issues of measurement. The sampling options available to social researchers are presented; and finally, an introduction to the combination of data gathering techniques (‘triangulation’) is given. Unit 6 Data Gathering Techniques This Unit discusses the many ways researchers gather data, including observation and ethnography; group and individual interviews; life history research; surveys; documentary research; secondary analysis; and content analysis. The advantages and disadvantages of each technique are discussed, as well as how to draft a measurement instrument for each of the techniques described. Unit 7 Qualitative Analysis and Presentation This Unit describes the general background to qualitative research and gives information on how different forms of qualitative data are analysed and described. Particular attention is given to considering issues surrounding the coding of data. The use of interviews, observational data and secondary data sources such as the media and archival information are all considered. The advantages and disadvantages of the qualitative approach are outlined.
1-18
disaster
riskcrisis MANAGE RISK
Unit 8 Statistical Tests The majority of commonly used statistical tests are relatively simple and straightforward and they do not require the researcher to possess a high level of mathematical knowledge in order to understand and implement them. Different experimental conditions demand different statistical tests for assessing results, and it is therefore essential for researchers to appreciate which statistical tests should be used in which situations. This Unit discusses the application of descriptive and inferential statistics together with a description of the appropriate statistical tests for different types of research design. Worked examples are provided for each of the recommended statistical tests. Unit 9 Writing a Research Proposal and Dissertation
1.13 Study Questions You should now write approximately 300 words in answer to each of the questions below. We think that this is an important exercise that will help you to assess your understanding of material and evaluate your progress on this course. Your answers are intended to form part of your course notes and should not be forwarded to the University. 1. In what ways do we acquire knowledge about the world we live in? 2. In what ways do quantitative and non-quantitative research methods differ in the social sciences? 3. Why is it important for research to be both reliable and valid? Is either quality more important than the other?
1.14 Bibliography Boon, J. C. W. and Davies, G. M. (1987) ‘Rumours Greatly Exaggerated: Allport and Postman’s Experimental Study’ Canadian Journal of Behavioural Science, 19: 430–40. Burns, R. (2000) Introduction to Research Methods, London: Sage.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
As part of the assessment process for all the post-graduate, distance learning programmes, students are required to prepare a research proposal and to write a dissertation following the completion of their MSc research project. This Unit discusses the requirements of these documents and outlines the structures and the key elements that should be incorporated into a research proposal and/or a dissertation.
PLANN NG
DISASTER
Cook, T.D. and Campbell, D.T. (1997) Quasi-experimentation: Design and Analysis Issues for Field Settings, Boston: Houghton Miffin Company. Loos, F. M (1995) Research Foundations for Psychology and the Behavioural Sciences, New York: HarperCollins. Simons, D.J. and Levin, Daniel, T. (1998) ‘Failure to Detect Changes in People during Real-world Interaction’ Psychonomic Bulletin and Review, 5(4): 644–9.
1-19
crisis
PLANN NG
MSC IN EMERGENCY PLANNING MANAGEMENT
1-20
crisisdisaster RISK MANAGE
risk
EMERGENCY
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 2 Research Ethics
disaster
riskcrisis MANAGE RISK
2 Unit Two: Research Ethics 2.1 Aims and Objectives of this Unit It is the aim of this Unit to introduce students to the contentious issues of values and ethics in social science research. Ethical considerations abound in all types of social research and students need to have a good understanding of the way(s) in which ethical dilemmas can arise during the course of a research project. The objectives are to: • encourage you to think about some of the problems and dilemmas inherent in studying and drawing conclusions from human behaviour; • enable you to critically examine the findings of other research; and • prepare you for possible criticisms of your own research in the future.
This Unit can be broadly divided into two sections. The first section discusses the role of values, their place in social research and the debate between positivism and phenomenology concerning their influence upon the research process. The second section considers the role of ethics and ethical dilemmas that may occur during the process of research at both a general and situational level. These two issues are, of course, closely interrelated. It is necessary, however, to understand some of the relevant terminology before we go on to consider the relationship between ethical issues and social research. The distinction between values and ethics is not readily apparent and is sometimes blurred. An allegation of unethical research practice generally implies criticism, suggesting that a researcher has acted in some way dishonourably by seeking to mislead, or by, for example, harming research subjects. Criticism based on identification of the researcher’s values, however, need not imply blame. This is clearly illustrated by the allegations, sometimes made by women, that male researchers have internalised values which colour their views of women to the extent that the research which they conduct must be treated with caution. This criticism is not (usually) that male researchers intend to treat women unfairly, rather that they do so unconsciously and that they are determined in their actions by what (or who) they are. In the Weberian (1949) sense, their maleness prevents them from being able to empathise with and understand women since their perceptions of the world are garnered through a male lens. Similar criticisms have been levelled at white researchers who study black people and at the comfortably-off when they study the poor. The same criticisms could presumably be made the other way round; the fact that they have not is a reflection of the historical prevalence of well-off, white male social scientists. Much more is known about the poor and powerless than the rich and powerful who, it may be argued, are in a stronger, more powerful, position to resist the intrusion of research.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
2.2 Introduction
PLANN NG
DISASTER
All social research, therefore, raises not only ethical and moral considerations but ‘political’ ones as well. All research is political in the ‘non-party’ political sense since there are always questions
2-3
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
concerning power relations and specifically the issues of access, privacy, honesty, trust, confidentiality and anonymity. Indeed, Punch (1986) has argued that ‘infiltration’ (getting in and getting out) is the key skill used in participant observation studies where the researcher spends lengthy periods of time (covertly or overtly) immersed in the life of the group. It might be argued that ‘getting in’ and ‘getting out’ unscathed is the important thing. For example, Yablonsky (1968) faced the threat of violence in a commune and Thompson (1967) was beaten up by Hell’s Angels. These are examples of ethical dilemmas that can arise in the course of research (to be discussed in more detail later in this Unit). However, the reader should now be sensitised to some of the dangers and difficulties involved in doing social research, especially when this entails the physical proximity of the researcher with people who are involved in criminal or deviant activities.
2.3 Values 2.3.1 Normative and Positive Statements
MSC IN EMERGENCY PLANNING MANAGEMENT
Social scientists engage in making statements about the nature of society and its members. It is important, therefore, to differentiate between the different types of statements they make. Positive statements are about what ‘is’, while normative statements are concerned with what the author would like to be. Positive statements are about what is, or predictions of what will come to be; they assert alleged facts about the universe which are amenable to proof or disproof. Normative statements are about what ‘ought’ to be. As they are dependent on judgements about what is good or bad, desirable or undesirable, they are bound up with our philosophical, cultural, political and religious beliefs. Another way of understanding this distinction is to think in terms of objective and subjective. A subjective belief is one which arises from the mind of an individual (influenced by preconceptions) and is, therefore, value-bound, while an objective belief is rooted in value-freedom and is based upon observable phenomena outside ourselves. However, the distinction between positive (fact) and normative (judgement) is not as clearly demarcated as this discussion suggests. It is important to be aware of a constant interplay and dialectic between the ideas in society and the ideas in science which, in turn, influence research (May, 1993). In the context of social science, the distinction between positive and normative is perhaps most easily exemplified by the difference between classicism and positivism. The latter, as its name suggests, sought to make positive statements about the world which could be tested by empirical examination. Classicists, on the other hand, based their ideas on the philosophical concept of a social contract, the existence of which was never subject to proof. This difference between positivism and classicism also underpins the epistemological debate between positivist and phenomenological approaches to the nature of knowledge, truth and social reality.
2.3.2 Positivism and Phenomenology The controversy surrounding the issue of values and their role in social research can be understood in terms of the debate between positivism and phenomenology. These two traditions of thought have tended to disagree about the role that values do or should play in social enquiry, and this can be explained by their competing perceptions of the nature of truth, social reality and human knowledge. In order to understand this complex debate it is important to locate it within an historical context.
2-4
disaster
riskcrisis MANAGE RISK
Classical sociologists of the late 19th and early 20th Centuries were influenced by the success of natural science and were keen to establish a science of society. There is no doubt that certain values underpinned and informed their work, particularly a desire to improve the societies in which they lived. Classical sociologists such as Marx (1818–1883) and Durkheim (1858–1917) were strongly influenced by the scientific approach of the natural sciences otherwise known as the ‘hypothetico-deductive’ method (or positivism). However, in their own work they relied largely upon historical and other sources, as opposed to conducting empirical research. As social theorists they made use of what has come to be known as the ‘comparative’ or ‘constant comparative’ method.
Positivism helped stimulate the development of quantitative (numerical) research methods, still very influential today. It was not until the 1960s that positivism encountered attack from phenomenological and interpretative sociologists who emphasised the processual nature of human knowledge. It was argued that knowledge does not exist independently of our observations and that there is no clear-cut distinction between the ‘knower’ and the ‘known’ since the two are interactive and inseparable. They argued further that there is no such thing as ‘objective’ knowledge or ‘absolute’ truth. Reality is socially constructed through the process of human enquiry, and knowledge does not simply exist ‘out there’ waiting to be discovered; rather, it is produced during the very process of research. The emphasis was, therefore, upon social interaction, consciousness and meaning, which were also central to the work of the Chicago School and the ethnographic research that it spawned. Interpretative sociologists argued that in order to understand social action we need to understand from the point of view of the actor. This is sometimes referred to as ‘methodological individualism’. This methodological approach helped stimulate methods which were more qualitative (non-numerical/verbal) than quantitative and more concerned with meaning and what Weber (1949) termed ‘verstehen’.
2.3.3 Values and Research These assumptions underpin the epistemological debate between positivism and phenomenology and have implications for the way in which research is conducted and the conception of the role of values in this process. Positivists believe it is possible to operate according to the principles of natural science. Therefore, we are able to separate ‘facts’ from ‘values’ and achieve knowledge which is both objective and independent. Importance is given to the issues of reliability and generalisability. In contrast, interpretative sociologists place greater emphasis upon meaning and validity and argue that all knowledge is produced rather than simply discovered. Thus, a qualitative researcher might ask: ‘What is the point of generating lots of data which are replicable (given the same methods) and representative, if invalid?’
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The dominant paradigm of this period was the positive method (or positivism) which held certain fundamental assumptions about the world. According to positivism, an independent reality exists which is discoverable by the application of the appropriate research methods. There was also a belief in ‘absolute’ truth and ‘objective’ knowledge which is discoverable using the scientific approach. However, it was Max Weber (1864–1920) who was hesitant about this approach and who drew attention to a fundamental difference between the natural and social sciences, namely, its subject matter. Social scientists study people, who are rational, conscious beings, capable of making choices. Weber argued that it is not enough to study external forces as determinants of social behaviour since all action occurs within a social context and is founded upon shared ‘definitions’ and shared meanings. People do not respond to external stimuli in a passive way; instead, they make decisions based upon their interpretations.
PLANN NG
DISASTER
2-5
EMERGENCY
One of the first questions we need to ask is: what are value judgements? May (1993) intimates that value judgements can be conceptualised as falling along a continuum somewhere between ‘matters of fact’ and ‘matters of taste’. All value judgements are dependent upon both our beliefs and experiences, otherwise known as our preconceptions. In general, the debate concerning values has revolved around the issue of the extent to which values do or should impinge upon research. Those who believe in the existence of objective truth (positivists) argue that values should not impinge upon research. Weber distinguished between values which impinge upon our choice of topic, ‘value-relevance’, and values which enter the research process itself and bias our methods for collecting and analysing data. He argued for a sociology which maintains ‘value-neutrality’ during the process of research, but acknowledged that values would always play a part in the selection of a substantive topic. In contrast, those who hold that objective or absolute truth does not exist (phenomenologists) argue that truths are the negotiated outcome of a social process and that valuefree knowledge is impossible to achieve anyway.
MSC IN EMERGENCY PLANNING MANAGEMENT
Many positivists would, therefore, consider that science makes objective, value-free statements about our surroundings. Others, for example, phenomenologists, consider that it is impossible to eliminate human values from the research process. Indeed, some would go as far as to argue that this is not desirable. Critical (Marxist) and feminist researchers, for example, argue that preconceptions are central to the research process and tend therefore to begin research from a pre-formulated theoretical framework. Such social scientists have attempted to radicalise and politicise the research agenda in terms of class and gender. A good example can be found in the work of Janet Finch (1993) who provides a personal and honest account of this process in her chapter: ‘It’s Great to Have Someone to Talk to’. Reflecting upon two pieces of research, a study of clergymen’s wives and a study of women running and using pre-school playgroups, she reveals the ease with which female researchers can elicit material from other women using the technique of in-depth interviewing. She modestly attributes this, not to her skills as an interviewer, but to her identity as a woman. She argues that this is not a reflection of in-depth interviewing per se but more a reflection of the gender specific nature of the researcher–researched relationship. She argues that women are more used to intrusions into and questions about their private lives (facilitated through their experience of motherhood); that interviews conducted in the comfort of a woman’s own home can easily come to resemble an ‘intimate conversation’ and that the structurally disadvantaged position of women in the domestic and private sphere means that they are more likely to welcome an opportunity to have someone to talk to. Loneliness was very common among the women she interviewed and she cites the following from an interview with a 24-year-old mother of two living on a run-down council estate: No but I don’t, because I think once you get married and have kids, that’s it. To a lot of women round here – when you see them walking past – big fat women with all their little kids running behind them. And I think, God. That’s why I want to go to college and do something. But fellas don’t see it like that, do they? Like, he thinks it’s alright for me just going back to work in a factory for the rest of my life, you know. But I don’t want that. (To child) You have a career, won’t you? Prime Minister, eh? (Finch, 1993: 171)
2-6
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
disaster
riskcrisis MANAGE RISK
Finch continues with the following analysis of the interview text: Comments of this kind – albeit very differently conceptualised and articulated – would not have been elicited in a formal questionnaire nor if I, as interviewer, had been attempting to maintain an unbiased and objective distance from the interviewees ... . Comments like ‘fellas don’t see it that way, do they?’ ... indicate an identification between interviewer and interviewee which is gender specific. (Finch, 1993: 171) Finch therefore emphasises the value of in-depth interviewing as a technique which can elicit indepth and rich material as opposed to the relatively superficial data captured by the more structured and formal social survey type questionnaire. She highlights the exploitative potential of women interviewing other women when she argues that women are very trusting of female researchers and often willing to disclose very personal and private information about themselves. In return, they are afforded a very superficial guarantee of confidentiality and anonymity.
relations within the researcher–researched relationship (see Unit 7 for a little more on this). Finch, however, emphasises the exploitative potential of women interviewers in terms of the uses of such data and the fact that women researchers are not always in control of the data they collect. There is always a danger that it might be used against the interests of the women interviewed or women in society more generally. Nevertheless, Finch fails to consider the issue of class and the fact that many women interviewers tend to interview women in different and more structurally disadvantaged positions than themselves. Therefore, the general criticism levied against other (male) researchers that they focus their research on more vulnerable and less powerful members of society may also be applicable to feminist researchers, despite their commitment to the cause of feminism. More critically, Finch argues that Barnes’ (1979) discussion of ethics is male biased and couched in the language of the public domain of men, hence his use of the concept ‘citizen’ to denote the research ‘subject’ or ‘interviewee’. Discussion of ethics has tended to focus on issues such as access and data collection rather than the potential uses of such data, and Finch argues that information gained from research with women should not be used against women’s collective interests. In her own research, she found that clergymen’s wives were content to centre their lives around their husbands but she could not (as a feminist) generalise this to most women. This encouraged Finch to distinguish between women’s structural position and their experience of it. This example demonstrates the way in which preconceptions can be perceived by some researchers as an important part of the research process. Becker (1967) argued that participant observation studies could provide a voice for the powerless groups in society and he posed the question: ‘Whose side are we on?’ He further recommended that sociologists should favour the side of the ‘underdogs’. In answer to Becker’s question, Finch concludes that a feminist sociologist will always be ‘on the side’ of the women she studies and never take a detached and unbiased stance.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The in-depth interview approach advocated by Oakley (1981) is generally preferred by feminist researchers because it is supposed to subvert the hierarchical (and unequal) nature of power
PLANN NG
DISASTER
Gouldner (1970, 1976) wrote about the ‘myth of a value-free sociology’, which, he argued, has helped sociologists to evade the moral implications of their work. He attacked the notion of value-free, objective social science and argued that the position of moral indifference was a deeply immoral stance to take. Instead of using the myth of ‘value-freedom’ to hide behind their
2-7
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
values, he recommended that social scientists make their values explicit. In conclusion, while we may strive for ‘value relevance’ in our choice of topic, we need to take responsibility for, and try to anticipate, the likely uses to which our findings may be put. This will always be a moral issue which we should not evade.
2.3.4 Values and the Research Process In order to fully examine these issues, it is helpful to clarify the nature of the research process, which comprises at least four stages. Values can enter the research process at any one of these stages: 1. the identification and definition of a research problem; 2. the collection and analysis of data; 3. the interpretation of research findings; and 4. disseminating the findings.
MSC IN EMERGENCY PLANNING MANAGEMENT
The very first step towards ‘doing’ research involves the development of a research model, which, depending upon the nature of the research problem and the methodological approach adopted, can range from being very prescriptive to fairly loose and flexible. In general, quantitative researchers favour the use of a strict research design and are more prescriptive than qualitative researchers. Sarantakos (1994: 91) identifies the following assumptions employed by those who use a research model (the second assumption is considered a moot point for qualitative researchers): • research can be perceived as evolving in a sequence of steps, which are closely interrelated, and in which the success of the one depends on the successful completion of the preceding step; • the steps must be executed in the given order – this is imperative, particularly for models with a few steps; • planning and execution of the research is more successful if a research model is employed. Whether you are doing quantitative or qualitative research, it is impossible to overstate the importance of a well thought out and planned research design. Moreover, there are many important decisions which need to be made in all types of research design (for example, methodology, sampling, collection and analysis of data). Recognising the importance of preparation and planning will be essential when you come to conduct research of your own. It is important to recognise and document the way in which your preconceptions influence decisions regarding your work. You will need to ask yourself whether you are allowing your own values or beliefs to influence your work at any of these stages and, if so, how. For now, however, let us take each of these stages in turn and examine some of the ways in which values can influence this process. 2.3.4.1 Identification and Definition of a Research Problem Society is characterised by pluralism, whereby different groups in society have their own interests and frequently behave in ways which are designed to further those interests. Because of this, all research is vulnerable to group interests and values. Therefore, social research often involves us in all kinds of moral relationships with participants who usually have competing rather than unitary
2-8
disaster
riskcrisis MANAGE RISK
interests. The research process is subsequently characterised by a variety of groups/individuals with divergent (sometimes parallel) interests and obligations. The social scientist will therefore have the difficult job of trying to balance such interests in the process of conducting research. It is not uncommon for students to find a distinction in research methodology textbooks between an ‘idealised’ and more ‘realistic’ account of the research process. The discourse of ‘natural science’ has in the past prevented social scientists from writing openly and truthfully about their research experiences. The positivist tradition characterised by objectivity and detachment required a particular approach which did not include more qualitative approaches. Punch (1986: 13–14) comments that: ... it may well be that in our teaching and publications we tend to sell students a smooth, almost idealized, model of the research process as neat, tidy, and unproblematic. This presentation of research as straightforward, neat and tidy is misleading and suggests a linear model of research with a beginning, middle and an end. The reality is far more complex, and accounts by various researchers reveal the importance of interaction between researcher and researched flexibility is particularly important to fieldwork studies. Thus Lazarfield and Rosenberg’s (1955) standard textbook on methods chose to completely ignore participant observation on the basis of its unscientific nature and lack of systematic analyses of its methodology. It is only in recent years, therefore, that the fieldworker has been able to write more honestly about personal dilemmas encountered in the research process. (Punch, 1986: 14) notes that: ... only comparatively recently, it was considered academically appropriate for social scientists to abandon the dispassionate and detached image of science for the descriptions of their personal involvement in the field. There have been a number of detailed accounts of the trials and tribulations involved in doing fieldwork (see for example, Whyte, 1955; Polsky, 1971; Wax, 1971; Clarke, 1975; Holdaway, 1980). A good place to start is the classic account of Whyte (1955) and then consider other more general accounts of this research method by sociologists such as Becker (1970), Bell and Newby (1972), Johnson (1975), Junker (1960), McCall and Simmons (1969), Schatzman and Strauss (1973) and Van Maanen (1979), which should provide an appreciation of the importance of a full history of the research process and the light it can shed on the nature of data (Punch, 1986). Despite this more recent academic freedom to write and publish more candidly, social scientists are still constrained in their written accounts of the research process and may find it difficult to be completely open and frank about their experience (Van Maanen, 1978). In Britain, there are the libel laws and the Official Secrets Act to consider. For example, Colin Bell (1977) faced the threat
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
and its influence upon the direction and development of a particular research programme. Such
PLANN NG
DISASTER
of legal action from his colleagues simply because he wanted to expose some of the ‘realities’ (tensions and conflicts) surrounding their community study of Banbury (in Oxfordshire). Punch (1986: 19) observes that: ... the Bell and Newby collection illuminate the special dilemmas for British researchers, in terms of libel laws and the centralised control of access and funding in government ministries, which are in some respects less acute in North America.
2-9
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 2-10
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Where research is likely to be politically sensitive (for example, where it threatens to subvert the ideological status quo) or may illuminate aspects of local or central government policy, then ‘research is “political” in an ideological and even party-political sense’ (Punch, 1986: 20). Barnes (1979) identifies four groups/individuals who are involved in the process of social enquiry – when considering the way(s) in which values can impinge upon research, it is helpful to think in terms of these four categories: (i) scientists; (ii) sponsors; (iii) gatekeepers; and (iv) citizens. (i) Scientists (value-relevance) The very first stage of the research process — defining the research problem – is one of the most difficult to ‘cleanse’ of value judgements. What to me is an issue worth investigating might not be to you, and vice versa, and what we choose to research is itself affected by our own values. The choice of research topic can thus lay one open to the ‘charge’ of having submitted to value judgements. Hans Eysenck, for example, has been accused of racism because he investigated an hypothesised link between race and intelligence. In criminology, accusation and counter-accusation have flown between those on the political left, who call for research to be conducted on crimes committed by the powerful, and the self-styled ‘realists’ who prefer to investigate the ‘real problem’ of street crime. The choice of topic is determined therefore by the personal and political interests and values of researchers. Moreover, in the world of contract research, researchers may have little influence over choice of topic, which may be exclusively determined by the subcontractors of research. (ii) Sponsors (funders) Research can be influenced by the value judgements of sponsors and funders (the two are not necessarily synonymous). Typical sponsors range from direct state sponsorship (for example, a government department) to relatively autonomous government research councils (for example, the ESRC) through to local government support, private charitable organisations (for example, the Joseph Rowntree Foundation), university departments and employer organisations (Bulmer, 1982a). The sponsors of research bring different political agendas to the researcher–sponsor relationship. For example, the theory that crime can be reduced by the application of situational prevention methods such as target-hardening, informal and formal surveillance and environmental design is well suited to testing by empirical study. Largely because of its perceived pragmatic advantages, the British Home Office, possibly with government approval, has been generous in allocating research funds to projects designed to test the efficacy of situational crime prevention, with the result that such projects flourished throughout the 1980s. The question of sponsorship has been widely debated in the American literature in relation to Project Camelot and the involvement of the government in social science research. Project Camelot (sponsored by the US military) was conducted in 1965 under the Special Operations Research Office of the American University in Washington DC before it was cancelled suddenly. Data were to be collected in Latin American countries and were expected to be of use to the army in its counterinsurgency missions. When an attempt was made to recruit staff for the project in Chile (without indicating the source of the funds) the project was exposed. Several aspects of the project were controversial. First, the goal was to find out how to prevent peasants and disadvantaged groups in developing countries from taking independent political action to oppose a dictator. Second, some
disaster
riskcrisis MANAGE RISK
researchers were unaware of the source of funds and third, the people and government of Chile were not informed about the project. Questions were therefore raised about the extent to which social scientific research would be jeopardised by the intervention of the military and the ethics of using social science skills in the service of the government. Research is dependent on financial support, and if projects of one type or another are treated favourably because of the value judgements of sponsors/funders, the ‘knowledge’ we gain will tend to be affected. Sponsors, therefore, influence the type of research which is conducted and can exert a controlling influence preventing other types of research. Barnes (1979) indicates that even the major sponsors of research have ‘codified’ the ethical terms and conditions which will influence their decision to provide or not provide financial support. As May (1993: 34–5) asserts: The majority of social research in Britain is sponsored by governments, or other agencies with a vested interest in the results. This is not to say that this necessarily invalidates any conclusions because the work is ‘interested’ as opposed to ‘disinterested’ (often assumed to be a characteristic of scientific activity).
Lee (1993) argues that much criminological research can be defined as ‘sensitive’ because it tends to illuminate the ‘hidden corners of society’, not least the criminal justice system itself. A good example is the study by Cohen and Taylor (1972) which highlights the closed nature of the criminal justice system (see below). However, there are also examples of a more open criminal justice system as in the study by Genders and Player (1989) entitled Race Relations in Prisons, which was both commissioned and funded by the Home Office Research and Planning Unit (HORPU). This research found evidence of racial stereotyping and differential (less favourable) treatment of black as opposed to white prisoners. The authors put forward a set of recommendations to the Prison Department, which shows that the Home Office is not necessarily always and for ever closed. A more recent example which highlights the influence of political power on the research process is a study by Brogden and Shearing (1993) into the raison d’etre of the ‘old’ South African police force. State opposition meant they had to rely on secondary sources of data for their analysis despite the distinct lack of research on the South African police. They observe that: It is no fault of critical South African academics that research on policing in South Africa is notable for its relative barrenness .... Parliamentary debates have indicated that the line between criticising the police and subverting the ‘national interest’ has been a very thin one.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
He argues that an understanding of the research process should entail an awareness of questions such as who funds the research, its aims and the possible dissemination of findings. This does not make the research process invalid but should increase awareness of the political and ethical context of the whole research process in the social sciences.
PLANN NG
DISASTER
(Brogden and Shearing, 1993: 192) Sponsors can also influence the outcomes of research and the way in which it is (or is not) disseminated. Those who sponsor research tend to feel they have exclusive rights over the data and may resent their widespread publication, especially when they consider the findings to be detrimental or antithetical to their practice/ethos. An example of this type of problem can be found in the doctoral work of Punch (1977), which raises significant questions about the rights
2-11
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 2-12
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
of sponsors and the freedom of researchers to publish. The research was carried out in 1967 at Dartington Hall School, which since its opening in 1926 had been labelled ‘progressive’ and had an apparently ‘flexible, experimental, and idyllic’ regime (Punch, 1986). A Mr and Mrs Child had been appointed as joint heads following the resignation of the former head in the 1950s and remained so for a period of ten years. The financiers of the research were the Elmgrant Trust at Dartington, who were motivated by the possibility of new developments for the school and a wish to discover what had happened to their former pupils. Early on in the study Punch encountered resistance and while being paid by the Elmgrant Trust to study their school he was effectively being denied access by their employees. Consequently, he ended up with a less than scientifically generated random sample. The findings proved less than favourable for the reputation of the school (particularly during the era of Mr and Mrs Child) and after a long period of battling to enable the research to continue (partly for personal reasons and the desire to complete his thesis), Punch found himself agreeing to (among other things) give Dartington exclusive rights over publication. It took more than a decade for Punch to finally achieve publication of Progressive Retreat (1977). His conclusions had been critical of the school and had highlighted links between the so-called progressive environment of the school and semi-delinquent adolescents. Moreover, there was evidence to suggest that some students found it difficult to integrate into wider society. Punch (1986: 77) concluded that: ... one aspect that emerges most strongly from my research is the difficulty of studying an institution composed of literate, articulate, self-conscious people with the power, resources, and expertise to protect their reputation. Punch concluded that researchers should never sign away their rights to publication and that the most they should ever concede is a right to be consulted prior to publication. (iii) Gatekeepers Social scientists are dependent upon the co-operation of ‘gatekeepers’, who can provide, or not provide, access (formal entry) to the citizens/respondents of research. This is especially the case when research is conducted within institutions or organisations (stratified research settings) and there are various levels at which we need to gain consent. Dingwall (1980) has termed this a ‘hierarchy of consent’. In such situations it is necessary to negotiate with various groups of people who may have different interests in the research and can influence the study accordingly. A classic British example is the study by Cohen and Taylor carried out in the late 1960s entitled Psychological Survival: The Experience of Long Term Imprisonment (1972). The focus of the study was the subjective experiences of life-sentence prisoners in the maximum security E Wing of Durham Prison. The researchers used their position as teachers of sociology to develop their work with prisoners. However, from the outset, the Prison Department of the Home Office were not happy about the research and tried to prevent its continuation. Cohen and Taylor adopted a ‘naturalistic’ approach and were keen to understand and portray the effects of longterm imprisonment from the point of view of the inmates. At the same time, a group of Home Office researchers were conducting psychological research using structured questionnaires and personality tests to examine the same phenomenon. The Home Office sought to undermine Cohen and Taylor’s research on the basis of its methodological and theoretical underpinnings. They suggested that the research was unsound and unscientific because of its small sample and subjective (qualitative) approach.
disaster
riskcrisis MANAGE RISK
Following publication of their book, Cohen and Taylor managed to obtain funding from the (then) Social Science Research Council for the costs of a formalised research proposal using a more conventional and politically acceptable questionnaire and interview research methods. The Home Office, however, armed with the Official Secrets Act, imposed a number of restrictions on their research: for example, they limited access and introduced censorship of the material. Cohen and Taylor were therefore forced to abandon their work. This case highlights the political nature of research and the difficulty of gaining access to ‘closed’ and politically sensitive settings due to restrictions that can be imposed by ‘gatekeepers’. The Home Office became increasingly suspicious of social science research following publication of Cohen and Taylor’s work and over the past two decades research in prisons has been on the decrease (Jupp, 1993). However, this may also be related in part to cutbacks in the research activities of the Home Office Research and Policy Unit.
(iv) Citizens Citizens fall into all of the above groups: that is, they are scientists, sponsors/funders and gatekeepers. Therefore, social research depends upon the acceptance of a plurality of interests and views within the wider community. Furthermore, the focus of social research is always on citizens within one social context or another and they are integral to the successful completion of any study. As mentioned above, citizens can act as ‘gatekeepers’ or bars to research and it is therefore essential to gain their consent. The rights of citizens/respondents and our obligations to them, are central to all codes of ethics which have been developed to protect their interests (see Appendix). 2.3.4.2 Collection and Analysis of Data
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Gatekeepers also control access at a more ‘informal’ level: for example, employees within a particular organisation may refuse to co-operate with researchers even though access has been facilitated via management at a more formal level. Moreover, managers may try to subvert research by, on the one hand, granting access at a formal level but, on the other hand, coercing employees not to co-operate or to withhold important information from researchers. Where research is being conducted within a bureaucratic organisation with various levels through which consent needs to be negotiated and agreed, the problem of gaining access is a more time-consuming endeavour. Further, gaining access is never a once and for all activity – the researcher must constantly negotiate and renegotiate his/ her presence and relations with participants, often referred to as the need to maintain good ‘field relations’. The researcher may find it difficult trying to balance such relations with various participants who will usually have different and competing interests. Finally, it is important to recognise that gatekeepers not only control access to citizens but to many other important sources of data/records and information, for example, archival and other public records.
PLANN NG
DISASTER
The need to collect data provides further opportunities for the intrusion of value judgements. How, for example, should one find out about people’s offending behaviour? Should one spend time with them and judge for oneself (participant observation), should one interview and ask them (self-report), should one look at officially-kept statistics (secondary sources), or should one ask victims of crime about their experiences? Each method has its advantages and disadvantages, but the final choice is again likely to reflect the researcher’s own (or the sponsor’s) preferences or values.
2-13
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
It is important to assess the relative advantages and disadvantages of the various methods available in terms of the nature of the research problem itself. Thus, it is generally accepted that no one research method is superior to any other, since all have their relative strengths and weaknesses and should therefore be selected in relation to their suitability to the research problem at hand. It always depends upon what you want to derive from the data (what you hope to ascertain) and the external constraints of the study (for example, financial and temporal). All of the above will need to be considered before a final decision is made but, in general, there is a consensus that an eclectic approach is always favourable and may help to strengthen the overall validity of research findings. There has also been considerable debate about the issue of choosing the subjects of research. The choice made may simply reflect the ease with which the researcher might gain access to different groups (convenience). Social scientists have been criticised for tending to study relatively powerless groups, such as the poor and juveniles, rather than more powerful groups who are perhaps more prepared or better placed to prevent research being conducted on themselves. Hence, the debate which emerged with Becker’s question ‘Whose side are we on?’
MSC IN EMERGENCY PLANNING MANAGEMENT
2.3.4.3 Interpretation of Research Findings Anxious to please a sponsor (and eager perhaps to secure further funding) a researcher can indulge in selectivity — highlighting those findings which he or she thinks the sponsor might be particularly pleased to hear. Deciding which findings are ‘relevant’ or ‘important’ may therefore constitute another opportunity for the entry of value judgements. Important findings may also escape the researcher’s notice because they do not tally with what he or she believes ‘society at large’ is concerned with. Therefore, it may be beneficial to have a number of researchers involved in data analysis to add to the overall validity of findings. 2.3.4.4 Disseminating the Findings Once findings have been communicated to the sponsor, the researcher loses control of the way in which they are used. An example of what can happen is provided by an incident that occurred during the Vietnam War. Social scientists from the USA had been contracted to collect data on the religious allegiances of the local population. They were led to believe that the information would be used to discover the moral or political allegiances of the research subjects. Later, there were allegations that the findings were used by the military to select bombing targets. Social scientists have a duty to report findings to the scientific/academic community and the funders/ sponsors of research will often expect a written report. The issue of why social scientists do research and what they should do with their findings has been debated by a number of social theorists, for example, Howard Becker (1967), Robert Lynd (1964) and C. Wright Mills (1959). Social scientists also find that however careful they might be to present their findings in a dispassionate manner, publication is followed by the drawing of conclusions by the public, perhaps fanned by media reporting, which were never intended. Moreover, many researchers find that in the world of contract research they may not be privy to the ultimate ends to which their research findings might be used.
2-14
disaster
riskcrisis MANAGE RISK
2.3.5 Towards Objectivity in Social Science
Max Weber (1949) had been of the opinion that social science could aspire to a measure of objectivity, although, unlike Nagel, he accepted that the subject matter of the social sciences is of a completely different nature from that of the natural sciences. Barnes (1979) highlights this distinction between the two branches of science in his discussion of the development of ethical considerations in each. Some critics (he argues) believe that social science is in a state of ‘permanent adolescence’ and as such is restricted to an ‘idiographic’ stage of development, whereas natural science has long since moved on to ‘nomothetic’ knowledge. In relation to ethics, Barnes argues that there exists a fundamental difference between the two sciences since a natural scientist cannot negotiate with the particles under study: ... there is no society for the prevention of cruelty to atoms, and although Heisenberg’s principle of uncertainty may perhaps be understood to entail some kind of feedback relation between the atomic physicist and the particles he studies, the principle does not permit negotiation between them. Only when atoms and particles impinge on citizens do ethical problems arise. (Barnes, 1979: 18) Nagel aims to explore people’s normative judgements about the way the world should be, a task which is impossible to accomplish in a pure scientific manner. However, while the choice of the desired goal is bound to be a value judgement, Weber argued that social science could be dispassionate in measuring the best way to achieve those goals. In other words, Weber was arguing for a distinction to be drawn between helping people decide what they should want and telling them how they should go about getting it.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Up to now we have been quite pessimistic about the prospects for an objective, positive social science. Not all commentators, however, have given up hope of achieving at least a measure of objectivity. Nagel (1961) argued for a distinction between two types of value judgement, one of which he said we might be forced to accept, but the second of which held out the prospect of eradication. He encouraged a distinction between what he called ‘characterising’ value judgements and ‘appraising’ value judgements (Nagel, 1961: 492–5). Characterising judgements merely assess the extent to which something is or is not present (these are similar to Max Weber’s notion of positive statements): for example, a social scientist might estimate the degree to which the public disapprove of the siting of US nuclear weapons on British soil. In contrast, appraising judgements are expressed through some form of approval or disapproval (these are similar to Weber’s notion of normative statements). Nagel takes the view that a researcher who chooses a research project on the basis of a characterising value judgement is guilty of a lesser sin than one who proceeds on the basis of an appraising value judgement that a concern is justified. Nagel acknowledged that it is sometimes difficult to distinguish between these two types of value judgement, but he nevertheless believed that his distinction held out the prospect for social science being seen as a sub-branch of the natural sciences.
PLANN NG
DISASTER
2-15
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
2.3.6 Conclusion You should now be aware of some of the ways in which research can be criticised for being biased or value-affected. Despite the arguments of Nagel and Weber, it seems difficult to imagine a way of eradicating value judgements completely from the social sciences, and from a more radical point of view we need to ask whether this itself is desirable. There are a myriad number of ways in which values can and do impact upon research at various stages of the research process (for example, in defining the research problem, gaining support from sponsors/funders, the collection and analysis of data and the interpretation/dissemination of findings). The discussion so far reveals that values are not only influential during the research process but help to shape the very form that research agendas take. In conclusion, research in general and criminological research in particular, does not take place in a moral, normative or political vacuum. Such research can be conceived of as political in both the ‘party political’ and ‘non-party’ political sense. The following section considers the politics and ethics of research.
MSC IN EMERGENCY PLANNING MANAGEMENT
2.4 Ethics In this section, we will identify the principles which help differentiate ethical research from unethical research and consider some important debates, such as that between the proponents of ‘moral absolutism’ and ‘moral relativism’ and that between supporters of ‘covert’ and ‘overt’ research. We shall also consider whether social scientists should adhere to a policy of ‘non-interference’: that is, the notion that some areas of social life be exempt from scientific study. Finally, we shall identify the main ethical considerations pertaining to different types of research methods. We will commence with some definitions.
2.4.1 Definitions Ethics are generally defined as a set of moral standards by which people regulate their behaviour. Barnes (1979: 16) goes further when he defines ethics as factors which: ... arise when we try to decide between one course of action and another, not in terms of expediency or efficiency but by reference to standards of what is morally right or wrong. Thus, when we consider whether or not something is ‘ethical’, we are entering into the realm of philosophy. While ethics are often outlined in sets of rules governing the conduct of professionals (medical practitioners and lawyers, for example, have their own codes of ethics) they are nevertheless essentially sets of value judgements about what is acceptable and what is not. Barnes’ (1979) definition makes an important distinction between matters of principle and matters of expediency. Ethics concern not what is of advantage to the researcher or the research project, but, rather what is ‘right’ or ‘just’ in the interests of those who are the subjects of research. The word ethics is derived from the Greek ethos, meaning a person’s character, nature or disposition, and the synonym morality is derived from the Latin moralis, meaning custom, manners or character (Kimmel, 1988). Social scientists have found it difficult to reach consensus regarding what constitutes exactly an ‘ethical’ or ‘moral’ problem in their research activities. This stems, partly, from the sheer diversity of ethical problems that exist. There is also confusion over whether we should distinguish
2-16
disaster
riskcrisis MANAGE RISK
between ‘ethical’ and ‘moral’ problems. What constitutes an ethical problem or dilemma as opposed to a moral one? Kimmel (1988) argues that ethical problems are in fact the same as moral problems, while Smith observes that: ... one might maintain that a psychologist acted ethically in the sense of not having violated the profession’s codified rules of proper behaviour, but still feel that the behaviour was immoral. (Smith, cited in Kimmel, 1988: 27) Moral concerns, therefore, relate to acts which are considered to be right or wrong. Ethical considerations, on the other hand, are those which conform to a codified set of principles or standards. It is important to recognise, however, that the terms ethical and moral are both inextricably linked with values and can therefore be used interchangeably. The important difference inheres more in the principles which are used to judge rather than the terms themselves. Kimmel (1988: 28) explains how we as social scientists know that an ethical or moral research dilemma exists:
action that cannot be maximised simultaneously. Kimmel (1988: 36) provides the following summary of some of the main characteristics of ethical problems in social research: (1) The complexity of a single research problem can give rise to multiple questions of proper behaviour. (2) Sensitivity to ethical issues is necessary but not sufficient for solving them. (3) Ethical problems are the result of conflicting values. (4) Ethical problems can relate to both the subject matter of the research and the conduct of the research. (5) An adequate understanding of an ethical problem sometimes requires a broad perspective based on the consequences of research. (6) Ethical problems involve both personal and professional elements. (7) Ethical problems can pertain to science (as a body of knowledge) and to research (conducted in such a way as to protect the rights of society and research participants). (8) Judgements about proper conduct lie on a continuum ranging from the clearly unethical to the clearly ethical. (9) An ethical problem can be encountered as a result of a decision to conduct a particular study or a decision not to conduct the study.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
... a dilemma is apparent in research situations in which two or more desirable values present themselves in a seemingly mutually exclusive way, with each value suggesting a different course of
PLANN NG
DISASTER
2.4.2 Approaches to Ethical Decision-Making There was little attention paid to ethical issues in social scientific research in the 19th Century. According to Barnes (1979) this was mostly to do with the methodological stance of social thinkers at the time, that is, the dominance of the ‘natural science paradigm’.
2-17
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 2-18
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Compared with the natural sciences, however, the growth of interest in ethics among social scientists has been a relatively recent development. Some have argued that a concern with ethics began following the Second World War when the atrocious medical experiments carried out by Nazi German physicians in concentration camps led to the Nuremberg code of medical ethics for human experimentation. However, Barnes (1979: 22) argues that the burst of interest in ethics has stemmed less from ‘diffusion’ of the natural sciences and more from: ... an historical shift in the balance of power between the four parties to the research process, and from the institutionalisation of social inquiry in the ambient culture of industrialised societies. It is the outcome of a movement away from positivism towards a hermeneutic view of knowledge, and from an evaluation of knowledge as a source of enlightenment to an evaluation in terms of power and property. Thus, he argues that the movement away from positivism or the ‘natural science paradigm’ has engendered certain changes within the social structure, such that citizens have become closer to scientists, and the power of gatekeepers and sponsors has increased over that of scientists. He further argues that: ... ethical problems encountered in natural sciences and, to a lesser extent, in medicine relate to the application of scientific knowledge in the real world rather than to the process of scientific discovery in the laboratory. ... In social science at the present time ethical problems are posed by the process of social inquiry itself at least as much as by the application of the scientific findings of these inquiries. (Barnes, 1979: 17) We shall now consider two separate approaches to the question of ethics before proceeding to outline the debate between ‘moral absolutism’ and ‘moral relativism’. 2.4.2.1 Deontological and Consequentialist Approaches It is possible to distinguish between two broad approaches to the question of ethics: the deontological and consequentialist (or teleological) approach. Ethical judgements which are deontological are those which concern the act, while consequentialist approaches determine the morality of behaviour on the basis of its consequences. Deontological ethicists can be differentiated from consequentialist ethicists on the basis of the importance they assign to rules. This is not to suggest that they do not consider consequences, but rather that they perceive certain behaviour as morally right or morally wrong irrespective of the consequences. There are also variants within the deontological and consequentialist approaches. Kimmel (1988) identifies three types of deontological position: rule-deontological; act-deontological and rights-based. Rule-deontologists (for example, the philosopher Immanuel Kant [1965] and his notion of categorical imperatives) believe that rules which guide behaviour represent ‘morality’ and should always be followed, a position similar to the perspective of ‘moral absolutism’ (sometimes referred to as normative ethics). Implicit in this perspective of normative ethics is the belief in the existence of objective moral truths. For example, the rule that one should always gain the consent of respondents when conducting research might, from an act-deontological perspective be negated on the basis that deception in certain circumstances is necessary in order to obtain valid data. In contrast, rule-deontologists would argue that the principle of ‘informed consent’ should never be violated.
disaster
riskcrisis MANAGE RISK
On the other hand, act-deontologists adopt a more flexible position, arguing for particular judgements in certain situations, often referred to as situation ethics (Kimmel, 1988). The third deontological position can be found in the work of John Rawls (1971) and is known as the rights-based position. This perspective derives from the fundamental premise that rights and obligations are logically related. According to Kimmel (1988) the rights-based approach grew out of a general dissatisfaction with the utilitarian/consequentialist (or teleological) approach.
... the ethics of human research should be governed by the proposition that individuals are ends in and of themselves and must not figure as means to an end beyond their own interest ... in this view, research participants are believed to have certain inalienable rights that cannot be violated for knowledge. (Kimmel, 1988: 45–6) The distinction between deontological and consequentialist approaches has a parallel in Weber’s distinction between ends and means. A frequent justification for questionable means is that the ends justify them. This is, however, a dangerous path to tread as it might conceivably lead to justification for what most people’s value judgements would reject. Expressed as a generalisation, a statement such as ‘the ends justify the means’ is more plausible than when it is used about specific situations. During the late 1980s, for example, it might have led to compulsory universal HIV tests (questionable means) to control the spread of AIDS (laudable aim). There is also the question of who decides which aims are laudable. Earlier in this discussion we referred to the ethical position akin to ‘moral absolutism’ known as normative ethics. This perspective believes that rules and codes of conduct should be followed at all times. As argued already, this suggests an acceptance of objective moral truths. However, there is a contrasting position known as metaethics (or ‘moral relativism’) which asks important questions about how we arrive at such rules and how these come to be defined as absolute truths. From this perspective, codes of rules for guiding conduct are an infringement of the freedom of social scientists to make their own decisions and moral choices. They therefore question the objective nature of such rules and the way in which they can be justified (Frankena, 1973).
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The consequentialist/teleological approach asserts that an activity or rule of conduct is right if it produces the greatest well-being of everybody affected by it. The consequences of an act therefore determine its value. Kimmel (1988) identifies two variants of this approach: ethical egoism and utilitarianism. The former can be found in the philosophies of Nietzsche, Hobbes and Epicurus and represents the view that it is the greatest good of ‘the agent alone’ which matters most, rather than all persons affected. This contrasts with the latter position, which is best represented by John Stuart Mill’s (1957) principle of utility, the ethical principle that an individual ought to do that which promotes the greatest good, happiness or satisfaction for the most people (Kimmel, 1988). Faced with a moral dilemma, therefore, the utilitarian will consider the net consequences of his/her chosen action rather than his/her individual interests alone. This cost/benefit approach to the question of ethics has been heavily criticised by deontologists, however, on the basis that knowledge pursued for the greater good of society may violate the rights and dignity of individual participants. Thus:
PLANN NG
DISASTER
‘Moral absolutists’ adopt the point of view that strict adherence to ethical guidelines is always necessary, whereas ‘moral relativists’ hold that strict adherence to ethical codes of practice is destructive to the project of social science and therefore argue against the use of such codes.
2-19
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
2.4.3 Ethics and the Research Process Like value judgements, ethics have the potential to impinge at every stage of the research process – although they are often associated with the data collection stage. As in the case of value judgements, there are no absolute ‘right’ and ‘wrong’ ethical answers. However, social scientists, like other groups, are able to impose standards of conduct on those who wish to join their ranks. If certain standards of conduct are generally accepted, for example, by the publishers of social science journals, then one will find it difficult to disseminate work which has not been conducted in the approved manner.
MSC IN EMERGENCY PLANNING MANAGEMENT
An important ethical principle is the doctrine of ‘informed consent’. Subjects should consent freely to involvement in research, and in order to do so they must be told the precise nature of the research, who is doing it and what the findings will be used for. This requires research participants to have the research explained to them in terms which are meaningful. This may apply particularly to research with minors, where, in addition, consent may need to be gained by proxy. However, in certain circumstances it may be detrimental to research projects if participants are fully aware of the objectives of the research, for example, in experiments or field research. Similarly, in certain types of participant observation studies the researcher may have to choose between using covert methods or not doing research. In such cases, therefore, it may be appropriate to debrief and gain the informed consent of participants at the end rather than the beginning of research. While the quest for knowledge is a worthy goal in its own right, some consider this an insufficient justification for intruding into the lives of people without their full and ‘informed consent’. Others justify their intrusions by arguing that the knowledge gained can be used to improve the lives of research subjects; however, many would argue that this is a dubious justification (see 2.4.2.1). Ethical issues are, therefore, moral dilemmas which relate to the protection of subjects versus the freedom of social scientists to conduct research and disseminate findings. In recent years various professional associations have devised their own codes of ethics, for example, the British Psychological Association (BPA) and the British Sociological Association (see Appendix). These codes are useful guides which can alert us to important issues. However, it is important to recognise the distinction between ethical guides, which provide basic moral principles, and ‘situational’ ethics, which are more difficult to predict, anticipate or prevent and can arise within the research context itself (see Bulmer, 1987). We will now examine the main ethical principles in relation to different research methods.
2.4.4 Ethics and Different Research Methods You should now be aware of some of the main principles used to differentiate between ethical and unethical research. We shall now consider the ways in which these principles are applied to different techniques of research. 2.4.4.1 Experimental Research Experimental research is part of the quantitative tradition of social research and is usually associated with psychology and social psychology. It is considered positivist in its orientation and is particularly suited to research problems which allow the researcher to manipulate and control certain conditions in the pursuit of identifying relationships between independent (cause) and dependent (effect) variables. Through its use of statistical techniques, such as correlation coefficients and
2-20
disaster
riskcrisis MANAGE RISK
multivariate analysis, it seeks to establish causal relations. However, not all experiments take place in a controlled environment: for example, field experiments are conducted in natural settings as opposed to artificial ones.
who was situated in another room but they could hear the pupil. The equipment was rigged so that the pupil would not actually receive electric shocks but would feign pain upon receiving the shocks. The true aim of the study was to observe the extent to which subjects obeyed the authority of the experimenter. Milgram reports that, ... subjects were observed to sweat, tremble, stutter, bite their lips, groan and dig their fingernails into their flesh. These were characteristic rather than exceptional responses to the experiment. (Milgram, 1963: 375) The study demonstrated a surprisingly high percentage of subjects who would administer electric shocks to very dangerous levels. Ethical concerns centred around the use of deception and the emotional stress experienced by subjects. Critics argued that the experiments should have been abandoned at an early stage as soon as it became evident that participants were experiencing undue stress. Milgram, however, counter-argued that subjects were fully debriefed after the research and were free to withdraw from the experiment at any stage. He further argued that the ethicality of the experiment was less to do with the deception involved and more to do with the results obtained. Bickman and Zarantonello (1978) found support for this contention when they presented research subjects with four different written accounts of the Milgram experiment. They found that subjects were much more likely to designate an experiment unethical when the results indicated a high degree of obedience and were far less likely to do so when the results showed a low level of obedience. A similar kind of experiment which took place in a more ‘natural’ setting was conducted by Zimbardo et al. (1974). In this experiment that took place in a simulated prison environment (the basement of a Stanford University building) male students were divided into two groups: ‘guards’ and ‘prisoners’. The latter were informed that they would lose some of their civil rights but assured that they would encounter no physical abuse. They were dressed in standard uniforms and assigned numbers (deindividualised) while the ‘guards’ wore military-style uniforms. Unexpectedly, the volunteers overidentified with their roles and the experiment had to be abandoned. The guards became aggressive and dehumanising while the prisoners became passive and disorganised.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Ethical considerations are a significant issue in experimental research because it is an ‘intrusive’ form of research that ‘interferes’ with people’s lives. In contrast to field experiments (based in ‘natural’ settings), the classic experiment involves placing research subjects in an artificial setting and attempting to manipulate their thoughts, feelings or behaviour to see how they respond. A classic example is Milgram’s obedience study (1974), which sought to discover how the horrors of the Holocaust under the Nazis could have occurred by examining the strength of social pressure to obey authority. Milgram was a social psychologist at Yale University who conducted a series of experiments between 1960 and 1964. Volunteers were recruited and were falsely led to believe that they were involved in an experiment on the effects of punishment on learning. The volunteers were recruited and asked to sign ‘informed consent’ forms. They were then assigned to the role of ‘teacher’ to test a ‘pupil’s’ memory of word lists and administer electric shocks (increased up to 450 volts) to pupils (actually experimental accomplices) when they made mistakes. The teacher could not actually see the pupil,
PLANN NG
DISASTER
2-21
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 2-22
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
The major ethical issue in experimental research is the issue of deception that is so common to this method. Researchers must be careful not to place participants in dangerous or anxiety-inducing situations. They are not inanimate objects but human beings and should therefore be treated with dignity and respect. The major strength of experimental research is the ability to exercise control over research subjects but, paradoxically, the more control one has, the greater the likelihood of ethical problems. 2.4.4.2 Survey Research Survey research is probably the most widely used social research technique. It can be traced back to the census, for example, the Domesday Book, which was a famous census of England conducted by William the Conqueror (1085–1086). The survey approach has been developed within the positivist tradition of social science and facilitates the collection, classification, codification and quantification of large amounts of data. As you will see in Unit 6, surveys can generally be administered in one of three ways: the face-to-face interview, mail questionnaire and telephone interview. They are appropriate for finding out about people’s self-reported opinions, beliefs, attitudes and past or present behaviour. The advantages and disadvantages of survey research will vary according to the way in which it is administered. However, one of the major strengths of this research technique lies in its ability to collect large (or small) amounts of data from a large section of the population. Once the data have been collected the method allows for precise comparisons to be made between respondents (and responses). The method is also considered extremely reliable. The major ethical issue in survey research relates to the issue of privacy. For a discussion of ethical concerns in relation to survey research, see Marsh (1982: 125–46). Like the experimental method, the survey method is also intrusive as it probes into the lives of people, often in pursuit of private and personal information. Researchers involved in conducting any type of research are always confronted with the question: ‘Why should they talk to me?’ Experience has shown, however, that respondents are more likely to agree to take part in research when they are assured that information provided will be treated in confidence and they will remain anonymous. A second issue concerns the principle of ‘informed consent’. Respondents have the right to participate ‘voluntarily’, to refuse to answer questions at any time, and/or withdraw from the research altogether. Experience also shows that researchers will encounter fewer problems from respondents if their questions are well developed (always piloted) and carefully constructed. Participants should be treated with due respect and questions must be asked sensitively. Researchers should always try to minimise anxiety or discomfort to respondents. The traditional social survey type interview places the interviewee in a less powerful position than the researcher. Feminist researchers have, therefore, generally preferred the in-depth interview approach in order to transcend this imbalance of power relations (Oakley, 1981). The in-depth interview is thought to provide respondents with greater opportunity to control and dictate both the content and form of the data. Thus, feminist researchers, in reaction to the patriarchal nature of academic life, have allied together to construct their own epistemology and methodology (see for example, Cook and Fonow, 1990; Harding, 1987; Nielsen, 1990; Ramazanoglu, 1992; Reinharz, 1983, 1992).
disaster
riskcrisis MANAGE RISK
2.4.4.3 Nonreactive or Unobtrusive Research Both the survey method and the experiment are reactive or obtrusive techniques: that is, the people who are being studied are aware this is the case. Researchers are in direct contact with respondents (close proximity) and this can introduce both personal and procedural reactivity into the research context, for example, interview bias. Examples of nonreactive (unobtrusive) measures include: content analysis and secondary analysis of existing statistics, documents and survey data. This type of research does not involve the researcher in close physical proximity with respondents and raises a different set of ethical questions. The major ethical implications of nonreactive methods concern the use of data collected by others and the rights of respondents to privacy and confidentiality. A second issue concerns the socially (and politically) constructed nature of official statistics: political and social values can influence the generation of secondary data and this should be treated with caution. 2.4.4.4 Historical Comparative Research
2.4.4.5 Field Research The field researcher becomes deeply immersed in the life of the group, organisation or community under study and this personal involvement raises a host of ethical dilemmas. For instance, the debate between overt and covert methods has produced an extensive literature (see for example, Klockars and O’Connor, 1979; Sjoberg, 1968; Bulmer, 1982b; and special issues of the American Sociologist, 1978 and Social Problems, 1973, 1980). A classic example of a piece of covert research which caused much debate and controversy was the study by Humphreys (1970) of male homosexual encounters in public toilets. The researcher adopted the role of voyeur (watchqueen) and observed men engaging in ‘secret’ homosexual activities. The ethical problem at this stage of the research was that participants were not given the opportunity to consent freely. However, more controversially, Humphreys followed the men to their cars, took down their car licence number plates and about one year later approached the men to interview them in their own homes. This was under the guise of a different research project (a social health survey). The study raised problems concerning the ethical issues of consent and deception. However, it was also criticised for putting the men at risk of criminal prosecution, extortion and disruption to their family lives.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Ethical considerations are less relevant to this method since researchers do not have direct contact with the people they study. This method elicits the same ethical concerns, however, as other nonreactive methods (see above), and special considerations apply when the researcher is collecting primary historical sources. Errors in documentation can lead to the integrity of the research being damaged, and people may be resistant to research and try to destroy, damage or hide data. Furthermore, there is a need to be sensitive to the beliefs and customs of other societies when doing cross-cultural research and to avoid accusations of ethnocentrism, a term first coined by W.G. Sumner (1906), and which reflects prejudicial attitudes between in-groups and out-groups. Sensitivity to these issues can aid in the development of good working relations and may also help facilitate (rather than close off) the possibilities of future research. It may also help to protect researchers from getting into trouble with the authorities of other countries.
PLANN NG
DISASTER
2-23
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
We will now discuss the main three ethical issues/dilemmas that arise during field studies: (i) deception; (ii) confidentiality; and (iii) involvement with criminals/ deviants. (i) Deception Deception can arise in field research in a number of ways. For example, in the case of ‘covert’ research, participants are not aware that they are taking part in the study. However, more often than not some participants will be aware of their involvement in the research (overt) but it may become very tedious having to inform everybody you meet during the course of the study. Moreover, some researchers pose the question of whether field research (like daily life) is inevitably interactionally deceitful (Punch, 1986). During the course of research some participants will inevitably be deceived. In addition, once researchers have gained access they need to develop a suitable role and ‘good enough’ field relations. Sometimes a researcher will adopt a particular role in order to gain acceptance, ‘fit in’ or ‘pass’. This can also lead to people deception.
MSC IN EMERGENCY PLANNING MANAGEMENT
The most heated debate to have emerged in relation to the ethics of field research has centred on the debate between ‘covert’ and ‘overt’ research methods. Those who favour covert research methods (Douglas, 1976; Johnson, 1975) argue that the ends justify the means: that is, some areas of social life would be excluded from analysis if it were not used. This is the view of the ‘conflict methodologists’, who argue that deception is acceptable and legitimate because the explicit purpose of social research is to expose the powerful. Thus, Douglas (1979) vehemently protests against the use of codes of ethics because he argues they are a ‘deceit and a snare’. Others disagree with the use of covert research methods (Erikson, 1967; Bulmer, 1982b) and argue that it contravenes the principle of ‘informed consent’, undermining any trust between researcher and researched and breaching a subject’s right to privacy. According to Erikson (1967) covert research is ‘bad science’. This dilemma has encouraged a debate over whether sociologists should, in some instances, adopt a policy of ‘non-interference’: that is, should some areas of social life be exempt from study? Moreover, if a researcher uses covert participant observation they will usually have to become a member of the group and this can lead to further ethical dilemmas. For example, Patrick (1973) became a member of a violent gang in Glasgow in the late 1960s, hiding his true status from all but one of the gang members. He changed the names of those he wrote about and waited a number of years before publishing his findings in order to avoid any possibility that they would be identified. Nevertheless, he still felt compelled to publish under a pseudonym (perhaps this was in his own self-interest, as once his identity was exposed to the rest of the gang they beat him up). Field research often places researchers in difficult and uncompromising situations whereby ethical dilemmas are much harder to anticipate, predict or control. In such instances, on-the-spot decisions will have to be made: ... laughing and joking about buying a bundle of dope, Ralphie turned to the anthropologist and said, ‘Here’s a present for you’. The anthropologist felt Ralphie drop a heavy object into his raincoat pocket. It was a loaded snub-nosed revolver ... Urban American fieldwork, then, may confront the researcher with moral, ethical, and legal crises on an almost daily basis. (Soloway and Walters, 1977 cited in Punch, 1986: 11)
2-24
disaster
riskcrisis MANAGE RISK
(ii) Confidentiality During any field research study a researcher will often have access to a multitude of private and confidential data/information. As in other research methods it is important to take all the necessary precautions to protect the privacy of all members of the group and to make use of pseudonyms in field notes and other documentation. (iii) Involvement with Criminals/Deviants Doing field research with criminals and deviants raises special difficulties for researchers as they will always be likely to come into possession of what Fetterman (1989) terms ‘guilty knowledge’ or what Klockars (1979) calls getting ‘dirty hands’. This raises a series of interlinked ethical questions: • to what extent should researchers become involved in such illegal activities? • to what extent does a researcher’s inaction condone illegitimate/illegal behaviour? • to what extent should researchers be protected from the legal state apparatus?
Social scientists, like other professionals, have developed their own code of ethics, but these are not generally as inflexible as those laid down for doctors and lawyers. In Australia, however, research is no longer simply guided by ‘codes’ but has become part of the legal structure (Sarantakos, 1994). The academic freedom previously enjoyed by researchers is now controlled by strict rules and regulations such that research can be a very dangerous practice. It is up to individual researchers to accept responsibility for their actions. Codes can operate as useful guides, but researchers should also rely on academic convention and consultation with academic peers. They should always consider the ethical principles of privacy, harm, deception, anonymity, confidentiality and consent. In conclusion, in terms of the debate between ‘conflict methodologists’ and those who support the strict adherence to ‘codes of ethics’, a balance somewhere in the middle might be the best solution. Thus, researchers should not feel they have the right to ride roughshod over the rights of research participants in their quest for knowledge and must take these ethical principles seriously. However, perhaps we should question rigid ‘codes’ of practice, for they may be detrimental to the project of social science, especially if they do inadvertently protect the powerful rather than the weak (Galliher, 1982).
2.5 Main Points • The distinction between values and ethics is not readily apparent and is sometimes blurred. An allegation of unethical research practice generally implies criticism, suggesting that a researcher has acted in some way dishonourably by seeking to mislead, or by, for example, harming research subjects. Criticism based on identification of the researcher’s values need not imply blame.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
2.4.5 Conclusion
PLANN NG
DISASTER
• All social research raises not only ethical and moral considerations but ‘political’ ones as well. All research is political in the ‘non-party’ sense since there are always questions concerning power relations and specifically the issues of access, privacy, honesty, trust, confidentiality and anonymity.
2-25
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
• Social scientists engage in making statements about the nature of society and its members. It is important, therefore, to differentiate between the different types of statements they make. • The controversy surrounding the issue of values and their role in social research can be understood in terms of the debate between positivism and phenomenology. The assumptions that underpin this debate have implications for the way in which research is conducted. • The in-depth interview approach is generally preferred by feminist researchers because it is supposed to subvert the hierarchical (and unequal) nature of power relations within the researcher–researched relationship (Oakley, 1981). • Gouldner (1970, 1976) wrote about the ‘myth of value-free sociology’, which he argued has helped sociologists to evade the moral implications of their work. • Values can enter the research process at various stages.
MSC IN EMERGENCY PLANNING MANAGEMENT
• The very first stage of the research process – defining the research problem – is one of the most difficult to ‘cleanse’ of value judgements. What we choose to research is affected by our own values. • Research can be influenced by the value judgements of sponsors and funders. • Sponsors can influence the outcomes of research and the way in which it is (or is not) disseminated. • Social scientists are dependent upon the co-operation of ‘gatekeepers’, who can provide, or not provide, access (formal entry) to the citizens/respondents of research. • Gatekeepers also control access at a more ‘informal’ level: for example, employees within a particular organisation may refuse to co-operate with researchers even though access has been facilitated via management at a more formal level. • The need to collect data provides further opportunities for the intrusion of value judgements. • Anxious to please a sponsor (and eager perhaps to secure further funding) a researcher can indulge in selectivity – highlighting those findings which he or she thinks the sponsor might be particularly pleased to hear. • Once findings have been communicated to the sponsor, the researcher loses control of the way in which they are used. • Nagel (1961) argued for a distinction between two types of value judgement: (i) characterising judgements that merely assess the extent to which something is or is not present; and (ii) appraising judgements that are expressed through some form of approval or disapproval. • Ethics are generally defined as a set of moral standards by which people regulate their behaviour, and when we consider whether or not something is ‘ethical’ we are entering into the realm of philosophy.
2-26
disaster
riskcrisis MANAGE RISK
• Moral concerns relate to acts which are considered to be right or wrong. Ethical considerations, on the other hand, are those that conform to a codified set of principles or standards. • The growth of interest in ethics among social scientists has been a relatively recent development. • It is possible to distinguish between two broad approaches to the question of ethics: (i) the deontological approach, which focuses on the act; and (ii) consequentialist (or teleological) approaches, which determine the morality of behaviour on the basis of its consequences. • Deontological ethicists’ approaches can be differentiated from consequentialist approaches on the basis of the importance they assign to rules. • Like value judgements, ethics have the potential to impinge at every stage of the research process – although they are often associated with the data collection stage.
• Ethical considerations are a significant issue in experimental research because it is an ‘intrusive’ form of research that ‘interferes’ with people’s lives. • The major ethical issue in survey research relates to the issue of privacy. • The major ethical implications of nonreactive methods concern the use of data collected by others and the rights of respondents to privacy and confidentiality. • There is a need to be sensitive to the beliefs and customs of other societies when doing cross-cultural research and to avoid accusations of ethnocentrism. • There are three ethical issues/dilemmas that arise during field studies: (i) deception; (ii) confidentiality; and (iii) involvement with criminals/deviants. • Social scientists have developed their own code of ethics, but these are not generally as inflexible as those laid down for doctors and lawyers.
2.6 Study Questions You should now answer each of the following study questions in approximately 300 words. We consider that this is an important exercise that will assist your understanding of material and help you to assess your progress on the course. Your answers are intended to form part of your course notes and should not be forwarded to the University.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• An important ethical principle is the doctrine of ‘informed consent’.
PLANN NG
DISASTER
1. What are the differences between the ethical dilemmas faced by natural and social scientists? 2. Describe the implications of research sponsorship for the production of value-free findings. 3. Describe the various ways in which values enter social science research.
2-27
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
2.7 Appendix – BSA: Statement of Ethical Practice (1993)1 The British Sociological Association gratefully acknowledges the use made of the ethical codes produced by the American Sociological Association, the Association of Social Anthropologists of the Commonwealth and the Social Research Association. Styles of sociological work are diverse and subject to change, not least because sociologists work within a wide variety of settings. Sociologists, in carrying out their work, inevitably face ethical, and sometimes legal, dilemmas which arise out of competing obligations and conflicts of interest. The following statement aims to alert the members of the Association to issues that raise ethical concerns and to indicate potential problems and conflicts of interest that might arise in the course of their professional activities.
MSC IN EMERGENCY PLANNING MANAGEMENT
While they are not exhaustive, the statement points to a set of obligations to which members should normally adhere as principles for guiding their conduct. Departures from the principles should be the result of deliberation and not ignorance. The strength of this statement and its binding force rest ultimately on active discussion, reflection, and continued use by sociologists. In addition, the statement will help to communicate the professional position of sociologists to others, especially those involved in or affected by the activities of sociologists. The statement is meant, primarily, to inform members’ ethical judgements rather than to impose on them an external set of standards. The purpose is to make members aware of the ethical issues that may arise in their work, and to encourage them to educate themselves and their colleagues to behave ethically. The statement does not, therefore, provide a set of recipes for resolving ethical choices or dilemmas, but recognises that often it will be necessary to make such choices on the basis of principles and values, and the interests of those involved. Professional Integrity Members should strive to maintain the integrity of sociological enquiry as a discipline, the freedom to research and study, and to publish and promote the results of sociological research. Members have a responsibility both to safeguard the proper interests of those involved in or affected by their work, and to report their findings accurately and truthfully. They need to consider the effects of their involvement and the consequences of their work or its misuse for those they study and other interested parties. While recognising that training and skill are necessary to the conduct of social research, members should themselves recognise the boundaries of their professional competence. They should not accept work of a kind that they are not qualified to carry out. Members should satisfy themselves that the research they undertake is worthwhile and that the techniques proposed are appropriate. They should be clear about the limits of their detachment from and involvement in their areas of study. Members should be careful not to claim an expertise in areas outside those that would be recognised academically as their true fields of expertise. Particularly in their relations with the Media, members should have regard for the reputation of the discipline and refrain from offering expert commentaries in a form that would appear to give credence to material which as researchers they would regard as comprising inadequate or tendentious evidence.
2-28
disaster
riskcrisis MANAGE RISK
Relations With And Responsibilities Towards Research Participants Sociologists, when they carry out research, enter into personal and moral relationships with those they study, be they individuals, households, social groups or corporate entities. Although sociologists, like other researchers, are committed to the advancement of knowledge, that goal does not, of itself, provide an entitlement to override the rights of others. Members must satisfy themselves that a study is necessary for the furtherance of knowledge before embarking upon it. Members should be aware that they have some responsibility for the use to which their research may be put. Discharging that responsibility may on occasion be difficult, especially in situations of social conflict, competing social interests or where there is unanticipated misuse of the research by third parties. 1. Relationships With Research Participants
well as those more powerful than themselves, research relationships are frequently characterised by disparities of power and status. Despite this, research relationships should be characterised, whenever possible, by trust. In some cases, where the public interest dictates otherwise and particularly where power is being abused, obligations of trust and protection may weigh less heavily. Nevertheless, these obligations should not be discarded lightly. (b) As far as possible, sociological research should be based on the freely given informed consent of those studied. This implies a responsibility on the sociologist to explain as fully as possible, and in terms meaningful to participants, what the research is about, who is undertaking and financing it, why it is being undertaken, and how it is to be promoted. (i) Research participants should be aware of their right to refuse participation whenever and for whatever reason they wish. They should also not be under the impression that they are required to participate. (ii) Research participants should understand how far they will be afforded anonymity and confidentiality and should be able to reject the use of data-gathering devices such as tape recorders and video cameras. (iii) Where there is a likelihood that data may be shared with other researchers, the potential uses to which the data might be put may need to be discussed with research participants. (iv) When filming or recording for research purposes, sociologists should make clear to research participants the purpose of the filming or recording, and, as precisely as possible, to whom it will be communicated. Sociologists should be careful, on the one hand, not to give unrealistic guarantees of confidentiality and, on the other, not to permit communication of research films or records to audiences other than those to which the research participants have agreed.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
(a) Sociologists have a responsibility to ensure that the physical, social and psychological well-being of research participants is not adversely affected by the research. They should strive to protect the rights of those they study, their interests, sensitivities and privacy, while recognising the difficulty of balancing potentially conflicting interests. Because sociologists study the relatively powerless as
PLANN NG
DISASTER
(v) It should also be borne in mind that in some research contexts, especially those involving field research, it may be necessary for the obtaining of consent to be regarded, not as a once-and-for-all prior event, but as a process, subject to renegotiation over time. In addition, particular care may need to be taken during periods of prolonged fieldwork where it is easy for research participants to forget they are being studied.
2-29
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
(vi) In some situations access to a research setting is gained via a ‘gatekeeper’. In these situations members should adhere to the principle of obtaining informed consent directly from the research participants to whom access is required, while at the same time taking account of the gatekeepers’ interest. Since the relationship between the research participant and the gatekeeper will continue long after the sociologist has left the research setting, care should be taken not to inadvertently disturb that relationship unduly.
(c) It is incumbent upon members to be aware of the possible consequences of their work. Wherever possible they should attempt to anticipate, and to guard against, consequences for research participants which can be predicted to be harmful. Members are not absolved from this responsibility by the consent given by research participants. (d) In many of its guises, social research intrudes into the lives of those studied. While some participants may find the experience a positive and welcome one, for others the experience may be disturbing. This can be particularly so if they perceive apparent intrusions into their private and personal worlds, or where research gives rise to false hopes, uncalled for self-knowledge, or unnecessary anxiety. Members should consider carefully the possibility that the research experience may be a disturbing one and, normally, should
MSC IN EMERGENCY PLANNING MANAGEMENT
attempt to minimise disturbance to those participating in research. It should be borne in mind that decisions made on the basis of research may have effects on individuals as members of a group, even if individual research participants are protected by confidentiality and anonymity. (e) Special care should be taken where research participants are particularly vulnerable by virtue of factors such as age, social status and powerlessness. Where research participants are ill or too young or too old to participate, proxies may need to be used in order to gather data. In these situations care should be taken not to intrude on the personal space of the person to whom the data ultimately refer, or to disturb the relationship between this person and the proxy. Where it can be inferred that the person about whom data are sought would object to supplying certain kinds of information, that material should not be sought from the proxy. 2. Covert Research There are serious ethical dangers in the use of covert research but covert methods may avoid certain problems. For instance, difficulties arise when research participants change their behaviour because they know they are being studied. Researchers may also face problems when access to spheres of social life is closed to social scientists by powerful or secretive interests. However, covert methods violate the principles of informed consent and may invade the privacy of those being studied. Participant or non-participant observation in non-public spaces or experimental manipulation of research participants without their knowledge should be resorted to only where it is impossible to use other methods to obtain essential data. In such studies it is important to safeguard the anonymity of research participants. Ideally, where informed consent has not been obtained prior to the research it should be obtained post-hoc. 3. Anonymity, Privacy And Confidentiality (a) The anonymity and privacy of those who participate in the research process should be respected. Personal information concerning participants should be kept confidential. In some cases it may be necessary to decide whether it is proper or appropriate even to record certain kinds of sensitive information.
2-30
disaster
riskcrisis MANAGE RISK
(b) Where possible, threats to the confidentiality and anonymity of research data should be anticipated by researchers. The identities and research records of those participating in research should be kept confidential whether or not an explicit pledge of confidentiality has been given. Appropriate measures should be taken to store research data in a secure manner. Members should have regard to their obligations under the Data Protection Act. Where appropriate and practicable, methods for preserving the privacy of data should be used. These may include the removal of identifiers, the use of pseudonyms and other technical means for breaking the link between data and identifiable individuals such as ‘broadbanding’ or ‘micro-aggregation’. Members should also take care to prevent data being published or released in a form which would permit the actual or potential identification of research participants. Potential informants and research participants, especially those possessing a combination of attributes which make them readily identifiable, may need to be reminded that it can be difficult to disguise their identity without introducing an unacceptably large measure of distortion into the data.
obligations in this respect. By the same token, sociologists should respect the efforts taken by other researchers to maintain anonymity. Research data given in confidence do not enjoy legal privilege: that is they may be liable to subpoena by a court. Research participants may also need to be made aware that it may not be possible to avoid legal threats to the privacy of the data. (d) There may be less compelling grounds for extending guarantees of privacy or confidentiality to public organisations, collectivities, governments, officials or agencies than to individuals or small groups. Nevertheless, where guarantees have been given they should be honoured, unless there are clear and compelling reasons not to do so. (e) During their research members should avoid, where they can, actions which may have deleterious consequences for sociologists who come after them or which might undermine the reputation of sociology as a discipline. Relations With And Responsibilities Towards Sponsors And/Or Funders A common interest exists between sponsor, funder and sociologist as long as the aim of the social inquiry is to advance knowledge, although such knowledge may only be of limited benefit to the sponsor and the funder. That relationship is best served if the atmosphere is conducive to high professional standards. Members should attempt to ensure that sponsors and/or funders appreciate the obligations that sociologists have not only to them, but also to society at large, research participants and professional colleagues and the sociological community. The relationship between sponsors or funders and social researchers should be such as to enable social inquiry to be undertaken as objectively as possible. Research should be undertaken with a view to providing information or explanation rather than being constrained to reach particular conclusions or prescribe particular courses of action.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
(c) Guarantees of confidentiality and anonymity given to research participants must be honoured, unless there are clear and overriding reasons to do otherwise. Other people, such as colleagues, research staff or other employees given access to the data must also be made aware of their
PLANN NG
DISASTER
1. Clarifying Obligations, Roles And Rights (a) Members should clarify in advance the respective obligations of funders and researchers, where possible in the form of a written contract. They should refer the sponsor or funder to the relevant
2-31
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
parts of the professional code to which they adhere. Members should also be careful not to promise or imply acceptance of conditions which are contrary to their professional ethics or competing commitments. Where some or all of those involved in the research are also acting as sponsors and/ or funders of research the potential for conflict between the different roles and interests should also be made clear to them. (b) Members should also recognise their own general or specific obligations to the sponsors whether contractually defined or only the subject of informal and often unwritten agreements. They should be honest and candid about their qualifications and expertise, the limitations, advantages and disadvantages of the various methods of analysis and data, and acknowledge the necessity for discretion with confidential information obtained from sponsors. They should also try not to conceal factors which are likely to affect satisfactory conditions or the completion of a proposed research project or contract. 2. Pre-empting Outcomes And Negotiations About Research
MSC IN EMERGENCY PLANNING MANAGEMENT
(a) Members should not accept contractual conditions that are contingent upon a particular outcome or set of findings from a proposed inquiry. A conflict of obligations may also occur if the funder requires particular methods to be used. (b) Members should try to clarify, before signing the contract, that they are entitled to be able to disclose the source of their funds, its personnel, the aims of the institution, and the purposes of the project. (c) Members should also try to clarify their right to publish and spread the results of their research. (d) Members should be prepared to clarify with sponsors the methods of analysis to be used. 3. Guarding Privileged Information And Negotiating Problematic Sponsorship (a) Members are frequently furnished with information by the funder who may legitimately require it to be kept confidential. Methods and procedures that have been utilised to produce published data should not, however, be kept confidential. (b) When negotiating sponsorships members should be aware of the requirements of the law with respect to the ownership of and rights of access to data. (c) In some political, social and cultural contexts some sources of funding and sponsorship may be contentious. Candour and frankness about the source of funding may create problems of access or co-operation for the social researcher but concealment may have serious consequences for colleagues, the discipline and research participants. The emphasis should be on maximum openness. (d) Where sponsors and funders also act directly or indirectly as gatekeepers and control access to participants, researchers should not devolve their responsibility to protect the participants’ interests onto the gatekeeper. Members should be wary of inadvertently disturbing the relationship between participants and gatekeepers since that will continue long after the researcher has left. 4. Obligations To Sponsors And/Or Funders During The Research Process (a) Members have a responsibility to notify the sponsor and/or funder of any proposed departure from the terms of reference of the contracted research.
2-32
disaster
riskcrisis MANAGE RISK
(b) A research study should not be undertaken on the basis of resources known from the start to be inadequate, whether the work is of a sociological or inter-disciplinary kind. (c) When financial support or sponsorship has been accepted, members must make every reasonable effort to complete the proposed research on schedule, including reports to the funding source. (d) Members should be prepared to take comments from sponsors or funders or research participants. (e) Members should, wherever possible, spread their research findings. (f) Members should normally avoid restrictions on their freedom to publish or otherwise broadcast research findings. APPROVED AGM ’92; AMENDED AGM ’93. The authors would like to thank the British Sociological Association for their permission to use this statement. 1
American Sociologist (1978) Special Issue on ‘Regulation of Research’, 13. Barnes, J. (1977) The Ethics of Inquiry in Social Science: Three Lectures, Oxford: Oxford University Press. Barnes, J. (1979) Who Should Know What? Social Science, Privacy and Ethics, Harmondsworth: Penguin. Becker, H. S. (1967) ‘Whose Side Are We On?’ Social Problems, 14: 239–47. Becker, H. S. (1970) Sociological Work, London: Allen Lane. Bell, C. (1977) ‘Reflections in the Banbury Restudy’, in C. Bell and H. Newby (eds) Doing Sociological Research, London: Allen & Unwin. Bell, C. and Newby, H. (1972) Community Studies, London: Allen & Unwin. Bell, C. and Newby, H. (1977) Doing Sociological Research, London: Allen & Unwin. Bickman, L. and Zarantonello, M. (1978) ‘The Effects of Deception and Level of Obedience on Subjects’ Ratings of the Milgram Study’, Personality and Social Psychology Bulletin, 4: 81–5.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
2.8 Bibliography
PLANN NG
DISASTER
BPS (1993) Code of Conduct, Ethical Principles & Guidelines, The British Psychological Society, Leicester. Brogden, M. and Shearing, C. (1993) Policing for a New South Africa, London: Routledge.
2-33
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
BSA (1993) Statement of Ethical Practice Ratified by the Annual General Meeting of the British Sociological Association, April 1992; Amended AGM 1993. Bulmer, M. (1982a) The Uses of Social Research, London, Allen & Unwin. Bulmer, M. (1982b) Social Research Ethics, London: Macmillan Press. Bulmer, M. (ed.) (1986) Sociological Research Methods: An Introduction, London: Macmillan. Bulmer, M. (1987) ‘Ethics in Social Research’, in J. Kuper (ed.) Methods, Ethics and Models, London: Routledge. Clarke, M. (1975) ‘Survival in the Field: Implications of Personal Experience in Field-work’, Theory and Society, 2(1): 95–123.
MSC IN EMERGENCY PLANNING MANAGEMENT
Cohen, S. and Taylor, L. (1972) Psychological Survival: The Experience of Long Term Imprisonment, Harmondsworth: Penguin. Cook, J. A. and Fonow, M. M. (1990) ‘Knowledge and Women’s Interests: Issues of Epistemology and Methodology in Feminist Sociological Research’, in J.M. Nielsen (ed.) Feminist Research Methods: Exemplary Readings in the Social Sciences, London: Westview Press. Diener, E. and Crandall, R. (1978) Ethics in Social and Behavioural Research, Chicago: University of Chicago Press. Dingwall, R. (1980) ‘Ethics and Ethnography’, Sociological Review, 28(4): 871–91. Dixon, B. R., Bouma, G. D. and Atkinson, G. B. J. (1987) A Handbook of Social Science Research: A Comprehensive and Practical Guide for Students, Oxford: Oxford University Press. Douglas, J. D. (1976) Investigative Social Research, London: Sage. Douglas, J. D. (1979) ‘Living Morality Versus Bureaucratic Fiat’, in C. B. Klockars and F. W. O’ Connor (eds) Deviance and Decency, Beverly Hills, CA: Sage. Erikson, K. (1967) ‘A Comment on Disguised Observation in Sociology’, Social Problems, 14: 366–73. Fetterman, D. M. (1989) Ethnography: Step by Step, Newbury Park, CA: Sage. Finch, J. (1993) ‘“It’s Great to Have Someone to Talk To”: Ethics and Politics of Interviewing Women’, in M. Hammersley (ed.) Social Research: Philosophy, Politics and Practice, London: Sage. Frankena, W. K. (1973) Ethics (2nd edn), Englewood Cliffs, NJ: Prentice-Hall. Galliher, J. F. (1982) ‘The Protection of Human Subjects: A Re-examination of the Professional Code of Ethics’, in M. Bulmer (ed.) Social Research Ethics, London: Macmillan.
2-34
disaster
riskcrisis MANAGE RISK
Genders, E. and Player, E. (1989) Race Relations in Prisons, Oxford: Clarendon. Gilbert, M. (ed.) (1993) Research, Theory and Method, London: Sage. Gouldner, A. W. (1970) The Coming Crisis of Western Sociology, New York: Basic Books. Gouldner, A. W. (1976) ‘The Dark Side of the Dialectic: Toward a New Objectivity’, Sociological Inquiry, 46: 3–16. Hammersley, M. (ed.) (1993) Social Research: Philosophy, Politics and Practice, London: Sage. Harding, S. (ed.) (1987) Feminism and Methodology, Bloomington: Indiana University Press. Holdaway, S. (1980) ‘The Occupational Culture of Urban Policing: An Ethnographic Study’, Unpublished PhD thesis, University of Sheffield.
Humphreys, L. (1970) Tearoom Trade: Impersonal Sex in Public Places, Chicago: Aldine. Johnson, J. M. (1975) Doing Field Research, New York: Free Press. Junker, B. H. (ed.) (1960) Field Work, Chicago: University of Chicago Press. Jupp, V. (1993) Methods of Criminological Research, London: Routledge. Kant, I. (1965) The Metaphysical Elements of Justice, Indianapolis: Bobbs-Merrill (originally published in 1797). Kimmel, A. J. (1988) Ethics and Values in Applied Social Research, London: Sage. Klockars, C. B. (1979) ‘Dirty Hands and Deviant Subjects’, in C. B. Klockars and F. W. O’ Connor (eds) Deviance and Decency: The Ethics of Research with Human Subjects, Beverly Hills, CA: Sage. Klockars, C. B. and O’Connor, F. W. (eds) (1979) Deviance and Decency: The Ethics of Research with Human Subjects, Beverly Hills, CA: Sage. Lazarfield, P. F. and Rosenberg, M. (eds) (1955) The Language of Social Research, Glencoe, IL: Free Press.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Homan, R. (1991) The Ethics of Social Research, Harlow: Longman.
PLANN NG
DISASTER
Lee, A. M. (1978) Sociology for Whom? New York: Oxford University Press. Lee, R. M. (1993) Doing Research on Sensitive Topics, London: Sage. Lynd, R. S. (1964) Knowledge for What? The Place of Social Science in American Culture, New York: Grove.
2-35
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
McCall, G. and Simmons, J.L. (eds) (1969) Issues in Participant Observation, Reading, MA: AddisonWesley. Marsh, C. (1982) The Survey Method: The Contribution of Surveys to Sociological Explanation, Boston: George Allen & Unwin. May, T. (1993) Social Research: Issues, Methods and Process, Buckingham: Open University Press. Milgram, S. (1963) ‘Behavioural Study of Obedience’, Journal of Abnormal and Social Psychology, 6: 371–2. Milgram, S. (1974) Obedience to Authority, New York: Harper & Row. Mill, J. S. (1957) Utilitarianism, New York: Bobbs-Merrill (originally published in 1861). Mills, C. W. (1959) The Sociological Imagination, New York: Oxford University Press.
MSC IN EMERGENCY PLANNING MANAGEMENT
Nagel, E. (1961) The Structure of Science, London: Macmillan. Nielsen, J. M. (ed.) (1990) Feminist Research Methods: Exemplary Readings in the Social Sciences, London: Westview Press. Oakley, A. (1981) ‘Interviewing Women: A Contradiction in Terms’, in H. Roberts (ed.) Doing Feminist Research, London: Routledge & Kegan Paul. Patrick, J. (1973) A Glasgow Gang Observed, London: Eyre Methuen. Polsky, N. (1971) Hustlers, Beats and Others, Harmondsworth: Penguin. Punch, M. (1977) Progressive Retreat, Cambridge: Cambridge University Press. Punch, M. (1986) The Politics and Ethics of Fieldwork, London: Sage. Ramazanoglu, C. (1992) ‘On Feminist Methodology: Male Reason Versus Female Empowerment’, Sociology, The Journal of British Sociological Association, 26(2): 207–12. Rawls, J. (1971) A Theory of Justice, Cambridge, MA: Harvard University Press. Reinharz, S. (1983) ‘Experiential Analysis: A Contribution to Feminist Research’, in G. Bowles and R. Duelli-Klein (eds) Theories of Women’s Studies, Boston: Routledge & Kegan Paul. Reinharz, S. (1992) Feminist Methods in Social Research, New York: Oxford University Press. Reiss, A. J. (1979) ‘Conditions and Consequences of Consent in Human Subject Research’, in K. M. Wulff (ed.) Regulation of Scientific Inquiry: Social Concerns with Research, American Association for the Advancement of Science, pp. 161–84. Sarantakos, S. (1994) Social Research, Basingstoke: Macmillan.
2-36
disaster
riskcrisis MANAGE RISK
Schatzman, L. and Strauss, A. (1973) Field Research: Strategies for a Natural Sociology, Englewood Cliffs, NJ: Prentice-Hall. Sjoberg, G. (1968) Ethics, Politics and Social Research, Cambridge, MA: Schenkman. Social Problems (1973) Special Issue on ‘The Control of Social Research’, 21:1. Social Problems (1980) Special issue on ‘Ethical Problems of Fieldwork’, 27:3. Soloway, I. and Walters, J. (1977) ‘Working the Corner’, in R.S. Weppner (ed.) Street Ethnography, Beverly Hills, CA: Sage. Sumner, W.G. (1906) Folkways, New York: Dover. Thompson, H. (1967) Hell’s Angels, Harmondsworth: Penguin.
Van Maanen, J. (ed.) (1979) ‘Qualitative Methodology Special Issue’, Administrative Science Quarterly, 24(4): 519–680. Wax, R. H. (1971) Doing Fieldwork: Warnings and Advice, Chicago: University of Chicago Press. Weber, M. (1949) The Methodology of the Social Sciences, Glencoe, IL: The Free Press. Whyte, W. F. (1943, 1955) Street Corner Society, Chicago: University of Chicago Press. Yablonsky, L. (1968) The Hippy Trip, Harmondsworth: Penguin. Zimbardo, P. G. et al. (1974) ‘The Psychology of Imprisonment: Privation, Power and Pathology’, in Z. Rubin (ed.) Doing Unto Others, Englewood Cliffs, NJ: Prentice-Hall.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Van Maanen, J. (1978) ‘On Watching the Watchers’, in P.K Manning and J. Van Maanen (eds) Policing: A View from the Street, Santa Monica, CA: Goodyear.
PLANN NG
DISASTER
2-37
crisis
PLANN NG
MSC IN EMERGENCY PLANNING MANAGEMENT
2-38
crisisdisaster RISK MANAGE
risk
EMERGENCY
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 3 Reviewing the Literature
disaster
riskcrisis MANAGE RISK
3 Unit Three: Reviewing the Literature 3.1 Aims and objectives of this Unit The literature review is a crucial part of the research process. Research in any discipline is incremental. It builds slowly and steadily on the work of past researchers, with each new effort adding to the overall body of knowledge. Anyone interested in carrying out a new research project needs to have a good understanding of the current state of knowledge on the subject and of the methods and perspectives previous research efforts have used. The better an awareness a prospective researcher has of this, then the more likely it is that their own research will be well planned, appropriate and significant. On the other hand, a poor awareness of past efforts can lead to badly designed studies which are poorly carried out and which produce findings of little usefulness or importance. Thus, the potential researcher always needs to have a good grasp of the relevant research literature for the subject at hand. The best way to acquire and demonstrate this Overall, this Unit is designed to achieve the following aims: • to explore the role of the literature review in research; • to explain the process of conducting a literature review; • to describe the two main types of literature review to be found in professional and academic use.
3.2 What is a Literature Review? ‘Literature’ n. – the material in print on a particular subject ‘Review’ n. – a general survey or assessment of a subject or thing (The Oxford English Reference Dictionary, 1995) In the words of Fink (1998: 3), a literature review is ‘a systematic, explicit, and reproducible method for identifying, evaluating, and interpreting the existing body of recorded work produced by researchers, scholars, and practitioners’. For academics and scholars the potential body of recorded work is vast indeed. The world is literally filled with information and the amount of information is increasing at an incredible pace on a daily basis. The current version of the British Books in Print has over one million entries, and each day, hundreds if not thousands of new titles are being published. Added to this there are tens of thousands of specialist journals and magazines which publish on a regular basis, not to mention the thousands of mainstream media newspapers and magazines. Added to the growing pile is the output of radio and television and the truly boggling amount of information which is added to the Internet each hour. In short, the world is literally drowning in information and this poses particularly serious problems for potential researchers.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
awareness is through the literature review and this will be the subject for this Unit.
PLANN NG
DISASTER
As we shall see, good research depends on at least a decent knowledge of what has gone before. But with mountains of new information accumulating so quickly these days, trying to stay abreast of
3-3
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
MSC IN EMERGENCY PLANNING MANAGEMENT
developments becomes an extremely demanding task. For those who need access to information, be it for academic, professional or other reasons, it is a major challenge to ‘keep up with the literature’. However, this idea of the need to keep up to date with the literature on a continual basis may be one that is somewhat overplayed. For example, interviews with some of the world’s most productive (in terms of research output) and influential researchers in the social sciences has revealed that the typical pattern of working of these eminent figures was to access the literature on a strictly ‘need to know’ basis. In other words, when a specific piece of research or writing demanded up-to-date knowledge, they would have an information ‘blitz’ to find the necessary literature. Nevertheless, it is the case that there are times when those engaged in academic study need access to published information and have to find the necessary journals, books and so on, and then must attempt to make sense of the information gathered. A literature review is simply the end product of this process of accessing, reading, summarising and integrating published information. Further, it is standard practice do a literature review when writing up any research work. Thus, for example, the introduction to an empirical paper in a journal will very probably contain a succinct review of the relevant literature; the introduction to a Master’s dissertation will give a fuller account of the relevant literature; while the introduction to a PhD thesis will undoubtedly contain a substantially longer review of the literature.
3.3 Why Write a Literature Review? A review of the literature is important because without it you will not acquire an understanding of your topic, of what has already been done on it, how it has been researched, and what the key issues are. In your written project you will be expected to show that you understand previous research on your topic. This amounts to showing that you have understood the main theories in the subject area and how they have been applied and developed, as well as the main criticisms that have been made of work on the topic. The review is therefore a part of your academic development – of becoming an expert in the field. (Hart, 1998: 1) Conceivably one might write a literature review just for interest’s sake. However, there are two main reasons why literature reviews are published. First, as knowledge increases in any field, there comes a point at which it is helpful simply to have available a summary of what is known. This type of ‘state of the art’ literature review is always of value to researchers, both those working in that particular field and those who wish to bring themselves up to date in a given area. Of course, teachers and students also find this type of review enormously useful! Indeed, there are publications whose function it is to offer this type of review: these publications are generally called ‘Annual Review of . . .’ (e.g. Annual Review of Psychology and Annual Review of Crime and Justice). Many books and book chapters perform a similar function in offering a review of a specialist field (e.g. Blackburn, 1993; Kapardis, 1997). Second, it is the case that many researchers and practitioners are concerned with more applied issues and questions, such as what is the best means by which to study a given phenomenon, or what is the preferred style of intervention in changing behaviour? Thus, to follow the above, in planning a research project one might reasonably want to know the best way to measure a particular
3-4
disaster
riskcrisis MANAGE RISK
behaviour. For example, say you were interested in gaining information on offending behaviour in the workplace. In theory you could try to gain such information by using a self-report methodology, or perhaps arrest statistics, or maybe court reports (among other options). An appropriate review of the literature will provide you with information on the strengths and weaknesses of each of these approaches to measurement and will allow you to make an informed choice for your own project. Similarly, policy makers and practitioners can also find literature reviews extremely useful. If we take the example of a prison, those charged with managing and treating violent offenders might want to know what method of intervention will give the best outcome in terms of reducing violent behaviour. Again, a review of the relevant literature will allow informed choices to be made regarding the costing, planning and implementation of such interventions. Thus, the literature review can be an invaluable asset to a range of people, including practitioners and researchers, teachers and students, in helping them work more quickly and effectively. For the particular purposes of a researcher though, reviewing the literature typically fulfills four main aims. In brief, these are that the review:
• acts as an aid to designing and doing the research; • provides guidance and tools for analysing the data gathered; and • gives the researcher a framework in which to place findings and conclusions, and also provides ideas for the style of presentation. The general importance of the literature review is that it provides a clear demonstration of the writer’s level of knowledge and understanding of the subject matter. In more academic terms, a literature review also serves the purpose of acknowledging a research debt to others. By giving account of past research, the writer demonstrates that they understand that research is a process and that current research always builds on previous work. Arguably the greatest general role of the review though is in how it directly influences your own research. Knowing what has gone before allows one to avoid unnecessary repetition and also to avoid the pitfalls and mistakes made by previous researchers, and to learn from their collective experiences. Table 3.1 below summarises again the main reasons for a researcher to conduct a literature review. Table 3.1: Reasons for Conducting a Literature Review
1.
As an aid to the area of research Helps identify and flesh out the research topic: what, where, why
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• acts as an aid in selecting and refining the research topic, the research question(s) and the relevant hypotheses/propositions;
PLANN NG
DISASTER
Helps set specific research question(s): identification, revision, focus, definition Helps establish any hypotheses/propositions that should be tested: if any, what and why (i.e. theory building or hypotheses testing) 2.
As an aid to designing and doing research Introduces methodological issues and debates (e.g. soft vs hard data, etc.).
3-5
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 3-6
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Provides insights into past practices and outcomes Helps with research design: describes different strategies, methods and procedures, levels of analysis, exploratory or explanatory, pros and cons, etc. 3.
As an aid to analysis Existing theory (range, type, ideological standpoint, etc.) Will the research be hypotheses testing or theory building What analytical ‘tools’ and practices are most appropriate
4. As an aid to findings and conclusions Style of presentation that should be used Are the findings specific or generalisable The level of analysis/abstraction that should be used How significant are the research results (e.g. does it simply reinforce existing knowledge or is it ground breaking).
3.4 What is Included in a Literature Review? If, as its name implies, a literature review is to offer a review of the literature, then the starting point must be the extant literature. Therefore any literature reviewer must begin by finding out what relevant literature is available. A good way to consider this issue, is to look at how one well known literature review was conducted. The example we will discuss here is the Correctional Drug Abuse Treatment Effectiveness Project, known as CDATE. In brief, CDATE is ‘a comprehensive detailed review of the evaluation research on rehabilitation programmes for offenders’ (Pearson et al. 1997: 2). CDATE has set its boundaries to include studies of offender treatment conducted between 1968 and 1996. In the annals of literature reviews CDATE is remarkable, even legendary, for the thoroughness of its approach. As described by Pearson et al. (1997), the literature search for this project encompassed several stages. The first stage in CDATE was a search of existing computerised bibliographic databases such as Psychological Abstracts and Social Science Citation Index. Typically available in university libraries, these databases are extensive cross-referenced records of published work: they are accessed via a computer, generally using keywords such as an author’s name or a particular topic, to highlight and identify relevant publications. Once the publications are available, in either electronic or paper form, they in turn become a source of further information: thus, the researchers on the CDATE project used the citations from the publications identified in its database search to generate more potential sources of information. It is worth noting here that as a student on a Civil Safety and Security Unit course, you can access these (and other) databases directly through the University of Leicester’s Distance Learning Unit. The Unit will give you both advice and practical assistance, and you can find out more by going to their web-page at http://www.le.ac.uk/li/distance/index.htm.
disaster
riskcrisis MANAGE RISK
Returning to the CDATE review, while computers are invaluable, the researchers there also found that a (time consuming!) hand search of relevant journals can throw up publications that might otherwise be missed. In terms of library time, the CDATE researchers also took the step of examining library collections for specialist reports and monographs: Pearson et al. note that ‘we located in these library collections many reports that we would not otherwise have obtained’ (p. 4). Not content with this level of searching, the CDATE researchers also contacted directly authors working in the field to ask for copies of published and unpublished work. Similarly, they contacted organisations such as correctional agencies, government departments, independent agencies and international bodies such as the United Nations, with requests for documents; they also placed advertisements in the major journals and set up a newsgroup and mailing list on the Internet. Importantly, this search for information extended beyond English-speaking countries, producing more than 300 foreign language reports (mainly in Spanish, German and Scandinavian languages). Many of the publications elicited by this level of searching were unpublished and therefore would not have appeared on a database search.
Clearly, CDATE is an exceptional project and not every literature review could aim to be so all inclusive. Nonetheless, there are lessons to be taken for all reviewers, including students, from the strategies used by the CDATE researchers. First, the use of databases to identify relevant published work: there are many such databases in existence and they should be one of the first ports of call for any reviewer. Second, the use of the references generated by the database search, having identified the most important key words and authors as sources of information for more literature. (For most purposes, it is highly likely that by this stage the most significant published references will have been identified: there is probably a law of diminishing returns for most reviewers after this point in the proceedings.) Third, government reports can be a source of helpful material: these reports are often held in university libraries, although they are also increasingly to be found posted on the Internet: for example, the Canadian Correctional System posts many research reports and other documents on its web-site, while in the UK, the Home Office web-site boasts an increasingly rich collection of reports and research studies which are freely available. Finally, it is often worthwhile directly contacting researchers: most researchers are happy to share their work and a polite letter requesting copies of an author’s recent work will often produce a helpful response. In collecting information it is easy to become engrossed in the chase for just one more paper, just one more critical research study. As suggested above, there is a law of diminishing returns in chasing publications (even the CDATE researchers had to stop at some point!): most reviewers will satisfy themselves with a thorough search of the databases, identifying the most recent publications, the important web-sites, and perhaps a few letters to eminent researchers. In fact, rather than chasing publications, the critical issue for the reviewer once the material is gathered lies in deciding what to leave out of the eventual review.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Cleland et al. (1997) have revealed that (at that time) the search strategy they used for the CDATE project produced a staggering 9,183 citations that could potentially be used in their review.
PLANN NG
DISASTER
3-7
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
3.5 What is Left Out of a Literature Review? The CDATE researchers identified more than 9,000 studies but they then applied stringent exclusion criteria in order finally to include in the review only the most robust information. The exclusion criteria, shown in Table 3.2 below, help to shape the strength of the review by eliminating weak and poorly reported studies. It is important here to make two points: first, the exclusion criteria are explicit and open to scrutiny; second, the strategy of excluding weak studies gives more weight to the eventual conclusions of the review. It perhaps says a great deal about the quality of research in the field of offender treatment that after the exclusion criteria were applied to the 9,000 or so studies, the eventual number of studies included in the review was still 787. To place this figure of 787 studies in context, the next largest review available in the offender treatment field is the meta-analysis reported by Lipsey (1992) which included ‘just’ 397 studies. Table 3.2: Exclusion Criteria in the CDATE Review (adapted from Cleland et al., 1997: 3)
MSC IN EMERGENCY PLANNING MANAGEMENT
Studies were excluded from the review if: 1.
After-only design, with no control or comparison group
2.
Only a programme description
3.
Not in the 1968–1996 time frame
4.
Sampling methods were not reported
5.
Only a pilot study
6.
Unknown size of groups
7.
An insufficient description of treatment was provided
8.
Treatment cannot be replicated
9.
Only a clinical speculation of outcome
10. Only anecdotal outcome 11. Outcome measures too misleading 12. Findings inextricably confounded by external factors 13. Not a population in criminal justice custody, including probation and parole 14. Only a subjective evaluation, no outcome data As seen in the case of the CDATE effort, a search through the literature can easily turn up hundreds and even thousands of separate articles and papers, all of which could potentially be included in the finished review. However, all researchers face limits in terms of the time they can invest in the review and the amount of space they can devote to it. For example, MSc dissertations at the Civil Safety and Security Unit have a word limit imposed on them: they cannot be longer than 20,000 words. A literature review accounts for a only proportion of this, and as a result as a writer you have to be selective and discriminating.
3-8
disaster
riskcrisis MANAGE RISK
The good news is that while there might be hundreds of papers, articles and reports on the topic in question, in reality only a proportion of these articles will be of real relevance to your study. For example, if you were interested in reviewing articles on how to prevent airline disasters, a general search will produce articles on the causes of aircraft crashes, the physical, human and economic impact of such disasters, safety measures which have been introduced in the past, relevant legislation which exists, investigations into particular cases, and so on. Some of these papers will be very useful to your review, but many will not be. Before starting to review them all, you must sort through them to identify the ones that contain information on prevention.
good studies from poor ones. In the end, a literature review is always filtered through two screens (Fink, 1998). The first screen is essentially concerned with practical issues. It identifies studies that are potentially usable in that they cover the right topic, are in a language you read, and appear in a publication that you respect and can obtain in the time available. The second screen is about quality. Here, the concern is to produce the best available studies in terms of their adherence to methods that scientists and scholars regard as good for gathering reliable and valid evidence. In carrying out a literature review, you use the ‘practicality’ screen first, and the ‘quality’ screen second.
3.6 What Issues Should a Review Address? The obvious answer to the question of what a review should address is, of course, that the issues focused on in a review depend entirely on the purpose of the review! If for the sake of example we take a review that aims to offer an overview of an area, say the accuracy of recall of children as witnesses, what specific topics might reasonably be found in a review of such a subject? Once the relevant studies on children as witnesses have been identified and then selected for review, taking into account any exclusion criteria the reviewer might wish to apply, the literature review may seek to present some broad conclusions as to the accuracy of children as witnesses. However, the findings from the empirical research will need to be set against a background of methodological issues. Thus, the empirical findings regarding the accuracy of children as witnesses may vary as a function of sampling: children from certain backgrounds, or with certain types of experience, or of different educational ability may perform significantly differently as witnesses. Similarly, the method of stimulus presentation in different empirical studies might be important: for example, does accuracy of memory vary between live and videotaped presentations?; is there a difference in ability to be found when the children are observers or actually involved in the incident?
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Let us say that after sorting through the initial items, you then find that there are 50 studies that have focused on your general topic: preventing airline disasters. More than likely, some of these papers will be methodologically rigorous, deriving sound conclusions which are based on valid and reliable evidence. In contrast, some of the other papers will be methodologically weak: perhaps their conclusions rely on a few anecdotal cases, a flimsy analysis of evidence or maybe they only report the writer’s personal opinions on the matter and nothing else. To ensure the accuracy and quality of your review, you must engage in a screening process so that you can correctly distinguish
PLANN NG
DISASTER
3-9
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Yet further, there may be differences in research findings according to the type of questioning employed with the children. Thus, children’s ability to remember what they have witnessed may vary as a function of open versus closed questioning or narrative versus structured recall (see Unit 6 in this Module for a longer discussion of some of these issues with data gathering methods). Next there are issues of reliability and validity of measurement to consider: do the experimental designs of the various studies incorporate a style of measurement that will detect variations in the accuracy of memory? (A typical difficulty in this type of memory research is to use a methodology such as a highly structured questioning procedure that is too blunt, producing a ‘floor effect’ of uniformly low recall. Alternatively, an open questioning procedure, such as free narrative recall, can produce a ‘ceiling effect’ of uniformly high recall.) Yet further, not all studies will study the same exact variables, and reviewers must pick their way through a range of different ways of studying the same phenomenon. Finally, different studies will use different statistical methods, leaving the reviewer to try to make sense of experimental results that come from styles of different statistical analysis.
MSC IN EMERGENCY PLANNING MANAGEMENT
As is plain from the above, the reviewer’s task is far from simple: if you consider that the reviewer may typically be trying to make sense of more than one hundred studies, then the magnitude and complexity of the task becomes increasingly daunting. Ultimately, when beginning to tackle a literature review on any topic, it is useful to remember the following list of dos and don’ts:
Do … • identify and discuss the key relevant landmark studies on the topic; • include as much up-to-date material as possible; • check the details, such as how names are spelled; • examine your own biases and make them clear; • critically evaluate the material and show your analysis; • use extracts and examples to justify your analysis and argument; • be analytical, evaluative and critical and show this in your review; • manage the information: have a system for records management; • make your review worth reading; explain why the topic is interesting.
Don’t … • omit classic works or discuss core ideas without proper reference; • discuss outdated or only old materials; • misspell names or get the date of publication wrong; • use concepts to impress or without definition; • use jargon and discriminatory language to justify a parochial standpoint;
3-10
disaster
riskcrisis MANAGE RISK
• produce a list of items, even if annotated; a list is not a review; • accept any position at face value or believe everything that is written; • only produce a description of the content of what you have read; • drown in information by not keeping control; • make silly mistakes or be boring. (Hart, 1998: 219)
PLANN NG
DISASTER
EMERGENCY
3.7 Narrative Review
It is the case that narrative reviews, particularly when regularly updated, offer an excellent opportunity to keep abreast of a field of research. The major issue, of course, is that the content and conclusions of a narrative review are open to the subjective interpretations of the individual reviewer. An excellent example of the influence of the opinions of a single reviewer comes from the research looking at the effectiveness of treatment for criminal offenders. A narrative review of the treatment effectiveness literature reported by Martinson (1974) came to the view that there was little in the way of supportive evidence for the proposition that treatment could have any effect on recidivism, i.e. treatment in prison seemed to have no effect whatsoever on preventing criminals from reoffending. While it would be wrong to say that Martinson single-handedly changed the course of history, there is little doubt that his review was influential in prompting the widespread view that ‘nothing works’ in the treatment of offenders. In fact, in reaching the conclusion that ‘nothing works’ Martinson adopted the tactic of ‘vote counting’: that is, he counted the number of studies that had a positive effect in terms of reduced offending, the number of studies that failed to show any effect, and the number of studies that showed a negative effect (i.e. more offending after treatment). As there were fewer studies showing a positive effect than there were showing either no effect or else a negative effect, Martinson concluded as it were, that the ‘noes’ have it. However, Thornton (1987) took exception to this method of reviewing and performed a more sophisticated analysis of exactly the same studies that were included in the Martinson review. Thornton though focused much more closely on methodological issues, in other words, he looked much more closely at the quality of the different studies. He balanced the findings of each study in the review against the robustness of its methodology and design (in effect applying exclusion criteria to the studies used by Martinson).
MSC IN EMERGENCY PLANNING MANAGEMENT
The narrative (or descriptive) review is the classic style of review in the human and social sciences. In essence, the narrative review is a qualitative analysis, offered by the reviewer, of the material gathered together from the publications selected for inclusion in the review. Thus, the reviewer may comment on the methodological issues across the published literature; may attempt to compare and contrast the findings of the various studies, perhaps with cross-reference to methodological issues; and may attempt to draw conclusions with reference to the main messages to emerge from the body of research.
Thornton’s first conclusion, based on this review and analysis of the literature, was that many of the studies were methodologically very weak or flawed and would have been excluded from a
3-11
EMERGENCY
crisisdisaster RISK
MANAGE
more conservative review (cf. the exclusion criteria in the CDATE review). Thornton’s second conclusion, based on just examining the high quality studies, was that the literature actually showed that treatment could be effective in reducing offending! However, given the variability across studies, there were still a lot of unanswered questions and in particular, it was not clear from the evidence ‘what works, for whom, and under what conditions’. Nevertheless, Thornton’s assessment was far more optimistic than that reached by Martinson, and significantly it helped to catalyse renewed interest in offender treatment programmes. The variation in the conclusions drawn by Martinson and Thornton — based on a review of exactly the same literature — points to a fundamental weakness of narrative reviews. The reader is very much open to the views of the reviewer: even with identical literatures, different reviewers can sometimes reach very different conclusions. (It is perhaps worth noting in Martinson’s defence that he came to realise his mistake and he did later retract the ‘nothing works’ conclusion (Martinson, 1979).)
MSC IN EMERGENCY PLANNING MANAGEMENT
Prior to the mid-1980s the influence of subjectivity was the accepted state of affairs and narrative reviews were the main currency in the academic literature. However, the development of the technique of meta-analysis was to change radically the way in which many literature reviews are conducted.
3.8 Meta-Analysis Izzo and Ross (1990) offer a concise description of meta-analysis as ‘A technique that enables a reviewer to objectively and statistically analyse the findings of each study as data points . . . . The procedure of meta-analysis involves collection of relevant studies, using the summary statistics from each study as units of analysis, and then analysing the aggregated data in a quantitative manner using statistical tests’ (p. 135). Thus, unlike a narrative review in which the conclusions are based on the views of the reviewer, with meta-analysis the conclusions are based on the quantitative statistical analysis of the pooled findings of relevant individual studies. The main statistic used in meta-analysis is called the effect size. Simply, the effect size is an index of the magnitude of difference between two groups, typically an experimental (or treatment) group and a control group. For example, Lipsey and Wilson (1998) were concerned with the effects of intervention with serious juvenile delinquents. They defined arrest rates as their outcome variable and then calculated an effect size for each of the 200 individual studies of the effects of treatment on violent juveniles to be included in their meta-analysis. The effect size index used to represent the outcome for each study was the difference between the treatment and control group means on the selected recidivism measure, standardised by the pooled standard deviation [the meta-analysis revealed that] the overall mean recidivism value for treated juveniles was thus .12 standard deviation units less than that for the control group, and this effect was statistically significant. To put this in perspective, a mean effect size of .12 is equivalent to the difference between a 44% recidivism rate for treated juveniles and a 50% rate for the untreated control group. This 6-percentage-point difference represents a 12% decrease in recidivism (6/50). (Lipsey and Wilson, 1998: 318)
3-12
crisis
PLANN NG
risk
disaster
riskcrisis MANAGE RISK
There are two important points to make at this point: first, not all the pooled studies in a metaanalysis will use the same research methodology; second, the pooled studies are likely to be conducted in different settings, with differing populations and using different methods of treatment. While calculating overall treatment effects across all the studies is informative, knowledge of the difference between ‘high effect’ and ‘low effect’ studies is potentially of much greater practical and theoretical significance. The technique of meta-analysis allows the reviewer both to control for the effects of different methodologies and to search for high and low effect studies. In practice this fine grained analysis is achieved by coding studies across a number of factors and then controlling for these factors in the analysis. At this point, almost inevitably, the statistics become increasingly more complex both in concept and execution.
freedom for the statistical test, number of treatment subjects in the analysis, number of control subjects in the analysis, significance of difference’ (pp. 6–7); third, for the treatment engaged in by offenders, ‘type of treatment, type of difference between conditions, whether each treatment is targeted to specific treatment needs, planned treatment length, actual treatment length, number of contacts, rate of contacts, sequence of the treatment in an overall period of custody, cost of treatment, implementation of treatment’ (p. 7). Thus, each of the individual conditions – age, statistical details, type of treatment, and so on – will be individually noted for each study and entered into the meta-analysis. Of course, this is all very time consuming: ‘Based on 787 studies in which coding time is available, the median time to fully code a study was about 120 minutes. However, longer, more complicated studies took more than 10 hours to code’ (Cleland et al., 1997: 7). It is clear that meta-analysis is a time-consuming, statistically complex endeavour and, again as might be expected, like all research methods, meta-analysis is not without its critics. Sharpe (1997) notes three specific criticisms of meta-analysis. First, the ‘apples and oranges’ criticism that refers to mixing together in the meta-analysis studies that are so dissimilar in nature that any results produced by the analysis are meaningless; second, the ‘file drawer’ criticism of selecting for analysis only published studies and hence potentially ignoring those that remain filed away because they do not produce the ‘right’ results; third, the criticism of ‘garbage in garbage out’, in which poorly designed studies exert a disproportionate influence within a meta-analysis. Of course, each of these criticisms can be answered: first by careful filtering of studies for inclusion in meta-analysis thereby reducing the apples and oranges effect; second by trawling for a range of source material, including unpublished as well as published sources (cf. CDATE above); third, by coding within the analysis to allow for the relative methodological strength of different studies, so that more weight is given to better designed studies, thereby reducing the ‘garbage in garbage out’ effect.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The process of coding can be complex in itself, Cleland et al. (1997) describe their coding of the 787 studies included in their meta-analysis of the effects of treatment on offending. First, in coding for population variables the reviewers coded ‘age, setting of treatment, allocation of subjects to treatment group’ (p. 6); second, for data presentation, ‘type of analysis, measure of central tendency, variance, direction of the difference between treatment groups, type of statistical test, degrees of
PLANN NG
DISASTER
It is perhaps advantageous to see meta-analysis as a catalyst for new research rather than a final, definitive statement. As the number of meta-analyses across a given field increases, so narrative
3-13
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 3-14
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
reviews of (meta-meta-analyses!) have followed to try to extract the main messages from the metaanalyses. Following the theme of offender rehabilitation, there are several such narrative reviews of the meta-analyses which try to point to ‘what works’ in the treatment of offenders (Gendreau and Andrews, 1990; Hollin, 1994; Lösel, 1995). It is undoubtedly the case that the meta-analyses have precipitated renewed interest in the possibilities of reducing offending by working with offenders (e.g. McGuire, 1995). In pointing towards the components of effective work with offenders, the meta-analyses have inspired a new generation of treatment programmes based on ‘What Works’ principles. The evaluative research testing these programmes, not the findings of the meta-analyses, will be the acid test of this approach.
3.9 Conclusion … the literature review should be regarded as a process fundamental to any worthwhile research ... in any subject irrespective of the discipline. The research student has the responsibility to find out what already exists in the area in which they propose to do research before doing the research itself. The review forms the foundation for the research proper. The researcher needs to know about the contributions others have made to the knowledge pool relevant to their topic. It is the ideas and work of others that will provide the researcher with the framework for their own work; this includes methodological assumptions, data-collection techniques, key concepts and structuring the research into a conventional academic thesis. (Hart, 1998: 26) Hart (1998) provided a quick guide to the qualities of excellent, average and bad literature reviews. In his opinion, excellent reviews are characterised by showing a thorough review of the relevant literature. This literature has been systematically analysed and importantly all the main variables and arguments have been clearly identified. Further, the critical evaluation used by the reviewer has been firmly linked to justification and methodology. Moving to competent reviews, Hart noted that these showed a good survey of the main literature and also clearly identified the main variables and arguments. Only some links though were made to methodology and justification. Finally, poor reviews displayed an inadequate review of the literature (e.g. important and well-known studies were left out); there was a lack of critical evaluation and therefore no arguments or key variables relevant to the topic were identified. Further, Hart commented that poor reviews often lacked a proper bibliography or else reported a bibliography that was so large that it could not feasibly have been used! Ultimately though, there is no such thing as the perfect literature review. All reviews, regardless of their subject matter, are written from a particular perspective or standpoint. This perspective often originates from the school of thought, professional experience or ideological standpoint in which the reviewer is located. Writing a good review though need not be too difficult. Indeed, it is often regarded as one of the easier tasks in the research process, and the personal pay-offs can be considerable. Completing a good literature review not only increases substantially your level of expertise, it also develops skills and intellectual abilities you did not have before you began your research.
disaster
riskcrisis MANAGE RISK
As should be clear by now writing a literature review can be a time-consuming and highly skilled task. Not only must the reviewer know his or her field, they must also be able to extract the relevant messages, with due regard for methodological issues, from the research for the benefit of their readers. It is precisely because the time (and effort) spent by the reviewer in writing and publishing a review can be translated into time saved by researchers, teachers and students, that literature reviews are so valued. However, a literature review, regardless of whether it is a narrative or meta-analytic review, is a summary of a field of knowledge. As a summary, it should not be taken at face value: reading a review is not a substitute for reading original sources, nor can any review be the final, irrefutable word on any topic.
3.10 Main Points • A literature review is a systematic method for identifying, evaluating and interpreting the existing body of work produced by researchers and scholars.
As an aid to understanding the area of research
As an aid to designing and doing research
As an aid to deciding the appropriate method of analysis
As an aid to dealing with findings and conclusions
• High quality literature reviews are systematic and explicit, and base their findings on evidence from high quality studies. High quality studies are those which adhere to rigorous research standards. • A literature review is always filtered through two screens: the first screen is concerned with practicalities (i.e. the literature which can be feasibly accessed); the second screen is concerned with quality, that the best available studies are focused on. • Two methods are commonly used in literature reviews to combine the results of studies. The narrative review is primarily descriptive, while the second method, meta-analysis, uses formal statistical techniques to sum up the outcomes of similar studies and then to pool them.
3.11 Guide to Reading
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• In research, literature reviews are used for the four following reasons:
PLANN NG
DISASTER
Students interested in reading more on the issues raised in this Unit can find useful material in the following books: Fink, A. (1998) Conducting Research Literature Reviews: From Paper to the Internet, Thousand Oaks, CA: Sage Publications. Hart, C. (1998) Doing a Literature Review: Releasing the Social Science Research Imagination, Thousand Oaks, CA: Sage Publications.
3-15
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
3.12 Study Questions You should now write approximately 300 words in answer to each of the questions below. We believe that this is an important exercise that will assist your comprehension of the material and aid your progress on the course. Your answers are intended to form part of your own course notes and should not be forwarded to the University. 1. When carrying out a literature review, is it necessary to go to the lengths of the CDATE researchers in seeking out every publication? 2. Sometimes it may prove extremely difficult to locate accounts of directly relevant research for a literature review. What strategies would you recommend a researcher adopts in such circumstances? 3. This Unit proposed that literature reviews are used for four reasons. Do you think these reasons are all equally important or are some more significant than the others?
MSC IN EMERGENCY PLANNING MANAGEMENT
3.13 Bibliography Blackburn, R. (1993) The Psychology of Criminal Conduct: Theory, Research and Practice, Chichester: Wiley. Cleland, C. M., Pearson, F. S., Lipton, D. S. and Yee, D. (1997, November) ‘Does age make a difference? A meta-analytic approach to reductions in criminal offending for juveniles and adults’, paper presented at the annual meeting of the American Society of Criminology, San Diego, CA. Fink, A. (1998) Conducting Research Literature Reviews: From Paper to the Internet, Thousand Oaks, CA: Sage Publications. Gendreau, P. and Andrews, D. A. (1990) ‘Tertiary Prevention: What the Meta-analyses of the Offender Treatment Literature Tells us about “What Works”’, Canadian Journal of Criminology, 32: 173–84. Hart, C. (1998) Doing a Literature Review: Releasing the Social Science Research Imagination, Thousand Oaks, CA: Sage Publications. Hollin, C. R. (1994) ‘Designing Effective Rehabilitation Programmes for Young Offenders’, Psychology, Crime, & Law, 1: 193–9. Izzo, R. L. and Ross, R. R. (1990) ‘Meta‑analysis of Rehabilitation Programs for Juvenile Delinquents: A Brief Report’, Criminal Justice and Behavior, 17: 134–42. Kapardis, A. (1997) Psychology and Law: A Critical Introduction, Cambridge: Cambridge University Press.
3-16
disaster
riskcrisis MANAGE RISK
Lipsey, M. W. (1992) ‘Juvenile Delinquency Treatment: A Meta‑analytic Inquiry into the Variability of Effects’, In T. D. Cook, H. Cooper, D. S. Cordray, H. Hartmann, L. V. Hedges, R. J. Light, T. A. Louis and F. Mosteller (eds), Meta‑analysis for Explanation: A Casebook (pp. 83–127), New York: Russell Sage Foundation. Lipsey, M. W. and Wilson, D. B. (1998) ‘Effective Intervention for Serious Juvenile Offenders’, in R. Loeber and D. Farrington (eds) Serious and Violent Juvenile Offenders: Risk Factors and Successful Interventions (pp. 313–45), Thousand Oaks, CA: Sage.
PLANN NG
DISASTER
EMERGENCY
Lösel, F. (1995) ‘Increasing Consensus in the Evaluation of Offender Rehabilitation? Lessons from Recent Research Syntheses’, Psychology, Crime & Law, 2: 19–39. McGuire, J. (ed.) (1995) What Works: Reducing Reoffending, Chichester: Wiley. Martinson, R. (1974) ‘What Works? Questions and Answers about Prison Reform’, The Public Interest, 35: 22–54.
Pearsall, J. and Trumble, B. (1995). The Oxford English Reference Dictionary, Oxford: Oxford University Press. Pearson, F. S. Lipton, D. S. and Cleland, C. M. (1997, November) ‘Rehabilitative Programs in Adult Corrections: CDATE Meta-analyses’, paper presented at the annual meeting of The American Society of Criminology, San Diego, CA. Sharpe, D. (1997) ‘Of Apples and Oranges, File Drawers and Garbage: Why Validity Issues in Metaanalysis Will Not Go Away’, Clinical Psychology Review, 17: 881–901. Thornton, D. M. (1987) ‘Treatment Effects on Recidivism: A Reappraisal of the “Nothing Works” Doctrine’, in B.J. McGurk, D. M. Thornton and M. Williams (eds), Applying Psychology to Imprisonment: Theory & Practice (pp. 181–9), London: HMSO.
MSC IN EMERGENCY PLANNING MANAGEMENT
Martinson, R. (1979) ‘New Findings, New Views: A Note of Caution Regarding Sentencing Reform’, Hofstra Law Review, 7: 243–58.
3-17
crisis
PLANN NG
MSC IN EMERGENCY PLANNING MANAGEMENT
3-18
crisisdisaster RISK MANAGE
risk
EMERGENCY
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 4 The Social Context of Research
disaster
riskcrisis MANAGE RISK
4 Unit Four: The Social Context of Research 4.1 Aims and Objectives of this Unit It is the aim of this Unit to understand how the research setting and the researchers themselves have an effect on the process of conducting research and the research findings. Specifically, the objectives are for students to: • develop an understanding of how external factors (i.e. the social environment) and internal factors (i.e. the individual characteristics of the researcher) influence the research process and research findings; • have an awareness of the safety issues that may arise while conducting research; • understand how and why ‘traditional’ research methods have been criticised; • be able to describe and evaluate the ways in which these criticisms have been addressed;
• use research methods appropriately in order to reflect social diversity and maintain participant well-being; • develop an awareness and knowledge of the political issues arising from social research. On completion of this Unit students should be able to design and conduct research that considers and respects social diversity.
4.2 The Research Setting It has traditionally been assumed that research is carried out within a social void; however, this is far from the case. Rather than research demonstrating an objective measure of social reality, it is now recognised by many that research actually shows a subjective construction of social reality which is influenced, or even determined by, the research setting. Figure 4.1 shows some of the factors surrounding the research setting that must be taken into consideration when designing research. Figure 4.1: Factors Influencing the Research Process
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• understand the need to consider and respect social diversity while planning research;
PLANN NG
DISASTER
4-3
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
4.2.1 The Relevance of the Social Setting The relevance of the social setting of research has been increasingly highlighted alongside a general decline in positivism. Under the ‘rules’ of positivism the social setting was not deemed important, with the belief that as long as objective research was conducted the truth would be uncovered. The ‘rules’ of positivism are summarised by Kolakowski (1993) as being: • the belief that reality is what is available to the senses; • the idea that science deals with facts, not values; • the proposal that the natural and social sciences share the same common logic and methodology. Researchers who adopt a positivistic framework therefore assume that research produces knowledge that is objective because it is based on observable social ‘facts’ and ‘laws’. Hence, social science is viewed as being a ‘science’ alongside the natural sciences.
MSC IN EMERGENCY PLANNING MANAGEMENT
Replicability, for example, is a traditional positivistic measurement of how ‘good’ a piece of research is. A ‘good’ piece of research should be able to be carried out by a different researcher with similar participants, in a similar research setting and yield the same findings. Over the last few decades, however, it has been argued that the social and historical setting combined with the individual characteristics of the researcher are central to the research and that the replicability of research may be difficult, if indeed at all possible. Warren and Hackney (2000: ix), for example, highlight that the misguided idea that the researcher is a: ... person without gender, personality, or historical location, who could objectively ... produce the same findings as any other person has been dispelled. Similarly, Guba and Lincoln argue: ... it is virtually impossible to imagine any human behaviour that is not heavily mediated by the context in which it occurs. (1981: 62) Arguments such as these have led researchers to accept that replicability and generalisablity are at least more limited than once thought (e.g. Schofield, 1989), indeed if possible at all (e.g. Guba and Lincoln, 1981). Researchers who adopt a more interpretative position argue that research provides insights into the social world of the research participants. This position builds on the work of Weber with the notion of verstehen, or understanding, of the individual’s own meanings associated with their behaviour being of paramount importance in the understanding of the research process. Burgess, for example highlights that: ... the ultimate aim is to study situations from the participants’ point of view … it is this theoretical perspective that has been the major influence on field research. (1984: 3) It is not claimed, however, that this approach is scientific in the same way that positivists argue. The methods used, for example participant observation, are very different from the methods
4-4
disaster
riskcrisis MANAGE RISK
used in the classic scientific model, with research placing less emphasis on control and more on interaction. Closely associated with interpretism is the concept of ‘grounded theory’. This term, coined by Glaser and Strauss (1967), refers to theory developed inductively from the situation being researched, with its emphasis on the interpretations of those being studied. Instead of a researcher starting from a theory based on their suppositions and then going out to test it, the main features of the theory are dictated by the definitions and descriptions of the people being studied. This approach therefore takes into account the social setting of research; however, the researcher still has to enter the research setting with some idea of the issue they wish to study. This therefore gives the research direction and focus and hence is still inextricably linked with the research setting; and indeed the researcher. Different researchers have therefore placed differing emphasis on the social setting of research, and this can be seen as a sliding scale as summarised in Figure 4.2 below: Figure 4.2: The Importance of the Social Setting
Grounded theory
Interpretism
Unimportant Positivism
The research setting is particularly important when conducting ethnographic or comparative research, whereby it is essential to consider the cultural context of the setting. Research concerned with ageing, for example, must take into account the way that ageing is portrayed in each setting. Similarly, there may be religious norms that must be respected and it must be remembered that the social position of women varies from culture to culture. The safety of the researcher also varies between research settings, especially when conducting research in an environment that is unfamiliar to the researcher.
4.2.2 Safety Within the Research Setting The issue of safety within the research setting is double sided, with the safety of both the participant and the researcher being of paramount importance. Some research may be potentially dangerous for both the participant and the researcher; for example, if women who are experiencing domestic violence are interviewed in their home this could be dangerous should the perpetrator return home unexpectedly. In other cases, however, it is primarily the safety of the researcher that is of importance. This is particularly pertinent when researchers enter settings where they do not know the area in depth. ‘Corridor tales’ are often told of researchers who have had, for example, their car tyres slashed or have experienced violence within the research setting.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Important
PLANN NG
DISASTER
Due to the increased concerns that researchers have voiced regarding their safety in the research setting, along with a more general increase in society regarding concerns over safety at work, following the Suzy Lampugh abduction, many organisations have now developed safety guidelines for researchers. The Social Research Association, for example, has produced a Code of Practice for the Safety of Social Researchers (available at: http://www.the-sra.org.uk/safe. htm). The code is aimed specifically at social researchers who are conducting field research on
4-5
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
their own, and they highlight a number of dimensions to the risk that researchers take when entering the research setting: • the risk of physical threat or abuse; • the risk of psychological trauma, as a result of actual or threatened violence or the nature of what is disclosed during the interaction; • the risk of being in a comprising situation, in which there might be accusations of improper behaviour; • the increased exposure to risks of everyday life and social interaction, such as road accidents and infectious illness; • the risk of causing psychological or physical harm to others. (Social Research Association)
MSC IN EMERGENCY PLANNING MANAGEMENT
They highlight that the safety of the researcher is not determined solely by the research setting, but also by the issues that are being researched: The topics for discussion in many social research interviews – for example, poverty, unemployment, relationship breakdown, social exclusion, bereavement and ill-health – may provoke strong feelings in respondents and prompt angry reactions. Some research may be concerned explicitly with phenomena where the threat of violence is likely – investigating criminal behaviour, working across sectarian divides or studying homophobic violence, for example. (Social Research Association) The safety issues relating to research should therefore be addressed before entering the research setting, and should preferably be considered at the design stage. Researchers who have not entered potentially dangerous research settings before should be given particular support and guidance and should be accompanied where possible. It must also be remembered that even research settings which are not generally considered to be ‘dangerous’ can change suddenly and unexpectedly. Planning for all eventualities is therefore important before entering any research setting. Always be aware of the fastest and safest way to exit the setting, and this could be due to anything from the threat of violence to a fire in the building.
4.2.3 The Physical Characteristics of the Research Setting The social implications of the physical characteristics of the research setting are important to researchers and can be very revealing. Although it is easy to focus directly on the participant, while ignoring the influence of the physical characteristics of the research setting, when conducting research it is important to make notes regarding observations made of the setting. Indeed, this can be seen to be as much a part of the research process as the actual method used. It is important to use all our senses when recording the research setting, and Bailey highlights that: ... watching is certainly an important part of collecting data in the field setting; however so are listening, smelling, touching and tasting. (1996: 65)
4-6
disaster
riskcrisis MANAGE RISK
If the research is taking place within an organisation, it is important to either record or gather existing data about the organisation so as to inform readers of your research about the organisational context of research. Relevant data might include size of the organisation, founding philosophy or mission statement, existing decision-making hierarchy, and any recent organisational changes. If the research setting is an individual’s home (for example conducting interviews) their personal surroundings can often help build up a more complete picture. For example, what type of books do they read? What social class are they likely to be? Do they have family photos placed around their room? These physical aspects are important to (discreetly) note, as they are often forgotten about after leaving the research setting. When the research setting is outside of the home, for example at work, there are also observations that can be made to help build a more complete picture. Does the participant have their own office? Is it neat and tidy or are there piles of papers scattered around? If they do not have their own office, where is the interview held? What does that tell you about the individual? Are they being constantly interrupted? Do other workers ignore them or smile and wave?
Another important aspect of non-verbal communication is clothing. The research setting should again be taken into consideration here; for example, workers in statutory agencies may have very specific ideas of what a researcher ‘looks like’, and these expectations may be very different depending on the research setting. It is not necessary to worry too much about clothing, but it is necessary to use common sense. While it is important that you feel comfortable in your clothing, it is important that the participant also feels comfortable. When interviewing in individuals’ homes it must be remembered that the participants’ neighbours may see you enter and leave the research setting. While this is not always an issue, if the participant does not wish to explain if questioned why you were visiting, it is often easier to explain the presence of a researcher if they are dressed casually (a friend?) than if an official presence is suggested (social services?).
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
It is also interesting to note where the participants place themselves in relation to the researcher. Do they sit close by or as far away as possible? Do they sit forward or lean back in their chair? Do they sit at the opposite side of a table or desk to you? Taking notes about non-verbal communication can often say as much as, if not more than, verbal communication and can also guide the researcher in their approach. For example, a table may be used as a barrier if the participants feel uncomfortable about revealing too much about themselves, while sitting forward in their chair may indicate nervousness. Participants (and researchers!) often become more relaxed as the interview proceeds, and this may be reflected in body language. It is also important to pay attention to your own non-verbal communication, for example many researchers use a clipboard to hold their interview schedule, which may give an overly official, business-like stance which researchers may wish to avoid.
PLANN NG
DISASTER
4.2.4 Gaining Access to the Research Setting One of the first stages of the research process is gaining access to the research setting. While this is sometimes relatively easy, some populations are more difficult to access. In some situations a researcher may require a ‘gatekeeper’ to assist them to gain access to the research participants or indeed the research setting itself. One form of gatekeeper is someone who is in a powerful position and can authorise or prevent researchers from entering the research setting. This will be
4-7
EMERGENCY
crisisdisaster RISK
MANAGE
discussed in more depth in Section 4.4, Political Issues. A further type of gatekeeper is an ‘insider’, someone who is already accepted in the research setting and whom the potential participants trust. For example, to carry out research in a community not familiar to the researcher being introduced by a local resident may afford more access than the researcher could obtain alone. This, however, can also be problematic in that only those who ‘trust’ or know the gatekeeper will be available to the researcher and possibly create a situation where only one aspect of the issue being researched is revealed. A gatekeeper can therefore provide an ‘in’ into an area that might otherwise not be accessible to the research, and once established as an acceptable researcher by the community other opportunities to gain access to further areas may be established.
MSC IN EMERGENCY PLANNING MANAGEMENT
Other factors, such as the individual characteristics of the researcher, may affect access to the research setting. Some research, for example, may be largely gender specific, whereby access may be denied or made difficult depending on the gender of the researcher. This is particularly pertinent when the research method used is participant observation; especially if the observation is covert. For example, a female researcher would find it almost impossible to conduct observational research in a Working Men’s Club, while a male researcher would be unlikely to gain access to a Women’s Refuge or a Rape Crisis Centre. Burgess highlights that other overt researcher characteristics, such as social class, age, race and ethnicity may also influence access. He points out that: Such characteristics create an immediate impression of the interviewer and will, in part, place limits on the roles that an interviewer may adopt. (1984: 105) With any research therefore it is important to consider how access will be gained, the effect the contact within the research setting will have on the findings and what if any personal characteristics may affect the research process. Once access to a chosen population has been gained the research process itself may further be complicated by the differences in norms and values of the researcher and the participants. This can affect the research process at any stage with the research process itself not only affected by the norms, values and beliefs of society but also those of the researcher and participants which may cause conflict and misinterpretation of data. It is therefore necessary to consider and respect human diversity when conducting social research.
4.3 Human Diversity and Social Research The research process is intersected with the norms, values and beliefs of society, and May (1997: 46) argues there are five stages where social values enter the research process: • interests leading to research • aims, objectives and design of research process • data collection process • interpretation of the data • the use made of the research findings.
4-8
crisis
PLANN NG
risk
disaster
riskcrisis MANAGE RISK
Similarly, Silverman highlights that social researchers often have a pre-defined agenda and from the beginning have set ideas regarding the research findings. He argues that it is a common tendency to ‘make claims to know how things really are, while all too often ignoring what people are actually saying and doing’ (1985: 188). It is therefore necessary to consider how prejudice and discrimination have affected social research, not only in terms of how research is conducted, but also which issues are addressed through research and what is ignored.
4.3.1 Sexism and Social Research The gender of the researcher is a factor that plays a large part throughout the research process, while also interacting with other factors such as age, class, ethnicity and sexuality. Social research has been criticised for being situated within a male reality, and this has always been the case. In 1983, for example, Jayaratne pointed out that:
(1983: 146) She highlighted that it is only since the mid-1970s that some journals have started to include research regarding women’s issues, and also that some journals have been introduced that are concerned exclusively with research with women (e.g. Psychology of Women Quarterly; Sex Roles). Since Jayaratne’s article the number of papers written by women regarding women’s issues has increased, as have the number of journals specifically aimed at women (e.g. Women’s Studies International Forum; Feminist Review; Violence Against Women). Despite these increases, most feminist researchers would still agree that the majority of social research is still sexist. Indeed, Jayaratne’s comment that ‘the great majority of research addresses issues of importance to white male academics’ (146) is still true nearly thirty years on. Jayarantne (1983) suggested that some of the reasons for the reduction of sexism within research were related to the increasing numbers of women in graduate study, the increased media coverage of women’s issues and the growing number of women entering the paid workforce. Nearly thirty years on, we cannot deny the influence of these factors; however, it is also important to recognise the powerful influence of second wave feminists. Feminist academics such as Renate Duelli-Klien, Maria Mies, Liz Stanley, Sue Wise, Shulamit Reinharz, Ann Oakley and Helen Roberts have played a groundbreaking role in the development of the critique of existing methods while developing ideas of feminist methodology.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
... anyone who glances through the indices of social science journals for the past thirty or forty years cannot deny that the great majority of research addresses issues of importance to white male academics.
PLANN NG
DISASTER
Research that does not take women into consideration is often referred to as being ‘malestream’, whereby the male perspective is seen as normal and universal. This is therefore also reflective of society as a whole where male reality is seen as the reality and is generally associated with the positivist paradigm. Sexism within research has been studied by Eichler (1991), who has identified ‘The Seven Sexist Problems’ of social research: • Androcentricity – which she defines as ‘a view from a male perspective’ (5). Research that is androcentric is based in a male reality, whereby the male perspective is seen as the ‘normal’, most important, or even the only view.
4-9
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
• Overgeneralisation/ Overspecificity – whereby ‘overgeneralization occurs when a study deals with only one sex but presents itself as if it were applicable to both sexes. Its flip side is overspecificity, which occurs when a study is reported in such a manner that it is impossible to determine whether or not it applies to one or both sexes’ (6). Overspecificity is often caused due to the use of sexist language. • Gender Insensitivity – this occurs when researchers assume that gender is not a relevant factor in their research and hence ignore every aspect of gender within their analysis. • Double Standards – research that uses double standards ‘involves evaluating, treating, or measuring identical behaviours, traits, or situations by different means’ (7). In this respect, one research instrument may be used with male participants while another is used for female participants.
MSC IN EMERGENCY PLANNING MANAGEMENT
• Sex Appropriateness – this occurs when a phenomenon is assumed to affect one gender, while in reality it can affect both females and males. She notes that while it is not an issue when attributes are truly sex-specific, in practice this is rarely the case. Other feminists, however, disagree with Eichler on this issue and argue that gender specific research is very necessary, if only to redress the balance caused by many years of androcentric research. • Familism – which she describes as a specific form of gender insensitivity, whereby the family is treated as being ‘the smallest unit of analysis in instances in which it is, in fact, individuals within families (or households) who engage in certain actions, have certain experiences, and so on’ (8). In this case research may show, for example, that families with pre-school children have difficulties accessing adequate child-care, but further analysis may demonstrate that it is generally, but not always, women who are most concerned and affected by this. • Sexual Dichotomism – which she describes as a form of double standards, which ‘involves the treatment of the sexes as two entirely discrete social, as well as biological, groups, rather than as two groups with overlapping characteristics’ (p. 9). Again, other feminist researchers may disagree with Eichler, claiming there is a convincing argument to be made for thinking about men and women as having very different experiences, interests and indeed lifestyles. There is not, therefore, a uniform ‘feminist response’ to the issue of sexism within social research, and this is linked to the fact that there is no one feminism. Hence, the plural term ‘feminisms’ is now often used in place of the singular ‘feminism’ (see Kemp and Squires, 1997 for further discussion). It is equally important to remember that feminists have not simply worked towards the reduction and elimination of the sexist bias within research; indeed this can be seen as purely the tip of the iceberg. The link between feminists and the wider Women’s Liberation Movement plays a key part in the conducting of feminist research, and while in some cases this link is distant (e.g. postmodern feminism), for others it is central (e.g. radical feminism). It is also essential to demystify what is meant by feminist research methods. Students (and indeed some established academics!) often become confused over the term ‘feminist research methods’, searching for methods that are specifically ‘feminist’. However, the term refers not to any specific
4-10
disaster
riskcrisis MANAGE RISK
methods per se, but rather to a way of using research methods. Therefore it is the overall approach that is feminist, as Kelly et al., highlight: ... what makes feminist research feminist is less the method used, and more how it is used and what it is used for. (1992: 150) Similarly, Ramazanoglu (1992) highlights the diversity within feminism and points out that: What one means by feminist methodology depends in part on which authors one takes as examples. (1992: 208) It is therefore important to recognise the divisions within and between feminisms, as well as the divisions between feminist and non-feminist researchers (Westmarland, 2001). Many attempts have been made to define what research can be classed as ‘feminist’ and although there is no solid agreement, the following principles are generally found in feminist research:
• The rejection of total objectivity, with the acknowledgement that all knowledge is partial and situated. DuBois (1983) was one of the first to develop a feminist critique of objectivity, and described it as ‘an excuse for a power relationship’ (p. 108). • The valorisation of qualitative methods. Although the 1980s saw a rejection of quantitative research methods (for example surveys) in favour of qualitative methods (for example semi-structured interviews), it is now generally acknowledged that both are important in feminist research. In contrast to other research traditions, however, most feminist researchers still see qualitative methods as equally, if not more, important as quantitative methods. • The suggestion made by feminist standpoint feminists that women, due to their sex-class position, can produce more valid knowledge through research than men who have only a limited view of the world (see for example Hartstock, 1983). • The acknowledgement that research is a political activity, and that feminist research should have the emancipation and empowerment of women as its primary aim. Jagger and Struhl (1978), for example, argue that research should be judged by the effect it has on improving women’s lives. • The elimination of sexist bias in research.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• The elimination of hierarchies within the research process. A more equal relationship between the researcher and the researched is aimed for (e.g. Stanley and Wise, 1990), and terms such as ‘research subject’ are rejected.
PLANN NG
DISASTER
These are just some of the elements that are likely to be present within feminist research, and different researchers have argued for differing definitions of feminist research. Reinharz (1992), for example, argues for self-definition, whereby feminist research should be defined as such if the researcher identifies herslf as being a feminist carrying out research using feminist methods. A further issue addressed by feminists is the use of language. Sexist language has been used, and often continues to be used, within social research and is defined by Eichler (1991) as ‘language that
4-11
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 4-12
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
uses male terms for generic purposes’ (p. 49). Many associations now have anti-sexist language codes which are designed to be used both in the writing of research and also within the academic setting as a whole. The British Sociological Association, for example, has produced an Anti-Sexist Language Code which aims to assist its members in the understanding of what is meant by sexist language, while also suggesting alternatives such as: Sexist
Non-Sexist
founders
founding fathers
the rights of man
people’s/citizens’ rights; the rights of the individual
work-hours
man-hours
chairman
chair
disseminate
broadcast, inform, publicise
master copy
top copy/original
Feminist researchers have therefore challenged all aspects of the research process, and many of the ideas of feminist researchers who were writing in the 1980s have now been accepted as the way that research in general should be conducted, and are no longer perceived as being radical or revolutionary. Just as there is no one feminism there is no one feminist method, and it may be useful to think of feminist methodology in the way described by Cealey-Harrison and HoodWilliams (1998) – more variety than Heinz!
4.3.2 Racism and Social Research Racism has been defined in different ways by different researchers. One recent definition is offered by Anthias (1999: para. 3.7), who suggests that: A racist practice, as well as being one that has explicit racist facets or is ethnocentric can be any practice that produces racist effects, and where ethnic markers correlate with differential treatment. Goldberg (1993) highlights that racism is a ‘fluid, transforming, historically specific concept parasitic on theoretic and social discourses for the meaning it assumes at any historical moment’ (p. 74). Many sociologists (e.g. Lawrence, 1982) have criticised the racist values associated with ‘white sociology’, arguing that research stereotypes black people, and has perpetuated, maintained, reflected and reproduced racism in Britain. Orbe (2000) points out that concerns have been raised in relation to the epistemological and ontological assumptions associated with the traditional social scientific approach to the study of race and ethnicity. Traditional methods in social research have been accused of ‘caricaturing’ certain racial and ethnic groups, and Orbe suggests that ‘at best, what has been achieved is a commuter’s view of racial/ethnic minority group cultures’ (2000: 604). While large-scale empirical research strives to make ‘objective’ generalisations, it is argued that the voices of minority groups are marginalised. Strine (1997), for example, criticises empirical research for categorising minority groups, and reducing their voices to ‘predetermined categories for analysis or behaviour variables for testing’ (1997: 604).
disaster
riskcrisis MANAGE RISK
Booth (1988) states that alongside concerns about the ‘race problem’ there has also been an increase in the collection of statistics that take ‘race’ or ethnicity into consideration. Although historically this research has contributed to racist attitudes and policies rather than being the basis for anti-discriminatory attitudes and policies, the 1980s saw the utilisation of statistics based on ethnicity to reveal racist discrimination. Ahmed and Sheldon, for example, show that: ... ‘ethnic data’ have thus become the major tool for gaining ‘race’ equality in the new formalised, bureaucratised form of ant-racism. (1991: 27)
• Some sources of central government funding are available for services for ethnic minorities and it is necessary for authorities to have statistics in order to apply for this funding. • Local authorities can better tailor their services to the needs of ethnic minorities if they have appropriate figures. • Services can be situated appropriately within local authorities with relevant information. • Evidence of organisational discrimination can be highlighted and tackled at the appropriate levels if the statistical data are available. • Statistics on ethnicity can provide baseline data for the formulation of policies. • Ethnic minorities themselves can use data to highlight their need for specific services and policies. The data can also be used to highlight inequality and discrimination. There are therefore a number of valid reasons for having a question on ethnicity within the Census, with various examples available to demonstrate how the Census has been used to highlight racial inequality. Cross (1980) points out that Census data have been used to reveal that ethnic minorities are housed in the most overcrowded and least socially desirable housing, have higher unemployment rates and, contrary to popular belief, are less likely to ask social services for assistance. The census data have therefore also been used to challenge racial stereotypes and myths about ethnic minorities.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Statistics regarding ethnicity have therefore been used to promote both racism and anti-racism, and the usefulness of the collection of ethnicity data has been debated for several decades. Ahmed and Sheldon (1991) highlight that the main concern is that the statistics will be used to play the ‘numbers game’ and, more specifically, that they can be used to develop racist police and immigration policies. The relevance of the ethnicity question in the Census has become the focus of much debate, with the main arguments put forward for the collection of statistics on ethnicity summarised by Ahmed and Sheldon (1991) as:
PLANN NG
DISASTER
Although strong arguments have been put forward for the usefulness of the collection of data regarding ethnicity, there are also valid arguments to be made against its collection. It has been argued that ethnicity data contribute to racism by maintaining the ‘us versus them’ dichotomy, while also distinguishing between different sorts of ‘them’, while ‘us’ remains a homogenous category (Sivanandan, 1991). Ahmad and Sheldon (1991) highlight the problematic nature of the terms ‘race’ and ‘ethnicity’, and argue that as ‘race’ has lost favour as a valid analytical category it has been substituted with ‘ethnicity’.
4-13
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 4-14
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
They point out that the problem with the way the Census records ethnicity is that it uses categories of race, but simply substitutes the word ‘race’ for ‘ethnicity’, and they argue: ... the ‘ethnic’ question in the Census is both rigid and externally imposed: it uses a culturalist, geographical and ‘nationalist’ notion of ‘race’ dressed up as ‘ethnicity’. (1991: 127) As with feminist critiques of androcentrism within the research process, there are therefore methodological implications related to the study of race and ethnicity. It is argued that existing methods used in social scientific research are neither adequate nor appropriate for research in this area, and the debate is not only centred on critiques of positivistic quantitative research, but has also led to the critique of qualitative research methods. DeAndrade (2000), for example, argues: There is a serious disjuncture between qualitiative research methodology and the dominant conceptual perspective of race and ethnicity. (2000: 269) She argues that rather than race and ethnicity being seen as dynamic, they are imposed onto the research process and assumed to be externally determined. Additionally, it has been argued that qualitative research methodology within racial and ethnic studies has remained static (Stanfield, 1993), despite race and ethnicity being a dynamic presence within research (DeVault, 1995). Researchers should therefore consider the suitability of research methods for the study of race and ethnicity, while recognising the changing relationship between race and ethnicity.
4.3.3 Disablism and Social Research Social research on disability has been ignored and silenced for many years, although the last decade has seen the introduction of specific journals (e.g. Disability and Society) while some universities have introduced modules and/or courses focused around disability studies. The methods used in social research have also been criticised, for example some methods may limit some disabled individuals from participating in research. Kennedy (1996) questions the assumptions that researchers make when designing the research process and points out that ‘most research methods presume the respondent to have no physical or sensory impairments or learning difficulties’ (1996: 121–2). She also highlights the importance of considering these issues at the design stage of the research, as most research projects do not budget for additional research assistance that may be required such as Sign Language Interpreters. The assumptions made by non-disabled researchers must also be questioned and considered at the design stage. The framing of questions has been highlighted in all areas of social research, whereby the way a question is asked will affect the way it is answered. Similarly, questions not asked cannot be answered and therefore silence certain aspects of the area being researched. The discourse of disability is often dominated by non-disabled researcher perspectives and a common perspective used to frame research is the idea of disabled individuals being victims, disadvantaged and dependent. In this respect, disability is defined as a ‘problem’ for the individual which is known as the ‘personal tragedy model’ (Oliver, 1992).
disaster
riskcrisis MANAGE RISK
In direct contrast to the ‘personal tragedy model’, the idea of a ‘social model of disability’ has been proposed to reflect more accurately and respectfully the experiences of disabled individuals. This model turns the situation around, whereby the ‘problems’ that disabled individuals face are deemed to be the problems of society, rather than the individual. For example, rather than taking the view that an individual’s disability makes it difficult to travel by public transport (personal tragedy model), it is the poorly designed public transport that causes the problems (social model). Dowling and Dolan (2001) provide an example of research using the social model of disability in their research with families when they conclude that: ... the lives of these families are often characterised by financial hardship, stress and anxiety as a result of social barriers, prejudices and poorly conceived service provision. (2001: 21) The overall importance of this shift in the way research with individuals with disabilities is conducted and analysed is summarised by Beckett and Wrighton (2000), who highlight that:
(2000: 991) Despite this paradigmatic shift, academics generally continue to conduct research based on the personal tragedy, or ‘medical’ model of disability, and Goodley and Moore (2000: 861) highlight that the relationship between academics and the disability movement remains problematic. They believe that ‘real efforts must be made to bridge these boundaries’ in order to make research more inclusive. Feminist research has been criticised for excluding disabled women, and Morris highlights that: ... the way some feminist sociologists have excluded disabled women’s subjective reality from their research has colluded with … prejudicial attitudes. (1996: 6) She points out that feminist research claims to be research about women, yet if disabled women are not considered then research cannot be said to reflect women’s experiences. Rather, it is able women that the research speaks of. As an example, she discusses how research on domestic violence has been on the feminist agenda for many decades, but disabled women’s voices have been absent despite domestic violence being as much, if not more, of an issue for women with disabilities. Indeed, she notes that it has been only recently that the question of access to women’s refuges has been considered and some refuges still do not have any facilities for disabled women.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Moving from a medical to a social model of individual disability is a political process of change with implications for understanding of and relationship to borders between individual, social life and political participation.
PLANN NG
DISASTER
Similarly, feminist researchers have studied child sexual abuse for several decades, and disabled children have only recently been considered. Kennedy, for example, points out that ‘this concern has been slow to come and has certainly lagged behind efforts to protect non-disabled children’ (1996: 116). In 1989 she conducted research with 156 deaf children and found that 83 percent of females had been sexually abused, and that in 45 percent of the cases the abuse occurred
4-15
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
before their eighteenth birthday. It must therefore be questioned why the experiences of disabled individuals are so often neglected and omitted from research. Children and young people’s voices are rarely heard within research on disability, with carers, parents or guardians generally being asked questions on their behalf. Returning to the example of child sexual abuse, this is particularly pertinent, as children are overwhelmingly more likely to be abused by someone close to them. It is therefore important to listen directly (i.e. research with children), as opposed to indirectly (i.e. research with carers/parents/guardians) to avoid at best misinterpretation, or at worst the silencing of children and young people’s experiences of disability. Researchers should therefore avoid using the personal tragedy or ‘medical’ model in favour of the social model of disability when designing, conducting and analysing their work. Pragmatic issues should be considered, while the experiences of young people and women must also be listened to through research.
4.3.4 Heterosexism and Social Research
MSC IN EMERGENCY PLANNING MANAGEMENT
Although early research characterised gay men, and to a lesser extent lesbian women, as being mentally ill or criminal, this is now rarely the case. There are some, however, who continue the search for an elusive (invented?) ‘gene’ for homosexuality. It must also be remembered that until relatively recently homosexuality was classified as a mental disorder; for example the American Psychiatric Association did not remove it from their classification until 1973. This view of homosexuality as a mental illness has been reflected in the way research has been conducted, as well as the issues which are addressed through research. As with the other issues of human diversity, research on sexuality has aided the rejection of some stereotypes associated with homosexuality. Empirical research has used statistics to demonstrate that although society remains mainly heterosexual, a significant number are gay, lesbian or bisexual. This highlights the inadequacy of heterosexist policies and assumptions, although there is still clearly a problem in practice due to the homophobic attitudes prevalent in our society. Additionally, lesbian and gay issues are still invisible in many academic journals, with a content analysis of psychology journals showing that less than 1 percent (0.65 percent) took gay and lesbian participants into consideration (Buhrke et al., 1992). Harry points out that early research on homosexuality was problematic for two primary reasons relating to sampling strategies. Firstly, the numbers of participants were small and secondly they were derived from particular settings. These two issues are generally interrelated; for example he highlights that: ... the practice of taking respondents from the prison and the psychiatric couch necessarily limited the number of persons available in a sampling frame. (1986: 22) The small number of participants in early studies led to the inability to perform elaborate statistical analysis, but Harry (1986) suggests that this problem has now been largely overcome. As with feminist research, concerns have been raised regarding the appropriateness and usefulness of certain social scientific methods. Kitzinger (1992), for example, argues that the uses of quantitative
4-16
disaster
riskcrisis MANAGE RISK
methods, such as large-scale questionnaires, are oppressive to gay men and lesbian women. She highlights concerns over the way quantitative research findings are used to inform policy, an issue also highlighted by Bonell (1999) who succinctly titles his paper ‘Gay Men: Drowning (and Swimming) by Numbers’. Recommendations are offered by Buhrke et al. (1992: 96) regarding future research with gay and lesbian participants, which include: • The diversity of samples used in research needs to be increased. Ethnic and other group differences or similarities should be noted, if not fully examined or analysed. • Issues relating to multiple oppression and the difficulties involved in balancing multiple identities (e.g. black, lesbian and female) are of particular relevance and concern. • In studies specific to lesbian or gay male populations, researchers should not assume similarity of these two groups or of their experiences.
• Researchers should consider and clearly specify limits of generalisability. It is important to remember that most studies of lesbian women and gay men are studies of those who have come out. The extent to which findings generalise to those who remain hidden remains unknown. • Researchers should not make assumptions as to the sexual orientation of sample participants…it is not sufficient to assume that patrons of a lesbian or gay bar who agree to fill out an anonymous questionnaire are lesbian or gay. Nor is it safe or empirically acceptable to assume that all members of a ‘general’ sample are heterosexual. • Researchers should reconsider the use of a heterosexual paradigm when conducting research on lesbian and gay related issues and populations. Therefore, researchers need to take into account not what is considered ‘normal’ to themselves, but what others would define as ‘normal’. Put simply, although a relationship between a man and a woman may be considered the ‘norm’ by some; for others it is same-sex relationships that are the ‘normal’ way of life. Additionally, it has been argued that the knowledge gained by researchers who have conducted lesbian or gay related studies should pass on their techniques for research design, sampling and data collection to others to increase the number of skilled researchers in these areas of study (Buhrke et al., 1992).
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• Gender and sex role issues are likely to be as salient in these groups as they are for heterosexual women and men.
PLANN NG
DISASTER
4.4 Political Issues in Social Research Political issues in social research are always present, whether visibly or as part of the hidden agenda as Punch (1986) highlights: ‘to a greater or lesser extent, “politics” suffuses all sociological research’. Political issues refer not only to the state itself, but also to the powers and policies of governmental research, university departments, the funding of research units and even the micropolitics of interpersonal relationships (Punch, 1986).
4-17
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 4-18
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Stavenhagan (1971) argues that knowledge gained though social research, as with all scientific knowledge, has increasingly become controlled by power, and May highlights that: ... knowledge is not simply a politically neutral product as would be maintained by positivism and empiricism. (1997: 54) In this respect, it is often no longer possible to simply conduct research in an area of interest; it must be deemed interesting and necessary by those in power such as editors and publishers and those who determine funding priorities and opportunities. Some research topics, by their very nature, are more highly politicised than others, and this is particularly pertinent when research is explicitly linked to a social movement, for example the Women’s Liberation Movement or the Anti-Racist Movement. Auerbach considers that research can be used as a tool to inform and change politics, and emphasises the need for research to be conducted within governmental organisations: In order for feminist social science research to influence the transformation of society toward more equitable gender arrangements, it must both be undertaken and be applied in the policy arena. (2000: 30) Similarly, Millen points out that ‘as feminists, we can view the entire research process as situated within politics, rather than set aside from politics’ (1997: para 1.1). Politics within research can therefore be viewed as a two-way interaction; with politics influencing research and also research informing politics. Where research is likely to be politically sensitive (for example, where it threatens to subvert the ideological status quo) or may illuminate aspects of local or central government policy, researchers are likely to be asked to sign the Official Secrets Act. In cases such as these, not only is the research process itself intersected with politics, but also the publication of the findings. Research carried out on behalf of the government often takes years before it is published, and in many cases the results are never published with the researcher banned from discussing the findings with anyone outside of the research in order to adhere to the Official Secrets Act. Although this may sound extreme it is more common than it sounds, and the power of the government in research should not be underestimated. Researchers should be conscious of the possibility that their work may be controlled or censored, and strive to resist compromising the quality of good scientific research.
4.4.1 The Research Agenda The emphasis placed on academic staff to gain funding to carry out research has increased significantly over the last decade, with the Research Assessment Exercise (RAE) score given to a university being increasingly important. It is important to highlight that a postgraduate thesis is likely to be the last opportunity for an academic to spend so much time carrying out research on exactly what they want to do, in the way they want to do it. After this the macro-politics of funding and also the micro-politics of university departments primarily determine the research agenda.
disaster
riskcrisis MANAGE RISK
Research can be influenced by the value judgements of funding agencies. Typical funding agencies range from direct state sponsorship (for example, a government department) to relatively autonomous government research councils (for example, the ESRC) through to local government support, private charitable organisations (for example, the Joseph Rowntree Foundation), university departments and employer organisations (Bulmer, 1982). In addition to determining what is studied, funding agencies can also determine how it is studied, and McCartney (1970) found that major funding agencies were more likely to fund research that used statistical, or ‘scientific’ research methods. Similarly, Useem (1976) concluded that research funded by the government was preferentially allocated to researchers proposing the use of quantitative methodologies. More recent research conducted by Platt produced similar findings in that quantitative research was more likely to be funded than research which was qualitative. However, she argues that this does not necessarily mean that there is a funding bias towards quantitative research. She argues:
(1996: para. 4.2) Research is therefore continually intersected with politics, and an awareness of this can help identify the issues that may arise within the research process. If writing a research proposal for a funding agency, always remember and take into consideration the particular political stance of the agency, along with their more individual value judgements.
4.5 Conclusion This Unit has demonstrated the effect that the social context has on research. In contrast to positivistic approaches to social science, more modern approaches highlight the necessity for contemporary research to take the research setting, human diversity and political issues into consideration. To pretend that these issues do not exist is dated, and is likely to result in research which is criticised for being at best flawed, and at worst prejudiced and discriminatory. It is also important to remember that although issues of human diversity have been discussed in isolation in this Unit for reasons of clarity, these issues often overlap and interact with each other. For example, it is possible that a participant may be female, black, disabled and lesbian. It is therefore necessary to consider the methodological contributions that researchers have made to all of these issues. Other issues, such as ageism and classism, may also intersect the research process and it is important to identify at the design stage which issues are likely to be relevant to your research. Also remember that most issues are of some relevance to every research design, even if they are not immediately highlighted.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The foundations did play a significant role in the funding of quantitative work, and of the development and diffusion of quantitative methods, but it does not follow that they were thereby showing a bias in that direction. Two points demonstrate that they were not. The first is that they quite often gave grants in such a form that they had no control over what they were used for; the second is that they also funded much qualitative, and indeed non-empirical, work.
PLANN NG
DISASTER
4-19
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
4.6 Main Points • It is important that the research setting be taken into consideration when planning, conducting and analysing research. • Safety issues should always be addressed before entering the research setting. • Think carefully about how you are going to gain access and be accepted in the research setting. • Pay attention to the physical characteristics of the research setting. • Make sure human diversity is reflected and respected within your research, even if it does not initially appear to be important. • Remember that political issues intersect the research process from the beginning to the end, even if they are not always overt.
MSC IN EMERGENCY PLANNING MANAGEMENT
4.7 Guide to Reading Students interested in reading more on the issues raised in this Unit can find useful material in the following books and/or articles: British Sociological Association (undated) Anti Sexist Language Code, available at www.britsoc.org.uk May, T. (1997) Social Research: Issues, Methods and Process, Buckingham: Open University Press. Oliver, M. (1992) ‘Changing the Social Relations of Research Production’, Disability, Handicap and Society, 7(2): 101–14. Reinharz, S. (1992) Feminist Methods in Social Research, Oxford: Oxford University Press Stanfield, J. H. and Dennis, R. (eds) (1993) Race and Ethnicity in Research Methods, London: Sage.
4.8 Study Questions You should now write approximately 300 words in answer to each of the questions below. We believe that this is an important exercise that will assist your comprehension of the material and aid your progress on the course. Your answers are intended to form part of your own course notes and should not be forwarded to the University. • What factors relating to the research setting need to be taken into consideration when designing research? • Why have qualitative research methods been argued to better reflect human diversity than quantitative methods? • How can political issues control or influence the research process?
4-20
disaster
riskcrisis MANAGE RISK
4.9 Bibliography Ahmed, W. I. U. and Sheldon, T. A. (1991) ‘Race and Statistics’, Radical Statistics, 48: 27–33. Anthias, F. (1999) ‘Institutional Racism, Power and Accountability’, Sociological Research Online, 4(1) available at www.socresonline.org.uk/socresonline/4/lawrence//anthias.html. Auerbach, J. D. (2000) ‘Feminism and Federally Funded Social Science: Notes from Inside’, Annals of the American Academy of Political and Social Science, 571: 30–41. Bailey, C. A. (1996) A Guide to Field Research, London: Pine Forge Press. Beckett, C. and Wrighton, E. (2000) ‘“What matters to me is not what you’re talking about” – Maintaining the Social Model of Disability in “Public and Private” Negotiations’, Disability and Society, 25(7): 991–9.
Booth (1988) ‘Identifying Ethnic Origin: The Past, Present and Future of Official Data Production’, in A. Bhat, R. Carr-Hill and S. Ohri (eds) Britain’s Black Population (2nd edn), Aldershot: Gower. British Sociological Association (undated) Anti Sexist Language Code, available at www.britsoc.org.uk Buhrke, R. A., Ben-Ezra, L. A., Hurley, M. E. and Ruprecht, L. J. (1992) ‘Content Analysis and Methodological Critique of Articles Concerning Lesbian and Gay Male Issues in Counseling Journals’, Journal of Counseling Psychology, 39(1): 91–9. Bulmer, M. (1982) The Uses of Social Research, London: Allen & Unwin. Burgess, R. (1984) In the Field: An Introduction to Field Research, London: Routledge. Campbell, R. and Wasco, S. M. (2000) ‘Feminist Approaches to Social Science: Epistemological and Methodological Tenets’, American Journal of Community Psychology, 26(6): 773–91. Cealey-Harrison, W. and Hood-Williams, J. (1998) ‘More Varieties than Heinz: Social Categories and Sociality in Humphries, Hammersley and Beyond’, Sociological Research Online, (3)1, available at http://www.socresonline.org.uk/socresonline/3/1/8.html>
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Bonell, C. (1999) ‘Gay Men: Drowning (and Swimming) by Numbers’, in S. Hood, B. Mayall and S. Olive (eds) Critical Issues in Social Research: Power and Prejudice, Buckingham: Open University Press.
PLANN NG
DISASTER
Cross (1980) cited in K. Leech (1989) A Question in Dispute: The Debate about ‘Ethnic’ Questions in the Census, London: Runnymead Trust. DeAndrade, L. L. (2000) ‘Negotiating from the Inside: Constructing Racial and Ethnic Identity in Qualitative Research’, Journal of Contemporary Ethnography, 29(3): 268–90. DeVault, M. (1995) ‘Ethnicity and Expertise: Racial-Ethnic Knowledge in Sociological Research’, Gender and Society, 9: 612–31.
4-21
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Dowling, M. and Dolan, L. (2001) ‘Families with Children with Disabilities – Inequalities and the Social Model’, Disability and Society, 16(1): 21–35. DuBois, B. (1983) ‘Passionate Scholarship: Notes on Values, Knowing and Method in Feminist Social Sciences’, in G. Bowles and R. Duelli Klein (eds), Theories of Women’s Studies, London: Routledge and Kegan Paul. Eichler, M. (1991) Nonsexist Research Methods – A Practical Guide, London: Routledge. Glaser, B. and Strauss, A.L. (1967) The Discovery of Grounded Theory, Chicago: Aldine. Goldberg, D. (1993) Racist Culture, Oxford: Blackwell. Goodley, D. and Moore, M. (2000) ‘Doing Disability Research: Activist Lives and the Academy’, Disability and Society, 15(6): 861–82.
MSC IN EMERGENCY PLANNING MANAGEMENT
Guba, E. G. and Lincoln, Y. S. (1981) Effective Evaluation: Improving the Usefulness of Evaluation Results through Responsive and Naturalistic Approaches, San Francisco: Jossey-Bass. Harry, J. (1986) ‘Sampling Gay Men’, The Journal of Sex Research, 22(1): 21–34. Hartstock, N. (1983) ‘The Feminist Standpoint: Developing the Ground for a Specifically Feminist Historical Materialism’, in S. Harding, and M. Hintikka (eds), Discovering Reality, Dordrecht, Holland: Reidel Publishing Company. Jagger, A. M. and Struhl, P. R. (1978) Feminist Frameworks: Alternative Theoretical Accounts of the Relations between Women and Men, New York: McGraw-Hill. Jayaratne, T. (1983) ‘The Value of Quantitative Methodology for Feminist Research’, in G. Bowles and R. Duelli Klein (eds), Theories of Women’s Studies, London: Routledge and Kegan Paul. Kelly, Regan and Burton (1992) ‘Quantitative Methods and Feminist Research’, in H. Hinds, A. Phoenix and J. Stacey (eds), Working Out: New Directions in Women’s Studies, Lewes: Falmer Press. Kemp, S. and Squires, J. (1997) Feminisms, Oxford: Oxford University Press. Kennedy, M. (1989) ‘The Abuse of Deaf Children’, Child Abuse Review, 3(1): 185–7. Kennedy, M. (1996) ‘Sexual Abuse and Disabled Children’, in J. Morris (ed.), Encounters with Strangers: Feminism and Disability, London: The Women’s Press. Kitzinger, J. (1992) ‘Sexual Violence and Compulsory Heterosexuality’, in S. Wilkinson and C. Kitzinger (eds), Heterosexuality, London: Sage. Kolakowski, L. (1993) ‘An Overall View of Positivism’, in M. Hammersley (ed.), Social Research: Philosophy, Politics and Practice, London: Sage.
4-22
disaster
riskcrisis MANAGE RISK
Lawrence, E. (1982) ‘“In the Abundance of Water, the Fool is Thirsty”: Sociology and Black “Pathology”’, in Centre for Contemporary Cultural Studies (eds), The Empire Strikes Back: Race and Racism in 70s Britain, London: Hutchinson. McCartney, J. L. (1970) ‘On Being Scientific: Changing Styles of Presentation of Sociological Research’, The American Sociologist, 5(1): 30–5. May, T. (1997) Social Research: Issues, Methods and Process (2nd edn), Buckingham: Open University Press. Millen, D. (1997) ‘Some Methodological and Epistemological Issues Raised by Doing Feminist Research on Non-Feminist Women’, Sociological Research Online, 2(3) available at http:// www.socresonline.org.uk/socresonline/2/3/3.html Morris, J. (1996) ‘Introduction’, in J. Morris (ed.), Encounters with Strangers: Feminism and Disability, London: The Women’s Press.
Oliver, M. (1992) ‘Changing the Social Relations of Research Production’, Disability, Handicap and Society, 7(2): 101–14. Orbe, M. P. (2000) ‘Centralizing Diverse Racial/Ethnic Voices in Scholarly Research: The Value of Phenomenological Inquiry’, International Journal of Intercultural Relations, 24: 603–21. Platt, J. (1996) ‘Has Funding Made a Difference to Research Methods?’, Sociological Research Online, 1(1) available at http://www.socresonline.org.uk/socresonline/1/1/5.html Punch, M. (1986) The Politics and Ethics of Fieldwork – Muddy Boots and Grubby Hands, London: Sage. Ramazanoglu, C. (1992) ‘On Feminist Methodology: Male Reason Versus Female Empowerment’, Sociology, 26(2): 207–12. Reinharz, S. (1992) Feminist Methods in Social Research, Oxford: Oxford University Press. Schofield, J. W. (1989) ‘Increasing the Generalisability of Qualitative Research’, in E. W. Eisner and A. Peshkin (eds), Qualitative Enquiry in Education: The Continuing Debate, New York: Teachers College Press.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Oakley, A. (1981) ‘Interviewing Women: A Contradiction in Terms’, in H. Roberts (ed.), Doing Feminist Research, London: Routledge & Kegan Paul.
PLANN NG
DISASTER
Silverman, D. (1985) Qualitative Methodology and Sociology, Aldershot: Gower. Sivanandan, A. (1991) ‘Black Struggles against Racism’, in Curriculum Development Project Steering Group (eds), Setting the Context, London: CCETSW.
4-23
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 4-24
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Social Research Association (undated) Code of Practice for the Safety of Social Researchers, available at http://www.the-sra.org.uk/safe.htm. Stanfield, J. H. (1993) ‘Methodological Reflections: An Introduction’, in J.H. Stanfield and R. Dennis (eds), Race and Ethnicity in Research Methods, London: Sage. Stanley, L. and Wise, S. (1990) ‘Method, Methodology and Epistemology in Feminist Research Processes’, in L. Stanley (ed.), Feminist Praxis, London: Routledge. Stavenhagan, R. (1971) ‘Decolonialising Applied Social Sciences’, Human Organisation, 30(4): 333–44. Strine, M. (1997) ‘Deconstructing Identity in/and Difference: Voices under Erasure’, Western Journal of Communication, 61: 448–59. Useem, M. (1976) ‘Government Influence on the Social Science Paradigm’, Sociological Quarterly, 17: 146–61. Warren and Hackney (2000) Gender Issues in Ethnography (2nd edn), California: Sage. Weber, M. (1984) The Methodology of the Social Sciences, 1904-1917, London: Sage. Westmarland, N. (2001) ‘The Quantitative/Qualitative Debate and Feminist Research: A Subjective View of Objectivity’ [28 paragraphs] Forum Qualitative Sozialforschung/ Forum: Qualitative Social Research [On-line Journal] 2(1). Available at: http://qualitative-research.net/fqs/fqs-eng.htm
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 5 Research Design
disaster
riskcrisis MANAGE RISK
5 Unit Five: Research Design 5.1 Aims and Objectives of this Unit It is the aim of this Unit to understand the processes involved in designing a piece of research. Specifically, the objectives are for students to: • understand and be able to use the key terms involved in designing research; • develop an understanding of how research is measured in terms of quality and be able to use these criteria when designing and critically evaluating research; • develop the ability to integrate theory and research; • understand how measurements are used in research; • be able to choose and use appropriate sampling procedures in your research.
5.2 What is Research Design? Research is carried out for many reasons, but generally falls into two forms: descriptive research (what is going on?) and explanatory research (why is it going on?). The form of research we are attempting to carry out often dictates the type of design we will go on to use. Research design therefore refers, in its most basic form, to the design of the research we want to carry out. It can be seen as one of the most important parts of the overall research process because, like a house, if the foundations are not solid it is likely that the rest will fall down sooner or later. Good preparation is essential in research, and de Vaus (2001) also uses a building analogy to describe the important decisions that need to be made at the design stage explaining: ... when constructing a building there is no point ordering materials or setting critical dates for completion of project dates until we know what kind of building is being constructed. The first decision is whether we need a high rise office building, a factory for manufacturing machinery, a school, a residential home or an apartment block. Until this is done we cannot sketch a plan, obtain permits, work out a work schedule or order materials. (2001: 8)
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
On completion of this Unit students should have an understanding of the different forms of research design and be able to choose an appropriate design for their research topic.
PLANN NG
DISASTER
It is commonly assumed that research design means the same thing as research methods, and de Vaus (2001) highlights that research methods textbooks often use these terms interchangeably, and hence incorrectly. He explains that: There is nothing intrinsic about any research design that requires a particular method of data collection … How the data are collected is irrelevant to the logic of the design. (2001: 9)
5-3
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
In designing your research, there are several areas that need to be taken into consideration. Although these will vary depending on what you are researching, de Vaus (2001) has identified six main elements of research design: 1. The number of groups in the design – Design will vary from those with only one group of participants to designs which wish to compare two or more groups. 2. The number of ‘pre-test’ measurement phases – Designs vary from those with no ‘pre-test’ to those with a series of ‘pre-tests’ which establish pre-existing trends before an event. The majority of ‘real world’ research projects will have no ‘pre-test’ because the focus of research is often when something has happened rather than controlling situations to measure the ‘before’ and ‘after’ effects. It may therefore be impossible to study the ‘before’ effect because the research will start after an event.
MSC IN EMERGENCY PLANNING MANAGEMENT
3. The number of ‘post-test’ measurement phases – All designs require at least one ‘posttest’ – the measurement of an outcome variable. In some designs there will be one ‘post-test’, while other designs can have many post-tests to help distinguish between short and long-term outcomes. For example, we may wish to do one ‘post-test’, or ‘after’ effect immediately after the event and then conduct a follow-up test in 12 months time to see if there are any changes. 4. The method of allocations of cases to groups – If you are splitting a sample into two or more groups you may want the two groups to consist of similar participants. Statistical sampling procedures must be used to allocate participants into groups if you want your research to have external validity. 5. The nature of the intervention – Studies that rely on existing variations have no interventions. Other designs rely on interventions between a pre-test and a post-test. These ‘interventions’ may be either active or natural. 6. The number of interventions – Designs with an intervention can have either a single intervention or multiple interventions. Multiple interventions can be used to identify the effect of cumulative ‘treatments’. (taken from de Vaus, 2001: 48) There are therefore many interrelated parts to research design and it is necessary to decide which parts are relevant to your own particular piece of research.
5.3 Variables Experimental research is primarily concerned with the strict controlling of variables. However, all research designs use variables to some extent and are concerned with the question ‘what exactly do we want to study’? Variable – A variable is, quite simply, anything that can be changed. In other words, something that is ‘able’ to be ‘varied’ is a variable. Variables can be things that are always changing and which the researcher has little control over, for example days of the week, or time of day. They can also be
5-4
disaster
riskcrisis MANAGE RISK
more individual, such as gender, age, ethnicity or class. Although some of these variables may be fixed within an individual they are still variables as they vary between individuals. Variables can also be things that are conceptual and hence more difficult to identify and measure, for example emotional states such as well-being, stress, happiness or sadness. They can also be things that are directly manipulated by the researcher, such as the amount of time a participant is given to carry out a particular task or the difficulty rating of a task.
choose the more complicated your design becomes. Any variables other than IVs which you think may have an effect on the DV(s) must be controlled for where possible. Dependent variables – Often summarised as DVs, these are the variables that are measured. In the above example the DV would be the typing speed. However, you may want to use more than one DV. A further DV that could be measured in the above example may be the number of typing errors. Confounding variables – Also known as intervening variables, these are anything other than the independent variable(s) that may have an effect on the dependent variable(s). This is related to the controlling of variables, and any variable which is not controlled could confound the results of your research. Manly (1992) explains: … [a] conclusion may be invalid because of the confounding effects of uncontrolled variables … there may be no way of knowing whether an effect observed in the data is due to a change in the variable of interest, or is instead due to changes that happen to also occur in other variables at the same time. (1992: 4) Coolican (1994) offers a definition whereby: Confounding occurs when a variable related to the independent variable obscures a real effect or produces the false impression that the independent variable is producing observed changes. (1994: 22)
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Independent variables – Often summarised as IVs, and also known as ‘interventions’ these refer to the variables that have been changed and which may have an effect on the dependent variable(s). An IV can have more than one level; for example if you want to test whether a person’s gender has any effect on typing speed your IV would be gender. This would have two levels: male and female. You may have IVs other than gender that you think may have an effect on typing speed, such as age. You have a choice of two options here: you can use age as a second IV, which may have several levels (e.g. 4 levels: 21 years and under, 22–40 years, 41–60 years and over 61 years). Alternatively, you may want to control this variable. This could be achieved by only testing the effect of gender on participants in one age group. You may have as many IVs as you want; however, the more you
PLANN NG
DISASTER
Because these can affect the results of your research they must be taken into consideration at the design stage. For example, A may cause B to change, but this may be affected by C. Here, C would be a confounding variable. The controlling of confounding variables has typically been portrayed as only important in experimental research; however, they must be taken into consideration in all research topics and designs. Extraneous variables – These may also affect your research findings and are related to external factors which may affect the DV. For example, A may be correlated with B, but C may also affect
5-5
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 5-6
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
B. In this example C is an extraneous variable. These are often referred to as irrelevant variables because usually they are just that – irrelevant, but they are still variables because they are still factors in your research that vary. They may include: the time of day that the research is carried out (are the participants tired?), the heat of the room (are participants uncomfortable?), the way the researcher is dressed (are the participants intimidated?), or background noise (are the participants distracted?). Usually these factors are irrelevant outside of experimental designs but it is worth at least attempting to keep variables as constant as possible within your given research project. The difference between confounding and extraneous variables is often confusing. As a general rule confounding variables are those that directly intervene within the research process and which affect the relationship between the IV and the DV. Extraneous variables, in contrast, are those which are factors external to the research, such as the time of day that the research is carried out or the temperature of the room. Extraneous variables may affect the DV, and may or may not affect the overall results of the research, while confounding variables will affect the results of the research.
5.4 Benchmarks of Quality The quality of research has traditionally been measured using three interrelated concepts that form the positivistic perspective to knowledge and research: Figure 5.1: Measuring the Quality of Research Objectivity
Benchmarks of Quality Validity
Reliability
The use of these concepts to measure the quality of research is based on a scientific logic model stating that a research question can be ‘truthfully’ answered if all the ‘rules’ are followed:
disaster
riskcrisis MANAGE RISK
Figure 5.2: The Traditional/Positivistic Process of Integrating Quality into Research
PLANN NG
DISASTER
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Critiques of positivism claim that this model is too scientific to be applied to social research and argue that different measures of knowledge should be used when carrying out ‘real life’ research with ‘real people’ (see 5.4.6, Alternative Measures of Quality).
5.4.1 Objectivity Objectivity is linked with the notion of social research as a form of scientific research and the positivistic tradition. It refers to the ability to make an unbiased, value-free assessment of a situation, behaviour, set of data or facts without allowing personal, or subjective, meanings to guide your assessment. The idea is that the facts are allowed to speak for themselves and that the researcher is simply another tool in the research process. This is valued within the positivistic
5-7
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 5-8
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
tradition, although objectivity has increasingly come in for severe criticism and has been rejected as a measure of quality from other disciplines. Westmarland (2001: para. 7), for example, rejects the view that social research can ever be purely objective. She explains that there are a number of subjective decisions that invariably play a part: Humans, be they female or male, are not computers, and are unable to process information without some degree of subjective interpretation. This starts with the first stage of research: identifying the topic to be studied invariably involves subjectivity. As the process continues this is highlighted further, indeed, the introduction, or literature review, at the beginning of a report is actually a review of the literature that the researcher has deemed to be relevant. The rejection of total objectivity has led to a debate regarding alternative measurements of knowledge (see 5.4.6).
5.4.2 Validity There are two forms of validity that should be taken into consideration when designing research: internal and external. 5.4.2.1 Internal Validity Internal validity refers to whether the conclusions drawn in respect of the specific population studied are valid. Is there a real difference between groups or are they due to a confounding factor? For example, if it rains heavily then puddles will contain more water; however, there may also have been water from another source adding to the puddle, someone may have been washing their car! This would be classed as a confounding variable making the research lack internal validity. Control over the research and knowledge of other variables that occur during the research data collection phase is therefore paramount to ensure internal validity. Furthermore, internal validity can be jeopardised in the analysis of the data. Threats to internal validity Internal validity can be threatened if an inappropriate statistical test for the type of data generated or a low power statistical test is used. It is essential to use the appropriate statistical test for the data generated. Statistical tests should only be used when the data meet the required assumptions. Furthermore, exploring the data set by using multiple tests may increase the chances of obtaining a desired result when in reality it is not present. Internal validity may also be threatened by sampling bias, for example an over-representation of a specific type of person within the population. In certain types of research, such as experimental research where a control group is required as a comparison to the experimental group, there may be problems with internal validity if they (the control group) become aware of what the experimental group is doing. This could lead to rivalry (with the control group trying to perform as well as the experimentation group) or demoralisation of the control group (because they are not receiving the same treatment as the control group) both of which may lead to problems of internal validity. Internal validity may therefore be affected by many different factors during the research process.
disaster
riskcrisis MANAGE RISK
5.4.2.2 External Validity External validity refers to the extent that research findings can be generalised. Put crudely, this measure of knowledge is asking ‘how useful are your findings?’ It is therefore important to decide at the design stage to whom, and to what degree, you want your findings to be generalised. Imagine, for example, that you want to study the effect of caffeine on typing speed and you ask all the students on your course to take part. This research is likely to be designed using a simple experiment, which will be discussed in more depth later in the Unit. However, your results may show that on average typing speed increases by 10 percent after a participant has drunk a cup of coffee. If the findings can be generalised to those whom you intend to be able to apply your findings then your research has external validity. However, overly ambitious generalisation will threaten your external validity. Threats to external validity
• on average the students on your course may have more computer experience than the general population; • on average the students on your course may be younger than the general population; • on average the students on your course may be from a higher social class than the general population. In this respect, you may be able to conclude that the typing speed of the students on your course increases after drinking caffeine, but to conclude that typing speed increases after drinking caffeine per se would demonstrate an error in external validity. External validity does not need to be a problem in research as long as it is tackled at the design stage. This is done through the use of sampling procedures, which will be discussed later in this Unit.
5.4.3 Reliability Reliability refers to how reliable your measure is; in other words, if you used your measure again would it give the same result? It is therefore concerned with how reliable your research instrument is at measuring what it is supposed to measure. This can range from simplistic problems such as the measurement of time. For example, if your watch tells a different time from the clock on the wall you may be initially concerned over which time is correct; which one is reliably telling the correct time? In order to decide which one is correct you may look for other measurements of time, in other words you may look for consistency in your measuring instrument. While this is relatively simple when your measuring instrument is time, other concepts, such as intelligence or happiness, are more difficult to measure. Through conceptualisation and operationalisation (discussed later in this Unit) you must ensure that you are in fact measuring what you think you are measuring. There are also issues to be taken into consideration when it is the researcher, or the rater, that is the measuring instrument.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
You may conclude from your findings that typing speed is increased after drinking coffee, but maybe there are several problems in the design of this research and these can threaten the external validity of your research. For example the following factors need to be taken into account:
PLANN NG
DISASTER
5-9
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
5.4.4 Intra-rater Reliability While with time it is relatively easy to obtain reliability, other measures are more difficult. For example, if you are using observational methods, it is the researcher who must have reliability, in other words if a researcher observed a type of behaviour more than once would they record it as the same? This is called intra-rater reliability because you are concerned with how reliable one researcher is at rating what they have observed.
5.4.5 Inter-rater Reliability If the research is on a larger scale, several researchers may be involved in the data collection stage. It is important to ensure that all the researchers are measuring the same thing in the same way, and this is known as inter-rater reliability. This generally means that researchers must be trained in the technique used in the research. To achieve this, observers would observe the same phenomena and then compare their findings, and once an adequate level of agreement had been achieved the researchers could gather data knowing that they all used the same rating techniques.
MSC IN EMERGENCY PLANNING MANAGEMENT
De Vaus (2001) highlights that although validity and reliability are linked, they are not the same and a piece of research can be reliable without being valid. He uses as an example the problems related to self-report studies on alcohol and explains that: ... a measure can be consistently wrong ‌ people consistently underestimate their level of alcohol consumption in questionnaire surveys. Alcohol consumption measures are reliable but do not accurately tell us the true level of alcohol consumed. (2001: 31) Therefore, to measure alcohol consumption the self-report questionnaire technique might be a reliable measure but is not the most valid method to gather such data. This shows that the design stage of research requires intensive planning to prevent problems with validity and reliability.
5.4.6 Alternative Measures of Quality Feminist researchers have argued that the traditional ways of measuring the quality of research (objectivity, validity and reliability) are overly scientific, based in a male reality and are inappropriate for research with women. They argue that the subjective nature of research should be highlighted due to the necessary role that subjectivity plays within the research process. Ramazanoglu (1992), for example, writes: ... it is more logical to accept our subjectivity, our emotions and our socially grounded positions than to assume some of us can rise above them. (1992: 211) Feminists have now broadly rejected the idea of objectivity being used to measure social knowledge, and this rejection of pure objectivity is not limited to feminist researchers. Many other social researchers have questioned and rejected the notion, preferring to make claims of inter-subjectivity whereby findings are corroborated by other research.
5-10
disaster
riskcrisis MANAGE RISK
5.5 Types of Research Design The type of research design you choose will depend on the nature of your research topic and the amount of the depth and/or breadth you want to use to study your topic. The chosen design will also depend on the amount of control you have over the variables involved in the research. Three of the most commonly used designs are now discussed in further detail.
5.5.1 Correlational Design
individual will possess. However, we could not say if the intensity of the red hair caused the freckles or the freckles caused the intensity of the hair colour, only that they were associated to each other. One of the most common errors made in correlational research is to say that because two variables are associated then one causes the other. It must be remembered that although correlations can predict values for one variable based other variables, we cannot infer causality from this. The ‘golden rule’ of correlational design that must be remembered at all times is therefore that correlations cannot imply causality. The reason why we cannot infer causality from correlational research is because we do not know which variable causes the change – just that they change together. Therefore we cannot conclude that ‘A’ causes ‘B’ because equally, ‘B’ may cause ‘A’. All we can conclude is that when ‘A’ increases or decreases so does ‘B’, and therefore they are associated or related to each other. Other variables that we have not measured (extraneous variables) may have affected our findings. For example when testing hair colour and freckles it would be essential to establish that the hair colour is natural – hair dye would become a confounding variable! Correlations are statistically measured using correlation coefficients, on a scale of +1 to –1, with a coefficient of +1 showing a strong positive correlation or –1 showing a strong negative correlation. Variables that are not associated in any way have a correlation coefficient of 0. Positive correlations If we return to our simple example we may predict, or hypothesise, there is a relationship between the intensity of the red hair and the number of freckles. We may go one step further and predict the direction in which our findings are likely to go. In this example we may predict that as hair red hair colour intensifies, so does the number of freckles. To test our prediction we must go out and collect the data by measuring the hair colour intensity and the amount of freckles. After collecting the data they can be entered into a scattergram:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Correlational designs look for associations, or relationships, between variables. A correlation can be defined as the extent to which pairs of related variables change together or how or to what degree they are associated with each other. Correlational design is useful when you want to investigate whether variables are associated with each other. For example, you may have noticed that people with red hair have freckles. You may wish to investigate if red hair is associated with freckle numbers, predicting that the more intense the redness of the hair the more freckles the
PLANN NG
DISASTER
5-11
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
Figure 5.3: A Positive Correlation
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
When we draw the line of best fit (if using a computer package this will be entered for you) we can see that as one variable increases so does the other. Therefore, as we predicted, as the redness intensifies so does the number of freckles. This is known as a positive correlation. No correlation No correlation would indicate no association between the intensity of redness in hair colour and freckle numbers; in other words the intensity of redness of hair is not associated with number of freckles. Figure 5.4: No Correlation
No line of best fit can be entered because there is no correlation and hence no line of best fit. Negative correlations A negative correlation would indicate that as redness of hair colour intensifies freckle numbers decrease. There is still an association in this case, but the association is negative.
5-12
disaster
riskcrisis MANAGE RISK Figure 5.5: A Negative Correlation
PLANN NG
DISASTER
EMERGENCY
In contrast, if your aim is to show what effect one variable has on another variable, rather than to show that they are associated, then you should choose an experimental design.
5.5.2 Experimental Design An experiment, in its simplest form, attempts to measure what effect one variable has on another variable. It is also important to have a control group in an experiment. A control group has the same pre- and post-tests as the experimental group and should ideally have the same characteristics as the experimental group but is not given the ‘intervention’, or IV. This allows comparison between the pre- and post-tests of both groups to establish if the ‘intervention’ has effected the outcome. The pre-test scores allow us to establish the participants’ baseline abilities, while the post-test scores allow us to assess how much effect the intervention (IV) had on the ability (DV). Figure 5.6: A Simple Experimental Design
MSC IN EMERGENCY PLANNING MANAGEMENT
Therefore, if the aim of your research is to demonstrate an association between two variables a correlational design should be chosen. You cannot show a causal effect; you cannot say that freckles cause red hair, as equally, red hair may cause freckles. There may even be a confounding variable that is causing the association between freckles and red hair. For example, the association may be due to a gene or a skin pigment. An extraneous variable may be the time of year that you count the number of freckles because the sun may cause an increase in the number of freckles. Extraneous variables must therefore be controlled.
5-13
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 5-14
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Figure 5.6 demonstrates, in a simplistic form, the experimental research process. The ideal format of experimental research is to control all relevant variables whilst altering only the independent variable. The best way of explaining this is to design a hypothetical example. If we wanted to assess the effect that caffeine has on typing speed we would design an experiment which would enable us to control all variables except the independent variable; therefore any change in typing speed could be attributed to the caffeine. The independent variable would be caffeine with two levels, caffeine and no caffeine. The dependent variable would be the speed of typing. The typing test would first have to be developed to ensure that we had an adequate test of ability. The sample population would need to be divided into two groups and this could be performed by random allocation or by matched sample methods. A matched sample involves placing participants into groups based on various factors; for this example it could be typing ability as by chance the experimental group could contain people who have better typing skills. You may also want to consider how much caffeine people usually drink as this may affect the response to the caffeine you give. Once you have established the two groups you would want to control the experimental setting so that both groups experience the same environmental situations (in terms of location, time of day, heat, etc.) as these factors may confound the results if not taken into account. The research diagram would now look like this in our example: Figure 5.7: A Working Example of a Simple Experiment
To further control the experimental situation, in the control group decaffeinated coffee could be given as a ‘placebo’. A placebo is a substitute that is given in place of the actual intervention that will not have an affect on the post-test. This would mean that the participants would not be aware of which group they belonged to which would increase internal validity. It would also mean that participants in this instance could take part in the experiment at the same time which would further reduce confounding variables as the experimental conditions would be the same for both groups (in terms of time of day, heat, duration, etc.). Once the pre-test, intervention and post-tests have been performed analysis of the results can be carried out. By comparing the scores of the two groups we can see if the independent variable has had any effect on the dependent variable.
disaster
riskcrisis MANAGE RISK
5.5.3 Case Studies A case study involves an in-depth complex analysis of one particular ‘case’ using as many different methods as are appropriate. A ‘case’ is often wrongly assumed to mean an individual; however, de Vaus (2001) highlights that although case studies can be conducted on individuals this design can be applied to other ‘cases’ such as: • places • organisations • events • decisions • time periods. Case studies differ from other forms of research design in that they do not look for differences between groups, but rather concentrate on the differences within a group, or a case (de Vaus,
As there are different types of ‘cases’ that can be used for a case study, similarly there are different types of case studies, which Stake (1994) suggests can be separated into three distinguishable groups: • The intrinsic case study – in which the study is conducted in order to give a deeper understanding into a particular case. This is therefore a single case study design and it is the case itself that is the primary topic of research. • The instrumental case study – in which a case is studied in order to give insight into a particular issue or theory. This is also a single case study, but it is the issue or theory that is the primary topic of research. • The collective case study – which is the same as the instrumental case study with the variation that more than one case is studied. This is therefore a multiple, or comparative, case study design. Case studies are often criticised for not being generalisable, and Punch (1998) argues that because of the frequency of this criticism it must be taken into consideration. He argues that the first point to be considered should be whether we actually want to be able to generalise from the case study. He argues that cases that are unique, unusual, atypical, important, interesting or misunderstood can be argued to warrant study in their own right, without generalisability being an issue. It must be remembered, as Denzin (1983) highlights, that generalisability is not the be-all and end-all of research, regardless of the chosen design. Furthermore, Punch (1998) points out that although the results of a case study may not be generalisable, the concepts or propositions that have been developed may be able to be generalised to a wider population.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
2001). Although multiple cases can be used for analysis these are in order for the results to be replicated or to look for comparisons between cases as opposed to looking for causal features (de Vaus, 2001). Case studies further differ from other designs in the amount of depth they enter into. They are able to delve much deeper into the subjective meanings of the individual(s) in the case and hence produce a more complete picture of the case in question.
PLANN NG
DISASTER
5-15
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
5.6 Comparative Research It is important to note that many research designs, for example experiments, include some element of comparison. Any design that uses a control group, for example, can be said to be comparative because it is comparing the results of an experimental group with a control group. Similarly, May highlights that ‘we all use the idea of comparison when making judgements in everyday life’ (1993: 154). Despite this, experiments are not generally referred to as being a form of comparative research. Comparative research, in effect, uses two or more groups to compare but does not necessarily have a control group or an experimental group.
MSC IN EMERGENCY PLANNING MANAGEMENT
The use of comparative research has grown in popularity over the last few decades and now increasingly attracts both those who fund and those who publish research in a way that it has not in the past (May, 1993). Although comparative research has grown in popularity in academia, this is not to imply that it is new in any way and it is important not to neglect the historical bases of comparative research. Deutsch (1987), for example, states that ‘states, kingdoms and principalities have been compared for approximately 2,500 years’ (1987: 5). May (1993) suggests that the increase in comparative research can be attributed to two key factors: • the development of nation states and increased globalisation; • increases in communication through technological advances. It is also related to the acknowledgement that we can learn from other countries, particularly those similar to our own. In this respect, comparative research differs from other research designs in that it does not generally aim to test a theory. Teune (1990) states that ‘the goal is lessons rather than creating or testing theory. Countries that are similar are more likely to borrow from one another’ (1990: 58). Comparative research, although often consisting of cross-national research, can also be carried out within a nation. May (1993) offers a useful distinction, suggesting that comparative research can be classified as either an intra-societal comparison whereby comparisons are made within a society or an inter-societal comparison whereby comparisons are made between societies. Sensitivity to the cultural, ethnic, religious and/or historical contexts is imperative in comparative research, as these are likely to vary between the groups you are comparing. For example, the same question in different times and/or places may have very different meanings to the participants involved. A further form of comparative research is longitudinal research. This follows one group, or ‘cohort’, of participants and makes comparisons at different points in their lives, allowing the study of variables over time. One of the main advantages of longitudinal research is that there are no individual differences of participants that may confound the research because they are the same participants; simply at different stages in their lives. A problem often encountered with longitudinal research is ‘losing’ participants. They may, for example, move house, get married and/or change their name without informing the research team. Furthermore, participants may decide that they no longer wish to take part in the research and
5-16
disaster
riskcrisis MANAGE RISK
may opt out of the study. Therefore, the number of participants is likely to be drastically reduced throughout a longitudinal study. Longitudinal studies are, by their very nature, time consuming and hence are also an expensive form of carrying out research. A more time and cost efficient variation on longitudinal research is a cross-sectional study. In a cross-sectional study a chosen variable, for example attitude towards illegal drugs, is measured at the same moment in time with participants of different ages. Although this is a more manageable design, there may be problems with the use of non-equivalent groups. It does not give a developmental perspective in the same way as longitudinal research can; however, the likelihood of ‘losing’ participants is much reduced and results are obtained much faster. Additionally, a crosssectional study can compare groups based on variables other than age, for example social class, health status, ethnicity or occupational groups.
5.7 Historical Approaches
The historian … is typically engaged in tracing a chronological sequence of past events and with showing how certain events led on to others; time is thus a major dimension of his [sic] work. In contrast to this the [social scientist] is seen as being centrally concerned with the functional relationships which exist between the analytically separate elements in societies (or ‘social systems’) – time notwithstanding. The general propositions which he [sic] seeks are, when true, timeless and have no existential implications. (1977: 163) The final distinction identified by Goldthorpe (1977) is the relationship of each discipline to science. While many researchers working in the social sciences make claims to their work being scientific, historians make no such claims. Since Goldthorpe’s writing in the 1970s, the roles of historians and social scientists can be argued to have merged in many respects, with some academics now describing themselves as ‘social historians’ or ‘modern historians’. This is, in part, related to the widening and overlapping of disciplinary boundaries and the increasing emphasis on multi-disciplinary research and study. The vagueness of the boundaries of the two disciplines is highlighted if we imagine a piece of research that, for example, looks at housing quality since the 1970s. Is this historical research or social research? Clearly it is both and transgresses disciplinary boundaries. In another respect, we must bear in mind that nearly all research is historical, because even something that has happened a year ago, a week ago, or an hour ago is historic per se. It is therefore no longer useful to polarise historical and social research.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
While there are many similarities and overlaps between the work of historians and social scientists, it has also been argued that there is a sharp distinction between the two, which Goldthorpe (1977) argues to be due to three factors. He argues that the first distinction is related to the level of abstraction, whereby social scientists are interested primarily with theoretical analysis while historians are interested in finding out everything possible about a given subject. The second distinction made by Goldthorpe (1977) is that of time, and he argues that:
PLANN NG
DISASTER
Historical approaches generally involve secondary sources in the form of documents; however, oral histories have also been used since the early days of the Chicago School and have recently
5-17
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
gained in popularity. Alternatively, a historical approach to research may use both documentary sources and oral histories, as suggested by Plummer’s (1983) life history approach to research whereby all sources available are used to document an individual’s life, as far as possible in their own words. One of the major factors that needs to be taken into consideration when designing historical research is the availability of research materials. Some materials may have been destroyed, others may never have existed, while others may exist but may not be available to the public and/or researchers. Pragmatic considerations therefore play a key part when designing historical research.
5.8 Applied Research
MSC IN EMERGENCY PLANNING MANAGEMENT
Applied research, quite simply, relates to research which is not only applied to ‘real life’ issues but that is also of value to the ‘real world’. Academics are often criticised for living in ‘ivory towers’ and having little contact with the concerns of the general population; however, applied research aims to bridge this gap. Governmental departments or commercial organisations often commission applied research, and applied, or ‘real life’ research can take many forms. Robson (1993) explains the heterogeneity of situations that could be classed as ‘real life’ research: The ‘real life’ situation refers in part to the actual context where whatever we are interested in occurs, whether it be an office, school, hospital, home, street or football ground. (1993: 2) A large portion of applied research is therefore conducted in the workplace because, for many people, this is where they spend a substantial proportion of their life. However, in its broadest sense, applied research can be classified as any research that is conducted outside of a laboratory.
5.8.1 Evaluation Research Evaluation research aims to ‘evaluate the effectiveness of different types of actions in meeting needs or solving problems’ (Reinharz, 1992: 189). An action could be, although is not limited to, the following: • a policy or law • a set of guidelines • a management structure • a project • a programme • an agency • a group • an individual. Robson highlights that the simplest way to think of an evaluation is a ‘study which has a distinctive purpose’ (1993: 171) and explains that there are many different designs that an evaluation can take:
5-18
disaster
riskcrisis MANAGE RISK
[Evaluations] can be done using experimental, survey or case study research strategies – or some appropriate hybrid or combined strategy. (1993: 170) Therefore it may be necessary to ‘mix and match’ designs depending on the nature and the size of the evaluation. Different designs may be used to evaluate different viewpoints from different people. Evaluations are often commissioned in times of change; either when change is needed or to analyse the effectiveness of an existing action. A key term in an evaluation is effectiveness; however, what one person defines as effective may be different to another’s definition depending on their interests in the subject of the evaluation. Evaluations can range from relatively simple pieces of research to extremely large-scale complex studies. The complexity often arises in the controlling of variables outside of the action that is to be evaluated; in other words, to what extent is the ‘output(s)’, or effectiveness, attributable to the ‘input(s)’ and to what extent are the outputs attributable to other factors?
Additionally, there is the question of how effectiveness is defined and measured. Many evaluations are concerned with the cost-effectiveness of the intervention being evaluated, while this may need to be balanced with other definitions of effectiveness. Although Figure 5.8 shows the situation before the action being measured, it should be noted that this is an ideal situation. Unless the research is carefully planned in advance of the action being introduced this will not be possible. Indeed, if it is an evaluation of an existing action it will be impossible to travel back in time and collect data prior to the action.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Figure 5.8: A Simple Evaluation Process
PLANN NG
DISASTER
Robson (1993) highlights that the number of evaluations that are being commissioned is rising substantially, especially within public services. He argues that a major reason for this is the call for accountability, whereby public services are now having to be accountable not only for providing the service, but also to be cost-effective and efficient. This increase in the number of evaluations that are being commissioned has led to a variety of different forms of evaluations, and Patton (1981) has identified over 100 different types! The
5-19
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
purposes of evaluations also vary, and the Evaluation Research Society (1980: 3–4) has identified six broad reasons for evaluations to be conducted: 1. Front-end analysis (pre-installation, context, feasibility analysis) – takes place before the programme starts, to provide guidance in its planning and implementation. 2. Evaluability assessment – assesses feasibility of evaluation approaches and methods. 3. Formative evaluation (developmental, process) – provides information for programme improvement, modification and management. 4. Impact evaluation (summative, outcome, effectiveness) – determines the programme results and effectiveness, especially for deciding about programme continuation, expansion, reduction, funding. 5. Programme monitoring – checks for compliance with policy, tracking of services delivered, counting of clients.
MSC IN EMERGENCY PLANNING MANAGEMENT
6. Evaluation of evaluation (secondary evaluation, meta-evaluation, evaluation audit) – critiques of evaluation reports, re-analysis of data, external reviews of internal evaluations. Different writers identify different models or types of evaluation, and it is necessary to consider the purpose(s) of your evaluation in detail. Robson (1993) reminds us that: ... what is particularly important is the usefulness of the data for the purposes of the evaluation, and not the method by which it is obtained. (1993:186)
5.8.2 Policy Analysis Policy analysis involves the evaluation of existing and competing policies or programmes. This can take a historical perspective if comparing existing policies with those used in the past or may take a comparative approach if comparing with other existing policies which may draw on those from other nations. A specific design must be chosen, for example an experiment may look at participant well-being before and after a policy has been introduced, while a case study may concentrate on the effect a policy has on a particular person or group of people. According to Patton and Sawicki (1986) there are six steps that should be followed in order to conduct a successful policy analysis: Figure 5.9: The Process of Policy Analysis • Verify, define and detail the problem Make the objectives of the analysis clear – often the objective is to relieve or lessen some form of social, economic or physical problem. Sometimes the objectives are contradictory because of more than one interested party. For example, the policy analyst may need to take into account the objectives of, for example, a service provider and the service users. The views of the effectiveness of a particular policy may vary depending on the interests of different groups.
5-20
disaster
riskcrisis MANAGE RISK
• Establish evaluation criteria Criteria must be developed to enable the analysis and comparison of different competing policies. Factors such as cost-effectiveness, administration and political acceptance must be taken into consideration. Often the group or individual that has commissioned the policy analysis will make it clear what evaluation criteria are important for their needs. • Identify alternative policies The first two steps must be completed before attempting this stage. Alternative policies can be useful in order to present alternatives that have not previously been considered. Similarly, alternatives can be combined in order to develop the most useful and effective policy. Looking at past policies and/or policy analyses can help provide a more thorough analysis. • Evaluate alternative policies
• Display and distinguish among alternative policies The alternatives should now have been analysed in terms of the evaluation criteria and this will allow you to list the degree to which each of the competing policies meets the criteria. Comparisons can now be drawn between the policies. It may be necessary again at this stage to consider the different interests of the groups or individuals who have a vested interest in the policy. If a new policy is introduced this is likely to be as heavily influenced by politics as it is by the analysis. • Monitor the implemented policy Once the new policy is implemented it is necessary to measure whether the policy is having the impact on all of the interested groups or individuals as was intended. It may be necessary for the policy to be revised or even discontinued if it is not having the desired effect. The basic questions addressed in policy analyses are therefore (a) what do we want from a policy? (b) are there any better policies that we can incorporate? (c) does the policy make a lasting difference? and (d) is this difference what was intended? It is also important to contrast the actual effect of the policy with the intended effect of the policy. If they do not match then the policy may need to be continually monitored (to see if they will eventually match) or may need to be amended (if they are vastly different) or even discontinued.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
This is the next stage and incorporates the previous two stages and involves taking the evaluation criteria and applying them to the alternative policies. In some cases the initial stage of verifying, defining and detailing the problem may need to be amended slightly to take into account new information gained at this stage.
PLANN NG
DISASTER
5-21
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
5.8.3 Audits An audit is generally used to examine and assess how effectively a service or business is functioning and is usually thought of in terms of a cycle, whereby practice is assessed by independent observations and checked against agreed standards. Any deficiency is highlighted and brought to the attention of the appropriate managers so that appropriate changes can be agreed upon and implemented. Subsequently practice is again observed to complete the cycle (see Figure 5.10). Audits are therefore used to ensure that high standards of practice are both achieved and maintained and can therefore be seen as a system of quality assurance. Li Wan Po (1998) offers a useful model that demonstrates the ongoing process that an audit is likely to take: Figure 5.10: Suggested Audit Model (Taken from Li Wan Po, 1998: 7)
MSC IN EMERGENCY PLANNING MANAGEMENT An audit is therefore a continuous process of complex evaluations, where practice and standards are constantly changing in order to develop the best practice possible.
5-22
disaster
riskcrisis MANAGE RISK
Clarke (1998) argues that there has been an increase in the commissioning of audits within the public sphere, and that this increase is related to the decentralisation of control from the government and the breaking down of traditional top-down management systems. Since 1983 the Audit Commission has monitored local authorities and the NHS and aims to conduct audits which take into consideration the ‘three Es’: • economy • efficiency • effectiveness. Additionally, the ‘voices’ of the service users are being increasingly listened to and integrated into audit research (Clarke, 1998); however, this is a relatively recent advancement. Audits are therefore concerned with constant control, regulation and accountability.
5.8.4 Collaborative or Participatory Research
Can it be that a radically different approach is required in social science? Can the human beings who constitute the subject-matter of social science be regarded, not as objects for experimental manipulation, but as participants in what is being planned? If this can be so, it requires a transformed attitude towards social experiment. Traditionally, attention is concentrated on the precautions needed to objectify results, and this entails treating the participants as lay figures to be observed before and after subjection to a series of external stimuli. In contrast, the new approach entails the acceptance and encouragement of conscious co-operation by all concerned. There are then no longer an investigator and his [sic] passive subjects, but a number of human beings, one of whom is more experienced than the others and has somewhat more complex aims, but all of whom are knowingly collaborating in a research project. While participatory research has been the subject of much theorising since the 1950s, the basic premise remains the same. Dockery (2000) points out that there are political implications relating to the participatory research perspective:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Reinharz (1992) defines participatory or collaborative research as research where ‘the people studied make decisions about the study format and data analysis’ (1992: 181; emphasis in original). It involves the participants in the research being fully involved in the research process; from the design stage to the writing up and/or publishing of research. Although this form of research, as with action research, is assumed to be new this is not the case. The idea of participants playing a key role in research can be documented for several decades. It first appeared in the 1950s, when Madge (1953) theorised that this idea required a marked move away from the ideas of natural science and asked:
PLANN NG
DISASTER
Supporting or enabling participation in the strongest sense becomes a political act through establishing partnerships between the researcher and the researched, whereby ownership, empowerment, and responsibility for accountability are shared throughout the research process. Participatory research can play an important role in fostering or stimulating community activism at both the individual and collective levels. (2000: 95)
5-23
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 5-24
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Collaborative or participatory research therefore entails acknowledging that in many cases the real ‘experts’ in your chosen topic of research are the participants themselves. It is a perspective which is particularly useful when carrying out research on (with?) disempowered and/or marginalised groups. In 1946 Kurt Lewin introduced the concept of action research, whereby social research should, he argued, contribute in a more direct way to the needs of society. It is often used in circumstances where the interests of the researcher(s) meet and overlap with the interests of the participants and hence is often used when researching issues of a political nature. It takes a collaborative structure, and Rapoport (1970) argues that the aims of action research are: … to contribute not only to the practical concerns of people in an immediate problematic situation and to the goals of social science by joint collaboration within a mutually acceptable framework. (1970: 499) It was not until the 1970s that Lewin’s ideas around action research gained popularity. This was the result of a general decline in positivism, and feminist researchers adopted action research as a more appropriate, woman friendly way of conducting research (see for example Mies, 1983). Since then, action research has been employed to research issues such as race, ethnicity, sexuality and peace. Fullan (1991) offers a simplified model to demonstrate action research. Although in reality the procedure is more complex, this model is useful in demonstrating that action research is not a unilinear procedure. Rather, the two-way arrows (in Figure 5.11) highlight that while change is the ultimate aim of action research, change is a process rather than an event (Fullan, 1991). Figure 5.11 Fullan’s Simplified Model of Action Research
Although the idea of research being integrated with action has been criticised over the years, action research continues to be successfully used and often represents a more ethical way of conducting research than other research strategies.
5.9 The Integration of Theory Research which attempts to answer the ‘why is it going on’ question (explanatory research) in contrast to the ‘what is going on’ question (descriptive research) is concerned with theory, and de Vaus (2001) highlights that good descriptive research is likely to lead to explanatory research. Explanatory research generally falls into one of two forms in relation to theory; it can be used to either build a theory or to test a theory. de Vaus (2001) offers two models which effectively demonstrate how each process works:
disaster
riskcrisis MANAGE RISK
Figure 5.12: Theory Building Approach (taken from de Vaus, 2001: 6)
This model therefore uses inductive reasoning to develop and build a hypothesis, or theory. Observations are the starting point in the development of theory. Figure 5.13: Theory Testing Approach (taken from de Vaus, 2001: 6)
You will see that both models move between an empirical level (observations) and a conceptualabstract level (theory), with the starting point and direction changing depending on the approach. This move requires conceptualisation and operationalisation, and can be one of the most difficult parts of research design.
5.9.1 Conceptualisation Most pieces of social research use more abstract and complex concepts than the simplistic examples used in this Unit. While time, age and gender may be relatively easy to measure and record, de Vaus (2001) shows that concepts are less simply measured. He points out that: … concepts are, by their very nature, not directly observable. We cannot see social class, marital happiness, intelligence etc. To use concepts in research we need to translate concepts into something observable – something we can measure.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
This model is also known as the hypothetico-deductive model because it uses deductive reasoning to test a hypothesis, or theory. A theory is the starting point and observations, or data, are used to test the theory. This can also be used if you are testing an existing theory with new observations.
PLANN NG
DISASTER
(2001: 24) Conceptualisation therefore involves ‘descending the ladder of abstraction’ (de Vaus, 2001: 24). Put simply, conceptualisation refers to the identification, development and defining of concepts, while operationalisation refers to the actual observation and measurement of these concepts. The process of conceptualisation is often one of the most difficult parts of research design, and it will vary immensely depending on your individual research topic. De Vaus argues that this can be done by moving through a number of stages starting from a nominal definition which ‘specifies the meaning
5-25
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
of the concept but remains abstract’ (2001: 25), moving through to an operational definition. He suggests the stages involved in obtaining a nominal definition may be as follows: Figure 5.14: Developing a Conceptual Definition
EMERGENCY
When you delineate the dimensions of the concept you may find that the conceptions have subdivisions and this will assist you in formulating your operational definition. Indeed, until you begin to think in these terms about your concept you may not even realise that these sub-divisions exist.
5.9.2 Operationalisation
MSC IN EMERGENCY PLANNING MANAGEMENT
Operationalisation refers to the process where your conceptual definition becomes an operational definition and further becomes operationalised through the carrying out of empirical research. It is the process of linking your concept with your data collection method and it is important to return at this stage to your original research question and reconsider it in terms of your empirical indicators. If they do not appear to match then you must return to the beginning of your conceptualisation process and find a more appropriate operational definition and hence more appropriate empirical indicators. If you think the empirical indicators are appropriate for your concept and your original research question you are now ready to consider what, if anything, you will hypothesise.
5.9.3 Hypotheses When designing research you must decide on your research aim(s) or hypothesis (or hypotheses if you have more than one). Generally a piece of research will have either aims or hypotheses but will rarely have both. There are no particular advantages or disadvantages to using aims or hypotheses, although as a general rule aims are more commonly used in qualitative, non-positivistic research while hypotheses are generally used in quantitative, positivistic research. A hypothesis is basically a thesis, a theory, or a prediction about what you think your research will find. Hypotheses generally fall into two categories, although they may vary in how they are named. It does not matter which term you choose to use as long as you remain consistent throughout the research: 1.
Alternative/experimental hypotheses
This is what you think will happen. It is abbreviated to H1 if you have only one hypothesis, or increases numerically (i.e. H2, H3) if you have multiple hypotheses. It may be one tailed or two tailed. One-tailed/directional/uni-directional hypothesis This is when you predict the direction that your results will take. For example, in our rainfall example we could have a one-tailed hypothesis, stating that as rainfall increases puddle size will increase. We are therefore stating the direction of our theory.
5-26
disaster
riskcrisis MANAGE RISK
Two-tailed/non-directional/bi-directional hypothesis This is when you do not predict the direction that your results will take. For example, in our rainfall example we could have a two-tailed hypothesis, stating that as rainfall varies, so will puddle size. We are therefore saying that there will be an effect, but not stating which direction our results will take. There are no rules regarding the use of one- or two-tailed hypotheses, with the choice depending on how confident you are about the direction your results will take. In other words, if you are fairly confident about what your research will find you should use a one-tailed hypothesis, but if you are less certain then a two-tailed would be more appropriate. If you have a more complex design whereby you are making more than one prediction you should use more than one hypothesis. For example, if you theorise that when it is cold and rainfall increases that puddle size will increase, two separate hypotheses should be used, i.e. H1 – When it is cold puddle size will increase.
H2 – When rainfall increases puddle size will increase.
2.
Null/Non-experimental hypothesis
This is the opposite of an alternative/ experimental hypothesis and states what you predict will not happen. It is abbreviated to H0, and in our rainfall example would read:
H0 – When rainfall increases puddle size will not increase.
It is necessary to have a null hypothesis for every experimental, or alternative hypothesis. Therefore, if you have ten experimental hypotheses then you must have ten null hypotheses.
5.10 Measurement After the conceptualisation and operationalisation stages have been completed the next stage is to decide how you are going to measure your concept(s). The measurement stage is summarised by Punch as being ‘the process of using numbers to link concepts to indictors, where a continuum is involved’ (1998: 90). The measuring instrument you use will generally be guided by your research topic. For example, if you want to measure how long it takes a participant to carry out a task you may use time as your measure. In contrast, if you want to measure how the task makes them feel you may use a questionnaire designed to measure emotion. Often there will already be a measuring instrument available which you can use and/or amend for the purposes of your research; however, in original research you may have to design your own. Whether you use an existing measure or design your own, it is essential that it be both reliable and valid.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
PLANN NG
DISASTER
Scales Scales are often used to measure research, and may take several forms: • Nominal – this scale simply means counting. For example, how many participants answer ‘yes’ to a question, how many answer ‘no’ and how many answer ‘no opinion’.
5-27
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
• Ordinal – this scale literally means ordered; i.e. data that can be put in order. One of the most frequently used ordinal scales is the Likert scale. This is where participants have the choice of an ordered scale which can be continuous: Figure 5.15: Continuous Likert Scale Strongly Agree
Strongly Disagree
Where participants are required to put a dash through the scale, or they can be discrete where they are asked to choose between set categories: Figure 5.16: Discrete Likert Scale Strongly agree Agree No opinion Disagree Strongly disagree
MSC IN EMERGENCY PLANNING MANAGEMENT
• Interval – this scale can tell you the amount of difference between two scores, for example temperature. With ordinal data we could say that room A was hotter than room B, but with interval data we can say how much hotter, e.g. room A is 5 degrees Fahrenheit hotter than room B. • Ratio – this scale, as with the interval scale, can tell you the amount of difference between two scores. It varies in that it has a pure zero. For example, we cannot say that 80 degrees Fahrenheit is twice as hot as 40 degrees Fahrenheit because the Fahrenheit scale does not have a pure zero, i.e. it has a value. Time, or distance, on the other hand does have a pure zero; i.e. the zero has no value at all. Therefore, time can be measured on a ratio scale because we can say that 80 metres is twice as far as 40 metres. It is easy to become confused between interval and ratio scales, but most statistical tests treat interval and ratio data as being the same thing. Additionally, some measures can be measured on any scale. For example, time can be nominal, ordinal, interval or ratio. The scale chosen will depend on which statistical test is chosen and is related to the power of your analysis. This will be discussed in further detail in the statistics Unit.
5.11 Sampling Procedures This section is intended only as an introduction to the main principles of sampling. The statistical ideas upon which the concept is based are covered in more detail in Unit 8. Sampling is important if you want to be able to generalise your findings to a wider population. It is also therefore related to the validity of your research, as mentioned previously in this Unit. Put simply, sampling refers to the selection of a sample number of cases from a wider population so that the results of the sample can be inferred back to the wider population.
5-28
disaster
riskcrisis MANAGE RISK
Figure 5.17: The Relationship Between a Sample and a Population (from Punch, 1998: 106)
PLANN NG
DISASTER
EMERGENCY
• How many participants you intend to use in your research. – Take into consideration how long each participant will be needed; for example a questionnaire may take 10 minutes to complete while an in-depth interview may take an hour. – The prevalence of the phenomena you are investigating in the general population will affect the number of participants you require. If the phenomena you are investigating occurred frequently in the general population then you may require a large sample whereas if you are investigating a workplace or a specific group of people you may not require a large sample to make your findings generalisable to the specific group. – The financial costs must also be assessed; for example a questionnaire may be relatively cheap to produce, distribute and analyse in large numbers. However, with postal questionnaires the costs will significantly increase. Interviews are more labour intensive in terms of researcher’s time and the transcription time. • To what extent you want your results to be generalisable. – If you want your results to be generalisable you are likely to choose a probability sampling procedure. – If you do not want your results to be generalisable you are likely to choose a nonprobability sample.
MSC IN EMERGENCY PLANNING MANAGEMENT
There are many different sampling procedures and the one you choose to use in your design must take into account two key factors:
– Remember that even if you use a probability sampling procedure some individuals are more likely to participate than others. You then need to choose which sampling procedure will be most appropriate for your research. Sampling procedures can be divided into two categories: probability sampling or non-probability sampling.
5-29
EMERGENCY
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
Probability sampling
Non-probability sampling
Random
Opportunity or convenience
Systematic
Purposive
Stratified
Snowball
In probability sampling every population member has an equal chance of being included in the research. Non-probability sampling, on the other hand, means that some members of the population are more likely to be included in the research than others.
5.11.1 Probability Sampling Procedures Probability sampling procedures are chosen if the researcher wishes to show that the results of a sample can be generalised to a wider population. They are likely to use larger samples than researchers using non-probability sampling. There are many different types of probability sampling; however, the three most commonly used are described in this Unit.
MSC IN EMERGENCY PLANNING MANAGEMENT
Random Sampling The most frequently used example of a random sample is to draw names from a hat. In this sense, every individual has an equal, random, chance of having their name drawn. For example, if there are 100 names in a hat each individual has a one in a hundred chance of having their name drawn. Although this is the most commonly used example, there are few researchers who actually spend their time writing names on pieces of paper and drawing them out of a hat! This is particularly unsuitable for large-scale studies; for example if we wanted to send a survey to 1 percent of the population of the UK we would need a very large hat! More frequently used are random number generators. These are computer-based programs which allocate numbers to individuals and then select numbers, and hence individuals, randomly. Returning to our nation-wide survey, this may use electoral roles or phone books to allocate numbers to individuals before using a random number generator. Although this is referred to as a random sample, it must always be remembered that some individuals are still likely to be excluded from the sample. In the previous example those without telephones or those who are homeless would not have a random chance of being included in the research. Systematic Sampling Systematic sampling is a form of random sampling. Again using a list of names such as those found on the electoral role or in a telephone directory a systematic decision is made to decide who will be included in the research. Depending on the size of your population and your sample this may be, for example, every 5th name on the list or every 100th name on the list. Stratified Sampling Stratified sampling uses complex statistical procedures to gain a truly representative sample of a given population. This is a time consuming, although highly valued, sampling procedure which can be used only if you know the demographic characteristics of the overall population to which your results are to be generalised. This procedure is described in more depth in the statistics Unit as sample size in relation to population size must also be taken into consideration. However, gender can be used as a brief example here.
5-30
disaster
riskcrisis MANAGE RISK
We already know that 51 percent of the population in the UK is female. Therefore, if we want a truly representative sample 51 percent of our sample must be female. Although this appears to be relatively straightforward it must be remembered that this is only one variable that must be taken into consideration. Other variables such as age, class and ethnicity are generally taken into consideration, although other variables will be specific to your research project.
5.11.2 Non-probability Sampling Procedures There are both advantages and disadvantages to using non-probability sampling procedures. They are particularly useful for obtaining participants for research that is of a specific group that may be difficult to locate using more conventional means of sampling. For example, groups of people who would not necessarily see themselves as a group with specific features and meeting places but rather share one common feature thereby knowing a few people who have the same feature. It is also a relatively quick and simple form of sampling. There are many different types of nonprobability sampling, and the three most commonly used are described in this Unit.
The terms opportunity sampling and convenience sampling mean the same thing, and are often used interchangeably. However, it is advisable to choose one term and use it consistently within any one research project. There are no particular advantages to choosing either term and both are widely used. This sample is based on the opportunities a researcher has to obtain participants at their convenience. If, for example, you design a questionnaire and you give it to friends, family, other students to complete this is an opportunity or convenience sample. Purposive Sampling Purposive sampling requires the researcher to ‘handpick’ the participants they would like to use in their research. They may be picked for a variety of reasons, such as their knowledge of the issues in question, how likely they are to agree to participate in the research and/or if they are known to have had certain experiences. The participants are therefore chosen depending on how likely they are to produce rich, valued and varied data. Snowball Sampling Snowball sampling is named after the process whereby a snowball starting at the top of a hill may start as one solitary snowflake, and as it rolls down the hill it picks up more and more snowflakes until it is a large snowball! In less abstract terms, snowball sampling works by asking one person to participate in your research and then asking them if they know anyone else who may be interested in taking part. If each participant suggests two further participants, your snowball sample may grow quickly as demonstrated in Figure 5.18:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Opportunity or Convenience Sampling
PLANN NG
DISASTER
Figure 5.18: How a Snowball Sample Increases in Size
5-31
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Therefore, the sample size has grown in the above example from one to 15 participants and this process will continue ‘rolling’.
5.12 Data Gathering Techniques After deciding how you are going to measure your concepts and how you are going to operationalise them, the next part of your design will be to choose which data gathering techniques you are going to use. A research method is, quite simply, a way of collecting data and although these will be discussed in detail in another Unit, if you are planning to use more than one research method it is necessary to take this into consideration at the design stage.
5.12.1 Triangulation
MSC IN EMERGENCY PLANNING MANAGEMENT
The term triangulation was coined by Denzin and refers to ‘the combination of methodologies in the study of phenomena’ or the ‘use of multiple methods’ (1970: 300). These are sub-divided into two categories: • within-method triangulation; and • between-method triangulation. Furthermore, the concept of triangulation involves four other sub-strands which need to be considered: • data triangulation • investigator triangulation • theory triangulation • methodological triangulation. The concept of triangulation was designed with the intention of measuring both the internal and external validity of different research strategies. By considering ‘data’ and ‘method’ it is feasible to focus on several interrelated themes. Among the many advantages the concept of triangulation offers is that it enables a researcher to monitor the findings of different kinds of information, not only to check the validity of each of these individual parts, but also to highlight the novel dimensions of a particular field of enquiry which may not have been previously covered. Doing research in this style presents the opportunity to engage in a dialogue with the previous work done in similar areas. Hence the process of data collection and analysis involves constant cross-referencing and contextual reading. Moreover, as Porter (1994) notes: It is less a case of checking a ‘fact’ collected by one method, using another method, than using one method and then justifying the results by another. (1994: 70) In a research project it is possible to draw on various sources of data, including (1) secondary data, (2) official documents and reports and (3) fieldwork data. The first two categories could form the contextual reading part of the research project. The researcher may want to check the validity of these textual analyses by testing them against another source of evidence.
5-32
disaster
riskcrisis MANAGE RISK
The significant contribution of triangulation is that it provides a set of procedures to do research which wishes to draw on different techniques, strategies and sources of data. Different sources of data reflect the agenda of the institutions out of which they come. One may find that official sources are preoccupied with administrative matters whereas secondary sources may be overtheoretical. Attempts to synthesise radically different perspectives on the same issues may lead to new methodological as well as theoretical insights.
5.13 Conclusion
Many of the designs and approaches to research can be overlapped; for example, historical research could also be comparative research and vice-versa. There are therefore no hard and fast rules to the design of research, and it is more important that you amend a design to fit your research topic than to impose an unsuitable design on your topic. Similarly, keep in mind what you want your research to be used for and be prepared to amend your design if problems arise. Similarly, you may start, for example, with a probability sampling procedure and then realise that you will not obtain enough participants so switch to a non-probability sample. Alternatively, you may use more than one form of non-probability sampling; for example, you may combine an opportunity sample with snowball sampling. Again, it is much more important to amend your design than to carry on with one that you have found to be inappropriate.
5.14 Main Points • There are many interrelated parts to research design and it is necessary to decide which parts are relevant to your own particular piece of research. • The quality of research has traditionally been measured using three interrelated concepts: objectivity, reliability and validity. However, the usefulness of these concepts has been questioned by those working outside of the strict positivistic tradition. • An experiment, in its most simple form, attempts to measure what effect one variable has on another variable.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Design forms the starting point and overall basis of any piece of research, although the nature of the design will change depending on the nature of your research. While it is important not to rush this stage as this will result in a flawed piece of research, if you do complete your research and then realise that there have been problems with your design stage do not worry too much. This is part of the learning process, and as long as the problems are recognised and hence limitations placed on the generalisability and/or value of your research then this is recognised academic practice.
PLANN NG
DISASTER
• Correlations look for associations between variables – they cannot infer causality. • Correlations can be positive (as one variable increases so does the other) or negative (as one variable decreases so does the other). • A case study involves an in-depth complex analysis of one particular ‘case’ using as many different methods as are appropriate.
5-33
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
• Applied research, quite simply, relates to research which is not only applied to ‘real life’ issues but that is also of value to the ‘real world’. • Evaluation research aims to ‘evaluate the effectiveness of different types of actions in meeting needs or solving problems’ (Reinharz, 1992: 189). • Policy analysis involves the evaluation of existing and competing policies or programmes. • An audit is a continuous process of complex evaluation processes, where practice and standards are constantly changing in order to develop the best practice possible. • A hypothesis is basically a thesis, a theory, or a prediction about what you think your research will find. • Sampling refers to the selection of a sample number of cases from a wider population so that the results of the sample can be inferred back to the wider population.
MSC IN EMERGENCY PLANNING MANAGEMENT
• The advantages of using random sampling procedures are that they can be generalised to a wider population and are more valued in the positivistic tradition. • The advantages of using non-random sampling procedures are that they are relatively simple, fast, and it is not necessary to know the characteristics of the overall population.
5.15 Guide to Reading Students interested in reading more on the issues raised in this Unit can find useful material in the following books and/or articles: de Vaus, D. (2001) Research Design in Social Research, London: Sage Punch, K.F. (1998) Introduction to Social Research, Quantitative and Qualitative Approaches, London: Sage. Robson, C. (1993) Real World Research – A Resource for Social Scientists and Practitioner-researchers, Oxford: Blackwell.
5.16 Study Questions You should now write approximately 300 words in answer to each of the questions below. We believe that this is an important exercise that will assist your comprehension of the material and aid your progress on the course. Your answers are intended to form part of your own course notes and should not be forwarded to the University. 1. Describe the traditional benchmarks of quality (objectivity, validity and reliability). 2. Compare and contrast any two different research designs. 3. Describe and assess the advantages and disadvantages of using non-probability sampling.
5-34
disaster
riskcrisis MANAGE RISK
5.17 Bibliography Clarke, J. (1998) ‘Managing and Delivering Welfare’, in P. Alcock, A. Erskine and M. May (eds) The Student’s Companion to Social Policy, Oxford: Blackwell Publishers. Coolican, H. (1994) Research Methods and Statistics in Psychology (2nd edition), London: Hodder and Stoughton. Denzin, N. K. (1970) The Research Act, Chicago: Aldine. Denzin, N. K. (1983) ‘Interpretative Interactionism’, in G. Morgan (ed.) Beyond Method: Strategies for Social Research, California: Sage. de Vaus, D. (2001) Research Design in Social Research, London: Sage.
Dockery, G. (2000) ‘Participatory Research – Whose Roles, Whose Responsibilities?’, in C. Truman, D.M. Mertens and B. Humphries (eds) Research and Inequality, London: UCL Press. Evaluation Research Society (1980) Standards for Evaluation, Washington, DC: Evaluation Research Society. Fullan, M. (1991) The New Meaning of Educational Change, 2nd edition, London: Cassell. Lewin, K. (1946) ‘Action Research and Minority Problems’, Journal of Social Issues, 2: 34–6. Li Wan Po, A. (1998) Dictionary of Evidence Based-Medicine, Oxon: Radcliffe Medical Press. Madge, J. (1953) The Tools of Social Research, London: Longman. Manly, B. F. (1992) The Design and Analysis of Research Studies, Cambridge: Cambridge University Press. May, T. (1993) Social Research – Issues, Methods and Process, Buckingham: Open University Press. Mies, M. (1983) ‘Towards a Methodology for Feminist Research’, in G. Bowles and R. Klien (eds) Theories of Women’s Studies, London: Routledge and Kegan Paul.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Deutsch, K. (1987) ‘Prologue: Achievements and Challenges in 2000 Years of Comparative Research’, in M. Dierkes, H. Weiler and A. Berthoin Antals (eds) Comparative Policy Research: Learning from Experience, Aldershot: Gower.
PLANN NG
DISASTER
Patton, C. V. and Sawicki, D. S. (1986) Basic Methods of Policy Analysis and Planning, Englewood Cliffs, NJ: Prentice-Hall. Patton, M. Q. (1981) Creative Evaluation, London: Sage. Plummer, K. (1983) Documents of Life, London: Allen and Unwin.
5-35
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 5-36
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Porter, M. (1994) ‘ “Second Hand Ethnography”: Some Problems in Analysing a Feminist Project’, in A. Bryman and R. G. Burgess (eds) Analysing Qualitative Data, London: Routledge. Punch, K. F. (1998) Introduction to Social Research, Quantitative and Qualitative Approaches, London: Sage. Ramazanoglu, C. (1992) ‘On Feminist Methodology: Male Reason Versus Female Empowerment’, Sociology, 26(2): 207–12. Rapoport, R. N. (1970) ‘Three Dilemmas in Action Research’, Human Relations, 23: 499–513. Reinharz, S. (1992) Feminist Methods in Social Research, Oxford: Oxford University Press. Robson, C. (1993) Real World Research – A Resource for Social Scientists and Practitioner-researchers, Oxford: Blackwell. Stake, R. E. (1994) ‘Case Studies’, in N. K. Denzin and Y. S. Lincoln (eds) Handbook of Qualitative Research, Thousand Oaks, CA: Sage. Teune, H. (1990) ‘Comparing Countries: Lessons Learned’, in E. Øyen (ed.) Comparative Methodology, London: Sage. Westmarland, N. (2001) ‘The Quantitative/Qualitative Debate: A Subjective View of Objectivity’, Forum: Qualitative Social Research, 2(1) [on-line journal; accessed 6.7.2001].
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 6 Data Gathering Techniques
disaster
riskcrisis MANAGE RISK
6 Unit Six: Data Gathering Techniques 6.1 Aims and Objectives of this Unit The aim of this Unit is to introduce you to primary and secondary data gathering techniques. The objectives are to: • describe and explain a wide range of data gathering techniques; • provide information and advice on how to draft a measurement instrument for each research method discussed; and • address the advantages and disadvantages of using each data gathering technique discussed.
6.2 Introduction Once research has been designed to address a series of aims and objectives, the researcher must make careful choices about the appropriateness of certain data gathering techniques. Each technique has its own advantages and disadvantages. Often, researchers combine techniques to increase the validity of their findings. Throughout the data gathering process, the ethical considerations discussed in previous Units come into play. An initial choice of data gathering techniques concerns the collection of primary or secondary data. Primary data are those observations collected firsthand for the specific purpose of addressing the research issues in question. The way in which the research is designed, and the categories which are chosen to give a framework to the collection of observations and their subsequent analysis, are predominantly in the hands of the social scientist (Jupp, 1989). They are influenced by the issues the researcher is addressing and the theoretical ideas brought to bear on such issues (Jupp, 1989). Observational methods, interviews, life history research and surveys are all ways of collecting primary data. These methods are discussed in detail in the first part of the Unit. Secondary data are those observations collected by other people or other agencies with other purposes in mind. The ways in which observations are collected, categorised, organised and presented are very much in the hands of others and may not be influenced by the theoretical ideas in which the researcher is interested (Jupp, 1989). Documentary research is a technique using existing research material and is discussed in the latter part of the Unit, accompanied by a discussion of secondary analysis and content analysis of existing data and documents.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
By the completion of this Unit you will have developed a broad understanding of how data can be collected. You will possess the necessary information to assist you in designing your own research project.
PLANN NG
DISASTER
6-3
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
6.3 Observation and Ethnography 6.3.1 Types of Observation
MSC IN EMERGENCY PLANNING MANAGEMENT
One of the most basic techniques for data gathering is observation. All observation is guided by a hypothesis or research objective whereby researchers have to decide what it is they are looking for. The main virtue of observation is its directness; it makes it possible to study behaviour as it occurs. The researcher does not have to ask people about their own behaviour and the actions of others; he or she can simply watch them do and say things. Observation can be conducted in either a controlled or uncontrolled observational system (Nachmias and Nachmias, 1981). A controlled observational system is typified by clear and explicit decisions on what, how and when to observe; a non-controlled system posits fewer commitments on the part of the researcher and allows greater flexibility. Controlled observation is most frequently used with experimental research designs and occurs through laboratory experimentation which involves the introduction of conditions in a controlled environment (laboratory) that stimulate certain features of a natural environment. Uncontrolled observations tend to occur in the field, in a natural setting. This section will focus on the least controlled method of observation, participant observation, which is closely associated with the ethnographic tradition of the social sciences. Participant observation occurs through experiments in the field and is an appropriate method for investigating complex interactions, processes and changes in natural settings. Non-participant observation, where the observer is, in effect, an eavesdropper, someone who attempts to observe people without interacting with them and, typically without their knowing that they are being observed, is comparatively less frequently used and is often used in conjunction with participant observation (Singleton and Straits, 1999).
6.3.2 Understanding Ethnography Ethnography is a qualitative research method used for empirical data collection and analysis. It is practised through personal participant observation in a social grouping, with the distinct aim of providing an insider’s account of some cultural feature (or features) of that group. Ethnography has for most of this century been the dominant method of enquiry for social and cultural anthropologists; more recently, its popularity has increased among the other social sciences, in particular sociology and psychology. There has been considerable debate among theorists and practitioners as to what actually can be considered an ethnographic method. It is essential, therefore, to discuss the meaning of the term ‘ethnography’, which is often used in two distinct senses: • in reference to the practice of ethnographic fieldwork; and • the production of the ethnographic monograph. These terms are pragmatic subdivisions used for describing substantive parts of the ethnographic method. It is important that you are aware that in practice no part of the ethnographic research process is conducted in isolation from the other constituent parts. Ethnography can best be described as a holistic research method. All aspects of ethnographic enquiry are mutually reliant and influenced by the entire research process.
6-4
disaster
riskcrisis MANAGE RISK
Ethnographic Fieldwork Ethnographic fieldwork consists of observation (usually, but not always, by means of participation) in a particular group, culture, subculture, organisation or cultural system. The purpose of this method of enquiry is to gain an empathetic understanding of the activities of the subject group. Fieldwork usually involves the researcher living among or participating in the activities of the group being studied for a period of time. The desired outcome is that the researcher gains a view of the world as seen by the members of that group themselves. Doing fieldwork is about observing and analysing real-life situations, studying actions and activities as they occur. The main instrument of social investigation is the researcher, who relies upon learning firsthand about a people and a culture. However, if the researcher is to obtain an insider’s view of situations, it is vital to maintain membership in the culture in which they were reared while establishing membership in the groups which they are studying. For other research techniques the researcher does not need to experience the same event in order to analyse or understand it.
Ethnographic Monograph The ethnographic monograph is the report produced by the ethnographer. The style and presentation of these monographs will vary considerably owing to the highly personal and subjective nature of their production. Perhaps the most contentious issue surrounding the production of any ethnographic monograph is the types of information which the author considers suitable for inclusion and exclusion. All social researchers are selective in their use of data and ethnographers are no exception. This observation may give the impression that the production of the monograph constitutes the analysis phase of ethnography. This is not necessarily the case. Ethnographic analysis is a continuous process that takes place throughout the entire research period. The central issue to be addressed is the extent to which the researcher is aware of his or her subjective selection of data and acknowledges this in the monograph. The term ‘reflexivity’ is essential to your understanding of ethnographic research for it refers to the issue of the presence of the ethnographer in the text, in other words, the degree to which the ethnographer can display a reflexive awareness of him- or herself within an ethnographic context.
6.3.3 Conducting Ethnographic Research
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
When conducting fieldwork the researcher has to decide how close to a group they should get, that is whether they should adopt a covert or overt role. Covert observation is a highly contentious issue which some researchers repudiate on ethical grounds while others justify it on the basis that some groups, especially powerful elites, would otherwise be closed to research (Fielding, 1982 cited in Fielding, 1993). In practice the distinction between ‘covert’ and ‘overt’ research is often blurred, as it is impossible to consult every person who is being studied.
PLANN NG
DISASTER
First it is necessary to decide what research topic is appropriate to study through observation. The theoretical perspective pursued by the researcher will then influence the types of questions they will ask in the field and the data they will obtain. Once this is achieved there are four basic stages which are fundamental to most types of ethnographic study: 1. Gaining access 2. Preliminary socialisation
6-5
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 6-6
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
3. Gaining acceptance 4. Going native There are no rigid rules about how these stages should be conducted. Some stages may take a long time to complete or even act as a complete block to proceeding further with the research, while others may be quite unproblematic. But central to each stage of ethnographic work is a continual process of analysis. It is a process that commences when you first consider doing ethnography and continues throughout, often beyond even the final production of the monograph itself. 1. Gaining Access to the Research Field The ethnographer needs to be able to gain access to the organisation or social group which is to be studied. Gaining access will depend on the skill of the ethnographer as a social negotiator and in any investigation the researcher has to decide whom to contact. In overt research, access is accomplished through an explicit negotiation with a ‘gatekeeper’. ‘Gatekeepers’ may be group leaders or those who are otherwise institutionally powerful. These ‘gatekeepers’ are crucial to the research process; they can control access to the subjects of the research, or to the secondary sources such as organisational records or statistics. Often there are layers of gatekeepers to be negotiated, some formal others informal but both with hierarchies of authority and power between them. 2. Preliminary Socialisation Once access and entry to the research field has been achieved the researcher must go through a process of preliminary socialisation. This is an important process which enables both observer and observed to feel comfortable in each other’s company and hence behave in a naturalistic way. However, the ethnographer may find the organisation or group different from what had been anticipated and experience a period of culture shock. Planning for this problem in the research design is difficult, although to a degree it can be reduced by copious background research. The ethnographer in the field is faced with learning new forms of communication, new definitions of acceptable behaviour and perhaps most importantly identifying social roles. The process of doing this is often a traumatic experience involving stepping in and out of society’s cultural boundaries. The process through which the researcher socialises into a group can also create severe ethical problems. With the need to remain within the law yet at the same time gain the trust of group members, this may prove an impossible if not hazardous task. The ethnographer must also learn to adapt his or her behaviour to the social surroundings, simply in order to account for their presence. The need for flexibility in the field is vital to this stage of fieldwork. 3. Gaining Acceptance The ethnographer will need to gain acceptance from those being studied. This is not always unproblematic, for gender, race and social status may all operate as barriers to acceptance despite the social skills that the researcher may have. Gaining acceptance is an issue worth considering early on in ethnography. Certain individuals may never become fully accepted into a group because of who they are, or at least only be selectively tolerated in a particular part of that group’s activities. Problems often relate to marginality, the ethnographer may have concerns as to whether they have been accepted or, in terms of covert research, whether the deception has been discovered (Fielding, 1993).
disaster
riskcrisis MANAGE RISK
4. Going Native ‘Going native’ is a term suggestive of ethnography’s early roots in social anthropology (Jupp, 1989: 60). At this stage the researcher needs to establish some limits to the extent that they will go in order to become accepted by the research subjects. At the same time, it is important that the researcher becomes accepted to the extent that they will be able to operate within the group. There is a danger that during this careful balancing act the observer will begin to take actions and statements for granted rather than as data to be examined, questioned and treated as anthropologically strange. This situation poses a dilemma for the ethnographer; the impartiality of the observer will diminish in proportion to the level of active involvement with the observed, but failure to go native may result in no more than a tertiary understanding (Friedrichs and Ludtke, 1975). It is precisely by moving in and out of these two states that the ethnographer is able to both collect and analyse data simultaneously.
6.3.4 Data Collection record. Ethnographers use varied methods of data collection. Methods used will to a certain extent depend on the background of the researcher and the purpose of the study and also whether it is overt or covert in nature. A flexible strategy has the advantage of maximising both the types and quantity of data collected. The ethnographer, ideally, should make notes of anything interesting as and when it happens during the fieldwork stage. Notes should be regularly recorded in a journal or field diary. Field notes, which may be simply jottings made at various opportune moments, are based on observations that usually form the basis of writing up fieldwork. The diary, on the other hand, is personal. This should ideally take the form of a chronological journal which is written up as soon as possible after each encounter with the field (Okely, 1994: 23). They can contain anything from accounts of observations to highly personal abstract thoughts and ideas about the entire research project. Fielding (1993) highlights how it is important to have data on one’s own attitude in order to document one’s evolving relationship to others in the setting. A field diary provides a record of the entire research process up until the point of writing up. Note taking is often impractical while participating in the activities of the subject group. People may already feel too aware of the researcher to behave in a naturalistic way and note taking would simply aggravate this situation. In sensitive settings it may not be feasible to scribble notes. Tape recorders may also be employed, thus allowing the ethnographer more practical freedom for participation, but the use of these devices will have particular drawbacks. In addition to the already mentioned problems of not facilitating naturalistic behaviour, the researcher can find himor herself fiddling with tapes, batteries and recording levels during vital moments. Furthermore, the transcribing of recordings is a particularly tedious and time-consuming process. However rich and interesting the data, they are no use unless they are recorded accurately. For ‘covert’ research, neither note taking nor tape-recording is practical, and notes have to be made from memory as soon as possible after the event. However, memory can often prove unreliable. Erosion of memory is not related to time so strongly as it is to new input; that is, the more stimuli to which you are subjected during a day the more detail is forced out (Fielding, 1993).
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
A key concern for researchers in the field is what data to record, when to record, and how to
PLANN NG
DISASTER
6-7
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
6.3.5 Data Analysis Data analysis is addressed in greater detail in subsequent Units. For the ethnographer, however, analysis continues throughout the research process and provides an outline for many of the conclusions contained in the final monograph. In fact, the question of data analysis cannot be considered in isolation from data collection. When you collect data, you are by definition analysing them. When you are analysing data you are generating new thoughts and ideas, in other words, more data.
MSC IN EMERGENCY PLANNING MANAGEMENT
Becker (1971, cited in Fielding, 1993: 167) has suggested a procedure termed ‘sequential analysis’. In Becker’s approach the analysis of ethnographic data is carried out sequentially in the sense that analysis begins while one is still gathering data. In the periods between observation, one may ‘step back’ from the data so as to reflect on their possible meaning. Further data gathering is then directed to particular matters to which the observer has become sensitive by provisional analysis. Further observation may oblige the researcher to abandon the original hypothesis about that part of the process and investigate one more consistent with the setting. Consequently hypotheses become gradually refined. This method has the distinct advantage over methods like surveys, where, once the instrument is designed, analytic interests cannot affect the data collected.
6.3.6 Problems Associated with Ethnography By now you will be aware that ethnography has the distinct advantage of allowing the researcher firsthand experience of a particular group and can be a very effective method for investigating complex behaviour in natural settings. However, there are a number of issues that can render ethnographic research problematic. The main issues are reliability and validity. Reliability refers to the extent to which a study is replicable: that is, if another researcher or the same researcher, at another time, were to repeat the study would they obtain the same results? Validity is concerned with accurately measuring a concept and in terms of observation a number of difficulties are apparent. First, it is important to acknowledge that the subject matter of the social sciences, human social behaviour, is shrouded in fuzzy concepts such as alienation or aggression, and these are by definition culturally defined concepts which are likely to vary across societies and history. This problem makes social phenomena difficult to measure and compare. The social researcher also has to contend with ‘multiple social realities’. People within the same social grouping will frequently disagree about the nature of social reality. Hammersley (1990) suggests: It is argued that it is characteristic of human beings that they create multiple social worlds or realities, that all perception and cognition involves the construction of phenomena rather than mere discovery. From this point of view, there are multiple social realities or worlds, not just a single one; and it may be concluded that contradictory views of ‘the same’ phenomena by different cultural groups are equally ‘true’ in their own terms. As a consequence of these problems, repeat studies on the same society, when contrasted with the original often produce results at variance. Consequently we are faced with the question: can we legitimately say that these studies represent valid academic knowledge or are they merely the idiosyncratic introspection of a particular ethnographer? It is important to note that social research
6-8
disaster
riskcrisis MANAGE RISK
is not a mechanical process, and the creative stages which precede and follow the standard procedures of enquiry are largely interpretative and imaginative. Researchers are also human, consciously or not interpreting the events that they are observing. This leads us on to the issue of objectivity. For the positivistically inclined, the problem is how the observer can remain objective in order to provide demonstrable proof, while for the anti-positivist it is how to get close enough to the actors to understand their meanings properly. Some theorists have disputed whether any method in either the pure or social sciences can ever be objective. Scientists will try to minimise their effect on the phenomenon observed, but this is impossible without first removing the phenomenon from the context in which it normally operates. One approach for achieving validation is to accept that ‘objective’ observation is impossible and ask on what grounds might there be credibility of the ethnographer’s account (Fielding, 1993). It is here that a number of commentators have offered direction.
Fielding (1993: 166) emphasises the importance of keeping a notebook evaluating one’s observations and suggests that Bruyn’s (1966) criteria of subjective adequacy are useful for this purpose: 1. Time The more time the ethnographer spends with the group the greater the likelihood of adequacy. 2. Place The closer the observer works to the group the greater is the likelihood of adequacy. 3. Social Circumstances The more varied the status opportunities within which the observer can relate to the members and the more varied the activities witnessed the more likely the interpretations will be true. 4. Sensitivity to Language (such as the argot, slang or jargon) It is argued that the more familiar the observer is with the group’s language the greater the accuracy of observation.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Fielding (1993) draws our attention to how understanding is derived from experience. Based on this premise, some researchers have pursued a test of congruence or principle of verifiability. The idea is that in any natural setting there are norms or rules of action in which members are competent. Understanding on the part of the observer is achieved when the observer learns the rules. In Hughes’ (1976: 134, cited in Fielding, 1993: 164) description of the principle of verifiability, ‘understanding’ is achieved when the researcher knows the rules and can communicate them to both members and colleagues in such a way that if a colleague were to follow them he or she could become a member of the group. However, the test of congruence may comprise an ideal check on the validity of observations but it has to be recognised that many consumers of research do not have the time to perform it (Fielding, 1993).
PLANN NG
DISASTER
5. Intimacy Bruyn (1966) suggests the observer record how he or she experienced and encountered social openings and barriers in seeking accurate interpretation of setting specific meanings. Intimacy, or how close one is supposed to get, can be constrained by one’s own reserve as well as by members.
6-9
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
6. Social Consensus ‘Social consensus’ is fulfilled by maximising confrontation of the group’s expressive meanings either directly – by checking interpretations with members – or indirectly – by observing what members say about an interpretation. This section has highlighted the different forms of observation available to researchers and has paid particular attention to the mechanics of the ethnographic process using participant observation. You will have developed an understanding of how to conduct an ethnographic study and be aware of the problems you are likely to encounter. You will now be introduced to another method of primary data collection, interviewing.
6.4 Interviews
MSC IN EMERGENCY PLANNING MANAGEMENT
Interviewing is one of the most widely used research techniques and can take many different forms. It can be administered either one-to-one, where respondents are seen individually, or to a group, where the interviewer or group leader guides the discussion amongst a small group of respondents. Both approaches will be discussed in this section. Interviews can serve a number of purposes. Fielding (1993) highlights how interviews are often used to identify the main behavioural groups to be sampled, and lend an insight into how they should be defined. Interviews can be used to familiarise researchers with the phraseology and concepts used by a population of respondents. Interviews have often been used to establish the variety of opinion concerning a particular topic. They have also been used to establish relevant dimensions of attitudes. In addition, although this use is much debated, interviews are often used to form tentative hypotheses about the motivation underlying behaviour and attitudes.
6.4.1 Varieties of Research Interviews The simplest method of differentiating types of interview is by the degree of structure imposed on its format. Three types of interview structure will now be addressed. Structured Interview In the structured interview the wording of questions and the order in which they are asked is the same from one interview to another, for example, the archetypal survey interview used in market research. This is done to ensure that when variations appear between responses, they can be attributed to the actual differences between respondents and not to variations in the interview (Nachmias and Nachmias, 1981). The interviewer will possess an ‘interview schedule’ which conveys the formality of this type of interview. Interviews which are rigidly structured and ask close-ended questions encourage monosyllabic or short answers from respondents. The archetypal questionnaire style interview with tick box answers completed by the interviewer also engenders a relationship with the respondent that is remote, distant and short lived. Structured interviews are suitable when you have some idea of what is happening with your sample in relation to the research topic and where there is no danger of loss of meaning as a result of imposing a standard way of asking questions (Fielding, 1993). Semi-Structured Interview In semi-structured interviews the interviewer asks certain, major questions in the same way each
6-10
disaster
riskcrisis MANAGE RISK
time but is free to alter their sequence and to probe for more information. The interviewer is thus able to adapt the research instrument to the level of comprehension and articulacy of the respondent, and to handle the fact that in responding to a question, people often also provide answers to questions we were going to ask later (Fielding, 1993). This type of interview encourages a friendly rapport to develop between the interviewer and interviewee. Unstructured Interview
To elicit rich, detailed materials that can be used in qualitative analysis. Its object is to find out what kinds of things are happening rather than to determine the frequency of predetermined kinds of things that the researcher already believes can happen. Fielding (1993) emphasises how because of its simplicity of design and correspondence to the conversational procedures which are routine in social life, the unstructured approach is very often the type of interview students conduct in their own research projects. The following discussion is applicable to interviewing in general; however, many points will specifically be made to the latter type of interview technique for the structured approach is discussed in more detail in Section 6.6.
6.4.2 Designing an Interview Guide Lofland (1971: 75–84, cited in Fielding, 1993: 142–3) provides an excellent discussion of the basis of designing a guide to conducting an unstructured interview. First, it is necessary for you to identify a topic which is appropriate to study by interviewing. Once this is settled, the first step is to think about what you find interesting and problematic about this particular topic; Lofland calls these ‘puzzlements’. It is then necessary to write down questions which express each ‘puzzlement’. Consulting other people as to what they find puzzling about the topic is also useful, for you are teasing out what is puzzling about the phenomenon in the context of your particular ‘cultural endowment’. To clarify, we are located in a particular social context, with a particular biographical background, and we must recognise that our point of departure is always what is puzzling relative to our particular cultural perspective.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
In unstructured or focused interviews the interviewers simply have a list of topics which they want the respondent to talk about, but are free to phrase the questions as they wish, ask them in any order that seems appropriate at the time, and even join in the conversation by discussing what they think of the topic themselves (Fielding, 1993). Although it may appear to be without a structure, the researcher must conduct the interview within a particular framework which is ‘flexible’ but also controlled, thus the interviewer will often possess an ‘interview guide’ which allows the interviewer to take their own path within certain guidelines. The questioning should be as open-ended as possible, in order to gain spontaneous information about attitudes, beliefs, values and actions, rather than a rehearsed position. Similar to semi-structured interviews this technique encourages good relations between the interviewer and interviewee. Lofland (1971: 76, cited in Fielding, 1993: 137) summarises the objective of the unstructured interview as being:
PLANN NG
DISASTER
The next stage is to write each ‘puzzlement’ on a separate piece of paper and sort them into piles of which seem to be topically related. This process may have to be repeated several times to obtain an order that seems to express the social phenomenon. Some ‘puzzlements’ will be disregarded for as your knowledge of the phenomenon develops, you will identify some as being
6-11
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
irrelevant, others will emerge as being related and you can amalgamate them. The ‘puzzlements’, expressed as questions, can then be written in the form of a list which should display a logical, orderly sequence, taking the form of an outline. The last step before you pilot the interview, to check if there are any problems in practice which may have been overlooked in the design, is to design probes. ‘Probing’ involves follow-up questioning to get a fuller response; it may be verbal or non-verbal (Fielding, 1993). As this is an unstructured interview the probes can be couched in informal terms or can be written flexibly so that the exact words used will depend on the interviewee and their level of comprehension and ease of response. Fielding (1993) highlights how designing probes is at least as important as generating the main questions in the guide. A more detailed discussion of probes and prompting is found in the following discussion.
6.4.3 Conducting Interviews The following points are presented by Thompson (1978), developed to assist in the delivery of life
MSC IN EMERGENCY PLANNING MANAGEMENT
history/oral history interviews, but are generally applicable when conducting any type of interview. 1. Adequate Preparation and Background Information Reading to increase one’s knowledge may be useful, but in some instances exploratory interviews at the beginning of a project may also help to define and focus the research problem. Such interviews are termed ‘general gathering interviews’, which are akin to the ‘pilot interview’ of a big survey. 2. Phrasing of Questions Great care must be taken in the phrasing of questions to ensure that they are clear, straightforward and couched in a familiar language that the interviewee will understand. For a more comprehensive discussion of the phrasing of questions refer to Section 6.6.4. 3. Leading Questions Leading questions must be avoided because of the problem of ‘reactivity’ and interviewer bias. If your personal views influence the responses of the interviewee, then this could damage the reliability of your data. However, Thompson (1978) points out that in certain circumstances it may be important to show the interviewee that you sympathise with their (strong) views or else they may never feel free to express them at all. This is particularly relevant when you are interviewing somebody with a ‘minority standpoint’ or somebody who may espouse unconventional views. 4. Terminology It is important to encourage the interviewee to discuss ideas and concepts in their own words and terminology. Therefore, avoid using terms like ‘social class’, ‘organised crime’, ‘peer group pressure’, etc. as this may lead the interviewee to provide data using your terminology. Your data will be more convincing when couched in the language of the participants themselves. 5. Sequence of Topics As already discussed in all kinds of interview situations, whether semi-structured and open-ended or rigidly structured and closed-ended, an interview schedule or guide should be used. It is your
6-12
disaster
riskcrisis MANAGE RISK
responsibility to guide the interviewee, and at certain times during the interview it is likely that he or she will digress. You will need to reign him or her back in and an interview schedule or guide is most helpful in this task. 6. Equipment
While all methods of recording appear to have their respective problems, the tape recording method is probably more favourable as it allows the natural flow of the conversation to develop as well as enabling the material to be recorded exactly as it is revealed. 7. Aids to Memory Visual aids can be quite useful to take along to the interview and may serve two purposes. First, they may help to trigger memories for the interviewee and second, they might encourage him/ her to dig out various documents such as letters, diaries, photo albums, newspaper cuttings, etc. Thompson (1978) argues that this could be the ‘most valuable by-product of an interview’. 8. Interview Location The location of the interview will have important ramifications for the general atmosphere in which the interview is conducted. An examination of the literature on interviews reveals that interviewees are generally more responsive when they are made to feel at ease, usually within the comfort of their own home. It is generally best to conduct the interview in complete privacy, with nobody else present, so as to elicit more truthful responses and to minimise pressure on the interviewee to feel that they need to provide socially acceptable answers. The locale of the interview and the people present will of course need to be taken into consideration when interpreting the data. 9. Making Contact
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
At an early stage, decisions must be made as to which method should be used to record the details of the interview. Basically there is a choice between note taking, tape recording or recalling from memory the details and writing them down as soon after the interview as possible. This latter approach may result in vital pieces of information being lost since it depends upon the memory of the researcher. Some interviewees may find it difficult to relax with a tape recorder running and may hold back on very useful information. This may be overcome if the interviewee is assured of confidentiality and (usually) anonymity before the interview begins. It is also very important to ensure that any technical equipment used is working properly. Others may find that note taking disturbs them since a sense of formality is created within the interview. Furthermore, note taking may lead to invaluable pieces of information being missed, including important signals, often nonverbal, from the interviewee.
PLANN NG
DISASTER
Establishing contact with your chosen respondent(s) can initially be accomplished by writing a letter or making a telephone call. When you attempt to arrange a visit you must always provide the interviewee with the opportunity to refuse or to postpone to a later date. You may not be clear how much to reveal about the nature of the research at this stage and should always consider factors such as the age and literacy of the interviewee and the way in which it may affect your relationship to them. For example, a young person may find it a little daunting to receive a longwinded letter or telephone call detailing the nature of the research, which may actually deter them. In general, it is best to provide more detail when you finally meet.
6-13
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
10. Recording Quality It is not enough to ensure that the machinery is working. You must also try to minimise any acoustic problems in the room where the interview takes place. The room should be quiet and preferably free from disturbances. Noise from traffic, a telephone ringing every few minutes, etc. can actually ruin the quality of the recording. Remember not to place the microphone next to the recorder, as you will find that you have recorded the noises from the machine over and above anything else. 11. The Interview
MSC IN EMERGENCY PLANNING MANAGEMENT
An interview is not a dialogue, or a conversation. The whole point is to get the informant to speak. You should keep yourself in the background as much as possible, simply making supportive gestures, but not thrusting in your own comments and stories. It is not an occasion which calls for demonstrations of your own knowledge or charm. And do not allow yourself to feel embarrassed by pauses. Maintaining silence can be a valuable way of allowing an informant to think further, and drawing out a further comment. The time for conversation is later on when the recorder is switched off. Of course you can go too far in this direction, and allow an informant to falter for lack of come-back. To grind to a halt in silence at the end of an exhausted topic is discouraging, and a firm question is needed before this point. But in general, you should ask no more questions than are needed, in a clear, simple, unhurried, manner. Keep the informant relaxed and confident. Above all, never interrupt a story. (Thompson, 1978: 178) Fielding (1993) explains how ‘prompting’ is an important technique used in interviewing which involves encouraging the respondent to produce an answer. In structured interviews, great care must be taken to get a response without having to put the words into the respondent’s mouth. The mildest technique is to merely repeat the question. If this fails the interviewer may be permitted to slightly rephrase the question, the interview schedule should list acceptable rephrases to ensure that the question is delivered in the same way to each respondent. In unstructured interviews the interviewer will be more flexible and maintaining consistency in question structure is not as important. It is also important to stimulate the interviewee, and this can be achieved by asking probing questions. Nachmias and Nachmias (1981) describe the two major functions of probing. First, they motivate the respondent to elaborate or clarify an answer or to explain the reasons behind the answer. Second, they help focus the conversation on the specific topic of the interview. In general, the less structured the interview, the more important and acceptable probing becomes. It is important to note that a probe should be as neutral as possible and should not incline the respondent to a particular response. 12. Sensitive Questions More often than not, sensitive questions should be left until much later in the interview, or preferably when you meet with the interviewee on another occasion. Remember that during the first few times you meet you are building and developing a relationship based on trust and mutual respect. Therefore, it is important that the interviewee does not feel pressurised to disclose very private and personal (sensitive) information to you. You will usually find that after a while they will
6-14
disaster
riskcrisis MANAGE RISK
volunteer the information to you with very little prompting when you delicately mention the issue. If the interviewee finds it difficult to talk about an issue, then leave it for now, as you may be able to come back to it at a later date. 13. Ending the Interview Do not exhaust the interviewee; and pay attention to their feelings. An hour and a half to two hours will normally be a sensible maximum. Try to end the interview on a positive note. It is important not to just get all the information you can from the respondent and then leave. They may actually feel slighted once you have gone or may start to worry about what they have told you. It is important to stay for a while and give something of yourself. This is something that feminist interviewers are quite strong on, that is, trying to minimise power differentials and sharing something of their self with the respondent. Often this process is referred to as ‘humanising’ the interviewee and/or the interview process. It is also important to make contact with the interviewee as soon as possible after the initial interview has been conducted, whether or not there will be a follow-up interview to assure the respondent that what they said was fine and perhaps reiterate confidentiality.
A small number of individuals, brought together as a discussion or resource group, is more valuable many times over than any representative sample. Such a group, discussing collectively their sphere of life and probing into it as they meet one another’s disagreements, will do more to lift the veils covering the sphere of life than any other device that I know of. (Blumer, 1969: 41, cited in Flick, 1998: 116) The majority of research interviews are one-to-one, but researchers interested in consensus formation, interactional processes and group dynamics may find the group interview useful. Group discussions have special value to those who want to assess how several people work out a common view, or the range of views about a topic. They allow you to see how people interact in considering a topic and how they respond to disagreement. They can assist in identifying attitudes and behaviours which are considered socially unacceptable. Group discussions are used for various reasons. To social scientists, the strength of group discussions is the insight they offer into the dynamic effects of interaction on expressed opinion (Fielding, 1993). Group discussions correspond to the way in which opinions are produced, expressed and exchanged in everyday life (Flick, 1998). Generally, group interviews have the advantage of being quicker and cheaper to conduct than individual interviews with the same number of respondents. Group discussions are also useful in obtaining more information from some respondents for once one respondent has launched a line of discussion others are often more willing to join in.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
6.4.4 Group Discussions and Focus Groups
PLANN NG
DISASTER
Flick (1998) draws our attention to the different forms of groups used in interviewing. A common feature of the varieties of group discussions is, against using the purposive questioning of one person, to use as a data source the discussion on a specific topic in a natural group (i.e. existing in everyday life) or an artificial group (i.e. put together for the research purpose according to certain criteria). Furthermore, there is a distinction between homogenous and heterogeneous groups. In homogenous groups members are comparable in the essential dimensions related to the research question and have a similar background. In heterogeneous groups, members should be different in the
6-15
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
characteristics that are relevant for the research question. This is intended to increase the dynamics of the discussion in order that many different perspectives will be expressed and also that individual participants’ reserve will be broken down by the confrontation between these perspectives. The main strength of the group interview, increasing the dynamics of the discussion, is also the main source of problems when applying the research method. Maintaining control of the group and remaining focused on the subject at hand is often problematic. Often a more fully elaborate interview schedule is needed to stay on track. Sharing the running of the session or splitting roles so that one person maintains the discussion while the other looks ahead to new topics and introduces them can lessen this problem. Because each group session will vary considerably from the next, it is often difficult to design relatively common conditions for the collection of data. A further problem is that it is difficult to tape record group discussions; you will probably need more than one microphone, and you should check that your equipment can record people speaking at different levels and at different distances from the microphone before conducting the actual session.
MSC IN EMERGENCY PLANNING MANAGEMENT
The method of ‘group discussions’ was dominant in earlier studies, especially in the German speaking area; the method has more recently had some kind of renaissance as ‘focus group’ in Anglo-Saxon research (Flick, 1998). Focus groups are a popular method used in marketing and media research. Focus groups are used as a method on their own or in combination with other methods such as surveys, observations and single interviews. Morgan (1988: 11, cited in Flick, 1998: 122) sees focus groups as useful for: • orienting oneself into a new field; • generating hypotheses based on informants’ insights; • evaluating different research sites or study populations; • developing interview schedules and questionnaires; and • getting participants’ interpretations of results from earlier studies. Morgan (1988) also offers insight into conducting focus groups. The number of groups to be conducted depends on the research question and on the number of different population subgroups required. It is generally suggested by commentators on this method that it is more appropriate to work with strangers instead of a group of people that are friends or know each other because the level of things taken for granted which remains implicit tends to be higher in the latter. On the other hand, Morgan also suggests that the researcher(s) should begin with groups as heterogeneous as possible and then run a second set of groups that are more homogenous.
6.4.5 Problems Associated with Interviews Throughout this section you will have become aware of several potential problems when designing and conducting interviews. One particular area where researchers have been criticised is in relation to the effects of the interviewer. The work of Fielding (1993) is useful in highlighting the effect an interviewer can have on the research process. As you will already be aware, structured and unstructured approaches vary
6-16
disaster
riskcrisis MANAGE RISK
greatly in the role they permit the interviewer to play in the interaction with the respondent. Consequently, the extent to which the interviewer will have an effect on the outcome of the interview may well depend on which approach is adopted. Fielding discusses how there was an early concern with whether the demographic characteristics of the interviewer and respondent should be matched, for characteristics such as race, age, sex, social class and religion have all proven to impact on the interview process. Hyman (1954, cited in Fielding, 1993) found, for example, that white interviewers received more socially acceptable responses from black respondents than from white respondents. Similarly black and Oriental interviewers obtained more socially acceptable answers than did white interviewers, with the differences predictably being greatest on race. Fielding (1993) emphasises how socially acceptable responses are particularly likely to represent convenient ways of dealing with interviewers rather than expressing the respondent’s actual view. It is for these reasons that structured interviews try to match interviewers to the characteristics of the research population wherever possible.
Validity refers to the question of whether a research tool actually measures what it purports to measure: for example, during an interview does the interviewee reveal their attitudes (truthfully) or simply what they profess or think the researcher would like to hear? People may also try to conceal opinions which are not politically correct. This is a difficult issue and is closely tied to the problem of reliability. Through interviewer training, careful question design and probing and comparison with results using other methods, research findings can be strengthened. However, as Fielding (1993) emphasises, it seems that we need a better theory of why people do and do not act as they say they do for this is a frequent problem exposed when using other research methods to check the reliability and validity of responses. This section will have provided you with an understanding of the different types of interviews frequently conducted. Generally speaking the chosen interview technique will reflect the methodological stance (qualitative or quantitative) of the researcher and the nature of the research problem at hand. You will be aware of the importance of possessing an interview guide or schedule and the problems you are likely to encounter when designing and conducting an interview. The next section focuses on gathering data through life history research in which the interview process plays an integral part.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The problem of interviewer bias has mainly been directed at unstructured interviews. There is always the danger of ‘personal’ and ‘procedural’ reactivity which results from the close and intimate nature of the relationship between interviewer and interviewee. This is related to the problem of ‘objectivity’ and ‘detachment’, a common difficulty of qualitative research in general. Thus, the researcher may become too close to research subjects and unable to maintain the required degree of objectivity. This is a difficult task for qualitative researchers (generally) who need to maintain a balance between their ‘insider’ and ‘outsider’ status. Significantly, bias can enter research at both the stage of interviewing and during the process of data analysis. Thus it is important for researchers to be extremely ‘self-reflective’ in an effort to avoid (or at least minimise) the problem of interviewer bias. Fielding (1993) emphasises how it is crucial to have as full and accurate a record of the interview as possible, for scrutiny during analysis.
PLANN NG
DISASTER
6-17
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 6-18
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
6.5 Life History Research 6.5.1 What is Life History Research? The life history technique is both a method of data collection and a source of data (see Section 6.7). Life histories are possibly the best method for providing detail about the development of people’s beliefs and attitudes over time and are a valuable tool for researchers interested in making connections between psychological development and wider social processes. Life histories also provide a synthesis of an ethnographic tradition, emphasising the need to understand from the point of view of the individual (in the present), and the historical tradition of trying to understand the present within the context of the past, and vice versa. The life history method is able to achieve a recognition and appreciation of the complexities of life. Social science, in its quest for rationality, can oversimplify the complex nature of life and fail to cater for areas of ambiguity. It is important to recognise that ambiguity and inconsistency are inescapable features of human existence. The life history technique is dynamic, as it allows for ambiguity and inconsistency in research to be explored. This point is illustrated by Plummer (1990: 68) who comments: Most social scientists in their quest for generalisability impose order and rationality upon experiences and worlds that are more ambiguous and more chaotic in reality. ... Researchers seek consistency in subjects’ responses, when subjects’ lives are often inconsistent. The life history technique is peculiarly suited to discovering the confusions, ambiguities and contradictions of everyday lives. The life history technique is further unique in that it enables human experiences to be placed within a life context, thereby providing a deeper understanding of the reality of social experience, demonstrating that experiences cannot conveniently be extracted from the complex process of life. In addition, in areas of study where there exists a paucity of knowledge the life history can be particularly useful as an exploratory tool. It can provide new insights and open up new areas for social investigation and is useful for the generation of new hypotheses and the development of new concepts. Methodologically, therefore, the life history needs to be placed within the broader spectrum of qualitative research strategies with an emphasis upon studying people in their ‘natural’ surroundings. This is a similar goal to that of participant observation, but the focus of the life history is mainly concerned with the subjective meanings of individuals. Such strategies depend upon observing people in their own territory and interacting with them in their own language. The life history method like participant observation adopts an ‘holistic’ approach to the individual, thus providing a focus on ‘totality’ (Plummer, 1990). As a form of historical data the ‘life history’ combines both written and oral evidence and is sometimes referred to as ‘documents of life’ (Plummer, 1990). The main sources of data include ‘oral history’ (story-telling) in conjunction with the scrutiny of other private and public documentary evidence, for example, letters, contemporary reports or newspaper descriptions. These are used to expand upon and check the validity of information provided.
disaster
riskcrisis MANAGE RISK
6.5.2 Conducting Life History Research Plummer (1990) describes the classic life history technique where the interviewee relays an entire life history to the researcher. This obviously takes a number of years and may not be suitable for a short project. However, the life history approach may still be used in order to obtain in-depth, focused interviews for a short project which may also include the collection of other sources of data. Life histories do not necessarily cover the entire span of a person’s life and may simply focus upon a certain aspect of interest.
After the research subject(s) have been selected, it is important for the researcher to be honest about his or her motives. Plummer (1990: 90) clearly illustrates the importance of this: The subject will undoubtedly be curious as to why you the researcher are interested in him or her and you should be ready with an honest response, a response which will almost certainly include career and professional advantages, side by side with some tangible and political and/or moral concern for a social problem. At the outset, it is necessary to come fairly clean with subjects who will very likely sense a whiff of exploitation unless you do. The next stage is conducting the life history interview. The previous section outlined the different interviewing techniques from which social researchers can choose. Your choice of technique will be influenced by the nature of your investigation. Unstructured interviews are frequently employed in life history research because of the depth of information they can extract. You should consult the interview and questionnaire sections for guidance on how to prepare an interview schedule or guide and conduct an effective interview. Following the collection of data, it is important that careful thought be given to the mode of storage, for life history research can produce a mass of data. It is important that any notes made during the interview are clearly typed and the transcripts from tape recordings must be stored in systematic order. It is also suggested that you should keep a log with all your personal impressions experienced during the interview process, as this assists in clarifying all the ethical and personal problems as they occur.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The initial stage of any research project after deciding what it is you wish to study consists of choosing suitable research subject(s). This will largely be dictated by the nature of the study (for example, the question of whether a scientifically generated random sample is required). An important point to consider when selecting research subject(s) is accessibility. The lengthy and detailed nature of the life history method will require the research subject(s) to be available over an extended period and be willing to dedicate the amount of time necessary for the research. The nature of the ‘informal’ interview requires a degree of intimacy and close contact between the researcher and interviewee. While this may be difficult to attain at the beginning of a research project, it is important to be aware of this in order to minimise conflict between yourself and the interviewee.
PLANN NG
DISASTER
6.5.3 Problems Associated with Life History Research Many of the problems associated with conducting life history interviews are outlined in the problems associated with interviewing in general. However, life history research has received its own set of criticisms.
6-19
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
1. ‘Atheoretical’ The rich and in-depth nature of data elicited by this method has often led critics to compare life history research with that of journalism rather than social science. It has therefore been criticised as being ‘atheoretical’, that is, providing no link with theoretical understanding. However, it must be recognised that this is not an inherent property of this research method. Watson (1976, quoted in Plummer, 1990: 122) observes that: Theory is everywhere, and is intimately connected to issues of problem, method, and substance. Thus, any particular problem brings in its wake a most satisfactory theoretical approach and a preferred methodological stance. ... The same is true of those who are interested in life history research. It stems from the problems of trying to understand how concrete people give meaning to their everyday lives (or some aspect of them), is thus highly connected to the theorisations of phenomenologists and symbolic interactionists, and in turn becomes the prime tool of such theoreticians.
MSC IN EMERGENCY PLANNING MANAGEMENT
Plummer identifies the five following uses of theory in life history research: theory as orientation, building theory, falsification, illustrating theory and life history as ‘text’. He concludes: The claim is sometimes made that personal documents are inherently atheoretical, and hence of little value. ... I have tried to show that historically theory has frequently been a key part of the tradition, and have outlined a number of uses and objects to which it can be put in contemporary work. ... I hope to have shown the role that personal documents can play in social science: no longer should we plead theoretical ignorance in using them. (Plummer, 1990: 133) 2. Time Consuming Life history research has been criticised for being wasteful, time-consuming and inefficient (Gottschalk, 1945). It is possible that a researcher may spend many months, even years, collecting life history interview data. Furthermore, the collection of interview data is often combined with documentary sources which can sometimes prove inaccessible. 3. Representativeness The life history technique has often been dismissed due to a lack of representativeness which refers to the ‘typicality’ of the research participant(s) or sample. To what extent are the cases or case representative? How far can we generalise on the basis of only one detailed case or a few? Life history cases are consequently deemed atypical. It has been viewed as a journalistic, subjective account which is incapable of representing the objective order. However, as Plummer (1990) argues, this need not be the case. While recognising the difficulty of avoiding this problem, he suggests that researchers can minimise the effect through explicitly identifying the relationship of life histories to the wider population: Where it is possible to relate the characteristics of a ‘case’ to a ‘sample’, confidence in its generalisability may be considerably increased. (Plummer, 1990: 100)
6-20
disaster
riskcrisis MANAGE RISK
However, it is not essential that researchers do this if the focus of their research is a detailed description (unique story) of one case. In such circumstances, however, they should avoid making sweeping generalisations, though, of course, it is perfectly legitimate to make speculations about wider implications. 4. Subjectivity Some critics argue that the life history technique is not a viable research method because of its extreme subjectivity. Consequently, the method is deemed unscientific, providing no more than a journalistic account of social experience. However, it is important to emphasise that it is the subjective nature of this method which is also of benefit to social science and therefore this accusation is to a certain extent unjustified. Life histories have allowed the subjective experience of individual actors to be communicated. By allowing these personal interpretations, social scientists are able to become familiar with new social phenomena through the connection with people’s everyday lives. Without this connection and familiarity, theoretical perspectives become mere invalid speculation as social scientists fall into the false ‘objectivity’ trap. This point is clearly illustrated by Faraday and Plummer (1979: 776) who remark:
5. Reliability and Validity Reliability and validity are issues for all research methods. In this instance, the life history method has been criticised for being unreliable, a product of the extreme subjectivity of the approach. Whyte (1989) explains how the interviewee may inadvertently distort information in order to please the interviewer. People may also try to conceal opinions which are not ‘politically correct’. Whyte (1989) explores the problem of recollecting past feelings, as individuals often recollect the past in a manner which fits with their current point of view. The unreliable nature of life histories is also recognised by Blumer, who remarks: Many critics charge that the authors of personal accounts can easily give free play to their imagination, choose what they want to say, slant what they wish, say only what they happen to recall at the moment, in short to engage in both willing and deliberate deception. They argue accordingly, that accounts yielded by human documents are not trustworthy.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Whatever is of interest to the sociologist, one pivotal perspective which should always be entertained, is that of the participant experience itself. This is not to say that social scientists should always rest content with such a perspective; it is to assert that if they fail to consider people’s concrete experiences and have ‘ultimate familiarity’ with them, then they will invariably run the extreme risk of simply being wrong – of speculating and abstracting about a phenomenon which does not really exist, of communicating the fallacy of objectivism, of substituting your own perspective for that of your subjects.
PLANN NG
DISASTER
(Blumer, 1979, in Plummer, 1990: 46–7) However, it is important to recognise that it will always depend upon what the researcher is actually trying to achieve. If it is a subjective account/perspective that one is after, then this is possibly the most valid method available. The problem of reliability can also be minimised, according to both Plummer (1990) and Whyte (1989), by conducting a series of validity checks. Researchers can check for inconsistencies and misinformation in a number of ways. For example, it is possible to ask the research subject to read
6-21
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
the document and provide a critique. Another strategy for analysing the validity of the data is to compare the factual parts of the account provided with official statistics and records. For example, a respondent’s verbal account of the past and how they felt, etc. can be checked against other personal documents such as a diary. Alternatively, validity checks can be carried out in comparisons with others who occupy similar roles, or who have a close relationship with the informant. In conclusion, it is evident that life histories can be used as both a primary and secondary source of data. The interviewing process is central to this data gathering technique and it has received many distinct criticisms. However, it is a useful research tool, possessing the ability to investigate the complexities of social life. The final primary method of data collection to be discussed is the survey.
6.6 Surveys
MSC IN EMERGENCY PLANNING MANAGEMENT
The survey is one of the most popular research tools available for describing and explaining different aspects of social life. The first modern surveys were conducted to obtain information about living conditions and the findings were used to inform social reform programmes. Throughout the 20th Century social surveys have not only been of utility to social scientists but also various organisational groups in the commercial and public sectors as well as market researchers. This section focuses on major large-scale surveys and it is necessary to bear in mind that ‘surveys designed and implemented by the lone researcher’ are relatively rare (Cjaza and Blair, 1996: 2). Nevertheless, it is important to recognise that the same principles apply to the smaller research projects that you may undertake for your dissertations.
6.6.1 What is a Survey? Surveys are mainly used to gather information and make generalisations about the demographic and socio-economic characteristics as well as the lifestyles, attitudes and beliefs of a particular population. Fink (1995a: 1) provides an all-encompassing definition of a survey: ... a system for collecting information to describe, compare, or explain knowledge, attitudes, and behaviour. Surveys involve setting objectives for information collection, designing research, preparing a reliable and valid data collection instrument, administering and scoring the instrument, analysing the data, and reporting the results. Surveys are usually based on samples. Instead of directly studying whole populations, surveys typically collect evidence from a small sample of people selected from the population. The word ‘population’ is used in the statistical sense rather than geographical sense and can refer to any group to which we want our results to apply (Jupp, 1989). The intention is to infer that the findings and conclusions drawn from the sample are representative or typical of the wider group to which they belong. There are a number of different methods used to select a sample and they are discussed in Unit 5. There is no one ‘ideal typical’ survey. This is a reflection of two particular issues at the heart of the complexities of researching social life. In all social research, theory and method have a mutual influence on each other. For instance, in the words of de Vaus (1993: 3), ‘the distinguishing features of surveys are the form of data collection and the method of analysis’. Although various kinds of
6-22
disaster
riskcrisis MANAGE RISK
data and methods have been used in survey research – including in-depth interviews, various forms of observation and content analysis – questionnaires and standardised interview schedules are the most widely used techniques. This is in accord with the observation of Rose (1982: 10) that surveys are largely quantitative and used for ‘theory-testing’ rather than the kind of ‘theory construction’ associated with qualitative research. It should be noted, however, that this distinction is not always clear-cut and that some social scientists have used surveys in research projects which combine qualitative and quantitative data (Mason, 1994). The emphasis in this section is, nonetheless, on the latter. Second, there are other significant factors affecting the type of survey used, including (i) the nature of the topic being researched, (ii) the environment in which the research is being undertaken and (iii) the importance of the role of the researcher in obtaining the best possible results from a survey.
6.6.2 Different Types of Survey
1. Factual Surveys Factual surveys are used to gain information from individuals and organisations about knowledge they have of particular social phenomena (Fink, 1995b: 74–7). These surveys are characterised by the collection of ‘hard/quantitative’ evidence instead of ‘soft/qualitative’ information. An example is the British Crime Survey (BCS) which measures some of those crimes recorded by the police and published as official crime figures as well as those that are not (Hough and Mayhew, 1983; Mayhew et al., 1989; Mayhew et al., 1994; Mirlees-Black et al., 1996). Some critics have questioned the value of such facts and have argued that such statistics need plausible accounts to explain how they are joined together, or to use some sociological jargon, how they are socially constructed (Coleman and Moynihan, 1996: 20–1). These commentators argue that ‘factual’ findings tell us what is going on but do not tell us why it is going on. In other words, they provide essentially descriptive rather than explanatory accounts. 2. Explanatory Surveys To some extent all surveys are explanatory. They ask questions about, for example, voting behaviour and seek to explain how the attitudes or intentions of people are linked to their socio-economic background. However, these types of survey are specifically designed to test hypotheses which are derived from theories. In order to achieve this, there is an extensive use of statistical procedures to prove or disprove such theories. de Vaus (1993: 31) observes that the use of this type of survey involves the researcher having to consider the following questions:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
There are a multiplicity of approaches to social surveys employing a diversity of research techniques and strategies. These are tailored to fit the research topic, the timescale and resources available. In this sub-section you will be introduced to five broad categories of social survey.
PLANN NG
DISASTER
1. What am I trying to explain? 2. What are the possible causes? 3. Which causes will I explore? 4. What are the possible mechanisms? Many explanatory surveys have been used in an attempt to identify the attitudes of members of particular research populations.
6-23
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
3. Attitude Surveys The use of attitude surveys involves a move away from the collection of objective facts towards the acquisition of data about the attitudes of people: what people think about life in general and events in particular. The notion of ‘public opinion’ is perhaps the key to this type of survey. Societies with democratic aspirations seek to gauge the beliefs and opinions of their citizens. Often a policy is justified by the notion of ‘what the public demands’. However, how do we know what the public demands? Clearly, attitude surveys make a significant contribution towards providing this information.
MSC IN EMERGENCY PLANNING MANAGEMENT
Likert scales are used to place people on an ‘attitude continuum’. Statements are devised to measure a particular aspect in which the researcher is interested and the respondent indicates their degree of agreement on a five- or seven-point range (Bryman and Cramer, 1994: 64). An example is the semantic-differential technique, developed as a quantitative measure of meaning of subjective dimension. In this technique, people are asked to tick a box between pairs of opposite adjectives. This yields rating scales: for example, between good/bad; fast/slow; mild/strong; cool/hot etc. Other scaling methods include: the ‘Thurnstone’ scale; and Arbitrary scales (see Nachmias and Nachmias, 1976: 109–17). There are also many other types of scale (see Sudman and Bradburn, 1982: Ch. 6). 4. Social Psychological Surveys Social psychological surveys are concerned with the relationship between attitudes and beliefs and actual behaviour. The question of exactly how far attitudes correspond with behaviour is a source of contentious debate within social psychology and sociology (including various strands of criminological theory). Consequently, there has been a movement away from statistical profiles of a general population – as in factual and attitude surveys – towards an interest in the differences between individual and group behaviour. The general conclusion, though, is perhaps not surprisingly that there is ‘no simple and direct relationship between attitudes and behaviour’ (Smith, 1991: 12). 5. Hybrid Surveys The four types of survey we have discussed should not be viewed as distinct research tools that are used on their own. In practice, most researchers – when designing their research project – utilise a combination of the different types of survey in order to achieve their research aims, as well as using other research methods. The researcher should assess the aims and objectives of his or her research project and then design the most appropriate research tool possible, taking into account the resources available. For the lone researcher it is possible, for example, to begin with a mail survey and make further contacts later on by telephone. Another approach could be to do multiple interviews in households: first, initial face-to-face interviews and second, leaving self-administrated questionnaires for other household members to complete. We will now discuss the technique most widely used in conducting social surveys, questionnaires.
6.6.3 Designing a Questionnaire There is a simple truth worth remembering: a questionnaire is only as good as its design. There are three broad types of questionnaire which determine the method of design:
6-24
disaster
riskcrisis MANAGE RISK
• postal or self-completion questionnaires; • telephone survey questionnaires; and • interview schedule questionnaires. Clearly, the choice of which type of questionnaire to use will depend on the aims and objectives of the research and the time and resources available. It is worth remembering that if a researcher strives for the ‘ideal’ in their research and this involves a clash with the ‘practical’, it is invariably the case that the latter wins through. The above types of questionnaire will now be discussed in relation to their advantages and disadvantages. Postal or Self-Completion Questionnaires
The postal or self-completion questionnaire offers a relatively cheap method of data collection, in comparison with the telephone survey and personal interview. But the cost of envelopes, questionnaires and stamps, then the follow-up envelopes, letters and stamps must be considered, especially if you are a student receiving no funding for your research project. This aside, postal questionnaires can be sent to a large number of individuals across a wide area. It may also be possible to gain a response from an individual whom it would be difficult to interview because of their busy schedule for example. This approach provides the respondent with the opportunity to anonymously express beliefs, give details of activities and provide sensitive information. Questions on sensitive issues may not be answered in a face-to-face interview or over the telephone, but might in an anonymous survey. A further advantage is that respondents can take more time to consider their answers and can thus give the questions more thought and be more accurate. A further advantage of this approach is a substantial reduction in the possibility of bias that may result from the ‘interviewer effect’. Most importantly, it allows information to be compiled from uniform questions across a group, so that it can be easily analysed and legitimately compared. One of the main disadvantages of this type of approach is the almost certain low response rate. Unless individuals have an incentive, either through interest in the research subject of the survey or some other, then the response rate will be low. Obviously the lower the response rate the harder it is to draw conclusions from the results. The response may also be biased in that some groups of people are more likely to respond than others, for example, those with a low level of education or lower levels of literacy are less likely to respond than those with higher levels. In addition, respondents may provide only the minimum level of information required.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Self-administered (or self-completed) questionnaires are, as their name implies, intended for the respondents to complete themselves. Questionnaires are usually sent through the post but can be distributed by hand, to a sample of a population for respondents to fill in and return. Surveys conducted through the mail must be totally self-explanatory and avoid the use of any ambiguities. If these fundamental requirements are satisfied then such surveys have the following advantages.
PLANN NG
DISASTER
Furthermore, the researcher has no understanding of the considerations of individuals when completing the questionnaire: in other words, not being present when it is filled in means that the ‘context’ of an answer is not understood. Neither is it possible to probe answers and seek further clarification.
6-25
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Moreover, the researcher has no control over how individuals are interpreting questions. If the question is not clear it may lead to very different answers being given. Most importantly, the researcher does not even have control over who fills out the questionnaire. Although it may be addressed to a specific person, there is no guarantee that that same person actually filled out the questionnaire. Telephone Surveys Telephone surveys involve the respondent being telephoned by the researcher and verbally answering questions that are read out to him or her. Developments in computer technology have now made telephone interviewing easier. A computer assisted telephone interviewing (CATI) system is able to sample the specified population, provide guidance for the interviewer’s introduction, display the interview schedule and record the interviewees’ responses (Newell, 1993).
MSC IN EMERGENCY PLANNING MANAGEMENT
There are three main advantages in using a telephone survey. First, a wide sample can be drawn from a relatively large geographical area and surveyed (as in the case of postal surveys). Second, the response rate is usually much higher than with postal surveys as the respondent can answer the questions immediately (ranging between 60 and 90 percent compared with 40 percent for postal surveys). Moreover, it is more difficult for a respondent to say no to a voice than to a letter. There is also some evidence to suggest that some people are usually more willing to talk because the interviewer can sell the research to the respondent more convincingly over the telephone. Third, the approach enables questions to be clarified and answers to be probed and expanded. There are also a number of disadvantages with using telephone surveys. They are potentially expensive. Telephone calls may last for a long time and prove particularly expensive if the calls are national or international rather than local and regional. A number of the advantages of the postal survey are lost, such as anonymity and time to consider answers, and there is a danger that a limited interviewer effect might develop. Certain groups such as the poor, the young, the sick and disabled, and those who are frequently away from the telephone may be under-represented (Newell, 1993). Telephone surveys are not amenable to open-ended questions and this line of questioning produces longer answers. The respondent may also be subjected to questions in an environment in which they feel that they cannot speak freely or may be easily distracted. Interview Schedules You will have already been introduced to the importance of interview schedules in Section 6.4. Using this method, the researcher or a team of researchers administers the questionnaire. It enables the interviewer, if required, to record the context of the interview and non-verbal gestures. Unlike the methods discussed above, there is an interactional element between the interviewer and interviewee. There are distinct advantages in using the interview schedule approach. In comparison to the other survey methods there is a possibility of gaining a high response rate. The interview allows questions to be explained, answers to be probed and the context of the answer to be explored. The body language of the interviewee and interview context can be assessed. It is possible for the interviewer to develop more of a rapport with the respondent than is feasible over the phone. The construction of a more complex questionnaire is possible, including the use of visual aids.
6-26
disaster
riskcrisis MANAGE RISK
The main disadvantage of the interview schedule approach is that only a limited sample can be interviewed for there may be constraints on where the interviewers can travel and time and resources available are likely to mean fewer people can be interviewed. The interview schedule approach is also the most expensive form of survey. Some individuals may be less willing to give an interview than reply to a postal questionnaire. If additional researchers are being employed there is also the potential of the ‘interviewer effect’ yielding a different quality of data from different interviewers. Furthermore, there is a tendency for respondents to over-report desirable social attitudes and behaviour (Sapsford and Jupp, 1996: 5).
PLANN NG
DISASTER
EMERGENCY
6.6.4 Creating Questions
1. Target Questions for Respondents The first consideration is whether the respondent has the knowledge to answer a question. It would be wrong to assume that they will, and if they do not, you will be wasting your time devising the questions and the respondent’s time trying to answer them. The second consideration is wording questions in the most appropriate way for the respondent. Cjaza and Blair (1996: 56) argue that it is necessary to avoid: ... cling[ing] tightly to the language of our [as academic researchers] hypotheses, constructs or research concepts that few people other than experts can understand. The questioning should be oriented to the lowest common denominator, that is, what the least educated respondent would be likely to understand. 2. Question Wording Questions should be specific and related to the answers that are required. Avoid using vague questions that can be interpreted in very different ways. Vague words such as occasionally, regularly, frequently should all be avoided. And if they are used they should be defined, for instance occasionally, once a week; regularly, three times a week, etc.
MSC IN EMERGENCY PLANNING MANAGEMENT
The most important part of the actual design of a questionnaire is to present unambiguous questions. It is necessary to provide concrete questions which specify the exact nature of the information that is required. A survey can be ruined if the questions are not clear. Also slight changes in wording can produce very different answers. To reiterate an earlier point: a survey – no matter how good the sample and the response rate – is only as good as the questionnaire. The following are pointers that should be borne in mind when designing a questionnaire. They refer mainly to postal questionnaires, although in many cases they apply equally to telephone surveys and interview schedules.
3. Avoid Double-Barrelled Questions Double-barrelled questions include two or more questions in one. The problem with such a question is that it may confuse the respondent. They may only answer one of the questions or have different answers for both. It is best to divide double-barrelled questions into two separate questions.
6-27
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
4. Avoid Leading Questions The term ‘leading question’ refers to a question phrased in such a manner that it appears to the respondent that the researcher expects a certain answer. Nachmias and Nachmias (1981) explain how the term social desirability refers to the tendency of respondents to agree with questions that support accepted norms or provide answers that are perceived as socially desirable. Questions that reflect a socially undesirable behaviour or attitude are endorsed less frequently than those high on the scale of social desirability. Emotionally loaded questions should also be avoided, as they tend to make respondents react not so much to the issue posed by the question but to the loaded phrase itself. For example, using words such as: Conservative, Labour, Socialist, Communist, etc. could lead to answers based on the respondent’s political views and not their objective consideration of the question. 5. Avoid Presuming Questions
MSC IN EMERGENCY PLANNING MANAGEMENT
Another consideration is, do not presume the respondent can answer the question. For instance, do not presume that they have or have not done something. If necessary, ask two questions. The first question should be used to ascertain what the respondent has or has not done, the second to ask them about it. 6. Avoid Hypothetical Questions Questions that seek to ask about future behaviour should also be avoided, as individuals are generally not good at predicting their future behaviour (Moser and Kalton, 1971: 326). What people say and subsequently do is often very different. 7. Avoid Embarrassing Questions Questions which are embarrassing should also be avoided wherever possible, as asking questions about personal behaviour and sensitive information may lead to a substantial drop in the response rate (Miller, 1983: 111). However, various methods can be used to address this problem if the questions are essential: first, offer increased anonymity and confidentiality such as a reminder or offer a self-completion component; second, normalize the embarrassing behaviour (‘most people at one time or another have…’); third, destigmatise the behaviour by avoiding language that elicits social desirability (using the word ‘take’ instead of ‘steal’, for example); finally, you can consider addressing the question in the third person. 8. Closed or Open Questions Closed questions limit the number of possible answers to be given and are generally coded to allow ease of analysis. The advantages are that they take up less time and they allow comparability between answers. However, they also compartmentalise respondents and are problematic if respondents have not thought about the issue (if it is assumed that they have). The researcher has to be sure that all potential answers are listed; otherwise the results will be inaccurate or lead the respondent to a particular set of answers. With other questions it might not be possible to break the answer into yes or no alternatives especially when many may have complex views on a subject. Open questions give the respondent greater freedom to answer, because they reply in a way that suits their frame of reference. However, the purpose and wording of a question have to be very clear, otherwise it might lead to wide variations in answers. Open questions are a useful ‘follow-
6-28
disaster
riskcrisis MANAGE RISK
up’ to closed questions and when situations to which responses are being measured are changing quickly: Finally, as survey responses are increasingly used as a basis for historical research, open responses have the value of enabling researchers to explore raw data and to devise new coding categories. (SCPR Newsletter, Autumn 1981: 7) 9. Order of Questions There is evidence to suggest that the order of questions may affect the response rate (Moser and Kalton, 1971: 346). The questionnaire should start with questions that the respondent will be interested in and will find no difficulty answering. The questions should proceed logically and variations in questions should be explained. Any sensitive questions should be left until last. Demographic questions about the respondent’s background should also be left until the end because they may cast doubts in their minds about the core purposes of a piece of research.
To keep the respondent interested, question filters should also be used wherever possible. Thus if there is a series of questions that are relevant to a respondent who answers yes or no to a question, then for those for whom the question is not relevant there should be a filter, for example ‘if the answer is no please move on to question y’. The questionnaire should also be designed so that it is not possible to take short-cuts. Respondents may answer a particular way to avoid answering lots of questions. If the questionnaire first lists the alternative answers and then asks for an explanation for the answer after all the selections have been made, the short-cut is made more difficult.
6.6.5 Presentation of Questionnaire It goes without saying that the presentation of a questionnaire is very important – although this is less important with interview and telephone questionnaires. The respondent’s views on the survey in a postal questionnaire will be almost totally formed by the questionnaire and covering letter he or she sees, which should be: ... eye-catching (yet professional), clear (but brief), and compelling (but neutral). The letter must stand out from the welter of junk mail most people receive and must speak for the researcher to the respondent, addressing the key obstacles to co-operation.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The sequence of questions, depending on subject, should follow the ‘funnel sequence’ or ‘inverted funnel sequence’ (Nachmias and Nachmias, 1976: 106). Generally, they should follow the former, which means that questions should start broadly and then gradually narrow with each question being related to the previous one. However, when the subject is likely to arouse no interest in the respondent, starting with a broad question might lead to no response. In this case, starting narrowly and gradually broadening out to the wider questions might be more appropriate.
PLANN NG
DISASTER
(Cjaza and Blair, 1996: 82) The letter should be brief and should explain the purpose of the questionnaire; the aims of the research; what the respondent should do; the importance of the subject being studied and the need for an answer. If sensitive issues are being tackled, there should also be a statement ensuring confidentiality and anonymity.
6-29
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
In terms of the questionnaire it is essential that it is attractive, that the size of print is readable, that there is ample space for answers and there are no unintended marks on the questionnaire. Indeed, research has found that an aesthetically pleasing cover, with a title that arouses interest, attractive page layout and with a size and style of type that is easily read under poor illumination, may increase the response rate (Miller, 1983: 112–15).
6.6.6 Piloting
MSC IN EMERGENCY PLANNING MANAGEMENT
Piloting is essentially concerned with identifying any deficiencies in the design of a questionnaire such as problems in the administration of the questionnaire or the changes in the phrasing and sequence of questions. Large-scale research projects often undergo pre-testing. However, it is not practicable in terms of time and money for a lone researcher who is limited to relatively small-scale studies. It is therefore advisable to ‘stick to well-tested field and sampling procedures’ (Cjaza and Blair, 1996: 105). Conducting a literature review would be helpful to see how previous researchers have collected data, obtained co-operation and utilised sound interviewing techniques. To avoid potential problems with the design of a questionnaire, it is essential that the questionnaire be piloted on a sample that will be similar to the respondents of the main survey.
6.6.7 Maximising Response A good questionnaire design will no doubt affect the response rate but there are a number of additional strategies that can be used to increase the likelihood of response. One of the most common methods is to send a reminder to the respondent a week or so after the questionnaire was posted. Alternatively, the researcher could telephone the non-respondents and ask if they have received a questionnaire. The researcher could then, after another period of time has elapsed, send another questionnaire and letter. If there are sufficient resources, the last resort could be to send yet another questionnaire by registered mail. Research has found that the response rate increases on average by 20 percent after the first follow-up, another 12 percent following the second and a further 10 percent following the third (Miller, 1983: 110). Inducements have also been used to increase the response by up to 40 percent (Miller, 1983: 112), although the inducement offered should be suitable for the type of respondent. Nevertheless, an offer of a financial reward, for example, implies losing anonymity for the respondent and this must also be borne in mind when considering it as an option. A more common incentive is to supply a stamped addressed envelope, but this may be costly and is likely to be a very small incentive to a major organisation or a senior official on a large salary. Combined with the development of more sophisticated methodologies and techniques, social surveys have proved to be a fairly accurate and reliable source of information to inform social action. However, it is important to note that a survey should be manageable, especially given the innumerable restrictions and limitations imposed on a research project undertaken by a single person. Overall, the above four sections will have provided you with a basis to decide what is an appropriate tool with which to pursue your research aims when conducting primary data gathering. You may, however, decide to focus on collecting data from secondary, rather than primary sources or use a combination of approaches. The next three sections will introduce you to secondary gathering techniques discussing documentary research, secondary analysis and content analysis.
6-30
disaster
riskcrisis MANAGE RISK
6.7 Documentary Research Documents are often neglected as a primary research tool. They are frequently only used in order to support other methods of research such as surveys, interviews and ethnographic studies as a point of reference. This section will be of use not only for those using documents as the primary basis of research, but also for those using them to support other research methods.
6.7.1 What is Documentary Research?
In a sense, primary documents are the substance of history. They tell us about human aspirations and intentions and record the details of events that are beyond our understanding because we did not have firsthand experience of the events which they detail. They are the only source of evidence about what happened several generations ago. Therefore, whether we consider documents a valid means of social research or not, they provide a wealth of information and facts about the decisions which people have made on a routine, day-to-day basis in the context of broad developments in history. Importantly, they offer a version of how past events unfolded or occurred. In this sense, history and our understanding of it can be informed by reading documents or legacies of the past. However, because it is not possible or even desirable to record everything, these documents themselves may be based on the use of selective evidence. Thus, what people decide to record, to leave in or take out, is itself informed by decisions, which also relate to the social, political and economic environment of which they are a part (Carr, 1961). History, like all social and natural sciences, is amenable to manipulation and selective influence. In undertaking documentary research, we should be aware of these influences and not assume that documents are simply neutral artifacts from the past. In Finnegan’s (1996: 143) words:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Documentary research is, perhaps, one of the least-explained techniques in social research, and is generally regarded as less important than other more conventional sources of data, such as questionnaires and interviews. This is quite surprising, however, especially if one takes on board that the founders of modern social science, Durkheim (1951), Marx and Weber (1930), drew extensively on documentary sources in their seminal works. Documents, consisting of official records (governmental and non-governmental), private papers and statistical data, are an important tool for enhancing our understanding of the social world. Documentary evidence shares one thing in common, namely that it refers to past human behaviour. There are, for example, two main categories of documentary sources: primary and secondary. The distinction, although not always maintained, refers to the ‘“contemporaneity” of the source and closeness to the origin of the data’ (Finnegan, 1996: 142). Primary sources are produced close to the specific period being investigated. Secondary sources are produced at a later date. Moreover, ‘secondary sources copy, interpret or judge material to be found in primary sources’ (Finnegan, 1996: 141).
PLANN NG
DISASTER
... all sources do not just arise automatically through some natural process [but are] the results of human activity. They are produced by human beings acting in particular circumstances and within the constraints of particular social, historical or administrative conditions.
6-31
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
6.7.2 What are Documents? Documents are the accounts, returns, statutes and proclamations that individuals and groups produce in the course of their everyday practice and that are geared to their immediate and practical needs. (Scott, 1990: 12)
MSC IN EMERGENCY PLANNING MANAGEMENT
There are multiplicities of sources of information that can reasonably be regarded as documents. It would be wrong to limit the scope of documents available to the researcher to the stereotypical view of an archive of formal historical texts such as Green Papers, White Papers and Acts of Parliament and communications between heads of state, as often used by the historian. Jupp (1996: 209) makes a distinction between a ‘document’ and a ‘text’. The former is ‘the medium on which the message is stored’. The latter is ‘the message that is conveyed through the symbols which constitute writing’. Documents therefore include conventionally defined texts, but there are various other sources such as company accounts and reports, newspaper articles, autobiographies and letters. Documents may even include items such as photographs, maps, charts, microfilm or microfiche, audio-visual and electronic forms and even gravestones. Whether a text is written on paper, on stone or recorded on a magnetic disk does not matter; as long as the text is the primary purpose of its being and it contributes to the research one is trying to undertake, it may be classed as a document. There have been a number of attempts by authors to classify different types of documents. For instance, McNeil (1985: 109–12) – like many writers – distinguishes between personal and public documents. The former are items such as letters and diaries, and the latter official records such as Acts of Parliament and reports commissioned by the government. The main difference between personal and public documents is that the latter are produced by or for a public body, such as a government department or company, and the former are written by private individuals, often with no intention of publishing them, although key players in history often do. A more recent classification is Scott’s (1990) authorship classification: ‘personal’, ‘official private’ and ‘official state’. Here the ‘personal’ includes life histories, biographies, diaries, oral histories, letters, literature of fact and visual data and various other sources. ‘Official private’ documents comprise official information about private organisations. ‘Official state’ documents include a host of government publications at local, regional, national and supranational levels. When doing documentary research, as with other data gathering techniques, there are potential pitfalls. First, there is the difficulty of getting access to some kinds of data. Second, there is the perennial problem of the time and resources available. Further problems appear once a researcher has a document in their hands: its authenticity; credibility; representativeness and meaning. The main guideline is that documents and the phenomena they describe are multifaceted and should be examined by recognising their inherent complexity.
6.7.3 Methodological Issues Finnegan (1996: 146–9) raises eight methodological issues that need to be considered by a researcher consulting documentary sources. 1. Has the researcher made use of the existing sources relevant and appropriate for his or her research topic?
6-32
disaster
riskcrisis MANAGE RISK
This is an important issue because the conclusions a researcher reaches are often criticised if relevant sources have not been consulted. That is why the literature search or literature review is so important. If this is done thoroughly there should be no important unidentified sources. However, it is necessary to bear in mind financial constraints and time limits because it is not possible to cover everything. An ability to identify the most relevant literature in a particular field of enquiry is therefore of the utmost importance. 2. How far has the researcher taken account of any ‘twisting’ or selection of the facts in the source used?
3. What kind of selection has the researcher made in her or his use of the sources and on what principles? The range of material that is available in a particular field of enquiry may be huge, and given the constraints of time and resources it will not be possible to look at everything. It will therefore be necessary to select certain documents and ignore others. Care must be taken to ensure that the selection of particular sources is ‘fair’ and ‘representative’ and can be justified as such while at the same time remembering that these criteria are, like the ‘truth’, politically contested. On a methodological level it may be possible to triangulate different types of document to see if there are any similarities. The use of sampling techniques may also be required. 4. How far does a source, which describes a particular incident or case, reflect the general situation? It is important to consider the extent to which a research finding can be considered typical. Certainly do not claim a finding to be typical when this may not be the case. For example, if media evidence is being used there is a tendency for such accounts to focus on the unusual and the novel rather than the common and mundane. This is not necessarily a problem, as long as this is explicitly recognised and the findings are not promoted as something which they are not. 5. Is the source concerned with recommendations, ideals or what ought to be done? In most cases it is a relatively straightforward exercise to distinguish each of these from the others, especially in the case of explicitly normative statements. However, when it comes to policy statements, distinguishing between rhetoric and reality is less clear-cut.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
It is necessary to remember that all sources of data, not necessarily intentionally, reflect the social values of the individual or group producing particular documents. The social and political contexts in which individuals and collectivities work are important and exert a great deal of influence. However, do not under any circumstances deliberately falsify evidence or use bogus data. An additional problem is that in the social sciences, in particular those kinds favouring certain qualitative methodological approaches, the ‘truth’ is to some extent relative and negotiable rather than absolute.
PLANN NG
DISASTER
6. How relevant is the context of the source? The economic, social and political context in which a document is produced is very important and it is essential that the researcher is aware of this background. What was the author trying to say? Are the statements typical of a particular time and place or were they groundbreaking?
6-33
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 6-34
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
7. With statistical sources: what were the assumptions according to which the statistics were collected and presented? Statistics are socially constructed in the same way as documents. Take the different assumptions underlying the official crime figures and those compiled by the British Crime Survey (BCS) and their respective implications for understanding the reality of crime. Finnegan (1996: 149) observes: In statistics referring to ‘the crime rate’, for example, what is the category ‘crime’ assumed to mean? If it is ‘crimes known to the police’, that will result in one figure; if ‘people prosecuted’, in another (lesser) figure; if ‘persons found guilty’, yet another (lesser still). For this reason, always look carefully at the title of tables and figures cited by researchers. 8. And, finally, having taken all the previous factors into account, do you consider that the researcher has reached a reasonable interpretation of the meaning of the sources? In other words, how convincing is a researcher’s interpretation of the data? This is a moot point and can lead to a great deal of conflict between the competing arguments and interpretations of others. What one person may deem to be important is of no significance to another. This is an inherently political issue. Finnegan makes a further methodological observation about the status of documentary sources by drawing our attention to the distinction between evidence that is gathered purposefully or without purpose, or in the words of the social historian Arthur Marwick, evidence that is gathered ‘wittingly’ or ‘unwittingly’ (Marwick, 1977, cited in Finnegan, 1996: 150). It is also necessary to take into consideration whether the evidence gleaned from research is a reflection of preconceived ideas or whether the theory is developed out of the data. In addition, the researcher should be constantly inquisitive about the origins of a particular document. What does it tell us about the individual or the organisation that created it and the wider social and political context in which it was socially constructed? This section has examined the many issues that need to be considered when using documents for research. You will be aware that there are a wide variety of documents available to the social researcher. Documents are not free from subjective bias, thus each document must be assessed on its own merits and weight attached to it accordingly. Similar considerations must also be borne in mind when conducting secondary analysis.
6.8 Secondary Analysis It is a truism of social research that almost all data are seriously under-analysed: unless the data collection is tightly designed to test a specific hypothesis the original researcher will explore only a fraction of its potential (Procter, 1993). In ethnography it is difficult for further analysis to be conducted because it is extremely hard for another researcher to understand the complexities of the study. Survey data, however, are rather easier for an adoptive researcher to understand almost as well as the original researcher because much survey research is in any case planned and executed in the first place by several researchers. As a consequence secondary analysis of survey data is fast growing in importance (Procter, 1993).
disaster
riskcrisis MANAGE RISK
Most social scientists, when contemplating the initiation of a research project, will automatically think in terms of collecting new data. Few will think of re-analysing existing datasets. Researchers like to think that their data for a research project is original, and hence to assume that relevant data could not yet have been collected by anyone else. But original research can often be done with old data. (Hakim, 1982: 1)
6.8.1 Data Sources An important source of high quality data often used in secondary analysis is government work. A selection of some of the research undertaken by public and private bodies is listed in Box 6.1. A fuller listing of the variety of research available can be found in the annual Social Trends produced by the Central Statistical Office, while The Directory of British Official Publications by Richard (1992) and the Home Office Statistical Bulletin (Home Office, 15 December 1992) list all the statistics collected by government on the criminal justice system.
Author Publication British Retail Consortium
Retail Crime Costs Survey
Employment Department
Labour Force Survey
Employment Department
New Earnings Survey
Employment Department
Unemployment Statistics
Employment Department
Workplace Industrial Relations Survey
Home Office
Criminal Statistics for England and Wales
Home Office
Probation Statistics
Home Office
Statistical Bulletins
Home Office
The British Crime Survey
Jordans and Son
Britain’s Security Industry
OPCS
Census
OPCS
General Household Survey
OPCS
The National Prison Survey
Scottish Office
Criminal Statistics
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Box 6.1: Selected Research Available for Secondary Analysis
PLANN NG
DISASTER
Note: OPCS is the Office of Population Censuses and Surveys
The advance of technology has led to a multiplicity of on-line databases, some of which may be useful to the researcher. POLIS is of use for finding information on the business of Parliament. Some databases may also be accessed through the Internet, such as DATA-STAR which lists all government press releases. Such databases are also usually constantly updated, thus providing greater accuracy. It is important, nevertheless, to be cautious when considering the use of such tools.
6-35
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 6-36
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
1. The database may not have all the information and will, therefore, have to be supplemented. For instance, if it is a database of limited companies and one is searching for security companies, the sample could not be claimed to be representative of the wider industry, as there are many security firms that are not limited companies. 2. Some databases are easy to use, while others are very complex and may require training. A great deal of data are also collected by academic social scientists. Procter (1993) emphasises how there would be very little point in doing this kind of work without the assumption that the findings would be communicated, mainly through the medium of books, articles in academic journals, and conference papers. Procter (1993) explains how this idea has now extended, at least in the case of survey research, to making raw data available to other competent researchers, and it is usually deposited at the national data archive. He therefore suggests that when you read an account of survey based research and wonder why the researcher did not analyse it differently, it may be possible for you to contact the archive and see if the data are available for reanalysis.
6.8.2 Conceptualising in Secondary Analysis The real challenge in secondary analysis lies in finding ways of forcing the data, collected by someone else, quite often with entirely different theoretical and analytical intentions, to answer your questions (Procter, 1993). According to Stewart (1984, cited in Procter, 1993: 262) a secondary analyst should try to get answers to the following questions about data: 1. What was the purpose of the original study? Was it designed around a specific hypothesis which limits its usefulness for a researcher with different ideas? 2. What information was collected? Were key variables defined in such a way as to be compatible with the new analyst’s ideas? 3. What was the sample design and what sample was actually achieved? Is it adequately representative of those groups that play a key part in the new researcher’s theory? 4. What are the credentials of the data? Can the original researchers be assumed to be competent? What evidence is available about the reliability and validity of the data? 5. When were they collected? Is topicality important, or is it reasonable to assume that the relationships being studied are relatively invariant across time? 6. Are the data nationally representative? Is this crucial, or is it reasonable to assume, as under the last point, that the relationships hold across a wide range of geographical localities?
disaster
riskcrisis MANAGE RISK
6.8.3 Advantages and Disadvantages of Secondary Analysis Procter (1993: 256–8) outlines the pros and cons associated with secondary analysis. Advantages Cost The most obvious advantage of secondary analysis is cost. For some researchers there is no possibility of obtaining funding at the level required for survey work and secondary analysis provides an alternative. Time
considerations also apply to student researchers who have, in addition, externally imposed deadlines to cope with. However, the process can still take many months and it is important to discuss the pitfalls of this approach with a more experienced adviser before you commit to this form of research. Quality The quality of data obtained in this way is likely to be higher than a relatively inexperienced researcher can hope to obtain unaided. Government survey organisations, in particular, have great expertise in their chosen method, and because their clients are usually government departments they can often bring to bear the political pressure that is needed to legitimise the relatively expensive technology they employ: national representative probability samples, highly trained and closely supervised interviewers, the necessary sequence of pilot studies, etc. Reaching Inaccessible Populations Even though the original purpose of a survey may not have been connected with the particular subgroup you wish to focus on, sample sizes are often so large that otherwise inaccessible populations can be reached, especially if it is legitimate to merge successive years’ samples. Disadvantages Relevance It is important to ensure that the questions asked in the data are relevant to your research problem. There is always a temptation for the subject of research to be determined by what is convenient rather than by what is scientifically important, and the balance of effort between data collection and data analysis makes this particularly dangerous in planning secondary analysis.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
In using secondary analysis a great deal of time can be saved. If the data already exist the new analysts can get to work almost as soon as they think of the idea, instead of having to face years of preparation and fieldwork if they are responsible for their own data collection. Secondary analysis is especially useful to academic researchers of whom the majority are also employed as teachers who often find it difficult to find the time to run their own project from start to finish. Similar
PLANN NG
DISASTER
Sample Design There can be pitfalls in the sample design, into which one would be unlikely to fall if planning the project from scratch. To cite a classic example, the UK Family Expenditure Survey, which is a
6-37
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
survey of private households and thus excludes people living in pubs and bars, has proved to be entirely inappropriate as a source of information about alcohol assumption. Time Span It is often unrealistic for a researcher, especially an inexperienced one, to plan to collect data over a period of years, if only because one’s career requires that one ‘publish or perish’ over a relatively short time span. And yet data collected over an extended period of time are often precisely what is required by the logic of the research problem. The solution may be to make use of an existing longitudinal data set, so that the waiting period has already taken place when the researcher gets to work. Ethical Issues
MSC IN EMERGENCY PLANNING MANAGEMENT
Secondary analysts might think that, because ethical issues mainly concern data collection and the intrusive nature of research on individuals, they are exempt from such concerns. Ethical issues are discussed in detail in Unit 2 but it is worth stressing the importance of continuously maintaining anonymity. This is perhaps even more important than in the primary analysis, because the respondents are not in general aware of the uses to which their answers will be put in subsequent work. Secondary analysis is an alternative to collecting your own data. It is especially useful when time and resources are limited. Content analysis also involves using existing forms of research to carry out worthwhile study and will now be addressed.
6.9 Content Analysis 6.9.1 What is Content Analysis? Maxfield and Babbie (2001: 329) define content analysis: Content analysis involves the systematic study of messages. Dixon et al. (1987: 95) observe that: Content analysis is very much like an observation study. In a content analysis a check-list is developed to count how frequently certain ideas, words, phrases, images or scenes appear. It is like an observation study, but what is being observed is a text, or a film or television programme. Content analysis is a method of data analysis as well as form of observation. Instead of observing people’s behaviour directly, or asking them about it, the researcher takes the communications that people have produced and asks questions of the communication (Nachmias and Nachmias, 1981). Content analysis can document in quantitative terms a theory you may have based on unsystematic observation. Content analysis allows a researcher to reveal the content (i.e. messages, meanings, symbols) in a source of communication (i.e. a book, article, movie) which may otherwise be difficult to see (Neuman, 1997). Neuman (1997) explains how content analysis is useful for three types of research problems. First, it is helpful for problems involving a large volume of text (e.g. years of newspaper articles) with sampling and multiple coders. Second, it is helpful when a topic must be studied ‘at a distance’, for
6-38
disaster
riskcrisis MANAGE RISK
example, when studying historical documents, the writings of someone who has died or broadcasts in a hostile foreign country. Finally, content analysis can reveal messages that are difficult to see with casual observation. The creator of the text or those who read it may not be aware of all its themes, biases or characteristics. Nachmias and Nachmias (1981) explain how although content analysis is always performed on a message, it may also be used to answer questions about other elements of communication. Lasswell (1965, cited in Nachmias and Nachmias, 1981) formulated the basic question that can be raised by researchers: ‘who says what, to whom, how and with what effect?’ More explicitly, Nachmias and Nachmias (1981) explain how a researcher may analyse messages to test hypotheses about: • Characteristics of the text. The most frequent application of content analysis has been to describe the attributes of the message. • Antecedents of the message. A text is analysed in order to make inferences about the sender of the message and about causes of the message. recipients.
6.9.2 Carrying out Content Analysis The content analysis procedure involves the interaction of two processes: specification of the content characteristics to be measured, and application of the rules for identifying and recording the characteristics when they appear in the texts to be analysed (Nachmias and Nachmias, 1981). First you must decide what it is you wish to measure through content analysis. For example, Neuman (1997) explains how a researcher may want to determine how frequently television dramas portray elderly characters in terms of negative stereotypes. You will then have to develop a measure of the construct, which in Neuman’s example would be ‘negative stereotypes of the elderly’. Constructs in content analysis are then operationalised with a coding system, which is a set of instructions or rules on how to systematically observe and record content from text. Coding represents the measurement process in content analysis (Maxfield and Babbie, 2001). A researcher will tailor the coding system to the type of text or means of communication being studied. The form the coding system will take will also depend on the researcher’s unit of analysis. Nachmias and Nachmias (1981) distinguish between recording units and context units. The recording unit is the smallest body of content in which the appearance of a reference is counted (a reference is a single occurrence of the content element). The context unit is the largest body of content that may be examined in characterising a recording unit. For example, the recording unit may be a single term; but in order to note whether the term is treated favourably, one has to consider the entire sentence in which the term appears (the context unit). Thus the sentence is taken into account when recording (and subsequently when coding) the term.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• Effects of the communication. Inferences are made about the effects of the message on
PLANN NG
DISASTER
There are five major recording units frequently used in content analysis (Nachmias and Nachmias, 1981): 1. Words or Terms The word is the smallest unit generally applied in research. Its application results in a list of frequencies of selected words or terms. Computer programs are often used for this purpose. They allow individuals
6-39
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
to search for individual words or phrases; establish the frequency of occurrence of words; isolate ‘chunks’ of texts which contain key words; or even analyse correlations between codes or categories of behaviour. More recently the NUDIST (Non-Numerical Unstructured Data Indexing, Searching and Theorising) package has been introduced on to the market (Richards and Richards, 1994). 2. Themes In its simplest form a theme is a simple sentence; that is, subject and predicate. Because in most texts themes can be found in clauses, paragraphs and illustrations, it becomes necessary to specify which of these places will be searched when using the theme as a recording unit. For example, one may consider only the primary theme in each paragraph or count every theme in the text. Themes are most frequently employed in the study of propaganda, attitudes, images and values. 3. Characters In this case the researcher counts the number of persons rather than the number of words or themes. This in turn, enables the examination of traits of characters appearing in various texts.
MSC IN EMERGENCY PLANNING MANAGEMENT
4. Paragraph The paragraph is rarely used as a recording unit because of difficulties in classifying and coding the various and numerous things implied in a single paragraph. 5. Items The item is the whole unit employed by the producer of a message. The item may be an entire article, a book, a speech, or the like. Analysis by the item is appropriate whenever the variations within the item are small and insignificant. For example, news stories can often be classified by subject matter such as crime or sport. Recording units are then classified and coded into categories. Nachmias and Nachmias (1981) explain how categories must relate to the research purpose, and they must be exhaustive and mutually exclusive. Exhaustiveness ensures that every recording unit relevant to the study can be classified. Mutual exclusivity means that no recording unit can be included more than once within any given category-system. The researcher also has to specify explicitly the indicators that determine which recording units fall into each category. It is important that you carefully design and document procedures for coding in order to make replication possible and improve reliability. Researchers often conduct a pilot study and refine coding on the basis of it. There are many ways of quantifying the data in content analysis. Neuman (1997) explains how coding systems identify one or more of four characteristics of text context: frequency, direction, intensity and space. A researcher measures from one to all four in a content analysis research project. 1. Frequency Simply means counting whether or not something occurs and, if it occurs, how often. For example, how many elderly people appear on a television programme within a given week? What percentage of all characters are they, or in what percentage of programmes do they appear?
6-40
disaster
riskcrisis MANAGE RISK
2. Direction Direction is noting the direction of messages in the content along some continuum (e.g. positive or negative, supporting or opposed). For example, a researcher devises a list of ways an elderly television character can act. Some are positive (e.g. friendly, wise, considerate) and some are negative (e.g. nasty, dull, selfish). 3. Intensity Intensity is the strength or power of a message in a direction. For example, the characteristic of forgetfulness can be minor (e.g. not remembering to take one’s keys when leaving home, taking time to recall the name of someone one has not seen in years) or major (e.g. not remembering one’s own name, not recognising one’s children). 4. Space
As in other research methods the researcher faces a fundamental choice between depth and specificity of understanding when measuring a phenomenon. A survey researcher must choose between open or closed questioning for example. The content analyst has a choice of searching for manifest or latent content. Coding the visible surface content in a text is called manifest coding and more closely approximates the use of closed-ended items in a survey questionnaire (Maxfield and Babbie, 2001). It can involve, for example, counting the number of times a phrase or word appears in a written text or whether a specific action appears in a movie scene or a photograph. The coding system lists terms or actions which are located in the text. As discussed earlier, a computer can be utilised for this purpose. Neuman (1997) highlights how manifest coding is highly reliable because the phrase or word is or is not present. However, manifest coding does not take of account the connotations of words or phrases. Neuman (1997) explains how the same word can take on different meanings depending on the context. Thus the possibility that there are multiple meanings of a word limits the measurement validity of manifest coding. In the most general sense, manifest and latent coding can be distinguished by the degree of interpretation required in measurement (Maxfield and Babbie, 2001). When using latent coding the researcher looks for the underlying meaning in the content of a text (Maxfield and Babbie, 2001). For example, a researcher reads an entire paragraph and decides whether it contains particular themes or moods. The researcher will possess, in the coding system, general rules to guide his or her interpretation of the text and for determining whether particular themes or moods are present. In terms of reliability and validity, Neuman (1997) explains how latent coding tends to be less reliable than manifest coding for it depends on a coder’s knowledge of language and social meaning. Training and practice, and written rules all improve reliability but it is still difficult to consistently identify themes, moods, and the like. However, Neuman explains how in terms of validity latent coding can exceed that of manifest coding because people communicate meaning in many implicit ways that depend on context, not just specific words.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
A researcher can record the size of a text message or the amount of space or volume allocated to it. Space in written text is measured by counting words, sentences, paragraphs or space on a page (e.g. square inches). For video or audio text, space can be measured by the amount of time allocated. For example, a character may be present for a few seconds or continuously in every scene of a two-hour programme.
PLANN NG
DISASTER
6-41
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Researchers can use a combination of manifest and latent coding and often, if the two approaches produce similar outcomes, the results are strengthened. If different results are produced Neuman (1997) suggests the researcher may want to re-examine the operational and theoretical definitions. Maxfield and Babbie (2001) explain how reliability in content analysis can be tested, if not guaranteed, in two related ways. First inter-rater reliability can be determined by having two different people code the same message and then computing the proportion of items coding the same. The second technique is the test-retest method, in which one person codes the same message twice with time lapsing between the two coding operations.
EMERGENCY
6.10 Conclusion Singleton and Straits (1999) highlight how available data research, including the secondary analysis of existing survey data, is currently the most popular method of social research. Compared to other research strategies, it is best suited to the analysis of social structural variables, to crosscultural research and to studies of the past and to social change. The research methods that you
MSC IN EMERGENCY PLANNING MANAGEMENT
decide to use for your own research project will of course depend on what you are going to research and the resources you have. This Unit has provided you with an in-depth understanding of different ways to collect data, how to use specific data gathering techniques and their associated advantages and disadvantages. You will have become aware that there are four basic choices to data collection: observation, interviews, survey and use of available data. Since the general approach to data collection affects decisions about measurement and sampling, the approach to be taken is determined early in the research design phase. Specifically, you will have to decide whether to collect primary or secondary data, whether it will be qualitative or quantitative, or whether you will use a combination of approaches. Multiple method studies are often very desirable and can strengthen the reliability and validity of your findings. For example, combining attitude measures with say direct observation, can be used to confirm that people actually do what they say they do (Fielding, 1993). The use of multiple data gathering techniques can also reduce the impact of the researcher; for example, supplementing interviews with a self-completion survey can help reduce interviewer bias. Overall, the Unit will have given you the basis to decide what is an appropriate tool with which to pursue your research aims and the limitations and the problems of the tool that must be borne in mind while undertaking such research. Good research involves a sound choice regarding appropriate data gathering techniques, the careful design and piloting of measurement instruments or strategies, and the systematic collection of data.
6.11 Main Points • Data gathering techniques can be divided into those that collect primary data and those that collect secondary data. • Data gathering techniques are accompanied by measurement strategies or instruments which should be carefully designed and piloted, taking into account not only the aims and objectives of research but also the sample at hand and ethical considerations.
6-42
disaster
riskcrisis MANAGE RISK
• Observation is the most basic of data gathering techniques and is at the heart of virtually every other technique. Observation lets us watch what others say or do. • Ethnography is a qualitative research method practiced through participant observation in a social grouping. • Interviews can be one-on-one or in a group; interviews vary in their degree of structure. Interviews let us question people on knowledge, attitudes, opinions and motivations for behaviour. • Life history research is both a method of data collection and a source of data. Life histories enable us to understand people’s beliefs, attitudes and behaviour over time, associate individual choices with historical processes and immerse ourselves in the research subject’s point of view. • Surveys are one of the most popular data gathering techniques. Surveys can be carried out by post, telephone, or face-to-face. to question wording, question sequence, presentation and layout, and introductory and ethical statements. • Documentary research is often overlooked as a potential data gathering technique. • Most data collected is not fully exploited in analysis; consequently, secondary data analysis provides the opportunity to do original research with existing data. • Content analysis involves the systematic study of messages. • Good research involves a sound choice regarding appropriate data gathering techniques, the careful design and piloting of measurement instruments or strategies, and the systematic collection of data.
6.12 Guide to Reading Students interested in reading more on the issues raised in this Unit can find useful material in the following: Gilbert, N. (ed.) (1999) Researching Social Life, London: Sage, the following section: Section II, ‘Into the Field’ (Chapters 6–10); Chapter 13 ‘Analysing Other Researchers’ Data’.
6.13 Study Questions You should now write approximately 300 words in answer to each of the questions below. We believe that this is an important exercise that will assist your comprehension of material and aid your progress on the course. Your answers are intended to form part of your own notes and should not be forwarded to the University.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• Questionnaire design is an integral part of conducting a survey. Attention should be paid
PLANN NG
DISASTER
1. Contrast the advantages and disadvantages of surveys and interviews. 2. What are the purposes of conducting a pilot study? 3. What are the advantages of using secondary data gathering techniques?
6-43
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
6.14 Bibliography Bryman, A. and Cramer, D. (1994) Quantitative Data Analysis for Social Scientists, revised edition, London: Routledge. Carr, E. H. (1961) What is History? Harmondsworth: Penguin. Cjaza, R. and Blair, J. (1996) Designing Surveys: A Guide to Decisions and Procedures, Thousand Oaks, CA: Pine Forge Press. Coleman, C. and Moynihan, J. (1996) Understanding Crime Data: Haunted by the Dark Figure, Buckingham: Open University Press. de Vaus, D. A. (1993) Surveys in Social Research, Third Edition, London: University College of London Press.
MSC IN EMERGENCY PLANNING MANAGEMENT
Durkheim, E. (1951) Suicide, Glencoe: Free Press. Faraday, A. and Plummer, K. (1979) ‘Doing Life Histories’, Sociological Review, 27(4): 773–92. Fielding, N. (1993) ‘Qualitative Interviewing and Ethnography’, in N. Gilbert (ed.) Researching Social Life, London: Sage. Fink, A. (1995a) The Survey Handbook: The Survey Kit 1, Thousand Oaks, CA: Sage. Fink, A. (1995b) How to Ask Survey Questions: The Survey Kit 2, Thousand Oaks, CA: Sage. Finnegan, R. (1996) ‘Using Documents’, in R. Sapsford and V. Jupp (eds) Data Collection and Analysis, London: Sage. Flick, U. (1998) An Introduction to Qualitative Research, London: Sage. Gilbert, N. (ed.) (1999) Researching Social Life, London: Sage. Gottschalk, L. (1945) ‘The Historian and the Historical Document’, in L. Gottschalk, C. Kluckhohn and R. Angell (eds) The Use of Personal Documents in History, Anthropology and Sociology, Bulletin 53, New York: Social Science Research Council. Hakim, C. (1982) Secondary Analysis in Social Research, London: Allen & Unwin. Hammersley, M. (1990) The Dilemma of Qualitative Method: Herbert Blumer and the Chicago Tradition, London: Routledge & Kegan Paul. Home Office (1992) Home Office Statistical Bulletin, London: HMSO. Hough, M. and Mayhew, P. (1983) The British Crime Survey: First Report, Home Office Research Study No. 85, London: HMSO.
6-44
disaster
riskcrisis MANAGE RISK
Jupp, V. (1989) Methods of Criminological Research, London: Routledge. Jupp, V. (1996) ‘Documents and Critical Research’, in R. Sapsford and V. Jupp (eds) Data Collection and Analysis, London: Sage. McNeil, P. (1985) Research Methods, London: Tavistock Publications. Maxfield, M. G. and Babbie, E. (2001) Research Methods for Criminal Justice and Criminology, Third Edition, Wadsworth Thomson Learning. Mayhew, P., Elliot, D. and Dowds, L. (1989) The 1988 British Crime Survey, Home Office Research Study No. 111, London: HMSO. Mayhew, P., Mirlees-Black, C. and Aye-Maung, N. (1994) Trends in Crime: Findings from the 1994 British Crime Survey: Research Findings No. 14, London: Home Office Research and Statistics Department.
Mirlees-Black, C., Mayhew, P. and Percy, A. (1996) The 1996 British Crime Survey: England and Wales, Home Office Statistical Bulletin, London: Home Office Research and Statistics Directorate. Moser, C. A. and Kalton, G. (1971) Survey Methods in Social Investigation, London: Heinemann. Nachmias, D. and Nachmias, C. (1976) Research Methods in the Social Sciences, London: Edward Arnold. Nachmias, D. and Nachmias, C. (1981) Research Methods in the Social Sciences, Alternate Second Edition without Statistics, London: Edward Arnold. Newell, R. (1993) ‘Questionnaires’, in N. Gilbert (ed.) Researching Social Life, London: Sage. Neuman, W. L. (1997) Social Research Methods. Qualitative and Quantitative Approaches, Third Edition, Boston: Allyn and Bacon. Okely, J. (1994) ‘Thinking Through Fieldwork’, in A. Bryman and R. Burgess (eds) Analyzing Qualitative Data, London: Routledge. Plummer, K. (1990) Documents of Life: An Introduction to the Problems and Literature of a Humanistic Method, London: Unwin Hyman.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Miller, D. (1983) Handbook of Research Design and Social Measurement, London: Longman.
PLANN NG
DISASTER
Procter, M. (1993) ‘Analysing Survey Data’, in N. Gilbert (ed.) Researching Social Life, London: Sage. Richard, S. (1992) The Directory of British Official Publications, London: Mansell Publishing. Richards, L. and Richards, T. (1994) ‘From Filing Cabinet to Computer’, in A. Bryman and R. G. Burgess (eds) Analysing Qualitative Data, London: Routledge.
6-45
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 6-46
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Rose, G. (1982) Deciphering Sociological Research, London: Macmillan. Sapsford, R. and Jupp, V. (1996) ‘Validating Evidence’, in R. Sapsford and V. Jupp (eds) Data Collection and Analysis, London: Sage. Scott, J. (1990) A Matter of Record, Cambridge: Polity Press. SCPR Newsletter (1981) Group Discussion and In-Depth Interviewing, Technical Manual No. 4, London: SCPR. Singleton, J. A. and Straits, B. C. (1999) Approaches to Social Research, Third Edition, New York: Oxford University Press. Smith, D. J. (1991) ‘The Origins of Black Hostility to the Police’, Policing and Society 2(1): 1–15. Sudman, S. and Bradburn, M. (1982) Asking Questions, San Francisco: Jossey-Bass. Thompson, E. P. (1978) The Voice of the Past: Oral History, Oxford: Oxford University Press. Weber, M. (1930) The Protestant Ethic and the Spirit of Capitalism, London: Allen & Unwin. Whyte, W. F. (1989) ‘Interviewing in Field Research’, in R.G. Burgess (ed.) Contemporary Social Research 4 (Series editor: M. Bulmer), Field Research: A Sourcebook and Field Manual, London: Unwin Hyman.
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 7 Qualitative Analysis and Presentation
disaster
riskcrisis MANAGE RISK
7 Unit Seven: Qualitative Analysis and Presentation 7.1 Aims and Objectives of this Unit This Module has already discussed how to choose methods and design research, and this Unit is concerned with the ‘now what?’ question. If qualitative methods have been used then you are likely to have pages and pages of text or images, i.e. ‘data’, which need analysing. You are also likely to want to develop a theory from your data. This Unit will assist you in this process. Specifically, the objectives of this Unit are to: • understand the value of qualitative research; • learn how to index qualitative data; • describe the different ways of coding qualitative data;
• have the ability to conceptualise, or develop themes from, qualitative data; • be able to apply these basic rules to different methods of qualitative data collection; • understand how qualitative data are analysed to provide meaning in order to develop theories. On completion of this Unit students should be able to analyse and present qualitative data and develop theories based on qualitative data.
7.2 Introduction to Qualitative Analysis There have been many debates around what actually constitutes qualitative research, whether it can be defined and what its relationship to quantitative data should be. Mason (1996: 4) offers a loose definition, which is made up of three parts, stating that qualitative research is: • Grounded in a philosophical position which is broadly ‘interpretivist’ in the sense that it is concerned with how the social world is interpreted, understood, experienced or produced. Whilst different versions of qualitative research might understand or approach these elements in different ways (for example, focusing on specific meanings, or interpretations, or practices, or constructions) all will see at least some of these as meaningful elements in a complex – possibly multi-layered – social world. • Based on methods of data generation which are flexible and sensitive to the social context in which the data are produced (rather than rigidly standardised or structured, or removed from ‘real life’ or ‘natural’ social context, as in some forms of experimental method).
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• recognise the importance of writing memos;
PLANN NG
DISASTER
• Based on methods of analysis and explanation building which involve understandings of complexity, detail and context. Qualitative research aims to produce rounded understandings of the basis of rich, contextual, and detailed data. There is more emphasis on ‘holistic’ forms of analysis and explanation in this sense, than on charting surface patterns, trends and correlations. Qualitative research usually does include some form of quantification, but statistical forms of analysis are not seen as central.
7-3
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Qualitative research does not aim, therefore, to produce universal laws about society. Rather, it serves as an exploration and a way of developing theory that is not detached from the original data. Qualitative research allows a deeper understanding of the meanings and values related to the topic of research than quantitative data; however, it has been criticised for this very understanding. Positivists, who adhere closely to the scientific paradigm of research, argue that qualitative research is less valid than quantitative research because it relies upon the researcher’s own subjective interpretation of the data. It is this debate that has added to the increased polarisation of quantitative and qualitative methods (and indeed researchers). In contrast, qualitative researchers argue that quantitative data only give a surface view of any given phenomenon that is not adequate to describe in any depth the meanings that lie behind the majority of social research (see Westmarland, 2001 for a full discussion of the quantitative/qualitative debate).
MSC IN EMERGENCY PLANNING MANAGEMENT
The increased valorisation of qualitative methods has been influenced in the last few decades and this can be explained in two ways. First, the criticisms of quantitative research made by feminists in the 1980s and second, the development of different forms of analysis of qualitative data which has in turn led to more discussion of qualitative methods and analysis in research methods textbooks. There are also now international journals specifically geared towards the development of new ideas and discussion around the analysing of qualitative research.
7.2.1 Feminism and Qualitative Research Second wave feminism arose in the UK in the 1960s and began to question the methods and epistemological stance taken by (mostly male) social scientists (see for example Bernard, 1975). It was argued that quantitative methods could not adequately reflect the complex social experiences of women, as highlighted by Graham (1983: 146): Women do not experience the world from a position of insularity and equality, but from a complex web of asymmetrical social relationships. Their experiences, moreover, are defined through a language which originates from outside these experiences, making survey speak an incomplete and alienating form of communication. Additionally, concern was not simply aimed at the validity of the research but also the well-being of the participants. Quantitative research was argued to further oppress women, and through the influential research of Ann Oakley, qualitative methods were amended to be more ‘woman friendly’. The term ‘subject’ has subsequently been rejected by feminist researchers, as explained by Westmarland (2001: para 21): They reject the use of the word ‘subject’ that implies the participant is an insensate object to be experimented on and observed like an animal in a zoo. Although a more equal relationship between the researcher and participant is often cited as increasing the validity of the research, this is not the primary reason feminist researchers insist upon this relationship. Feminist researchers are working within the wider women’s liberation movement and are working towards the overall aim of all women being free from oppression. It is hence clearly not acceptable for researchers to further oppress women in the name of academic research. Feminist researchers therefore re-emphasised the usefulness of qualitative research while also contributing to the furthering of its development.
7-4
disaster
riskcrisis MANAGE RISK
7.2.2 Development of New Forms of Analysis Also the subject of further development were the forms of analysing qualitative data. Until the groundbreaking work of Glaser and Strauss (1967) and their discovery of grounded theory, qualitative research was analysed purely on the ‘hunches’ of the researchers. Analysis was very much down to the individual researcher, as Kvale (1996: 188) highlights: Analysis took place through listening to repeated replaying of the tapes, or by cutting and pasting selections from the transcribed pages. The analyses more often terminated because of time limits or exhaustion, rather than a feeling of having analysed the material to have worked out its main structures and meanings. There was therefore the need for a comprehensive description of exactly how researchers got from their pages of data to their condensed version in the form of final report. 7.2.2.1 Grounded Theory Grounded theory was first developed in 1967 by Glaser and Strauss, and most qualitative textbooks refer to this approach. The theory has been subsequently revised over the last three decades; The methodological thrust of the grounded theory approach to qualitative data is towards the development of theory, without any specific commitment to specific kinds of data, lines of research or theoretical interests. So, it is not really a specific method or technique. Rather, it is a style of doing qualitative analysis. It is called grounded theory because it aims to develop a coherent theory that is grounded in the data. In this sense, the theory emerges through the analysis while never losing hold of the original data. It is based on a ‘concept-indicator’ model: Figure 7.1: Concept-Indicator Model (taken from Strauss, 1987: 25)
Figure 7.1 above shows that a concept is directly coded from empirical indicators (the Is) and also that the indicators are constantly compared (see arrows). Despite the discovery of grounded theory there remain few textbooks that explain the structural approaches needed to analyse qualitative data effectively and efficiently, and grounded theory remains the most comprehensive description of qualitative analysis with many computer software packages based upon this model of analysis.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
however, its basic structure remains the same. Strauss (1987: 5) explains:
PLANN NG
DISASTER
7.2.2.2 The Use of Computers for the Analysis of Qualitative Research The development of new forms of analysis has also been aided significantly through the advances in computer technology that have speeded up the process significantly. Additionally, specific computer software programs such as NUD*IST (Non-numerical Unstructured Indexing, Searching
7-5
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
and Theory-building) and more recently Ethnograph, ATLAS/ti and HyperResearch, have again speeded and simplified the process of qualitative analysis. The use of these programs to analyse data is known as Computer-Aided Qualitative Data Analysis (CAQDA). These software programs, however, are not essential for analysing qualitative data and are more geared towards large-scale studies with numerous interviews. They are particularly useful if more than one researcher has collected the data; however, simple word processing packages can be used just as effectively when dealing with small to medium sized studies.
EMERGENCY
7.3 Basic Principles for the Analysis of Qualitative Data
MSC IN EMERGENCY PLANNING MANAGEMENT
The aim of qualitative research is to ‘provide an explicit rendering of the structure, order and patterns found among a set of participants’ (Lofland 1971: 1). The term ‘qualitative research’ is an umbrella term under which a host of different methods of data collection fall, and many methods can be used for either qualitative or quantitative analysis. The easiest way of remembering the distinction between the different categories of analysis is that qualitative analysis is concerned with some form of words or images, while quantitative analysis is concerned with numbers. To describe the processes involved in the analysis of qualitative data, Robson (1993: 377) offers some basic principles that should be followed. These are relevant regardless of the qualitative method used: 1. Analysis of some form should start as soon as the data are collected. Don’t allow data to accumulate without preliminary analysis. 2. Make sure you keep tabs on what you have collected (literally – get it indexed). 3. Generate themes, categories, codes, etc. as you go along. Start by including rather than excluding; you can combine and modify as you go on. 4. Dealing with data should not be a routine or mechanical task; think, reflect! Use analytical notes (memos) to help to get from the data to a conceptual level. 5. Use some form of filing system to sort your data. Be prepared to re-sort. Play with the data. 6. There is no one ‘right’ way of analysing these kind of data – which places even more emphasis on your being systematic, organised and persevering. 7. You are seeking to take apart your data in various ways and then trying to put them together to form some consolidated picture. Your main tool is comparison. Different research designs and methods use different tools of analysis; however, all qualitative data must be indexed, coded and conceptualised in some format in order to reduce the data to a manageable amount. Memos should also be written to assist the generation of theory. As these principles apply to all qualitative research, they will now be described in depth, before looking at specific tips for analysing specific methods that are used to collect qualitative data. Before moving on, however, principle number six must be re-emphasised – that there is ‘no one
7-6
disaster
riskcrisis MANAGE RISK
right way’ to analyse qualitative data. In this sense, the only rule that should be followed when analysing qualitative research is that ‘there are no rules’! As Plummer (1990: 99) explains: In many ways this is the truly creative part of the work – it entails brooding and reflecting upon mounds of data for long periods of time, until it ‘makes sense’ and ‘feels right’ and key issues and themes flow from it. It is also the hardest process to describe. The standard technique is to read and take notes, leave and ponder, reread without notes, make notes, match notes up, ponder, reread and so on. This may sound difficult and confusing; however, on the plus side – because there is no one right way of analysing qualitative research this also means that there is no wrong way! Even though there are rules there are basic principles that can aid you in your analysis and a basic model can be suggested: Figure 7.2: Basic Model of Qualitative Analysis
Any given piece of qualitative research is likely to generate large amounts of data, and it is necessary to organise this in a way that allows you to quickly retrieve relevant parts. This is called indexing and works in a similar way as an index in a book whereby you need to be able to quickly locate any given piece of datum. For example, you may have ‘chapters’ which encapsulate broad thematic areas. Within these ‘chapters’ you may have sub-sections, for example you may have different kinds of data. For any one theme you may have secondary data (for example newspaper reports, diary entries), pictorial data (for example photographs) and transcripts (for example interviews, focus groups, television reports). Comprehensive field notes (which are also classed as data) should also be written up and included as a sign of good practice. It is important that all data that are indexed
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
7.3.1 Indexing Qualitative Research
PLANN NG
DISASTER
7-7
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
dated, and this can serve as one option for arranging your index at the early stages. You should then develop an index for your ‘chapters’ which specifies exactly what is included, and, like in an index some parts may be repeated in several different sections and your index will allow crossreferencing. The indexing phase should be carried out alongside the coding and conceptualisation processes and is an ongoing procedure. It may be either computer based (cutting and pasting different sections into folders and sub-folders) or paper based (literally cutting with scissors and putting into different paper folders and sub-folders).
7.3.2 Coding Qualitative Data
MSC IN EMERGENCY PLANNING MANAGEMENT
Alongside the development of themes the coding of data must begin. It is rare for researchers conducting unstructured forms of qualitative research to wait until they have collected all their data before starting the coding process. This is because themes often emerge through coding (and vice versa), and as the theory building process begins the research may change focus or discover new areas that warrant detailed study, that were not previously considered before beginning the research. It is not unusual for interview schedules (if used) to be altered after the coding of the first few texts. Miles and Huberman (1994: 56) define codes as: ... tags or labels for assigning units of meaning to the descriptive or inferential information compiled during a study. Codes usually are attached to ‘chunks’ of varying sizes – words, phrases, sentences or whole paragraphs, connected or unconnected to a specific setting. Coding can therefore be seen as a form of minimising and abbreviating large pieces of text, as Neuman (1997: 422) highlights: ‘coding data is the hard work of reducing mountains of raw data into manageable piles. In addition … coding allows a researcher to quickly retrieve relevant parts of it.’ This process may sound like a long and tedious procedure, and unfortunately this is indeed the case! This is one good reason for not leaving the entire coding process until the end of the data collection. There have been several attempts to formulate a typology of coding techniques, and the one most frequently used is that proposed by Glaser and Strauss (1967) that they refer to as a ‘coding paradigm’. Since their first proposal, Strauss (various years) has further developed this paradigm which is made up of open coding, axial coding and selective coding. 7.3.2.1 Open Coding Open coding is the initial coding that should be conducted and Strauss (1987: 28) describes this as the ‘unrestricted coding of the data’. More specifically, open coding refers to ‘the analytic process through which concepts are identified and their properties and dimensions are discovered in data’ (Strauss and Corbin, 1998: 101). The aim of open coding is to ascertain provisional concepts that fit the data. This stage is generally conducted by studying the data (whether it be an interview transcript, document, diary or whatever) and writing brief codes and/or notes within or at the side of the data. The most appropriate way of explaining open coding is though the use of an example. If an interview were to be transcribed, that looked at the problems faced by mothers in paid employment, it may be coded as demonstrated below (with codes in [ ] ):
7-8
disaster
riskcrisis MANAGE RISK
Interviewer
Tell me how an average day may start.
Participant
Well, first of all I wake up with the inevitable screaming [frustration] and I’m always tired because I stay up late trying to get all the housework done on an evening [lack of time]. I get up and get her dressed for nursery [planning] but I also have to get ready for work myself and this gets me stressed [stress/lack of time]. Breakfast time is always difficult [stress] because she throws all her breakfast up the walls [frustration] but sometimes she looks so sweet and innocent I can’t help but laugh [pleasure] she’s just so adorable [love]. What happens after you’ve had breakfast?
Participant
Well that’s when my husband usually gets up. I’m sure he waits until she’s neatly dressed* and fed before coming down to help [frustration/blame] and then he plays with her while I make the packed lunches [planning]. I’m always worried at this point whether I will make it to work on time [stress/lack of time]. There’s never enough time to wash the breakfast dishes [lack of time] and they inevitably get left until the evening [frustration].
Through the coding of this very small excerpt we can see that the participant’s morning is characterised by frustration, stress and a lack of time. Further interviews may reveal similar experiences that can be coded in a similar way. However, new codes will need to be developed throughout the interview process and some of the original codes may need to be amended slightly. The codes can be inserted within the text if working on a computer but it is essential that they stand out clearly when skimming through the document at a later date. This can be done through the use of bold, coloured and/or italicised fonts. Alternatively, codes can be handwritten alongside the printed out (or hand-written) text. However, it is a good idea to write these in pencil to start with in order to make amendments if necessary. If you feel confident in your codes (or have spare photocopies of your data) you may wish to allocate specific codes to different coloured pens, and these can be used to highlight areas in your data. For example, if ‘stress = red’ then all areas with stress written next to them would simply be highlighted in red. Remember though to use relatively thin pens because some areas may need to be highlighted in more than one colour if they demonstrate more than one code. It is important to highlight that these are not ‘correct codes’, and coding is the result of the researcher’s subjective reading of the data. While this is not a problem for research where one researcher will conduct the whole of the analysis, more detailed coding schemes need to be developed for projects consisting of more than one analyst. Cross-referencing may also be needed. In practice these are often large-scale studies that are likely to use CAQDA.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Interviewer
PLANN NG
DISASTER
The above example is a line by line coding strategy it does not have to be done in this way (remember – there are no rules!). For larger amounts of data this excerpt of text may need to be coded as one category at the open coding stage (e.g. getting ready to go to work) and subsequent coding (axial and selective) can be conducted in relevant areas. Although the above example uses text, pictorial images can be coded in a similar way. Pictures are arranged on paper or a board surrounded by plenty of space to add codes and notes around the image.
7-9
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
7.3.2.2 Axial Coding This is the second stage of coding, where the researcher takes the set of codes to develop clusters or themes from the data. Several related concepts may be collapsed into one theme, or some concepts may need to be split and re-coded. Neuman (1997: 424) highlights the analytical nature of axial coding: Axial coding stimulates thinking about linkages between concepts or themes, and it raises new questions. It can suggest dropping some themes or examining others in depth. In addition, it reinforces the connections between evidence and concepts. As a researcher consolidates codes and locates evidence, he or she finds evidence in many places for core themes and builds a dense web of support in the qualitative data for them … the connection between a theme and data is strengthened by multiple instances of empirical data. Axial coding is therefore concerned with the generation of theory along with data analysis. It is worthwhile making extensive analytical memos at this stage to keep track of why you decided to cluster certain concepts and always keeping in mind what the evidence, or data, is ‘telling’ you. Remember not to lose sight of the empirical data.
MSC IN EMERGENCY PLANNING MANAGEMENT
7.3.2.3 Selective Coding Selective coding is the final stage and involves re-reading the data in light of the themes that have been developed. Comparisons are made, and elaboration may be necessary for themes that are of particular interest and/or importance. At this stage ‘key’ quotes and/or images may be highlighted for use in the final report. These are prime examples of themes – those that best support your theory. It is at this stage that your theory should begin to take shape; however, the exact nature of it may still be ambiguous. This will be discussed further in Section 7.9 when suggestions for the development of theory are offered. Depending on the heterogeneity of the participants, documents or pictures there may be many themes or only a few. The number of themes is also led to some degree by the scale of research and by the breadth of the topic being studied. It is important to consider previous literature and theories while developing themes and working out how your theory is being built in comparison to other research. This marks a key difference between quantitative and qualitative data analysis; in the former, theory is developed via a discussion of the results, while the latter involves the continuous development of theory throughout the analytical stages.
7.3.3 Writing Analytical Memos Analytical memos are basically ideas that come into the researcher’s mind while reading the transcripts. They are used when you want to emphasise or query something that is not part of the coding process. Glaser (1978: 83–4) explains: A memo is the theorising write-up of ideas about codes and their relationships as they strike the analyst during coding ... it can be a sentence, a paragraph or a few pages … it exhausts the analyst’s momentary ideation based on data with perhaps a little conceptual elaboration. The important of the use of memos cannot be understated as they play an essential role in theory building. They should always be indexed and if they refer to a specific section this should be highlighted. They could alternatively be general ideas about what you think the research as a whole
7-10
disaster
riskcrisis MANAGE RISK
is ‘saying’, and as with all indexed material these should be dated. If at a later date you contradict something you wrote in an earlier memo the original one should never be destroyed – simply highlight on the bottom of the memo to see an updated idea and cross-reference these (i.e. see [date]). The * sign in the above example of coding can be used to show where a memo might be used. It may look something like this: *01.08.2001 Interview with Participant # It is interesting that she is so busy but is still concerned about her daughter looking smart and tidy. She doesn’t mention her own clothes or appearance. Memos are incredibly personalised and any two researchers are extremely unlikely to pick up on the same pieces of text to write memos. Similarly, a memo may be about something that happened during the interview when this part of text was spoken: *01.08.2001 Interview with Participant #
The most important rule of memoing is never rely on your memory – write it down! You may think of a fantastic theory and be sure that you will never forget it, but this idea is likely to be superseded by another one and memories are rarely as clear as written memos.
7.4 Analysing Transcripts Speech is generally recorded using an audio recorder, and advances in technology are improving the quality that is obtained. It is important to remember that transcription is a very long process – it will take a fast typist around four hours to transcribe one hour of tape and this is only if the quality is good. In practice it cuts into researchers’ time to a significant degree and the level of detail required in the transcription can lengthen this process even further.
7.4.1 Interviews In their most simple form, interviews can be defined as: Encounters between a researcher and a respondent in which the latter is asked a series of questions relevant to the subject of the research. The respondent’s answers constitute the raw data analysed at a later point in time by the researcher.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
At this point participant looked down at her daughter and smiled. She appears to get a lot of pleasure from seeing her daughter looking tidy.
PLANN NG
DISASTER
(Ackroyd and Huges, 1983: 66) Unit 6 discusses the use of interviews as a method, this Unit is concerned primarily with the analysis. 7.4.1.1 Face-to-Face Interviews Face-to-face interviews are generally analysed using a system such as that described above (i.e. indexing, coding, analytical memoing); however, it is also important to make field notes. These
7-11
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
will help contextualise the transcript when you start analysis. Remember, the whole interview process will not be reflected fully in a transcript. Some meanings may be lost if notes are not kept regarding body language, the physical environment, etc. Additionally, some participants are wary of being interviewed using an audio recorder and may refuse to have their interview taped. It is their right as participants to choose and you must be prepared to take extensive notes if this occurs. Additionally, there is often a lot of useful information given in pre and post interview discussions, and it is necessary to document this information in the form of field notes, or analytical memos if the information gives you some clue about what your end theory may include. Field notes should always be written up, dated and indexed and used as qualitative data in the same way as a transcription. 7.4.1.2 On-line Interviewing
MSC IN EMERGENCY PLANNING MANAGEMENT
In relation to recent advances in computer technology and the Internet there has been a developing interest over the last five years regarding ‘on-line’ interviewing. This may take place in a ‘chat room’ or the ‘bouncing’ back and forth of e-mails. There are, however, specific ethical issues that must be taken into consideration when using the Internet as an interviewing tool. You must make sure that you receive informed consent from your participant in the same way as you would in a face-toface interview. Additional problems may arise if you are interviewing on a sensitive topic. It would not be good practice to leave a participant upset and distraught after a face-to-face interview and the same applies to on-line interviews. However, it may be more difficult to gauge the feelings of a participant when they are (possibly) thousands of miles away than when they are sitting in the next chair. These factors must be taken into consideration before starting an on-line interview. Analysis can take place in a similar way to face-to-face interviewing, although little has yet been written on the subject. The basic rule is to treat the on-line interview out and use it as a transcript, although arguments could be made for a qualitative content analysis in some cases.
7.4.2 Conversation Analysis Conversation analysis, or CA as it is generally referred to, is interested in the ‘normal’, ‘mundane’ conversations that go on in everyday life and is centred around language. It is concerned with the ‘sequential organisation of talk’ (Silverman, 1993: 120) and therefore differs from most forms of interview analysis. Tape recordings are vitally important as a data collection tool for CA, as Pomerantz and Fehr (1997: 70) explain: 1.
Certain features of the details of actions in interaction are not recoverable in any other way.
2.
A recording makes it possible to play and replay the interaction, which is important both for transcribing and for developing an analysis.
3.
A recording makes it possible to check a particular analysis against the materials, in all their detail, that were used to produce the analysis.
4.
A recording makes it possible to return to an interaction with new analytic interests.
Unlike interview data, CA starts its analysis at the transcription stage, and as Ashmore and Reed (2000: para 11) highlight, transcription ‘is routinely understood as a craft process, as itself a part of the practice of analysis’.
7-12
disaster
riskcrisis MANAGE RISK
7.4.3 Discourse Analysis Discourse analysis (DA) is also centred around language and starts its analysis at the transcription stage, but it varies from CA in that it generally uses a more detailed transcription. Certain symbols are used to denote different styles of speech: Table 7.3: Selection of Commonly Used Transcription Symbols Pause no longer than one tenth of a second.
I was going to (.) the park
(.3)
Pause recorded in tenths of a second.
When I came across John (.3) the park warden
.hhh
Out-breath (number of h’s indicates length). Without the dot indicates in-breath.
He was .hhh looking kinda sad
CAPITAL
Word or words are emphasised And he said I CAN’T BELIEVE IT through volume.
(word) :
Possible hearings. So I asked (him) what was up Prolongation of the prior sound. And he said you will ne:ver The number of colons represents guess what’s happened the length of prolongation.
(
Transcriber unable to hear what was said.
)
(
)
This is the most time-consuming form of data to transcribe, although some of the more basic signs are often used as good practice in general interview transcriptions. DA is also interested in the power discourses that form the data. Does one individual dominate the conversation? Do the individuals overlap each other’s speech? Are there any gender dynamics that need to be taken into consideration? In this sense, DA attempts to analyse more deeply than other forms of transcription. It may pick up important findings that are not visible from a basic interview transcription, and it is always worth remembering that sometimes a silence can say as much as, if not more than, words.
7.5 Analysing Observational Data
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
(.)
PLANN NG
DISASTER
Observational data may be gathered through the use of ethnographic note-books, or field notes from participant observation. It differs somewhat from other forms of qualitative research in that codes generally need to be developed to some degree before the observation begins due to the time limits involved in recording observational data. A pilot study is likely to determine the most frequently used codes. However, if this is not possible due to limited observational opportunities it may be necessary to consider using a pre-designed coding frame or even adopting one that has
7-13
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
been previously developed and used by other researchers. This can be particularly useful if the coding frame has been used for a similar topic to the one you wish to use it for. The coding frames therefore become the method of data collection.
7.6 Pictorial Images Pictorial and other visual images are often neglected in qualitative analysis because they are more difficult to analyse than text. However, images such as photographs, cartoons, advertisements, films, etc. can be used alongside other qualitative methods or on their own. One of the first studies to use photographic data was carried out by Bateson and Mead (1942) entitled Balinese Character and this remains the largest visual ethnography with around 25,000 images collected and analysed. Pictorial and visual images should be indexed and then analysed by writing memos, developing codes and themes as specified earlier in this Unit.
MSC IN EMERGENCY PLANNING MANAGEMENT
7.7 Content Analysis Qualitative content analysis differs from quantitative content analysis in that it does not simply try to calculate the frequencies with which something occurs. May (1993: 146–7) explains: Qualitative content analysis … starts with the idea of a process, or social context, and views the author as a self-conscious actor addressing an audience under particular circumstances. The task of the analyst becomes a ‘reading’ of the text in terms of its symbols. Content analysis was described in further detail in Unit 6.
7.7.1 Analysis of Media Sources (Newspapers, Web-sites etc.) Media sources (such as advertisements, magazines, television, films, newspapers) can be analysed though qualitative content analysis. They can provide excellent sources for analysis, and Weitz (1977: 194) highlights their importance, stating that ‘the cultural products of any given society at any given time reverberate with the themes of that society and that era’. Qualitative analyses of media sources aim to go deeper into the meaning of the sources than that provided by quantitative research. For example, they allow an analysis of how the frequencies of certain words and/or pictorial images are framed. Often qualitative analyses of media sources are mixed with some form of quantitative analysis; however, the latter analysis generally goes no further than frequencies or cross-tabulations. These aid the development of theory while the qualitative data are used to provide a deeper analysis through the use of important quotations and/or pictorial images. Web-sites may also form the data for qualitative analysis, and the actual design of the site may be as important as the text. Similarly, the links may also provide data for analysis.
7.8 Analysing Documents The idea of documentary analysis often lends itself to images of rooting through dusty historical archives (Mason, 1996), but this is far from the case. Documents for analysis may be those already produced, such as leaflets, Acts of Parliament or minutes of meetings, or they may be documents that you have requested for the purposes of research such as diaries.
7-14
disaster
riskcrisis MANAGE RISK
May (1993: 138) highlights that documents need to be analysed in terms of the social context and some form of engagement must take place in order for documents to be understood before analysis can begin. This is also necessary in order to discover any ‘hidden meanings’ within the document: ... the task of criticism is not to bring out the work’s relationship with the author, not to reconstruct through the text a thought or experience, but rather to analyse the work through its structure, its architecture, its intrinsic form, and the play of its internal relationships. (Foucault, 1984: 103)
7.8.1 Diaries
For unstructured diaries, involving coding of verbatim entries, the processing can be very labour intensive, in much the same way as it is for processing qualitative interview transcripts. Using highly trained coders and a rigorous unambiguous coding scheme is very important particularly where there is no clear demarcation of events or behaviour in the diary entries. Clearly, a well designed diary with a coherent pre-coding system should cut down on the degree of editing and coding. (Corti, 1993: 3) Diaries then need to be analysed in the same way as previously described.
7.8.2 Organisational Files Organisational files need to be analysed in a similar way to other texts and the nature of their development must also be taken into consideration, as Silverman (1993: 61) highlights: Like all documents, files are produced in particular circumstances for particular audiences. Files never speak for themselves … [it is necessary to] understand both the format of the file (for instance, the categories used on blank printed sheets) and the processes associated with its completion.
7.9 Secondary Analysis of Qualitative Data Sources
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Corti (1993) highlights that while diaries have long been recognised for their usefulness in recording historical data, their use in collecting qualitative data in contemporary society has been overlooked until relatively recently (known as self-completion diaries). She highlights that while much of the editing and coding process is carried out by the researcher in the field, the amount of time needed to analyse a single diary depends upon how structured it is:
PLANN NG
DISASTER
Secondary data analysis is defined by Hakim (1982: 1) as: ... any further analysis of an existing data set which presents interpretations, conclusions or knowledge additional to, or different from, those presented in the first report on the inquiry as a whole and its main effects. Although quantitative data are often the subject of secondary analysis, this approach has not been widely used in relation to qualitative research. Heaton (1998: 2) highlights that secondary analysis
7-15
EMERGENCY
crisisdisaster RISK
MANAGE
can be useful to qualitative researchers for three main purposes: • Additional in-depth analysis – whereby the analysis of a particular finding is elaborated upon. • Additional sub-set analysis – whereby a sub-set of the original sample is the area of focus. • New perspective/conceptual focus – whereby all or part of the sample is re-analysed from a different perspective using different concepts than those developed in the original analysis. One of the main problems in the past for researchers wishing to conduct secondary analysis of qualitative data has been the lack of archival material. This issue has now been addressed, and the ESRC (Economic and Social Research Council) have developed ‘Qualidata’ which archives qualitative data.
MSC IN EMERGENCY PLANNING MANAGEMENT
Secondary analysis can in some circumstances be the only way research can be conducted. For example, costs for certain research proposals may be prohibitive with primary data, or participants may be difficult or impossible to obtain due to the nature of the research. Secondary analysis that takes into account the methodological implications of using secondary data can provide just as useful and valid data as these produced through primary research.
7.10 Looking for Meaning – Developing Theory In order to develop theory some form of meaning must be sought that will make sense to those reading your research. Miles and Huberman (1994: 245–6) highlight 13 tactics that can assist researchers in moving from description to explanation and aid the development of theory: • Noting patterns and themes • Seeing plausibility (does it make sense?) • Clustering (seeing what goes with what) • Making metaphors (integrating diverse pieces of data) • Counting (noting the frequencies of various codes and/or themes) • Making contrasts/comparisons (to sharpen understanding) • Partitioning variables (differentiation) • Subsuming things under the general • Factoring • Noting relations between variables • Finding intervening variables • Building a logical chain of evidence • Making conceptual/theoretical coherence
7-16
crisis
PLANN NG
risk
disaster
riskcrisis MANAGE RISK
Researchers use different strategies to use these steps in a move towards explanation. However, one of the most frequently used methods without CAQDA uses ‘post-it notes’, sticky labels or scissors and glue. Themes can be written on notes and stuck in a random pattern on a board, wall or the back of a door. Sometimes drawing pins are used in one corner of each note and string is used to temporarily link themes together to develop a coherent order. Alternatively, they may be arranged by sticking them in different positions to develop a model that adequately reflects the original data. It is essential to leave enough time at the end to develop the structure that your theory will take, as this can be very time consuming unless you have a clear view of what your theory will look like. It is more likely to include a lot of gluing, cutting, drawing and so on before you are happy with your finished theory. When you are, then it is time to think about presenting your theory to others.
7.11 Presenting Your Theory
Your final report should reflect all the stages of analysis and all the forms of data that were originally gathered. Neuman (1997: 425) shows how these are brought together through the index that should have been developed: Figure 7.4: Production of Final Report (Taken from Neuman, 1997: 425)
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Excerpts from your original data set, be they visual, textual or whatever, need to be integrated into your theory to demonstrate how your theory is supported by your original data. It is important that these are integrated as smoothly as possible to enable easy and enjoyable reading – as Taylor and Bogdan (1984: 149) highlight, ‘the purpose of research is not only to increase your own understanding of social life, but also to share that understanding with others’.
PLANN NG
DISASTER
The final report must therefore be fully reflective of the whole research process, from design to analysis.
7-17
EMERGENCY
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
7.12 Advantages and Disadvantages of Qualitative Research Despite common preconceptions, qualitative research is often more time consuming than quantitative research, and it is certainly no easier to analyse. The advantages and disadvantages of qualitative research can only be adequately decided upon in reference to any given research topic, in other words what may be seen as an advantage in one piece of research may be seen as a disadvantage in another. General guidelines can be offered and these are described in Table 7.1 below. However, these must always be considered in terms of the requirements and limitations of your individual research topic. Table 7.1: Advantages and Disadvantages of Qualitative Research Advantages Disadvantages
MSC IN EMERGENCY PLANNING MANAGEMENT
More suitable for sensitive topics of research.
Can be expensive.
End result (i.e. report) is less detached from the original data than in quantitative research.
Time consuming.
May be more suitable for research with women.
Can be emotionally tiring/ distressing for the researcher.
Enables a deeper knowledge of the Is not as highly regarded as quantitative subjective meanings that are held research in certain academic disciplines. by participants. Allows more freedom when it comes to analysis.
Is criticised as being ‘non-scientific’.
Analysis is an ongoing process and this can make it easier to guide the research throughout the whole process.
More difficult to stay detached from the data to provide an ‘objective’ analysis.
7.13 Conclusion Due to the lack of literature on ‘how to do it’, qualitative researchers often learn to analyse through trial and error, and indeed this is to some degree also how experienced researchers continue to analyse qualitative data. There are no hard and fast rules, and it is often a matter of playing around with the data until a theory is formed that adequately reflects your data – a theory that is grounded. Time spent on indexing and writing memos along with coding will always be time well spent as it will inevitably save you time in the long run. It is important to conclude by highlighting the fact that qualitative research can be successfully used alongside quantitative research, and that they may prove to be mutually beneficial in the development of a full explanation or theory. As Westmarland (2001: para 28) highlights:
7-18
disaster
riskcrisis MANAGE RISK
... there is no need for the dichotomous ‘us against them’, ‘quantitative against qualitative’ debates … their success depends solely upon the researcher employing them. Therefore, although qualitative research can be useful for some research areas, similarly, quantitative methods also have their advantages. A full, detailed theory needs to analyse all necessary data that is appropriate to their area of study.
7.14 Main Points • Qualitative research is argued to give the researcher a deeper understanding of the topic of study than quantitative research. • There are no hard and fast rules that must be followed when analysing qualitative research. • Grounded theory (developed by Glaser and Strauss, 1967) remains the most comprehensive explanation for qualitative analysis.
• All forms of qualitative data must be coded to reduce the data to a manageable size. • There are three elements to coding data using grounded theory; open, axial and selective. • Analytical memos should be written throughout the analysis to aid the development of theory. • Codes and memos are then conceptualised and developed into themes. • Theory is developed from these themes. • Secondary data analysis is concerned with the re-analysis of existing data sources. • Discourse analysis (DA) differs from conversation analysis (CA) in that the former uses a more detailed transcription. • Qualitative content analysis differs from quantitative content analysis in that is does not simply try to calculate the frequencies with which something occurs. • In order to develop theory some form of meaning must be sought that will make sense to those reading your research. • Excerpts from your original data set, be they visual, textual or whatever, need to be integrated into your theory to demonstrate how your theory is supported by your original data. • Your final report should reflect all the stages of analysis and all the forms of data that were originally gathered.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• Advances in computer technology have speeded up the process of qualitative analysis.
PLANN NG
DISASTER
• Despite common preconceptions, qualitative research is often more time consuming than quantitative research, and it is certainly no easier to analyse. • Qualitative researchers often learn to analyse through trial and error. • There are no barriers to using both qualitative and quantitative research methods in the same piece of research and this can result in a fuller, more detailed analysis and overall theory.
7-19
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
7.15 Guide to Reading Students interested in reading more on the issues raised in this Unit can find useful material in the following books and articles: Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory, Chicago: Aldine Publishing Co. Kvale, S. (1996) Interviews – An Introduction to Qualitative Research Interviewing, London: Sage. Silverman, D. (1993) Interpreting Qualitative Data: Methods for Analysing Talk, Text and Interaction, London: Sage. Westmarland, N. (2001) ‘The Quantitative/Qualitative Debate and Feminist Research: A Subjective View of Objectivity’ [28 paragraphs] Forum: Qualitative Social Research [On-line Journal] 2(1). Available at: http://qualitative-research.net/fqs/fqs-eng.htm
MSC IN EMERGENCY PLANNING MANAGEMENT
7.16 Study Questions You should now write approximately 300 words in answer to each of the questions below. We believe that this is an important exercise that will assist your comprehension of the material and aid your progress on the course. Your answers are intended to form part of your own course notes and should not be forwarded to the University. 1. Why might qualitative methods be used rather than quantitative methods? 2. Describe the three stage coding paradigm developed by Glaser and Strauss (1967). 3. How is theory developed from qualitative data?
7.17 Bibliography Ackroyd, S. and Hughes, J. (1983) Data Collection in Context, London: Longman. Arksey, H. and Knight, P. (1999) Interviewing for Social Scientists – An Introductory Resource with Examples, London: Sage. Ashmore, M. and Reed, D. (2000) ‘Innocence and Nostalgia in Conversational Analysis: The Dynamic Relations of Tape and Transcript’ [45 paragraphs]. Forum: Qualitative Social Research [On-line Journal] 2(1). Available at: http://qualitative-research.net/fqs/fqs-eng.htm Bateson, G. and Mead, M. (1942) Balinese Character: A Photographic Analysis, New York: New York Academy of Sciences. Bernard, J. (1975) Women, Wives, Mothers, Chicago: Aldine Publishing Co. Corti, L. (1993) ‘Using Diaries in Social Research’, Social Research Update 2, Surrey: SRU.
7-20
disaster
riskcrisis MANAGE RISK
Foucault, M. (1984) ‘What is an Author?’, in P. Rabinow (ed.), The Foucault Reader, Harmondsworth: Penguin. Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory, Chicago: Aldine Publishing Co. Glasser, B. (1978) Theoretical Sensitivity, California: Sociology Press. Graham, H. (1983) ‘Do Her Answers Fit His Questions? Women and the Survey Method’, in E. Gamarnikow, D. Morgan, J. Purvis and D. Taylorson (eds), The Public and the Private, London: Heinemann. Hakim, C. (1982) Secondary Analysis in Social Research, London: Allen and Unwin. Heaton, J. (1998) ‘Secondary Analysis of Qualitative Data’, Social Research Update 22, Surrey: SRU. Kvale, S. (1996) InterViews – An Introduction to Qualitative Research Interviewing, London: Sage.
Mason, J. (1996) Qualitative Researching, London: Sage. May, T. (1993) Social Research – Issues, Methods and Process, Buckingham: Open University Press. Mayring, P. (2000) ‘Qualitative Content Analysis’ [28 paragraphs] Forum: Qualitative Social Research [On-line Journal], 1(2) Available at: http://qualitative-research.net/fqs/fqs-e/2-00inhalt-e. htm (accessed 1.08.2001). Miles, M. B. and Huberman, A. M. (1994) Qualitative Data Analysis: an Expanded Sourcebook (2nd edn), London: Sage. Neuman, W. L. (1997) Social Research Methods – Qualitative and Quantitative Approaches (3rd edn), London: Allyn and Bacon. Oakley, A. (1981) ‘Interviewing Women: A Contradiction in Terms’, in H. Roberts (ed.), Doing Feminist Research, London: Routledge & Kegan Paul. Plummer, K. (1990) Documents of Life: An Introduction to the Problems and Literature of a Humanistic Method, London: Unwin Hyman. Pomerantz, A. and Fehr, B. (1997) ‘Conversation Analysis: An Approach to the Study of Social Action as Sense Making Practices’, in T. A. Van Dijk (ed.), Discourse as Social Interaction, London: Sage.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Lofland, J. (1971) Analyzing Social Settings, New York: Wadsworth.
PLANN NG
DISASTER
Robson, C. (1993) Real World Research – A Resource for Social Scientists and Practitioner-Researchers, Oxford: Blackwell. Silverman, D. (1993) Interpreting Qualitative Data: Methods for Analysing Talk, Text and Interaction, London: Sage.
7-21
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 7-22
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Strauss, A. L. (1987) Qualitative Analysis for Social Scientists, California: Cambridge University Press. Strauss, A. L. and Corbin, J. (1998) Basics of Qualitative Research – Techniques and Procedures for Developing Grounded Theory (2nd edn), London: Sage. Taylor and Bogdan (1984) Introduction to Qualitative Research Methods – the Search for Meaning, New York: John Wiley & Sons. Weitz, R. (1977) Sex Roles, New York: Oxford University Press.
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 8 Statistical Tests
disaster
riskcrisis MANAGE RISK
8 Unit Eight: Statistical Tests 8.1 Aims and Objectives of this Unit Most research projects can produce large amounts of data and it is therefore not very helpful to the reader of a research paper or thesis to be presented with every item of raw data. It is preferable to be provided with a summary of the data in a form that highlights the most important trends, correlations, differences and conclusions. However, if a researcher is presenting summary data, it is important that this information correctly represents the results obtained without distortion and does not claim conclusions that are not substantiated by the results. Researchers should therefore avoid presenting the results of their research in a way that aims to support their preconceived assertions and ideas rather than in a way that provides an unbiased view of the research conclusions. The essence of research is therefore systematic but detached analysis of data.
to understand which tests should be used in which circumstances. Whilst most statistical tests can normally be completed through the use of standard computer packages, it is important to understand the underlying principles of the tests. The aims of this Unit are to: • outline which statistical tests are the most appropriate in different circumstances; • present some of the commonly used statistical tests; and • provide examples of how to use these tests.
8.2 Descriptive Statistics Descriptive statistics are used for presenting and summarising research data in the form of: • simple descriptions of results; • tables, charts and graphs; and • numerical measures. For example, in an experiment to determine whether men or women complete crosswords in the shortest time, ten men and ten women of similar ages and academic background were asked to complete the same crossword puzzle. The data obtained from this experiment are presented in Table 8.1.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The majority of most commonly used statistical tests are straightforward and their use does not require a high level of knowledge of statistical theory. Instead, it is more important for researchers
PLANN NG
DISASTER
8-3
EMERGENCY
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
Table 8.1: Time to Complete a Crossword Puzzle as a Function of the Participant’s Gender Participant number
Time to complete crossword, minutes
MSC IN EMERGENCY PLANNING MANAGEMENT
Men
Women
1
57
123
2
93
87
3
28
94
4
75
143
5
63
221
6
41
42
7
54
24
8
71
127
9
117
158
10
32
191
From the ten data points obtained for both men and women, it is possible to reach some very general conclusions: • the women appeared to take almost twice as long as the men to complete the crossword; • one woman took the shortest time and another woman took the longest time to complete the crossword; and • the women’s times (24 to 221 minutes) varied much more than the men’s times (28 to 117 minutes). This represents a simple summary description of the data obtained because it provides an indication of the average, the range and the variation in the results. However, it is possible to define these variations in more formal and precise statistical terms. These are referred to as central tendency and dispersion characteristics: Central tendency is that value obtained from a group of values which represents the most typical value for the sample or the value around which all other values are clustered evenly. This is normally interpreted as the average value but there are in fact a number of other ways of describing central tendency and these are discussed below. Dispersion provides a measure of how much the individual data sample values vary from the central value.
8.2.1 Measurement of Central Tendency There are three measures that provide information on the central tendency of research data; these are referred to as the mean, the median and the mode values.
8-4
disaster
riskcrisis MANAGE RISK
8.2.1.1 The Mean The mean, which is the most commonly used measure, is also referred to as the average of the results but its full name is the arithmetic mean. The mean (XMean) is obtained by summing (∑) all of the individual values (Xi), from i=1 to N, and dividing the total by the number of values (N): XMean
=
∑ Xi N
The mean is a good measure of central tendency when the data contain values that are spread fairly evenly over the range of values. If the data contain a small number of very high or very low values then the mean value will be significantly distorted by these, and the subsequent mean will be less ‘meaningful’ or useful to represent central tendancy. Considering the data presented in Table 8.1, the mean value for the men was: XMean
Men
= 63.1
and for the women was: Women
XMean
= (123 + 87 + 94+ 143 + 221 + 42 + 24 + 127 + 158 + 191) / 10 = 121.0
An examination of the two sets of data shows that in the case of the men there were 4 values greater and 6 values less than the mean; whilst for the women there were 6 values greater and 4 values less than the mean. The mean values have therefore provided a good central representation of the two data sets, and so the mean values are therefore meaningful and useful measures. The mean is the statistic that is used in estimating sample parameters using interval and ratio scales and this value is used as the basis for parametric testing of data, which will be discussed later. The mean can, but should not be used to obtain average values for variables that have been measured on nominal or ordinal scales. Nominal scales are purely descriptive e.g. Yes/No, Male/Female, True/ False, and provide no relationships between the individual responses measured. Ordinal scales, for example select 1, 2, 3, 4 or 5 etc provide a relationship between the responses but only in terms of their rank order; there is no relationship between the differences in the rank order. 8.2.1.2 The Median The median value refers to the middle value in a dataset when the values have been placed in rank order. If there is an even number of values in the data set the median value is defined as the midpoint between the two middle values. Whilst the median value is the best measure of central tendency for data that have been classified on an ordinal scale, it can also be used to describe data measured on ratio and interval scales. Considering the data presented in Table 8.1, the individual values for the men can be placed in rank order in the following way:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
= (57 + 93 + 28+ 75 + 63 + 41 + 54 + 71 + 117 + 32) / 10
PLANN NG
DISASTER
Rank 1 2 3 4 5 6 7 8 9 10 Median Value 28 32 41 54 57 value 63 71 75 93 117
8-5
EMERGENCY
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
The median value for the men, in this case, was 60, which was halfway between the 5th and 6th values, i.e., 57+63=120, 120÷2=60. The individual values for the women can similarly be placed in rank order in the following way:
Rank 1 2 3 4 5 Median 6 7 8 9 10
Value 24 42 87 94 123 value 127 143 158 191 221 The median value for the women, in this case, was 125, which was halfway between the 5th and 6th values, i.e., 123+127=250, 250÷2=125.
MSC IN EMERGENCY PLANNING MANAGEMENT
The median value cannot be used to make statistical estimates of sample parameters. The median value is a good measure of central tendency when there are exceptionally high or low values in a data set because these so called outlier values will have less influence on the median value than they will have on the mean value. For this reason the median value is often used to describe such skewed data sets. However, the median is not a good measure where there are only a small number of values because the results can easily be distorted. In addition to providing a measure of central tendency for individual data sets, the median can also be used to summarise frequency data. For example, the data presented in Table 8.1 can also be presented in a frequency table, such as that shown in Table 8.2. Table 8.2: Frequency Distribution of the Men and Women’s Times to Complete a Crossword Time to completion, minutes
Men (f) (cf)
Women (f) (cf)
0 – 50
3
3
2
2
51 – 100
6
9*
2
4
101 – 150
1
10
3
7*
151 – 200
0
10
2
9
201 – 250
0
10
1
10
(f) = frequency (cf) = cumulative frequency For frequency distributions, the median value is not given by the middle value of the categories, in this case 101–150 minutes but by the category in which the middle value appears. In this case (because there are an even number of values), the median value is provided by the category in which the 5th/6th* values appear. For men this is the time range 51 to 100 minutes and for women it is the time range 101 to 150 minutes. 8.2.1.3 The Mode The mode, which is defined as the value that occurs most frequently in a data set, provides a measure of central tendency because it reports the most commonly occurring value in a data set. The mode can be used to describe ratio, interval, ordinal and nominal data. The mode therefore provides the
8-6
disaster
riskcrisis MANAGE RISK
only measure of central tendency that can be used with nominal data because it defines only the most commonly occurring value or category. There may, however, be more than one mode value in a data set if several values are equally common. Using the frequency data presented in Table 8.2, the mode value for men is 51–100 minutes (6 values) and for women it is 101–150 (3 values). It is worthwhile now assessing how the three measures of central tendency (mean, median and mode values) for the data presented in Table 8.1 compare with each other; see Table 8.3. Table 8.3: A Comparison of the Measures of Central Tendency for the Data Presented in Table 8.1 Measure of central tendency
Women
Mean
63.1
121.0
Median
60
125
Mode
51–100
101–150
Despite the differences in definition and the methods of calculation, the results are consistent. In fact for a distribution of results that follows a normal, Gaussian (after Gauss), or Bell curve distribution (see section 8.2.4), the results for these three measures would all be the same.
8.2.2 Measurement of Dispersion The measure of central tendency alone can often provide misleading information because this value gives no information about the spread of the values around the central value. There are a number of ways in which the spread of values can be expressed. 8.2.2.1 The Range The simplest measure of dispersion is the range of the results; this measures the difference between the top and bottom values in the data set. From the results presented in Table 8.1, the following ranges for the times for men and women to complete the crossword can be calculated as: Men
= (117 – 28) = 89
Women
= (221 – 24) = 197
Therefore, in this example, the range of values for the women was twice that of the men. This shows the greater variation or dispersion in the results obtained from the women.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Men
PLANN NG
DISASTER
8.2.2.2 The Inter-quartile Range The inter-quartile range provides a measure of the dispersion of values around the median value. The median divides a ranked data set into halves and the inter-quartile values further divide the data set into quarters. The upper quartile value is the mid-point between the median and the highest value and the lower quartile value is the mid-point between the median and the lowest value in the data set. Using the results presented in Table 8.1, the lower and upper quartiles are defined for the women as:
8-7
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 8-8
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
Lower quartile
Median
Upper quartile
24 42 87 94 123 127 143 158 191 221 Data sets can in fact be split into any number of divisions in order to indicate the dispersion of the data. A common approach is to split the data into divisions of 10%, which are then referred to as percentile values. The range, median and inter-quartile values, however, are all based on limited information from the data set; the standard deviation value, which is discussed later, provides more useful statistical information about a data set. 8.2.2.3 The Variation Ratio The variation ratio is a useful measure of dispersion for results where the mode has been used as the measure of central tendency. The variation ratio is defined as the proportion of the total number of values that are not equal to the modal value:
Variation ratio =
Number of non-modal values
Total number of values
From the data presented in Table 8.1: Variation ratio for men
=
(3 + 1) / 10
=
0.4
Variation ratio for women
=
(2 + 2 + 2 + 1) / 10
=
0.7
The value of the variation ratio obtained for the women again shows that the results for the women are more widely dispersed than the results obtained for the men. 8.2.2.4 The Mean Deviation The key concept behind any measure of dispersion is to define how much variation there is within the results and how closely the individual values are distributed about the mean value. If the deviations are small then the values will be closely packed around the mean, whereas if the deviations are large, they will be widely spread around the mean. One approach for defining this variation might be to determine the average deviation of all the values from the mean value. Unfortunately, this statistic will always be equal to zero, because of the way in which the mean value is defined. Consider the data presented in Table 8.1 and the resulting values obtained for the deviations from the mean value; see Table 8.4.
disaster
riskcrisis MANAGE RISK
Table 8.4: Calculation of the Mean Deviation for the Data Presented in Table 8.1 Men Women Participant Absolute Absolute number Value Deviation deviation Value Deviation deviation 57 -6.1 6.1 123 2
2
93 29.9 29.9 87 -34 34
3
28 -35.1 35.1
4
75 11.9 11.9 143 22 22
5
63 -0.1 0.1 221 100 100
6
41 -22.1 22.1
7
54 -9.1 9.1 24 -97 97
8
71 7.9 7.9 127 6
94 -27
2
27
42 -79
79
6
9 117 53.9 53.9 158 37 37 10
Mean
32 -31.1 31.1 191 70 63.1
0
20.7
121.0
0
70 47.4
It can be seen from the results presented in Table 8.4 that if a simple mean of the deviations is taken the result will always be zero. However, the mean deviation statistic, which uses the absolute deviation of the values from the mean, provides a measure of the size of the deviation by ignoring the arithmetic sign of the deviations. In this case, the absolute deviation for the women (47.4) is more than twice the value of the men (20.7). This statistic is very seldom used, however. 8.2.2.5 The Standard Deviation and Variance The standard deviation, which is defined as the square root of the variance of a data set, is an important statistic. The variance is defined as the mean of all the squares of the deviations of the values.
Variance
=
∑ d2
N
and
Standard deviation (S)
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
1
PLANN NG
DISASTER
= √ (Variance)
For example, taking the values for the men in Table 8.1, the variance and standard deviation can be calculated as shown in Table 8.5.
8-9
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 8-10
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
Table 8.5: Calculation of the Variance and Standard Deviation (men)
Value
Mean value
Deviation, d
Squared deviation,
(Value – Mean)
d2
57
63.1
-6.1
37.21
93
63.1
29.9
894.01
28
63.1
-35.1
1232.01
75
63.1
11.9
141.61
63
63.1
-0.1
0.01
41
63.1
-22.1
488.41
54
63.1
-9.1
82.81
71
63.1
7.9
62.41
117
63.1
53.9
32
63.1
-31.1
2905.21 967.21
N = 10
∑ d2 = 6810.9
Mean = 63.1
Mean (d2) = 681.09
This version of the standard deviation is used when the deviation of the sample of values (S) alone is required. The standard deviation of the total population (s), which is the more important statistic used in parametric tests (see section 8.3.1), is obtained by dividing the sum of the deviations by (N – 1) called Bessel’s correction. Therefore, in this example:
S = √ 6810.9 / 10
=
26.10
s
=
√ 6810.9 / 9
=
27.51
and
8.2.3 General Distributions In addition to presenting information about the central tendency and standard deviation of the data, researchers often wish to present information about the distribution of the data. If there is only a small amount of data, it may be appropriate to present the actual data values. However, this approach is unreasonable where the number of data points exceeds one hundred values, in which case, it would be more common to summarise the data in the form of, for example, a frequency distribution. In this case, the data are split into categories or class intervals for display either as a table or as a figure. For example, using the data presented in Table 8.1 for all 20 male and female participants, the data can be summarised in the format shown in Table 8.6.
disaster
riskcrisis MANAGE RISK
Table 8.6: Frequency Distribution of the Times to Complete a Crossword
Time to completion, minutes
Number of respondents (f)
Cumulative frequency (cf)
% cumulative frequency (%cf)
0 – 50
5
5
25
51 – 100
8
13
65
101 – 150
4
17
85
151 – 200
2
19
95
201 – 250
1
20
100
(f) = frequency (cf) = cumulative frequency An alternative method of display for this data is to present the data as a histogram shown in Figure 8.1.
If the histogram was presented with a single point at the top of the centre of each column and these points were joined together the resulting graph would be referred to as a frequency polygon. The cumulative frequency values presented in Table 8.6 represent the total number of items in the interval shown and less than the value shown. For example, 17 people completed the crossword in 150 minutes or less. The percentage cumulative frequency distribution for these values is shown in Figure 8.2.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Figure 8.1: Frequency Distribution of the Data Presented in Table 8.6
PLANN NG
DISASTER
8-11
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Figure 8.2: Percentage (%) Cumulative Frequency Distribution of Times to Complete Crossword
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Interestingly if the distribution was re-drawn using the cumulative frequency (and not the percentage cumulative frequency as shown), the resulting distribution is called an ogive, which for a normal distribution (sometimes called Gaussian or Bell curve distribution – see 8.2.4) resembles one side of an Arabesque or ogival arch. Note actual cumulative frequencies were, 5, 13, 17, 19, and 20). The horizontal axis or x-axis of a histogram uses a continuous scale with no gaps displayed between the value ranges; histograms should therefore only be used where the data can be presented on a continuous scale. Bar charts are used to display data that refer to parameters measured on a nominal scale. Using the mean values for completion of the crossword that were presented in Table 8.3, it is possible to compare the mean completion times for the men and women using a vertical or horizontal bar chart; such as that shown in Figure 8.3. Figure 8.3: Bar Chart of the Mean Times for Men and Women to Complete the Crossword
The main difference between a bar chart and a histogram is the fact that the independent variable scale is not continuous in a bar chart and merely represents discrete parameters or criteria.
8-12
disaster
riskcrisis MANAGE RISK
8.2.4 Normal Distributions (sometimes referred to as Gaussian or Bell curve distributions) If the measurement intervals in a histogram are reduced further and further the graph will eventually begin to look like Figure 8.4. When the increments in the histogram become so small that they cannot be distinguished they will produce a continuous distribution curve. Figure 8.4: Histogram Approximating to a Continuous Distribution
PLANN NG
DISASTER
EMERGENCY
From statistical theory, it can be shown that approximately: • 68.3% of all experimental results will fall in the area of a normal distribution between the values of (mean – s) and (mean + s); i.e. + or - one standard deviation from the mean. • 95.4% of all experimental results will fall in the area of a normal distribution between the values of (mean – 2s) and (mean + 2s); i.e. + or - two standard deviations from the mean.
MSC IN EMERGENCY PLANNING MANAGEMENT
If this distribution curve is symmetrical it will represent a Gaussian or normal distribution curve. Normal distribution curves are symmetrical about the mid-point of the x-axis. The mean, median and mode values of a normal distribution curve all occur at the same value and the ends of the graph will never actually reach the x-axis. In practice, most experimental curves only approximate to a normal distribution curve because the number of data points is insufficient to obtain the level of discrimination required. It should be remembered that the term ‘normal’ is used solely for statistical purposes and does not indicate or imply that curves that are described as ‘normal’ are the correct type of curve that should be obtained for all experimental results or that all other types of curve are wrong.
• 99.7% of all experimental results will fall in the area of a normal distribution between the values of (mean – 3s) and (mean + 3s), i.e. + or - three standard deviations from the mean. If a distribution has many results that are either particularly high or particularly low then the distribution will be skewed in the direction of these scores. When this skewness is particularly
8-13
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 8-14
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
pronounced, the use of the median or the mode values is more appropriate than the mean value. If the results in a distribution appear packed together to produce a ‘peak’ effect, the distribution is described as showing KURTOSIS.
8.3 Inferential Statistics Descriptive statistics simply describe or summarise data, they do not allow any generalised conclusions about the data to be reached. More often, researchers would prefer to test, for example, whether a research hypothesis (whether the hypothesis is Null or Directional) is correct or not, and whether the data obtained from the sample are representative of the total population or whether the results obtained from one sample group are significantly different from the results obtained from another sample group. In order to achieve this important goal, it is necessary to turn to inferential statistical tests. The first stage in using inferential statistics is to identify the type of tests required. There are basically three types of tests: • a measure of specific population characteristics; • a comparison or contrast of data, such as whether the data are greater, smaller or equal to a defined value; and • a measure of association or relationship between data sets. There are a very large number of statistical tests available and the problem for most researchers is a question of which one of all the tests available is the most appropriate for the data and the hypotheses under test in the research project.
8.3.1 Parametric and Non-parametric Tests Statistical tests are divided into these two types and these tests enable a researcher to calculate the probability (p) that the experimental results obtained could have occurred purely by chance. If the probability that the results could have occurred by chance is small enough then the results can be deemed to be significant, see section 8.3.2.2. The type of data available determines the choice of parametric and non-parametric tests. Parametric tests provide a higher level of statistical testing of research data. In order to carry out parametric tests, the data must have been obtained by using a scale with equal intervals; that is interval or ratio scales. Non-parametric testing is usually carried out on data from nominal and ordinal scales. Researchers do sometimes manage to get round this by using pseudo or quasi interval scales for ordinal data. This approach is frequently used, for example, when assessing people’s perceptions and attitudes through the use of Likert scales. In this case, a typical ordinal response scale may be presented as a pseudo or quasi interval scale by presenting it in the following format: strongly agree (1), agree (2), neither agree nor disagree (3), disagree (4) and strongly disagree (5). A second criterion for parametric testing is that the data should be, or closely approximate to, a normally distributed frequency. This means that if the data sample set is normally distributed, the mean of the data set will be meaningful as a measure of central tendency. Parametric tests
disaster
riskcrisis MANAGE RISK
generally compute or calculate statistical values using the mean as a basis for calculating/comparing differences, and thus need meaningful means which are only found in normal distributions. Some parametric tests are more sensitive to ‘normality’ of data than others, although some are quite robust to violation of this criterion. A third criterion for using parametric testing is that the errors associated with the data should be randomly distributed amongst the parameters associated with the measurements. Researchers should normally check that these three criteria are valid before completing the statistical testing. However, some statisticians claim that parametric testing is not sensitive to variations from these criteria, and that parametric tests are robust to abuse in terms of violating these criteria.
type of statistical testing is greater. The power of a statistical test refers to its ability to identify the level of probability of significance in the results and therefore this indicates the likelihood of Type 1 and Type 2 statistical errors. It is sometimes possible to perform data transformations to assist in using the more powerful Parametric tests available in most Statistical (computer) packages. The Box-Cox conversion is one which allows the data in a distribution which is clearly not normal to be converted to a more normal distribution to allow the more powerful Parametric tests to be used. It is recommended that where possible the issue of conducting Reliability analyses is considered before running any appropriate Statistical tests looking for probabilities of significance. It should be noted that not all data can be subjected to such Reliability analyses and care should be taken not to run such analyses on inappropriate data.
8.3.2 Selecting Statistical Tests for Hypothesis Testing First, it is possible to test for differences in population characteristics; second, it is possible to test for comparisons between data and third, it is possible to test for associations between data. Statistical tests for population characteristics are normally associated with the population mean or the nature of the population distribution. The selection of the most appropriate statistical test for comparisons and associations, however, depends on the experimental design that has been used, such as whether the same subjects were used to measure different variables or whether different subjects were used to measure the same variables. Additionally, whether the data available are ratio, interval, ordinal or nominal - see also sections 8.2.1.1 and 8.3.1 will affect the choice of the statistical test. The choices of statistical tests for use with two variables and two groups of subjects are summarised in Figure 8.5.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Whenever parametric testing can be carried out on a data set, non-parametric testing can also be carried out on the same data. However, the reverse of this statement is not true because nonparametric testing for rank order and categorical data requires fewer assumptions than parametric testing. In non-parametric testing, the underpinning rationale does not involve statistical parameters such as the mean and standard deviation or any other parameters defining a sample distribution. Wherever possible, researchers should always use parametric testing because the power of this
PLANN NG
DISASTER
8-15
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Figure 8.5: Selecting Appropriate Statistical Tests for Comparisons and Associations of Data
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The previous figure 8.5 although it provides information on some of the more commonly used tests, the selection is not exhaustive. The reader’s attention is drawn to the following which are also quite extensively used; • ANOVA (analysis of variance) – both Parametric and Non-Parametric alternatives are available for examining differences between more than two groups on variable(s). • Kolmogoroff – Smirnoff test - a Non-Parametric two group test of differences on variable(s). • Kendall Tau rank correlation coefficient – a Non-Parametric test which assesses the correspondence between two rankings and the significance. Other useful tests worth investigating are the MANOVA (Multiple analysis of Variance, Functional Discriminant analysis, Cluster analysis and Multiple Regression analysis.
8-16
disaster
riskcrisis MANAGE RISK
The following sections discuss some of the issues related to the specific statistical tests included in Figure 8.5 and the procedures for manually calculating the statistical tests. However, these tests can also be calculated using standard computer programs, such as Microsoft Excel, the Statistical Package for the Social Sciences (SPSS), and CSS Statistica to name some available. 8.3.2.1 Degrees of Freedom An important parameter in statistical testing is the concept of the number of degrees of freedom. Degrees of freedom (df) simply refer to the number of free choices in a sample population and is equal to (N – 1), where N is the number of values in the sample. This concept can be explained, for example, by considering a pack of 52 cards; there are choices available for the value of each card for the first 51 cards dealt face up onto a table. However, there remains only one possibility for the value of the 52nd card because there is only one card left available and therefore there are no choices remaining. 8.3.2.2 Hypothesis Testing of Population Distributions
PLANN NG
DISASTER
EMERGENCY
The chi-square (x2) sometimes written c2 one-sample test for testing a population distribution (non-parametric test) The chi-square test is a non-parametric test of significance for nominal data that is used to test whether an observed distribution is significantly different from the expected distribution. Data used in a x2 one-sample test must be grouped in categories and must be absolute values and not relative frequencies, such as percentages. For example, a researcher, who interviewed 140 subjects, asked television viewers to identify their favourite type of television programme from a choice of the following (4), documentary, sport, game show and drama categories. If there were no differences in the viewers’ preferences, the researcher would anticipate that 35 of the 140 subjects would identify each category of programme. The actual results obtained in this study are shown in Table 8.7. Table 8.7: Viewers’ Preferences for Television Programmes Programme type
Observed number of viewers, O
Expected number of viewers, E (140÷4)
Documentary 11 35 Sport
30 35
Game show
76
Drama
23 35
35
MSC IN EMERGENCY PLANNING MANAGEMENT
The most common statistical tests are for assessments of the mean of a population and the nature of the population distribution.
In this example, the null hypothesis is that there is no significant difference in the preferences of television viewers for documentary, sport, game show and drama programmes. Therefore the expected frequency in each category would be 35 (that is the sample population of 140 divided by the 4 categories of programmes). The x2 statistic is calculated from the equation:
x2
=
∑{(O – E)2 / E}
8-17
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Therefore for the data presented in Table 8.6, the x2 statistic is: x2 = {(11 – 35)2 / 35} + {(30 – 35)2 / 35} + {(76 – 35)2 / 35} + {(23 – 35)2 / 35} = 16.46 + 0.71 + 48.03 + 4.11 = 69.31
MSC IN EMERGENCY PLANNING MANAGEMENT
The significance of this value is assessed from the x2 distribution, which is available from statistical tables. Most Statistical computer packages have ‘built-in’ tables which are used to generate ‘p’ values (probabilities of significance). Most statistics books show such tables, for example see: Berenson, Levine and Krehbiel (2006). Basic Business Statistics: Concepts and Applications, Pearson International Edition. See also Bibliography / Suggested reading. The critical values are dependent on the number of degrees of freedom in the distribution. In this example there are 4 categories and the number of degrees of freedom is therefore 3 (N-1). The critical x2 values for 3 degrees of freedom are 7.81 at p<0.05 and 11.34 at p<0.01; therefore the null hypothesis can be rejected at the p<0.01 level and it can be stated that there were significant differences in the choices of favourite television programmes by the viewers. Note to reader: The following is a simplistic generalisation to aid comprehension. The probability of significance (p) values which have been generally accepted by researchers are; • For p≤ 0.05 indicates significance • For p≤ 0.01 indicates increased significance (very significant) • For p≤ 0.001 indicates even more increased significance (highly significant) Those represent chance occurrences at 5% (0.05), 1% (0.01) and 0.1% (0.001). Thus when carrying out statistical tests the ‘test statistic value’ (which varies according to the test) although important, is not as crucial as its probability of significance (p). If a ‘p’ value is greater than 0.05 then technically this is a non-significant result, although interestingly more and more research papers are quoting results where p is greater than 0.05 and up to 0.07 as “approaching” significance. For valid results from the x2 distribution test, there should always be a minimum number of five values in each of the categories. 8.3.2.3 Hypothesis Testing for Comparisons of Data If the research required a comparison of the same variable across two populations then the results should be compared using the x2 two-sample test for nominal data, the Mann-Whitney U test for ordinal data and the two sample t-test for parametric data. If the research requires a comparison of different variables across the same population then the results should be tested using the sign test for nominal data, the Wilcoxon T test for ordinal data and the paired sample t-test for parametric data. The paired sample t-test (parametric test) Consider an experiment in which subjects were asked to first estimate the time for the completion of a crossword and then measure how long it actually took them to complete it. The results for this experiment are presented in Table 8.8.
8-18
disaster
riskcrisis MANAGE RISK
Table 8.8: A Comparison of the Estimated and Actual Times for Completing a Crossword using the Paired-sample t-test
Time to complete crossword, minutes Estimate (E)
1
57
123
-66
4356
2
93
87
6
36
3
28
94
-66
4356
4
75
143
-68
4624
5
63
221
-158
24964
6
41
42
-1
1
7
54
24
30
900
8
71
127
-56
3136
9
117
158
-41
1681
10
32
191
Actual (A)
d
-159
d2
25281
Total
-579
69,335
Mean
-57.9
6,934
63.1
121.0
The null hypothesis is that there is no significant difference between the participantsâ&#x20AC;&#x2122; estimates and the actual times to complete the crossword. A paired-sample t-test is used to test the significance of the difference between the estimated and the actual values. It has been discussed previously how the means of samples from a population distributed themselves normally about the true population mean. In the same way, the differences between pairs of sample means will also be described by a normal distribution. The first step in the pairedsample t-test is a calculation of the differences (d) between the estimates and the actual times for each participant and a calculation of the mean of all these values. If the differences between the estimates and the actual values were purely down to chance then the positive and negative values would cancel each other with the result that the mean of the differences would be very close to zero. If, however, the mean value deviated by a large amount from zero then the differences in the results would be considered to be statistically significant. How large this difference needs to be away from zero for the results to be significant depends on the number of data points and their variability. The standard error of the results reflects these factors and therefore the standard error will be larger the smaller the number of data points and the larger the variation in the data. A test statistic (t) is derived from the mean of the deviations (Md) and the standard error of the mean of the differences (SEd).
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Participant number
PLANN NG
DISASTER
8-19
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
t = Md
SEd
When the value of t has been calculated it is compared with the values presented in standard statistical tables in order to determine whether the test value of t is greater than the critical value shown in the statistical table. If the calculated value is greater (at the defined level of confidence) than the critical value then the results are deemed to be significant at the confidence level specified. The standard error of the differences (SEd) is defined as the square root of the sum of squared deviations between the individual values and the mean of the deviations (SSd) divided by the number of degrees of freedom (df) and the number of values (N). The sum of the squares (SS) is given by: SSd
=
∑(d – Md )2
which is equal to:
MSC IN EMERGENCY PLANNING MANAGEMENT
SSEd
=
∑d2 – (∑d)2
N
Therefore, the standard error of the mean of the differences is: SEd
=
√ SSd / (N x df)
In the example above: Md
=
∑d / N = – 579 / 10 = – 57.9
SSd
=
∑d2 – (∑d)2 = 69,335 – {( – 579)2 / 10}
N
=
69,335 – {335,241 / 10} = 35,811
SEd
=
√ SSd / (N x df) = √ 35,811 / (10 x 9)
=
19.95
=
Md / SEd = 57.9 / 19.95 = 2.90
Therefore, t
Because it is only differences that are important it does not matter whether the value of Md is positive or negative. In order for the result to be significant, the value must be greater than the values of t for 9 degrees of freedom, which at p<0.05 is 2.26 and at p<0.01 is 3.25. Because the calculated value for t is 2.90, the differences between the estimated and actual results are deemed to be significant at the p<0.05 level but not at the p<0.01 level. The conclusion from this statistical test is therefore that the estimates for the time to complete the crossword were on average significantly (p<0.05) shorter than the actual times taken and because the result is significant, this conclusion is valid for the whole population of people completing the crossword and not just for the sample in the test. The conclusion therefore is that the people were optimistic about the time taken to complete the crossword.
8-20
disaster
riskcrisis MANAGE RISK
The Wilcoxon T test (non-parametric test) The Wilcoxon matched pairs, signed-ranks test first calculates the differences between the values and then ranks the differences based on size irrespective of whether the value is positive or negative; see Table 8.9. Table 8.9: A Comparison of the Estimated and Actual Times for Completing a Crossword using the Wilcoxon T Value Time to complete crossword, minutes Participant number Estimate Actual (E) (A)
1
57
123
PLANN NG
DISASTER
EMERGENCY
d
– 66
Rank of d
6.5
93
87
6
2
3
28
94
–66
6.5
4
75
143
–68
8
5
63
221 –158
9
6
41
42
–1
1
7
54
24
30
3
8
71
127
–56
5
9
117
158
–41
4
10
32
191 –159
10
The second stage is to determine whether there are more positive or negative values for the difference. In this case, there are 8 negative and 2 positive values. The third stage is to add up the ranks of all the values in the group with the least number of values in it (which is the group of positive values in the example here i.e., d=6, d=30, ranked 2, and 3 respectively); this gives the T statistic a value of 5 (obtained from ranks 2 + 3). In this test the value of T is small where most of the values are either positive or negative and large if the values are fairly evenly distributed about the zero value. The critical T value is obtained from standard statistical tables; the number of values taken in this test is equal to the total number of values minus any values where the difference is equal to zero. In this case there are no differences equal to zero so the number of values used is 10. The statistical tables show that T values are 8 at p<0.05 and 3 at p<0.01; therefore, because the calculated T value is less than 8 but greater than 3, the differences in the results are significant at p<0.05.
MSC IN EMERGENCY PLANNING MANAGEMENT
2
8-21
EMERGENCY
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
The Sign test (non-parametric test)
MSC IN EMERGENCY PLANNING MANAGEMENT
The t-test used original values and the Wilcoxon T test used ranked values in the statistical tests for comparing different results from the same group. However, even when using ranked scores, there may be some concern that the differences in values of, for example, 5 units between 5 and 10 and between 20 and 25 are the same. Therefore instead of assessing the magnitude of the differences in the absolute scores or the rank order of the positive and negative differences, the Sign test only assesses how many differences are positive and how many differences are negative. Similarly to the Wilcoxon T test, any values where the differences are zero are ignored in the test. From the data presented in Table 8.9, there are 2 positive and 8 negative differences. The smaller number is always taken as the test statistic in the Sign test (which is therefore 2 in this case). From statistical tables, it can be found that the maximum number of values that should be obtained for 10 valid results are 1 at p<0.05 and 0 at p<0.01. Therefore because the test statistic obtained is 2, which is greater than these critical values, the results are not considered to be significant. Although this result is different from the results obtained from the t-test and the Wilcoxon T test, this is not too unusual. The reason for this difference is that the Sign test makes use of less information than the other two tests and is therefore a less sensitive test. The x2 two-sample test Consider the responses on a Likert scale from a group of academics and a group of students to the statement that post-graduate students work very hard for their degrees; see Table 8.10. Table 8.10: Views of Academics and Students on the Statement that Students Work Very Hard in Order to Obtain a Post-graduate Qualification
Number of responses (expected responses) Group
Strongly agree Agree Disagree
Strongly disagree
Total responses
Academics
9 (12.8)
14 (13.9)
12 (12.3)
18 (13.9)
53
Students
15 (11.2)
12 (12.1)
11 (10.7)
8 (12.1)
46
Total responses 24 26 23 26 99 The null hypothesis in this example is that the groups and views are not related. It is assumed that the groups are independent of each other and that the data are ordinal. The first step is to calculate the expected frequencies for each cell in Table 8.10. For example, the expected frequency for the group ‘academics’ and the category ‘Agree’ is given by: Expected frequency = 26 x 53 / 99 = 13.9 All other expected frequencies which are shown in Table 8.10 are calculated in the same way. The x2 statistic (see section 8.3.2.2) is now calculated for each cell and the results summed:
8-22
x2
= 1.13 + <0.01 + 0.01 + 1.21 + 1.29 + <0.01 + 0.01 + 1.39
= 5.04
disaster
riskcrisis MANAGE RISK
The number of degrees of freedom associated with the test are 3 (4 – 1) for the categories and 1 (2 – 1) for the groups. This gives a total of 3 (3 x 1) degrees of freedom for the two groups and four categories. The critical x2 values are now obtained from statistical tables and these are 7.81 at p<0.05 and 11.34 at p<0.01. Therefore, because the test statistic is less than the critical values, the null hypothesis that there is no significant difference between the views of the two groups is accepted. The Mann-Whitney U test (non-parametric test) This test is used for comparing data for the same variable from two unrelated samples, such as a group of men and a group of women. The test is normally only applied to small samples, such as less than 25 in each group. To determine the U statistic, the values obtained from the two samples, such as those presented in Table 8.1, are ranked as though they originated from the same sample; see Table 8.11. Table 8.11: Ranking Values for the Mann-Whitney U Test
Men (N1) Values, minutes Rank
Women (N2)
Values, minutes Rank
57 7
123 15
93 12
87 11
28 2
94 13
75 10
143 17
63 8
221 20
41 4
42 5
54 6
24 1
71 9
127 16
117 14
158 18
32 3
191 19
Totals
631 R1 = 75
1210 R2 = 135
Clearly if the sum of the values in one group is smaller than the sum of the values in the other group then the sum of the ranks for this group will also be smaller than the sum of the ranks in the other group. The U statistic is now calculated for the group that has the largest sum of the values and is given by the equation:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
PLANN NG
DISASTER
U = (N1 x N2) + {N1 x (N1 + 1)} / 2 – R In this case, the R2 value is the greater and therefore the U value is: U = (10 x 10) + {10 x (10 + 1)} / 2 – 135
= 100 + 55 – 135
= 20
8-23
EMERGENCY
crisisdisaster RISK
MANAGE
The critical values for the U statistic are obtained from standard statistical tables for the U statistic using a sample size of 10 for both groups. These tables show that the U value at p<0.05 is 23 and at p<0.01 is 16. Because the calculated U statistic must be ≤ critical U statistic value, it can be stated that the differences are significant at the p<0.05 level. Therefore the conclusion from the statistical test is that the women took significantly (p<0.05) longer to complete the crossword than the men. The two-sample t-test (parametric test) The two-sample t-test is calculated in a similar way to the one-sample t-test discussed earlier; the difference is that the two-sample t-test is used to compare the results for two unrelated groups and measures of the same variable. Using the data presented in Table 8.1, the first stage is to calculate the sum of the squares of the differences for the men (SSM) and for the women (SSW); see Table 8.12. Table 8.12: Calculation of the Two-sample t-test using the Data Presented in Table 8.1
Men (M)
M2
Women (W)
W2
MSC IN EMERGENCY PLANNING MANAGEMENT
57
3,249
123
15,129
93
8,649
87
7,569
28
784
94
8,836
75
5,625
143
20,449
63
3,969
221
48,841
41
1,681
42
1,764
54
2,916
24
576
71
5,041
127
16,129
117 13,689
158
24,964
191
36,481
32
1,024
Totals 631 46,627 1,210 180,738
Therefore: MM
=
63.1
SSM
=
∑ M2 – (∑ M)2 / NM = 46,627 – 39,816
=
6,811
MW
=
121.0
SSW
=
∑ W2 – (∑ W)2 / NW = 180,738 – 146,410
=
34,328
SE
=
√(SSW + SSM) x (1/NW + 1/NM) / (dfW + dfM)
=
17.49
t
=
(MW – MM ) / SE
=
3.31
and
8-24
crisis
PLANN NG
risk
disaster
riskcrisis MANAGE RISK
From standard statistical tables, the critical values for t with 18 degrees of freedom are 2.10 at p<0.05 and 2.88 at p< 0.01. Therefore, the null hypothesis can be rejected with the conclusion that, on average, men take significantly (p<0.01) less time than women to complete the crossword. 8.3.2.4 Hypothesis Testing for Associations of Data If the values of two parameters are compared, the tests described previously will identify whether they are significantly different, such as the comparison between the estimated and actual times to complete a crossword shown in Table 8.8. These tests, however, do not indicate whether people who provided a high estimate of the time also took a long time to complete the crossword and vice versa. For this, it is necessary to plot a graph and show whether there are any relationships between the values; such as that depicted in the scatter plot shown in Figure 8.6. Figure 8.6: Relationship Between the Estimated and Actual Times to Complete a Crossword
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The scatter plot, therefore, provides a visual estimate of whether the two parameters have any relationship. In this case there appears to be a possible trend that the higher the estimated value (the independent variable) the higher the actual time (the dependent variable) taken to complete the crossword. There are two aspects to a correlation; the first is its significance and the second is its strength. It is possible to have a very significant but small correlation at one extreme and an insignificant strong correlation at the other extreme, as previously mentioned any test statistic (e.g. correlation coefficient) is important, but it is the probability of its significance (p) that is crucial. Generally correlations are defined as either positive, in which case the higher the value on the independent axis the higher the value on the dependent axis, or negative, in which case the higher the value on the independent axis the lower the value on the dependent axis. The correlation value therefore varies between â&#x20AC;&#x201C;1 (a perfect negative or inverse correlation), through 0 (no correlation whatsoever) to +1 (a perfect positive or direct correlation).
PLANN NG
DISASTER
Tests for associations are very important in some social sciences research and the two most common tests used for this purpose are the Pearson product-moment correlation coefficient, which uses raw interval data, and the Spearman correlation, which uses the order of raw ordinal data.
8-25
EMERGENCY
crisisdisaster RISK crisis
PLANN NG
risk
MANAGE
The Pearson correlation (r) The Pearson correlation (r), which is also referred to as the product–moment correlation, can be calculated using paired data values (X, Y) with the following formula: r =
N ∑{XY} – ∑X∑Y
√ {N ∑X2 – (∑X)2} {N∑Y2 – (∑Y)2 }
The X value represents the independent variable, Y represents the corresponding dependent variable and N represents the number of pairs of data for X and Y. This formula can now be used to assess the data presented in Table 8.8 and illustrated in Figure 8.6; see Table 8.13. Table 8.13: Calculation of the Pearson Correlation Coefficient for the Data Presented in Table 8.8
Estimated value, X
Actual value, Y
XY
X2 Y2
MSC IN EMERGENCY PLANNING MANAGEMENT
57
123 7,011 3,249 15,129
93
87 8,091 8,649 7,569
28
94 2,632 784 8,836
75
143 10,725 5,625 20,449
63 41
221 13,923 3,969 48,841 42 1,722 1,681 1,764
54
71 117 32
Totals
631
24 1,296 2,916
576
127 9,017 5,041 16,129 158 18,486 13,689 24,964 191 6,112 1,024 36,481 1,210
79,015
46,627
180,738
From the data presented in Table 8.13, the value for r can be calculated as:
r
=
(10 x 79,015) – (631 x 1,210)
√ {10 x 46,627 – 6312} {10 x 180,738 – 1,2102 }
=
26,640 / √(68,109 x 343,280)
=
0.17
This result confirms the visual impression obtained from the scatter plot that there is a positive correlation between the values of X and Y but that the correlation is fairly weak. The Spearman correlation (rs) One of the problems with the Pearson correlation coefficient is that outlier values can significantly influence and distort the statistical test. This problem can be minimised by using a Spearman
8-26
disaster
riskcrisis MANAGE RISK
correlation, which first ranks the scores and then uses the ranked data sets rather than the raw data values in the calculation. The Spearman correlation again produces a correlation value between –1 and +1, with the same meaning as that for the Pearson correlation coefficient. For this test, the data used in the previous example are ranked in the way shown in Table 8.14. Table 8.14: Calculation of the Spearman Correlation Coefficient for the Data Presented in Table 8.8
Estimated Actual value, X value, Y
Difference (X – Y) Rank, Rank, X Y d d2
57 123 5 5 0 0 3 7 49
4
28
94
1
– 3
9
75 143 8 7 1 1
63
221
6
10
– 4
16
41 42 3 2 1 1 54 24 4 1 3 9 71 127 7 6 1 1 117 158 9 8 1 1
32
191
2
9
N = 10
– 7
49
∑ d2 = 136
When the scores have been converted to ranks these values can be used in the same formula as that used for the Pearson correlation or in the simpler equation: rs = 1 – 6 ∑ d2
N (N2 – 1)
Therefore, in this case, the Spearman correlation coefficient is: rs = 1 – 6 x 136
10 x (100 – 1)
= 0.18
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
93 87 10
PLANN NG
DISASTER
This result, as expected, is similar to the result obtained previously using the Pearson correlation.
8-27
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
8.4 Factor Analysis Measuring concepts in the social sciences can be quite complex because individual concepts are often affected by a number of items. If each of the items affects the individual concept then it might be anticipated that these items would therefore be interrelated. Items that correlated together well would constitute a factor and the process of factor analysis is the statistical technique that identifies these factors. Factor analysis can be used in the following ways: • a large number of items, variables or questions related to an issue can be reduced to a smaller number of factors by identifying which items produce similar responses from a sample population and these may therefore be grouped together as a single factor. This represents an example of exploratory factor analysis;
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• where the responses to a number of items or questions have been identified as indicative of, for example, a particular personality characteristic, factor analysis can be used to confirm that the responses are related. The technique is therefore used to test the validity of the questionnaire and this represents an example of confirmatory factor analysis. These are more complex statistical techniques and information about their application should therefore be obtained from specialist texts such as Bryman and Cramer (2001). An interesting and less complex way of looking at “groupings” in data is using Cluster Analysis - see section 8.3.2.
8.5 Linear Regression and Structural Equation Modelling In addition to identifying whether there is an association between two sets of data, it is often useful to define the exact relationship between the two variables, which requires defining the line of best fit between the two data sets. The technique used to define the line of best fit is referred to as linear regression. If a number of interactions between variables are involved in a research project it is also possible to determine how these variables interact with each other using a technique referred to as structural equation modelling.
8.5.1 Linear Regression Any straight line through a series of points is defined by the equation: Y = mX + c Where m is the slope of the line and c is the intercept on the y-axis, when the value of X is equal to zero. When the values of m and c are known, it is possible to predict values of the dependent variable (Y) for any value of the independent variable (X). For the regression of Y on X, the values of m and c are calculated from the following equations: m = N ∑{XY} – ∑X∑Y
N ∑X2 – (∑X)2
and c = ∑Y – m ∑X
N
For the data presented in Table 8.8 and presented in Figure 8.6, the values for m and c can be calculated from the information presented in Table 8.15.
8-28
disaster
riskcrisis MANAGE RISK
Table 8.15: Calculation of the Linear Regression Equation for the Data Presented in Table 8.8
Estimated time (X), Actual time minutes minutes (Y),
X2
XY
57
123
3,249 7,011
93
87
8,649 8,091
28
94
784 2,632
75
143
5,625 10,725
63
221
3,969 13,923
41
42
1,681 1,722
54
24
2,916 1,296
71
127
5,041 9,017
117
32
158 13,689 18,486 191
1,024 6,112
Totals 631 1,210 46,627 79,015
Therefore, m = 10 x 79,015 – 631 x 1,210
10 x 46,627 – 631 x 631
=
0.39
and, c = 1,210 – 0.39 x 631
=
10 96.4
This regression line can be superimposed onto the scatter plot presented in Figure 8.6 to show how well this fits the data; see Figure 8.7.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
PLANN NG
DISASTER
8-29
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Figure 8.7: Linear Regression for the Relationship Between the Estimated and Actual Times to Complete a Crossword
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
8.5.2 Structural Equation Modelling Structural equation modelling or path analysis entails the use of the more complex version of linear regression analysis called multiple regression analysis, see also Section 8.3.2 (Bryman and Cramer, 2001). The main purpose of structural equation modelling is to provide quantitative assessments of the causal links between interconnected variables. The technique allows the relationships between several variables to be explored and, linked with factor analysis, provides a very useful technique in the social and behavioural sciences. The technique is particularly useful for assessing abstract concepts such as risk perceptions and attitudes. More detailed information on the application of this technique should be obtained by consulting specialist texts, such as Bryman and Cramer (2001).
8.6 Main Points The following key points have been raised and discussed within this Unit: • Statistics can be used to provide simple descriptions of results, tables, graphs and numerical measures. • The mean, median and mode provide measures of the central tendency of data sets. • For a normal distribution the values of the mean, median and the mode are the same. • The range, variation ratio, mean deviation and standard deviation provide measures of the dispersion of data sets. • Descriptive statistics describe or summarise data but they do not enable generalised conclusions to be reached about the results. • Inferential statistics enable hypotheses about population characteristics, comparisons of data and associations of data to be tested.
8-30
disaster
riskcrisis MANAGE RISK
• Parametric statistical tests enable statistical tests of data obtained from interval and ratio scales. • Non-parametric tests enable statistical tests of data obtained on nominal and ordinal scales. • Factor analysis enables a range of variables that behave in a similar way to be identified and grouped together to form a single factor. • Structural equation modelling enables the interactions that exist between variables to be modelled in order to generate models of organisational performance.
8.7 Study Questions
and should not be forwarded to the University. 1. Outline how the statistical analysis of performance data could be used within your organisation. 2. Discuss why inferential statistics may provide more useful information for an organisation than descriptive statistics. 3. Using a current political issue that is reported in a quality newspaper article and also in a popular newspaper article, calculate the mean, median and mode of the length of words used in the two newspaper articles.
8.8 Bibliography / Suggested Reading Berenson, Levine and Krehbiel (2006) Basic Business Statistics: Concepts and Applications. Pearson International Edition. Bryman, A. and Cramer, D. (2001) Quantitative Data Analysis with SPSS Release 10 for Windows, London: Routledge. Bryman, A. (2008) Social Research Methods. New York: Oxford University Press. Clegg, F. (2006) Simple Statistics. London: SAGE.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
You should now write approximately 300 words in answer to each of the questions below. We believe that this is an important exercise that will assist your comprehension of the material and aid your progress on the course. Your answers are intended to form part of your own course notes
PLANN NG
DISASTER
Dodge, Y. (2003) The Oxford Dictionary of Statistical terms. Oxford University Press. ISBN: 0-19-920613-9. Field, A. (2009) Discovering Statistics using SPSS. London: SAGE. Freeman, L. C. (1965) Elementary Applied Statistics. New York: John Wiley and Sons (pp. 40-43).
8-31
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 8-32
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Gilbert, N. (2008) Researching Social Life. London: Sage Publications. Nachmias, D. and Nachmias, C. (1976) Research Methods in the Social Sciences. Edward Arnold Ltd. Robson, C. (2002) Real World Research. UK: Blackwell.
MSC IN EMERGENC Y PLANNING MANAGEMENT
UNIT 9 Writing a Research Proposal and Dissertation
disaster
riskcrisis MANAGE RISK
9 Unit Nine: Writing a Research Proposal and Dissertation 9.1 Aim and Objectives of this Unit Research and development are important subject areas in academia but they are also equally important in other social, commercial and industrial contexts. Therefore, whilst understanding the underpinning theories of research is critical from an academic perspective, it is also an equally significant issue within these other spheres. There are three important aspects of implementing a research programme, which are related to defining the scope of the project and explaining how the study will be carried out; carrying out the research programme; and writing up the results in a form that accurately reflects the methods, results and conclusions in order that the research can be read and referred to by others working in the field. successfully carrying out research. The aims of this Unit are to discuss the first and last points described above, viz.: â&#x20AC;˘ defining the scope of the research project and explaining how the project will be carried out; and â&#x20AC;˘ writing up the results in a form that accurately reflects the methods, results and conclusions in order that the research can be referenced and understood by those studying in a similar area. This will be outlined in the context of writing up results in the form of a dissertation. Advice on the formal process of submitting a dissertation to the Civil Safety and Security Unit will also be provided.
9.2 General Issues Honesty in research is essential. Honesty is related to the values that a researcher places on personal integrity and to their responsibilities to the people and organisations taking part in the study. It is assumed that what is written in a research proposal or dissertation is the original work of the author. It will also be assumed that the ideas presented in the dissertation are original unless reference has specifically been made in the text to the contrary. Copying or presenting the work of other researchers is clearly an unacceptable practice. In this respect, authors should be fully aware of the issues related to plagiarism. Very little research, however, is entirely original and most builds on the ideas and results of previous studies and incorporating these into a dissertation or proposal is normal practice, which is quite acceptable provided that the sources of the ideas are referenced.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The previous Units in this Module have discussed in detail a number of factors associated with
PLANN NG
DISASTER
In addition to plagiarism, there are several other issues of research honesty, relating to the honesty of the information that is presented in the document. It is important that a researcher describes exactly, for example, how the research project was conducted, the make up of the sample population, the results obtained and the statistical tests that were carried out on the results. Researchers should not be selective in their presentation of results; for example, by ignoring those results that do not fit with the proposed theory or adjusting other results in order to ensure that they fit the theory more closely.
9-3
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
The researcher must also be honest with the subjects, groups or organisations taking part in a research project. Nothing should be done to compromise their position, feelings, perceptions or attitudes. The researcher should therefore plan the project in order to be sensitive to the perceptions and feelings of those from minority communities.
MSC IN EMERGENCY PLANNING MANAGEMENT
More broadly, social research may raise ethical, moral, financial, legal and political considerations. In fact, one could argue that all research is political in the social sense as there are always questions concerning power relations and issues of access, privacy, honesty, trust, confidentiality and anonymity. Social science research invariably makes statements about the nature of societies and the members of these societies. It is important, therefore, to understand the different types of research statements that may be made. Positive statements relate to what exists, whilst normative statements relate to what an author feels does or should exist. It could be said therefore that positive research statements report factual information, which should be indisputable, whilst normative statements present only the views and opinions of the researcher and therefore they may be disputable. Other researchers should be able to reproduce the results that give rise to positive statements in future projects; however, there is no necessity that normative statements should either be accepted or be reproduced by others because these statements may be based solely on the personal perceptions, beliefs and/or attitudes of the author towards the subject rather than on objective research.
9.2.1 Identifying a Research Problem Before selecting and defining a research project, it is important to appreciate what is meant. Research has been defined by Walliman (2001) as: • a gathering of facts or information; • moving facts from one setting to another; and • an esoteric activity removed from practical life. There are in reality a very large number of definitions of research, some of which are more appropriate than others to academic work. Whichever definition is considered to be the most appropriate, research should be differentiated from knowledge and logic. Everyone acquires knowledge from his or her experiences of life and society achieves collective knowledge by pooling or integrating the lessons acquired from each individual’s experiences. However, knowledge acquired solely from life’s experiences is limited because it is normally acquired from a random sample of activities and any conclusions reached are not normally tested. Experience and acquired knowledge, however, still form the basis of much research. Logic provides the means whereby conclusions can be reached through the processes of inductive and/or deductive reasoning. Deductive reasoning begins from general information and moves towards reaching specific conclusions, whereas inductive reasoning starts with specific information and moves towards reaching general conclusions. The application of inductive and deductive reasoning coupled with acquired knowledge forms the basis of scientific research. The results of scientific research are invariably exposed to rigorous testing through public examination and constructive criticism. Scientific research displays a number of characteristics (Walliman, 2001):
9-4
disaster
riskcrisis MANAGE RISK
• it is generated by a question; • it necessitates clarification of a goal; • it entails a specific programme of work; • it is aimed at increasing understanding by interpreting facts and/or ideas; • it involves reasoned arguments to support research conclusions; and • it follows a reiterative process. An alternative view of research, which is put forward by many social scientists, is that it is related to its social context and that the world’s experiences are a creation of individuals’ minds. In other words, results obtained are a function of the researchers’ preconceptions and beliefs (Walliman, 2001). The interpretive approach expounded here is therefore that research can only study the appearance of things rather than the things themselves and this leads to an understanding of issues rather than to the causal relationships that exist between the issues.
When research in sociology is compared with that in the natural sciences, certain differences are apparent. It is very difficult, for example, for sociologists to undertake experiments in the same way that natural scientists can and sociological factors cannot usually be defined with the same precision. Some sociologists, who are referred to as positivistic sociologists, have attempted to emulate research in the natural sciences. From a positivist perspective sociology is a science. There is an assumption that society has objective and regular features, like those that have been discovered in the natural world, and that these can therefore be measured using similar approaches and techniques to those used in the natural sciences. Sociologists, like natural scientists, use a variety of theories about the societies that they study. These are essentially a collection of logically linked ideas and generalisations that attempt to demonstrate the reasons behind particular events and processes. These theories therefore provide a conceptual framework within which social science research can be carried out. Without underpinning theory, natural scientists and sociologists could only observe. A researcher, however, should have a clearly defined problem within which to conceptualise the observations. Social science theories may be very elaborate or they may be little more than a number of assumptions and generalisations. In many ways, selecting the right research project is the key to a successful research proposal and dissertation. A project should be one’s own research and should be based on sound theoretical and methodological considerations that have been discussed within this Module. The Module has been concerned with central research issues, the extent to which these can be examined by a number of theoretical approaches and the methods by which these issues may be addressed. The interchange between the research question, the underpinning theories and the research methodology provides the framework for the research and provides the basis on which the research proposal and dissertation will be assessed. The key issues that must be considered within a research project are:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
These two views are clearly polarised and it is quite possible to study both causal relationships and the meanings of the causal relationships. In fact, most research lies on the spectrum between these limits and to ignore this fact diminishes the potential benefits that can be obtained from effective research.
PLANN NG
DISASTER
9-5
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 9-6
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
• the definition of a substantive research question, problem or issue that is related to the course of study; • an assessment of the practical and ethical problems that may arise before, during and after the research work; • identification of the central elements of one or more theoretical approaches within the subject area that will address the research question; • identification and description of one or more appropriate methods of investigation in order to address the research question; and • interpretation of the results of the research work and an assessment of the wider implications of the conclusions reached. A single, short piece of research work, such as that required for an MSc degree, is unlikely to provide results and conclusions that will change the way in which the world operates. However, as with all research, it should, for example, move existing research methods in the subject area forward, apply existing methodology in new organisational settings or confirm results obtained by earlier studies. 9.2.1.1 The Types of Research Problem There are fundamentally two types of research problem that can be identified: pure research that involves developing theories and applied research that involves testing existing theories in the real world. However, it is often easier to consider slightly wider definitions of research using the following headings: • Exploratory research – which involves investigating a new problem or issue; • Testing-out research – which involves determining the limits of the application of exploratory work; and • Problem-solving research – which involves starting out with a practical problem from the real world and applying the available knowledge and intellectual resources to solving the problem. Intensive research sets out to identify how causal processes work in a particular situation; this approach is often exemplified by the undertaking of case studies. Extensive research, on the other hand, sets out to describe and explain, usually through statistical analysis, how large groups of people who have, for example, similar perceptions, attitudes or social climates, behave under certain defined conditions. There are many arguments for and against each of these research approaches and as is normally the case in arguments of this type, the viewpoints are strongly influenced by which camp the protagonists reside in. Those advocating extensive research will argue, for example, that the intensive research approach fails to produce objective results that are relevant to anything other than the specific conditions in which the project was carried out. However, those advocating intensive research will argue that their investigations solve real problems in the real world.
disaster
riskcrisis MANAGE RISK
Much research carried out in the pure and social sciences is inter-disciplinary because it involves working and understanding theories and problems from many social and scientific disciplines. Frequently, projects will require the researchers to take into account social, moral and legislative aspects of the organisational setting in which the investigation is conducted. Whilst this certainly makes the work very interesting, it does require researchers to broaden their views about what issues are both relevant and important to a project. This will often require the researcher to consider several sources of information and/or data and several techniques in order to support the project. In selecting a research project, it is important to acknowledge the strengths and weaknesses of the researcher and the implications of the project in terms of financial and time commitments. 9.2.1.2 The Context and Setting of the Research Problem
9.2.2 Access to Research Data Having defined a research question, it is necessary to turn the attention to the collection of data. Depending on the approach used, data can be either quantitative or qualitative in nature. As covered in Unit 6, data-gathering techniques vary enormously in how well they are able to get these two different types of data. Depending on the type of information that a researcher is interested in they will have to design their data-gathering techniques to match. Data refers to observations, information and measurements that are systematically collected in any discipline. It is useful to distinguish between data collected about individual characteristics and those relating to social structures and processes. Some theories raise issues or seek explanations in terms of the behaviour, experience and understandings of individuals. These perspectives involve research on people and are founded on the assumption that the key factors in the explanation are to be found within people. Social perspectives, however, premise that it is the social structure that determines behaviour. Some methods of enquiry, such as personal interviews, are particularly suited to collecting data from individuals and about individuals, whilst other sources, such as official statistics and historical information, have been used to study social structures.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
An important aspect in defining a project relates to the context and setting in which the research study is undertaken. For instance, this will refer to the subject area of the research such as ‘a study of eating habits’ and to whether the project is implemented within, for example, particular social groups, commercial and industrial sectors, individual organisations or political groups. Therefore, a suitable study may be ‘an investigation into the eating habits of female managers within the banking sector’. It is important to define the limits of a project and this should take into account the resources and timescales available. There is a great tendency for new researchers to feel that they must and will solve the problems of the world with their work. It is more important to refine and limit a project in order to successfully complete it within the time available than to unsuccessfully attempt a much wider ranging investigation. This can be achieved by the introduction of a further variable into the research question as this may create a very large number of more manageable projects within the general area. For example, in the case above, the project could address the influence of organisational culture, training programmes, mentoring, office conditions, lighting, workload or stress on the eating habits of female managers within the banking sector.
PLANN NG
DISASTER
For a project to be successful, the researcher must have access to the data required within the limiting issue of the total resources available. It would be difficult to study the social behaviour of
9-7
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
crofters in the Outer Hebrides if the researcher worked and lived in London because the time and cost of travel to undertake the observational assessments would be prohibitive. Similarly, it would be inappropriate to attempt to review the UK national crime statistics within the 20th Century, during a six-month study, because the time required to access the information would be far in excess of the time available. Consideration should also be given to the research sample and to whether this is representative of the total population covered by the question. In some projects, it may be necessary to resort to statistical tests in order to confirm that the sample employed is valid. A further issue is to ensure that the variables measured are both relevant and representative of the research question. In a project measuring employees’ job satisfaction, it would clearly be unacceptable simply to rely upon answers to a question of whether respondents were satisfied with their level of remuneration because job satisfaction covers a much broader range of issues than monetary reward.
MSC IN EMERGENCY PLANNING MANAGEMENT
An important issue in some projects is the availability, per se, of the data required. An investigation of whether there was a relationship between a bank manager’s willingness to arrange a home visit to a client in order to discuss investment options and the client’s bank balance would prove difficult for reasons related to the release of personal financial data about the bank’s clients. Similarly, a project related to an assessment of the motivating factors behind large-scale fraud in the banking sector may prove difficult through lack of access to a suitable sample of successful and/or unsuccessful fraud cases. Access to data in some situations may be expedited through a letter of introduction, explanation and support from the university. If this is the case, the requirement should be identified as early as possible and thoroughly discussed with a suitable tutor in order to ensure that a letter of introduction is made available in time.
9.2.3 Writing Research Proposals and Dissertations A well-written proposal or dissertation is a very satisfying and rewarding achievement. However, quite often the readers of the material do not share the same views about the quality of the written material as those of the researcher. The problems of writing are summarised by Rudestam and Newton (2001: 57): As a general rule, if you have difficulties in your basic writing skills – that is, in constructing grammatical sentences, using appropriate transitions, and staying focused and concise – a research dissertation will glaringly reveal these weaknesses, and the logic and persuasiveness of your arguments will be diminished. One suggestion is to obtain remedial help in strengthening basic writing skills. Furthermore, the style of writing that is appropriate to research papers is somewhat different from the style of writing associated with literary prose. … Effective writing is an acquired skill. An important skill in writing and presenting research proposals and dissertations is the ability to present a discussion that contains valid statements and arguments in support of the results obtained. We cannot develop critical skills unless we have developed a skill in handling language: investigating and defining meanings, appreciating the effects of grammatical forms, and understanding the thread of an argument through an extended piece of text. (Walliman, 2001: 118)
9-8
disaster
riskcrisis MANAGE RISK
Language can be split into three categories (Walliman, 2001): Informative: which is used to communicate information, such as describing and discussing a research project; Expressive: which is used to impart feelings and emotions, such as how beautiful a flower looks; and Directive: which is used to give commands, such as whether or not to cross the road. Research proposals and dissertations are invariably centred on presenting informative statements. However, there are also a number of different types of statements (Walliman, 2001):
Relational statements provide information about relationships between variables, ideas or concepts, such as ‘an individual’s spending power is related to his or her annual income’. This type of statement is an essential aspect of scientific argument and discussion. Relationship statements explain and predict situations and include: • associations, such as those provided by statistical correlations, where the associations can range from +1 (a perfect positive correlation) through 0 (no correlation) to –1 (a perfect negative correlation); and • cause and effects, such as those provided by a statistical association whereby an independent variable (the cause) has a measurable impact on a dependent variable (the effect). Causal statements can also be linked to probability values. Relational and associational statements can be made with various, what are referred to as, levels of abstraction (Walliman, 2001): Theoretical statements provide statements about theoretical concepts and ideas, such as ‘an individual’s ability to read depends on lighting levels’. Operational statements provide statements about theoretical concepts but which can be supported by measurable data, such as ‘the lighting levels in an office are related to the total window area in the office’. Concrete statements provide specific results or information based on measurements, such as ‘the average illuminance measured in four offices was 550 lux’.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Existence statements state that a particular concept or idea exists, such as ‘Most people own a motor car’. Existence statements become more complex as the information available becomes greater; this is particularly relevant when describing and discussing results. For example, a researcher may state: ‘Seventy-four per cent of the men in the sample population owned a motor car that was less than ten years old’. Existence statements provide a process of classification or categorisation.
PLANN NG
DISASTER
When a series of statements are connected together they form a discourse and a discourse, which is made up of informative or assertive statements, such as a dissertation, would be referred to as an assertive discourse (Walliman, 2001). Where statements are presented in such a way as to demonstrate that there are logical connections between them, they are referred to as arguments. In this case, not only are assertions made but also some assertions are used to support other assertions. Walliman (2001: 123) has defined the minimum requirements for an argument as:
9-9
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
• at least one statement that is reasoned for (this is the conclusion of the argument); • at least one statement that is alleged to support it (this is the premise of the argument); • some signal or suggestion that an argument is under way (where this is a word or phrase, we shall call it the logical indicator). The quality of the arguments presented defines the quality of a research proposal and/or dissertation. Arguments can be related to inductive and deductive reasoning. A deductive argument is said to be valid when its premises, if true, do provide conclusive grounds for its conclusion; and, the other way round, when premises and conclusions are logically related in such a way that it follows that the premises could not possibly be true if the conclusion was valid. (Walliman, 2001: 128) Deductive arguments must be either valid or invalid; on the other hand, inductive arguments can be said to be neither valid nor invalid:
MSC IN EMERGENCY PLANNING MANAGEMENT
An inductive argument contains the claim that its premises only provide some support for the conclusion, rather than furnishing conclusive grounds for its truth. (Walliman, 2001: 128)
9.2.4 Assessment of Proposals and Dissertations Research proposals and dissertations, which are submitted to the Civil Safety and Security Unit in fulfillment of the requirements of an MSc or Postgraduate Diploma, are assessed using the following criteria: The Research Problem The research proposal and dissertation should have an introduction that: • provides the background to the area of the research, including a literature review; • explains the organisational setting for the research; • justifies why the proposed research is important; and • explains the aims and objectives of the proposed research. Theoretical Perspectives and Concepts The research proposal and dissertation should show that the student has: • researched, selected and justified the chosen theories; • considered the relationships between theories and the research methods; and • developed a coherent and reasoned theoretical framework for the proposed research. Research Methods The research proposal and dissertation should show that the student has: • understood the principles of research design; • applied an appropriate methodology to the research problem;
9-10
disaster
riskcrisis MANAGE RISK
• considered feasibility issues, such as access to subjects and organisations for the collection of data; • considered sampling, measurement, validity and reliability issues; • discussed the advantages and disadvantages of various data gathering techniques; and • considered the constraints of time and effort in the research project. Anticipated Problems The research proposal and dissertation should demonstrate that the student has: • anticipated problems in conducting the research, such as whether there are ethical, data, access or other issues; and • considered ways to resolve anticipated problems and discussed the limitations of the results.
9.3 Research Proposals The assessed assignment for Module 3 requires students to produce a research proposal within their subject area or discipline. In doing this, a topic should be chosen that can be narrowed down to a tightly focused problem and where appropriate theories, which underpin the proposed area of study, can be discussed together with the methods that will be used to investigate the chosen problem. The main issues to avoid in preparing a research proposal are uncertainty and vagueness in the presentation. Therefore, avoid phrases such as ‘attempt to’ and ‘consider whether’ and instead write positively by stating exactly what the research will involve by using phrases such as ‘measure’ and ‘determine’. In developing the research proposal, the following factors should be taken into account: • there is no clear and unequivocal relationship between the particular problem chosen and the particular theories and methods discussed. There may be a number of plausible and coherent research routes that can be identified; • the ideas presented in a proposal can be borrowed from a range of sources, provided that they can be shown to fit together in a coherent and complementary way; • question the value, content and structure of the research proposal continuously during its development.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
See section 9.4.3 below for a summary of what examiners may look for when formally assessing a dissertation.
PLANN NG
DISASTER
It was indicated earlier how the research proposal will be assessed and therefore it is suggested that it should be structured around the headings of research problem, theoretical perspective(s), methods of investigation and anticipated difficulties. These headings provide a clear and logical way of thinking about research and they should therefore be incorporated into the planning and development of the proposal.
9-11
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 9-12
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
It is important to recognise that the way in which a research proposal is developed is very much an individual matter; however, the following aspects are important issues: • choose a broad problem area, place it in a particular context and then explain and justify its importance; • choose a particular theoretical approach, which is considered appropriate to the problem at hand, and outline the central concepts and convert these into questions which could be addressed during the research; • choose a range of research methods that are appropriate to the collection of data about the research question; • write a few notes on the problems that might arise, such as political, institutional or ethical constraints; • return to the original research question and consider whether it would be more appropriate to use a different theoretical perspective and/or methods of investigation. This approach can be summarised as follows: The idea is to use a variety of viewpoints: you will, for instance, ask yourself how would a political scientist you have recently read approach this, and how would that experimental psychologist, or this historian? You try to think in terms of a variety of viewpoints and in this way let your mind become a moving prism catching light from as many angles as possible. In this connection, the writing of dialogues is often very useful. (Mills, 1970: 235–6) Comparisons with the original research idea should identify where improvements can and should be made. This strategy will identify whether there is potential for further insight into the problem by merging ideas from different theoretical perspectives. Combining different methods of investigation should be considered together with the possible use of a wider range of data in order to improve the validity and reliability of the research. Whatever the starting point, this strategy will assist the development and refinement of the proposal and those theoretical ideas and methods that are not considered to be useful should be discarded whilst those that are useful should be incorporated into the final proposal. Ensure that the final research proposal is clearly focused by avoiding projects that are vague and/ or unstructured. The research proposal should be presented with a clear structure. The proposal should contain an introduction, which outlines what the research proposal is about, which issues are being researched and provide some background information about why the research is relevant. How the issues will be measured or assessed and how the results will be analysed should follow this.
disaster
riskcrisis MANAGE RISK
9.4 Dissertations A dissertation, which presents an author’s research findings and conclusions, is submitted in support of a higher degree. It is, in its dictionary definition, described as a ‘reasoned argument’ and therefore is not a ‘thesis’ which is the term used for the extended research carried out for a doctorate, nor a ‘project’ or ‘long essay’. The main object of a dissertation is to provide you with an opportunity to conduct an in-depth and focused study on a subject of your choice. The dissertation does not, however, seek to provide the last word on the subject. At Master’s level you will be expected to work with ‘received information’ but you should aim to add something to the body of knowledge which already exists. You must, therefore, examine critically existing knowledge on your chosen subject which will enable you to identify gaps where you can make your own contribution. When writing and submitting a dissertation, students should be fully aware of the current University of Leicester submission requirements for their course. In particular, they should follow the standard requirements for submission procedures and specifications on format – see section The purpose of a dissertation is to communicate the results and conclusions of research work to other interested people in a format that is logical and consistent. The dissertation should therefore contain information about: • what you did; • why you did it; • how you did it; • what you found; and • what you think it means. The dissertation also represents the means by which a student’s understanding of ‘research’ is assessed. Therefore the dissertation should not only report the results but also demonstrate an appreciation of the research philosophy expounded within the course material.
9.4.1 Writing the Dissertation: The Various Stages Outlined A common question asked about writing a dissertation relates to whether to write-up as the research progresses or whether to complete the whole project and then commence the writing. Established researchers take different views and approaches to writing. For new researchers, it is quite daunting to attempt writing on completion of the project because at this stage he or she may have to deal with:
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
9.9 below.
PLANN NG
DISASTER
• a large pile of references, which have been accumulated during the literature search, and which must now be assimilated and presented in a coherent structure; • some brief, handwritten notes on possible research theories that must be reviewed; • a mass of research data that must be analysed and assessed; and • some notes on possible conclusions.
9-13
EMERGENCY
crisisdisaster RISK
MANAGE
It is invariably easier to write each stage of the research process as it progresses and it is also satisfying and reassuring to see the dissertation taking shape well before the deadline for submission approaches. Although the process of writing up may take place as the research progresses, the researcher must always retain an open mind and be prepared to revisit each section at a later date. It may be quite feasible that it is necessary to revisit the published literature in the light of the final methodology adopted, the results obtained or the conclusions reached. It may also be necessary to re-analyse data and to look for new associations between independent and dependent variables as the outcomes are discussed and the conclusions are obtained.
MSC IN EMERGENCY PLANNING MANAGEMENT
On completion and before submission, it is essential to re-read the dissertation carefully from a critical point of view. The structure, content and conclusions should all be reassessed and changed if they no longer fit together to form a coherent presentation. This reappraisal should also include a thorough examination of the sentence structures, grammar, punctuation and spelling used within the dissertation. Finally, it should always be remembered that a computer grammar and spellcheck alone do not guarantee a dissertation free from mistakes.
9.4.2 The Formal Dissertation Process You need to complete and return the dissertation registration and proposal forms by the due dates. This is very important as these forms enable the Civil Safety and Security Unit to allocate dissertation supervisors and to review dissertation topics to ensure their relevance and viability. Clearance of the research problem by the course team is a compulsory part of the requirements for the dissertation. At this stage, a supervisor will be allocated to you who will provide guidance, advice and support throughout the process of completing your dissertation. Writing a dissertation is clearly a very different exercise to writing assessed essays, although the study skills developed through essay writing are an essential prerequisite of conducting research. The experience of designing a research proposal at the end of Module 3 will be invaluable.1 The key difference between dissertations and essays is that you decide what you are going to investigate and how. This is a challenging experience, but because this is your opportunity to carry out a piece of your own work of relevance to your personal/ professional interests it is also very rewarding.
9.4.3 Approaching the Dissertation: The Examiners’ Expectations The dissertation will allow you to develop and demonstrate to the examiners your research skills, powers of critical analysis and ability to write. Building upon the advice contained in section 9.2.4 above, a well-executed dissertation will: • ask interesting questions; • investigate clearly thought-out themes; • make thorough use of available resources; • reach thoughtful and relevant conclusions; • be well written; and • be clearly structured.
9-14
crisis
PLANN NG
risk
disaster
riskcrisis MANAGE RISK
What you have studied so far on the course has been structured by the course team who have set the questions and provided the necessary materials to enable you to answer them. The dissertation element of the course has different and additional demands. Although your work will be supported by a dissertation supervisor, you will be expected to: • design your own research problem; • collect your own data; • find and review relevant existing literature; • identify weaknesses in existing research and theoretical work; • select appropriate methodologies; and • analyse and interpret your research data.
There are a number of distinct elements through which your dissertation will be judged. These include: • the exploration of the relevant literature through which the topic is defined; • a critical and focused in-depth examination of the literature; • the construction of research aims and objectives, and the clear definition of the research problem; • data collection; • the analysis and interpretation of the data collected within a framework chosen by you and showing a clear understanding of your findings; • the development of your own argument through your new understanding of your chosen topic. You should aim to use the material gathered to question and possibly modify your original framework or even develop a new one according to your findings. This will clearly require a critical approach; • a demonstration of the validity of your argument and your competence as a researcher. Your findings may even persuade others to think in a new way or to re-examine an issue in a fresh light.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Clearly the demands of a dissertation place a greater responsibility upon you to take control of your own academic work. At Master’s level, we expect you to examine academic debates critically and demonstrate your ability to interpret your data and relate them to a research problem.
PLANN NG
DISASTER
9.5 Empirical Research The distinctive quality of empirical dissertations is that you use primary source materials generated by yourself in addition to secondary materials. Data which you collect are analysed and interpreted in the light of existing knowledge. So, a literature review also forms an important part of this type of research. The data collected are related to existing debates and themes in the field. Dunleavy (1986: 117–20) analyses two approaches to linking empirical research to existing theoretical debates:
9-15
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
(i) a focus-down model whereby the dissertation begins with a review of the empirical themes and theoretical controversies in the research field, then ‘focuses down’ on the relatively micro-considerations of the empirical research study itself and concludes with a re-analysis of the themes and controversies in the light of the research findings.
EMERGENCY
Difficulties can arise with this approach if the themes raised in the initial chapter are not followed through in the actual research. This produces a ‘lack of fit’ between the existing knowledge and the student’s contribution. Extended literature reviews also have the tendency to crowd out a student’s use of research findings.
MSC IN EMERGENCY PLANNING MANAGEMENT
(ii) an opening-out model where the dissertation begins with a short outline of the rationale behind the research and its aims and objectives. This provides the reader with an orientation to the research. Following this the substantive empirical findings of the primary research are presented. The findings are then analysed and interpreted in terms of the aims and objectives of the research. This process thus enables students to launch into a discussion of the wider implications of the findings in the concluding chapters.
This model means that existing knowledge is discussed only in terms of the empirical research. In other words, only relevant material and debates are discussed. As such, it often produces more integrated pieces of work than the focus-down model.
9.5.1 Choosing a Topic The most important factor to remember when choosing a dissertation topic is that the subject must be of interest to you. In fact, it needs to enthuse you. This is important as it must hold your interest for the duration of the research and writing process. The place to start is the course materials. Think back to the subject areas which have interested or intrigued you and which you would like to examine in greater depth. It may be worth thinking about a topic which allows you to utilise knowledge, insight and contacts gained through your job or other activities. Often the most original and interesting research has been developed through the researcher’s own network. It is still important to remember, however, that access restrictions or ethical dilemmas must be considered. There may be a conflict of interests and this should be discussed with all relevant people including the course team. The subject must be achievable. We can all think of at least twenty interesting and exciting pieces of research which we would like to conduct but you must think of practical issues such as gaining access, time and money. There must be sufficient accessible sources available which address your subject choice to enable you to carry out the essential exploratory literature review. You need to think carefully in advance about where you can best obtain the necessary background information; which libraries you can access; which agencies or organisations you need to contact. If you are considering an empirical study you must also consider the practicalities relating to access to your research population. The research topic must also be manageable. There is no point is setting out to interview 1,000 people in the town centre if you work office hours and usually, also, work on Saturdays. This is an extreme example, but do not set yourself up to fail. A good research topic is usually one which is
9-16
disaster
riskcrisis MANAGE RISK
sufficiently focused and capable of being conducted in the required time. You need sufficient data to facilitate a dissertation of the required length, but not so much that it overwhelms you. Try not to overestimate what you can do. Think small!! Your chosen research topic should have sufficient breadth to enable you to satisfy the requirements of the examiners. If you are in any doubt about your chosen subject area or cannot decide upon a topic area please do not hesitate to contact the course team for advice. Two further warnings should be heeded before choosing your research topic: • do not try to tackle a subject area in which you have little or no expertise; • do not choose a topic for which the evidence is patchy or particularly difficult to interpret, without full guidance from your supervisor.
9.5.2 Choosing Methods
• the nature of the research; • the research problem; • the time available; • access to sources; • the level of resources available to support the research; • the practical difficulties that may be presented when you try to utilise any particular research method; • potential ethical problems; • how confident you feel about using a particular research method. A good starting point in choosing your methodologies is to consider the potential problems you may encounter in obtaining the relevant material or gaining access to an appropriate research group. Many potentially excellent pieces of research have had to be abandoned because access to the sources of information required proved too difficult to gain. Remember, you do not have to carry out primary research; you could use secondary sources. Primary research is, however, exciting and many students find this the most fulfilling part of the course.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Choice of methodology will play a significant part in influencing the final form of your dissertation. The merits of particular methods have been outlined in other Units in this Module and the accompanying textbooks, and it is advisable to refresh your memory of the material. You should be very clear in your mind about the strengths and weaknesses of the different research methods and the type of data produced by them. You must choose your research methods according to a number of criteria:
PLANN NG
DISASTER
9-17
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
9.5.3 Planning the Research Once you have focused on a topic and chosen your methodologies you will need to complete the dissertation registration and proposal forms. These will help to clarify your ideas. They should include the following: • the key problems addressed or research questions to be investigated; • the ways in which these questions are to be investigated, that is, the research methods to be used, for example, sample surveys, interviews, library research; • the primary and secondary sources which you envisage using; • a tentative dissertation outline detailing the sequence and general content of chapters. The research proposal acts as the ‘walking frame’ of a dissertation, supporting you and your supervisor in the research process. However, the final dissertation will inevitably deviate from this. A research proposal is not fixed and the research process is certainly not static. The nature of your research will change over time. However, the construction of a proposal is an important aid
MSC IN EMERGENCY PLANNING MANAGEMENT
in planning and organising your work. It compels you to question the purpose of your dissertation, helping you to specify what direction your research should take and what sources you need to use. A research proposal also provides supervisors with the opportunity to respond to your ideas, suggest alternative directions, possible pitfalls and provide encouragement. It is important to get this feedback early on, thereby giving you the maximum time to develop your ideas before beginning the research. It is useful to think of writing a research proposal as a rolling process in which a number of summaries are drawn up and revised as the research proceeds. It is useful to have some idea of how you intend to write up your dissertation before you start the research process. You should, therefore, consider how you intend to present your findings and the overall structure of your dissertation before you start. The best way of doing this is to construct a plan which indicates chapter by chapter what you are intending to cover. The process of producing this will help to structure, focus and clarify your ideas. It is also useful to refer to this during the research to keep you focused directly on your research subject. However, it is worth noting that many finished dissertations bear only a passing resemblance to this initial structure. It is important to write up findings and observations as you go, otherwise important information will be forgotten. Keeping written drafts of your thoughts will also help you monitor how well the research is working, whether you are actually uncovering information which relates to the questions and problems you are investigating, or whether these questions need revision.
9.5.4 Literature Review Dissertations An alternative to conducting primary empirical research for your dissertation is to undertake a dissertation solely based on existing literature. These ‘literature review dissertations’ are often referred to as focused syntheses. The aim is not to generate original or ‘primary’ data on a particular topic but to critically review existing literature and secondary data in order to reinterpret the material. The aim of a literature review is to develop a new angle on a body of research or theoretical debate. You will be expected, therefore, to have read and used a substantial amount of the existing literature. For guidance on the composition and style of a focused synthesis you should consult review articles in journals and edited collections by established academics. The principal problems encountered in this type
9-18
disaster
riskcrisis MANAGE RISK
of dissertation are conducting a sufficiently comprehensive search of the literature and the development of a distinctive interpretation. Literature reviews should not be simply a regurgitation of an existing review. These basic difficulties indicate further dilemmas in conducting a literature review. If the topic area chosen is relatively unstudied it will be easier to be comprehensive but discovering sufficient material to sustain a 20,000-word dissertation may be difficult. Conversely, if the topic has been extensively researched it can be difficult to précis the literature and develop an innovative angle or interpretation. The place to start is the course materials, in particular the bibliographies of the relevant Units. Another important source of information are the various computer library services, including international bibliographic databases, which can be accessed through libraries.
9.5.5 Timetables
• designate a limited period for reading and theoretical clarification; • clarification of the subject matter of the research must occur early so that empirical enquiry can begin. This is especially important if the research is going to involve work ‘in the field’. It should be noted that the relationship between theory development and applied research is a reciprocal one in which ideas initially suggesting a research direction are redeveloped as their application suggests revision. It is important not to spend too long on abstract theorising, the fruits of which may prove redundant; • if the dissertation involves empirical research then remember that data analysis can be very time consuming; • the time you have available for all the elements of research must be thoughtfully allocated to allow enough time to produce the final document. Always over- rather than underestimate the time required; • the process of writing up the dissertation is likely to take at least a month. Two chapters should be submitted to your supervisor as drafts, and up to three weeks should be allowed for feedback on these chapters;
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
A crucial aid to students in planning the organisation of a dissertation is the formulation of timetables which specify deadlines for specific sections. Timetables provide students and supervisors with a mechanism for avoiding last minute rushes which damage the overall quality of the dissertation. Timetables are an indispensable aid in coping with large pieces of work which take place over a long period of time. It is advisable to have an overall ‘strategic’ timetable which maps out the entire dissertation process and a number of intermediate timetables which deal with the completion of specific sections. You are required to produce a research timetable as part of your dissertation proposal. There are a number of considerations which should be borne in mind:
PLANN NG
DISASTER
• leave sufficient time for producing a final draft. Allow time for the typing, proof reading and binding of the dissertations. Typing can take up to four weeks, proof reading two to three days and binding from 24 hours to two weeks. • students must fulfil all the above before the submission date. Extensions will be granted only in exceptional circumstances.
9-19
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
9.5.6 Researching the Topic Once you have decided on the research problem you can begin to search out the relevant material in earnest. This process should have commenced during the exploratory literature search but it should now become more focused. What you get out of the literature will be determined ultimately by the questions you ask and how you ask them. Please remember that if you have to order or reserve literature, especially from libraries or from overseas, this may take a considerable time to arrive. So the earlier you order it the better. Furthermore, this process will establish the main lines of your enquiry, relevant theoretical debates and ascertain the nature and content of other research already carried out in your chosen field. Try to use a wide variety of secondary sources including books and journals, documents, television programmes, newspaper material now available on CD-ROM, the Internet, magazines, pamphlets, films, statistics, paintings, photographs, and even cartoons where appropriate. Leave no stone unturned in your search for data.
MSC IN EMERGENCY PLANNING MANAGEMENT
You can find out about available information sources through your supervisor, Centre staff, friends and colleagues, people working in the field, professional and campaigning groups, other interested parties, contacts established while conducting your research and by examining the bibliographies in the secondary sources you have consulted. Other sources are the bibliographic databases held in libraries including Leicester University and the Internet. Please remember that recent journals will contain the most up-to-date findings of current research. Archive material may also be available for your subject; if this is so, telephone in advance and ask for relevant material to be made available; be as specific as possible. It is usually possible to pay for a search to be carried out in advance of your visit. It is important to take brief notes on the material you read. You should note down the factual information and also the arguments presented and any points of controversy in the material discussed. Quotations when relevant should also be noted. There are three important points to remember when writing notes for a dissertation: (i) be selective. You cannot take notes about everything or you will be overwhelmed by your material. Refer constantly to the aims and objectives of the research. Do not go off on a tangent; keep to the points in hand. Be ruthless and disregard anything which is not directly relevant to your research. If something is not directly relevant, do not read it regardless of how interesting it may seem; this sort of reading is merely a diversion from the work in hand; (ii) remember to take a note of all information required for referencing. Careful attention should be given to the recording of names of the author(s), title of the publication, publishing details and, where relevant, the page numbers. This information will be crucial when you try to utilise the data. Without this information, you will be unable to reference your data or complete your bibliography. This includes relevant details from all secondary sources including newspapers, films, photographs and so on. There is nothing worse than searching at the end of your research for missing or incomplete references; (iii) make sure that you organise your notes so that you can access relevant material easily. Preferably design a system of note-taking that divides the material into specific issues
9-20
disaster
riskcrisis MANAGE RISK
and subject areas. There are many ways of doing this but it is always advisable to take notes on the computer, loose leaf paper or small index cards rather than note books. You can then sort and structure your notes in a logical fashion. This is relevant for interview transcriptions and observational notes as well as secondary data.
9.5.7 Exploratory Literature Searching It is vitally important to specify your research topic at the outset and have clear aims and objectives. This will give purpose to the dissertation. If you do not have a clear idea about what the dissertation is intending to investigate you will find yourself down many blind alleys, off on many tangents and lost in a mound of literature, data and information. Once you have chosen a subject area you will undoubtedly need to limit and focus down your idea into a manageable dissertation. The best and only way of doing this is by reviewing the existing literature in the area. The initial literature review will also provide a guide to the most appropriate methodologies and some of the pitfalls which have arisen during previous research.
The relationship between reviewing the literature and developing the ideas which direct a research project is a reciprocal one. As students conduct their literature review the nature of the research problem may change due to the occurrence of alternative ideas or through external constraints such as lack of access to information. However, it is vital to have a framework informing the research at any one time even if this is re-conceptualised during the process of ‘doing’ the study.
9.6 Writing Up At some point, you will have to begin the process of writing up your research. Try to keep to the timetable you have prepared for yourself as you must leave enough time for this stage of the process. It is very easy to continuously feel that you are not yet ready to begin writing up. Before you begin to write you should prepare a final structured plan of your dissertation. There are likely to be modifications from your original chapter guidelines. The earlier you begin the process the easier it will be to take full benefit of the supervision on offer. The supervisor will be able to provide feedback in the knowledge that you will have time to act upon the comments made. Dissertations should be written within the allotted word limits and ‘padding out’ should be strictly avoided. Proof reading your own work is essential. Spelling and grammar are important, and most examiners will take this into account. A good dictionary and thesaurus are essential.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Having specified the topic, students can then begin to formulate the research questions and the argument the dissertation is going to pursue.
PLANN NG
DISASTER
The aim of the dissertation is to show that you are presenting a clear and logical argument that directly examines current academic debates relevant to your chosen topic. You do not have to prove that your hypothesis is correct, rather that you have questioned all your main ideas thoroughly. You should be showing the examiners what a good researcher you are even if you ultimately fail to support your initial hypothesis.
9-21
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
These are a few points in relation to writing in general which need to be considered: • keep your writing concise. Do not waffle: ask yourself constantly ‘Is this word or sentence really necessary?’; • make sure that what you say is what you mean. It is easy when writing a lengthy piece to become unclear; • avoid using jargon. Write in plain English as if you are writing for a member of your peer group. In this way you will be careful to define your terms and clarify your main points; • avoid using long and complicated sentences. Unless you are an expert in the use of grammar, keep sentences simple. In that way your reader will not get ‘lost’ in your work; • avoid repetition in your writing. If you structure your work well and keep to a well thought-out plan most repetition can be avoided;
MSC IN EMERGENCY PLANNING MANAGEMENT
• use quotations effectively to illustrate a relevant point. Keep them reasonably brief (no more than 30 words) and do not use them too often. Remember to reference other people’s work and remind yourself of the regulations relating to plagiarism. If in doubt – reference; • do not use abbreviations unless absolutely necessary. Make sure that if you do abbreviate a name that is frequently discussed you have been absolutely clear about what the abbreviation stands for; • avoid aiming your writing at someone whom you consider to have more knowledge than yourself. It is always best to write for someone who has a broad knowledge of your subject but has slightly less specific knowledge about your chosen topic. Overall, your writing should be clear and concise, and your argument should be logical and easily accessible. No examiner is going to thank you for writing a piece that requires them to reach regularly for their own dictionary or thesaurus.
9.6.1 Starting to Write Many people have difficulties applying themselves to the actual writing of the dissertation. ‘Writer’s block’ can sound romantic but it is also very common and can produce enormous anxiety – the feeling that you cannot do it, yet the acute knowledge that you must. Blocks can come at any stage of the dissertation process. Whatever stage it hits, it is an enormous time waster. There are a variety of difficulties which can be identified and these can be divided up into three types: (i) getting down to the work; (ii) starting the writing process; (iii) finishing off and submitting the work. Often the person who finds it difficult to get down to work at all is thought of, and also thinks of themselves, as lazy. They are likely to be lacking in confidence, perhaps intimidated by the size of the task in front of them. Often such people find it impossible to begin work until they are confronted with a rapidly looming deadline. Sadly, a lot of useful time is wasted. It is very easy to
9-22
disaster
riskcrisis MANAGE RISK
find excuses for not starting work, and many diversionary tactics will be used to delay the inevitable. The house will be tidy, forgotten and neglected friends visited, other work brought up-to-date. Setting yourself a weekly timetable (in addition to your overall timetable) and providing yourself with frequent deadlines may help. It is also important to make sure that you fully understand what is expected of you; you may have far greater expectations of the task than are realistic. Discuss this with your supervisor as soon as you encounter difficulties. Many students find the data collection aspect of the dissertation the most rewarding and get stuck when they are required to write it up. There are three main problems associated with this type of block: (i) you have gathered too much material and cannot stop reading. You are overwhelmed with the sense that you may miss something vital; (ii) you cannot sort out your ideas. There seem to be so many different perspectives that you feel completely confused; expose you. All of these are common problems. We all feel insecure about our work. Reading too much can be a diversionary tactic: anything to avoid the inevitable and get down to work. Try to sort out your notes, check that you are not being repetitious in your reading. Keep to a timetable and unless something is absolutely essential allow yourself to stop researching. Remember we cannot find out everything. Once you have finished reading, spend some time thinking about your argument, write it down, construct a new dissertation plan, and then start to write about the sections in their logical order. Writing a dissertation is like doing a jigsaw puzzle: it is best approached methodically. It may help to remember that your initial writing is merely the first draft. By the time the work is completed many changes will have been made and the process of typing and publication will leave your work looking professional. Knowing when to finish and giving yourself permission to submit work takes courage. Some students beg extensions and then carry a completed piece of work around too anxious to submit it. There is many a dissertation that has to be gently peeled out of a studentâ&#x20AC;&#x2122;s hand. Proof reading is important, but over-enthusiastic editing that holds back your work is unlikely to be productive. Overall, be realistic about your dissertation, follow the instructions provided and remember this is a student dissertation not the ultimate piece of work on your chosen topic.
9.6.2 Structure Many researchers who will read dissertations will only be looking for specific information on certain aspects of the research, such as the sample population, average group responses and conclusions. Dissertations are therefore normally presented in a standardised format in order to assist this process as well as providing a structured description of the research. A good general focal point to remember when writing a dissertation is the person who will read the dissertation.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
(iii) you cannot bear to see your ideas on paper. You are scared that your examiner will
PLANN NG
DISASTER
Informed researchers, who may or may not be familiar with the specific issues or context of the research, normally read dissertations. Most researchers will read the dissertation title first; if the title conveys a message that the contents are likely to be of interest then the researcher will move
9-23
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 9-24
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
on to read the abstract. Only if the abstract indicates that the dissertation may be worth their time and effort to read it will a researcher read the whole dissertation. Each piece of research information should appear only once in a dissertation and therefore if issues are repeated, the presentation is most likely to be incorrect. There is only one exception to this general rule and that relates to the abstract, and in this case the abstract should contain only information that has already been presented elsewhere within the dissertation. A dissertation generally follows an hourglass format. The introduction should introduce the broad area of the research and then concentrate down to the specific area of research, which defines the project. The method and results sections should contain information only about the specific research subject area. The discussion should start by discussing the specific results of the research and then broaden back out again in order to assess the possible wider implications of the research results. The dissertation should therefore take the reader full circle. In order to achieve this structured approach to a dissertation, it is essential to prepare a plan of the dissertation before starting to write. The following headings provide a useful guide for structuring a dissertation. 9.6.2.1 Dissertation Title and Abstract (what you did) The title should be succinct and accurate and reflect the content of the dissertation. It should provide a very brief summary, normally no more than one sentence, of what the research is about. Titles, such as: ‘The impact of speed cameras on the number of road traffic accidents on European motorways’, and ‘The effect of management training programmes on the performance of senior managers in FTSE100 companies’, would be acceptable as dissertation titles. However, titles, such as: ‘Reducing road traffic accidents on European motorways through the installation of speed cameras’, and ‘Improving the performance of senior managers in FTSE100 companies through management training programmes’, would be unacceptable as dissertation titles as they represent statements of intent rather than answers to research questions. The abstract provides a short summary of the dissertation but provides significantly more information than the title. The abstract should contain a brief description of the rationale behind the research and should include a summary of the research method, the important results and the conclusions. Abstracts are typically between 200 and 300 words in length and therefore the abstract must be carefully considered and constructed in order to ensure that all of the important information is included. Abstracts from research dissertations are often included on library databases, which are used by other researchers as they search the literature for relevant publications. Therefore, in order to make searches easier, many publications recommend a format for abstracts that will include sub-headings or paragraphs entitled research objectives, research methods, results and conclusions.
disaster
riskcrisis MANAGE RISK
9.6.2.2 Introduction (why you did it) The introduction provides the reasoning behind the research study and should contain the relevant information from the literature survey. After reading the introduction, an informed researcher should have a well-defined idea of what issues the research project will be addressing. In addition, the introduction should inform a non-expert in the particular area of research what the salient issues are and why the research is important. The majority of the references included within a dissertation should be found within the introduction.
The introduction should contain an explanation of why any previous research work is deficient or inadequate in order to support the decision to undertake the proposed research. The explanation may relate to issues such as the methodology, the need to confirm the results from previous studies or the inappropriateness of the published research within the proposed organisational setting. Clearly, if the previous research were unequivocal, there would be no justification or need for further research. There may be a need to explain how the proposed research goes beyond, improves upon or extends previous research in the specific area. The introduction should conclude with a clear statement of the proposed research question, hypothesis or area of study. 9.6.2.3 Research Methodology (how you did it) The research methodology should contain sufficient detail about the methods used in order to allow a reader to repeat the research if needed. This will vary considerably depending on whether a quantitative or qualitative approach has been used, but where appropriate, the methodology should include information about the following issues: Sample population: this should state the organisational setting involved in the research, which may be a particular social or political group, industrial or commercial sector or individual organisation. Where individuals are studied within the research project, the method should include information, if appropriate, about, for example, gender, age and occupations of the subjects studied. These issues may be very important or of little interest, the level of interest being dependent on the research area. It will clearly not be possible to make generalisations about the whole population if the study included exclusively male, female, old or young subjects.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
The introduction should start with a general background to the subject area and then progress through to the specific issues related to the research study. This should include a brief review of the theoretical and practical reasons for undertaking the research. The introduction should include a description of the research area with an explanation of why the research is of interest, if this is not clearly apparent to an informed reader. There should be a summary of the previous published research studies that are relevant to the study; this may include information about the methodology adopted, the organisational setting for the research and/or the subject area of the research work.
PLANN NG
DISASTER
Research instruments: questionnaires, interviews, audits and focus groups, for example, are all research instruments and their formats and contents should be explicitly described. Similarly, where archive material is used, the source should be described in full so that later researchers can, if necessary, re-access and re-assess the material. If it is necessary to provide a full copy of a research instrument rather than a summary, then it is normal to include this in an Appendix at the end of the dissertation.
9-25
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT 9-26
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Research procedures: it is very important to provide details of how the research was undertaken; this may include how questionnaires or interviews were developed, piloted and implemented amongst the sample population. The procedures should incorporate information that was presented to the subjects, how variables were controlled and how variables were measured. This should include a description of the measurement scales so that they can be identified as nominal, ordinal, interval or ratio scales for the purpose of validating any statistical tests used in the analysis of the data. It may also be relevant to discuss the timescales associated with the research project and/or the date or year in which the study was conducted. This will be particularly important if the sample data is compared to larger national databases of information over the same time period. Statistical tests that are used to analyse the data and the level of statistical significance that will be accepted are normally presented; however, it has been argued that statistical tests are research tools rather than research methodology and as such should not be incorporated within the research methodology. 9.6.2.4 Results (what you found) This section should provide a clear and concise summary of the research data obtained. Again, the type of research data will vary enormously depending on whether qualitative or quantitative approaches have been used. (see Units 7 and 8). For example, quantitative methods will be more likely to involve raw data, mean scores, standard deviations, ranges and the results of statistical tests. Tables and figures provide a convenient way of summarising research data. Tables are normally used to summarise and present data and other information, whilst figures are used to present diagrams and graphs. It is important to remember that all tables and figures included in a dissertation should be referenced in the text. Tables and figures should be numbered and labelled; they should never be included in a dissertation if the information that they contain is not required and/or not used. By convention, tables are normally labelled above the table and figures are labelled below the figure. The results section of a dissertation is normally relatively short and should never contain a discussion of any of the results presented. 9.6.2.5 Discussion and Conclusions (what you think the results mean) This section should present the authorâ&#x20AC;&#x2122;s interpretation of the research results and is where their wider significance can be discussed. The discussion should relate to the research results presented in the results section, which in turn should relate to the research issues raised within the introduction. All results discussed should therefore already have been presented in the results section; new results should only be introduced within the discussion if they relate to another researcherâ&#x20AC;&#x2122;s work, in which case they should be referenced and included in the reference list of the dissertation. It is quite possible that the results obtained from the research study do not enable the researcher to reach a clear conclusion or to answer the research question in full. This does not mean that the research has failed or is invalid; it does mean, however, that, because the research question has not been fully answered, it may be appropriate within the discussion to explain why this has occurred and to suggest areas for future research. Where conclusions have been reached from the research, these should be included at the end of the discussion.
disaster
riskcrisis MANAGE RISK
9.7 Common Problems There are a variety of problems associated with dissertation writing and this section will identify the most common mistakes made by students. For many students the temptation to fix upon a particular issue and plunge straight into the primary research, with little or no consultation or consideration of the issues and without a sufficient exploratory literature review, leads to early disaster. Preparation and planning are essential if you wish to produce a useful or relevant piece of work. Many dissertations are also over-ambitious: there is little point in setting out a research proposal that would require several years to complete and unlimited resources when you have a short period of time (approximately four months) and are also likely to have extremely restricted resources. The most successful student research utilises easily available resources and focuses upon a narrow but precise problem.
Problems can also be encountered when not enough time and energy have been devoted to designing the research instrument, for example, a questionnaire or interview schedule. The research instrument is of vital importance as once data have been collected you rarely have a second chance to go back for more. It is, therefore, a mistake to conduct the fieldwork without thinking clearly in advance about what you want to find out, the best way of doing this and how you intend to analyse the findings. The way in which you intend to analyse the findings will have implications for the methodology which you use. It is, therefore, essential that the issue of analysis be addressed before you start the project. Many students fail to place their research in a wider context. It is important to contextualise the debates you are discussing. Nothing comes out of thin air and everything has a historical, political, economic and social context. Inclusion of this in a dissertation is one of the things which makes it an academic piece of work. Dissertations often lack reference to research methods literature. It is very important that you ground your chosen research methods within the relevant literature on theory and practice. So you must discuss the theoretical tradition from which the method you have adopted emerged and critically analyse its effectiveness at meeting its objectives. Referring back to your Module 3 assessment and the course materials related to research methods should help. Remember you must always justify why you have conducted the research in the way that you have and the problems which arose with the approach.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Many distance learning students try to carry out research for their own organisations as part of their academic studies. This can be successful and clearly may offer some advantages such as resourcing and negotiating access. However, there may be pitfalls such as conflicts of interest. It must also be borne in mind that the dissertation is not merely a report; it must be grounded in academic debates. An academic piece of research should build upon a previous body of knowledge and add to it. Discussion of the proposal with the course team and/or your supervisor will help you decide whether or not work-based research will provide the basis of a viable dissertation. Often the student will be advised to write up two quite separate versions of the research.
PLANN NG
DISASTER
Another common problem arises with the presentation of data. Many findings are simply produced without any discussion or interpretation. It is, therefore, important to draw out the themes of your
9-27
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
research and discuss some possible interpretations and consequences of your findings. This will provide the essential ingredients of interpretation and critical analysis. Also bear in mind that you are unlikely to use all the material you have collected. Much of it will remain unused in the final dissertation. Only include data which are relevant to the argument you are trying to make.
9.8 Supervision You will be allocated a supervisor to provide some further guidance on your dissertation. The supervision has two distinct elements: • an initial supervision session in which the dissertation proposal is discussed and suggestions about possible resources, amendments and methodological issues are examined;
MSC IN EMERGENCY PLANNING MANAGEMENT
• the second element of supervision is through reading and commenting on two draft chapters. You may choose which draft chapters you submit, but our advice is the methodology and concluding chapters. Feedback provided by the supervisors should help students to avoid many of the most common problems associated with dissertations. The role of supervisors is: • to help students clarify their research problem; • to assist and advise students in undertaking the dissertation; • to aid students in summarising the themes emerging from the literature review and findings; • to locate the findings into broader theoretical and academic debates and the social, political and economic context. Dissertation supervisors can help you avoid many of the problems associated with completing dissertations and are a source of support and guidance. Students who fail to maintain contact with their supervisors are more likely to have difficulties completing this part of the course. The onus is on you to keep in touch with your supervisor. You should contact the Centre if you have any queries or problems with supervision.
9.9 Dissertation Regulations and Requirements Candidates who satisfy the examiners in all six formally assessed module assignments with marks of 50 percent or over will be permitted to submit a dissertation of between 15,000 and 20,000 words (excluding bibliography and appendices) for the degree of MSc. The dissertation should normally be completed within two years of commencing the course. Any candidate may be invited to attend viva voce (oral) examination of their dissertation. In such cases attendance at the examination and satisfactory assessment by the examiners will be a condition of the successful completion of the MSc.
9-28
disaster
riskcrisis MANAGE RISK
Submission of Dissertation Distance learning students should submit three bound copies of a soft covered dissertation on the due date to: University of Leicester, Civil Safety and Security Unit, 14 Salisbury Road, LE1 7QR Leicester. They must be accompanied by a completed plagiarism declaration form and five loose copies of the abstract. Format Dissertations must be typewritten, word processed or printed and the following requirements must be strictly observed: • size: A4 (210 x 297 mm) (except with special permission); • one side of paper only to be used; • spacing: double or one-and-a-half; • font size: at least 10;
• head, foot and outer margins: 15 mm minimum; • explanatory footnotes at the bottom of the relevant page; • bibliography: to follow text and any appendices; • title page: to include, as well as the full title of the dissertation, the degree for which the work is submitted, Centre/University, the year of submission, and the candidate’s name; • an acknowledgements page (optional) can be included; • an abstract is deemed to be an integral part of the work to be examined and must be produced in strict accordance with the following requirements: • five loose copies of the abstract are to be submitted at the same time as the dissertation; • the abstract must not exceed 300 words, must be produced with single spacing on one side of A4 paper and must be suitable for photographic reproduction; • the abstract must show the author and the title of the dissertation in the form of a heading. • three bound copies of the dissertation should be submitted on or before the due date; • a signed plagiarism declaration form must be submitted with your completed dissertation. Dissertations will not be accepted without one;
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
• inner margin: 35 mm minimum;
PLANN NG
DISASTER
• a list of contents, including chapter numbers/titles and page numbers, should follow the abstract; • the body of the dissertation is followed by the appendices and then the bibliography; • subheadings should be typed in lower case, underlined or in bold, and begin from the margin. The first line of paragraphs should be indented.
9-29
EMERGENCY
crisisdisaster RISK
MANAGE
crisis
PLANN NG
risk
Extensions It is vital that you submit your dissertation on time and failure to do so may delay your graduation. If you require an extension please submit a request (preferably via email) containing a brief outline of your extenuating circumstances to your course administrator. Applications for extensions must be received prior to the submission date.
9.10 Main Points The following key points have been raised and discussed within this Unit: • Honesty in research is essential; this refers to honesty in the context of accuracy and truthfulness of the presentation of the research methods and results and to honesty towards the subjects and organisations taking part in the research programme.
MSC IN EMERGENCY PLANNING MANAGEMENT
• Positive research statements relate to factual information and what exists, whilst normative statements relate to a researcher’s opinions and views about the research subject. • The natural sciences view of research is that it is related to the acquisition and understanding of knowledge through the processes of inductive and deductive reasoning. • The social sciences view of research is that it is related to the social context within which investigations are undertaken and knowledge is acquired. • Selecting a research topic that is clearly focused is a key factor in successful research proposals and dissertations. • Research work includes pure and applied aspects of study and covers a variety of approaches, such as exploratory, testing-out and problem-solving research. • Access to the appropriate data is a fundamental prerequisite for all research projects and is an issue that should be thoroughly examined before a research proposal is finalised. • Research proposals and dissertations are assessed against the criteria of research problem, theoretical perspectives and concepts, research methods and anticipated problems. • In a research project there are no preconceived right or wrong theoretical or methodological approaches; these are defined by the researcher and are based on his or her knowledge and experience of the research area. • A well-written dissertation has a clear structure based around issues of previously published research in the area of work, methodology, results, discussion and conclusions.
9-30
disaster
riskcrisis MANAGE RISK
9.11 Study Questions You should now write approximately 300 words in answer to each of the questions below. We believe that this is an important exercise that will assist your comprehension of the material and aid your progress on the course. Your answers are intended to form part of your own course notes and should not be forwarded to the University. Identify a research problem within an area of your course and then consider: 1. the theoretical perspective(s) and concepts that would underpin the study; 2. the research methods that you would use to investigate the issue; and 3. the problems that you may encounter in carrying out your research study.
9.12 Bibliography
Glaser, B. and Strauss, A. L. (1967) The Discovery of Grounded Theory, Chicago: Aldine. Mills, C. W. (1970) The Sociological Imagination, Harmondsworth: Penguin. Rudestam, K.E. and Newton, R. R. (2001) Surviving Your Dissertation: A Comprehensive Guide to Content and Process, London: Sage Publications. Swetnam, D. (2000) Writing Your Dissertation, Oxford: How To Books Ltd. Walliman, N. (2001) Your Research Project, London: Sage Publications.
EMERGENCY
MSC IN EMERGENCY PLANNING MANAGEMENT
Dunleavy, P. J. (1986) Studying for a Degree in the Humanities and Social Sciences London: Macmillan.
PLANN NG
DISASTER
9-31
crisis
PLANN NG
MSC IN EMERGENCY PLANNING MANAGEMENT
9-32
crisisdisaster RISK MANAGE
risk
EMERGENCY
This module serves as an introduction to the course and to the subject area of risk, crisis and disaster management, and it is also a conceptual tool box for the rest of the course. In particular, it introduces a range of theoretical perspectives on the concepts of risk and crisis such as how risk is assessed and managed. The overarching aim of the module is to identify different perspectives and examine the extent to which they inform practice and ultimately to lay a foundation upon which future modules will build.
MSc in Emergency Planning Management
Module 2 Managing Risk and Crisis
Module 3 Research Methods in Risk, Crisis and Disaster Management This Module aims to provide students with comprehensive knowledge and understanding of methodological issues in investigation studies research. The Module introduces students to research methodology on both a theoretical and practical level. Students are encouraged to analyse critically the process of social scientific enquiry and to examine the relationship between research problems, theoretical perspectives and methodological approaches.
Module 4 Case Studies of Crises and Disasters In this module a number of case studies of crises and disasters are examined. The case studies act as heuristics ‑ vehicles for exploring some of the issues and concepts introduced in modules one and two. Such issues include the impact of personality on crisis and disaster management, the influence of cultural factors and national preferences on crisis and disaster management techniques, and the impact on disaster investigations of paradigmatic interpretations of evidence. The rationale for the module is that important lessons can be learned from the detailed, objective analysis of past crises and disasters. The unit also provides an insight into the politics of the 1974 Health and Safety at Work Act, which set up the United Kingdom’s Health and Safety Executive, and into subsequent legislation on the regulation of developments close to hazardous complexes.
Module 5 Models of Risk and Crisis This module addresses the possibility that risks, crises and disasters may be understood in different ways by different people. An air crash, for example, may be understood primarily as a potential blow to profitability by an aircraft manufacturer, as a case for investigation by the relevant police service and national accident investigation bureau, as a destabilizing influence on the stock market by brokers and investors and as a human tragedy by the tabloid press (for whom disasters provide many column-inches of material) and relatives, partners and friends of the victims. Thus the same event may be ‘constructed’ or experienced differently by different parties. This module examines how parties with different ‘investments’ (reputational, financial, emotional etc.) in crises and disasters may experience them in quite different ways.
Module 6 Emergency Planning Management This module looks at the ‘front line’ management of risks, crises and disasters. The emphasis is on practical risk, crisis and disaster management, from risk assessments produced by Britain’s Health and Safety Executive to the factors that need to be considered by emergency planners when drafting an evacuation plan. The module aims to be as eclectic as possible, including, for example, a unit on the identification and management of post-traumatic stress disorder.
The course material is and remains the property of the University (and must be immediately returned to the University upon request at any time) and is either the copyright of the University or of third parties who have licensed the University to make use of it. The course material is for the private study of the student to whom it is sent and any unauthorised use, copying or resale is not permitted. Unauthorised use may result in the course being terminated. The course material was created in the academic year 2011/2012 Civil Safety and Security Unit • University of Leicester • 14 Salisbury Road • Leicester • LE1 7QR
RESEARCH METHODS IN RISK, CRISIS AND DISASTER MANAGEMENT
In this module some contemporary debates about security are explored. It brings together broad developments in theories of risk in the social sciences with risk issues of relevance to security managers. It also examines the relationship between these different perspectives on risk and a general theory of security. An attempt is made to highlight the relationship between the theory and practice of risk management and security.
MODULE 3
(updated August 2011)
Research Methods in Risk, Crisis and Disaster Management EMERGENCY
EMERGENCY
Module 1 Theories of Risk and Crisis
MODULE 3
MSc in Emergency Planning Management
PLANNING