Research Design and Data Collection 1

Starting Your Research Journey: Research Design & Data Collection: 1

BML224: Data Analysis for Research

Learning Outcomes Aims:  To discuss and contextualise the key elements of the research

design and data collec6on process

 To discuss and consider the differences between different types of

data

 To demonstrate how different types of data influence the type of

analysis that can be executed

Step Step 1: 1: Decide Your Decide on Research your research Topic topic

1

Cra1ing Research What?

• What puzzles/intrigues me?

• What do I want to know more about/understand be@er?

• What are my key research quesCons?

Why? • Why will this be of enough interest to others to be published as a thesis, book, paper, guide to pracCConers or policy-­‐ makers?

• Can the research be jusCfied as a ‘contribuCon to knowledge?’

How – Conceptually?

• What models, concepts and theories can I draw on/develop to answer my research quesCons?

• How can these be brought together into a basic conceptual framework to guide my invesCgaCon?

How – Prac:cally?

• What invesCgaCve styles and techniques shall I use to apply my conceptual framework (both to gather material and analyse it)?

• How shall I gain access to informaCon sources/date

Step 1: Decide on Your Topic

 Undertake your literature review – Read!

Step 1: Decide on Your Topic

 Undertake your literature review

What is the func6on and value of your literature review?

The FuncCon of the Literature Review

 Demonstrates your knowledge of the research topic/problem –

originality, crea6vity, innova6on! (not merely reinven6ng the wheel)

 Convinces your reader that your proposed research will make a

significant and substan6al contribu6on to the literature

 Demonstrates your understanding of the theore6cal and research

issues related to your research ques6on

 Shows your ability to cri6cally evaluate, integrate and synthesise

relevant literature informa6on

The FuncCon of the Literature Review

 Provides new theore6cal insights or develops a new model as the

conceptual framework for your research

 Contextualisa6on and different perspec6ves – how has the topic

been researched before?

 Helps to plan out the intended line of analysis and data

requirements

 Helps you learn from other researchers’ mistakes  It is expected and addresses your assessment criteria!

Assessment Criteria

 Evidence of clear research aims and

objec6ves informed by background research

Managing the Literature Review  Housekeeping -­‐ keep a record of all your reading  Note the full reference (using Harvard conven6on)  You must evidence the literature review throughout (in-­‐text

referencing/cita6ons)

Data and Data Sources

Principal Forms of Data:  Quan%ta%ve -­‐ the observa6ons or responses are expressed

numerically

 Qualita%ve – use of comments, case studies or observa6ons with

responses oUen subsequently assigned to categories

 Ideally aim for a balance of both quan6ta6ve and qualita6ve

methodologies/data (mixed methods)

Data and Data Sources Primary Data ata • Primary Direct Dm easurement of actual phenomena, e.g. ques6onnaires, interviews, focus groups Secondary Data

• Data collected by someone other than the user and which Primary ata generalised, filtered or modified in some way, e.g. have bDeen census data, marke6ng sta6s6cs, economy sta6s6cs, visitor numbers

Sources of Secondary Data

Sources of Secondary Data

Sources of Secondary Data

Approaches Sampling to Sampling  Types of Sampling

Random Random SystemaCc StraCfied Cluster MulC-­‐Stage

Non-­‐Random Judgement Quota Convenience

Can you define each approach to sampling?

Step 2: Research Aims & Objectives

2

Step 2: Research Aims and ObjecCves

What is the difference between a research aim and objec6ve?

Research Aims  Aims are broad statements of desired outcomes, or the general

inten6ons of the research, which 'paint a picture’/ ‘tell a story’ of your research project

 Aims emphasize what is to be accomplished (not how it is to be

accomplished)

 Aims address the long-­‐term project outcomes, i.e. they should

reflect the aspira6ons and expecta6ons of the research topic

Research ObjecCves  Research objec6ves help to explain the way in which the research

ques6on is going to be answered (they should be understandable to you and to others!)

 Steps taken to answer your research ques6on/list of tasks

needed to accomplish aims

 Must be sensible and precisely described and act as statements

to convey your inten6ons

 Objec6ves should link to established theory and research

Research ObjecCves  Research objec6ves are

usually headed by an infini6ve verb

• To iden6fy • To establish • To describe

 Research objec6ves should

be met

• To determine • To es6mate • To develop • To compare • To analyse • To collect

Services

Step 3: Questionnaire Design

3

Step 3: QuesConnaire Design

What are the key elements of good ques6onnaire design?

QuesConnaire Design Layout and ordering of ques%ons  Clear presenta6on and instruc6ons on about how to respond  Guide the respondent clearly through the process  Use pages and sec6ons in BOS  Place broad and general ques6ons at the beginning of the

ques6onnaire followed by more specific ques6ons

 Using funnel ques6ons

QuesConnaire Design Layout and ordering of ques%ons  Numeric coding  Avoid placing survey ques6ons out of order or out of context  Format and layout of ques6ons – e.g. horizontal or ver6cal

presenta6on

 BOS allows you to insert a variety of media into your ques6ons

(e.g. images)

QuesConnaire Design Layout and ordering of ques%ons  Don’t ask two ques6ons in one – e.g. is your job interes6ng and

well paid cannot be answered with a simple yes or no

QuesConnaire Design Avoid Bias  Be careful with the use of wording – the tone and wording of

ques6ons can have a significant impact on your results (e.g. might/ should/could)

Capture the Voice  Even within quan6ta6ve surveys open-­‐ended ques6ons provide an

opportunity to ‘capture’ the voice of the respondent

QuesConnaire Design Be Specific and Exact  Unclear survey ques6ons produce answers that lack meaning and

relevance

 e.g. How regularly do you watch TV – ‘regularly’ needs defining  Make sure mul6ple choice ques6ons are mutually exclusive so that

a clear choice can be made

QuesConnaire Design Be Considerate  Avoid intrusive ques6ons (ethics!)  Consider your audience – ensure they understand your language,

terminology and the ques6on being asked

 Keep ques6ons short and to the point – respondents will not

complete long and tedious surveys

 Incen6ve – what do I get for comple6ng your survey?

QuesConnaire Design Choose the Right Ques%on  Your choice of ques6on will influence the type of data you get  Do the ques6on and answer formats provide enough robustness to

meet analysis requirements?

 The type of data will influence the nature and scope of your

analysis (and ability to meet assessment criteria!)

Assessment Criteria

 Clear and logical structure of the

presenta6on/poster demonstra6ng progression from basic to advanced sta6s6cal techniques referencing a specific aspect of the research process/results

Data Requirements  Accurate -­‐ there is no ‘systema6c bias’ in measurements  Precise -­‐ they are measured with sophis6cated instruments /

methodologies

 Reliability -­‐ they are comparable over 6me and/or spa6ally/

geographically (longitudinal analysis)

 Valid -­‐ they are a ‘true’ representa6on of the underlying more

complex phenomenon, e.g. ‘quality of life’ or ‘economic well-­‐being’ etc.

Data Requirements  Fit for purpose – provide the basis for basic descrip6ve to

advanced sta6s6cal analysis

 Informed – methodologies have been drawn from a review/

cri6que of the available literature

Step 3a: Linking Data Types to Analysis

3a

Data Types

NOIR

Data Types Types of Data  Nominal:

NOIR

 List of categories to which objects can be a@ributed; objects

can be counted but not be measured numerically (classed as qualitaCve data)

 No assumpCon on their order, only that objects in different

categories are “different”

 Classed as non-­‐parametric data  Examples:

Which supermarket do you normally shop at?: Asda (1), Sainsburys (2), Tesco (3), Morrisons (4)  Gender: Male (1), Female (2) 

Data Types Types of Data  Ordinal:

NOIR

 List of categories but this Cme ordered or ranked; differences

are in relaCve magnitude (greater than; less than)

 No assumpCon is made on the ‘distance’ between categories  Classed as non-­‐parame:c data

 Examples: 

How would you rate the quality of service provided by your mobile phone company?: 5 – Excellent to 1 – very poor

Data Types Types of Data  Interval:

NOIR

 ObservaCons are made on a scale comprising equal intervals

but the zero value is arbitrary

 Classed as parametric data

 Examples: 

Fahrenheit or Celsius scale to measure temperature.

Differences make sense but raCos do not (20o/10o is not twice as hot!)

Data Types Types of Data  Ra:o:

NOIR

 ObservaCons are made on a scale comprising equal intervals

with a true zero point

 Classed as parametric data

 Examples: 

e.g. age, height, weight, response Cme, grade

Data Types: Further DefiniCons Further DefiniCons

QUANTITATIVE DATA

Discrete

Con:nuous

No. of people in the class

Distance traveled: 12.654 miles

INTEGER

Data Types: Further DefiniCons Types of Data -­‐ Summary

NOIR NOMINAL

ORDINAL

NON-­‐PARAMETRIC

INTERVAL

RATIO

PARAMETRIC

Linking Data Types to Analysis Nature of the QuesCon

NOMINAL ORDINAL Type of Data

INTERVAL RATIO

Type of Analysis DESCRIPTIVE INFERENTIAL

Linking Data Types to Analysis General Purpose

Descrip:on (only)

Specific Purpose

Summarise Data

Compare Groups

Finds Strengths of AssociaCon, Relate Variables

Type of Ques:on/ Hypothesis

Descrip:ve

Difference

Associa:onal

DescripCve StaCsCcs (e.g. mean, percentage, range)

(e.g. t-­‐test, Mann Whitney)

(e.g. correla6on)

General Type of Sta:s:c

Explore Rela:onship Between Variables

[Source: Morgan, G. et al (2011), IBM SPSS for Introductory Sta6s6cs, Routledge, London, p. 6]

Linking Data Types to Analysis General Purpose

Descrip:on (only)

Specific Purpose

Summarise Data

Compare Groups

Finds Strengths of AssociaCon, Relate Variables

Type of Ques:on/ Hypothesis

Descrip:ve

Difference

Associa:onal

DescripCve StaCsCcs (e.g. mean, percentage, range)

(e.g. t-­‐test, Mann Whitney)

(e.g. correla6on)

General Type of Sta:s:c

Explore Rela:onship Between Variables

[Source: Morgan, G. et al (2011), IBM SPSS for Introductory Sta6s6cs, Routledge, London, p. 6]

Linking Data Types to Analysis QuesCon: What grade do you expect to get?

Type of Data =

Type of Analysis

Linking Data Types to Analysis QuesCon: What grade do you expect to get?

Type of Data = NOMINAL

Type of Analysis

Linking Data Types to Analysis Type of Analysis

Tabular What grade do you expect to get for the module?

2010/2011

%

2011/2012

%

Grade A

6

8.5%

3

12%

Grade B

17

23.9%

16

32%

Grade C

37

52.1%

19

38%

Grade D

11

15.5%

11

22%

Grade E

0

0%

1

2%

Total

71

100%

50

100%

Linking Data Types to Analysis Type of Analysis

Graphical Expected(Grade(for(BML224(

F"(<40%):" 2%"

A"(70%+):" 6%"

D"(40,49%):" 22%"

B"(60,69%):" 32%"

C"(50,59%):" 38%"

Linking Data Types to Analysis Type of Analysis

Graphical Expected)Grade)for)BML224) 40%

35%

30%

Percentage)(%))

25%

20%

15%

10%

5%

0% A)(70%+):

B)(60169%):

C)(50159%):

Expected)Grade)

D)(40149%):

F)(<40%):

Linking Data Types to Analysis QuesCon: How confident do you feel about star%ng this module?

Type of Data =

Type of Analysis

Linking Data Types to Analysis QuesCon: How confident do you feel about star%ng this module?

Type of Data = ORDINAL

Type of Analysis

Linking Data Types to Analysis Type of Analysis

Tabular How confident are you about star:ng this module?

2010/2011

%

2011/2012

%

7 -­‐ Very confident

0

0.0%

0

0.0%

6 -­‐ Quite Confident

4

5.60%

1

2.0%

5 -­‐ Confident

16

22.50%

10

20.0%

4 -­‐ Uncertain

28

39.40%

26

52.0%

3 -­‐ Anxious

14

19.70%

7

14.0%

2 -­‐ Quite Anxious

4

5.60%

4

8.0%

1 -­‐ Very Anxious

5

7.00%

2

4.0%

Uncertain to very anxious

51

72%

39

78%

Sample (n)

71

50

Linking Data Types to Analysis Type of Analysis

Tabular How confident are you about star:ng this module?

Average confidence level 2010/2012: 3.82 Average confidence level 2011/2012: 3.82

2010/2011

%

2011/2012

%

7 -­‐ Very confident

0

0.0%

0

0.0%

6 -­‐ Quite Confident

4

5.60%

1

2.0%

5 -­‐ Confident

16

22.50%

10

20.0%

4 -­‐ Uncertain

28

39.40%

26

52.0%

3 -­‐ Anxious

14

19.70%

7

14.0%

2 -­‐ Quite Anxious

4

5.60%

4

8.0%

1 -­‐ Very Anxious

5

7.00%

2

4.0%

Uncertain to very anxious

51

72%

39

78%

Sample (n)

71

50

Linking Data Types to Analysis Type of Analysis

Graphical Student&Confidence&Levels&2011&

Very&Anxious:!

4%!

Quite&Anxious:!

8%!

Quite&Confident:!

2%!

Confident:!

20%!

Anxious:!

14%!

Uncertain:!

52%!

Linking Data Types to Analysis QuesCon: ALtudes to Sta%s%cs

Type of Data =

Type of Analysis

Linking Data Types to Analysis QuesCon: ALtudes to Sta%s%cs

Type of Data = ORDINAL

Type of Analysis

Linking Data Types to Analysis Type of Analysis

Tabular

A`tudes Towards Sta:s:cs

This is my first ever staCsCcs class I am worried about this module If I could avoid taking this module I would I've never enjoyed maths Passing is my main goal for this module I do not see the relevance of this module

Strong Agree [5]

Agree [4]

No Opinion [3]

Disagree [2]

Strongly Disagree [1]

32%

37%

4%

14%

13%

17%

26%

18%

32%

7%

14%

32%

25%

20%

9%

10%

24%

24%

30%

13%

32%

38%

17%

10%

1%

7%

9%

24%

39%

21%

Linking Data Types to Analysis Type of Analysis

Graphical Student$Attitudes$to$Statistics$ I,do,not,see,the,relevance,of,this,module

7%$

9%$

Statement$

Passing,is,my,main,goal,for,this,module

24%$

39%$

32%$

21%$

39%$

17%$

10%$ 1%$

Strongly,Agree Agree

I've,never,enjoyed,maths

10%$

24%$

24%$

30%$

13%$

No,Opinion Disagree Strongly,Disagree

If,I,could,avoid,taking,this,module,I,would

14%$

I,am,worried,about,this,module

32%$

17%$

This,is,my,first,ever,statistics,class

25%$

25%$

18%$

32%$

0%

20%

20%$

32%$

37%$

40% Percentage$

4%$

60%

14%$

80%

9%$

7%$

13%$

100%

Linking Data Types to Analysis Type of Analysis

Graphically Attitudes$to$Statistics$

I+do+not+see+the+relevance+of+this+module

4$

Response$

Passing+is+my+main+goal+for+this+module

Ranking$Scale$ + 1+=+Strongly+Agree+ 2+=+Agree+ 3+=+No+opinon+ 4+=+Disagree+ 5+=+Strongly+Diagree+ +

2.5$

I've+never+enjoyed+maths

3.4$

If+I+could+avoid+taking+this+module+I+would

3.2$

I+am+worried+about+this+module

3$

This+is+my+first+ever+statistics+class

2.3$

0

1

2

3 Mean$Rank$

4

5

Linking Data Types to Analysis Type of Analysis

Graphically Attitudes$to$Statistics$

Plotting the mean score (rank) for each response

I+do+not+see+the+relevance+of+this+module

4$

Response$

Passing+is+my+main+goal+for+this+module

Ranking$Scale$ + 1+=+Strongly+Agree+ 2+=+Agree+ 3+=+No+opinon+ 4+=+Disagree+ 5+=+Strongly+Diagree+ +

2.5$

I've+never+enjoyed+maths

3.4$

If+I+could+avoid+taking+this+module+I+would

3.2$

I+am+worried+about+this+module

3$

This+is+my+first+ever+statistics+class

2.3$

0

1

2

3 Mean$Rank$

4

5

Linking Data Types to Analysis QuesCon: Business Turnover in 2010

Type of Data =

Type of Analysis

Linking Data Types to Analysis QuesCon: Business Turnover in 2010

Type of Data = RATIO

Type of Analysis

Linking Data Types to Analysis Type of Analysis

AnalyCcal Descrip:ve Sta:s:cs – Turnover 2010 Turnover 2010 Mean

£41,311.40

Median

£44,640.00

Mode

£44,760.00

Standard DeviaCon

£9191.0316

Linking Data Types to Analysis Type of Analysis

Graphical

Distribution of the Data

Box plot

Linking Data Types to Analysis Type of Analysis

Distribution of the Data

Graphical

Linking Data Types to Analysis Type of Analysis

Tabular Size of Business in 2010 by Category of Turnover Turnover 2010

No. of Businesses

Percentage

£0 to £9,999

0

0

£10,000 to £19,999

0

0

£20,000 to £29,999

48

16

£30,000 to £39,999

73

24

£40,000 to £49,000

118

39

£50,000 to £59,000

59

20

£60,000

2

1

300

100%

Total

Linking Data Types to Analysis Type of Analysis RECODING

RATIO

NOMINAL

Tabular Size of Business in 2010 by Category of Turnover Turnover 2010

No. of Businesses

Percentage

£0 to £9,999

0

0

£10,000 to £19,999

0

0

£20,000 to £29,999

48

16

£30,000 to £39,999

73

24

£40,000 to £49,000

118

39

£50,000 to £59,000

59

20

£60,000

2

1

300

100%

Total

Linking Data Types to Analysis Type of Analysis RECODING

NOMINAL

RATIO

Tabular Size of Business in 2010 by Category of Turnover Turnover 2010

No. of Businesses

Percentage

£0 to £9,999

0

0

£10,000 to £19,999

0

0

£20,000 to £29,999

48

16

£30,000 to £39,999

73

24

£40,000 to £49,000

118

39

£50,000 to £59,000

59

20

£60,000

2

1

300

100%

Total

Linking Data Types to Analysis Type of Analysis

Graphical Size%of%Business%by%Turnover%Category% £60,000% 1%% £50,000%to%£59,000%% 20%%

£20,000%to%£29,999%% 16%%

£30,000%to%£39,999%% 24%%

£40,000%to%£49,000%% 39%%

Linking Data Types to Analysis QuesCon: % Change in Turnover 2008-­‐2010

Type of Data =

Type of Analysis

Linking Data Types to Analysis QuesCon: % Change in Turnover 2008-­‐2010

Type of Data = INTERVAL

Type of Analysis Same process of analysis for RATIO data would apply

Types of Data Summary TYPE OF MEASUREMENT

QUALITATIVE NOMINAL In nominal measurement the variables consists of named categories. The categories have no mathemaCcal properCes.

QUANTITATIVE VARIABLES ORDINAL

INTERVAL

RATIO

The scores indicate only rank order in terms of size. It is not correct to calculate means on the scores.

The steps between the scores are equal in size though there is no proper zero point. The scores can be added, means calculated etc.

This is the same as ‘interval’ measurement but the scale has a proper zero point. RaCos can be calculated as a consequence

[Source: Howitt, D. and Cramer, D. (2011), Introduction to Research Methods in Psychology, Pearson, London]

Learning Outcomes At the end of this session you should be able to:  Iden6fy and discuss the importance of key elements of the research

design and data collec6on process

 Contextualise the research design and data collec6on process via an

ar6cula6on of the requirements of the undergraduate management project

 Discuss and consider the differences between different types of data  Demonstrate how different types of data influence the type of analysis

that can be executed through the design of your own survey ques6ons