Lecture10 research methods in architecture data collection

Data Collection Educational Research by John W. Creswell. Copyright © 2002 by Pearson Education. All rights reserved.

Slide 1

Key Ideas  Obtaining permissions for data collection  Selecting participants for data collection  Identifying data options

 Recording and administering data collection

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 2

Procedures for Collecting Quantitative Data  Obtain permissions

– identify the unit of analysis – secure permissions – obtain informed consent from participants

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 3

Procedures for Collecting Quantitative Data  Select participants

– specify a population and sample – use probability and non-probability sampling – choose a sample size

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 4

Procedures for Collecting Quantitative Data  Identify data options

– specify variables – operationalize variables – select scales of measurement – choose types of data measures

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 5

Procedures for Collecting Quantitative Data  Record and administer data collection

– locate or develop an instrument – obtain reliable and valid data – develop administrative procedures for data collection

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 6

Obtaining Permission: The unit of Analysis  Unit of analysis is the level (e.g. individual,

family, school, school district) the data will be gathered.  There may be different units of analysis – one for the dependent variable – one for the independent variable

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 7

Obtaining Permission: Obtaining Permissions  Institutional or organizational (e.g. school

district)  Site-specific (e.g. secondary school)  Individual participants or parents  Campus approval (e.g. university or college)

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 8

Obtaining Permission: Obtaining Informed Consent from I.R.B.  Obtain Institutional Review Board

Approval  Have participants sign an informed consent form

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 9

Select Participants: Specify a Population and Sample  A population is a group of individuals that

comprise the same characteristics  A sample is a sub-group of the target population that the researcher plans to study for the purpose about making generalizations about the target population.

– Samples are only estimates – The difference between the sample estimate and the true population is the “sampling error.”

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 10

Populations and Samples Target Population Sample

Sample

Population

Sample

-All teachers in high schools in one city -College students in all community colleges -Adult educators n all schools of education

-All high school biology teachers -Students in one community college -Adult educators in 5 schools of education in the Midwest

Educational Research by John W. Creswell. Copyright Š2002 by Pearson Education. All rights reserved.

Slide 11

Types of Quantitative Sampling Quantitative Sampling Strategies

Probability Sampling Simple Stratified Random Sampling Sampling

Multi-Stage Cluster Sampling

Non-Probability Sampling Convenience Sampling

Educational Research by John W. Creswell. Copyright Š2002 by Pearson Education. All rights reserved.

Snowball Sampling

Slide 12

Select Participants: Use Probability and Non-Probability Sampling  Probability sampling is the selection of

individuals from the population so that they are representative of the population  Non-probability sampling is the selection of participants because they are available, convent, or represent some characteristic the investigator wants to study.

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 13

Types of Probability Samples  Simple Random: selecting a sample from

the population so all in the population have an equal chance of being selected  Systematic: choosing every “nth” individual or site in the population until the desired sample size is achieved

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 14

Types of Probability Samples  Stratified sampling: stratifying the

population on a characteristic (e.g. gender) than sampling from each stratum.  Multi-Stage Cluster Sampling: a sample chosen in one or two stages because the population is not easily identified or is large

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 15

Proportional Stratification Sampling Approach Population (N=9000) Boys N=6000

.66 of pop.

200

Girls N=3000

.33 of pop

100

Sample = 300

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 16

Types of Non-Probability Samples  Convenience Sampling: participants are

selected because they are willing and available to be studied  Snowball Sampling: the researcher asks participants to identify other participants to become members of the sample.

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 17

Select Participants: Choose a Sample Size  Select a sample size as large as possible

from the individuals available  Select a sufficient number of participants for the statistical tests you will use (e.g. 15 per group for experiments)  Calculate the sample size using a sample size formula Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 18

Flow of Activities in Collecting Data Flow of Activities

Example

Identify the variable

Self-efficacy for learning from others

Operationally define the variable

Level of confidence that an individual can learn something by being taught by others

Locate data (measures, observations, documents with questions and scales) Collect data on instruments yielding numeric scores

13 items on a self-efficacy attitudinal scale from Bergin (1989)

Scores of each item ranged from 010 with 10 being “completely confident.”

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 19

Identify Data Options: Specify Variables  Independent Variables  Dependent Variables  Intervening Variables

 Control  Moderating

 Confounding

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 20

Identify Data Options: Operationalize Variables  Operational Definition: The specification

of how the variable will be defined and measured – typically based on the literature – often found in reports under “definition of terms”  Sometimes the researcher must construct it Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 21

Identify Data Options: Select Scales of Measurement  Nominal (Categorical): categories that

describe traits or characteristics participants can check  Ordinal: participants rank order a characteristic, trait or attribute

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 22

Identify Data Options: Select Scales of Measurement  Interval: provides “continuous” response

possibilities to questions with assumed equal distance  Ratio: a scale with a true zero and equal distances among units

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 23

Identify Data Options: Choose Types of Data Measures  An instrument is a tool for measuring, observing,

or documenting quantitative data  Types of Instruments – Performance Measures (e.g. test performance) – Attitudinal Measures (measures feelings toward educational topics) – Behavioral Measures (observations of behavior) – Factual Measures (documents, records) Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 24

Record and Administer Data Collection: Locate or Develop an Instrument  Develop your own instrument  Locate an existing instrument  Modify an existing instrument

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 25

Record and Administer Data Collection: Locate or Develop an Instrument  Strategies to use

– Look in published journal articles – Run an ERIC search and use the term “instruments” and the topic of the study – Go to ERIC web site for Evaluation and Assessment – Examine guides to commercially available tests Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 26

Record and Administer Data Collection: Obtain Reliable and Valid Data  Reliability: individual scores from an instrument

should be nearly the same or stable on repeated administrations of the instrument  Types of reliability – – – –

Test-retest (scores are stable over time) Alternate forms (equivalence of two instruments) Alternate forms and test-retest Inter-rater reliability (similarity in observation of a behavior by two or more individuals) – internal consistency (consistent scores across the instrument

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 27

Record and Administer Data Collection: Obtain Reliable and Valid Data  Validity: the ability to draw meaningful and

justifiable inferences from the scores about a sample or a population  Types of validity – Content (representative of all possible questions that could be asked) – Criterion-referenced (scores are a predictor of an outcome or criterion they are expected to predict – Construct (determination of the significance, meaning, purpose and use of the scores) Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 28

Record and Administer Data Collection: Develop Administrate Procedures for Data Collection  Develop standard written procedures for

administering an instrument  Train researchers to collect observational data  Obtain permission to collect and use public documents  Respect individuals and sites during data gathering

Educational Research by John W. Creswell. Copyright ©2002 by Pearson Education. All rights reserved.

Slide 29