ď‚Ą
How ________ is your thermometer?
ď‚Ą
Good measure is central to ability to assess well
Classical Test Theory Variance discriminate, but variance due to what (facets)?
=
Generalizability Theory Decompose variance into e.g. person, p and item, i. Identify facets causing desired, undesired variation
Item Response Theory •Probabilistic rather than deterministic •Item (question) level discrimination, not just test level •Account for person ability and item difficulty in same scale
ď‚Ą
A person should have greater probability of success on easier item than more difficult item.
ď‚Ą
A more able person should have greater probability of success on assessment items than a less able person.
ď‚Ą
Rasch analysis (in a nutshell)-how good is your test item fitting to these assertions?
(Person ability – Item Difficulty)
Personal best = 2.10m (Person ability)
Personal best = 1.72m (Person ability)
Personal best = 1.50 (Person ability)
Bar height = 1.80m (item difficulty)
What’s the probability he will clear the bar? 50:50? Anxiety? Preparation? Etc.
Personal best = 1.80m (Person ability)
Wright map
ICC and IIC
Polytomous IRT (e.g. partial credit model)
Fit statistics to Rasch model (chi-square)
Item 1
Item 2
Item 3
Item 1
Item 2
Item 3
Total
4.0 3.5
0.8
Information
Proportion Correct
1.0
0.6 0.4 0.2
3.0 2.5 2.0 1.5 1.0 0.5
0.0 -3
-2
-1
0 Ability
1
2
3
0.0 -3
-2
-1
0 Ability
1
2
3
Item with more than 1 mark, e.g. 4 marks Monotonic increase from 0 to 1 to 2 to 3 to 4 marks