IB Assessment Update, IBAP conference, 16th March 2012 Singapore
Richard Penrose, head of marking
IB Assessment “No other external factor has a greater effect on the quality of a student‟s learning than the quality of teaching and no other external factor has a greater effect on the quality of teaching than the nature of the final assessment of the student.”
© International Baccalaureate Organization 2007
IB Assessment We strive for IB assessments that are • Valid • Reliable • Challenging • Manageable • Meaningful
Page 3
© International Baccalaureate Organization 2007
MARKING EXAMINER RECRUITMENT & TRAINING
ASSESSMENT QA
CAO
of Head ent ssm n Asse Innovatio n&
SUBJECT MANAGEMENT
Desig
He Ma a d o r ki f ng
SUBJECT OPERATIONS
Head o f Scho Delivery ol
RESEARCH IN ASSESSMENT DESIGN
ASSESSMENT DESIGN AND INNOVATION
BUSINESS INTEGRATION
EXAMINATION PAPER PREPARATION
IBIS ADMINISTRATION (including academic ASSESSMENT honesty and special MATERIAL ADMINISTRATION educational needs)
SCHOOL DELIVERY CAO = Chief Assessment Officer © International Baccalaureate Organization 2007
Assessment e-initiatives (ECMS) Part of IB 2020 vision, allowing management of sustained IB growth E-marking E-coursework upload E-authoring E-assessment
© International Baccalaureate Organization 2007
E-Marking process IB provide data about components and candidates to RM Education for upload into scoris™ Scripts sent from schools to scanning centre (UK or US) Scripts scanned according to SLAs and made available in scoris™ Component senior team set-up practice, qualification and seed scripts Examiners „qualify‟ to mark and are quality assured during marking Raw marks are final (unmoderated) marks in IBIS © International Baccalaureate Organization 2007
ECMS Road Map - 2011 In Operation
2011 Q1
Q2
Web Assessor QIGing
Stage I : SharePoint Stage II: Item Authoring
E-Assessment
7
Q1
Q3
Q2
Q4
Q1
Q2
Q3
Q4
Eastgate Closure
Pilot web assessor marking client
E-Marking
E-Authoring
Q4
Academic division relocate to Hague
Business Milestones
E-Coursework Stage II
2013
2012 Q3
Mark Structure System
QIG marking
E-Marking Visual Arts and audio Ramp-up web assessor marking –pilot web assessor STM
Upload and emark all TOK
Audio upload pilot
Audio upload compulsory
Multiple file upload
Visual Arts upload optional
Extended essay upload optional VA upload compulsory
Procurement Complete Pilot E-Authoring Stage I
E-Authoring Stage II (Meta Data)
MYP Next Chapter
Š International Baccalaureate Organization 2007
ECMS Road Map (Stage II) 2012 In Operation
2012 Q1
Q2
2013 Q3
Q4
Q1
Mark Structure System
E-Marking
E-Coursework Stage II
QIGing Pilot
Q4
Q1
QIGing Phase I Live
E-Marking Visual Arts and audio Ramp-up web assessor marking –pilot web assessor STM Audio upload compulsory Visual Arts upload optional
E-Authoring Stage I : SharePoint Stage II: Item Authoring
2014 Q3
Q2
Q3
Pilot E-Authoring Stage I
E Structuring
Extended essay upload optional
QIGing Phase II Live Rich Media E-Marking
I/A E-Marking (Sample Moderation)
VA upload compulsory E-Authoring Stage II (Meta Data)
E-Authoring for EAssessment E-Assessment Trials
E-Assessment
8
Q4
Eastgate Closure
Business Milestones
Web Assessor QIGing
Q2
MYP Next Chapter
MYP Next Chapter Š International Baccalaureate Organization 2007
E-Marking: Examiners’ Script View
Annotation Bar
Applied annotations
Mark Entry Panel
Page 9
Š International Baccalaureate Organization 2007
E-marking in May 2012
216 components 629,259 scripts (67% of scripts) 3714 examiners 56,494 ToK essays
Page 10
© International Baccalaureate Organization 2007
E-marked components May 2012 Biology HL & SL p2 & p3
Chemistry HL & SL p2 & p3
Physics HL & SL p2 & p3
Computer science (All papers) English B (All papers)
Economics (All papers)
Psychology HL/SL p1 & HL p3
Environmental systems & societies (All papers)
Geography HL/SL p1 & HL p3
ITGS HL & SL p1, HL/SL p2 & HL p3
Maths SL (All papers)
Maths studies SL (All papers)
Philosophy (All papers)
History route 2 (All papers)
TOK essays
Social & Cultural Anthropology HL & SL p1, p2, HL p3 Further Maths
Maths HL (All papers)
Spanish B (All papers)
Spanish abinitio SL p1 & p2
German A1 HL & SL p1
French abinitio SL p1 & p2
French A1 HL & SL p1
Music HL & SL p1
Chinese A1 HL & SL p1 Page 11
Š International Baccalaureate Organization 2007
Benefits to IB of e-marking Examiner marking is visible Underperforming examiners are coached or removed from the session Re-allocation is rapid – no script movements No clerical checking – scoris does the sums Examiners mark to the PE standard No frenzy of re-marking inconsistent examiners close to issue of results Standardisation and Grade Award meetings can be remote Page 12
© International Baccalaureate Organization 2007
Benefit to schools and candidates Wealth of data on candidate performance available (IBResultsExtra) Digital return of EURs Candidate‟s work is marked anonymously, removing examiner bias Examiner work is constantly compared to the PE standard IB exam delivery can scale at the rate of IB growth Negligible risk of losing scripts
Page 13
© International Baccalaureate Organization 2007
E-coursework Upload of oral assessment recordings for groups 1 and 2 Upload of visual arts assessment material from 1st April (pilot). Please help
Page 14
© International Baccalaureate Organization 2007
Dynamic Sampling™ for IA e-marking Moderator downloads sample of 3
Teacher's marks within tolerance
Teacher's marks stand for all candidates
Teacher's marks consistent
Moderation applied to all candidates
Teacher's marks outside tolerance
Moderator downloads 2-7 more Page 15
Š International Baccalaureate Organization 2007
E-coursework upload - benefits All script / material movements are digital • No school to examiner postage cost • No examiner to TL, TL to PE, Examiner, TL and PE to Eastgate and Eastgate EUR CAT 2 to school postage costs • Nothing gets lost in the post • Nothing gets stuck at customs Ease of plagiarism checks, for school and IB Errors in submission can be corrected by candidate and school before submission Upload process requires authentication of candidate work Dovetails with e-marking Multi-media upload provides new possibilities for assessment tasks to curriculum review teams Page 16
© International Baccalaureate Organization 2007
Other assessment updates Re-moderation of internal assessment • Available from May 2012 • Replaces category 3 enquiry upon results internal moderation report Instructions to examiners about comments on scripts change in May 2012 Comparability study 2012 E-marking of internal assessment from 2014
Page 17
© International Baccalaureate Organization 2007
Mark changes in re-marks 5000
4760
4500
4000 3500 3000
2815
2500
2316
2000 1500 1110 1000
729 539
500
0
0
0
4
8
9
24
59
99
-9
-8
-7
-6
-5
-4
228
193
97
48
30
8
7
1
1
1
5
6
7
8
9
10
11
12
0 -12 -11 -10
-3
-2
-1
0
1
2
3
4
Amount of mark changes
Page 18
Š International Baccalaureate Organization 2007