Examiner newsletter July 2012 A note from the editorial team Welcome to the fifth edition of the examiner newsletter. The newsletter follows similar design to previous editions: interviews with IB examiners, articles on issues of interest to examiners, some statistical information and a collection of news in brief. A number of you have written to say how much you enjoy reading the interviews with examiners. In this edition we have included inspirational interviews with two examiners: Eunice Price, Deputy Chief Examiner for history, and Jeehan Abu Awad, Principal Examiner for English B. We have been asked to provide more information about the IB staff who manage the assessment sessions. We have an article on the new assessment structure together with information and photographs of some of the IB colleagues you might have communicated with during the marking sessions. The May 2012 session was, for nearly 2,000 IB examiners, their first experience of emarking. We have therefore included an article on scoris developments for the future. We know that many examiners find using scoris challenging at first, but we hope that you will persevere. We are always delighted to receive comments and feedback about our newsletter and hope that you enjoy this edition and write to let us know what you thought about it. Carolyn Adams Chief Assessment Officer IB trials the new scoris™ marking tool In the 2009 plans for introducing an e-marking tool for marking Diploma Programme (DP) examination scripts, the IB documented that part of the requirements for the tool include a multilingual application and PC and Mac compatibility. We are pleased to say that we are on the way to realising these requirements. In May 2012, IB examiners used scoris to mark 685,000 scripts online. This included over 55,000 theory of knowledge essays and almost 630,000 examination scripts. Examiners installed scoris, a Windows based application that runs in Internet Explorer, on their PCs (see images below). Examiners using PCs are able to mark relatively trouble free, though not all examiners embraced e-marking the first time they e-marked.
The marking experience for Mac users has been increasingly frustrating as examiners are required to access scoris through Terminal Services, essentially turning their Macs into a dumb terminal with scoris running on a file server in Abingdon near Oxford, UK. This approach does not work well for large numbers of Mac users and cannot be sustained for future sessions. Since March 2011, the IB has been working with RM Education, the developers of scoris, to produce a browser-based, platform-independent and multilingual version of scoris. We have
affectionately called this version “Web Assessor”, though the official RM name is “the new scoris marking tool”. This means that the new version of scoris will work from multiple browsers as a web-based tool. There is no marking application to install, though a free plugin called Silverlight (similar to Adobe Flash) is required. This new marking tool will also operate in English, French and Spanish so the marker can select their preferred working language for the interface. Web Assessor has other new features including an improved interface and the potential to download and mark to the concurrent limit offline. Download speed is improved and the tool is built to allow the division of scripts into single questions or multiple questions called question item groups (QIGs). Breaking up a paper into QIGs can be achieved for question papers the candidate writes on and essay-type papers where the candidate writes on answer paper rather than the question paper. The IB is still working on splitting IB scripts into QIGs and this change will be introduced soon. Live trials with Web Assessor took place in the November 2011 session. Many lessons were learned and significant code changes made. We trialled this tool again in the May 2012 session for both standardization set-up and marking. Over 50,000 scripts from 13 components were e-marked and we plan to use it for all components from November 2012. One of our greatest difficulties has been trying to stop calling it Web Assessor. Analysis of our e-marking examiners’ experiences as well as those of IB assessment staff shed new light on our assessment activities and provided the evidence needed to improve many of the things we do. Our experience of e-marking with seeding has underscored the importance of precise marking of seeds. To support principal examiners in the selection and definitive marking of seeds, we increased the number of face-to-face standardization meetings at the IB Assessment Centre last May. At these meetings, senior examiners refine component markschemes and agree definitive marks for selected scripts that become practice scripts, qualification scripts and seeds. The senior examining teams had technical support on hand to assist with standardization set-up in the scoris assessor e-marking tool. Senior examiners expressed that the face-to-face format was highly beneficial to the progress of standardization work and their confidence in the outcomes of the meetings. To offset the environmental impact, the cost of face-to-face standardization and the time away from home and workplace for senior examiners, many grade award meetings last June were held virtually rather than in person. E-marking provides that scripts are available electronically so that several senior examiners can view the same scripts simultaneously for grade boundary setting wherever the examiners are located. Some of these virtual grade awards took place synchronously; the meetings took place in real time using Skype and other web-based communication tools. Because some senior examiner teams are spread across several time zones, other meetings were held asynchronously using Moodle forums for discussion of grade boundaries. While we intend to continue to take advantage of the opportunities presented by electronic systems to reduce travel for IB examiners, the value of face-to-face meetings of senior examiners for complex discussions and innovation is not underestimated and some regular in-person meetings will continue for the foreseeable future. The response to the examiner questionnaire sent out to all May 2011 e-markers was very positive and the results are shown below. We will issue a new questionnaire to examiners using scoris after this session—please try to find a few minutes to respond, as your feedback is invaluable.
The IB has a new Assessment Centre The Assessment Division has restructured and is still based in Peterson House, Cardiff, UK, formerly the IB Curriculum and Assessment Centre. The overall objective of the division is to design, develop and deliver world-class academic assessments that support the mission and educational goals of the IB. There are three teams covering the main assessment processes, with functional teams working within them, as shown in the diagram below.
MARKING EXAMINER RECRUITMENT & TRAINING
ASSESSMENT QA
CAO
of Head ent ssm n Asse Innovatio n&
SUBJECT MANAGEMENT
Desig
He Ma a d o rkin f g
SUBJECT OPERATIONS
Head of School Deliver y
RESEARCH IN ASSESSMENT DESIGN
ASSESSMENT DESIGN AND INNOVATION
BUSINESS INTEGRATION
EXAMINATION PAPER PREPARATION
IBIS ADMINISTRATION (including academic ASSESSMENT honesty and special MATERIAL ADMINISTRATION educational needs)
SCHOOL DELIVERY
CAO: Chief Assessment Officer
Everyone in Assessment strives to provide the best possible support to IB examiners and the new structure helps us to work in the most streamlined and efficient way. Examiners will have most contact with the Marking Team, which is headed by Richard Penrose. The subject managers (led by Gareth Hegarty, Roxane Vigneault and Toby Wild) and subject operations staff (led by Hayley Smith) are in Richard’s team, as well as Craig Escott’s examiner recruitment and training (ERT) team. Sarah Perry manages the Middle Years Programme (MYP) assessment team.
The Marking team is organized in subject groups and manages all aspects of the marking process. The teams are structured so that there is a good mix of staff with knowledge of the three IB working languages and expertise in the subjects concerned. Marking staff work very closely with senior examiners, examiners and moderators to ensure that all assessment material is marked and checked by the time that the grade award meetings are held in each subject. Richard supports his team’s development through arranging assessment-related seminars, visiting schools where possible and continually reviewing procedures. The Marking team manages the virtual and face-to-face paper editing, standardization and grade award meetings with senior examiners and supports all examiners through their marking in both the DP and the MYP. The introduction of e-marking and e-coursework technology has been enthusiastically adopted by the team, who have needed to learn how scoris works and how all the administrative interfaces work with it as well, so they can monitor marking progress and help examiners who need it. A number of “super users” who have adapted very quickly to the new technology support the rest of the team and examiners who experience technical difficulties. The examiner recruitment and training team is responsible for ensuring that the IB has all the examiners and moderators required for each session for both the DP and MYP. It is also responsible for providing examiners and moderators with training materials to support them in their important role. The team has led the way in conducting online communications using a Moodle platform. We are using Moodle in grade award, standardization and other virtual meetings between examiners. Descriptions of the work of the School Delivery and Assessment Design and Innovation teams will be provided in later editions of the newsletter.
Interview with Eunice Price, Deputy Chief Examiner for history
How many years have you been an examiner with the IB? I first started examining in 1996. I chose to mark Paper 2 as I really like the global scope of the paper. Then I moved on to extended essays and have also, at different times, marked Paper 3 (Europe and the Middle East) and internal assessment. I was very lucky to have had the experience of working with Sonia Clarke, who was Chief Examiner when I started examining. I thought the world of Sonia and miss her terribly. Whenever I am unsure about what mark to award a script, I think of Sonia and try to remember all the invaluable advice she gave me. What attracted you to history? I intended to study English at university, but one of my courses was with Professor Ieuan Gwynedd Jones, whose special area of study was Victorian Wales, and he was really inspiring. Listening to him was to have the past open before you and, because it was about Wales, it seemed very familiar and relevant. Travelling the world has made me realize that it is difficult to get to know a country without knowing something of its history. More generally, I cannot imagine not knowing about the 20th century; without that knowledge, I would feel as if I had lost my memory. Teaching at the United World College Adriatic, Italy, reminds me constantly of the importance of history, as many of my students come from countries where the past is still approached as if it were, literally, a time bomb. In my last class of the term, before the IB exams began, I asked each of my students to mention a fact they had learned that had most surprised them. Lucie, who comes from a former Eastern Bloc country, said, “Trotsky”. She meant that although she had studied history before coming to the college, she hadn’t known he was such an important figure in the Russian Revolution. Every day brings experiences like this and I, too, will often learn something new from my students about their country or an historical event, especially when they research their extended essays. They go home in the summer to burrow away in archives and libraries and, as a result of this experience, quite a few decide to study history at university. In everyday life I find myself making comparisons between the past and the present. It is really impossible to discuss Vietnam without referring to very recent conflicts in the Middle East or to analyse the Great Depression without commenting on the current economic crisis. History can seem a very static subject but it is probably the most dynamic in the IB programme. Historians are constantly revising interpretations of the past and, although this is a bit of a cliché, the questions we ask and the answers we uncover tell us a lot about the times we live in.
What are your top three examining tips? 1. It is a good idea to re-acquaint yourself with how you marked last session, so take the time to read any feedback that you have received and, in the case of extended essays, take a look at samples returned to you with senior examiner comments. 2. Being an examiner is taking on a huge responsibility, as we are assessing the work of candidates who have been studying for two years. So, find a comfortable space (easy with e-marking) and give each script the attention it deserves. 3. When reading a script, if you come across an arcane fact that you think cannot possibly be true, go and check it. Experience has taught me that these facts are often correct and I am reminded that I do not know it all! What is your favourite place? I really have four favourite places. New York is my favourite city—I first taught the IB there in...er...1978. I love to go back every few years and to see how it has changed. It always makes me feel as though I am at the centre of the universe. Here in Duino, Italy, on a crisp winter morning, I can see the snow-capped Julian Alps from the window of my little office. My office is full of books, files and mounds of old exam papers. On the notice board there are postcards from students, random posters and magnetic finger puppets of historical figures (you can order them online!). My husband calls it the “parallel universe”. At home in Wales, in the summer, I love to stand on the top of Carn Ingli with the Pembrokeshire coast laid out before me. Finally, I always enjoy visiting the IB Assessment Centre in Cardiff, seeing my IB colleagues and being a part of the IB network. What do you like most about being an examiner for the IB? I really appreciate the confidence it gives me as a teacher. I feel better able to advise my students because I am so familiar with the exam process and I can assess their work clearly and accurately. Teaching, oddly enough, can be quite a lonely pursuit, although you spend all day with students, and examining has put me in touch with so many great people all across the world. It has really broadened my horizons. I look forward, very much, to each exam session (despite the workload) and being part of a process that carries with it a great deal of responsibility. It is always amazing (and rewarding) to read a good script that shows a real understanding of the question. I am involved with most stages of the exam process and seeing it through from start to finish is very special. I feel very fortunate to have this experience. Enquiry upon results: internal assessment One of several services offered by the IB is the enquiry upon results service that enables schools to request a re-mark of candidates’ externally assessed work, the return of work or a report on the moderation of marks for internal assessment based on sample work. In recent years this service has seen improvements in response to feedback from schools and with the aim of enhancing the integrity of assessment. With effect from May 2012, several changes will be introduced, but in particular a re-moderation procedure for internal assessment will replace the current reports for category 3. This is in direct response from schools requesting a service of this kind. The revised category 3 is a re-moderation of a school’s marks for internal assessment in a given subject and level based on the original sample material. (For some subjects this will be a combined higher level/standard level sample and therefore the re-moderation will affect both higher level and standard level candidates for the subject concerned.) Grades may be raised as a consequence of re-moderation, but not lowered. A re-moderation will only be undertaken in cases where the mean of the candidates’ moderated internal assessment marks differs from the mean of their raw marks (the marks awarded by the teacher) by 20% or more.
This new service aims to alleviate the concern in schools that currently there is no means to appeal against the outcome of moderating a school’s marks for internal assessment. However, for this new service to work efficiently, examiners must send candidates’ sample work for internal assessment to their team leader, or other senior examiner, as soon as possible if asked to do so. The IB would be most grateful to examiners for their cooperation in this regard, for the benefit of candidates concerned. Interview with Miss Jeehan Abu Awad, Principal Examiner, English B
Could you introduce yourself? I have been a DP English B teacher for the past 9 years with an overall teaching and administrative experience of 17 years. I am currently the Principal Examiner of Paper 1 SL, a language/English B workshop leader and online workshop facilitator in addition to my being an English B HL Internal Assessment Senior Examiner. This year, I am working as an English B Coordinator and a co-teacher at Cambridge High School in Amman, Jordan, and have held several similar and other positions in three other schools in my hometown. In November 2011 you were involved in assessing candidates’ work that was submitted using the IB’s audio upload facility. How did you find this experience? Being a staunch supporter of technology, I was quite eager to experience the newly introduced electronic upload option given to schools. In November 2011, I moderated some samples using the audio upload facility and found the experience a true enjoyment as it proved to be less time-consuming and more convenient than the conventional method used earlier. Does the audio upload function make your role as an examiner easier? Indeed! I have always found it agonizing to wait for couriers to deliver samples to my doorstep in addition to being bombarded with several CDs and 2/IA forms occupying a huge space of my working area. However, with this new audio upload facility, I can work on moderating samples at my own convenience with only a laptop and internet connection needed. I do not have to worry about samples arriving after the deadline, broken CDs, incomplete or missing 2/IA forms or even the space to spread all those around each time I start a new moderation session. Simply put, this facility has greatly made the process less strenuous and more efficient. Does your school use the audio upload function? Yes, it does. The facility would probably prove to be suitable for schools with a relatively small number of candidates.
Do you feel other components/subjects could benefit from the electronic upload/transfer of files? Of course! I strongly believe that each subject can benefit from this facility despite the individual differences pertinent to the nature of the internal assessment or components. The upload feature guarantees a safe and reliable transfer of files and/or suitable upload of any, such as lab reports, extended essays, etc, without worrying about loss or damage occurring. We are leading a highly competitive world in which technology plays a dominant role. Coping with and benefiting from any newly introduced facility means more effectiveness, reliability and ease. What has your relationship with the IB meant to you? Throughout the past nine-year experience with the IB I have continuously developed on both personal and professional levels. I have always viewed myself as an educator rather than a teacher, and that vision has constantly driven me to look for wider horizons to succeed in such a highly competitive environment and even lend a helping hand to all teachers and students seeking the same ambition. Working with the IB has equipped me with the necessary tools and made such realms accessible by allowing me to benefit from different sources of experiential knowledge and unique opportunities of professional development that ultimately helped me attain insight and gain the professional growth and satisfaction required. May 2012: some statistics May 2010 was a milestone for the IB; it was the first examination session for which there were over 100,000 candidates. Now, two years later, the number of candidates is nearly 120,000. Clearly, the growth in candidature is remarkable, with an equally positive growth in the number of subject registrations. A sustained increase in the number of candidates and subject registrations of course means that more examiners are required. Although we again have met our target for examiner recruitment to cover requirements for a May session, we must continue recruiting experienced educators for this role. So, if you have colleague who might be interested in becoming an examiner for the IB, please ask them to view the information on our public website, www.ibo.org. We are particularly interested in recruiting more examiners for history, psychology, philosophy and the language subjects for group 1 and group 2. Of the 119,438 candidates for May 2012, about 37,500 are registered for one or several individual subjects (known as certificate or course candidates), so a clear majority of candidates are pursuing the diploma. Looking ahead to May 2014, course candidates will be able to register for an extended essay and/or theory of knowledge. Until 2014 these have been confined to diploma candidates, so this change will also drive the recruitment of more examiners in the core subjects. Number of:
May 2011
May 2012
Increase
1,926
2,065
7.2%
111,500
119,438
7.1%
Raw marks required
1,784,946
1,923,996
7.8%
Grades to be issued
499,112
538,358
7.8%
2,308
4,088
77.1%
313,500
629,259
100.1%
Schools entering candidates Candidates
Examiners e-marking candidates’ work Candidates’ examination scripts e-marked
News in brief MYP: The Next Chapter The IB is to introduce an innovative new assessment for the MYP. The new optional assessments will be entirely online, include disciplinary and interdisciplinary material and are based on the MYP conceptual framework. We intend to trial the first versions of the eassessments in 2013 and they will be examined for the first time in June 2015. We will be appointing Chief Examiners in each of the subject groups later in 2012—if you are interested, please look out for advertisements on IBIS, www.ibo.org and Middle Years Programme coordinators’ notes. Disclosure of examiner role Examiners are reminded that their role as an IB examiner can only be disclosed under the conditions stated in the examiner contract, namely when applying for a position. The examiner role must not be disclosed in any other circumstance, such as when offering/advertising a service, on a social media profile or in an email footer. This is to ensure that the IB is not associated with any products or services that have not received official IB approval and whose quality we can’t assure. Chartered Institute of Educational Assessors One of our Chief Examiners is a member of the Chartered Institute of Educational Assessors (CIEA). She has asked us to let other examiners know about this association in case they might be interested in joining it. The IB does not actively support the CIEA, although we believe that it can offer helpful information to UK-based, English-speaking examiners. The CIEA is an independent professional body for the UK assessment community. It is dedicated to supporting the needs of examiners and individuals with an interest in or responsibility for assessment. Membership will give you access to the knowledge centre; you will regularly receive Make the Grade magazine and an e-newsletter with up-to-date information on UK assessments. You can find out more about the CIEA by going to www.ciea.org.uk. If you would like to become a member, email info@ciea.org.uk. Standards over time The primary method of maintaining standards over time is through the grade award process. However, it is necessary for further checks to take place to ensure that this process has been effective. While this is often achieved using various statistical approaches, such as looking at how candidates perform in one subject relative to another and tracking that performance over time, these methods have limitations. This is partly owing to a growing IB cohort with a potentially changing demographic (ie a sudden increase of candidates with a relative strength in a certain subject). The IB recently undertook a small-scale study looking at whether standards have remained consistent over time in some of our large-entry subjects. Boundary scripts (scripts chosen during the grade awarding process each session to represent the minimum standard required to achieve a certain grade), examination papers and markschemes in the chosen subjects from the May 1993, 2002 and 2012 sessions were scrutinized by either subject managers or senior examiners. The study evaluated whether the standard was the same in each year. In some instances this was complicated by the courses having changed. However, it was found that the standards were generally very consistent across the years. These findings will form part of a bigger project looking at standards over time and across
subjects being undertaken later in the year and further information will be provided about this in the next edition of the examiner newsletter. The examiners’ forum The forum, consisting of Nick Alchin, Vice-Chair of the Examining Board, senior IB staff and four examiners representing each IB region, is due to meet in October 2012. Please send any issues for the forum to consider to: examinerforum@ibo.org. The minutes of the last meeting are available on IBIS under Library > Examiner forum information. OFAC licence granted We’re delighted to inform you that the IB has been granted its Office of Foreign Assets Control (OFAC) licence. Though many in the academic community were surprised to hear that, as an apolitical foundation, the IB was required to comply with this legislation, no exemptions exist for charities or education services. As a foundation that abides by the laws in all territories where we hold a presence, we ensured our compliance by applying for a licence. We also implemented an OFAC policy and procedures to ensure compliance, including the difficult decision not to offer marking allocations to IB examiners from those countries until we had our licence. This licence will allow the IB to pursue its education mandate across the globe, further reaching students and IB educators in countries that were previously restricted by OFAC.
Useful contacts For i-expenses queries, please email IB Answers: ibid@ibo.org For e-marking queries, please email emarking@ibo.org or phone the e-marking help desk on: +44 29 2051 7114 To email any issues you wish to raise at the examiner forum: examinerforum@ibo.org For general queries or feedback and any suggestions for articles in further editions of the examiner newsletter please email: assessmentnews@ibo.org For examiner recruitment queries please email: examrecruit@ibo.org For examiner training queries please email: examtraining@ibo.org