Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers by NewGlobe

Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

Working Paper v2.6 Date: May 2016 Team: Catherine Naluwamba, Bridge International Academies. Jonathan Lehe, Harvard Kennedy School. Carol Yu, Bridge International Academies. Acknowledgments: Our great appreciation to Alex Eble at Columbia University insights; Beth Schueler at Harvard University for her insights. Finally, thank you to Republic of Uganda for their commitment to quality education. All errors are, of course, our own.

Submitted by:

https://newglobe.education/

Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

Contents 1. 2. 3. 4. 5. 6.

Executive Summary Bridge International Academies’ Teacher Vetting Process Recruiting Assessment Results Training Assessment Results Conclusion Reference

Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

Page 04 Page 06 Page 08 Page 10 Page 12 Page 12

Executive Summary Excellent teachers are central to Bridge's mission to provide pupils a high quality education. Accordingly, all prospective teachers who apply to Bridge go through a rigorous screening process. These candidates come to us with varied backgrounds and professional experiences. As part of its overall effort to refine its teacher selection model, Bridge considers these background data as it tracks the performance of candidates in its pipeline. The goal of these efforts is to examine which characteristics may predict better teacher performance. Ultimately, this allows us to put higher-performing teachers in classrooms for our pupils. This report examines data from 191 teachers who were hired to teach at 23 Bridge International Academies across Uganda in May of 2016. A key priority in Bridge's recruiting strategy was to hire as many government certified teachers as possible. Recruiters aggressively sought out government certified teachers from Primary Teachers Colleges (PTC) and the community. PTCs train teachers who teach in primary schools; the Grade III Teacher Certificate is the resulting certification for teachers who have graduated from PTCs after a two year training, and the certification required to teach in primary schools. Entrants must have at least a minimum of O-Level, with at passes in at least six subjects, including English, Mathematics, and two sciences (Teachers Initiative in Sub-Saharan Africa, 2013). PTC graduates or grade III teacher certificate holders do not necessarily have higher educational attainment than those that do not. As noted by the University of Sussex’s Centre of International Education, “some students join PTCs as an alternative to the higher school certificate (HSC) programme, either because they missed the grades, or because their parents cannot afford the fees for HSC” (Keyeyune, R., Mirembe, R., Baleetah, M., Sentongo, J., Nabi, R. and Katende, G). The Ministry of Education and Sports’ 2013 report states that “students Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

may apply to PTCs up to three years after completion of UCE – to allow those who fail A-level to apply for PTCs” (Teachers Initiative in Sub-Saharan Africa, 2013). Bridge is committed to providing a high quality education for underserved families and children in the poorest communities of Uganda. Given the dearth of PTC graduates in some areas, not all teachers who applied to Bridge were government certified, however. Moreover, recent UNEB guidelines to register schools as Primary Leaving Examination (PLE) examination centres allow for 1 out of 6 primary school teachers to work without a GIII certificate (Uganda National Examinations Board, 2016). This provided us with a data set that allowed for comparison between two groups: certified and non-certified teachers. Below, we summarise how government certified teachers and non-government certified teachers performed on recruiting assessments. Overall, Bridge finds that government certified teachers (who apply to teach at Bridge) perform slightly worse than teachers that are not government certified teachers on these assessments. This presents a challenge for Bridge as it seeks to hire more government certified teachers, and warrants further research and discussion. This trend is also present in training assessments; government certified teachers have slightly lower performance on Bridge Training assessments, and have higher voluntary drop-out rates from Bridge’s residential training than of teachers who are not government certified. Graph 1. Recruiting Examination Scores of Government Certified Teachers

Government Certified Not Government Certified

This paper does not discuss the performance of government certified teachers at large, and only describes the performance of government certified teachers who have applied and have been hired to teach at Bridge. Still, it is worth noting that the challenge presented at hand, and findings are not unique to Bridge alone. This is consistent with recent reports from the Uganda National Examinations Board (UNEB) describing low literacy rates of government certified teachers (Uganda National Examinations Board, 2015). According to the report, only 21.8% of pre-service teachers, or “Primary Teachers Colleges' Year-Two students,” (as described by UNEB Executive Secretaray Daniel Odongo) was rated as proficient in numeracy, and 38.8% proficient in literacy. Media outlets have Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

quoted UNEB Executive Secretary Daniel Odongo saying “these low results should be cause for worry because these students are now in the field teaching our children” during the release of the 2015 report (Ahimbisibwe, P., 2016). Studies in other contexts have suggested that formal teaching credentials are poor predictors of teacher effectiveness. A study in Andhra Pradesh notes “our results suggest that formal teaching credentials are poor predictors of primary school teacher effectiveness in India” (Muralidharan and Venkatesh, 2013), and studies in the U.S. have found that teaching credentials are poor predictors of teacher effectiveness as well (Kane, Rockoff and Staiger, 2008; Rockoff, Jacob, Kane and Staiger, 2008).

1. Bridge International Academies’ Teacher Vetting Process Bridge International Academies has a rigorous teacher recruiting and screening process, with four stages to ensure that our classrooms have teachers who demonstrate competence in content knowledge, pedagogy, and lesson delivery. We also ensure candidates have a strong connection to the community and are in good standing. The process is summarised as follows: Step 1: Sourcing of Candidates via Referrals and Community Outreach Candidates are sourced via recommendations from community leaders and stakeholders, local and state departments of education, and teacher training colleges. It is a priority that Bridge hire teachers from the community who are government certified. During this step Bridge actively engages Primary Teachers Colleges in order to recruit government certified teachers and meets with District Education Offices for referrals. Step 2: Initial Screen Bridge then reaches out to all qualified referred candidates and conducts a preliminary screening based on conversational English levels and other important logistical questions related to potential employment. For example, candidates are asked about their connection to the community, whether or not they can commit at least two years to the job, whether they are committed to work upon understanding work expectations, remuneration and benefits, and whether the academy is within close proximity to their current residence. The conversation is conducted in English as an initial verification of the candidate’s ability to teacher in English medium. Step 3: Written Examination of Content Knowledge Candidates who successfully pass the phone interview are then invited for a Written Examination and Interview. The Written Examination is an assessment composed of mathematics, reading comprehension, writing and grammar taught in primary school, in addition to basic cognitive ability to follow logical processes. This Examination was created to ensure that teaching staff have adequate content knowledge to teach a given grade level. Bridge also collects information about a candidate’s ties to the community to conduct reference checks on the candidate later on in the process.

Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

Step 4: Interview and Performance Assessments Successful candidates are then invited to interview with in person. This interview is composed of several performance assessments, and varies by grade level and subject specialty. All applicants are asked to deliver a lesson demonstration with a sample Bridge lesson guide that reflects the type of lesson guides taught for the grade level sought. This demonstration tests for the candidate’s level of confidence, clarity, and engagement in lesson delivery. Other assessments in this process include an in-person assessment of candidate reading fluency and pronunciation, letter sounds, phonics, blending, and a behavioral interview assessing job “fit.” These key English skills are key to successful teaching in an English medium. Step 5: Reference Check Bridge staff call members that share the same community connections as the candidate and complete a “blind” reference in order to determine the candidate’s work ethic, integrity, and passion for the teaching job. Step 6: Training Performance Assessments Successful candidates are then sent to Induction Training, and are hired as teachers with a 6 month probationary period. There are assessments throughout Induction Training to give formative feedback on key teaching techniques, but Induction Training also ends with a summative assessment that indicates teacher growth and overall performance.

Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

2. Recruiting Assessment Results Table 1 describes the cohort of teachers studied, a group of newly hired teachers attending a Bridge Induction Training in May of 2016. 42% of this cohort is government certified, and 58% is not. It is important to note, however, that these statistics are specific to this cohort; across the network, over 50% of our teachers are government certified. Table 1. Number of Government Certified teachers vs Not Certified in Cohort

Not a Government Certified TeacherGovernment Certified Teacher Government Certified Teacher

% of Teachers 111 (58%) 80 (42%)

Table 1 describes the cohort of teachers studied, a group of newly hired teachers attending a Bridge Induction Training in May of 2016. 42% of this cohort is government certified, and 58% is not. It is important to note, however, that these statistics are specific to this cohort; across the network, over 50% of our teachers are government certified. Table 2. Number of Government Certified teachers vs Not Certified in Cohort, by Gender Gender Male Female Total

Not Certified 29 26.13% 82 73.87% 111 100%

Government Certified 34 42.50% 46 57.50% 80 100%

Total 63 32.98% 128 67.02% 191 100%

Put another way, the percentage of female teachers who are government certified is lower (36%) than that of male teachers (54%). As described in the summary above, government certified teachers performed slightly worse than those that are not across all recruiting assessments. Overall, the range of scores for teachers who are not government certified teachers is higher than government certified teachers on all four Written Examinations (Mathematics, Grammar, Reading, Attention to Detail). In order to continue on to the next stage of the vetting process, a candidate must meet minimum viable scores across each exam. The difference in performance between the two groups translates into lower pass rates throughout the entire recruiting process for government certified teachers (versus not government certified teachers), creating a challenge for Bridge in its goal to hire as many government certified teachers as possible. Furthermore, as women were less likely than men to be government certified, hiring more certified teachers may have the unintended consequence of undermining another important goal: empowering women economically. Giving women in our communities the opportunity to become leaders and earn a steady income is critical, particularly as they are more likely to invest earnings into their own children’s education (Luke, Nancy, and Kaivan Munishi, 2011; Doss, 2006; Haddad, Lawrence, Hoddinott and Alderman, 1997; Schady, Norbert, and Rosero. 2008.) Table 3 summarises average scores for recruiting assessments, and also displays comparative pass rates for the two groups in the Examination portion of the recruiting process. Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

Table 3. Performance by Group on Recruiting Assessments

Average score

Recruiting Assessment

Description of Assessment

Combined Exam

Combined Score of Math, Grammar, Reading, Cognitive Ability

Math

Assessment of basic mathematics concepts

Max. Score

Government Certified Teacher

Not a Government Certified Teacher

58.5

62.1

13.8

14.1

Grammar

Assessment of basic grammar

16.7

Reading

Assessment of basic reading comprehension

14.2

15.6

Cognitive Ability (Attention to Detail)

Assessment of cognitive ability and attention to detail

13.9

15.3

Exam Pass Rate

Pass rate according to lowest minimum viable score

N/A

73.70%

78.40%

3. Training Assessment Results While Induction Training assessments occur after recruitment of candidates has concluded, they capture the quality of a candidate after a trainer has spent several weeks with the teacher, and see the teacher “in action” through simulated classrooms. The training assessment results further supports the findings from the recruitment assessments. Government certified teachers also had slightly lower scores on training assessments than not government certified teachers, as judged by teacher trainers (see Table 4). Government certified teachers also had higher voluntary drop-out rates than not government certified teachers (around 11 percentage points higher), adding further to Bridge’s challenge to hiring more government certified teachers. In exit interviews, the most frequently cited reasons for voluntary drop out for this group were a) needing to attend to family members who had fallen ill, and b) deciding to attend university. Other less frequently cited reasons included compensation, alternative job offers, and other types of family commitments.

Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

Table 4. Training Performance by Group

Average score

Description of Assessment

Training Assessment

Mid-Training Performance Assessment

Summative Performance Assessment

Max. Score

Trainer Assessment: How well teacher demonstrated techniques related to classroom management, lesson pacing, lesson delivery, feedback to students. Trainer assessment of teacher demonstration of teacher techniques related to classroom management, lesson pacing, lesson delivery, feedback to students.

Government Certified Teacher

Not a Government Certified Teacher

4.1

4.2

4.5

Table 5. Drop Out Rates at Training Overall rate Training Assessment Dropped out of training

Government Certified Teacher 17.50%

Not a Government Certified Teacher 6.30%

4. Conclusion Bridge is committed to providing a high quality education for underserved families and children in the poorest communities of Uganda. As these results demonstrate challenges not only for Bridge International Academies, but also for all non-state schools working in these communities, in hiring government certified teachers, Bridge will continue to track these statistics for future cohorts, and will also track the performance of government certified versus not government certified teachers over time throughout their employment. Persistent discrepancies in this direction may warrant further discussion of a longer term solution. In the short-term, Bridge will continue to aggressively recruit government certified teachers via increased engagement with communities, stakeholders, and Primary Teacher Training colleges, and welcomes feedback or suggestions from the broader education community regarding these findings. Bridge welcomes opportunities to collaborate with other researchers, and further investment to continue this body of research in Uganda, which will have implications for quality teaching in classrooms beyond Bridge academies. Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers

5. References Muralidharan, K., and Venkatesh, S. (20 October 2013). Contract Teachers: Experimental Evidence from India. National Bureau of Economic Research, Working paper no. 19440. Retrieved from http://www.nber.org/papers/w19440.pdf Rockoff, J., Jacob, B., Kane, T., and Staiger D. (2008). Can you recognize an effective teacher when you recruit one? National Bureau of Economic Research, Working paper no. 14485. Retrieved from http://www.nber.org/papers/w14485 Keyeyune, R., Mirembe, R., Baleetah, M., Sentongo, J., Nabi, R. and Katende, G. Main Features of Teacher Education. Retrieved from http://www.sussex.ac.uk/cie/projects/completed/tpa/uganda Teachers Initiative in Sub-Saharan Africa. (2013, August). Teacher Issues in Uganda: A Diagnosis for a Shared Vision on Issues and the Designing of a Feasible, Indigenous and Effective Teachers’ Policy. Retrieved from http://www.education.go.ug/files/downloads/TISSA%20Uganda%20Full%20Report_24%20A ugust%202013%20edited%20version%20moses.pdf Uganda National Examinations Board. (2015). Achievement of Primary School Pupils and Teachers in Uganda in Numeracy and Literacy in English. Retrieved http://uneb.ac.ug/downloads/2015%20NAPE%20PRIMARY%20SUMMARY%20REPORT% 20.pdf Uganda National Examinations Board. (2016, January). Validation of New Examination Centres for 2016 PLE (p. 1, Issue brief).

Teacher Quality at Bridge Recruiting Assessment Scores of Government Certified Teachers