14
15
B. Confirmation of Balance Between Treatment and Control Schools As some information was not available in the SUBEB Lagos 2018-19 census, we also surveyed schools prior to assessing pupils. For each grade, we noted the officially rostered number of teachers and pupils, then compared these numbers to how many teachers and pupils we observed at the school. We also collected information on the number of arms and physical classrooms. Class sizes are slightly larger in EKOEXCEL classrooms, though the difference is not statistically significant.
Figure 8 (A). Sample Size by Grade in Treatment vs. Control Schools Count of Pupils Across Unique Number of Schools, Physical Classrooms, Arms, & Teachers
Figure 7. Summary Statistics on Class Sizes, Pupil Attendance, & Teacher Attendance Baselines School Survey Data In schools with more than 20 pupils per grade, we randomly assessed 20 pupils proportional to the enrollment within each physical classroom and the distribution of boys and girls within the grade. When class sizes are unequal, we include more pupils from larger classrooms. Similarly, grades with more boys than girls would have proportionally more boys than girls in the sample. For each assessed pupil, we captured basic pupil demographic and schooling information, including gender, age, whether or not the pupil attended early childhood education, speaks English at home, or had breakfast, lunch or dinner the previous day. There were no meaningful differences between treatment and control schools, except for the percentage of pupils who speak English at home (KG and P5). In both cases, more pupils in EKOEXCEL schools reported speaking English at home: 8% more pupils in KG and 7% more pupils in P5. Figure 8 (B): Pupil Characteristics
C. Pupil Sample: Counts and Background Characteristics Our study design required the assessment of 20 pupils each in kindergarten, Primary 2, and Primary 5 in order to detect an effect size of 0.35 standard deviations per grade. We will see an EKOEXCEL effect only if pupils learn approximately 50% more than what they would have learned without the EKOEXCEL program. During baseline assessments, we discovered that certain grades in particular schools, especially kindergarten, had fewer than 20 pupils present. In these cases, we assessed all pupils present. We also encountered situations that did not match our initial census information: for example, at one control school, we discovered that there was no true kindergarten class. Pupils in “kindergarten” were much younger and should have been in nursery class. In some EKOEXCEL schools, not all teachers attended EKOEXCEL training. We did not include classes taught by these teachers as that teacher and those students are yet to start the program. This excluded 1 kindergarten and 3 Primary 5 classes from the study. The study sample (see Figure 8 (A) below) for treatment and control schools includes similar numbers of pupils, schools, arms, physical classrooms, and teachers at each grade level. Fewer pupils are in the study than our initial target of 1200 pupils per grade because some schools had fewer than 20 pupils per grade in attendance.
9
D. Identifying the Best Sample Amidst Assessment Disruption Our intended sample included nearly 2900 pupils roughly split between 30 schools participating in EKOEXCEL (treatment) and 30 schools not yet participating in EKOEXCEL (control). In the midst of our endline data collection, all Lagos schools were shut down due to the COVID-19 pandemic. At the time of the school closures, we had collected about 40% of the data for our endline report. For one group, Primary 2 girls, we were fortunate to have collected data from roughly 80% of the pupils at nearly all schools. This allowed us to estimate the EKOEXCEL effects for this group with reasonable precision. Thus, we present results from Primary 2 girls first.
The EKOEXCEL Effect
10
