Table 23. EGMA Reduction in Zero Scores: Linear Probability Model (9) Quantity Disc. KG-G2
(10) Addition 1 KG-G3
(11) Addition 2 G2-G3
(12) Subtraction 1 G1-G3
(13) Subtraction 2 G2-G3
(14) Word Problems G1-G3
Bridge
0.02 (0.87)
0.04 (0.55)
-0.12 (0.42)
0.12 (0.27)
0.07 (0.52)
0.08 (0.55)
Grade 1
0.07 (0.57)
0.14 (0.12)
Grade 2
0.13 (0.42)
0.18+ (0.08)
Grade 3
-0.07 (0.62)
achieved the reduction in zero scores goals in all three grades. This is particularly notable for Grade 3, for which the benchmark requires a zero score percentage of less than 5%.
7. Exploring Heterogeneous Impacts 7.1 Gains for Students by Percentile Although the results provide a strong signal that Bridge PSL has a differential impact on student outcomes overall, it is critical that every child is learning regardless of their incoming ability level. To investigate whether gains achieved vary by students’ baseline literacy and numeracy levels (heterogeneous impact), we can look at the performance of students across different percentiles across the academic year. Figure 10 shows student baseline and endline performance for the 10th, 25th, 50th, 75th, and 90th percentile at both Bridge PSL public and traditional public schools on Passage Fluency.
-0.20* (0.03) -0.32* (0.02)
0.17 (0.18)
0.18 (0.17)
-0.05 (0.75)
-0.02 (0.84)
Constant
0.75** (0.00)
0.68** (0.00)
0.74** (0.00)
0.71** (0.00)
0.55** (0.00)
0.96** (0.00)
Count R-squared
56 0.02
112 0.04
59 0.06
109 0.03
111 0.01
52 0.04
Figure 10. Passage Fluency Performance of Students by Percentile Passage Fluency 80 70
Words per Minute
6.5.3 Benchmarks for Reducing Zero Scores The benchmarking workshop that set literacy benchmarks by grade also set goals for reducing the percentage of students who score zero on non-word reading, passage fluency, and reading comprehension. These recommended percentages, described as “ambitious,”60 are shown in Table 24 below along with the progress of students at each school type towards these goals.
Table 24. Reducing Zero Score Benchmarks
60 50 40 30 20 10 10th
25th
50th
75th
90th
Percentile Traditional Baseline
Bridge PSL Baseline
Traditional Endline
Bridge PSL Endline
On some subtasks such as Passage Fluency, many students starting at the 10th percentile did not realize the gains of students with higher incoming skill levels. However, on the majority of subtasks, particularly on EGMA, gains were fairly evenly distributed across the entire spectrum of scores. In other words, students at Bridge PSL exhibit strong learning gains irrespective of baseline performance. Figure 15 and Figure 16 in Appendix A10. Heterogeneity of the Treatment show the heterogeneous impact of Bridge PSL across subtasks. As with the performance benchmarks discussed in the previous section, Bridge PSL public schools made noticeably more progress toward these goals, as the percentage of zero scores is lower at Bridge PSL public schools on all literacy skills in all grades. In fact, on Passage Fluency, Bridge PSL public schools actually 60
RTI International (2014).
Learning in Liberia: Literacy and Numeracy Gains in Year 1
Formal testing for equality of treatment effects at different points of score distributions can be seen in Table 50 and Table 51 in Appendix A10. Heterogeneity of the Treatment.61 Students were divided into terciles (low, medium, and high) based on their performance at baselines. We then tested to see whether 61
50
Methodology for testing heterogeneity of the treatment comes from Muralidharan et al. (2017).
Learning in Liberia: Literacy and Numeracy Gains in Year 1
51
