5.3
Learning outcomes
The raw treatment effects on test scores for each individual contractor shown in Figure 6 are internally valid, but not comparable. They are positive and significantly different from zero for three contractors: Rising Academies, Bridge International Academies, and Street Child. They are positive but statistically insignificant for Youth Movement for Collective Action, More Than Me, and BRAC. The estimates which we label as “comparable treatment effects” differ in two respects: They adjust for baseline differences and “shrink” the estimates for smaller contractors using the Bayesian hierarchical model. While the comparable effects are useful for comparisons, the raw experimental estimates remain cleaner for noncomparative statements (e.g., whether a contractor had an effect or not). Intention-to-treat (ITT) treatment effects are shown in Figure 6a. That is, the effect over both compliers and non-compliers (i.e., over all students enrolled in a treatment school in 2015/2016, regardless of whether they attended an actual PSL school in 2016/2017). Treatment-on-the-treated (ToT) treatment effects are shown in Figure 6b. That is, the effect over compliers (i.e., students who actually attended a PSL school in 2016/2017). Non-compliance can happen either at the school level (if a contractor opted not to operate a school or the school did not meet the eligibility criteria), or at the student level (if the student no longer attends a treatment school). Comparable ITT treatment effects across contractors from the Bayesian hierarchical model are also shown in Panel A of Table 12. Table A.12 in Appendix A has the raw experimental treatment effects by contractor.
44
