Figure 35 Sensitivity of coefficient estimate to varying maximum possible r-squared

Table 11 Comparison of Bridge with private schools

Regression with no controls (Table 10, Col 1) Regression with controls (Table 10, Col 2) Effect size if selection on unobserved variables = selection on observed variable (Beta, β if σ = 1)

Literacy

Coefficient on Bridge Dummy R squared Selection

0.351 0.045

0.233 0.128

0.161 1

Required relative selection on unobservables (Delta, σ) for true effect size to be equal to zero (β = 0) R Max (Assumed maximum possible R squared is equal 1.3 times the R-squared achieved in the regression with controls (Table 10, Col 2) 0.166 2.232

This estimate of σ is based on an assumption that the maximum possible R-Squared is 1.3 times the Rsquared reached in the fully controlled model. This value is proposed by Oster (2016) based on an analysis of 65 randomized studies from top five economics journals. 90 percent of the randomized studies are robust to a R Max of 1.3 times the r-squared reached in the fully controlled model. We demonstrate the sensitivity of the main estimate to varying the maximum possible r-squared in the figure below. For the true coefficient to be zero, the maximum possible R-squared would need to be 1.7 times the R-squared achieved in the fully controlled model.

Figure 35 Sensitivity of coefficient estimate to varying maximum possible r-squared

All of this suggests that the observed gap between Bridge and private schools in literacy is unlikely to be driven by selection on unobserved variables30 . We do not apply the Oster method for public schools. This is because the gap is so large and remains large even with factoring in for observables, that it’s highly unlikely that selection is driving the gap.

30 Note that crucially, this only applies to which are related to the observables and share the same covariance properties as the observables.

DFID’