Southwest Basic Education Project (SBEP)

controllable, the study required a minimum of 200 students in the control group of each grade at each phase of testing and 2% of student population in the project intervention group of at each phase. A stratified random sampling method was applied at the Baseline test to draw the school samples. It was advised that the same county and schools sampled in the Baseline test would be used for the MTR and EoP tests during the project period. Table 1 presents the actual numbers of schools and students drawn from project counties for the Baseline test.

This design implied a sample of above 30,000 students in the project counties and at least 800 or more students in the non-project counties being tested at the Baseline test (2007), mid-term review (MTR, 2009) and end of project evaluation (EoP, 2011) respectively. It was assumed that a much larger sample size than what a well-controlled randomized trial would require should give enough power to detect moderate changes in students’ mean test scores over time that could be attributable to project intervention.

As a concern of comparability of the two groups of students, Table 2 below presents 3 key student characteristics between project and non-project district samples for the Baseline test in 2007.

The project and non-project samples were comparable in their distributions of students’ gender and age overall, but there were large differences in the distribution of minority students. This was because the nonproject sample was matched by school and township socio-economic status. Only schools with a high concentration of minority students in those counties could be close to the similar social and economic status (SES, please refer to 3.1 for more details) status for the match. Future advanced data analysis should be able to take such difference into consideration.

Southwest Basic Education Project (SBEP): Analysis of the impact of SBEP on student achievement