Southwest Basic Education Project (SBEP)

Table 5: Frequency of repeated tests by schools

Thirdly, there were possible selection biases in non-project schools in favour of high performers. Table 6 shows overall mean test scores of students at Grades 3 and 5 in Chinese testing for project and nonproject counties by three phases of tests. While an increased mean score over time is observed in both project and non-project samples, such change in the latter at the EoP phase was dramatically large. The same pattern was found in Maths mean scores for the two grades too. Further enquiry in the EoP data revealed that the non-project county selected in Yunnan province came from the Xishan District of Kunming City, the capital of the province where schools had much better education quality with better resources than the SBEP project counties.

In the analysis of relative changes over time, such selection bias will clearly favour students from nonproject counties. If analysis showed that students from non-project counties were found making greater progress over time than those from project counties based on the data, the result would not be reliable due to such selection bias. However, if the analysis showed a greater progress from students of project counties than those from non-project counties over time, some degree of positive project impact which would be larger than that observed in the data if the selection bias was not in favor of the latter group could be reported