Annex 4: Learning Samples

Tetra Tech, August 2022 | 41

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Key: darker shades of green indicate higher values

Table 17: Baseline learning sample sizes by project and subtask (numeracy)

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Key: darker shades of green indicate higher values

Tetra Tech, August 2022 | 43

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Table 18: Midline learning sample sizes by project and subtask (literacy)

Tetra Tech, August 2022 | 44

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Midline learning sample

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Table 19: Midline learning sample sizes by project and subtask (numeracy)

Tetra Tech, August 2022 | 46

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Key: darker shades of green indicate higher values

Tetra Tech, August 2022 | 47

Annex 5: Weights for Learning Analysis

Project-equal weights

Project-equal weights, or “inverse sample weights”, were created with a view to give the same weight to each project in the portfolio-level analysis. They are equal to the inverse of the actual learning sample size of each project (number of girls who were given each subtask of the learning assessments). For the sake of clarity, this ratio has then been multiplied by 1000 (the value of the multiplicative coefficient does not matter as long as the same coefficient is applied throughout).

For example, a project who gave the EGRA familiar word subtask to exactly 400 girls at midline will be given a weight of: 1 / 400 * 1000 = 2.5. This means that the EGRA familiar word scores of each of the 400 girls will be assigned a weight of 2.5

Weights vary widely by project, and to a lesser extent by subtask. This reflects the fact that GEC-T projects have very different learning sample sizes, and that some subtasks have different sample sizes than others (for example when they were given to different grade cohorts).

Analysis based on project-equal weights estimates the average effect of the GEC-T across projects (weighting them equally). Weights are used for any type of analysis, from descriptive analysis (simple averages, cross-tabulations) to regression analysis and difference-in-difference estimates. In Stata, they are accounted for as analytical weights, using the standard option [aw = weight value].

Table 20: Project-equal weights – literacy assessments

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Key: darker shades of green indicate higher values

Tetra Tech, August 2022 | 49

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Table 21: Project-equal weights – numeracy assessments

Tetra Tech, August 2022 | 50

Independent Evaluation of the Girls’ Education Challenge Phase II –

Key: darker shades of green indicate higher values

Beneficiary-population weights

Beneficiary-population weights are the weights used by default in the report’s body. They are based on the target number of learning beneficiaries for each project and subtask. The principle is the same as for project-equal weights, except that instead of using an invariant multiplicative coefficient of 1000, the multiplicative coefficient is made proportional to the number of learning beneficiaries of each GEC-T project. The share of GEC-T learning beneficiaries by project has been obtained through FM documentation and is reproduced in Table 48 of the Learning analysis annex (Annex 9)

With beneficiary-population weights, learning scores at the portfolio level are made proportional to the relative ‘size’ of GEC-T projects, with larger projects (with higher numbers of learning beneficiaries) overweighted compared to smaller projects.

Beneficiary-population weights provide estimates of the effect of the GEC-T on the average (learning beneficiary) girl.

Table 22: Beneficiary-population weights – literacy assessments

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Key: darker shades of green indicate higher values

Tetra Tech, August 2022 | 53

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Table 23: Beneficiary-population weights – numeracy assessments

Tetra Tech, August 2022 | 54

Independent Evaluation of the Girls’ Education Challenge Phase II – Aggregate Impact of GEC-T Projects Between Baseline and Midline Study - Report Annexes

Key: darker shades of green indicate higher values

Tetra Tech, August 2022 | 55