VIII. Review and quality assurance of the rating system

VIII. REVIEW AND QUALITY ASSURANCE

OF THE RATING SYSTEM

The rating system will be reviewed regularly by IEO to ensure its applicability to the intended objective – the appropriate assessment of country programmes – as well as its consistent use among lead evaluators.

The process involves:

A. A peer working group, comprising the lead evaluators and those engaged in conducting ICPEs each year, will convene to facilitate the sharing of lessons and practices.

B. Internal and external review of ICPE draft reports: The rating appraisal will be part of existing IEO internal and external quality assurance reviews of all ICPE draft reports. Attention will be paid to ensure internal consistency between report narratives and the ratings awarded. Specific comments and requests for clarification on the rating summary, submitted by the country offices and Regional Bureaux, will be carefully documented to inform potential refinements of the rating system and its presentation.

As part of the internal and external reviews, each ICPE team will present the results of output-level rating analysis, together with the spreadsheet, to enable review of the application of the rating before the evaluation is finalized.

C. An external review mechanism will be constituted to annually review the quality of ICPEs as well as the rating process. The external review will assess the application of the rating system and check inter-evaluator variability. The quality of ICPEs will be assessed using an established set of parameters.

Consultants will be hired for this purpose.

Once the rating system is fully internalized, an external review will be carried out on a biennial basis (every two years).