2.3 Independent evaluations

2.3 Independent evaluations

Mott MacDonald conducted independent evaluations of the Kuunika Project in 2017 (baseline) and 2019 (midline). The 2017 baseline evaluation report used a theory-based, mixed method approach to generate a baseline for the Kuunika design objectives relating to: core information technology (IT) infrastructure; strengthening and scaling up EMR systems; training, mentoring and incentivising target users; and assisting in the establishment of MoH data governance structures and support use of quality for data-driven decision making. The baseline evaluation confirmed the presence of hybrid electronic and paper-based systems for collection of HIV data, and highly fragmented and parallel systems for registering and providing HIV (and other primary health care services) at facility level. It was noted that, at baseline, there were mixed stakeholder perceptions of HMIS data quality, and there was little use of electronic data for decision-making at each system level. The midline evaluation found that by 2019 paper-based data systems continued to predominate. The extent of use of an EMR to enter data had remained largely unchanged at just over one in three (38%) respondents – with power and connectivity issues being a key factor in consistent EMR use. There was some evidence of a stronger, more active ‘data culture’ by 2019; however, the enabling environment for using data within the larger health system remained weak. Very few respondents at facility and district levels had knowledge of the key technology deliverables relating to Unique Identifiers, the Health Facility Registry and the Demographics Data Exchange (DDE). Based on the findings from the baseline and midline evaluations, and a preliminary desk review for the endline evaluation, the evaluators have identified a number of topics for ‘deep-dive’ special studies. The topics and the specific questions for enquiry have been agreed with BMGF, GoM and the Kuunika leadership. They are informed by the Kuunika team’s own progress review in December 20206 - key points from this review are summarised in Figure 2 below.

6 Cooper-Smith. (2020). Kuunika: Data for Action - Investment Overview, Successes, Lessons, and Thoughts for the Future