10 minute read
2 The Kuunika Project in Malawi
This section presents an overview of the Kuunika Project and the ‘essentials’ of the governance components. This, in turn, provides the context for the specific questions underpinning this study enquiry.
2.1 Design features of the Kuunika Project
The Kuunika Project has aimed to establish a strong base of high-quality, routinely-available data and a culture of data use in the Malawi health sector, using HIV as a first use case. Together with the Government of Malawi (GoM), the Project has sought to strategically and efficiently strengthen the data systems architecture, while simultaneously evaluating targeted methods to increase data use for decision-making at facility, district and central levels.30
Notably, Kuunika has been characterised by an adaptive project design approach from the outset. Delivery of the Kuunika Project was based on a multi-phase implementation plan to allow for innovation, evaluation, and iterative responses. Phase I (scheduled for 2016-2019) included formative research and mapping of the ‘enterprise architecture’. This informed the design of a full package of support to five districts including Lilongwe, Blantyre, Zomba, Thyolo and Mangochi and three facilities in Mulanje. It was expected that Phase II would be based on scale-up to additional districts and sites, prior to a final phase of sustaining programme gains and translating lessons learned into policy and further roll-out.
The Kuunika Project was initially set up to be implemented by a consortium of four organisations: Lighthouse Trust (LHT); Baobab Health Trust (BHT), Luke International, Norway (LIN); and International Training and Education Center for Health (I-TECH); additional technical support has been provided by Cooper-Smith.
In practice, Phase I delivery proved slower than expected. The Kuunika team, therefore, agreed with BMGF to undertake a ‘project pivot’ to prioritise activities that would more rapidly ‘unlock defined key capabilities’ in HIV services. From November 2018, this involved a tighter focus on existing sites (rather than rolling out to new ones), accelerated delivery of specific data products to encourage data use, and a revised training approach. There were also new targets for improving the underlying system architecture, including the Health Facility Registry, the Demographic Data Exchange and the interoperability layer. On the back of this project pivot there was a significant reconfiguration of the consortium. This resulted in LIN becoming the sole Implementing Partner (drawing on consultant support from the original consortium partners), with additional technical assistance provided by Cooper-Smith.
Annex 1 shows the Kuunika theory of change developed by the evaluation team in 2019 which incorporates the 2018 / 2019 project pivot. This version of theory of change updated the baseline theory of change and shows the priority activities / deliverables agreed at each system level. This version of the theory of change shows that, by 2019, intended governance deliverables included: a) contributions to the National Standards Registry through Standard Operating Procedures (SOPs) and approval procedures and b) inclusion of data quality assessment tools within DHIS2.
The 2019 theory of change also included the evaluation team’s review of the activity pathways underpinning the theory of change. These activity pathways indicated key outputs for successful
30 Kuunika. (2019). Press Statement for the Kuunika Core Package Launch. Retrieved from: https://www.kuunika.org/?p=3340
delivery of intended Kuunika outcomes would be the establishment of a Master Health Facility Registry and a Demographic Data Exchange (to support data sharing and patient mobility).
A further iteration of the Kuunika theory of change is under development for the final project evaluation. However, this Special Study will reference the 2019 theory of change (Annex 1) to track further evolutions in the project design, focusing, in particular on digital health governance themes in the final phase of implementation.
2.2 Mapping key implementation milestones
Figure 131 below shows a reconstructed timeline of key project milestones relevant to review of Kuunika contributions to digital health governance themes in Malawi since 2016.
Figure 1: Key Kuunika milestones with implications for digital health governance
Figure 1 shows that in Phase 1 (2016-2018) there was a focus on formative research, with particular emphasis on opportunities to improve data use for decision-making at all system levels. This, in turn, was accompanied by efforts to improve access to timely, quality data through roll-out of EMRs, with the HIV programme as the use case. 32
As indicated above, in Phase 2 there was a rationalisation and restructuring of the project, initially to refocus efforts on systems strengthening and accelerated delivery of key data products to support data use.
Around this time, CDC which was using PEPFAR resources to co-fund BHT’s EMR activities, transferred its investment to the Elizabeth Glaser Paediatric AIDS Foundation (EGPAF) to ensure more timely reporting of age and sex disaggregated HIV data (a condition for PEPFAR fund releases). Also in 2019, Vital Wave conducted an assessment of EMRs in Malawi to inform the governments new National Digital Health Strategy. This highlighted wider issues of: fragmented EMR initiatives that limited holistic patient-care; a disproportionate focus on HIV; variability of EMR data quality; lack of a coherent interoperability framework: lack of standard
31 Adapted from Kuunika’s timeline in: Cooper-Smith. (2020). Kuunika: Data for Action - Investment Overview, Successes, Lessons, and
Thoughts for the Future. 32 Lighthouse Trust International. (2016). Grant Proposal Narrative for the Bill and Melinda Gates Foundation.
protocols for sharing patient records; ongoing issues of patchy use of EMRs; limited interconnectivity and intermittent power supplies across the country; and some fundamental concerns about sustainability and government ownership.33The convergence of these events had implications for Kuunika’s design focus on EMRs.34
A further significant milestone occurred at the end of Phase 2 when the GoM requested Kuunika to become the lead implementing partner for its new Digital Health Division – this ongoing role includes support to some 26 staff in the Division. A key initial task area was technical assistance for development of the National Digital Health Strategy, 2020-2025. Although a robust Strategy was produced, the period 2019-2020 was associated with two national elections, leadership changes in MoH, and the outbreak of the COVID-19 pandemic –all of which have had consequences for operationalising the Strategy.
2.3 Independent evaluations
Mott MacDonald conducted independent evaluations of the Kuunika Project in 2017 (baseline) and 2019 (midline). The 2017 baseline evaluation report used a theory-based, mixed method approach to generate a baseline for the Kuunika design objectives relating to: core information technology (IT) infrastructure; strengthening and scaling up EMR systems; training, mentoring and incentivising target users; and - importantly for the purposes of this study - assisting in the establishment of MoH data governance structures and support use of quality for data-driven decision making. The baseline evaluation confirmed the presence hybrid electronic and paper-based systems for collection of HIV data, and highly fragmented and parallel systems for registering and providing HIV (and other primary health care services) at facility level. It was noted that, at baseline, there were mixed stakeholder perceptions of HMIS data quality, and there was little use of electronic data for decision-making at each system level. Key baseline findings relating to digital health governance are summarised in Box 1 below.
Box 1: Key baseline evaluation findings on digital health governance
• Data extraction: Both EMR and DHIS2 data were perceived to ‘funnel up’ data from facilities and districts to the national level; zonal health teams in particular felt cut out of data use loops. • System time lags: There were concerns at national, zonal and district levels about the time lag between data collection and feeding into the DHIS2 system. • Donor priorities: There was a widespread perception that donor interests and priorities had led to multiple vertical reporting systems – including those relating to antiretroviral therapy (ART) and HIV services. • Standard Operating Procedures: Knowledge of health data Standard Operating Procedures (SOPs) and guidelines appeared to be low at health facility level; there was little coherent supervision on the application of the SOPs and guidelines. There was also lack of clarity among district level respondents as to how the DHIS2 and EMRs would apply governance systems, instructions and guidance.
33 Vital Wave. (2019). Assessment of EMR Systems in Malawi: Initial Landscape Assessment. Prepared for the Ministry of Health,
Republic of Malawi. February 2019. 34 We note these challenges are typical of EMR projects in low-income settings (i.e. a disease / indicator focus, a donor lens, lack of harmonised Supplier procurement, lack of digital health curation and lack of interoperability infrastructure). See Jawhari, B et al. (2016). Benefits and challenges of EMR implementations in low resource settings: a state-of-the-art review. BMC Med Inform Decis
Mak 16, 116 (2016).
The midline evaluation found that by 2019 paper-based data systems continued to predominate. The extent of use of an EMR to enter data had remained largely unchanged at just over one in three (38%) respondents – with power and connectivity issues being a key factor in consistent EMR use. There was some evidence of a stronger, more active‘data culture’ by 2019; however, the enabling environment for using data within the larger health system remained weak. Very few respondents at facility and district levels had knowledge of the key technology deliverables relating to Unique Identifiers, the Health Facility Registry and the Demographics Data Exchange (DDE). Key midline findings on progress relating to digital health governance are summarised in Box 2 below:
Box 2: Key midline evaluation findings on digital health governance
• Progress on governance objectives: Progress against Kuunika’s governance objectives remained unclear. At facility level, few respondents mentioned issues of data governance, data protection or institutional responsibilities for data safeguarding – although there was some concern about potential breaches of data privacy in the use of EMRs, especially in busy facilities. At district level, some respondents requested more consultation with them on key performance indicators and clearer guidance from Kuunika on data governance themes. At national level, one respondent observed:
“Oversight [or its lack] and overall data governance are a big issue and a Kuunika weakness.
Kuunika still hasn't got a clear, strong oversight structure to and from the MoH, and I just don't know what it is doing in terms of assuring data protection.” (National Senior Health Manager)
• Standard Operating Procedures: Although 10 SOPs had been drafted there was concern about the lack of stakeholder consultation in the drafting process, and the possible implications for SOP technical quality and relevance. For example, only one SOP on Data Access and Release made substantive reference to data governance and data confidentiality and was technically weak on issues of data sharing and requirements for informed patient consent. Similarly, the Guidelines on
Privacy, Security and Service Continuity of HMIS in Malawi focused on ICT personnel only, and made little reference to the responsibilities of other stakeholders. • Interoperability: The evaluators observed interoperability was as much an organisational and political challenge as a technical one – largely because it required complex, detailed and continuous agreement and enforcement across multiple organisations in the health system. By 2019, there was little evidence of progress on the non-technical aspects of the interoperability investment.
• Greater focus on data governance: The evaluators concluded that data governance remained a weak area in terms of Kuunika delivery and that, “data protection and governance [would] definitely require further attention, e.g. in terms of clear guidance and SOPs that speak to the importance of protecting what are both very personal and potentially commercially valuable data”.
Based on the findings from the baseline and midline evaluations, and a preliminary desk review for the endline evaluation, the evaluators have identified a number of topics for ‘deep-dive’ special studies. The topics and the specific questions for enquiry have been agreed with BMGF, GoM and the Kuunika leadership. They are informed by the Kuunika team’s own progress review in December 202035 - key points from this review are summarised in Figure 2 below.
35 Cooper-Smith. (2020). Kuunika: Data for Action - Investment Overview, Successes, Lessons, and Thoughts for the Future
Figure 2: Summary of Kuunika's progress review (Dec 2020)
Special Study 2 will focus on ‘Privacy, Data-Sharing, and Intellectual Property - the Regulatory and Policy Essentials for Kuunika’. The specific questions agreed for this study are summarised in Box 3 below. By addressing these specific questions, we will aim to address the study objectives of: a) surfacing the governance, regulatory and policy challenges / barriers to EMRbased data integration and sharing in Malawi and b) identifying specific recommendations for GoM, BMGF and other global health donors.
• Why has the important step of establishing a Demographic Data Exchange not yet been completed as a core technology deliverable? • What role did implementation and aid effectiveness issues, such as project design, project management and donor co-ordination play? • How useful would intellectual property regulation, data privacy & global digital governance standards have been in creating a more propitious context? • What are the lessons for future BMGF programmes?
