6. Synthesis of Key Findings

In summary, key lessons include:

Decentralization, Districts & Kuunika

• Neither a decentralized 'decision space' for health planning nor functions + capabilities have been effectively supported

• Development of digital health guidelines + systems should be supported more consistently at the District level

• That support should be based on a more coherent approach to the development of an organizational knowledge culture: starting with the users, not the system

• The Covid-19 pivot has not resulted in clear wins for DHOs + Health Facilities (+ perhaps communities): primarily seen as data extraction without effective support.

Sustainability

• Kuunika's key partner is the MoH; however, sustainability of digital health systems requires support to Districts and lower levels, as the central point of service deliveryfor HIV as well as universal health coverage

• The project support to the development of the MoH Digital Health Division represents not only a solid + (it is hoped) sustainable gain; the DHD could also serve as a central point for more aligned + harmonized MoH + partner engagement in work on digital health data systems + use, e.g. if the 'SWAp revival' concept note results in more coordinated inputs

• Kuunika inputs to e.g. the 2020-2025 National Digital Health Strategy represent normative + longer term contributions that can underpin more sustainable digital data architecture

Aid effectiveness

• There continues to be data extraction, imposition of donor partner data systems + indicators, lack of DHO + Council engagement in planning + creation of context-specific plans.

• There is scope (+ urgency) for greater coherence + genuine partnership at District level in digital data collection + use.

• There is fragmentation of effort, due to lack of donor co-ordination across the many health programs and projects. At District level, DHO, Council and Health Facility staff are overwhelmed by multiple demands and reporting formats. This affects the efficacy and efficiency of DHIS2. Alignment of all partners, with the active engagement of District actors, is essential.

• Districts continue to be the 'missing middle' in terms of the digital data architecture

Key findings

1. Decentralization of the health system and digital data is beyond the control of any project, including Kuunika. Information is that there are currently (mid-2021 onwards) changes being made at national government level specific to what might be seen as a (re) centralization of digital health data management and (re) focus on DHIS2. Thus the Digital Health Division has been moved back into the MoH Department of Planning and Policy Department.

2. The move might limit, even reduce, districts' and health workers' capacity to review and make speedy use of data; such moves might also have impacts on the building of digital data systems and knowledge culture at District and facility levels.

3. The partial, piecemeal and stop-start implementation of the government policy of decentralization is viewed as a cause of poor governance in the health sector Governance challenges are a significant barrier to achieving a more effective and equitable health system in three key domains: accountability (enforceability; answerability; stakeholder-led initiatives); health resource management (healthcare financing; drug supply); influence in decision-making (unequal power; stakeholder engagement).

4. Districts are the 'missing middle' in many respects in terms of GoM and donor partner engagement with the health sector; there is increased focus on community engagement (and quite rightly, not least for equity considerations), yet Districts are the entry point to the great bulk of health services delivered to Malawians.

5. When Kuunika was being designed, the GoM said it was decentralizing the health system, but most decisions and human resource management continued to be made at the central level. There is a centralized 'push system' in health. Most projects at District level have had to work very closely with the MoH, even if more focused on the Districts; this has been true for Kuunika.

6. The MoH is the owner of Kuunika - this has been made clear from project inception. Therefore, has the project made best use of the space available within the predominantly centralized health systems and structure to engage with Districts to build sustainable, standalone use of digital data? The answer is on balance no.

7. The cumulative experience of the Kuunika evaluation shows that there needs to be greater genuine ownership of data at the District level. This necessitates buy-in from all partners working in any one District and nationally, challenging in a donor landscape currently as fragmented as Malawi; this has obvious implications for effective coordination of digital health initiatives. The District Health Offices should ideally be integral partners in data collection and use

8 Kuunika has invested a very great deal of time, effort and resources into District level capacity development, training on and access to DHIS2 and digital data hardware and systems (e.g. dashboards, the mobile App, Cluster meetings). This was most apparent before 2019, at which point changes in consortium partners and project management, coupled with the sustainability pivot, led to Kuunika being seen as becoming more distant from the Districts (except in Zomba) More recent support to horizontal engagement, e.g. through helping to facilitate setting up Cluster meetings, are often not recognized as having Kuunika inputs, but are seen by those involved as useful channels for debate and decision-making processes.

9. The Covid-19 pivot, centrally facilitated by the MoH with Kuunika and other partner support, has enabled very considerable progress to be made by Malawi in terms of tracking the pandemic and developing a response. At District level there are mixed views as to how much such data collection has supported service delivery.

10. Despite being defined in terms of being a District-focus programme in the early days, Kuunika has given relatively limited attention to planning effectively for how it might most coherently and comprehensively support decentralized structures and systems and provide optimal inputs at District level, for access to and use of digital data, through the DHIS2 platform. The temporary involvement of the Districts in Kuunika planning after the first pivot appears to have been short-term and never properly integrated in project planning and processes

11. Kuunika might be most usefully regarded as sitting in the ‘functions and capabilities’ space, its contribution to decentralization of health services lying in its ability to empower Districts via access to better data to plan, manage and deliver services

12. The decision to focus on HIV as the data use case, a vertical system owned at the central level, (and where key datasets collected and managed by the DNHA were not uploaded to DHIS2) meant there was little space for the District level. Initially the Kuunika consortium was almost a parallel MoH, without any attention to decentralized health and data. Kuunika moved also; at one time it was in the HIV Department.

13 Kuunika was somewhat designed in a vacuum, without thinking of communication channels between the project and Districts. The initial big focus in Kuunika was data - not which entity/individuals had access, ownership, etc. Later focus was more on patient outcomes and use of data to support service delivery. That necessitates proper District buy-in if improvements in service delivery are to be achieved - and that step has not been properly taken by the project

14. At the end of Kuunika phase 1, there is little concrete evidence of sustained, systemic improvements in data use knowledge culture at District level, or of key planning and service delivery documents such as DIP being progressively informed by quality data derived from DHIS2. There continue to be considerable problems of access to DHIS2, of capacity to navigate its programs, of identifying, analysing and using quality data.

15 The existence of data 'super users' demonstrate the possibilities for building a knowledge culture based on interest twinned with capacity. The question is how to maximize such potential without thereby placing unrealistic burdens on individuals.

16 The Blantyre Prevention Strategy is said to be designed based on lessons learned regarding Kuunika District gaps. This new program has the DHO as the lead, from the start. 'We re-thought the process, based on Kuunika challenges with decentralization.'

7. Recommendations for Kuunika phase 2

Recommendation 1: engage throughout with Districts - leadership, ownership and governance

Rationale: the ultimate goal of any data system should be to deliver optimal health services that lead to improved patient outcomes. That necessitates proper District buy-in, which was not embedded into Kuunika from the outset.

Recommendation 2: work for greater aid effectiveness - alignment with other partners working on digital health data

Rationale: information from national level respondents is that a 'Sector-wide Approach (SWAp) Revival' Concept Note is in development. There is increasing emphasis on maturing away from a proliferation of pilot projects and toward proven and scaled solutions built on common standards within an architecture.

The consensus appears to be that any such action would seek to re-introduce SWAp principles to alignment, joint planning, working and Monitoring, Evaluation and Learning (MEL), but not (at least initially) any financial disbursements to government entities. Investments and implementation for digital health data require far more harmonization. Application of a number of SWAp principles could potentially link disbursements to District performance, monitored using digital data. This would, however, necessitate genuinely effective support and training to all those engaged with collecting and using data to plan and deliver services.

Kuunika (and the Bill and Melinda Gates Foundation) should continue work with the MoH to support greater alignment and harmonization of all partners' digital health data interventions. If there is a SWAp-light framework developed for digital health data, Kuunika should consider a role as an integral partner, calling upon cumulative project experience, expertise and its position at the center of national developments.

Recommendation 3:

continue to support the MoH Digital Health Division

Rationale: the Digital Data Division should continue to receive Kuunika support; it is the site of government technical capacity. In its inception phase Kuunika 2 should plan for greater DDH coordination with and support to District digital health data systems.

Recommendation 4: ensure digital data systems are designed with users in mind and work to maximise all levels having access to participation, training and ongoing support

Rationale: systems should not be designed without properly thinking of who will use them and how, and the capacity development each individual cadre will need. With hindsight, the adoption of HIV as a data use case for Kuunika may have limited scope and flexibility - collection of HIV data was and remains tightly managed at the national level. Districts and Health Facilities did not have oversight of such data, or effective ownership.

Recommendation 5:

Build In Sustainability From The Outset

Rationale: while there is increasing resistance to a 'proliferation of pilots' in the digital health data sphere, Kuunika 2 has options to build on the foundations not only of the project, but to engage closely with the Blantyre Prevention Strategy, which is focused on building Districts systems, capacity and ownership. Such relationships, allied to any development of greater partner alignment and harmonization, could enable not only economies of scale but evidence-based prioritization of interventions proven to be effective.

Recommendation 6: have more focus on equity aspects of digital data systems

Rationale: data are never neutral. Just one point is disaggregation of data - their collection, their identification, their analysis and their use. WHO, UN Women, many civil society organizations and others continue to press for greater equity of data disaggregation, and greater application of such principles in the context of digital data.