5 Study methodology

This chapter describes the methodology used for this Special Study. In this chapter, we describe our data collection methods, analytical framework and efforts made to mitigate methodological limitations.

5.1 Study design

As indicated in previous sections, this study aims to surface the governance, regulatory and policy challenges / barriers to EMR-based data integration and sharing in Malawi and generate recommendations for key stakeholders by focusing on four specific questions relating to: factors preventing delivery of the Demographic Data Exchange; implementation and aid effectiveness issues; the wider regulatory and governance context; and lessons learnt.

These retrospective and contextual enquiries required a qualitative study based on triangulation of the following methods: ● Document review to establish Kuunika project history over the course of design and implementation. ● Wider literature review to create a set of short comparative case studies from secondary sources to establish the current status of international legal and advisory conventions and standards for creating and sharing digital personal health data (especially in LMICs). ● Key informant interviews to (i) supplement the project document review and (ii) elicit lessons and recommendations.

5.2 Study participants and sampling method

Participants in the study were drawn from national and international health sector actors and commentators. These included: Kuunika Project staff; MoH and other GoM officials; and International Development Partners. Selection of study participants began with purposive sampling amongst the categories. Some convenience (i.e. dependent on availability and accessibility of the interviewee) and ‘snowballing’ sampling was also employed. The target sample size was 40 key informants. In practice, there was very slow take up of invitations over the timeframes of the study, despite repeated follow-ups. Consequently, it was only possible to conduct eight interviews (Annex 6). However, these interviews proved rich and informative and triangulated well with the findings from the extensive desk review and a group discussion with the Kuunika team.

5.3 Primary data collection instruments

Key informant interviews were conducted using a semi-structured topic guide developed following the project document and literature reviews. Interviews were not recorded to encourage full and frank responses. Due to COVID-19 travel restrictions, interviews were conducted remotely using Teams or Zoom platforms.

5.4 Secondary data sources

The review of project documents and the literature base was conducted using a standardised rapid literature review template developed for the purpose. Secondary data was sourced from: ● Kuunika project files, supplied by Project and BMGF Staff ● Mott MacDonald Kuunika evaluation document archive ● Internet searches and KII recommendations of publicly available material ● Grey literature provided by KII participants

5.5 Data management arrangements

Interview notes were recorded by the interviewer using a coded notes template. The notes were then transferred to an Excel spreadsheet matrix organised around key study themes. This allowed a simple thematic analysis, as well as a searchable record during writing up. Original notes were archived so they could be revisited if more detailed records were needed. All respondents were assured of confidentiality. In keeping with the study consent form, no respondent was cited or quoted without prior permission.

Interview records were stored on a Mott MacDonald SharePoint site, and were managed in line with Mott MacDonald/ Microsoft Teams data privacy and protection protocols. The Mott MacDonald SharePoint site was only accessible to named project management staff and permitted viewers. In keeping with Mott MacDonald procedures, a Data Protection Impact Assessment72will be completed before contract closure to ensure compliance with Mott MacDonald’s Privacy and Data Protection Group Policy Statement73 and the Group’s General Data Protection Regulation (GDPR) Framework guidance. 74

5.6 Data Analysis Plan

Analysis of documentary and interview data was continuous and iterative. Based on the study questions, review of the literature base and standard approaches to digital health governance, we developed a thematically-based analytical framework using Microsoft Excel worksheets and Word template. The interview guide and notes template for the KIIs was designed and coded on the basis of both.

As interviews are conducted and new evidence became available, the thematic framework was filled out with active triangulation and evidence comparisons. Additional columns were added for team reflections, and lessons / recommendations elicited.

The overarching analysis plan for this study is shown in Table 3 below.

Table 3: Overarching analysis plan for Special Study 2

Study Phase Analysis objectives Data sources Analysis outputs

1. Formative /

inception phase

2. Data collection

phase

- Rapid review of all secondary literature by document type and theme - Mapping of the project design, timeline and transitions showing implementation story - Mapping of relevant stakeholders, relevant policies, regulations, agreements etc to be considered - Rapid review of catalogued secondary literature - 2 expert interviews - Excel book: annotated bibliography - Annotated timeline - Stakeholder mapping - Tabulations of the policy, regulatory and institutional context - Coded data collection toolkit - Excel templates for capture of

KII and secondary data by theme (coded)

- Continuous iterative analysis to adjust data collection tools and analysis templates as lines of enquiry emerged - Document and literature review - KII - Populated data analysis templates - Archive of KII topic guides - Documented literature on global digital health standards and case studies

72 2020-01-21 Mott MacDonald DPIA Template v3.0.docx (sharepoint.com) 73 Privacy and Data Protection Group Policy Statement 74 General Data Protection Regulation (GDPR) (sharepoint.com)

3. Main data

analysis phase

4. Feedback/

verification phase

- Review of completed data analysis templates (Excel matrices aligning analysis theme to data source) - For each study question, evidence synthesis and triangulation, identification of constraints and success factors and benchmarking against global standards - Completed data analysis templates from document / literature review and KII - Excel findings sheet and

Word tables based on evidence synthesis from data analysis templates) - Factor analysis and benchmarking against global standards - Draft study report

- Preliminary findings and recommendations reviewed by key stakeholders to confirm factual accuracy, relevance and interpretation of study findings - Draft study report - Revisions to produce final study report

5.7 Study limitations & mitigating actions

Table 4 below summarises the key limitations to the study methodology and the mitigating actions taken.

Table 4: Study limitations, potential for impact and mitigating action Limitation Impact Mitigating action

Some key informants were unavailable Incomplete findings; small sample size limited information power

Events explored in interviews proved sensitive Interviewees limit or moderate their responses • All participants were asked to propose additional interviewees. • Schedule extended to include more interviews. • Interviews were complemented by reviews of available project literature and evaluation survey evidence. • Provision made for a group interview with the Kuunika team • Desk review extended and evidence triangulation intensified to confirm and test validity of findings • Interviews were handled sensitively, with issues depersonalised. • No recordings were made and respondents were assured of confidentiality / anonymity.

5.8 Study-specific ethics considerations

A standard consent statement was prepared and shared with all key informants. The consent statement explained the background and purpose of the study and how findings would be used. All key informants were: a) assured of confidentiality and anonymity (unless formally agreed otherwise); b) informed they were under no obligation to answer any question that made them feel uncomfortable; and c) informed they were free to terminate the interview at any time.

No respondents under 18 years old or in any way infirm were interviewed as key informants. As far as possible, interviews took place in workplace or populated environments for safeguarding purposes.

As indicated above, all interview notes and evidence documents were securely stored in line with GDPR requirements and Mott MacDonald’s data privacy and protection protocols.

The study design and methodology was subject to ethical approval by GoM authorities.