Engaged Research For Health: What, Why, and How

Engaged Research For Health:

What, Why, and How

Glossary

Community

A group of people with a shared identity, interest, practice, or location.

Community-Based Participatory Research (CBPR)

A collaborative approach to research where community members are equal partners in the research process.

Co-design

A collaborative approach to design that involves people with relevant experience and skills as equal partners in the process of creating a product or service.

Co-production

An approach to research with partners where power and decision-making is shared at every stage of the research process.

Engaged Research

Research that uses a range of approaches to embed the voices and perspectives of partners in decision-making throughout the research process.

Experience-Based Design

An approach that centres user experience in the design of new services or interventions.

Human-Centred Design

An approach to design that places people with relevant lived experience at the centre of problem-solving.

Integrated Knowledge Translation (IKT)

A collaborative research model where researchers work with knowledge users to identify problems and implement research recommendations.

Patient and Public Involvement (PPI)

An active partnership between researchers and members of the public, including patients, carers, and people who use health and social care services.

Research Partnership

A collaborative approach to research between organisations and/or individuals who have a stake in the intended research. Partners are involved at all stages of the research process and decision-making power is shared.

Stakeholder Engagement

An approach to working with people affected by research to incorporate their views, experiences and decisions into the process.

About this guide

This guide to engaged research for health was commissioned by the Wellcome Research Environment team and produced by Equitable Health Futures, led by Anna Ruddock and supported by advisory partners. The guide is informed by a rapid scoping review conducted between July-September 2024. You can learn more about the review methodology here.

The guide is primarily intended for Wellcome internal use and has a two-fold purpose. First, a narrative overview explores the “what” and the “why” of engaged research for health, highlighting key themes and common challenges in this evolving field. Second, the guide addresses the “how” of engaged research by linking to practical tools organised according to research stage and theme.

In this guide, we touch on a wide variety of different approaches to engagement. In describing the field in general, we use Wellcome’s terminology “engaged research” as an umbrella term that covers a range of approaches to working with different people throughout the research process. In this guide we refer to four broad stages of this process: 1. Conception, Planning, and Design; 2. Data Collection and Analysis; 3. Dissemination and Impact; 4. Evaluation and Sustainability.

Research engagement and participatory research methods share the goal of involving more people in the research process. Given the extensive guidance available about using participatory methods specifically for data collection and analysis, this guide mainly focuses on literature, tools and approaches for engaging partners at all stages of the research process.

Engaged Research

involves people at every stage

What and Why: A Landscape Review of Engaged Research for Health

The What

Engaged research is an expanding area of research practice. The volume of publications about different forms of engagement, and the number of tools and frameworks designed to help researchers engage effectively is also growing accordingly. This overview is informed by a rapid scoping review that initially yielded 2,771 results, with a search confined to open access sources and a focus on health research.1 Of the final 213 publications from which we extracted data, 44 were reviews of the literature, including systematic reviews (1-9), rapid or scoping reviews (10-27), and realist reviews (28-30). Six were umbrella reviews, or “reviews of reviews” (31-36). The emergence of umbrella reviews is a clear sign that the volume of publications on a particular theme has reached a point where there is value in collating and synthesising current knowledge.

While terminology and emphasis differ, meaningful engagement is broadly defined by the shift from a traditional paradigm of doing research “to”, “for”, or “about” subjects, to doing research “with” a wide range of people whom it affects in one way or another.2 Engagement can happen in different ways, to different degrees, and at different stages of research. Increasingly, researchers are encouraged to move from involving different stakeholders to actively partnering with them throughout the research process (9,13,31,32,37,38).

In our review, the lack of consistent terminology for describing engaged research was a common finding. There is a growing agreement that while there must be room for different approaches to engagement, the sheer range of descriptions and definitions is unhelpful for developing best practice guidance. This is particularly true for developing common evaluation criteria that would improve the reporting of engaged research and enhance understanding of its impact in different contexts (4,9,11,14,19,20,39).

The language used to describe engaged research varies widely among funders, national research bodies, international networks, journals, NGOs and community groups, and consequently among researchers. Terms include patient and public involvement (PPI), community-based participatory research (CBPR), community-engaged research (CER), coproduction, co-design, integrated knowledge translation (IKT), and stakeholder engagement (see glossary). The variation is so great that it has spawned its own review, which identified

1 Learn more about our methodology here

2 https://www.hra.nhs.uk/planning-and-improving-research/best-practice/public-involvement/

sixty different ways of describing engaged research (6).

These approaches are not necessarily interchangeable. They arise from different contexts with different histories. For example, CBPR originated in the US as a vehicle of social justice and is often associated with efforts to decolonise knowledge production in the Global South through partnership and power-sharing (11). PPI is generally associated with the UK and its promotion by the National Institute for Health and Care Research (NIHR), while IKT is an approach developed by Canadian research funders (40).

Co-production is an increasingly popular approach across contexts and arguably the most challenging given its commitment to power-sharing throughout the research process (41–43). It is also subject to a range of interpretations, which can be misleading. Reflecting on these multiple interpretations, Williams et al. coined the term “cobiquity”. This refers to “an apparent appetite for participatory research practice and increased emphasis on partnership working, in combination with the related emergence of a plethora of ‘co’ words, promoting a conflation of meanings and practices from different collaborative traditions” (43).

The Why

Funders have placed increasing emphasis on the importance of engaged research in recent years.3 In many cases researchers are now required to demonstrate in grant applications how they will engage non-academic stakeholders as part of their proposed project. This new emphasis arose from a recognition that too much health research, particularly that with an applied intention, was conducted without the involvement of the people it was intended to affect. Often, this was a reflection of powerful social hierarchies that were reproduced within academic institutions, particularly the medical and biological sciences (1). People who have historically been oppressed and marginalised in society have been excluded from research that affects them, which reinforces marginalisation. This includes women (44), people of colour and racially minoritised groups (45,46), people who are LGBTQ+ (1), people with disabilities (16,47,48), people who are neurodivergent (49), children and young people (35), people with lower levels of education (50), formerly colonised peoples (25,30), migrants (51), or people who combine these characteristics (26).

The failure to prioritise engagement with groups at high risk during COVID-19 research and vaccine development is a very recent example that has reinvigorated the push to institutionalise engagement as a non-negotiable component of health research (45,52,53). In the United States, 82% and 80% of participants in the Pfizer–BioNTech and Moderna SARSCoV-2 vaccine trials identified as White. Black (9.8%, 9.7%), Indigenous (0.6%, 0.8%), and Asian (4.4%, 4.7%) people accounted for less than a quarter of participants in both trials (45). As Adkins-Jackson et al. state:

3 https://www.ukri.org/news/shared-commitment-to-improve-public-involvement-in-research/

“The disconnect between a lack of diversity in vaccine trials and the racialized and minoritized populations the vaccine was to serve occurred despite robust literature from over a decade highlighting evidence-based practices for inclusive research participation. This disconnect demonstrated yet another instance of the pandemic revealing what was already there: the persistence of structural barriers in clinical research and in the knowledge production of science.” (45)

In a 2022 review, Harting et al. synthesised the literature to produce five categories of rationale for doing engaged research, which they illustrate through a case study of patient engagement in Lyme disease research (54). The democratic rationale holds that people have the right to be involved in research that affects them as more than “subjects” (35). The transformative rationale says that research has liberatory potential. It can be mobilised as part of wider efforts to dismantle racism, ableism, and other systems of oppression upheld, in part, through the exclusion of marginalised groups from knowledge production as illustrated by the COVID-19 vaccine trials (55–57). The substantive rationale aims to improve the quality of knowledge that research generates. The instrumental rationale reflects a drive to improve the efficiency and efficacy of the research. Finally, the consumerist rationale reflects the economic right of people to be engaged in research that impacts their interests (54).

Often a combination of these rationales motivates engaged research, with an overarching aim of remedying inequalities and improving health equity (32). Being clear about the rationale/s for conducting engaged research is helpful for planning and resourcing a project. It also helps when considering ethical challenges around power-sharing and equality of opportunity for engagement, which we discuss below.

Questioning engagement

There are strong arguments in favour of doing more, and more inclusive, engaged research. We have very little evidence, however, of how different approaches to engagement affect research quality and impact. Engagement, particularly efforts to conduct genuine co-production, also entails risks and challenges for everyone involved. In a 2019 paper, Oliver et al. draw attention to the risks for researchers and the lack of consensus around the most effective engagement strategies. They outline practical, personal, and professional costs, as well as potential costs to the quality of research and the reputation of the research profession (42). The authors have

been criticised for an overly technocratic interpretation of “co-production” (43). Ultimately, they ask that researchers be transparent about their motivations and carefully consider whether the costs of co-production outweigh the potential benefits for each project. Others challenge assumptions that engagement is a necessary component of all research projects, irrespective of topic, and argue against penalising researchers who present valid arguments for not incorporating community engagement into their work (58).

Engagement in different fields

Conducting research that is more engaged with the people it will affect means going beyond recruiting a more diverse range of research participants. It means establishing sustainable partnerships to involve more people at every stage of the process from priority-setting through to the uptake and use of findings.

The practicalities of this shift vary across different fields of health research. For social scientists, a more meaningful emphasis on engagement means a quite literal move from studies of people to studies with people whose experiences they are interested in. Particularly when studies focus on the experiences of minoritised and/or oppressed groups, integrating people as research partners can bring benefits to all concerned.

Our review found that common benefits of research partnerships included:

• More relevant research agendas.

• Research design that is better tailored to the population being studied.

• Improved knowledge and skills among both researchers and partners.

• A greater sense of ownership and investment in the research among partners and their communities.

• Improved likelihood of the research findings having a positive impact (12,13,27,33,35,59,60).

These new ways of working also entail significant challenges, as we explore below.

It isn’t surprising, given the nature of the work, that so much of the literature about research engagement in health comes from the social sciences. As with the field of evidence in decisionmaking, which studies the use of evidence by policy makers and practitioners, the process of engaged research is becoming its own sub-field of the social sciences. This is exemplified by the growth in popularity of realist reviewing, which seeks to understand the mechanisms through which engagement “works” (28–30).

Fields such as fundamental biology, and clinical and translational research, traditionally entail less engagement than the social sciences beyond the participation stage of studies. The journey to full engagement, or research partnership, is a longer one in these areas – as are research processes themselves. This makes sustaining engagement over time more challenging (61). Change is well underway, however. A growing number of research leaders are calling for greater engagement, and more clinical and pre-clinical research teams have begun sharing their experiences of engagement to showcase learning and good practice (52,60–65).

Challenges and Recommendations

Making engagement meaningful

Engaged research may be increasing, but that doesn’t mean it is done consistently, or done well. In a 2022 study, Lang et al. analysed 3000 papers published in the general health journal BMJ Open (which requires a statement on whether or not a study included PPI) to see how many included PPI and how PPI was associated with topics, methods, and funding sources. They found that only 20% of the analysed papers included PPI. Papers from the UK had the highest rate of PPI at 44%. Mental health was the topic with the highest rate of PPI at 36%. Studies funded by NIHR had the highest rate of PPI at 57% while 28% of studies funded by Wellcome included PPI (66).

Engagement efforts are regularly criticised for being tokenistic and superficial, referred to in one paper as “pantomime community engagement” (66–68). There are different reasons for this. Embedding engagement in some types of health research is more challenging than in others. In medical research, for example, studies may take several years, making sustained engagement more logistically challenging. Engagement work in fundamental biology, to take another example, is more resource intensive when it requires a training element to ensure partners understand the area and can participate effectively, and to ensure that researchers can communicate their work effectively to a range of partners.

Across all fields, there are cases in which researchers are expected to conduct engaged research, but they are not sufficiently supported or resourced to do so (69–71). This lack of support immediately compromises the depth and authenticity of the engagement (72). In other cases, there may be support for engagement at certain stages, such as agenda setting or data collection and analysis, but little for sustaining partnerships at the writing and dissemination stages (14,26,39,73).

There are contextual challenges too. In formerly colonised countries that remain resource constrained, both historical and contemporary practices of data extraction by dominant

institutions in the Global North create legacies of mistrust and scepticism about the motives of researchers (74,75). In the United States, the historical abuse and exploitation of Black communities by scientific institutions continues to negatively impact trust and participation in research (76). In the UK, the extent of distrust of research institutions among minoritised ethnic and religious groups became clear during the COVID-19 pandemic (46). Addressing the power relations that have excluded marginalised people from research is a priority in the growing field of research engagement, as we discuss below (also see Spotlights for resources).

Inclusive engagement

Partnership approaches and co-production aim to acknowledge and address longstanding power imbalances by integrating non-academic partners on a more equal footing with researchers. This means working with partners at every stage of the research cycle as part of a team and ideally establishing long-term working relationships. It also means valuing partners’ knowledge and lived experience as much as the academic knowledge that traditionally carries greater prestige. Challenging such entrenched hierarchies and institutional paradigms is no easy task and some people question whether truly transformative power-sharing is possible (41). (See Relationships: Power and Trust).

Nor does a partnership approach guarantee equitable engagement that reflects the diversity of people impacted by research. Engagement initiatives that do not include people from marginalised groups affected by research risk undermining common goals of engagement. First, they don’t reflect the democratic rationale for engagement which holds that people affected by research have a right to be involved in its production. Second, there is a risk that health inequalities may be exacerbated by research that is dominated by the perspectives of people who occupy comparatively privileged social positions (77,78).

In the UK, there is a recognition that White, well-educated, financially comfortable people over the age of 60 have been over-represented in PPI initiatives (77). A 2019 review of NIHR’s public involvement and engagement strategy showed that young people and people from minoritised ethnic groups were particularly under-represented. Only 2% of people were under 25, and 14% were 26-49. People from Asian and Black ethnic groups comprised 3% and 2% of PPI participants, respectively.4 Researchers have also shown how people with disabilities (47), people who are neurodivergent (49), people with less formal education (50), people with mental health conditions (17), people with dementia and/or life-limiting illness (79), and people for whom these different characteristics intersect (80), do not have equal access to opportunities for research engagement.

Conducting meaningful and inclusive research engagement is challenging. The process illuminates oppressive structures and systems that impede the full participation of marginalised people in different areas of life, from education to employment to healthcare to social safety. In their work on the exclusion of racialised groups in clinical trials, Rai et al. argue that the goal should not simply be to enable the participation of more diverse groups in current research practice, but to directly address the structural barriers that have excluded them from research until now (81).

4 https://www.nihr.ac.uk/taking-stock-nihr-public-involvement-and-engagement

Barriers to participation in research engagement include:

• Low literacy.

• Inexperience and low confidence in meetings or formal settings.

• Lack of money for hidden costs such as appropriate clothing.

• Not being part of groups through which opportunities are circulated.

• Not speaking the working language confidently.

• Lack of support for access needs.

• Lack of digital access.

• Lack of time.

• Uncertainty about the benefits of participation.

The lack of diversity within the academic workforce itself means that for many people designing engagement initiatives with the best intentions, these barriers are “hidden” and may not be considered at the planning stage.

However, the need to broaden access to engagement is increasingly recognised. Research teams working in collaboration with community partners have generated guidance for inclusive engagement (see Spotlights). Examples include the CHICO guidance for community involvement in health research (78), and the international consensus statement for engaging people with intellectual disabilities in health research (47). Reflecting a commitment to coproduction, some of these papers are co-authored by partners who share their experiences of being part of a research team (82). Funders are also developing guidance, such as NIHR’s Race Equality Framework and agenda for promoting inclusion in research partnerships (83,84).

Power and positionality

The lack of diversity within academia itself, and the barriers to academic careers for minoritised and marginalised people, is an under-recognised barrier to more inclusive engaged research and one that is rarely addressed. We found very few papers that made this connection (85,86).

The gulf in experience between many academics and the people they seek to engage with is highlighted by the shift from describing some groups as “hard to reach” (which is only the case if you have no connection with them), to “seldom heard” (87,88).

In the literature about partnerships in research on mental health and neurodivergence, we found examples of team members occupying dual positions as academic researchers and people with lived experience of the research topic (48,82). Evans et al. have written about coproducing work as a team of academic and non-academic researchers all living with energy limiting conditions (89). We also found one example of an “insider-only” participatory research council in which everyone, academics and community representatives, was autistic (49). On the whole, however, much of the literature and the guidance around engagement still suggests an “us and them” dynamic. In this dynamic, the partner is the “other” who is “engaged” by the academic researcher and power-holder (90). Even when the approach is highly sensitive to the uneven distribution of power, this dynamic poses the greatest challenge to achieving authentic co-production given the dominant social and financial power of the team members from the academic institution. This inequality is magnified in global health research funded by institutions in the Global North and conducted with people in the Global South (30,74,91) (see Relationships: Power and Trust).

Resources: Time, Money, Incentives

Meaningful and inclusive engagement throughout the research cycle requires intensive effort. Many researchers report that they underestimated the additional costs, time, skills and energy required, or that the resources required were not available to them. As Soklaridis et al. put it, reflecting on their co-production work in mental health research: “Although the research community may purport to espouse patient-centredness and the inclusion of [people with lived experience], the reality is that we operate in an environment with limited structural support to uphold these values and in which we are expected to get the work done as quickly as possible” (82).

Heney and Polykett note how the increasing reliance of universities on short-term, casualised academic labour is a structural impediment to the long-term relationship building that genuine engagement requires (72). Researchers also report a lack of incentives within academic employment and promotion structures for engagement work that is time- and energyintensive. Those who prioritise and invest in engagement have described a trade-off between this commitment and their career progression (92).

Recommendations

Meaningful and inclusive engagement in research requires coordinated efforts among funders and universities. These efforts should focus on four main areas.

1. Provision of greater expert support and guidance for developing comprehensive engagement plans. These plans must consider the direct and indirect costs, and realistic timelines for engagement.

2. Ring-fenced engagement budgets that cover all the direct and indirect costs involved.

3. Grant contracts that allow for the temporal flexibility required to establish and sustain longterm engagement relationships and to overcome unforeseen but inevitable challenges and hold-ups.

4. Academic performance evaluation and promotion structures that reflect the demands of meaningful engaged research and value it accordingly.

Resources: Skills

Effective research engagement with different partners and stakeholders at different stages of the research cycle requires a broads set of skills. Communication methods, facilitation skills, conflict resolution, self-reflection, and relationship management techniques for working with a wide range of people, including those who are marginalised, are all essential. Few researchers are trained in these skills. It is also essential that researchers have carefully considered the social context and potential impact of their work. Assuming that people should be able to conduct meaningful engagement without training underestimates the complexity involved and risks undermining professed commitment to the value of engagement. Similarly, for public and patient research partners, some form of induction and/or training is necessary to ensure they can contribute effectively, feel comfortable with the material and context, and make the most of the engagement opportunity (61).

Recommendations

1. Core competencies and skills for engagement should be agreed by funders and academic institutions.

2. Support should be available for developing these competencies and engagement skills should be part of ongoing professional development.

3. More engagement posts should be established within research institutions to embed engagement expertise and commitment within research cultures. These positions should be permanent and sustainable.

4. Engagement professionals should be included in senior decision-making groups, including strategy groups, scientific advisory boards, and funding committees, to advocate for engagement and embed it as part of institutional strategy.

Relationships: Power and Trust

Authentic partnership means power-sharing, which requires trust between all involved (15,74,87). Managing power dynamics between professional researchers and non-academic stakeholders arises in the literature as one of the most complex challenges of engaged research.

Even in scenarios where a team is assembled at the earliest stage of a research project to generate ideas and agree research priorities, there is a power difference between those who are employed by the academic institution that will hold the research grant and those who are not. The literature is rich with diagnoses of these power imbalances and the challenges they present for meaningful engagement, particularly co-production. We also found examples of research teams sharing the ways they navigated power imbalances and what they learned in the process. For these teams, acknowledging uneven power relations at the outset was crucial. Reflecting on their work towards co-production in rehabilitation research, Bourke et al. developed a set of three principles informed by challenges they encountered, accompanied by strategies to manage these: 1. Navigating power dynamics; 2. Building relational resilience; 3. Adopting a learning mindset (85).

Soklaridis and colleagues developed a set of values to guide their efforts to co-produce research in the area of mental health: 1. Navigating power relations together, with an emphasis on reflective practice; 2. Multi-directional learning, with informal as well as formal approaches; 3. Slow and steady wins the race, with an emphasis on health and wellness; 4. Connecting through vulnerability. They reflect that: “Co-production, through its emphasis on challenging traditional power dynamics and centring lived experience, asks something unique of the people involved. It asks, and arguably requires, the willingness to incorporate the concept of vulnerability as part of creating a psychologically safe collaborative environment. To centre lived experience and foster genuine collaboration, we found it essential to put our ‘professional hats’ aside and connect as people first” (82).

In global health research which sees academics employed by Global North institutions conducting engagement with partners in resource-constrained contexts in the Global South, power differences are often magnified. Vincent et al. conducted a realist review of if and how community engagement “works” in a malaria research programme in Kenya. They emphasised the importance of establishing and managing “working relationships across differences” of wealth, power, and culture between researchers and the community partners they seek to engage (30). These relationships are undermined by power expressed through what the authors describe as the “dominant health research paradigm context”, which includes: “research having historical links with colonialism, coupled with recent history of more or less vertically imposed health interventions; research being externally funded, designed and controlled; contemporary differences of wealth, power and culture between research centres, and surrounding settings of poverty and under-resourced health systems” (30). The review found that in many cases stakeholders accommodate these tensions partly as a necessary cost of access to vital healthcare that community engagement offers.

Another feature of power and relationships in engagement is the emotional labour involved, which is beginning to receive more attention in the literature. A recent review by Liabo et al. shows that there is often an emotional cost of engagement for both researchers and partners. For researchers, power dynamics and conflicts may be particularly difficult to manage. While for partners there is often an emotional cost when engagement is premised on the repeated sharing of painful experiences (18). Groot et al. discuss “ethics of care” as a theoretical approach that can help researchers manage their responsibilities towards partners. They emphasise the importance of self-care and existential safety for the ethical practice of engaged research (93).

Recommendations

1. Competencies, principles and values for managing power-sharing outlined in the literature should be synthesised and agreed by research institutions and funders.

2. Power analyses, and strategies to manage challenging dynamics, should be included in engagement planning.

3. Researchers and partners should be offered tailored support and learning opportunities to help them navigate power relations in engagement.

4. Funders and academic institutions should reflect on the power inequities that are illuminated by engagement work. They should explore if and how these might be mitigated by structural changes to grant conditions and support for engagement strategies (30,82).

Ethics

Engaged research involves different approaches to working with a wide range of people. Many of these people will be members of minoritised groups, and/or marginalised due to circumstance and experience. Funders increasingly expect thorough engagement plans but ethical review boards are not always equipped to manage proposals that demand flexibility in methodologies, timeframes, and degrees of engagement with people who are not traditional “research subjects” (94). Engagement plans may also be challenged by review boards that follow ethical guidelines to protect individuals who can’t provide traditional written consent for participation, or whose wellbeing is considered at risk (79). Walsh et al. explore this in their

paper about productive collaboration between researchers and ethics committees using a case study of their research about sexuality with young people with cerebral palsy who use communication methods other than speech (57).

Recommendations

1. Members of ethical review boards will benefit from training about the complexities of research engagement and the flexibility required.

2. There is scope for mutual learning between institutions, researchers, partners, and ethics reviewers to support engagement that is meaningful and inclusive of the fullest range of people (57,79,93).

Evaluation

The lack of consensus around the effectiveness and scope of evaluation methodologies for engaged research, together with the relative youth of this approach to research, means that long-term evaluations are rare (95,96). Crucially, we do not yet have multidimensional evaluations of the impact of different engagement strategies on the impact of research, including health outcomes over time (97).

Formative evaluation is an ongoing process of reflection and learning during a project. Often referred to as reflective practice, it is considered an important engagement principle, particularly in partnerships that aspire to the equal distribution of decision-making power (82,85). Formative evaluation methods in engagement vary, as do those intended to evaluate impact.

Those who are trying to promote more consistent evaluation of how engagement is conducted, and the impact it has on research practice regularly report the same obstacles (20,24,98,99):

• Inconsistent terminology used to describe engagement.

• Inconsistent reporting of how engagement was conducted.

• Lack of engagement evaluation embedded in the project design.

• Uncertainty about what should be evaluated and why.

Guidance does exist, but there is no consensus among funders or practitioners about what should be evaluated, why, and how.

Recommendations

1. Funders should expect evaluation planning as part of engagement plans and provide guidance where required.

2. Where engagement is a declared part of a project being written about, journals should require consistent reporting about what was done, and provide guidance around terminology to promote greater clarity.

3. All stakeholders should continue to work towards shared frameworks for evaluating the impact of engaged research for health.

How: Tools for Engaged Research

Principles

Many organisations have developed principles to inform engaged research partnerships. They are not the same, but they often overlap. Key shared themes include: Establishing and nurturing trust; open two-way communication; facilitating power-sharing; flexibility; ethical and inclusive practice; learning and accountability.

• NIHR Guiding Principles For Community Engagement and Involvement

• UKRI Key Principles of Co-production

• UK Standards for Public Involvement

• NCCPE/UKCPN Principles of Practice for Community-University Partnership Working

• Integrated Knowledge Translation (IKT) Guiding Principles

• Patient-Centred Outcomes Research Institute (PCORI) Foundational Expectations for Partnerships in Research

Engagement Throughout the Research Process

For this guide, we have segmented the research process into four stages: 1. Conception, Planning, and Design; 2. Data Collection and Analysis; 3. Dissemination and Impact; 4. Evaluation and Sustainability.

There is a growing amount of guidance about how to engage stakeholder or research partners throughout the whole research process. We begin with this, and in later sections you can find guidance specific to each stage in the process.

NIHR Learning for Involvement, the National Coordinating Centre for Public Engagement (NCCPE), Co-Productive Collective, and Mesh Community Engagement Network are rich repositories of engagement resources. In the sections below we have included key resources found during our search in September 2024. Each platform is constantly being updated and we recommend regular visits to keep up to date with what is available.

Quick Links Engagement Throughout the Research Process

Engaging Throughout the Research Process

Stage 1: Conception, Planning and Design

Spotlights

Engagement in Fundamental Research

Stage 2: Data Collection and Analysis

Stage 3: Dissemination and Impact

Stage 4: Evaluation and Sustainability

Engagement in Clinical and Translational Research

Engagement with Less Heard Groups Disability Inclusive Engaged Research

Engagement in Clinical Trials

Engagement with Racially Minoritised Groups

Engagement in Mental Health Research

Engagement with Children and Young People

Engagement for Policy and Practice

Engaging Throughout the Research Process

Guidance: Co-producing a research project

Guidance: Community engagement and involvement in global health research

Guidance: The Value of CoProduction

National Institute for Health and Care Research (NIHR)

NIHR

Co-Productive Collective

Guidance: Co-production in research UK Research and Innovation (UKRI)

Guidance: Equity in Public Engagement: A Guide for Practitioners

Guidance: Community-Based Participatory Research: A guide to ethical principles and practice

Podcast: Spotlight on community engagement and involvement

Toolkit: High Quality Engagement 101

Toolkit: The Principles of Partnership Working

Toolkit: A How-To Guide for Patient Engagement in Research

Toolkit: Experience-Based CoDesign Toolkit

Morris J. Wosk Centre for Dialogue, Simon Fraser University

Centre for Social Justice and Community Action, Durham University/NCCPE

Resource Source Description

A guide to how NIHR understands coproduction in research, including key principles, approaches, and challenges.

Principles and guidance for integrating meaningful community engagement and involvement in global health research.

A comprehensive resource designed to help you understand and communicate the value of co-production in your work.

A guide to how UKRI understands co-production in research, including key principles, and examples of good practice.

This guide proposes eight principles to support the meaningful and equitable inclusion of diverse voices in public engagement across sectors.

This guide outlines the central position of ethics in communitybased participatory research (CBPR), identifying the underlying ethical principles and offering guidance for putting them into practice.

NIHR

National Coordinating Centre for Public Engagement (NCCPE)

NCCPE

Canadian Institutes of Health Research (CIHR)

Point of Care Foundation

This podcast series from 2022 brings together guest speakers to discuss the importance of engagement and share tips for good practice.

A set of resources to support high quality public engagement, based on four principles: purpose, people, process, and evaluation.

Practical tools and content to help you develop your partnership approach and put it into practice.

A comprehensive set of resources to support patient engagement in research developed by the CIHR Institute of Musculoskeletal Health and Arthritis.

A step-by-step guide to improving patients’ experience of healthcare using experience-based co-design (EBCD).

Engaging Throughout the Research Process

Resource

Toolkit: Methods for HumanCentred Design

Toolkit: Patient Engagement Training and Resources

Framework: Spectrum of Public Participation

Framework: Power-sharing in CBPR

Framework: A design-led framework for engaged research: Using a design approach to understand and place the public at the core of health and social care

Framework: The Double Diamond Framework for Innovation

Course: Community Engagement and Involvement in Global Health Research

Course: Citizen Science and Scientific Crowdsourcing: an Introduction

Design Kit/Ideo.org

Source Description

A step-by-step guide to human-centred design, putting people at the centre of the design process to come up with new answers to difficult problems.

Patient Engagement Synapse

International Association of Public Participation (IAP2)

Engage for Equity

Galvin et al. (2021), Administration

Design Council

NIHR/Mesh Community Engagement Network

UCL

Course: Public Involvement in Research Coursera: Imperial College

A repository of resources for patient engagement in research, including training courses and capacity building materials, from a range of European organisations.

The IAP2 participation spectrum is a useful tool for understanding the different stages of engagement and what is involved.

This model promotes the liberatory potential of community-based participatory research (CBPR) and supports researchers to engage with partners as equals.

In this paper, the authors argue that a design-led approach can facilitate engaged research and support policymakers in the design of new policies and practices for health. They present a framework for facilitating dialogue among stakeholders.

Building on the Double Diamond approach to design, the Framework for Innovation includes key principles and methods that designers and nondesigners need to take, and the ideal working culture needed, to achieve significant and long-lasting positive change.

A free online course about community engagement and involvement in global health research.

This free online course offers an introduction to the theory and practice of citizen science and scientific crowdsourcing.

A free four-week online course looking at how to involve patients and the public throughout the research process.

Engaging Throughout the Research Process

Resource

Course: Applying Participatory Approaches in Public Health Settings

Course: Community Engagement: Collaborating for Change

Course: Stakeholder Engagement in Health Equity Research

Source Description

Coursera: Imperial College London

This free course includes learning how to critically analyse power dynamics between different stakeholders involved in programmes and research for public health.

edX: University of Michigan

Coursera: Johns Hopkins University

This free course is for anyone who wants to work more effectively with community members and organisations, including through research partnerships.

This free course looks at how to effectively engage community and organisational stakeholders in health equity research.

Engaging at Stage 1: Conception, Planning and Design

Guidance: Public coapplicants in research – guidance on roles and responsibilities

Guidance: PPI payment guidance

Guidance: A practical guide to planning an engagement strategy for your global health research funding application

Guidance: Priority-setting partnerships

Guidance: Building effective multi-stakeholder research teams

Toolkit: PPIE Planner

Toolkit: Community Engagement Studio

Toolkit: Tools for Co-Creation through Living Labs

NIHR

NIHR

Mesh

James Lind Alliance (NIHR)

Resource Source Description

A guide to including public coapplicants in research proposals and what is expected.

A guide to developing a payment policy and budget for patient and public involvement in research projects.

A guide from experienced practitioners to integrating a thoughtful engagement strategy into a research proposal.

A guide to how priority-setting partnerships work to identify and prioritise unanswered research questions that matter most to people.

Patient-centred Outcomes Research Institute (PCORI)

NIHR

Vanderbilt University

European Network of Living Labs

PCORI is the leading funder of patient-centered comparative clinical effectiveness research in the United States.

An interactive tool for planning patient and public involvement and engagement throughout your research.

The Community Engagement Studio is a structured forum for gathering research insights from patients or community partners.

A range of tools for co-creating solutions, covering need finding, ideation, strategy, experimentation, and feedback.

Engaging at Stage 2: Data Collection and Analysis

Toolkit: Involving People: A learning resource for systematic review authors

Toolkit: Supporting research partner involvement in data analysis and interpretation

Course: Peer Research Training Resource

Cochrane

PCORI

Imperial College

Resource Source Description

A resource to support systematic review editors and authors involving public, patients and healthcare professionals in the production of reviews.

A toolkit for involving research partners in data analysis and interpretation.

A free course for academics and engagement practitioners who want to train people with lived experience to become co-researchers in qualitative studies.

Engaging at Stage 3: Dissemination and Impact

Resource

Toolkit: European Science Engagement Platform

Source

The European Science Engagement Association

Description

The European Science Engagement Platform helps engagement practitioners find inspiration, resources, methods, and tools for participatory, dialogue-oriented science communication activities.

Toolkit: Research to Action

Reflection and learning: Identification and Reporting of Patient and Public Partner Authorship on Knowledge Syntheses

Course: People-Centred Policy Design

Course: Translating Research to Communities

Research to Action

Ellis et al. (2021), Journal of Participatory Medicine

Future Learn: Government Skills/Policy Lab

Coursera: University of Michigan

A range of tools for enhancing the positive impact of research, with a focus on international development.

This paper discusses the challenge of identifying and evaluating patient and public partner co-authorship of reviews, due to inconsistent terminology and reporting processes.

A free course focused on how to implement innovative methods that put people and their lived experience at the heart of public policy.

This free course includes examples of how to involve community members in sharing public health research findings.

Engaging at Stage 4: Evaluation and Sustainability

Guidance: Using dilemmas cafes to address ethical challenges in CBPR

Guidance: Evaluation of patient engagement in medicine development: A multi‐stakeholder framework with metrics

Toolkit: Public Involvement in Research Impact Toolkit

Centre for Social Justice and Community Action, Durham University

Vat et al. (2021), Health Expectations

Resource Source Description

This approach uses the World Café technique to discuss and resolve ethical dilemmas that may arise in communitybased participatory research (CBPR).

This paper describes a project designed to develop a monitoring and evaluation framework to demonstrate impact and enhance learning about patient engagement in medicine development.

Marie Curie Research Centre, University of Cardiff

Toolkit: Evaluation 101 NCCPE

Toolkit: GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research

Reflection and learning: Measuring what matters for advancing the science and practice of engagement

Staniszewska et al. (2017), BMJ

A set of tools to support researchers to plan and integrate public engagement in research; track public contributions and their impact on research; report impact against the UK Standards for Public Involvement.

An introduction to NCCPE’s four pillars of evaluation and how to apply them to your engagement work.

This paper shares the first international guidance for reporting of patient and public involvement in health and social care research, published in 2017.

This paper describes PCORI’s work towards harmonising approaches for measuring and evaluating patient engagement in research.

PCORI

Spotlights

In this section we spotlight engagement resources intended for use in specific areas of research, and/or with specific groups of people.

Spotlight on Engagement in Fundamental Research

Guidance: A practical guide to PPI in lab-based research

NIHR/Parkinson’s UK/ Alzheimer’s Society

Guidance: PPI – A Researcher’s Guide Versus Arthritis

Guidance: The need to set explicit goals for human germline gene editing public dialogues

Guidance: Adapting a conceptual framework to engage diverse stakeholders in genomic/precision medicine research

Guidance: Integrating patient and public involvement and engagement in translational medicine

Geuverink et al. (2024), Journal of Community Genetics

Watson et al. (2022), Health Expectations

Resource Source Description

Practical guidance about how to involve patients and the public in laboratory-based research.

Guidance about how to meaningfully involve people living with arthritis in research.

In this paper, the authors demonstrate the importance of setting clear goals in advance for engaging with people about human germline gene editing.

This paper offers a conceptual framework that aims to shape the evolution of genomic/precision medicine research by leveraging existing partnerships, engaging participants beyond recruitment, and embracing diverse perspectives.

Shaw et al. (2024), The Lancet In this paper, the authors argue that “Although patient and public involvement and engagement (PPIE) is increasingly being adopted in clinical and applied research, it is under-utilised and underreported in preclinical research where it is typically confined to the priority-setting stages of research and is often a one-time occurrence.”

Reflection and learning: The impact of patient and public involvement on COVID-19 immunology research: experiences from the UK Coronavirus Immunology Consortium

Aquino et al. (2023), Research Involvement and Engagement

This paper uses the example of the UKCIC to show that it is possible to conduct meaningful PPI in basic immunology research.

Spotlight on Engagement in Fundamental Research

Reflection and learning: Community Engagement in Precision Medicine Research: Organizational Practices and Their Impacts for Equity

Shim et al. (2023), AJOB Empirical Bioethics

Resource Source Description

In this paper, the authors show how “constraints of time and resources cascade into trade-offs” that can lead to “pantomime community engagement,” when engagement is experienced as “improvised and sporadic”.

Engagement in Clinical and Translational Research

Resource

Guidance: A short guide to successful patient involvement in EU-funded research

Source Description

Innovative Medicines Initiative/U-BIOPRED (Unbiased BIOmarkers in PREDiction of respiratory disease outcomes)

Guidance: PPI – Guidance for Researchers Parkinson’s UK

Guidance: PPI planning for blood and transplant research in precision cellular therapeutics

University of Birmingham

Guidance: Patient and student partnership University of Birmingham

Guidance: PPI – A Researcher’s Guide

Guidance: Enhancing the incorporation of the patient’s voice in medical product development

Versus Arthritis

US FDA

Tips from the development and use of a Patient Input Platform (PIP) in a respiratory disease research programme.

Guidance to support researchers to work in partnership with people affected by Parkinson’s.

A guide to developing a PPI plan designed for researchers working on blood and transplant research in precision cellular therapeutics.

This guide explains why it is important to involve patients in student research, and how to do it.

Guidance about how to meaningfully involve people living with arthritis in clinical research.

A set of guidance documents to facilitate systematic approaches to meaningful patient and caregiver engagement to better inform medical product development.

Spotlight on Engagement in Clinical Trials

Guidance: Embedding PPI in the culture of the MRC Clinical Trials Unit

Toolkit: Prioritisation tool for funders and patient groups

Reflection and learning: Inclusionary Trials: A Review of Lessons Not Learned

UCL/MRC

Clinical Trials Transformation Initiative, Duke University

Adkins-Jackson et al. (2022), Epidemiologic Reviews

Reflection and guidance about embedding high quality PPI into research cultures.

An interactive tool to help researchers and partners analyse, visualise, and act on opportunities for engagement.

In this paper, the authors show how the COVID-19 pandemic revealed the exclusion of marginalised groups from clinical research and suggest ways to address this.

Spotlight on Engagement in Mental Health Research

Resource

Guidance: Charter for Public Engagement and Involvement in Mental Health Research

Toolkit: The Influence and Participation Toolkit

Toolkit: Research Methods Toolkit

Reflection and learning: A balancing act: navigating the nuances of co-production in mental health research

Reflection and learning: Co-producing research on psychosis: a scoping review on barriers, facilitators and outcomes

NIHR/McPin

Foundation

Mind

Centre for Society and Mental Health (KCL)

Soklaridis et al. (2024), Research Involvement and Engagement

Jakobsson et al. (2024), International Journal of Mental Health Systems

This Charter sets out best practice for fully embedding public involvement and engagement into mental health research.

Templates and tools to support engagement with people living with mental health challenges.

A comprehensive set of tools for all aspects of research, including engagement.

In this paper, the authors reflect on their experience of co-producing research and share the key values they developed to help them navigate the process.

This review makes recommendations for encouraging co-production and improving the reporting of co-produced research with people living with psychosis or severe mental health conditions.

Spotlight on Engagement with Children and Young People

Resource

Guidance: Supporting children and young people to be young researchers

Toolkit: How to set-up and run a young persons advisory group

Reflection and learning: Positive Impacts of Adolescent Involvement in Health Research: An Umbrella Review

Reflection and learning: Laying the groundwork: Building relationships for public and patient involvement in pre-clinical paediatric research

Barnardo’s

Source Description

The Generation R Alliance/European Young Person’s Advisory Group Network

Warraitch et al. (2024), Journal of Adolescent Health

Costello and Dorris (2020), Health Expectations

Guidance and tools to use when supporting children and young people from a diverse range of backgrounds to be young researchers.

A toolkit to help you set up and run a young persons advisory group, allowing the voices of children and young people to be heard in research.

This review finds that the evidence supporting the positive impacts of adolescent involvement in research is substantial but limited by inconsistent reporting and a lack of rigorous evaluation.

This paper describes the creation of a successful partnership between pre-clinical researchers and young people living with arthritis in Ireland.

Spotlight on Engagement with Less Heard Groups

Resource

Guidance: Tips for researchers involving unpaid carers in health and care research

Guidance: Consolidated criteria for strengthening reporting of health research involving indigenous peoples: the CONSIDER statement

NIHR

Source Description

Huria et al. (2019), BMC Medical Research Methodology

Key factors for researchers to bear in mind when engaging with unpaid carers, and advice for carers getting involved in research.

In response to a legacy of exploitation in health research, this paper describes the development of consolidated criteria for strengthening the reporting of health research involving Indigenous peoples.

Spotlight on Engagement with Less Heard Groups

Reflection and learning: ‘Hard to reach’ or ‘easy to ignore’? Promoting equality in community engagement

What Works Scotland (2017)

Resource Source Description

This report examines evidence from Scotland and the UK on what is being done to overcome inequality in community engagement.

Reflection and learning: Participatory Health Research with Women from Refugee, Asylum-Seeker, and Migrant Backgrounds Living in HighIncome Countries: A Scoping Review

Reflection and learning: Equity and inclusivity in research: co-creation of a digital platform with representatives of marginalized populations to enhance the involvement in research of people with limited literacy skills

Vazquez Corona et al. (2024), International Journal of Qualitative Methods

Loignon et al. (2021), Research Involvement and Engagement

This review found that engagement with migrant women was more often done through community organisations or agents, rather than women themselves. The authors argue that better reporting is necessary to ensure that engagement with migrant women is equitable and inclusive.

This paper reports on the experience of co-creating a digital platform to address barriers to research involvement for people with limited literacy.

Spotlight on Disability Inclusive Engaged Research

Resource

Guidance: Co-producing accessible health information

Thinklusive/ Understanding Patient Data

The learning in this guide is from a project to co-produce accessible information about the uses of routinely collected patient health information in healthcare research and planning.

Spotlight on Disability Inclusive Engaged Research

Guidance: Making PPI more inclusive of people with Energy Limiting Conditions

Evans et al. (2023), Disbelief and Disregard Project

Resource Source Description

Recommendations for making PPI more inclusive of people with energy limiting conditions include enabling remote and/or asynchronous participation and including the cost of carers or personal assistants in engagement budgets.

Guidance: Ethical considerations for engaging frail and seriously ill patients as partners in research

Guidance: The inclusion of adults with intellectual disabilities in health research – challenges, barriers and opportunities

Ludwig et al. (2021), Research Involvement and Engagement

Bishop et al. (2024), Journal of Intellectual Disability Research

This review finds that common ethical principles for consideration when working with frail and seriously ill patients in research include autonomy, nonmaleficence, beneficence, and justice.

This review identifies non-accessible research design, lack of confidence with capacity and consent processes, limited resources including time, a need for training, and paternalistic ethics processes as barriers to meaningful research partnerships with people with intellectual disabilities.

Framework: A consensus statement on how to conduct inclusive health research

Framework: Shining a light on the road towards conducting principle-based co-production research in rehabilitation

Reflection and learning: In Pursuit of Ethical and Inclusive Research: What Ethics Committees and Disability Researchers Can Learn From Each Other

Reflection and learning: ‘Building our own house’ as an insider-only CommunityPartnered Participatory Research Council: Co-creating a safe space for Autistic knowledge production

Frankena et al. (2019), Journal of Intellectual Disability Research

Bourke et al. (2024), Frontiers in Rehabilitation Sciences

A collaborative consensus statement about how to design health research that is inclusive of people with intellectual disabilities.

In this article, the authors describe some principles essential to conducting coproduction research (sharing power, relational resilience, and adopting a learning mindset) and provide practical strategies to engage these values.

Walsh et al. (2024), International Journal of Qualitative Methods

Williams et al. (2024), Autism: the international journal of research and practice

In this paper, the authors describe productive collaboration between researchers and ethics committees using the case study of research about sexuality with young people with cerebral palsy who use communication methods other than speech.

In this paper, the authors explore what it means to be an insider-only community partnered participatory research council and reflect on the praxis they developed to facilitate autistic knowledge production.

Spotlight on Engagement with Racially Minoritised Groups

Guidance: NIHR Race Equality Framework NIHR

Guidance: Checklist for inclusive community involvement in health research (CHICO)

Guidance: Building research collaborations between universities and black and minority ethnic communities

Toolkit: Increasing Participation of Black, Asian and Minority Ethnic Groups in Health and Social Care Research

Reflection and learning: What would it take to meaningfully attend to ethnicity and race in health research? Learning from a trial intervention development study

Elizabeth Blackwell Institute for Health Research, University of Bristol

Common Cause Research

NIHR/East Midlands Academic Health Science Network

Resource Source Description

This Framework is a tool to help organisations deliver the change needed to address systemic inequities in health and care research.

This guidance was co-developed by community organisations and researchers and is intended to promote the engagement of racially minoritised groups in health research.

A report and set of principles for conducting fair and mutual research partnerships between universities and racially minoritised communities.

This toolkit is designed to help researchers address the inequitable engagement of racially minoritised groups by developing more relevant research questions, considering engagement in a more structured way, and improving the dissemination of research findings.

Reflection and learning: It should not require a pandemic to make community engagement in research leadership essential, not optional.

Rai et al. (2022), Sociology of Health and Illness

Reflection and learning: Resources to inform anti-racist co-production

Grumbach et al. (2021), Journal of Clinical and Translational Science

In this paper the authors show how “usual” research practices are unwittingly exclusionary and promote “methodological whiteness”. They argue that medical sociologists need to “recognise where we are complicit in replicating social inequalities and to actively engage with communities to produce more inclusive research.”

The authors of this paper argue that the failure to engage community stakeholders early in the COVID-19 research process cannot be divorced from systemic racism in the United States. They state that engagement should be obligatory in all clinical and translational research and that Black, Indigenous, and People of Colour should be centred in this process.

Co-Production Collective

A repository of resources to inform antiracist co-production practice.

Spotlight on Engagement with Racially Minoritised Groups

Reflection and learning: Antiracism and CommunityBased Participatory Research: Synergies, Challenges, and Opportunities

Fleming et al. (2023), American Journal of Public Health

Resource Source Description

The authors of this paper argue that health research institutions must invest in research that actively disrupts racial hierarchies, with leadership from racially marginalised communities and scholars.

Spotlight on Engaged Research in the Global South

The guides to community engagement in global health research above speak specifically to researchers working in countries in the Global South. Other resources included in this guide can also be adapted to different contexts. The material below considers specific challenges and experiences of engaged research in formerly colonised low- and middle-income settings.

Guidance: Stakeholder Engagement in Planning the Design of a National Needs Assessment for Cardiovascular Disease Prevention and Management in Nepal

Guidance: Mapping for Engagement: Setting up a Community Based Participatory Research Project to Reach Underserved Communities at Risk for Hepatitis C in Ho Chi Minh City, Vietnam

Reflection and learning: Challenges for research uptake for health policymaking and practice in low and middle-income countries: a scoping review

Archana et al. (2019), Global Heart

Resource Source Description

This paper describes an effective stakeholder engagement strategy built around the 7 Ps framework: Patients and the Public, Providers, Purchasers, Payers, Public Policy Makers and Policy Advocates, Product Makers and the Principal Investigators.

Nguyen Quoc et al. (2022), Frontiers in Public Health

Semahegn et al. (2023), Health Research Policy and Systems

In this paper, the authors describe how using information provided by local key stakeholders to shape the project helped them build solid relationships, gave the groups a sense of ownership from the outset, and made the project more relevant to the context.

This review finds that poor stakeholder engagement and partnership is one barrier to research uptake in low and middleincome countries.

Spotlight on Engaged Research in the Global South

The guides to community engagement in global health research above speak specifically to researchers working in countries in the Global South. Other resources included in this guide can also be adapted to different contexts. The material below considers specific challenges and experiences of engaged research in formerly colonised low- and middle-income settings.

Reflection and learning: Explore the practice and barriers of collaborative health policy and system researchpriority setting exercise in Ethiopia

Reflection and learning: ‘Working relationships’ across difference - a realist review of community engagement with malaria research

Reflection and learning: A critical examination of research narratives ‘rumours’ and passive community resistance in medical research

Tilahun et al. (2024), Health Research Policy and Systems

Resource Source Description

This study finds that “despite notable efforts to foster collaboration, stakeholders identified persistent challenges such as limited resources, institutional fragmentation, and inadequate coordination mechanisms as barriers to effective priority-setting processes.”

Vincent et al. (2022), Wellcome Open Research

Nyirenda et al. (2022), BMJ Global Health

In this realist review, the authors describe the importance of establishing “working relationships across differences” of wealth, power, and culture, often “in the context of suspicions and concerns about research”.

In this paper, the authors argue that negative “rumours” about medical research should be taken seriously by researchers seeking to engage with communities. They use ethnographic research to show that “rumours” contain “local critiques of medical research, legitimate concerns informed by historical experiences and local conceptualisation of health.”

Reflection and learning: Enhanced or hindered research benefits? A realist review of community engagement and participatory research practices for noncommunicable disease prevention in low- and middleincome countries

Klingberg et al. (2024), BMJ Global Health

This review offers practical recommendations for maximising benefits and minimising harm of research engagement by addressing institutional inflexibility and researcher capabilities, managing expectations, promoting solidarity in solving public health challenges, and sharing decision-making power.

Spotlight on Engagement for Policy and Practice

Toolkit: How do I monitor, evaluate and learn about policy engagement?

Toolkit: High Impact Research Training Toolkit

Toolkit: Research to Action

Course: Public Policy Engagement Series

Resource Source Description

University of Oxford A repository of resources to help researchers monitor, evaluate, and learn from policy engagement.

MEASURE Evaluation (USAID)

This toolkit aims to help researchers bridge the research-to-practice gap.

A range of tools for enhancing the positive impact of research, with a focus on international development.

UCL

The course is designed for early-career researchers to build their skills in academic policy engagement but is open to all career stages.

Methodology

This guide is informed by a rapid scoping review that we conducted based on the Arksey and O’Malley framework in July-September 2024 (100).

Search strategy

We conducted the following search on 18 July 2024 in the PubMed and Scopus research databases:

(“Engaged research” OR “participatory research” OR “participatory action research” OR “co-produced research” OR “community based participatory research” OR “stakeholder engagement” OR “public and patient involvement” OR “inclusive research”)

AND (“health research” OR “biology” OR “clinical research” OR “public health research” OR “population health research” OR “health systems research” OR “qualitative research”)

We limited our search to open access publications from 2015 onwards. For Scopus we also limited our search to the fields of social sciences, psychology, arts and humanities. Our search generated 3026 records out of which 255 were duplicates, leaving us with 2771 records for screening.

We also conducted a search of the grey literature for engaged research tools. We focused on the websites of funders, academic institutions, networks, think tanks, civil society organisations, and online learning platforms.

Record screening: inclusion and exclusion criteria

In the first screening round, two members of the team screened titles and abstracts of the 2771 identified records. A third member of the team screened a random 10% sub-sample of records for quality control purposes. Any disagreements were resolved through discussion.

Inclusion criteria

• Studies published in English.

• Studies focused on strategies for/approaches to engagement in research.

• Studies which included practical guidance for engagement (e.g., tools, frameworks, lessons learned).

Exclusion criteria

• Studies not published in English.

• Studies without an explicit focus on engagement approach.

• Studies not offering practical guidance for engagement.

The first round of screening eliminated 2040 records. Two reviewers retrieved and screened the full text of the remaining 731 records using the same inclusion and exclusion criteria. A

third member of the team again screened a random 10% sub-sample of the full-text records for quality control purposes. The full-text screening resulted in 193 records for data extraction. We extracted data from an additional 23 papers that we encountered through the initial full-text screening process and the grey literature search. The screening process is illustrated in the PRISMA diagram below.

Identification of Literature

Studies identified from Pubmed & Scopus n = 3026

Studies removed before screening: Duplicate records removed n = 255

Studies assessed in title and abstract n = 2771

Studies excluded n = 2040

Studies assessed in full text n = 731

Studies excluded n = 538

Studies included n=193. Additional studies identified through expert consultation and grey literature search n = 20. Total number of studies for data extraction n = 213

Data extraction

Three researchers extracted key study characteristics using an Excel proforma. The characteristics included details of the study e.g., title, authors, study design, and location. Additional data were included according to the themes agreed at the inception stage. Themes included: Research stage, research field, engagement approach, stakeholder group, EDI themes, challenges and barriers, key findings, and specific guidance.

Evidence synthesis

One researcher synthesised the data according to the themes agreed at the inception stage. Other team members provided review and feedback on the initial draft.

Acknowledgements

This project was led by Anna Ruddock, with research and advisory support from: Anna Gkiouleka, Keerty Nakray, Alison Allam, Hafsa Mohammed, Aaliyah Shaikh, Sayson Meya.

References

1. Ricks JM, Arthur EK, Stryker SD, Yockey RA, Anderson AM, Allensworth-Davies D. A Systematic Literature Review of Community-Based Participatory Health Research with Sexual and Gender Minority Communities. Health Equity. 2022;6(1):640–57.

2. Vaughan E, Manning MX. Are People with Aphasia Included in Stroke Trials? A Systematic Review and Narrative Synthesis. Clinical rehabilitation. 2023 Oct;37(10):1375–85.

3. Zabell T, Long KM, Scott D, Hope J, McLoughlin I, Enticott J. Engaging Healthcare Staff and Stakeholders in Healthcare Simulation Modeling to Better Translate Research Into Health Impact: A Systematic Review. Frontiers in health services. 2021;1:644831.

4. Mrklas KJ, Merali S, Khan M, Shergill S, Boyd JM, Nowell L, et al. How are health research partnerships assessed? A systematic review of outcomes, impacts, terminology and the use of theories, models and frameworks. Health research policy and systems. 2022 Dec;20(1):133.

5. Oldfield BJ, Harrison MA, Genao I, Greene AT, Pappas ME, Glover JG, et al. Patient, Family, and Community Advisory Councils in Health Care and Research: a Systematic Review. Journal of general internal medicine. 2019 Jul;34(7):1292–303.

6. Masterson D, Areskoug Josefsson K, Robert G, Nylander E, Kjellström S. Mapping definitions of co‐production and co‐design in health and social care: A systematic scoping review providing lessons for the future. Health Expectations. 2022 Jun;25(3):902–13.

7. Baxter H, Bearne L, Stone T, Thomas C, Denholm R, Redwood S, et al. The effectiveness of knowledgesharing techniques and approaches in research funded by the National Institute for Health and Care Research (NIHR): a systematic review. Health research policy and systems. 2024 Apr;22(1):41.

8. Harrison JD, Auerbach AD, Anderson W, Fagan M, Carnie M, Hanson C, et al. Patient stakeholder engagement in research: A narrative review to describe foundational principles and best practice activities. Health expectations. 2019 Jun;22(3):307–16.

9. Vaughn LM, Whetstone C, Boards A, Busch MD, Magnusson M, Määttä S. Partnering with insiders: A review of peer models across community-engaged research, education and social care. Health and Social Care in the Community. 2018;26(6):769–86.

10. Warraitch A, Wacker C, Bruce D, Bourke A, Hadfield K. A rapid review of guidelines on the involvement of adolescents in health research. Health expectations. 2024 Jun;27(3):e14058.

11. Brear M, Hammarberg K, Fisher J. Community participation in research from resource-constrained countries: a scoping review. Health Promotion International. 2018;33(4):723–33.

12. Coulter K, Ingram M, McClelland DJ, Lohr A. Positionality of Community Health Workers on Health Intervention Research Teams: A Scoping Review. Frontiers in public health. 2020;8:208.

13. Crockett LK, Scott SD, Driedger SM, Khan M, Prabhu D, Askin N, et al. Characterizing research partnerships in child health research: A scoping review. Journal of child health care. 2024 Feb;13674935241231346.

14. Ellis U, Kitchin V, Vis-Dunbar M. Identification and Reporting of Patient and Public Partner Authorship on Knowledge Syntheses: Rapid Review. Journal of participatory medicine. 2021 Jun;13(2):e27141.

15. Gilfoyle M, MacFarlane A, Salsberg J. Conceptualising, operationalising, and measuring trust in

participatory health research networks: a scoping review. Systematic reviews. 2022 Mar;11(1):40.

16. Hoekstra F, Trigo F, Sibley KM, Graham ID, Kennefick M, Mrklas KJ, et al. Systematic overviews of partnership principles and strategies identified from health research about spinal cord injury and related health conditions: A scoping review. The journal of spinal cord medicine. 2023 Jul;46(4):614–31.

17. Jakobsson CE, Genovesi E, Afolayan A, Bella-Awusah T, Omobowale O, Buyanga M, et al. Co-producing research on psychosis: a scoping review on barriers, facilitators and outcomes. International journal of mental health systems. 2023 Aug;17(1):25.

18. Liabo K, Asare L, Ruthen P, Burton J, Staunton P, Day J. Emotion in public involvement: A conceptual review. Health expectations. 2024 Apr;27(2):e14020.

19. Manafò E, Petermann L, Vandall-Walker V, Mason-Lai P. Patient and public engagement in priority setting: A systematic rapid review of the literature. PloS one. 2018;13(3):e0193579.

20. Mrklas KJ, Boyd JM, Shergill S, Merali S, Khan M, Moser C, et al. A scoping review of the globally available tools for assessing health research partnership outcomes and impacts. Health research policy and systems. 2023 Dec;21(1):139.

21. Murphy K, Branje K, White T, Cunsolo A, Latimer M, McMillan J, et al. Are we walking the talk of participatory Indigenous health research? A scoping review of the literature in Atlantic Canada. PloS one. 2021;16(7):e0255265.

22. Rajamani G, Rodriguez Espinosa P, Rosas LG. Intersection of Health Informatics Tools and Community Engagement in Health-Related Research to Reduce Health Inequities: Scoping Review. Journal of participatory medicine. 2021 Nov;13(3):e30062.

23. Semahegn A, Manyazewal T, Hanlon C, Getachew E, Fekadu B, Assefa E, et al. Challenges for research uptake for health policymaking and practice in low- and middle-income countries: a scoping review. Health research policy and systems. 2023 Dec;21(1):131.

24. Shahid A, Lalani IN, Rosgen BK, Sept BG, Longmore S, Parsons Leigh J, et al. A scoping review of methods to measure and evaluate citizen engagement in health research. Research involvement and engagement. 2022 Dec;8(1):72.

25. Tindana P, de Vries J, Campbell M, Littler K, Seeley J, Marshall P, et al. Community engagement strategies for genomic studies in Africa: a review of the literature. BMC medical ethics. 2015 Apr;16:24.

26. Vazquez Corona M, Hazfiarini A, Vaughan C, Block K, Bohren MA. Participatory Health Research With Women From Refugee, Asylum-Seeker, and Migrant Backgrounds Living in High-Income Countries: A Scoping Review. International Journal of Qualitative Methods. 2024;23.

27. Woodbury RB, Ketchum S, Hiratsuka VY, Spicer P. Health-Related Participatory Research in American Indian and Alaska Native Communities: A Scoping Review. International journal of environmental research and public health. 2019 Aug;16(16).

28. Dada S, Aivalli P, De Brún A, Barreix M, Chelwa N, Mutunga Z, et al. Understanding communication in community engagement for maternal and newborn health programmes in low- and middle-income countries: a realist review. Health Policy and Planning. 2023 Oct 11;38(9):1079–98.

29. Klingberg S, Adhikari B, Draper CE, Bosire E, Nyirenda D, Tiigah P, et al. Enhanced or hindered research benefits? A realist review of community engagement and participatory research practices for noncommunicable disease prevention in low- and middle-income countries. BMJ global health. 2024 Feb;9(2).

30. Vincent R, Adhikari B, Duddy C, Richardson E, Wong G, Lavery J, et al. ‘Working relationships’ across difference - a realist review of community engagement with malaria research. Wellcome open research. 2022;7:13.

31. Hoekstra F, Mrklas KJ, Khan M, McKay RC, Vis-Dunbar M, Sibley KM, et al. A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: a first step in synthesising the research partnership literature. Health research policy and systems. 2020 May;18(1):51.

32. Ortiz K, Nash J, Shea L, Oetzel J, Garoutte J, Sanchez-Youngman S, et al. Partnerships, Processes, and Outcomes: A Health Equity-Focused Scoping Meta-Review of Community-Engaged Scholarship. Annual review of public health. 2020 Apr;41:177–99.

33. Slattery P, Saeri AK, Bragge P. Research co-design in health: a rapid overview of reviews. Health research policy and systems. 2020 Feb;18(1):17.

34. Vinnicombe S, Bianchim MS, Noyes J. A review of reviews exploring patient and public involvement in population health research and development of tools containing best practice guidance. BMC public health. 2023 Jun;23(1):1271.

35. Warraitch A, Wacker C, Biju S, Lee M, Bruce D, Curran P, et al. Positive Impacts of Adolescent Involvement in Health Research: An Umbrella Review. The Journal of Adolescent Health. 2024 Apr.

36. Wyatt KA, Bell J, Cooper J, Constable L, Siero W, Pozo Jeria C, et al. Involvement of children and young people in the conduct of health research: A rapid umbrella review. Health expectations. 2024 Jun;27(3).

37. Chudyk AM, Stoddard R, McCleary N, Duhamel TA, Shimmin C, Hickes S, et al. Exploring patient and caregiver perceptions of the meaning of the patient partner role: a qualitative study. Research involvement and engagement. 2023 Nov;9(1):106.

38. Drinkwater J, Farr M, Hickey G, Van Vliet E, Söderholm Werkö S, Klingmann I, et al. Sharing power and building trust through partnering with communities in primary care research. The European journal of general practice. 2024 Dec;30(1):2328707.

39. Pinto RM, Park SE, Miles R, Ong PN. Community engagement in dissemination and implementation models: A narrative review. Implementation research and practice. 2021 Jan;2.

40. Nguyen T, Graham ID, Mrklas KJ, Bowen S, Cargo M, Estabrooks CA, et al. How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field. Health research policy and systems. 2020 Mar;18(1):35.

41. Paylor J, McKevitt C. The Possibilities and Limits of “Co-producing” Research. Frontiers in Sociology. 2019;4.

42. Oliver K, Kothari A, Mays N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health research policy and systems. 2019 Mar;17(1):33.

43. Williams O, Sarre S, Papoulias SC, Knowles S, Robert G, Beresford P, et al. Lost in the shadows: reflections on the dark side of co-production. Health research policy and systems. 2020 May;18(1):43.

44. Woitowich NC, Beery A, Woodruff T. A 10-year follow-up study of sex inclusion in the biological sciences eLife. 2020 Jun;9.

45. Adkins-Jackson PB, Burke NJ, Espinosa PR, Ison JM, Goold SD, Rosas LG, et al. Inclusionary Trials: A Review of Lessons Not Learned. Epidemiologic reviews. 2022 Dec;44(1):78–86.

46. Gillibrand S, Kapadia D, Watkinson R, Issa B, Kwaku-Odoi C, Sanders C. Marginalisation and distrust in the context of the COVID-19 vaccination programme: experiences of communities in a northern UK city region. BMC Public Health. 2024 Mar 19;24(1):853.

47. Frankena TK, Naaldenberg J, Cardol M, Garcia Iriarte E, Buchner T, Brooker K, et al. A consensus statement on how to conduct inclusive health research. Journal of Intellectual Disability Research. 2019;63(1):1–11.

48. Beeker T, Glück RK, Ziegenhagen J, Göppert L, Jänchen P, Krispin H, et al. Designed to Clash? Reflecting on the Practical, Personal, and Structural Challenges of Collaborative Research in Psychiatry. Frontiers in psychiatry. 2021;12:701312.

49. Williams GL, Ellis R, Holloway W, Caemawr S, Craine M, Williams K, et al. ‘Building our own house’ as an insider-only Community-Partnered Participatory Research Council: Co-creating a safe space for Autistic knowledge production. Autism. 2024 May.

50. Loignon C, Dupéré S, Leblanc C, Truchon K, Bouchard A, Arsenault J, et al. Equity and inclusivity in research: co-creation of a digital platform with representatives of marginalized populations to enhance the involvement in research of people with limited literacy skills. Research Involvement and Engagement. 2021;7(1).

51. MacFarlane A, Ogoro M, de Freitas C, Niranjan V, Severoni S, Waagensen E. Migrants’ involvement in health policy, service development and research in the WHO European Region: A narrative review of policy and practice. Tropical medicine & international health. 2021 Oct;26(10):1164–76.

52. Grumbach K, Cottler LB, Brown J, LeSarre M, Gonzalez-Fisher RF, Williams CD, et al. It should not require a pandemic to make community engagement in research leadership essential, not optional. Journal of clinical and translational science. 2021 Feb;5(1):e95.

53. Denegri S, Starling B. COVID-19 and patient engagement in health research: What have we learned? CMAJ. 2021 Jul;193(27):E1048–9.

54. Harting J, Kruithof K, Ruijter L, Stronks K. Participatory research in health promotion: a critical review and illustration of rationales. Health promotion international. 2022 Jun;37:ii7–20.

55. Chapman K, Dixon A, Cocks K, Ehrlich C, Kendall E. The Dignity Project Framework: An extreme citizen science framework in occupational therapy and rehabilitation research. Australian occupational therapy journal. 2022 Dec;69(6):742–52.

56. Fleming PJ, Stone LC, Creary MS, Greene-Moton E, Israel BA, Key KD, et al. Antiracism and CommunityBased Participatory Research: Synergies, Challenges, and Opportunities. American journal of public health. 2023 Jan;113(1):70–8.

57. Walsh M, Stead V, Sawyer SM, O’Shea A, Watson JM, Anderson KLM. In Pursuit of Ethical and Inclusive Research: What Ethics Committees and Disability Researchers Can Learn From Each Other. International Journal of Qualitative Methods. 2024;23.

58. Allman D. Community centrality and social science research. Anthropology and Medicine. 2015;22(3):217–33.

59. Béland S, Lambert M, Delahunty-Pike A, Howse D, Schwarz C, Chouinard MC, et al. Patient and researcher experiences of patient engagement in primary care health care research: A participatory qualitative study. Health expectations. 2022 Oct;25(5):2365–76.

60. Leader AE, Aplin AE. From the Community to the Bench and Back Again: The Value of Patient and Community Engagement in Cancer Research. Cancer discovery. 2021 Sep;11(9):2135–8.

61. Shaw KL, Calvert MJ, Borrow P, Chakera E, Chakraverty R, Hughes SE, et al. Integrating patient and public involvement and engagement in translational medicine. The Lancet. 2024 Aug.

62. Aquino EN, Moss P, Hafeez M, Jasper R, Kelly T, Laidlaw L, et al. The impact of patient and public involvement on COVID-19 immunology research: experiences from the UK Coronavirus Immunology Consortium. Research Involvement and Engagement. 2023 May 22;9(1):34.

63. Peters U, Turner B, Alvarez D, Murray M, Sharma A, Mohan S, et al. Considerations for Embedding Inclusive Research Principles in the Design and Execution of Clinical Trials. Therapeutic innovation & regulatory science. 2023 Mar;57(2):186–95.

64. Costello W, Dorris E. Laying the groundwork: Building relationships for public and patient involvement in pre-clinical paediatric research. Health expectations. 2020 Feb;23(1):96–105.

65. La Scala S, Mullins JL, Firat RB, Michalska KJ. Equity, diversity, and inclusion in developmental neuroscience: Practical lessons from community-based participatory research. Frontiers in integrative neuroscience. 2022;16.

66. Lang I, King A, Jenkins G, Boddy K, Khan Z, Liabo K. How common is patient and public involvement (PPI)? Cross-sectional analysis of frequency of PPI reporting in health research papers and associations with methods, funding sources and other factors. BMJ Open. 2022 May;12(5).

67. Shim JK, Foti N, Vasquez E, Fullerton SM, Bentz M, Jeske M, et al. Community Engagement in Precision Medicine Research: Organizational Practices and Their Impacts for Equity. AJOB Empirical Bioethics. 2023;14(4):185–96.

68. Wilkinson A, Slack C, Thabethe S, Salzwedel J. “It’s Almost as if Stakeholder Engagement is the Annoying ‘Have-to-do’…”: Can Ethics Review Help Address the “3 Ts” of Tokenism, Toxicity, and Tailoring in Stakeholder Engagement? Journal of Empirical Research on Human Research Ethics. 2022 Jul;17(3):292–303.

69. Fregonese F. Community involvement in biomedical research conducted in the global health context; What can be done to make it really matter? BMC Medical Ethics. 2018;19.

70. Papageorgiou V, Dewa LH, Bruton J, Murray KK, Hewlett N, Thamm W, et al. ‘Building bridges’: reflections and recommendations for co-producing health research. Research involvement and engagement. 2023 Dec;9(1):113.

71. Kapiriri L. Stakeholder involvement in health research priority setting in low income countries: The case of Zambia. Research Involvement and Engagement. 2018;4(1).

72. Heney V, Poleykett B. The impossibility of engaged research: Complicity and accountability between researchers, ‘publics’ and institutions. Sociology of health & illness. 2022 Dec;44(S1):179–94.

73. Harris J, Croot L, Thompson J, Springett J. How stakeholder participation can contribute to systematic reviews of complex interventions. Journal of epidemiology and community health. 2016 Feb;70(2):207–14.

74. Enria L, Lees S, Smout E, Mooney T, Tengbeh AF, Leigh B, et al. Power, fairness and trust: understanding and engaging with vaccine trial participants and communities in the setting up the EBOVAC-Salone vaccine trial in Sierra Leone. BMC public health. 2016 Nov;16(1):1140.

75. Nyirenda D, Sariola S, Desmond N. A critical examination of research narratives ‘rumours’ and passive community resistance in medical research. BMJ Global Health. 2022 Apr;7(4).

76. Antoine-LaVigne D, Hayes T, Fortenberry M, Ohikhuai E, Addison C, Mozee SJ, et al. Trust and Biomedical Research Engagement of Minority and Under-Represented Communities in Mississippi, USA. International journal of environmental research and public health. 2023 Jan;20(2).

77. Clark M, Van Vliet E, Collins M. Reflections from the COVID‐19 pandemic on inequalities and patient and public involvement and engagement (PPIE) in social care, health and public health research. Health Expectations. 2021 Oct;24(5):1547–50.

78. Jameson C, Haq Z, Musse S, Kosar Z, Watson G, Wylde V. Inclusive approaches to involvement of community groups in health research: the co-produced CHICO guidance. Research involvement and engagement. 2023 Sep 7;9(1):76.

79. Ludwig C, Graham ID, Lavoie J, Gifford W, Stacey D. Ethical considerations for engaging frail and seriously ill patients as partners in research: sub-analysis of a systematic review. Research involvement and engagement. 2021 Jan;7(1):8.

80. Aabe NO, Fox F, Rai D, Redwood S. Inside, outside and in-between: The process and impact of coproducing knowledge about autism in a UK Somali community. Health expectations. 2019 Aug;22(4):752–60.

81. Rai T, Hinton L, McManus RJ, Pope C. What would it take to meaningfully attend to ethnicity and race in health research? Learning from a trial intervention development study. Sociology of health & illness. 2022 Dec;44:57–72.

82. Soklaridis S, Harris H, Shier R, Rovet J, Black G, Bellissimo G, et al. A balancing act: navigating the nuances of co-production in mental health research. Research involvement and engagement. 2024 Mar;10(1):30.

83. Faluyi D, Ovseiko PV, Dziedzic K, Scott F, on behalf of the NIHR Race Equality Public Action Group, Tulloch A, et al. NIHR Race Equality Framework: development of a tool for addressing racial equality in

public involvement. Research involvement and engagement. 2024 May 7;10(1):44.

84. Imison C, Kaur M, Dawson S. Supporting equity and tackling inequality: how can NIHR promote inclusion in public partnerships? NIHR. 2022.

85. Bourke JA, Bragge P, River J, Sinnott Jerram KA, Arora M, Middleton JW. Shining a light on the road towards conducting principle-based co-production research in rehabilitation. Frontiers in rehabilitation sciences. 2024;5.

86. Cruz TH, Borrego ME, Page-Reeves J. Increasing the Number of Underrepresented Minority Behavioral Health Researchers Partnering With Underresourced Communities: Lessons Learned From a Pilot Research Project Program. Health promotion practice. 2020 Nov;21(6):865–71.

87. Islam S, Joseph O, Chaudry A, Forde D, Keane A, Wilson C, et al. ‘We are not hard to reach, but we may find it hard to trust’ …. Involving and engaging “seldom listened to” community voices in clinical translational health research: a social innovation approach. Research involvement and engagement. 2021 Jun;7(1):46.

88. Lightbody R. ‘Hard to reach’ or ‘easy to ignore’? Promoting equality in community engagement Edinburgh: What Works Scotland; 2017.

89. Evans B, Allam A, Bê A, Hale C, Rose M, Ruddock A. Being left behind beyond recovery: ‘crip time’ and chronic illness in neoliberal academia. Social & Cultural Geography. 2024 Oct 9;1–21.

90. Worsley JD, McKeown M, Wilson T, Corcoran R. A qualitative evaluation of coproduction of research: ‘If you do it properly, you will get turbulence’. Health Expectations. 2022 Oct;25(5):2034–42.

91. Akwataghibe NN, Ogunsola EA, Broerse JEW, Agbo AI, Dieleman MA. Inclusion strategies in multistakeholder dialogues: The case of a community-based participatory research on immunization in Nigeria PloS one. 2022;17(3).

92. Castleden H, Sylvestre P, Martin D, McNally M. ‘I don’t think that any peer review committee . . . would ever “get” what I currently do’: How institutional metrics for success and merit risk perpetuating the (re) production of colonial relationships in community-based participatory research involving indigenous peoples in Canada. International Indigenous Policy Journal. 2015;6(4).

93. Groot BC, Vink M, Haveman A, Huberts M, Schout G, Abma TA. Ethics of care in participatory health research: mutual responsibility in collaboration with co-researchers. Educational Action Research. 2019;27(2):286–302.

94. Calzo JP, Bogart LM, Francis E, Kornetsky SZ, Winkler SJ, Kaberry J. Engaging institutional review boards in developing a brief, community-responsive human subjects training for community partners. Progress in Community Health Partnerships: Research, Education, and Action. 2016;10(3):471–7.

95. Vawer M, Rutiz R, Tukuafu V, Whatley K, Hanchett K, Kahaleuahi L, et al. Hana Pu No Ke Ola O Hana (‘Working Together for the Health of Hana’): Our 14-year CBPR Journey. Hawai’i journal of medicine & public health. 2019 Mar;78(3):111–4.

96. Redman RM, Reinsvold MC, Reddy A, Bennett PE, Hoerauf JM, Puls KM, et al. A successful model for longitudinal community-engaged health research: the 2040 Partners for Health Student Program. Family practice. 2017 Jun;34(3):341–6.

97. Lavery JV. Building an evidence base for stakeholder engagement. Science. 2018 Aug 10;361(6402):554–6.

98. MacQueen KM, Bhan A, Frohlich J, Holzer J, Sugarman J. Evaluating community engagement in global health research: The need for metrics. BMC Medical Ethics. 2015;16(1).

99. Haenssgen MJ. New impulses from international development for more comprehensive and balanced public engagement evaluation. Global health action. 2019;12.

100. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005 Feb;8(1):19–32.