STEPS OVC - Data Demand Brief

Photo: Futures Group

January, 2016

Using Data to Monitor Performance STEPS OVC Care Groups

About

Geographic Reach

Sustainability through Economic Strengthening, Prevention and Support for Orphans and Vulnerable Children, Youth and Other Vulnerable Populations (STEPS OVC), is a six-year USAID-funded project that brings together the expertise and Zambia-wide coverage of international and local non-governmental organizations under the leadership of World Vision.

STEPS OVC implements activities countrywide with a focus on communities, community based organizations (CBOs), and government. In addition to the main office in Lusaka, coordination and roll-out of interventions is implemented out of partner offices dotted around the country which also support transition of activities and organizational capacity building, and gather information for learning.

Goal To strengthen capacity of Zambian communities to provide sustainable HIV prevention, care and support services in a harmonized manner to priority geographic areas, target populations, and stakeholder communities.

Dates July 2010 to March 2016

Funding US$75,000,000 million (USAID) US$17,374,797 (Matched Funds)

Targets STEPS OVC mitigates the HIV epidemic by strengthening comprehensive support services for over 400,000 orphans and vulnerable children (OVC); improving quality of life for 164,000 adults and children living with HIV (PLHIV); delivering HIV prevention information and behavior change skills to 80,000 HIV- and HIV+ persons; providing HIV counseling and testing services to over 630,000 people; increasing livelihoods of more than 40,000 beneficiaries through economic strengthening activities; involving an organized network of over 45,000 trained, equipped caregivers; building capacity of almost 400 Zambian-owned CBOs to respond to communities and households affected by HIV and AIDS.

Abstract The Sustainability Through Economic Strengthening, Prevention, and Support to Orphans and Vulnerable Children, Youth and Other Vulnerable Populations (STEPS OVC) project is a six-year national program that targets approximately 20 percent of all OVC in Zambia through community-based interventions in all of the country’s districts. The work of the project is sustained by more than 40,000 volunteer community caregivers who provide care and support to the target

populations and their households. These community caregivers routinely collect and report data related to the program and the services they provide, yet little research has been conducted on understanding how the data is used at the local and community levels. To explore data demand and use (DDU) at the local level, STEPS OVC conducted focus group discussions to learn how site coordinators collect, understand and use data.

using data to monitor performance of steps ovc care groups

Background Part of improving international health programs and systems is strengthening data demand and use (DDU). Considerable financial resources have been invested in strengthening the capacity of decision makers to use information for decision-making, in program planning, in allocating resources, and in policy formulation. Less research, however, has been conducted on understanding how information is used at the community level, especially by groups who provide health care, support services, and information to households with vulnerable populations. To help close this information gap, the USAID-funded STEPS OVC project conducted focus group discussions with site coordinators to better understand how data is collected, understood and used at the community level.

The USAID-funded STEPS OVC project is a national-level program under the leadership of World Vision. The overall goal of the program is to provide sustainable HIV prevention, care, and support services to two target populations in Zambia (1) orphans and vulnerable children (OVC) and (2) persons living with HIV (PLHIV).

To meet this goal, STEPS OVC has built on the expertise and geographic coverage of international and local organizations throughout Zambia. The work of the project is sustained by more than 40,000 volunteer community caregivers who provide care and support to the target populations and their households. Some of the volunteer community caregivers also act “site coordinators” supporting project coordination at the community (“site”) level. These site coordinators receive direction from district-level coordinators.

To explore DDU at the local level, STEPS OVC investigated how site coordinators collect, understand and use data.

Along with their community caregiver responsibilities, site coordinators are also accountable for overseeing

a team of community caregivers. They must be able to collect, review and use data to identify performance gaps among team members to ensure the quality of the services provided by the community caregivers is high. When issues arise, site coordinators are also responsible for identifying low performing team members and recommending corrective actions.

Because site coordinators review data submitted by community caregivers they also play an important role in ensuring data quality across the STEPS OVC project.

Study Objective purpose The purpose of this study was to gather information from site coordinators and community caregivers on how data is collected, understood and used within the STEPS OVC project. Findings informed a “Community Report”, which is a feedback tool to community caregivers showcasing information on program targets and community caregiver performance.

methodology The study team conducted four focus group discussions from November 26-27, 2013. There were a total of 42 participants comprised of community caregivers (n=33) and site coordinators (n=9) (see Table 1). Focus group participants were identified through the STEPS OVC database (COPIS) and were purposively selected to reflect diversity in educational attainment and caregiving experience.

Focus group sites were selected through a simple randomization of care group wards within the Kitwe District1. Each care group is managed by a subgrantee who assisted with notifying study participants, making logistical arrangements and introducing study personnel.

Table 1: Overview of Focus Groups focus group partipants

1

2

#

ward

criterion

1

Mindolo

2

community caregivers

site coordinators

total

≥ grade 10 education ≥ 5 years caregiving experience

9

5

14

Bupe

≥ grade 10 education ≤ 4 years caregiving experience

12

0

12

3

Luangwa

≥ grade 9 education ≥ 5 years caregiving experience

8

3

11

4

Kafue

≤ grade 9 education ≥ 4 years caregiving experience

4

1

5

Kitwe District was purposively selected due to distance from Lusaka and the diversity of organizations supporting community caregivers there.

using data to monitor performance of steps ovc care groups

Study Team The purpose of this study was to gather information from site coordinators and community caregivers on how data is collected, understood and used within the STEPS OVC project. Findings informed a “Community Report”, which is a feedback tool to community caregivers showcasing information on program targets and community caregiver performance.

Description The study team conducted four focus group discussions from November 26-27, 2013. There were a total of 42 participants comprised of community caregivers (n=33) and site coordinators (n=9) (see Table 1). Focus group participants were identified through the STEPS OVC database (COPIS) and were purposively selected to reflect diversity in educational attainment and caregiving experience.

Focus group sites were selected through a simple randomization of care group wards within the Kitwe District. Each care group is managed by a sub-grantee who assisted with notifying study participants, making logistical arrangements and introducing study personnel.

Findings

Photo: Marlene Lammers

INFORMATION DEMAND

INFORMATION USE

Moderators asked open-ended questions to better understand the types of information demands faced by participants. Moderators probed into the types of information requested of them, who requests it and how they responded. Participants reported that project beneficiaries, household members, community members and relatives of caregivers frequently ask why tangible services, like mosquito bed nets and school fees, are no longer being provided through the program.

According to participants, this change has caused community caregivers embarrassment and resulted in negative reactions, data collection difficulties, and loss of trust from community members.

When asked questions about how information is used and understood by site coordinators and community caregivers, participants indicated that most community caregivers do not use the data that they collect.

Participants agreed that they collect data because it is a project requirement. A few site coordinator respondents, however, noted that they use the information collected to monitor community caregivers’ performance. Site coordinators note how many caregivers have submitted reports, and make judgments on how well clients were served.

Participants also explained that beneficiaries questioned why they collect information at every visit and how this information is being used in the program. Specifically, beneficiaries were frustrated with questions around access to material items, such as soap, when the project does not provide these. However, this points to a larger issue of burdening beneficiaries by asking questions with possible responses for which there is no solution.

PERFORMANCE MONITORING When asked about how care groups monitor performance to ensure they are meeting the objectives of STEPS OVC, focus group participants explained that care groups meet at least once a month and sometimes more often. Monthly meetings cover performance monitoring topics including challenges, upcoming plans, reporting timeliness, and care group targets.

Only one site coordinator mentioned data quality as a topic covered at the monthly care group meetings.

3

using data to monitor performance of steps ovc care groups

Site coordinators do not visit households to monitor delivery of services.

INFORMATION NEEDS When asked questions about their information needs, community caregiver participants requested feedback on reporting performance, in particular about any errors in forms they submitted. They also requested additional information on what actions the program has taken in response to the data they have collected and reported.

FEEDBACK REPORT PREFERENCE At the end of the focus group discussion, the moderators showed participants two versions of a Community Report mock-up. The Community Report is intended to assist site coordinators with monitoring performance of their respective care group. Based on their feedback, STEPS OVC has design a single report that best suits preferences and needs.

Conclusions The focus groups provided insight on DDU at the community level. Results indicate that while community caregivers understand the importance of their role in data collection, there are not clear how data is used to improve the program or how it can be used at the community level to improve their ability to monitor care group performance.

It is apparent from the discussions that project beneficiaries and community members are also unclear why the project collects the array of data it does, and how this data is used to improve the program. Improving community caregivers ability to use information at the community level and equipping them with the skills to explain to beneficiaries whey data are collected, and what is done with them is critical.

As a result of this exercise, STEPS OVC designed a routine community report, which was incorporated into the project management information system.

Acknowledgments This paper was developed by Sonja Schimdt, M&E Advisor, from material developed by the STEPS OVC program management unit and partner staff. Inputs and insights were also provided by the M&E Technical Team: Mathew Ngunga, Daniel Phiri and Nachela Chelwa.

This technical briefing paper has been supported by the President’s Emergency Plan for AIDS Relief (PEPFAR) through the United States Agency for International Development (USAID) under the terms of the STEPS OVC cooperative agreement 611-A-00-10-00001-00. The views expressed are not necessarily those of PEPFAR, USAID, or the United States government.

recommended citation STEPS OVC. 2014. Using Data to Monitor Performance of STEPS OVC Care Groups. Lusaka, Zambia: STEPS OVC. 4

Photo: Michael Mistretta

contact information For more information about this study, please contact: Chief of Party STEPS OVC Program Plot No. 29E/377A Sable Road Kabulonga, Lusaka, Zambia Tel: +260 211 263 476 fax: +260 211 263 718 Email: steps_ovc@wvi.org