Evaluations of Promising Practices in Child Welfare

The story of NGOs serving vulnerable children and families is often one of striving to meet too many needs with too few resources. Out of necessity, NGOs respond nimbly to diverse and ever-changing needs, creating innovative solutions as they do so. These solutions may hold important implications and applications for other programs serving vulnerable children and families in other contexts around the world.

However, how can we be sure these innovative practices are achieving the results we hope for? How can we know they are producing good and not harm? Evaluation helps us to know what works, what doesn’t, and what can be better. Yet many NGOs don’t have the time or resources to carefully evaluate the practices they are using. That’s where this project comes in.

In 2022, CAFO’s Center on Applied Research for Vulnerable Children & Families offered small grants for projects to connect research with practice in child welfare. Four Researcher + NGO pairs were matched up and worked together over the course of a year to:

• Gather data about the NGO’s promising practice

• Analyze the data to identify the practice’s strengths and areas for growth

• Translate the data into a format that’s easy to understand and put into practice

This booklet shares the translated data that was prepared by the 2022 Research Challenge Grant pairs, highlighting the results and implications in a way that enables other NGOs to utilize effective solutions that have been evaluated for effectiveness.

Helping Children Worldwide + Dr. Sarah Neville, Brown University

Program Location: Sierra Leone

Promising Practice: Family Strengthening Program

Helping

Children Worldwide + Dr. Sarah Neville, Brown University

Helping Children Worldwide (HCW) has served 1,739 children and youth in 419 families since being founded in 2003. The Family Strengthening Program is delivered by program partners in Sierra Leone at the Child Reintegration Centre.

Promising Practice Under Review

The Child Reintegration Center offers a 2-day Family Strengthening Workshop focused on relationship resiliency and basic economic literacy for caregivers and their children with the ultimate goal of preventing family separation. Case Managers then provide ongoing content reinforcement through home visits to support families in practicing their learning at home once the workshop is completed.

The aim of this study was to determine whether this 2-phase Family Strengthening Program improves outcomes in families better than those participating in the workshop and receiving standard home visits without the added family strengthening program material. The effectiveness of the study is being evaluated in relation to

• Changing parenting behaviors

• Improving emotional regulation

• Improving caregiver-child attachment, and

• Improving financial literacy in dyads consisting of children ages 9-13 and their caregivers

“It is amazing to be able to work directly with a researcher who is so passionate about the topic. If our practice does prove to be promising, we would have research to support the marketing we already do around the efficacy of this practice, and we would be able to share with other organizations not just the practice, but proof of the impact of the practice.”

Basic Methodology

For this study, 100 child + caregiver pairs were selected to complete a baseline survey. Of those 100 pairs, 50 were selected to participate in the 2-phase program, which integrates Family Strengthening Program curriculum into their follow-up home visits. The other 50 pairs receive their standard care of receiving follow-up home visits without the added Family Strengthening Program curriculum. All 100 pairs completed an endline survey. The change in outcomes from start to finish (such as measures of emotional regulation, parenting practices, attachment, etc.) were compared between the group receiving home visits with Family Strengthening curriculum and the group receiving visits without the curriculum.

Basic Findings

Preliminary findings show that caregivers who participated in the program were more likely to say they apologized to their children, and said their children were more likely to apologize to them. Caregivers who participated in the program also said they comforted their child more. Children who participated in the intervention said their parents engaged in more “talk, time, and touch” with them after the intervention, such as holding conversations about their children’s lives, spending dedicated time with them, and showing physical affection. Children confirmed that they were more likely to apologize to their parents when they did something wrong after the intervention, and that their caregivers comforted them more often.

“I’m very excited because this may be the first quantitative evaluation of a family preservation program in LMIC (Low and Middle Income Country) that’s specifically meant to prevent children entering residential care. If the results are promising, HCW will be able to share the curriculum with other organizations so their clients can benefit from it as well. With promising results, HCW will be able to share their curriculum with other organizations so their clients can benefit from it as well.”

DR. SARAH NEVILLE

Single Biggest Takeaway

The family strengthening program teaches families useful, new attachment-building practices - things they wouldn’t necessarily have done if they were not taught about the importance of these practices. Thanks to the cultural relevance of the program, the families are willing to adopt these new behaviors, which will lead to better family relationships and trust.

Zoe Empowers + Obed Masese, Changing the Way We Care

Zoe Empowers + Obed Masese, Changing the Way We Care

Zoe Empowers has served over 160,000 vulnerable children and youth since 2007, with over 50,000 currently involved in the program. They operate in Kenya, Rwanda, Zimbabwe, Malawi, Liberia, Tanzania, Mozambique, and India, with support staff in the US.

Promising Practice Under Review

The study examined the impact of Zoe Empowers’ 3-year program on empowering child or youth-headed households, evaluating whether the skills learned and the network of support built by these young people while in the program equipped them to succeed on their own well past graduation. The program empowers vulnerable children and youth (13-21 years) to take the lead in their journey out of poverty, making their own decisions and assisting one another in permanently moving beyond dependence. Program participants join mutual support groups and receive training and resources from a Zoe program facilitator.

“We had not previously tracked beyond the program and were relying on anecdotal information in this area. We were extremely gratified to see that the progress of these young participants seemed only to strengthen up to four years after the completion of the program.”

Basic Methodology

A survey was used to gather data from a random sample of 385 of the 4767 program graduates who had graduated from Zoe’s program since 2018. Information was gathered about life conditions across several areas: food security and nutrition, housing, community reintegration, health and hygiene, child rights, education and vocational training, business development, and spiritual health.

Basic Findings

The data showed a significant, continued impact on Zoe program graduates across all measured areas: food security, safe housing, community reintegration, health and hygiene, child rights, education and vocational training, business development, and spiritual strength.

“The most surprising finding was that there were so many child and/ or youth headed households in the study area that the organization was not able to support even half of them. The model was easy to roll out and understand among the beneficiaries, and also efficient in terms of the cost of implementation. The model looked simple but its impact on household empowerment was surprisingly outstanding.”

OBED MASESE

Single Biggest Takeaway

Providing care for children while they remain at home with their kin, communities, families, and relatives is sustainable, permanent, costeffective, and has a greater impact in caring for vulnerable youths and children than institutional care. It is possible to design interventions that are not only sustainable but where a once-vulnerable family is able to put themselves on an upward trajectory, which should produce generational change.

Refuge: Zimbabwe + Dr. Pamhidzayi Mhongera, University of Johannesburg

Program Location:

Zimbabwe

Promising Practice:

Child-Directed Healing Play

Refuge: Zimbabwe + Dr. Pamhidzayi Mhongera, University of Johannesburg

Refuge: Zimbabwe brings hope and healing through connection for children, families and communities. They seek to ensure that every child has nurturing adults and safe spaces in which they experience hope and healing through connection with God and others. Refuge: Zimbabwe has served 400 children and families since its founding in 2013.

Promising Practice Under Review

The study evaluated the effectiveness of Child Directed Healing Play, a structured individual therapeutic play session between a caring adult and a primary-aged child aimed at emotional processing through play. Thirty teachers from 8 schools were trained in CDHP to participate in the study examining the effectiveness of CDHP in improving emotional regulation and student-teacher relationships.

“We have learned so much in the process. Working with a researcher who understands how to measure the impact of our work has been a delight. We have seen our relationships with partners deepen as they participate in evaluating the effectiveness of Child Directed Healing Play. As a practitioner, I have found myself reinvigorated by the prospect of being able to evidence our work in a different way.”

Basic Methodology

Three measures of evaluation were distributed to teachers and parents of 30 children who are receiving Child Directed Healing Play Sessions from their teachers and 30 children who are not. Measures of the child’s emotional regulation and the student-teacher relationship were administered before and after the intervention for those participating in the study, as well those in the control group. Trained teachers conducted 5 weekly 20-minute sessions of CDHP with one child from the intervention group. A comparison was made to identify changes in the emotional regulation and the student-teacher relationship in both the intervention and control groups.

Basic Findings

Preliminary qualitative findings show some positive outcomes in the student-teacher relationships. Feedback from teachers included observations on children becoming more relaxed, engaging in more calm and interactive play, and responding positively to the limits set out by the teacher. One teacher noted they are increasingly able to remain calm and regulated while irritated with a child’s behavior. Children have been increasingly willing to share stories and express themselves in response to the engagement strategies of the CDHP program. Quantitative data analysis is ongoing and will reveal more about the nature and mechanisms around program effects.

“The opportunity of partnering with a nonprofit to evaluate a promising practice, which is the first of its kind in Zimbabwe, was exhilarating. This program has great potential if it is to be replicated in public schools and communities where the incidences of trauma are higher than the current study area.“

DR. PAMHIDZAYI MHONGERA

Single Biggest Takeaway

Child-Directed Healing Play is an effective and sustainable way to promote mental health and foster successful student-teacher relationships. I was surprised by the transformative power of the intervention had to teachers’ way of thinking and reacting to some troublesome behaviors and how the training helped the development of empathy and curiosity helped produce better outcomes among children. The healing sessions during training generated much emotion and had an impact even on adults (teachers).

Agape Children’s Ministries +

Dr. Johanna Greeson

Associate Professor, School of Social Policy & Practice, University of Pennsylvania

Program Location: Kenya

Promising Practice: Family Strengthening Program

Agape Children’s Ministries +

Dr. Johanna Greeson, Associate Professor, School of Social Policy & Practice, University of Pennsylvania

Agape Children’s Ministry has worked in Kenya to meet the needs of children living on the streets and in governmental incarceration by reconciling them back to safe and loving homes throughout East Africa. Through Gospel-centered programs, the Agape Team has served over 5,000 children since 2010.

Promising Practice Under Review

Agape’s Family Strengthening Program is designed to prevent family breakdown after reunification where the ongoing child welfare worker needs additional support from a Family Strengthening Officer. The program is conducted by nine Family Strengthening Officers who provide intensive counseling to families who have recently had a child reunified from the streets and are at high risk of family breakdown.

“Our hope is that quantifiable change has been seen in the cohesiveness of families involved within the Family Strengthening intervention. If this is the case, then our hope is that Agape’s practices can be shared and adopted by other organizations looking to build stronger families postreintegration.”

Basic Methodology

This study utilized a customized version of the Family Togetherness Tool to ascertain the cohesiveness of 37 different high-risk families by assessing family togetherness, child wellbeing, and family stability. Scores were collected at both the start and completion of the Family Strengthening Intervention. Additionally, preand post-Child Status Index (CSI) scores were collected. Lastly, a “Pass/Fail” designation was established to determine if a family remained together or not during the time frame of the research. Semi-structured interviews with staff and clients were conducted to obtain additional insight, and a site visit was conducted to develop a cultural understanding of the contexts in which services are provided.

Basic Findings

Qualitative interviews with staff and clients indicate that basic needs and food insecurity remain a challenge to family stability, but that spiritual and relational connectedness are protective and provide families with hopefulness and a sense of resilience. Scores on the Family Togetherness Scale increased to a statistically significant extent after receiving family strengthening services.

“We were surprised to learn that if a family reunification lasts for 2 years, over 99% of those children will remain at home until they are 18. The hope is to increase the number of children who remain home for the first two years.”

DR. JOHANNA GREESON, Associate Professor

Single Biggest Takeaway

Working with recently reintegrated families is a complex challenge that requires organizations providing care to view them holistically. When families have access to basic needs, interventions can identify and address other domains that improve family functioning, leading to long-term stability. With adequate material, social, and emotional supports, families can successfully raise children in their homes and communities.