Child Outcomes & Volunteer Effectiveness: Selection Bias Study by Texas CASA

CHILD OUTCOMES AND VOLUNTEER EFFECTIVENESS

Selection Bias Study

Final Report | March 2017

The University of Texas at Austin Lyndon B. Johnson School of Public Aï¬&#x20AC;airs

Child Outcomes and Volunteer Effectiveness Study Selection Bias Study: Final Report

Deliverable 2: Selection Bias Analysis Report Original submission date: December 12, 2016 Revised submission date: March 20, 2017

Submitted by: Cynthia Osborne, Ph.D. Director, Child and Family Research Partnership

Child & Family Research Partnership childandfamilyresearch.org • info@childandfamilyresearch.org • 512.471.3536 • Toll Free 855.471.CFRP (2377) The University of Texas at Austin • The LBJ School of Public Affairs • P.O. Box Y, Austin, Texas 78713

childandfamilyresearch.org

Authors Cynthia Osborne, Ph.D. Director, Child and Family Research Partnership Associate Professor Lyndon B. Johnson School of Public Affairs The University of Texas at Austin Jennifer Lawson, Ph.D., MSSW Postdoctoral Fellow Child and Family Research Partnership

Data Support Holly Sexton, MA Senior Research Associate Child and Family Research Partnership Ruy Manrique-Betanzos, MPAff Research Associate Child and Family Research Partnership Pablo Varas Valenzuela, MSc Research Associate Child and Family Research Partnership

Research Support Sydney Briggs, MPAff Research Associate Child and Family Research Partnership Anna Lipton Galbraith, MPAff Senior Research Associate Child and Family Research Partnership David Mitnick Graduate Research Assistant Child and Family Research Partnership Erika Parks Graduate Research Assistant Child and Family Research Partnership

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

LIST OF FIGURES Figure 1: Summary of Characteristics Associated with CASA Appointment .................................................5 Figure 2: Percent of Children in Sample Appointed to Receive a CASA .....................................................15 Figure 3: Distribution of Child Race/Ethnicity by CASA Appointment........................................................17 Figure 4: Percent of Children with a CASA by Age Grouping .....................................................................18 Figure 5: Percent of Children with a CASA by Number of Siblings in Care .................................................19 Figure 6: Percent of Children with a CASA by Caregiver Risk Factors Contributing to the Removal ...........20 Figure 7: Percent of Children with a CASA by Presence of Domestic/Family Violence ...............................20 Figure 8: Percent of Children with a CASA by Number of Prior CPS Investigations ....................................21 Figure 9: Percent of Children with a CASA by Initial Placement type.........................................................22 Figure 10: Percent of Children with a CASA by Sum of Maltreatment Types (Removal Reasons)...............23 Figure 11: Odds of CASA Appointment for Child Demographic Factors.....................................................24 Figure 12: Odds of CASA Appointment for Family and Caregiver Factors..................................................25 Figure 13: Odds of CASA Appointment for Case-Level Factors..................................................................26 Figure 14: Summary of Characteristics Associated with CASA Appointment .............................................27

LIST OF TABLES Table 1: Children Included/Excluded from Analytic Sample......................................................................13 Table 2: Children in Sample Served by CASA by DFPS Region ...................................................................16 Table 3: Children in Sample Served by CASA in Texas' Five Largest CASA Programs ..................................16 Table 4: CASA Programs Included, Counties Covered, and Children Served ..............................................30 Table 5: List of Programs with No Roster or Unusable Roster, and Counties Covered ...............................33 Table 6: List of Counties Not Served by a CASA Program During Fiscal Years 2013-2014...........................33 Table 7: Frequency and Distribution of Individual Caregiver Risk Factors .................................................35 Table 8: Frequency and Distribution of the Number of Caregiver Risk Factors..........................................35 Table 9: Frequency and Distribution of Individual Maltreatment Types....................................................36 Table 10: Frequency and Distribution of the Number of Maltreatment Types ..........................................36 Table 11: Logistic Regression Findings .....................................................................................................37

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

EXECUTIVE SUMMARY Texas Court Appointed Special Advocates for Children (Texas CASA) contracted with Dr. Cynthia Osborne and the Child and Family Research Partnership (CFRP) at the Lyndon B. Johnson School of Public Affairs at the University of Texas at Austin to design a comprehensive study to evaluate the effectiveness of the Court Appointed Special Advocates (CASA) services provided in Texas. Court Appointed Special Advocates (CASA) is a national network of nonprofit organizations providing trained volunteer advocates to represent the interests of children who enter substitute care. Existing research on the effectiveness of CASA is scant and does not provide strong empirical evidence on whether CASA services are associated with improved safety, permanency, and wellbeing outcomes among the children they serve. To address this gap in the research, and to inform CASAâ&#x20AC;&#x2122;s program and service improvement efforts at the state and local levels, CFRP designed a four-phase evaluation project called the Child Outcomes and Volunteer Effectiveness (COVE) study. The broad goals of the COVE evaluation are to determine the extent to which CASA services improve the permanency, safety, and well-being outcomes of children in the custody of the state, and to identify the factors that enhance or limit the effective implementation of CASA services. This report details the activities and findings of the first phase of the COVE project: the Selection Bias Study. Children are often not randomly selected to receive a CASA volunteer advocate. Prior research and anecdotal information indicate that CASAs are typically assigned to the most complex or serious cases when the demand for CASA services exceeds the supply of volunteers. When the baseline characteristics of cases or children assigned a CASA volunteer differ from those not assigned a CASA volunteer, it is not possible to determine whether any observed differences in outcomes are due to pre-existing differences, or whether they are due to the CASA intervention itself. As a first step toward a rigorous analysis of outcomes associated with CASA services, the Selection Bias Study addresses this primary research aim: What are the child-, family-, and case-level differences between children who are appointed to receive a CASA volunteer advocate and children who are not? To answer this question, CFRP examined a cohort of children who entered substitute care in Texas in fiscal years 2013 and 2014 and compared the children who were appointed to receive a CASA volunteer advocate with the children who were not. Key findings indicate that, consistent with prior research, there are child-, family-, and case-level differences between children who are selected to receive CASA services and those who are not, and these differences generally indicate that CASA-appointed cases involve a greater degree of severity or complexity. Among the findings suggesting that more complex cases are more likely to get a CASA, we find that CASA is significantly more likely to be appointed to children in larger sibling groups, children who Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

experience more types of maltreatment leading to their removal, and older children. The presence of more caregiver risk factors, more prior CPS investigations, and indications of past or current domestic violence in the home are also associated with higher odds of CASA appointment. We also find that children who are placed in a relativeâ&#x20AC;&#x2122;s home as their initial placement upon entering care are less likely to be appointed to CASA compared to children placed in foster homes or other nonrelative initial placement types. In addition, the supply of CASA volunteer advocates relative to demand matters. Children who enter care in communities in which CASA is able to provide services to a high proportion of the children in substitute care are more likely to be appointed a CASA volunteer advocate. These findings are summarized in Figure 1. Figure 1: Summary of Characteristics Associated with CASA Appointment

Lower Odds of CASA Appointment Initial placement with relatives

Higher Odds of CASA Appointment Child age above 2 Having siblings in care Past or current DV indicated Three or more caregiver risk factors Prior CPS investigations, regardless of outcome Three or more maltreatment types leading to removal High percentage of local children in substitute care served by CASA

Understanding the factors associated with CASA appointment among children in substitute care in Texas provides a strong groundwork for conducting the next phase of the COVE study, comparing the outcomes of children who were appointed a CASA with those who were not, while accounting for the differences between these groups. The insight gained from the Selection Bias Study is a crucial step toward being able to identify the unique impact that CASA services have on the safety, permanency, and well-being children in substitute care in Texas.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Introduction and Purpose Texas Court Appointed Special Advocates for Children (Texas CASA) contracted with Dr. Cynthia Osborne and the Child and Family Research Partnership (CFRP) at the Lyndon B. Johnson School of Public Affairs at the University of Texas at Austin to design a comprehensive study to evaluate the effectiveness of the Court Appointed Special Advocates (CASA) services provided in Texas. The National CASA Association is a network of nonprofit organizations across the country providing trained volunteer advocates to represent the interests of children in substitute care, and Texas CASA is the state-level organization supporting the independent local CASA programs operating throughout Texas. The broad goals of the multi-phase Child Outcomes and Volunteer Effectiveness (COVE) evaluation are to determine the extent to which CASA services improve the permanency, safety, and well-being of children in substitute care, and to identify the factors that enhance or limit the effective implementation of CASA services. Through the COVE evaluation, we aim to learn not only whether CASA services work, but also why they work (or why not), for whom they work best, and what environmental or community factors may influence the delivery and effectiveness of these services. For the COVE evaluation, CFRP is conducting four separate, but related, studies to assess the effectiveness of CASA services statewide. Each study will enhance and support the other pieces of the COVE evaluation, but each stands alone as a separate project with distinct research aims. 1 The first study, and the subject of this report, is the Selection Bias Study, which began in the fall of 2014 and examines the baseline child-, family-, and case-level differences between children in substitute care in Texas who are appointed by a judge to receive a CASA volunteer advocate and those who are not. The Selection Bias Study is an important first step in the process of examining the effectiveness of CASA services because prior research has identified selection bias (which means the assignment of individuals to receive an intervention based on non-random characteristics) as a pervasive factor inhibiting the ability of researchers to measure CASAâ&#x20AC;&#x2122;s effectiveness at improving child outcomes. In this study, we aim to systematically identify the differences between children who are appointed a CASA and those who are not, in order to provide an understanding of the sources of potential selection bias specific to the Texas child welfare population. Specific approaches to providing CASA services can vary by state and locality, therefore the sources of selection bias may be different from state to state, and even across different jurisdictions within a state. Examining the dimensions of selection bias within the same population in which outcomes will be examined is important given the variation in the selection criteria for CASA volunteer advocates. This study describes the unique features of children in Texas who are appointed a CASA volunteer advocate and how they differ from children in Texas who only receive conventional child welfare services, laying the groundwork for a rigorous analysis of child and case outcomes within this population. Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Background and Motivation TEXAS COURT APPOINTED SPECIAL ADVOCATES (TEXAS CASA) Texas CASA, Inc. is a membership organization comprised of 72 a independent local CASA affiliate programs across the state. The first CASA program in Texas was established in 1980 in Dallas, and Texas CASA was founded in 1989 to support CASA affiliate programs in effectively advocating for abused and neglected children who have been placed in the custody of the state (substitute care). Specially trained community volunteers called CASA volunteer advocates (CASAs) are appointed by a judge to represent the child and serve as an advocate for the child’s needs and interests during the child’s time in substitute care. Currently, there are approximately 9,100 CASA volunteer advocates working across the 72 local programs in Texas, serving some 28,000 children in state care. 2 Children in the custody of the state face a wide variety of challenges and may have experienced many types of maltreatment and other adverse childhood experiences prior to their entry into care. CASA volunteer advocates serve as a voice for these children in court, with the goal of promoting better outcomes for children during and after their time in care. CASA volunteer advocates perform a unique role in child welfare cases by serving as the “eyes and ears” of judges, bringing a depth of knowledge about the child and his or her needs that child welfare caseworkers and attorneys may not have the time or resources to develop. Models of CASA representation can vary by local programs across Texas. In some areas of Texas, CASA volunteer advocates act as a “friend of the court” in an independent advisor capacity, whereas in other areas, CASA volunteer advocates serve as the child’s Guardian Ad Litem, an officer of the court with the designated function of making recommendations to the judge to promote the best interests of the child. In either of these capacities, all CASA volunteer advocates engage in a number of activities to meet the needs of the children they represent. Core activities of CASA volunteers include meeting with the child regularly to build a relationship and assess his or her needs; communicating with the parents, foster parents, service providers, caseworkers, attorneys, and other parties on the case; gathering information about the case; assisting in the family-finding process to identify relatives or other natural supports for the child; monitoring the service needs of the child and facilitating the provision of appropriate services; locating and monitoring placements; and making recommendations to the judge that reflect the child’s interests. CASA volunteer advocates aim to secure a safe and permanent home for each child, remain a strong and consistent source of support for the child, and ensure the needs of the child are met throughout the child’s time in care. a There are currently 72 local CASA affiliate programs in Texas, but there were only 71 affiliate programs at the time the COVE study began.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

In preparation for the COVE evaluation, leaders from Texas CASA and from local CASA program affiliates across the state contributed to the development of a logic model to inform the evaluation plan. This collaboratively created logic model (included in Appendix A) defines the inputs, activities, and outputs of CASA volunteer advocates, and links those to the outcomes that CASAs aim to positively influence through their representation.

SELECTION BIAS IS A BARRIER TO EVALUATING OUTCOMES Promoting positive child outcomes, especially ensuring that every child exits substitute care to a safe, stable, and permanent home, is the overarching goal of CASA. Prior researchers have attempted to examine the extent to which CASA is effective at achieving improved outcomes for the children they serve, however the existing body of research on CASA effectiveness is small and inconclusive due to widespread limitations in study design, most notably not accounting for selection bias in the CASA appointment process. 3 Prior research has shown that, when children who received CASA services and those who did not are compared, there are systematic differences between these groups on demographic, family, and case characteristics. 4 In research terms, this phenomenon is called selection bias. When children are selected to receive the CASA intervention based on non-random characteristics, such as more complicated family circumstances, higher risk, or more severe maltreatment, they are not comparable to children who were not selected to receive a CASA. When selection bias is present, the differential characteristics of the cases and children who receive the intervention may be responsible for differences in their outcomes, rather than the intervention itself. Aside from the widespread selection bias that calls into question the validity of the findings from prior research on CASAâ&#x20AC;&#x2122;s impact, the findings themselves are often contradictory or inconsistent. For example, several studies have examined whether CASA cases attain closure faster than cases without CASA, and whereas some studies find that CASA cases are of shorter duration, 5,6,7 other studies find that CASA cases remain in the system longer, 8 and still others find no differences in duration. 9,10,11,12 Findings are also equivocal in relation to the number of placements children with a CASA experience while in substitute care compared to children without a CASA. Some studies find that, in the aggregate, children with a CASA have fewer placements, 13,14 other studies find that children with a CASA have more placements, 15,16 and others find that the two groups have equivalent numbers of placements. 17,18,19,20,21 Even when conclusions from prior research are not contradictory, such as the consistent finding that children with CASA representation and their families receive more services than children without CASA representation, 22,23,24,25 the selection bias problem still limits confidence in the findings. If CASA cases are more difficult or complicated than non-CASA cases at the time a CASA is appointed, it is impossible to know whether the children or families would have received more Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

services even in the absence of a CASA. In other words, we cannot know whether CASA’s advocacy efforts caused more services to be provided, or if more services would have been provided anyway because more serious cases warrant additional services. CFRP solicited input from CASA staff and volunteers throughout Texas and found that, consistent with the findings from prior studies, the stakeholders perceive that judges appoint CASA volunteers to more difficult cases when a choice is necessary. Based on conversations with several judges and CASA executive directors, the process of selecting which children receive a CASA varies somewhat across the state, though the decision of when to appoint a CASA ultimately lies within the legal purview of the judge overseeing the case. Often, judges make the sole determination of which children they believe would benefit from the appointment of a CASA volunteer. In other instances, judges may work jointly with local CASA programs to “triage” which cases the judge will appoint a CASA volunteer when the local supply is not sufficient for all children to have one. In some areas, the local CASA agency may choose which children they will represent and communicate their selections to the judge for an appointment order. Whatever process is involved in selection, if a CASA does tend to be appointed to the cases with the highest need, this non-random selection may be beneficial for those children who receive extra support and advocacy, but it needs to be accounted for when comparing case and child outcomes. When selection bias is present in outcomes research, it presents a serious threat to attaining valid findings. If selection bias is not accounted for in the comparison of outcomes, then evaluation results may suggest that a program is not effective when it actually is, or vice versa. In the CASA context, a simple comparison of outcomes (without accounting for selection bias) may actually conclude that CASA-appointed cases have worse outcomes than cases not appointed a CASA volunteer, because CASA cases may be more difficult from the outset. For example, cases representing more severe maltreatment circumstances may require a longer timeframe to achieve permanency, because parents may have to complete more services to address the problems that led to the removal, or because adoptive placements take extra time to locate and prepare when reunification is not possible. If a CASA is appointed to represent cases involving more serious maltreatment, and these cases take longer to reach a final permanency outcome than less severe cases, it may appear that CASA is associated with worse outcomes, even though the longer case duration may be related to the severity of the case, not the effectiveness of CASA services. This potential scenario demonstrates why research that accounts for selection bias in the CASA appointment process is so important for determining what unique benefits CASA may impart to the children they are appointed to serve.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

THE CURRENT STUDY In the COVE project, we aim to avoid invalid conclusions by using the first phase of the evaluation to identify the areas in which selection bias is present, prior to evaluating outcomes. The proportion of children in Texas appointed a CASA volunteer varies by jurisdiction depending on the availability of volunteer advocates relative to the number of children in substitute care in that area. At present, there are 11 local CASA programs in Texas that are able to serve over 99 percent of the children in substitute care in their communities, and the number of programs able to fully meet local demand is continually growing. In most communities, however, there are not enough CASA volunteers for every child to receive CASA representation. Our strategy of carefully identifying the dimensions on which selection differences are present allows us to account for these differences when assessing child outcomes in subsequent stages of the COVE evaluation. Toward this end, the Selection Bias Study addresses the following research aim: What are the baseline child-, family-, and case-level differences between children who are appointed to receive a CASA volunteer advocate and children who are not? As discussed above, the strength and validity of the existing literature on CASA programs is hindered by selection bias, which has resulted in a limited evidence base supporting CASAâ&#x20AC;&#x2122;s effectiveness at improving child outcomes. The present evaluation enriches the existing evidence on CASA effectiveness by examining as many potential differences as possible, at the time of removal, between cases appointed and cases not appointed a CASA volunteer advocate. Identifying the systematic differences in the characteristics of cases appointed versus not appointed a CASA allows for these characteristics to be accounted for in subsequent analyses measuring differences in child and case outcomes. In addition, by identifying the multidimensional ways in which children with a CASA differ from their counterparts who only receive conventional child welfare services, Texas CASA gains a deeper understanding of their client system to support their ongoing training and program improvement efforts.

Data Sources and Analytic Method DATA SOURCES CFRP used two data sources for the selection bias analysis. These sources and the methodology that we used to create the study sample are outlined below. Information Management Protecting Adults and Children in Texas (IMPACT) Child Protective Services (CPS) caseworkers collect demographic and case information on every child welfare case that is opened in the state. Texas CASA and CFRP collaborated with the Texas Department of Family and Protective Services (DFPS, the parent agency of CPS) to securely access

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

the data records of all CPS cases in which a child entered substitute care between September 1, 2012 and August 31, 2014. The variables captured in IMPACT and included in the data records represent the demographic and case factors that were analyzed in the current Selection Bias Study. CASA Program Affiliate Case Management Data Because CPS case records in IMPACT do not consistently record which children in state care receive CASA representation, Texas CASA requested that each local CASA program affiliate provide a roster of all children they represented during the study timeframe (September 2012 through August 2014) for use in the research evaluation. In response to this request, 68 of the 71 local CASA programs that were in operation at the time securely provided complete rosters containing the names, dates of birth, and court cause numbers for each child they served during the study entry period. Two programs declined to supply a roster, and one program provided a roster that was of insufficient quality to use in the study, however all of these programs represent counties with relatively small numbers of children in care (see Table 1 below). The 68 usable CASA program rosters comprise a data set of children served by CASA during the study timeframe, which allowed the research team to conduct a linking process to match the children in the CASA data set with the children in the IMPACT data set. b

MATCHING METHODOLOGY To create a data set that allows us to compare the characteristics of children who were appointed a CASA to children who were not, CRFP merged the full IMPACT data file with the roster of children served by CASA programs using the probabilistic matching function in a statistical software package (Stata), supplemented by intensive hand-matching and quality assurance review, resulting in over 99 percent of the children on the CASA rosters being successfully matched to their IMPACT records in the final analytic data set. The result of this linking process is that the final data set contains all IMPACT case record data, and includes an indicator of which children in substitute care during the study window were appointed to receive a CASA volunteer advocate.

ANALYTIC SAMPLE The final sample includes children who entered substitute care between September 1, 2012 and August 31, 2014. This includes children who entered care during this time period in jurisdictions covered by one of the 68 CASA programs in Texas that provided a usable roster of the children they served during the study time frame. c b

See Appendix B for detailed information about the programs and counties that were included in the study. For children who had multiple entries to care during the study window, we used the entry that aligned with the date that CASA was appointed to the case for children who received CASA, and the last entry into care for children who did not. For children who were missing a date of removal in IMPACT, the first legal status date was used in place of removal date. Duplicate entries were dropped if the child had the same person ID number (a unique person-level identifier from IMPACT), date of birth, and date of removal. c

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

We also applied some exclusion criteria to maximize the validity of the study findings. Three local CASA programs did not provide a usable roster of children served. Two of these programs declined to provide any roster, and one provided a roster with invalid data that could not be retained for inclusion because it prevented the research team from being able to match the children to their IMPACT records. All children in the sample who were represented by one of these three programs were removed from the analytic sample, as were children from the comparison (no-CASA) group who would have been represented by one of these programs, based on their county of removal, had they been selected to receive a CASA volunteer. The sample also excludes all children who were removed in one of the 47 counties in Texas that were not covered by a local CASA affiliate program during the study timeframe. d In addition, as we began to analyze the sample data, we discovered that, out of the 313 unique courts that are represented in the study sample, a single court in a major urban area is an extreme outlier in regard to patterns of CASA appointment, particularly in regard to race and ethnicity. In this one court, the distribution of CASA appointment varies substantially based on the race/ethnicity of children whose cases are overseen by that court. For example, Hispanic children are significantly underrepresented and White children are significantly overrepresented among children appointed a CASA in this court, and these differences are far greater than what was observed among the rest of the courts in the analysis. Because the distribution of CASA by race/ethnicity is so extreme in this court compared to the rest of the state, and because the court is in a major urban population center with a large number of children entering care, its inclusion was dramatically changing the findings of the entire study, thereby painting a skewed picture of the true selection characteristics predicting CASA appointment in the state as a whole. As a result of the observations related to the CASA assignment patterns in this single court, we decided to exclude the 1,224 children whose cases were overseen in that court from the study, in order to avoid biasing the aggregate findings for the rest of state. The exclusion of children who entered care in an area either not covered by a CASA program, covered by a CASA program without a usable roster, or covered by the outlier court resulted in just under 10 percent of the total IMPACT sample being removed from the analysis, as summarized in Table 1 below.

d A list of the counties excluded from the analysis due to no roster, unusable roster, or no local CASA program is available in Appendix B.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Table 1: Children Included/Excluded from Analytic Sample Included or Excluded from Analytic Sample?

Number of Children in DFPS/IMPACT Sample

Percent of Total IMPACT Sample

Entered Care in a CASA Program Area with a Usable Roster

Included

31,371

90.5%

Entered Care in a CASA Program Area without a Usable Roster

Excluded

860

2.5%

Entered Care in an Area Not Covered by a CASA Program

Excluded

1,220

3.5%

Entered Care in the Area Overseen by the Outlier Court

Excluded

1,224

3.5%

The final analytic sample represents a total of 31,371 included children, of whom 56.1 percent (17,603) were served by CASA.

MEASURES Outcome Measure The primary outcome measure is a dichotomous variable indicating whether each child was “selected” or appointed to receive representation by a CASA volunteer advocate.

Predictor Measures CFRP chose a wide range of child, caregiver, and case characteristics (i.e., predictor variables) from the IMPACT data set to examine for selection bias. We only selected factors that were documented during the investigation stage of the case (prior to the opening of the substitute care case) or within 30 days of the removal date. The research questions for this study pertain to factors that may influence the appointment of a CASA, and CASA volunteers are typically appointed early in a removal case. By restricting our variables to characteristics that were present prior to or near the time of the CASA appointment, we ensured that we were not inadvertently capturing factors that may have been the result of CASA advocacy efforts after the volunteer advocate started working on the case. Child Characteristics For this analysis, we examined child characteristics including race/ethnicity, gender, and age at the time of removal. Additional child characteristics in IMPACT that provide information on the child’s well-being would likely inform the analysis, but we excluded these measures because of poor data quality in the IMPACT records. Specifically, the IMPACT data file includes 66 indicators for a variety of emotional, physical, and mental health issues that were suspected or diagnosed at the time of the removal. Caseworkers may select all that apply from these indicators. We grouped these indicators into four composite categories for use in the analysis: drug/alcohol Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

issues, emotional/behavioral problems, developmental/learning problems, and medical/physical problems. However, when examining the frequencies of these indicators and groupings in the final analytic data set, we discovered that the numbers of children who had these indicators were far less than would be expected based on external data sources. For example, only one percent of children in the sample are indicated as Limited English Proficient, which is dramatically lower than publicly available information indicating that over 17 percent of schoolage children in Texas during the 2013-2014 school year were in Bilingual/ESL programs. 26 Family and Caregiver Characteristics We also examined family-level characteristics, including whether the mother was ever in the custody of the state, the mother’s age at removal, whether anyone in the home had been a perpetrator or victim of domestic violence, and the number of each child’s siblings also in substitute care. In addition, we examined caregiver risk factors that were identified as contributing to the removal: alcohol abuse, drug abuse, emotional disturbance, inadequate housing, whether the caregiver is incarcerated, whether the caregiver is unable to cope, and whether the caregiver is deceased. Because caseworkers can select all that apply, we used a sum of the total number of possible caregiver risk factors that were indicated by the caseworker. Case Characteristics We examined case-level characteristics, including the number of prior CPS investigations involving the child, whether the child’s first placement upon removal was with relatives (compared to any other type of out-of-home placement), and the number of maltreatment types that resulted in the removal. To document the reason(s) that a child was removed, CPS caseworkers select all that apply from eight maltreatment types reflected in Texas child welfare statute: neglectful supervision, physical neglect, medical neglect, physical abuse, sexual abuse, emotional abuse, abandonment, and refusal to accept parental responsibility. There are also a handful of rarely used non-maltreatment removal circumstances (such as lack of needed medical, mental health, or juvenile justice services) that were grouped together into an “other” category for the analysis, resulting in a total of nine possible reasons for removal that each child could have. Because caseworkers can select multiple maltreatment types that led to the removal, we used a sum of the total number of the nine possible removal reasons selected by the caseworker.

ANALYTIC STRATEGY To understand the systematic differences that are present at or near the time of removal, we compared the group of children who were appointed a CASA volunteer advocate to the group of children who were not. Descriptive analyses of the case and child characteristics for both cohorts provide details on the differences between children with a CASA volunteer advocate and children without a CASA volunteer advocate. We employed chi-square tests to determine statistically significant observed differences in the child, case, and caregiver characteristics between the two Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

groups. Finally, we used logistic regression (with mixed effects to account for data clustering at the court/judge level) to determine which factors are associated with higher or lower odds (a measure of likelihood) of CASA appointment while controlling for all factors. â&#x20AC;&#x153;Controlling forâ&#x20AC;? all of the variables in an analysis is a statistical technique that allows us to hold each variable constant to isolate the association that each other variable has with the outcome (in this case, whether a CASA was appointed).

Findings Below are the descriptive summaries and results of statistical tests to examine differences in individual child, family, and case characteristics between children who are appointed a CASA volunteer advocate and children who are not. First, we present findings reflecting observed differences between the CASA and no-CASA groups on child, family, and case characteristics, without controlling for any other factors. The findings of the multivariate portion of the analysis, which used logistic regression to examine which factors are associated with higher or lower odds of CASA appointment while controlling for all variables, are presented subsequently.

DESCRIPTIVE AND GROUP COMPARISON FINDINGS The source of the data used in this analysis is linked Texas DFPS IMPACT and CASA Program Case Management records reflecting children who entered care between September 1, 2012 and August 31, 2014 and who met study inclusion criteria (N=31,371 children). Percent totals within data figures may not equal exactly 100 due to rounding of decimal places. Sample Description Of the 31,371 children in the final sample, 56 percent (17,603) were appointed to CASA. Figure 2: Percent of Children in Sample Appointed to Receive a CASA

No CASA 44%

Preliminary Selection Bias Report

CASA 56%

March 20, 2017

childandfamilyresearch.org

The 17,603 children in the sample who were appointed a CASA were served in every region of the state, but were clustered in regions with large urban populations, as reflected in Table 2. e Table 2: Children in Sample Served by CASA by DFPS Region f Children served by CASA Region 1 – Region 2 – Region 3 – Region 4 – Region 5 – Region 6 – Region 7 – Region 8 – Region 9 – Region 10 – Region 11 –

Lubbock area Abilene area DFW area Tyler area Beaumont area Houston area Austin area San Antonio area Midland area El Paso area Edinburg area

Total

n 613 498 4,393 1,810 742 2,934 2,278 2,352 673 370 940

% 3% 3% 25% 10% 4% 17% 13% 13% 4% 2% 5%

17,603

100%

Children in the analytic sample who were served by the five largest CASA programs in Texas account for one-third of all children in the CASA cohort for this study. Table 3: Children in Sample Served by CASA in Texas' Five Largest CASA Programs g Children served by CASA CASA of Travis County Dallas CASA CASA of Tarrant County Child Advocates Inc. (Harris Co.) Child Advocates of San Antonio All other CASA programs

n 997 1,023 733 1,812 1,476 11,562

% 6% 6% 4% 10% 8% 66%

Total

17,603

100%

A map of DFPS regions is included in Appendix C. Number of children served by each program who were included in the study sample. These figures may differ from local program records due to the inclusion and exclusion criteria applied in creating the study sample. g Number of children served by each program who were included in the study sample. These figures may differ from local program records due to the inclusion and exclusion criteria applied in creating the study sample. f

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Child Demographics We examined the observed differences in race/ethnicity, age, and gender between children who were appointed a CASA and children who only received typical child welfare services without a CASA. The figures below demonstrate that, at baseline, there are differences in the demographic characteristics of children who are appointed a CASA and children who are not. These differences are statistically significant for age and race, but not for gender. Looking at the observed differences in CASA appointment by child race/ethnicity, without “controlling for” any other variables, White children are overrepresented in the CASA group compared to their African American and Hispanic counterparts. Figure 3 shows that, of the children who are appointed a CASA, 38 percent are White, and 39 percent are Hispanic. By contrast, of the children who are not appointed a CASA, half are Hispanic, and only one-quarter are White. However, when all child, family, and case factors are “controlled for” in the multivariate analysis (discussed in a subsequent section), the racial and ethnic differences in CASA appointment disappear, and all children are equally likely to be appointed a CASA, irrespective of their race or ethnicity. This finding suggests that the baseline differences in CASA appointment by race or ethnicity are actually explained by other factors, such as whether children are placed with relatives upon entering care, the number of siblings in the family who are also in state care, and/or the number of maltreatment types leading to removal. Figure 3: Distribution of Child Race/Ethnicity by CASA Appointment 100%

80%

39%

50%

60%

40%

38%

25%

20%

17%

20%

CASA

No CASA

African American

White

Hispanic

Other

N=31,371; Significance test: X2=645.1; p<0.001

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Older children are more likely to be appointed a CASA than children under 2 years old (Figure 4). Of children under age 2, 51 percent are appointed a CASA, whereas among the older age categories, the proportion of children appointed a CASA is higher, reaching 60 percent in the 5 to 12 age group. Cases involving older children can present more challenging circumstances compared to cases involving infants, especially regarding placement. Older children are often more difficult to place, are more likely to experience placement disruptions, and may have more intensive emotional or behavioral needs compared to infants. 27,28 Figure 4: Percent of Children with a CASA by Age Grouping 13 to 17

56%

5 to 12

44%

60%

2 to 4

40%

56%

Under 2

44%

51% 0%

20%

49% 40%

CASA

60%

80%

100%

No CASA

N=31,371; Significance test: X2=163.4; p<0.001

Caregiver and Family Characteristics In addition to demographic differences between children who are appointed a CASA and those who are not, we also find baseline differences in characteristics related to the caregiver and family. As shown in the figures below, the observed differences are statistically significant regarding the number of siblings in care, number of caregiver risk factors, and whether past or current domestic violence was indicated. No statistically significant differences between CASA and no CASA groups are found regarding whether the mother was ever in DFPS custody, nor for maternal age at the time of the childâ&#x20AC;&#x2122;s removal. The observed comparison between the CASA and no CASA groups shows only modest differences related to the number of siblings in care (Figure 5). However, these differences become more pronounced when all variables are controlled for in the subsequent multivariate analyses. In the observed or bivariate comparison, between 56 and 60 percent of children with at least one sibling in care are appointed a CASA, compared to just over half of children with no

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

siblings. Cases with more siblings may present special challenges for caseworkers and other child welfare professionals. Sibling groups, especially larger sibling groups, are often difficult to place together. 29 This can make visitation plans to maintain sibling relationships more complicated. Cases with more siblings may also require more attention to ensure that each childâ&#x20AC;&#x2122;s needs are carefully assessed and that appropriate individualized services are provided. The special challenges of cases involving large sibling groups may lead judges and/or CASA administrators to select these cases for additional advocacy and support. Figure 5: Percent of Children with a CASA by Number of Siblings in Care

3 or more siblings in care

44%

56%

2 siblings in care

40%

60%

1 sibling in care

43%

57%

No siblings in care

49%

51% 0%

20%

CASA

40%

60%

80%

100%

No CASA

N=31,371; Significance test: X2=117.7; p<0.001

As shown in Figure 6 (below), children whose caregivers have more risk factors that contributed to their removal are more likely to be appointed a CASA. Over 60 percent of children whose caregivers had three or more risk factors contributing to the removal were appointed a CASA, compared to just over half of children whose caregivers had no risk factors indicated. Because CPS is mandated to provide services to address risks in the home that led to a childâ&#x20AC;&#x2122;s removal, cases involving more caregiver risk factors may require additional assessment and service provision compared to cases involving fewer risks.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Figure 6: Percent of Children with a CASA by Caregiver Risk Factors Contributing to the Removalh

3 or more caregiver risk factors

62%

38%

59%

41%

56%

44%

2 caregiver risk factors 1 caregiver risk factor No caregiver risk factors indicated

48%

52% 0%

20%

CASA

40%

60%

80%

100%

No CASA

N=31,371; Significance test: X2=117.5; p<0.001

CASA is slightly more likely to be appointed to children from families in which past or current domestic violence has been indicated by the caseworker. This difference is modest but statistically significant. Of cases in which domestic violence was indicated, 57 percent were appointed a CASA, compared to 55 percent appointed a CASA among cases with no domestic violence indicated (Figure 7). Domestic violence is a problem that frequently co-occurs with child maltreatment, and can require an additional set of services for families to complete as part of their treatment plans, which may add to the complexity of a CPS case. 30, 31 Figure 7: Percent of Children with a CASA by Presence of Domestic/Family Violence

Past or current DV indicated

57%

Past or current DV not indicated

43%

55% 0%

20%

CASA

45% 40%

60%

80%

100%

No CASA

N=31,371; Significance test: X2=17.0; p<0.001

The list of all caregiver risk factors that can be selected by the CPS caseworker is in Appendix D.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Case Characteristics We also examined case-level factors related to placement types, history of prior involvement with CPS, and reasons for the childâ&#x20AC;&#x2122;s removal, to determine whether these characteristics are significantly associated with the CASA selection process. As with the child and family variables, there are indicators that more difficult or complicated cases are associated with CASA appointment. Figure 8: Percent of Children with a CASA by Number of Prior CPS Investigations

3 or more prior CPS investigations

37%

63%

2 prior CPS investigations

58%

42%

1 prior CPS investigation

59%

41%

No prior CPS investigations

45%

55% 0%

CASA

20%

40%

60%

80%

100%

No CASA

N=31,371; Significance test: X2=95.0; p<0.001

As shown in Figure 8, above, children who have been involved in previous CPS investigations (cases that occurred prior to the case that resulted in the removal) are more likely to be appointed a CASA. This difference is most pronounced among children involved in three or more prior CPS cases, of whom, 63 percent are appointed a CASA, compared to children with no prior CPS cases, of whom only 55 percent are appointed a CASA. Prior CPS history can be viewed as a complicating factor in child welfare cases. Children who are re-reported to CPS after an initial report may be experiencing repeat traumatic victimization, and they are more likely to have multiple episodes of substitute care, both of which can disrupt family relationships and lead to greater emotional and behavioral needs to be addressed. 32 We also find that a slightly smaller proportion of children who are placed with relatives as their initial placement type are appointed a CASA (Figure 9, below). Of children who are placed in a non-relative foster home or other placement type, 57 percent are appointed a CASA, compared to 54 percent of children in relative placements. This modest difference, which is statistically

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

significant, becomes more pronounced in the subsequent stage of the analysis when all variables are included and controlled for. Having family members who are willing and able to accept placement of a child immediately upon removal may serve as a signal to the court that the child has an existing familial support network. Therefore, the lower likelihood of CASA appointment for children in kinship placements may reflect the triaging of CASA services to prioritize children who have less family support upon removal. Figure 9: Percent of Children with a CASA by Initial Placement type

Foster home or other placement

57%

43%

Relative/kinship placement

55%

46%

20%

CASA

40%

60%

80%

100%

No CASA

N=31,371; Significance test: X2=18.6; p<0.001

Children with more maltreatment types contributing to their removal are more likely to be appointed a CASA than children whose removal was based on a single type of maltreatment. As displayed in Figure 10, below, 63 percent of children with three or more removal reasons are appointed a CASA, compared to 55 percent of children with only one removal reason. Reasons for removal correspond to the statutory definitions of abuse and neglect in Texas, and therefore more reasons for removal may signal that a child experienced a more serious set of risks and potential harm in his or her home prior to removal. More reasons for removal may also indicate that the parents will have a more complicated set of services to complete to address all of the factors that contributed to the removal before the child can return home safely. We also examined whether each of the nine individual reasons for removal are associated with CASA appointment, but found no substantial differences in the maltreatment types leading to removal experienced between children in the CASA and no-CASA groups. In other words, the distribution of maltreatment types is substantially similar among children who are appointed a Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

CASA and children who are not. Because the individual types of maltreatment are not meaningfully different between groups, we chose to include the sum of maltreatment types contributing to the removal in the analysis rather than the individual types. Figure 10: Percent of Children with a CASA by Sum of Maltreatment Types (Removal Reasons) i

3 or more removal reasons

63%

2 removal reasons

37%

58%

1 removal reason

42%

55% 0%

20%

CASA

45% 40%

60%

80%

100%

No CASA

N=31,371; Significance test: X2=54.7; p<0.001

FACTORS ASSOCIATED WITH HIGHER OR LOWER ODDS OF CASA APPOINTMENT In the second phase of the analysis, we looked beyond the observed differences between CASA and no CASA groups to examine which factors are associated with higher or lower odds that a given child will be appointed a CASA while statistically controlling for all of the other factors simultaneously. This part of the analysis uses a statistical method called logistic regression, which quantifies the relationship between a binary outcome (in this case, either being appointed a CASA volunteer advocate or not) and multiple factors (predictors) that may be associated with whether that outcome is reached. This procedure calculates how each predictor is associated with the odds of the outcome occurring. Findings tell us both the direction (higher or lower) and strength of each predictorâ&#x20AC;&#x2122;s relationship to the CASA appointment outcome, while controlling for the influence of all the other predictors. In this analysis, we also employed a strategy called mixed-effects modeling, in which we accounted for the fact that CASA selection is clustered at the court level. j Any given judgeâ&#x20AC;&#x2122;s

The list of all removal reasons that can be selected by the CPS caseworker is in Appendix E.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

selection process for determining which children will be appointed a CASA may vary systematically from the selection process of other judges, and accounting for this court-level variation makes the logistic regression findings more reliable and precise. We present the findings below, separately by child, family, and case characteristics. k For each variable, we display how that factor is associated with higher or lower odds of being appointed a CASA. If the odds for any given variable are above 1, the odds of CASA appointment are higher; if they are below 1, the odds of CASA appointment are lower, holding all other characteristics between the groups constant. For the family and case factors, we only present findings for variables that are statistically significant, though all variables were included in the statistical analysis. l For the child-level factors, we chose to also display child race/ethnicity, even though it is not statistically significant, to demonstrate that the observed differences seen previously disappeared once all factors were controlled for in the analysis. Figure 11: Odds of CASA Appointment for Child Demographic Factors 1.8 1.6 1.4 1.2 1 0.8 0.6

Black

Hispanic

Other

(Odds compared to White)

Age 2-4*

Age 5-12* Age 13-17*

(Odds compared to infants)

N=31,198; Note: Odds are reported controlling for child gender, mother ever in DFPS custody, past or current domestic violence indicated, number of siblings in care, motherâ&#x20AC;&#x2122;s age at removal, number of caregiver risk factors, number of prior CPS cases, number of maltreatment types, initial placement with kin, percent of local children in substitute care with CASA, and court ID number. *p<=.05

Because regression analysis cannot include children for whom any data are missing, 172 children were dropped from this phase of the analysis due to missing court identification numbers, and one child was dropped for missing gender, resulting in N=31,198 for these statistical tests. k The full logistic regression findings, with statistical coefficients and other test output, are presented in table format in Appendix F. l If a variable has more than one category, all are shown, even if not every category attained statistical significance. Categories with statistical significance are indicated by *

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

The graph above (Figure 11) shows how each child characteristic is associated with the odds of CASA appointment, while controlling for all other child, family, and case characteristics. With all factors controlled for, there are no significant differences in the odds of CASA appointment by race or ethnicity. Children in all three age groups above age 2 have higher odds of CASA appointment compared to children in the youngest group, with children ages 5 to 12 having 59 percent higher odds of being appointed a CASA. Gender is insignificant as a factor; boys and girls are equally likely to be appointed a CASA. The graph below (Figure 12) displays the factors at the family level that are significantly associated with higher odds of CASA appointment. Children whose caseworkers indicated past or current domestic violence (DV) in the home have approximately 10 percent higher odds of CASA appointment compared to cases with no domestic violence indicated. Further, compared with having no siblings in care, having any number of siblings in care is associated with significantly higher odds of CASA appointment, and having two or more siblings in care more than doubles the odds of receiving a CASA. The number of caregiver risk factors that the caseworker selected as contributing to the removal was associated with 21 percent higher odds of CASA appointment when there were three or more risk factors compared to no risk factors. Neither the mother’s age at removal nor whether the mother was ever in DFPS custody were significantly associated with higher or lower odds of CASA appointment. Figure 12: Odds of CASA Appointment for Family and Caregiver Factors 2.2

2 1.8 1.6 1.4 1.2 1 0.8

DV indicated*

1 sibling* 2 siblings* 3+ siblings* (Odds compared to no siblings)

1 risk factor

2 risk factors

3+ risk factors*

(Odds compared to no risk factors)

N=31,198; Note: Odds are reported controlling for child race/ethnicity, child age, child gender, mother ever in DFPS custody, mother’s age at removal, number of prior CPS cases, number of maltreatment types, initial placement with kin, percent of local children in substitute care with CASA, and court ID number. In this figure, “DV” = domestic violence. *p<=.05

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

The findings shown below in Figure 13 represent the case-level factors examined in the analysis. Having a prior history of CPS involvement (regardless of the outcome of any previous cases) is significantly associated with higher odds of CASA appointment. The difference is most notable for children who have been involved with three or more prior CPS cases, who have 23 percent higher odds of being appointed a CASA than children with no prior CPS cases. Children in kinship placements as their initial placement type have 14 percent lower odds of CASA appointment than children in non-relative initial placement types. Children who enter substitute care may have experienced multiple types of abuse and neglect that contributed to their removal, and caseworkers select all that apply as part of the case record documentation. Children who have two or more maltreatment types resulting in their removal have significantly higher odds of being appointed a CASA than children with only one maltreatment type contributing to their removal. Finally, the supply of local CASA volunteers relative to demand is an important determinant of CASA appointment. For each CASA program area in Texas, we calculated the percentage of children in care in that area who were appointed a CASA. Controlling for all other factors, each 10 percentage point increase in local CASA coverage nearly doubles the odds of a child in that area being selected to receive a CASA. Figure 13: Odds of CASA Appointment for Case-Level Factors 2

1.8

1.6

1.4

1.2

1 1 prior case*

2 prior cases

3+ prior cases*

0.8 (Odds compared to no prior cases)

Initial placement with kin*

0.6

2 maltx types*

3+ maltx types*

(Odds compared to one maltx type)

Percent of children in care with CASA*

N=31,198; Note: Odds are reported controlling for child race/ethnicity, child age, child gender, mother ever in DFPS custody, past or current domestic violence indicated, number of siblings in care, motherâ&#x20AC;&#x2122;s age at removal, number of caregiver risk factors, and court ID number. In this figure, â&#x20AC;&#x153;maltx" = maltreatment. *p<=.05

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

SUMMARY OF FINDINGS The statistically significant factors that are associated with the selection of children appointed to CASA services are summarized below (Figure 14) based on whether each factor is associated with higher or lower odds of CASA being appointed.

Figure 14: Summary of Characteristics Associated with CASA Appointment

Lower Odds of CASA Appointment Initial placement with relatives

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Conclusions and Next Steps These findings indicate that, consistent with prior research from various states and jurisdictions around the country, there are child-, family-, and case-level differences between children who are appointed to CASA services and those who are not, and these differences generally suggest that CASA cases involve a greater degree of severity or complexity compared to their non-CASA counterparts. These differences persist after controlling for all variables simultaneously, suggesting that there is differential selection of children for CASA services based on non-random factors. These findings are based on the aggregation of all children included in the sample from across the state. Though we controlled for court jurisdiction and the proportion of local children served by CASA, the wide variation in child welfare service delivery within local contexts means that the patterns and predictors of CASA appointment may look different within individual communities. The Selection Bias Study provides detailed information on a comprehensive set of factors that are associated with the likelihood of CASA appointment. This study confirms that children in Texas are not selected at random to receive CASA services. Rather, there appear to be systematic differences between children who are appointed a CASA and children who are not, and the circumstances of children appointed a CASA have indicators of greater complexity or difficulty. Without this information, comparing the outcomes of these two groups would likely produce biased findings, given that more serious or complicated cases could be associated with worse outcomes, thereby masking the potential impact of the CASA intervention. The findings from the Selection Bias Study support what prior research has suggested, but with a new level of specificity. For the Selection Bias Study, we used a substantially larger sample size and a more comprehensive set of predictors than any prior CASA study has used. We conducted the analyses using a population sample representing a two-year entry cohort of children entering substitute care in Texas. Because we are using this same sample for the next phase of the COVE evaluation examining child outcomes, we know that findings we obtained regarding the differences between CASA and no-CASA children can be directly applied to the outcomes analysis to facilitate an unbiased, apples-to-apples comparison. Building on this phase of the COVE evaluation, the next step is to engage in a rigorous evaluation of child outcomes to determine whether children represented by CASA have better safety, permanency, and well-being outcomes, while empirically accounting for the presence of selection bias as demonstrated in the current study. We now have a more robust ability to isolate the effect of CASA on child outcomes than has previously been possible, and are poised to better understand the effectiveness of CASA services for children in substitute care.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Appendix A. Texas CASA COVE Logic Model INPUTS

ACTIVITIES

OUTPUTS

OUTCOMES While in Care

-Recruiting volunteer

-Advocate for child’s best interest

Number of … -Children served by CASA volunteer

- Child in safe and nurturing placement (kinship home if possible)

-Final placement type: reunification, adoption, guardianship, or PMC

- Right information to right people at right time: Communicate with all parties involved with the case including gathering and providing necessary information and asking the difficult questions

-Children placed in safe/stable home

-Child in correct grade for age

-Reduced time in substitute care

-Build relationships with involved parties

-Services provided to a child during the case

-Child receives needed services

-Reduced number of placements

-Services provided to the parents during the case

-Child’s mental and medical health needs are met

-Child less likely to experience new allegations of abuse and neglect, have a new substantiated case of abuse or neglect, or return to care

advocates - Initial training for volunteer advocates -Ongoing training for volunteer advocates - Supervision for volunteer advocates by CASA staff - Private and public funding invested in CASA programs - Quality assurance measures by Texas CASA for local CASA programs

-Write court reports and make recommendations - Request hearings and make court appearances -Assess child and family’s needs, strengths, and supports -Identify and refer child and parents to appropriate services -Create and help implement visitation plans for children and family and motivate parents to be involved in child’s life and needed services -Identify and secure appropriate placement such as conducting search for kinship resources

Final Case Outcomes

-Placements for child during a case -Visits child had with biological parent during the case -Reports volunteer submits and number of recommendations volunteer makes to the court during the case -Hours volunteer advocates spend communicating with other parties and service providers on a case

-Visitation occurs between child and family (parents & siblings) -Child does not change schools -CASA recommendations that are additional to or in conflict with those of CPS are accepted by judge

-Explain the process to the child -Act as stable adult in child’s life

-Number of contacts volunteer makes with other parties involved in the case

-Document everything

Preliminary Selection Bias Report

March 20, 2017

After Leaving Care -Child less likely to re-enter substitute care

childandfamilyresearch.org

Appendix B: Tables of Counties and Programs by Study Inclusion Table 4: CASA Programs Included, Counties Covered, and Children Served m

Program Name Amarillo Area CASA Big Country CASA Bluebonnet CASA Brush Country CASA CASA 69 CASA for Hunt County CASA for KIDS CASA for Kids of East Texas CASA for Kids of South Central Texas CASA for the Cross Timbers Area CASA for the Highland Lakes Area CASA in the Heart of Texas CASA of Bastrop County CASA of Bell & Coryell Counties CASA of Brazoria County CASA of Cameron & Willacy Counties CASA of Central Texas â&#x20AC;&#x201C; New Braunfels and San Marcos CASA of Collin County

Number of Children Included in Sample n

Coverage Rate (%) o

479

48.6

132

22.7

90.0

159

72.3

15 239 71 480

100.0 99.6 96.8 60.3

177

96.6

162

96.3

304

99.0

109 343

100.0 88.9

1,113

17.3

Brazoria

181

43.1

Cameron, Willacy

593

38.8

Caldwell, Comal, Guadalupe, Hays

644

51.4

Collin

431

97.7

Counties Covered Armstrong, Briscoe, Carson, Hutchinson, Potter, Randall, Swisher Taylor Edwards, Kimble, Mason, McCulloch, Menard Brooks, Duval, Jim Wells, Kenedy, Kleberg Dallam, Hartley, Moore, Sherman Hunt Delta, Lamar, Red River Smith, Van Zandt, Wood Austin, Colorado, Waller, Washington Bosque, Eastland, Erath, Hamilton Blanco, Burnet, Lampasas, Llano, San Saba Brown, Comanche, Mills Bastrop, Fayette, Lee Bell, Coryell

This list reflects programs (and their associated counties) that were in operation during the entire study timeframe of FY 2013 and FY 2014. n This reflects the number of children served by each program who were included in the study sample. These figures may differ from local program records due to the inclusion and exclusion criteria applied in creating the study sample. o Coverage rate refers to the proportion of all children in substitute care in that program service area who were appointed to receive a CASA volunteer advocate during the study timeframe. Children may have been served by a volunteer or a staff member. These figures may differ from local program records due to the inclusion and exclusion criteria applied in creating the study sample.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

CASA of Denton County, Inc. CASA of El Paso CASA of Ellis County CASA of Galveston County CASA of Grayson County CASA of Harrison County CASA of Hidalgo County CASA of Hood & Somervell Counties CASA of Johnson County CASA of Liberty/Chambers Counties CASA of Matagorda and Wharton Counties CASA of McLennan & Hill Counties CASA of Navarro County CASA of North Texas CASA of Northeast Texas CASA of South Texas â&#x20AC;&#x201C; Pleasanton Office CASA of Southeast Texas CASA of Tarrant County CASA of the Coastal Bend CASA of the High Plains CASA of the Permian Basin Area, Inc. CASA of the Pines, Inc. CASA of the Rolling Plains CASA of the Sabine Neches Region p CASA of the South Plains CASA of Titus, Camp and Morris CASA of Travis County CASA of Trinity Valley p

Denton El Paso Ellis Galveston Grayson Harrison, Marion, Panola Hidalgo

460 600 108 273 213 170 885

77.8 61.7 99.1 30.4 93.0 88.2 23.1

Hood, Somervell

119

98.3

Johnson

277

99.6

Chambers, Liberty

266

47.7

Matagorda, Wharton

97.0

Hill, McLennan

9.07

Navarro Cooke Bowie, Cass Atascosa, Frio, La Salle, Wilson, Karnes Jefferson Tarrant Aransas, Nueces, San Patricio Donley, Gray, Hansford, Hemphill, Lipscomb, Ochiltree, Roberts, Wheeler

58 121 327

89.7 100.0 79.5

216

70.8

326 1,517 655

95.4 48.7 49.5

98.4

Crane, Ector, Loving, Ward, Winkler

351

31.1

Angelina, Houston, Polk Childress, Collingsworth, Cottle, Hall, King Hardin, Jasper, Newton, Orange, Sabine, Tyler Cochran, Hale, Hockley, Lubbock, Terry, Yoakum

268

45.9

81.0

319

84.0

855

29.5

Camp, Titus, Morris

100.0

1,247 424

80.3 99.3

Travis Anderson, Cherokee, Henderson

At the time of data collection, CASA of the Sabine Neches Region was called Advocates for Children, Inc.

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

CASA of Walker County CASA of West Texas CASA of Williamson County CASA of Wise & Jack Counties CASA – Hope for Children, Inc. Child Advocates of Fort Bend Child Advocates of Montgomery County Child Advocates San Antonio Child Advocates, Inc. Child Advocates/CASA of Red River Children’s Advocacy Center of Tom Green County Dallas CASA East Texas CASA Fannin County Children’s Center (CASA & CAC) Frontier CASA Golden Crescent CASA Great Plains CASA for Kids, Inc. Hill Country CASA Lake Country CASA Lone Star CASA, Inc. North Star CASA Tri-County CASA Voices For Children Inc., CASA of Brazos Valley Voz de Ninos

Preliminary Selection Bias Report

San Jacinto, Trinity, Walker Andrews, Dawson, Gaines, Glasscock, Howard (Texas), Martin, Midland Williamson

116

67.2

71.9

305

36.1

Jack, Wise

152

96.7

Palo Pinto, Parker

361

99.7

Fort Bend

184

96.2

Montgomery

479

90.8

3,798 3,931

39.0 46.2

296

93.9

437

71.4

2,344 480

45.0 48.5

94.1

35.0

371

82.2

52.1

199 151 162 39 214

98.5 88.7 82.7 82.1 12.6

Brazos, Burleson, Grimes

180

75.6

Webb

502

13.3

Bexar Harris Archer, Clay, Montague, Wichita Coke, Concho, Crockett, Irion, Reagan, Runnels, Schleicher, Sterling, Sutton, Tom Green Dallas Gregg, Rusk, Upshur Fannin Brewster, Culberson, Hudspeth, Jeff Davis, Pecos, Presidio, Reeves Calhoun, DeWitt, Goliad, Gonzales, Jackson, Lavaca, Refugio, Victoria Bailey, Castro, Deaf Smith, Oldham, Parmer Bandera, Gillespie, Kendall, Kerr Franklin, Hopkins, Rains Kaufman, Rockwall Stephens, Throckmorton, Young Medina, Real, Uvalde

March 20, 2017

childandfamilyresearch.org

Table 5: List of Programs with No Roster or Unusable Roster, and Counties Covered Program Name

Counties Covered

CASA of Deep East Texas

Nacogdoches, San Augustine, Shelby

CASA of McLennan & Hill Counties

Hill, McLennan

Frontier CASA

Brewster, Culberson, Hudspeth, Jeff Davis, Pecos, Presidio, Reeves

Table 6: List of Counties Not Served by a CASA Program During Fiscal Years 2013-2014 Baylor Dickens Foard Jones Leon McMullin Scurry Val Verde

Bee Dimmit Freestone Karnes Limestone Milam Shackelford Wilbarger

Preliminary Selection Bias Report

Borden Edwards Garza Kent Live Oak Mitchell Starr Wilson

Callahan Falls Hardeman Kinney Lynn Motley Stonewall Zapata

March 20, 2017

Coleman Fisher Haskell Knox Madison Nolan Terrell Zavala

Crosby Floyd Jim Hogg Lamb Maverick Robertson Upton

childandfamilyresearch.org

Appendix C: Counties by DFPS Regions

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Appendix D: Caregiver Risk Factors Contributing to Removal with Frequency Distributions Table 7: Frequency and Distribution of Individual Caregiver Risk Factors Caregiver Risk Factor

Frequency

Percent

2,815

209

0.7%

Drug abuse

18,570

59.2%

Emotionally disturbed

3,196

10.2%

Inadequate housing

5,294

16.9%

Incarcerated

2,782

8.9%

Unable to cope

4,573

14.6%

Alcohol abuse Death

Table 8: Frequency and Distribution of the Number of Caregiver Risk Factors Number of Caregiver Risk Factors

Frequency

Percent

No risk factors

6,348

20.2%

1 risk factor

16,123

51.4%

2 risk factors

6,225

19.8%

3 or more risk factors

2,675

8.5%

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Appendix E: CPS Maltreatment Types/ Removal Reasons Table 9: Frequency and Distribution of Individual Maltreatment Types Maltreatment Type

Frequency

Percent

Abandonment

925

Emotional abuse

614

Medical neglect

1,349

4.3%

Neglectful supervision

24,948

79.5%

Physical abuse

7,006

22.3%

Physical neglect

4,092

13%

Refusal to accept parental responsibility

1,646

5.3%

Sexual abuse

1,768

5.6%

597

1.9%

Other

Table 10: Frequency and Distribution of the Number of Maltreatment Types Number of Maltreatment Types

Frequency

Percent

1 maltreatment type

22,119

70.5%

2 maltreatment types

7,420

23.7%

3 or more

1,832

5.8%

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

Appendix F: Logistic Regression Output Table 11: Logistic Regression Findings Variable

95% Confidence Interval

Odds Ratio

Standard Error

p-value

1.33 1.59

0.06 0.06

0.000 0.000

[1.23 [1.47

1.45] 1.72]

13-17 Female Child Race/Ethnicity (White baseline) African American Hispanic Other

1.51 1.02

0.08 0.03

0.000 0.41

[1.37 [0.97

1.68] 1.08]

1.04 0.94 0.96

0.05 0.04 0.06

0.46 0.12 0.56

[0.94 [0.87 [0.84

1.14] 1.02] 1.10]

Mother ever in DFPS custody Past or current DV indicated Number of siblings in care (none baseline) 1 2 3+ Motherâ&#x20AC;&#x2122;s age at removal (over 21 baseline) 18 to 21 Under 18 Number of caregiver risk factors (none baseline)

1.04 1.10

0.08 0.03

0.63 0.002

[0.90 [1.04

1.20] 1.17]

1.50 2.02 2.08

0.06 0.09 0.09

0.000 0.000 0.000

[1.38 [1.85 [1.90

1.63] 2.20] 2.26]

0.94 1.11

0.06 0.16

0.35 0.46

[0.82 [0.84

1.07] 1.48]

1 2 3+ Number of prior CPS cases (none baseline) 1 2 3+

1.01 1.10 1.21

0.04 0.05 0.07

0.71 0.052 0.002

[0.94 [1.00 [1.07

1.09] 1.20] 1.37]

1.13 1.12 1.23

0.06 0.07 0.07

0.013 0.09 0.000

[1.03 [0.98 [1.12

1.25] 1.27] 1.37]

1.01 1.37 0.86 1.98

0.04 0.09 0.03 0.06

0.04 0.000 0.000 0.000

[1.00 [1.20 [0.81 [1.87

1.15] 1.55] 0.92] 2.09]

0.012

0.002

0.000

[0.008

0.018]

Child Age (0-2 baseline) 2-4 5-12

Number of maltreatment types (1 baseline) 2 3+ Initial placement with kin Percent of local children in care with CASA Constant N=31,198; Wald Chi2(25)=1300.88; p<0.000

Random-Effects Parameters Court ID (sd_cons)

Estimate

Standard Error

1.212

0.07

95% Confidence Interval [1.08

1.36]

LR test vs. logistic regression: Chibar2(01)=3613.63; p<0.000

Preliminary Selection Bias Report

March 20, 2017

childandfamilyresearch.org

ENDNOTES 1 Osborne, C. (2015). Child Outcomes and Volunteer Effectiveness Study: Determining the effectiveness of CASA on child outcomes. Austin, TX: Child and Family Research Partnership, LBJ School of Public Affairs, University of Texas at Austin. 2 About. (n.d.). Texas Court Appointed Special Advocates for Children. Retrieved from http://texascasa.org/about/. 3 Lawson, J., & Berrick, J.D. (2013). Establishing CASA as an evidence-based practice. Journal of Evidence-Based Social Work, 10(4), 321337. 4 Lawson, J., & Berrick, J.D. (2013). Establishing CASA as an evidence-based practice. Journal of Evidence-Based Social Work, 10(4), 321337. 5 Calkins, C. A. & Millar, M. (1999). The effectiveness of Court Appointed Special Advocates to assist in permanency planning. Child and Adolescent Social Work, 16, 37-45. 6 McRoy, R. G. & Smith, S. (1998). CASA of Travis County evalution: Final report. Austin, TX: Center for Social Work Research, School of Social Work, University of Texas at Austin. 7 Powell, M. & Speshock, V. (1996). Arizona Court Appointed Special Advocate (CASA) Program: Internal Assessment. 8 Smith, S. (1993). CASA of Travis County evaluation final report. Austin, TX: Texas Department of Protective and Regulatory Services. 9 Caliber Associates. (2004). Evaluation of CASA representation: Final report. Fairfax, VA: Caliber Associates. 10 Litzelfelner, P. (2000). The effectiveness of CASAs in achieving positive outcomes for children. Child Welfare, 79(2), 179-193. 11 Poertner, J., & Press, A. (1990). Representation of children in child abuse and neglect cases. A comparison of court appointed special advocates with staff attorney models: Who best represents the interest of child in court? Child Welfare, 69(6), 537-549. 12 Siegel, G.C., Halemba, G.J., Gunn, R.D., Zawacki, S., Bozynski, M., & Black, M.S. (2001). Arizona CASA effectiveness study. Pittsburgh, PA: National Center for Juvenile Justice. 13 Calkins, C. A. & Millar, M. (1999). The effectiveness of Court Appointed Special Advocates to assist in permanency planning. Child and Adolescent Social Work, 16, 37-45. 14 Litzelfelner, P. (2000). The effectiveness of CASAs in achieving positive outcomes for children. Child Welfare, 79(2), 179-193. 15 McRoy, R. G. (1998). East Texas CASA: A program evaluation. Austin, TX: Center for Social Work Research, School of Social Work, University of Texas at Austin. 16 Smith, S. (1993). CASA of Travis County evaluation final report. Austin, TX: Texas Department of Protective and Regulatory Services. 17 Brennan, K., Wilson, D., George, T., & McLaughlin, O. (2010). Washington state court appointed special advocate program evaluation report. Seattle: University of Washington School of Social Work and Washington State Center for Court Research. 18 Caliber Associates. (2004). Evaluation of CASA representation: Final report. Fairfax, VA: Caliber Associates. 19 Leung, P. (1996). Is the court appointed special advocate program effective? A longitudinal analysis of time involvement and case outcomes. Child Welfare, 75(3), 269-284. 20 Poertner, J., & Press, A. (1990). Representation of children in child abuse and neglect cases. A comparison of court appointed special advocates with staff attorney models: Who best represents the interest of child in court? Child Welfare, 69(6), 537-549. 21 Siegel, G.C., Halemba, G.J., Gunn, R.D., Zawacki, S., Bozynski, M., and Black, M.S. (2001). Arizona CASA effectiveness study. Pittsburgh, PA: National Center for Juvenile Justice. 22 Caliber Associates. (2004). Evaluation of CASA representation: Final report. Fairfax, VA: Caliber Associates. 23 Litzelfelner, P. (2000). The effectiveness of CASAs in achieving positive outcomes for children. Child Welfare, 79(2), 179-193. 24 Poertner, J., & Press, A. (1990). Representation of children in child abuse and neglect cases. A comparison of court appointed special advocates with staff attorney models: Who best represents the interest of child in court? Child Welfare, 69(6), 537-549. 25 Siegel, G.C., Halemba, G.J., Gunn, R.D., Zawacki, S., Bozynski, M., and Black, M.S. (2001). Arizona CASA effectiveness study. Pittsburgh, PA: National Center for Juvenile Justice. 26 Students in Bilingual/ESL Programs. The Annie E. Casey Foundation, KIDS COUNT Data Center. Retrieved from http://datacenter.kidscount.org/. 27 Oosterman, M., Schuengel, C., Wim Slot, N., Bullens, R.A.R., Doreleijers, T.A.H. (2007). Disruptions in foster care: A review and metaanalysis. Children and Youth Services Review, 29(1), 53-76. 28 Leslie, L., K., Landsverk, J., Ezzet-Lofstrom, R., Tschann, J.M., Slymen, D.J., & Garland, A. F. (2000). Children in foster care: Factors influencing outpatient mental health service use. Child Abuse & Neglect, 24(4), 465-476. 29 Shlonsky, A., Webster, D., & Needell, B. (2003). The ties that bind: A cross-sectional analysis of siblings in foster care. Journal of Social Service Research, 29(3), 27-52. 30 Kohl, P. L., Edleson, J. L., English, D. J., & Barth, R. P. (2005). Domestic violence and pathways into child welfare services: Findings from the National Survey of Child and Adolescent Well-Being. Children and Youth Services Review, 27(11), 1167-1182. 31 Kohl, P. L., Barth, R. P., Hazen, A. L., & Landsverk, J. A. (2005). Child welfare as a gateway to domestic violence services. Children and Youth Services Review, 27(11), 1203-1221. 32 Proctor, L. J., Aarons, G. A., Dubowitz, H., English, D. J., Lewis, T., Thompson, R., Hussey, J. M., Litrownik, A. J., & Roesch, S. C. (2012). Trajectories of maltreatment re-reports from ages 4 to 12: Evidence for persistent risk after early exposure. Child Maltreatment, 17(3), 207-217.

Preliminary Selection Bias Report

March 20, 2017

Strengthening Families & Enhancing Public Policy Through Rigorous Research The Child and Family Research Partnership is an independent, nonpartisan research center under the direction of Dr. Cynthia Osborne at the LBJ School of Public Affairs at The University of Texas at Austin. CFRP specializes in rigorous research on policy issues related to young children, teens, and their parents. CFRP seeks to understand how current demographic trends affect parents and their children, what factors contribute to both positive and negative child outcomes, and what policy and programmatic changes can be implemented to improve child and family wellbeing.

Research Areas Family Supports Community Impact • Economic Security Fatherhood Paternity Establishment • Supports for Fathers • Child Support Early Childhood Home Visiting Programs • Public Pre-Kindergarten Child Welfare Child Welfare Workforce • Child and Family Outcomes Adolescent Health and Wellbeing Teen Pregnancy Prevention • Youth Homelessness

Join Our Email List http://eepurl.com/xEN3v childandfamilyresearch.org facebook.com/CFRPlbj

bit.ly/cfrplinkedin @CFRPlbj