2020—2021 Early Childhood Court Evaluation Deliverable #3: Final Report by fsuchildwelfare

RESEARCH REPORT

2020-2021 Early Childhood Court Evaluation Deliverable #3: Final Report SEPTEMBER 15, 2021 Principal Investigator: Lisa Magruder, Ph.D., MSW

CONTENTS Introduction....................................................................................................................................... 3 Evaluation Team....................................................................................................................................... 3 Florida Institute for Child Welfare...................................................................................................... 3 University of South Florida................................................................................................................ 3 Chapter 1: Background.................................................................................................................... 3 2018-2019 Early Childhood Court Evaluation.......................................................................................... 3 Literature................................................................................................................................................... 4 Early Childhood Court....................................................................................................................... 4

Co-Principal Investigator:

Demographics.............................................................................................................................. 4

Jennifer Marshall, Ph.D., MPH

Outcomes..................................................................................................................................... 4 Circle of Security............................................................................................................................... 4 Structure....................................................................................................................................... 4 Training......................................................................................................................................... 4 Fidelity.......................................................................................................................................... 4 Outcomes..................................................................................................................................... 5 Child-Parent Psychotherapy............................................................................................................. 5

Funded through a contract with ZERO TO THREE

Structure....................................................................................................................................... 5 Training......................................................................................................................................... 5 Fidelity.......................................................................................................................................... 5 Outcomes..................................................................................................................................... 6 Chapter 2: Providers’ Experiences Implementing Parenting-Focused Therapeutic Services in Florida’s Early Childhood Courts...........................................................6 Methodology............................................................................................................................................. 6 Sampling........................................................................................................................................... 6 Data Collection.................................................................................................................................. 6 Data Analysis.................................................................................................................................... 6 Results...................................................................................................................................................... 7 Provider Background........................................................................................................................ 7 Engaging with Early Childhood Court............................................................................................... 7 Onboarding.................................................................................................................................. 7 Team Dynamics............................................................................................................................ 8 Therapist’s Role........................................................................................................................... 9 Working with Early Childhood Court-Involved Families �� 10 Characteristics of Families Involved with Early Childhood Court.............................................. 10 System and Family Challenges................................................................................................. 10 Therapeutic Modalities............................................................................................................... 11 Implementation Decisions.......................................................................................................... 13 Provider Needs.......................................................................................................................... 13 Discussion.............................................................................................................................................. 13 Summary of Findings...................................................................................................................... 13 Limitations....................................................................................................................................... 14

Chapter 3: Clinician Survey.......................................................... 14

Chapter 5: Secondary Data Analyses of Child-

Methodology...................................................................................... 14

Parent Psychotherapy Outcomes............................................ 38

Sampling �� 14

Methodology..................................................................................38

Data Collection �� 15

Data Collection.......................................................................38

Data Analysis �� 15

Data Analysis.........................................................................38

Results �� 15

Results...........................................................................................38

Background and Preparation �� 15

Case Characteristics..............................................................38

Circle of Security-Parenting �� 15

Outcomes...............................................................................38

Training �� 15

Discussion.....................................................................................39

Delivery �� 15

Summary of Findings.............................................................39

Child-Parent Psychotherapy �� 16

Limitations..............................................................................39

Training �� 16

Chapter 6: Summary of Evaluation Findings.......................... 42

Delivery �� 16

Summary and Triangulation of Findings........................................42

Reflective Supervision.......................................................... 18

Delivery of Therapeutic Services...........................................42

Discussion �� 19

Parental Factors Influencing Delivery of Therapeutic Services............................................................ 43

Key Findings �� 19 Limitations �� 19 Chapter 4: Therapeutic Networks �� 19 Methodology...................................................................................... 19 Sampling �� 19 Data Collection........................................................................... 19 Data Analysis............................................................................. 20 Results �� 20 Parents’/Caregivers’ Overall Experience in ECC...................... 20 Therapeutic Networks Identified by Parents and Caregivers........................................................ 20

ECC Factors Influencing Delivery of Therapeutic Service.............................................................. 43 Relational Factors Influencing Therapeutic Services............44 COVID-19 Factors Influencing Therapeutic Services............45 Limitations......................................................................................45 Recommendations........................................................................45 Next Steps..............................................................................46 References................................................................................. 47 Appendices................................................................................ 50 Appendix A—Initiative One Interview Schedule............................50

Evaluation Question 1................................................................ 23

Appendix B—Initiative One Provider Survey................................52

Therapeutic Programs and Services �� 23

Appendix C—Initiative Two Interview Schedules..........................58

Provider Feedback on Specific Therapeutic Programs, Approaches, Methods, and Strategies................ 25 Evaluation Question 2................................................................ 28 Parents/Caregivers: Strategies and Support for Relationships with ECC Providers and Caregivers.............. 28 Providers: Strategies and Support for Relationships with ECC Parents and Caregivers........................................ 28 Evaluation Question 3................................................................ 30 Parents/Caregivers: Barriers to Engagement in Therapeutic Services and Supports...................................... 30 Improvements to Therapeutic Programs and Services........ 32 Evaluation Question 4................................................................ 35 Parents/Caregivers............................................................... 35 Providers............................................................................... 35 Discussion......................................................................................... 37 Summary of Findings................................................................. 37 Evaluation Challenges.......................................................... 37 Limitations.................................................................................. 37

FLORIDA INSTITUTE FOR CHILD WELFARE

Introduction Per Florida Statute 39.01304 (2) (2020), “the Office of the State Courts Administrator [OSCA] shall contract for an evaluation of the early childhood court programs to ensure the quality, accountability, and fidelity of the programs’ evidence-based treatment.” ZERO TO THREE (ZTT) contracted with the Florida Institute for Child Welfare (hereinafter, Institute) to develop and execute an evaluation plan to address this statute. The Institute, in consultation with ZTT and OSCA’s Office of Court Improvement (OCI), determined that the priority is evaluating the effectiveness of therapeutic modalities related to parenting and the parent-child relationship. To address the evaluation priorities, the Institute conducted two distinct but related evaluation initiatives: 1) a mixed-methods evaluation of the effectiveness of therapeutic modalities, and 2) a qualitative exploration of the perspectives of ECC-involved caregivers and providers regarding the therapeutic services and benefits of Early Childhood Court. The Institute led initiative one and subcontracted with Institute affiliate Dr. Jennifer Marshall (University of South Florida; USF) to lead initiative two. This final report includes individual chapters by data source as well as a summary chapter of triangulated findings and implications.

University of South Florida Jennifer Marshall, Ph.D., MPH, is an associate professor at the University of South Florida. As co-principal investigator, Dr. Marshall’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included conceptualizing and developing the evaluation plan; communicating with stakeholders; developing evaluation measures; collecting, analyzing, and interpreting data; and disseminating findings. Dr. Marshall is leading initiative two. Laura Kihlström, Ph.D., MS, MPH is a research associate at the University of South Florida Center of Excellence in Maternal and Child Health, Education, and Practice and Department of Anthropology, respectively. Dr. Kihlström’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included project development and data collection, analysis, and reporting for initiative two. Joanna Mackie, Ph.D., MPP, is a research associate at the University of South Florida College of Public Health. Dr. Mackie’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included project development and data collection, analysis, and reporting for initiative two.

The evaluation team would like to acknowledge the contributions of expert consultants Dr. Kimberly Renk, Dr. Diane Koch, and Ms. Meredith Piazza, who provided valuable feedback to the evaluation team at each phase of the project.

Tara Foti, Ph.D., MPH, is a research associate at the University of South Florida College of Public Health. Dr. Foti’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included project development and data collection, analysis, and reporting for initiative two.

EVALUATION TEAM

Shanda Vereen, MSPH, is a research assistant and doctoral student at the University of South Florida College of Public Health. Shanda’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included project development and data collection, analysis, and reporting for initiative two.

Florida Institute for Child Welfare Lisa Magruder, Ph.D., MSW, is the program director of science and research at the Institute. As principal investigator, Dr. Magruder’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included conceptualizing and developing the evaluation plan; communicating with stakeholders; developing evaluation measures; collecting, analyzing, and interpreting data; and disseminating findings. Dr. Magruder is leading initiative one. Michael Killian, Ph.D., MSW, is an assistant professor at the Florida State University College of Social Work. As co-investigator, Dr. Killian’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included data analysis of secondary administrative data and consultation on statistical analyses. Colleen McBride, M.A., is a professional research assistant at the Institute. Colleen’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included communicating with stakeholders, including serving as a point of contact for consultants; conducting literature reviews; assisting in the development of evaluation measures; and collecting, analyzing, and interpreting data for initiative one. Taylor Dowdy-Hazlett, MSW, is a research assistant at the Institute and current doctoral candidate at the Florida State University College of Social Work. Taylor’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included communicating with stakeholders, including consultants as needed; conducting literature reviews; assisting in the development of evaluation measures; and collecting, analyzing, and interpreting data for initiative one. Michae’ Cain, MSW, is a research assistant at the Institute and current doctoral student at the FSU College of Social Work. Michae’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included communicating with stakeholders, including consultants as needed; conducting literature reviews; assisting in the development of evaluation measures; and collecting, analyzing, and interpreting data for initiative one.

Caitlynn Carr, M.A., is a research assistant and current MPH and doctoral candidate at the University of South Florida Center of Excellence in Maternal and Child Health, Education, and Practice and Department of Anthropology, respectively. Caitlyn’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included project development and bilingual data collection, analysis, and reporting for initiative two. Megan Bell, MPH, is a research assistant at the University of South Florida College of Public Health. Megan’s responsibilities on the 2020-2021 Early Childhood Court Evaluation included project development and data collection, analysis, and reporting for initiative two.

Chapter 1: Background Lisa Magruder, Colleen McBride, Taylor Dowdy-Hazlett, & Michae’ Cain.1 2018-2019 EARLY CHILDHOOD COURT EVALUATION In 2018, the Florida OSCA’s OCI contracted with the Institute to conduct a broad-scale evaluation of Early Childhood Court (ECC) implementation in Florida. Using a mixed-methods approach, the evaluators explored a variety of topics, including parent and caregiver experiences of ECC; team fidelity to the ECC approach (i.e., based on Best Practice Standards); cost effectiveness; and relationships between community coordinator funding source and ECC processes and outcomes.1 Data indicated a number of strengths in Florida’s ECCs, though also illuminated areas for improvement. To build on the broad evaluation, the evaluators made several recommendations for more nuanced follow-up research, including evaluating the effectiveness of therapeutic modalities

1 Suggested Citation: Magruder, L., McBride, C., Dowdy-Hazlett, T., & Cain, M. (2021). Chapter 1: Background. In L. Magruder, 2020-2021 Early Childhood Court Evaluation (pp. 3-6).

FLORIDA INSTITUTE FOR CHILD WELFARE

used in ECC and seeking additional input from parents. The present evaluation implements both of these recommendations. LITERATURE The following is a brief review of the literature, including an overview of ECC generally, as well as of the two most frequently cited parentchild therapeutic modalities reported by providers in the present evaluation sample: Circle of Security (COS) and child-parent psychotherapy (CPP). Early Childhood Court Early Childhood Court, based on the Safe Babies Court Team approach, works to address child welfare cases involving children under the age of five who have been placed in out-of-home care. The goal of ECC is to improve child well-being, address trauma, repair family relationships, and promote permanency.2 Core components of ECC include: judicial and child welfare leadership, a local community coordinator, an active community team, family team meetings, a continuum of services, meeting parents where they are, enhancing parents’ relationships and social supports, frequent visitation, individualized concurrent case planning, and systemic quality improvement.3 ECC emphasizes a collaborative approach. Each case joins a team of parents, caregivers, a judge, a community coordinator, an infant mental health specialist, attorneys, a guardian ad litem, and a child welfare case manager. These “family teams” meet monthly to discuss the family’s progress and recommend changes to the case plan as needed.4 ECC was first adopted as a model in 20055 and has since been implemented in 99 sites across 30 states.6 Florida’s ECC launched in 2014 and is currently active in 27 sites statewide.2 Between 2015 and 2020, Florida’s ECC closed 577 cases involving children between the ages of 0 and 5 years.7 Demographics In April 2021, 39 percent of children removed from the home in Florida were ages 0-3 years; infants under the age of 1 year made up almost half of this group.8 These statistics are largely unchanged from earlier reports.9 Nearly half (48%) of children were removed as a result of parental drug abuse, while one-third (31%) were removed due to inadequate supervision and 20 percent were removed as a result of domestic violence.8 A study by the OSCA (2020)7 analyzed Florida’s ECC system from its inception through the end of 2019. The average age of a child served by Florida’s ECC was just over one year old. Almost twothirds (63%) were white, while just over one-quarter (28%) were Black. Children of two or more races accounted for eight percent of ECC children. Asian children made up 0.2%, while children whose race was unable to be determined accounted for 0.3 percent. Nearly half (44%) were sibling groups involved in the same ECC case. Outcomes While ECC-involved children achieve reunification at the same rate as non-ECC children, the time to reunification is approximately 8.5 months shorter for those whose cases are addressed within ECC. In addition, the adoption rate of ECC-involved children is much higher than that of non-ECC children, while the rate of children placed in permanent dianship is lower.2 Several studies have found that permanency is achieved sooner for ECC-involved children than non-ECC-involved children.10,2,11 There is also evidence that ECCinvolved children have improved developmental and behavioral outcomes.12,13 Circle of Security Circle of Security (COS) is a parenting program guided by Bowlby’s (1980)14 attachment theory, which teaches parents how to be a FLORIDA INSTITUTE FOR CHILD WELFARE

secure base for their children by responding sensitively to their needs. COS is a manualized program that utilizes videos to assist caregivers to understand attachment and reflect on how their feelings and relationship history impacts their parenting practices. COS aims to reduce insecure and disorganized attachment and increase parenting sensitivity. While this literature review includes information on both COS-P and COS-I (described below), participants in the Chapter 2: Providers’ Experiences Implementing Parenting-Focused Therapeutic Services in Florida’s Early Childhood Courts typically spoke specifically about COS-P. Structure COS consists of two models: The Circle of Security-Intensive (COS-I) model, a 16 to 22-week intervention, and the Circle of Security-Parenting (COS-P) model, an eight-week intervention (The Circle of Security International, 2019a). Both the COS-I and COS-P models were initially developed for group therapy but can be implemented in individual and family therapy settings (Powell et al., 2014). COS can be implemented with any caregiver (e.g., parent, foster parent, grandparent, teachers) and allows each caregiver to move through the content at their own pace to increase skills associated with the child’s cues, reflecting on the child’s feelings and thoughts, and reflecting on their own feelings, plans, and behaviors.15 Providers utilize a visual map to help parents understand attachment and determine where they are struggling to meet their child’s needs to assist in understanding how children develop a sense of security. Providers work with parents to view their child’s behaviors as a way of communicating needs in order to move away from punishment and reward for behavioral issues, teaching parents that when the child feels secure, the child becomes more compliant and cooperative.16 COS-P consists of eight chapters, each chapter consisting of one 15-minute video showing parent-child interactions and previous participants’ feelings about what they learned in the intervention. Chapters one and two cover psychoeducational materials on attachment, COS-P’s visual map, and children’s attachment needs. Chapters four and five discuss emotional responsiveness to the child and how this creates a safe haven for the child. Chapter five teaches caregivers about parenting struggles by guiding caregivers to reflect on their personal defensive processes that inhibit them from building a healthy attachment with their child. Chapters six and seven discuss disorganized attachment and the importance of repairing ruptured relationships with children. Chapter eight summarizes the material learned and celebrates the parent’s completion of the program.17 Training COS-P was developed from COS-I out of a need for rapid implementation, with training taking a shorter period, without post-training supervision, and the possibility of implementation in both group and individual settings.17 COS-I requires providers to: 1) be licensed clinicians, 2) participate in a minimum of one year of supervision, 3) participate in a 10-day COS-I training, and 4) pass a test. COS-P requirements only include a 4-day training and provides coaching after training completion.18 Fidelity The COS organization states there are two methods to COS fidelity. First, within COS-I, there is a required minimum of one year of supervision (approximately 70 hours) where the therapist and supervisor review assessment and treatment planning as well as videos and discussions. The total cost for supervision is $15,000. Second, COS-P offers optional fidelity coaching which begins at the start of COS-P training and runs in conjunction with a reflective journal requirement. The therapist writes in the reflective journal after each chapter is completed and asked to rate a series of items about the implementation and process on a scale of 1 to 4. The first set of questions is designed to assess fidelity of implementation while 4

most of the questions focus on the provider’s ability to manage their triggers and acknowledge when parent meet specific chapter goals. Because the COS-P fidelity coaching is optional, there are varying levels to which providers can move through the fidelity program: ● Level 1: Complete the four-day COS-P training only ● L evel 2: Complete two cases of COS-P and review weekly journal with fidelity coach ● L evel 3: Complete two cases of COS-P and review weekly journal and video reviews with fidelity coach ● L evel 4: Review video sessions with fidelity coach and coach other providers Outcomes COS-I decreases parents’ irritability;19 internalizing, externalizing, and behavior concerns in children;20 and parenting stress and psychological symptoms.21 In addition, COS-I is proven to increase well-being and attachment;15 protective factors;20 and parenting sensitivity.22 COS-P shows similar results, decreasing unresponsive behaviors in mothers;17 and stress, depression, and anxiety symptoms.23 Qualitative feedback suggests that parents felt the intervention positively impacted their parenting, personal emotions, and interactions with their child.23 A meta-analysis conducted on COS found that most interventions were conducted in a group therapy setting and improved child attachment quality, parenting quality, and parenting self-efficacy.24 Child-Parent Psychotherapy Rooted in infant-parent psychotherapy, child-parent psychotherapy (CPP) emerged as a therapeutic modality in the mid-1990s.25 CPP is used to treat children ages 0-5 years who experienced traumatic events (e.g., separation from a caregiver, violence or abuse, or the death of a loved one).26 Primarily based in attachment theory, CPP aims to strengthen the relationship between children and caregivers and help families appropriately process and respond to trauma.27 Structure CPP occurs in three phases: the foundational phase, the core intervention phase, and the termination phase.28 CPP is intended to be carried out over 52 weekly sessions lasting 1 to 1.5 hours which may take place in the family’s home, the provider’s office, or in a school or daycare setting.29 During the foundational phase, the provider meets with the child’s caregivers to establish a relationship and to develop a treatment plan. The provider uses these sessions to assess the child’s trauma history and subsequent symptoms, as well as any trauma symptoms the caregivers may have. The provider begins meeting with the child during the next phase— the core intervention phase. In the first session of this phase, the provider explains the CPP process to the child. In subsequent sessions, the child and their caregivers begin to examine the child’s trauma, largely through play, and the provider helps the family respond in a manner that strengthens the child-caregiver relationship. The termination phase occurs at the end of the family’s CPP treatment. These sessions focus on summarizing the family’s progress and preparing the family for the future.29 Training The CPP Dissemination and Implementation Team, a group dedicated to supporting the use of CCP nationwide, offers two training models for agencies that want to develop expertise in CPP: 1) a learning collaborative, designed for agencies that want to begin implementing CPP, and 2) the CPP Agency Mentorship Program (CAMP), intended for agencies that completed the learning collaborative and are looking to sustain the model through their own CPP supervisory team.30 The CPP learning collaborative is an intensive 18-month training designed for teams of licensed mental health professionals. The FLORIDA INSTITUTE FOR CHILD WELFARE

collaborative begins with a three-day didactic learning session where participants learn the basic components of CPP; each team member is also expected to have read the CPP manual. Participants then begin providing CPP to families; clinicians are expected to provide CPP to at least four families during the training period, while supervisors are required to provide CPP to at least two families. Providers participate in twice-monthly reflective supervision meetings, during which clinicians and their supervisors discuss and reflect on the clinicians’ cases. The team holds twicemonthly consultation calls with a licensed CPP consultant; each provider is expected to present two cases over the course of the collaborative. Trainees attend two intensive two-day competencybuilding workshops during the training period. Providers are also expected to complete fidelity measures for at least two families in their care. There are several optional components of the learning collaborative, including pre-work supports, supervisor calls, calls with senior leaders at the agency who are participating in the collaborative, and subject-specific trainings. After completing the learning collaborative, providers are invited to join the CPP roster.31 The cost of training varies by training entity, but generally ranges from $750 to $2,500 per participant.32,33,34,35 Studies have found that, while providers find the learning collaborative useful in broadening their knowledge of CPP, many find the structure of the training model to be challenging because: trainees are expected to treat families while in the initial training stages, face-to-face workshops are held infrequently, and supervision and consultation requirements require a substantial time commitment. Despite providers’ enthusiastic interest in continuing to treat families using CPP, a lack of ongoing trainings and “train the trainer” opportunities present a barrier for sustainability of the treatment.36 Agencies that wish to develop and sustain their use of CPP can, after completing the learning collaborative, apply to the CPP Agency Mentorship Program (CAMP). If an agency is selected as a CAMP Site, certified CAMP Mentors (either a member of the CPP Dissemination and Implementation Team or a senior CPP trainer endorsed by the Team) guide agency supervisory staff in training clinicians in the provision of CPP. The process to complete the CAMP training is similar to the learning collaborative, with enhanced supervision and consultation services for supervisory staff. At the end of the 18-month CAMP training, supervisory staff may be endorsed by the CPP Dissemination and Implementation Team as CAMP Agency Trainers who may continue training clinicians and may begin training future Agency Trainers.37 Fidelity Researchers developed six strands of CPP fidelity designed to help the provider understand their own adherence to the therapeutic procedure:38 ● T he content strand asks the provider about their understanding of the case and their treatment plan ● T he procedural strand asks the provider about any problems they have adhering to the processes and procedures for each treatment phase ● T he reflective practice strand asks the provider to reflect on the impact of their emotions on their interactions with the family during CPP sessions ● T he emotional process strand asks the provider to consider the emotional state of the family and the provider’s own responsiveness to those emotions ● T he dyadic relational strand asks the provider whether they are equally aware of and attentive to each family member’s needs and emotions ● T he trauma framework strand asks the provider how they integrate each family member’s trauma into the CPP process 5

Each phase of CPP has an accompanying fidelity form—a document to be completed at various points during the phase. The provider is asked to rate themselves on several domains related to each fidelity strand.39,40,41 In addition, providers are encouraged to review their supervisors’ adherence to the fidelity measures.28 Research suggests that CPP providers generally maintain fidelity to the treatment,42,36 but may find the lengthy fidelity forms to be burdensome.43 Outcomes The CEBC (2015)29 indicates that CPP is “supported by evidencebased research.” Several randomized controlled trials have found that CPP is effective for both children and their caregivers. Infants who experienced abuse or neglect had higher levels of secure attachment after being treated with CPP;44 a follow-up study demonstrated that these effects were sustained for at least one year.45 In a study of 4-year-old children who experienced abuse or neglect, researchers found that CPP had a positive impact on children’s self-representation, maternal attributions, and relationship expectations.46 Among preschoolers exposed to domestic violence, CPP was found to be effective in reducing behavior problems and PTSD symptoms;47 a follow-up study indicated that these effects persisted for at least 6 months.48 CPP was found to be especially effective for children who have experienced multiple traumatic events.49 Mothers of children treated with CPP show improvement in symptoms of avoidance;47 depression;50 PTSD;50,51 and general distress.48 Bernstein, Timmons, and Lieberman (2019) found that mothers perceived fewer negative emotions in infants’ faces after participating in CPP.52 Hagan and colleagues (2017) found that improvement in parents’ PTSD symptoms was correlated with improvement in their children’s symptoms.51 Parents in one study indicated that CPP allowed them to better process their own traumas and adjust their expectations for their lives and their children’s lives.53 According to the CEBC (2015),29 CPP “is commonly used to meet the needs of children…receiving child welfare services.” Several states incorporate CPP as a core component of their ECCs.54 One national study of ECCs found that, of the 51% of children who needed CPP, 94% received treatment.5 Several studies found that CPP is effective at increasing safety, permanency, and well-being among families served by ECCs .12,55,56

Chapter 2: Providers’ Experiences Implementing Parenting-Focused Therapeutic Services in Florida’s Early Childhood Courts Lisa Magruder, Taylor Dowdy-Hazlett, Colleen McBride, & Michae’ Cain.2 To gain an in-depth understanding of the various therapeutic modalities and techniques employed by ECC-affiliated providers (e.g., mental health providers, therapists, counselors), we conducted 13 individual telephone or Zoom interviews with providers. These interviews provided initial insight on 1) which therapeutic modalities are employed with ECC families; 2) consistencies and inconsistencies in implementation across providers; and 3) provider perspective on the utility of certain modalities based on circumstances; and 4) potential therapeutic approach modifications necessary for ECC-served families. The findings, detailed below, are informing the next phase of the evaluation—a survey of provider fidelity.

METHODOLOGY Sampling To establish a sampling frame, the OCI requested a list of ECC therapeutic providers whose work focuses on parent-child relationships from all community coordinators in Florida. Community coordinators responded directly to OCI or to Dr. Magruder, Principal Investigator, and provided a list of 52 clinicians. We removed two clinicians: one duplicate and one who is serving as a consultant on this evaluation. In addition, two providers listed were agency supervisors, so we requested individual contact information for those providers. One agency provided two additional provider names and contact details; the other opted to forward the invitation directly to their providers. Excluding the unknown number of individuals who received an invitation via their supervisor, the final sampling frame consisted of 52 therapeutic providers. We recruited participants in March 2021 by sending Qualtrics e-mail invitations to potential participants in one of three staggered panels, to which participants were randomly assigned. Each panel received an initial invitation, with one reminder e-mail sent a week later. In two instances, the PI sent personal invitations after the Qualtrics-generated e-mail was undeliverable. Fourteen providers completed the recruitment survey, and all provided active consent to participate. One interview was unable to be coordinated, resulting in a final sample size of 13. Data Collection The Institute team developed an interview guide which was reviewed by the expert consultants. Several revisions were made based on consultant feedback, resulting in a seven-item, semi-structured interview guide with additional probing items (see Appendix A). A research assistant was responsible for scheduling interviews, assigning them approximately equally (i.e., 4-5 interviews) to one of the three Institute research assistants. Interviewers reviewed the purpose of the study and offered to answer any participant questions before beginning the audiorecorded interviews. Interviews ranged in length from 44 to 86 minutes, with an average of 59 minutes. The PI e-mailed a USD $25 Amazon.com gift card to each participant following interview completion. All interviews were professionally transcribed then edited by the interviewers for clarity and confidentiality. Data Analysis Researchers applied thematic analysis to the interview data.57,58,59 This analysis includes six primary steps: 1) data familiarization, 2) initial coding, 3) theme searching, 4) theme reviewing, 5) theme defining, and 6) reporting of findings.58 The PI reviewed approximately 30% of the transcripts and created a codebook consisting of 38 substantive patterns which are primarily informed by the items in the interview guide (e.g., orientation to ECC, common characteristics of families, COS session trajectory). Some broader patterns included sub-codes; for example, the pattern “CPP Session Trajectory” included sub-codes for timeline of CPP (e.g., number of sessions), general overview of CPP phases, specific CPP session content, manualization/fidelity measures, and other. The codebook was shared with each research assistant to ensure it captured their experiences during the completed interviews. Using NVivo 12,60 the PI and three research assistants coded the data in teams of two. Transcripts were approximately equally divided, and each coder team independently coded a set of transcripts based on the codebook. Though initially the team planned to calculate Cohen’s Kappa, we opted instead to rely on multiple coders, including all interviewers, as a form of persistent engagement with the data to enhance validation of findings.61 Codes were merged in NVivo to ensure comprehensive representation.

2 Suggested Citation: Magruder, L., Dowdy-Hazlett, T., McBride, C., & Cain, M. (2021). Chapter 2: Providers’ experiences implementing parenting-focused therapeutic services in Florida’s Early Childhood Courts. In L. Magruder, 2020-2021 Early Childhood Court Evaluation (pp. 6-14).

FLORIDA INSTITUTE FOR CHILD WELFARE

Using patterns in the data, the PI arranged themes that were largely guided by the major sections of the semi-structured interview guide. As a form of debriefing,61 each team member reviewed the findings for validation. Results were also shared with consultants to provide contextual interpretation of the findings and assist in guiding relevant discussion points. RESULTS The 13 participants represented disparate areas of Florida. The Southern region was the only region not represented. We did not assess for demographics during this phase of the evaluation, though professional details were captured as part of the interview process (e.g., educational background, time involved in ECC). Given the small sample, the evaluation team decided to withhold certain nuances from the following results if those nuances could reasonably serve to identify the providers, therefore compromising the confidentiality of participants or their clients. This primarily resulted in the truncation of exemplar participant quotes. Provider Background Most providers completed graduate education in fields of counseling, social work, and child psychology and the majority reported licensure, or license-eligibility, in their respective fields (e.g., Licensed Clinical Social Worker, Licensed Mental Health Counselor). Excluding COS and CPP training, most participants indicated participating in trauma-focused trainings, including those focused on children, families, sexual abuse, and attachment. Approximately half reported training in cognitive behavioral therapy (CBT), with most being trained specifically in trauma-focused CBT. In addition, many participants reported training in infant mental health (IMH), though intensity of training varied widely. For example, while one participant reported (non-consecutive) years of training with various IMH experts, another reported they only completed a required six-week mental health training prior to CPP training. The providers reported several other modalities or treatments in which they received training, such as eye movement desensitization and reprocessing (EMDR), trust-based relational intervention, and sand play. When describing their paths to becoming therapeutic providers, many participants shared they previously held positions in child welfare or child advocacy. Several of these individuals noted the impetus to begin providing ECC services came when they recognized the significance of underlying trauma in the populations they served, or that they had a general desire to provide improved services for children and families. The latter sentiment was echoed by others without child welfare experience. One provider shared that when previously working with teen parents and their very young children, they began to notice a “gap in services”: When I say “gap in services” I mean in the evaluation piece. And there wasn’t a lot of providers in the community at that time to actually tailor the services to young children’s specific needs, their specialized needs, [ages] zero to five. Another participant shared, “…This may sound crazy, but I love child welfare, but I was getting burned out in the case management, just doing that whole, the case management part and the child safety part.” Following continued professional development, this participant began providing behavioral health services within child welfare, noting that their experience in both roles is an asset (e.g., “having the child welfare experience and now the therapeutic experience and blending those two systems to support the need”). Engaging with Early Childhood Court Onboarding Providers’ time with ECC varied widely. Most providers worked FLORIDA INSTITUTE FOR CHILD WELFARE

with ECC for two years or less, while others have been involved for up to 7 years. This variation seems to be based on the timing of the establishment of specific ECCs, as several providers shared that they have been active on their ECC team since its inception. Most participants, regardless of time involved with ECC, reported consistent engagement with their ECC team and ECC-served parents (e.g., consistent referrals, “100% of what I do is ECC”). Of the few with limited engagement, most indicated they worked within more recently established ECCs. In terms of joining their ECC team as a therapeutic provider, a few participants were approached by fellow community providers or the ECC itself to join as a therapeutic team member. For example: “… the court system here decided to create an ECC, some of our funders sent me and some other therapists to be trained in CPP and kind of help build the ECC team from the ground up that way.” However, the vast majority were onboarded through their employer. One provider shared: Actually, I was offered the opportunity to go to the training. And to put not too fine a point on it, the community behavioral health agency I worked with at the time was heavily marketing this newfangled service to [our circuit]. And at the time I did not understand I was the only provider they had with that training. So, I more or less got thrown into it and learned sink-or-swim style. Notably, this experience of a “sink-or-swim” orientation to ECC was shared by several participants. Participants, mostly those brought on through their agency, said there was no or minimal orientation to ECC as a program. A provider who said they received no orientation expressed, “I wish there was kind of more of that and that might be available. I just don’t know what it is. Like it’s not readily accessible.” Another provider elaborated: I guess you want transparency and honesty, right? This will help. I didn’t really have much, and that’s the honest truth, and that definitely needs to change…We’ve went through some changes since I’ve been in the program, multiple times with therapists, and that is really important – especially if you’re coming from maybe a private practice or an outpatient, or you’re just doing maybe some contract work… You know, understanding that ECC, therapeutically – or a case – it doesn’t look like any other type of therapists, because you juggle different roles. You can’t just treat it as a therapeutic role ‘cause you’re working with case management. You have to disclose things to the court, so you don’t have all this confidentiality. So, that could rupture relationships of trust, because you might have to say something that a parent doesn’t want you to have to say in front of the judge or in front of case management, but you have to, ‘cause that’s your role. And how do you keep an intact rapport with a family you’re trying to work on to break some of those cycles and for them to reflect and be aware if you’re constantly feeling like you’re rupturing a trust or relationship? So, that’s a very fine line of how you do that. And it can be done. So, the orientation piece, to a therapist, is very important. For those who reported minimal orientation, several shared they learned about ECC and their role through shadowing and speaking with other providers, including outgoing therapists in their own agencies (e.g., “through my coworker who was doing that and then I started picking up the cases …I guess through my coworker and my supervisor”). Others shared that they read books, engaged with reflective supervisors, or connected with external trainers. For those who did not express the same concern regarding limited orientation—which included newer and more seasoned ECC providers—several shared that their previous professional experiences had afforded them opportunities to understand the major tenets of ECC (e.g., “Well, I mean, I was oriented to ECC when I worked for the child welfare agency”). Moreover, several 7

participants mentioned being “oriented” by virtue of either helping set up their circuit’s team or working with their ECC while it was starting. Team Dynamics Providers’ perceptions of their team’s functioning and experiences were mixed. Nearly half of participants explicitly discussed the strength of their team, seen as a benefit to both the general ECC process and parents; for example, a provider noted that their team had a “great dynamic” due to the team-focus: “You really feel like you have the support to address the person’s needs as you’re working with them.” Another participant shared how the team approach empowers families: “…what makes ECC different – is we’re not there doing something to them or because of them. We’re there to partner with them. And if they feel that, that gives them this empowerment to take back their life.” The sentiment that the team-based ECC approach was uniquely supportive of families was shared by numerous participants; for example: “We met them where they were at, and that’s – it’s a relationship, again. I think that’s the special thing about the team – is that we’re so intensive, they know we’re there. And so, they don’t have an excuse.” Participants also shared how, when ECC is implemented well, the benefits are visible: I have seen kind of just like, the rewards in other cases where the parent is renovated, and that extra support is helping, and it is working the way it is kind of designed to. So, that’s really, really neat to see that happen and to see like, a parent and a child’s relationship really improve and to see it prioritized. So, that’s, I think could be a huge reward of the process. In addition to noticing improved outcomes for families, providers experienced positive aspects related to the ECC process. They noted that open communication is useful for all team members, including the families. More frequent or “standard” meetings and hearings were described as beneficial to cases, with one provider noting that it came with benefits to professionals as well: “Being part of the family support team meetings, I’m getting subpoenaed a lot less.” Conversely, the intensive team-based approach does present certain challenges, including a lack of understanding of “who is in charge” (e.g., “…sometimes, it feels like the blind leading the blind…”) as well as each team member’s role. A provider described their team as such: I would say our relationships tend to be more strained, I think, and – more strained and just kind of confusing, finding ourselves in like, these conversations with other team members where they’re like, all “This, this, and this is gonna happen” and having to say like, “Actually, no. That’s not appropriate. That’s not what’s gonna happen.” So, it can cause some friction…it can be kind of strained and confusing, I think, when everyone – me included, sometimes – are not entirely sure of our role and our boundaries with those roles. The notion of role or boundary confusion emerged primarily in discussions of the lack of unified messaging around the purpose of ECC as well as the different foci of each team member. One provider said: I think, also, just a lack of clarity – a lack of cohesive clarity in the ECC team about the purpose of ECC. And sometimes, as a provider, I felt like I’m fighting an uphill battle with other team members, explaining to them that – how important the child is. Clinicians expressed challenges in articulating the nuance of client progress, or lack thereof, to team members whose work focuses on parents meeting more explicit goals, such as whether a parent has obtained housing or employment. For example, a provider shared

FLORIDA INSTITUTE FOR CHILD WELFARE

that even when parents complete a program, it does not always translate into action: Their attorney jumped all over me, saying that “the family understands Circle of Security; I don’t know why [the clinician] thinks they don’t understand it.” And I’m like, “It’s the difference between understanding and implementing.” That’s why in the treatment plan I say “implement” and “verbalize and demonstrate” the key concepts. Another participant elaborated on how these nuances should impact reunification decisions: And the team was saying that “Oh she has done everything.” Like they’re just checking the boxes. “She did the parenting class. She did the Circle of Security. She is attending all the CPP sessions.” Ok. Attending all the CPP sessions are not enough. Are you listening what is being discussed? Are you acknowledging what your child is saying? And if a child has a memory or disclosure about abuse. Yes, it happened one year ago. Now the kids are safe. But if they have a memory, acknowledge that memory. Validate their feeling. But [the parent] denied it. She completely denied it…That’s my concern. And believe me, that’s my hardest thing working with this case. And I was hoping that it would be different. We deal with so many cases like this and other cases also. But of course with working with the ECC…I was expecting that my concern will not be dismissed. Will be validated. Yes, that’s a genuine concern what mom’s actions are. Did mom make any progress, make her safe, herself safe or her children safe? So they should not be reunified because what has changed? Several participants shared that providers feel “pressure” to be the team member that brings up “the elephant in the room” versus “tiptoeing around issues.” Providers described themselves as having to be “the bad guy.” As one participant expressed: “But there are times that it feels kind of like – again, like I’m the sole voice going, ‘I’m worried about this.’” Several providers shared concerns about their role as the team member who raises difficult dialogue, noting how it could harm the therapeutic alliance with the client (e.g., “… And I don’t’ want to break down that therapeutic relationship with the person served, for her to hear me say the things that I needed to say to the team”). When clinicians do provide their input, they reported disparate experiences as to the extent to which their opinions are taken into consideration. While some shared sentiments of, “They don’t discount anybody,” and “I love, love, love how much emphasis is placed on our clinical care…,” others reported their concerns were “minimized” or otherwise unwelcome. One provider explained: Yeah, so they don’t want to hear therapeutic opinions unless it serves their needs. So you can’t divorce the clinical from the team. You can’t say that therapy exists in a vacuum. If it is existing in a vacuum, they’re not doing ECC. They’re just doing traditional therapy with the child and they’re having no impact on the team. That’s how it is… So the whole nexus of the clinician and the team is very important. The team doesn’t want to hear from the clinician. The team doesn’t want to hear from the clinician because we create work. Engaging with Specific Team Members Providers shared their perspective on a number of specific team member roles, primarily expressing their thoughts on the judges, community coordinators, case managers, and attorneys with whom they collaborate. Fewer participants spoke about the roles of guardians ad litem, other service providers (e.g., substance abuse, domestic violence), parents, and caregivers. Importantly, the below findings specific to roles only speak to the varied experiences of clinicians in this sample. Examples are provided, though the

researchers were unable to identify saturated patterns of specific positive or negative experiences with the roles presented, with the exception of judicial leadership. Generally, the clinicians reported positive experiences with the judges; one saying, “…I think the best part of ECC is the judge.” Judges were praised both for their leadership skills as well as their belief in and support of families. One provider shared: The way she responded to the parents, to the team, to the lawyers, everybody. In my experience the judges are like very robotic, very clinical. Not clinical, very professional. But this judge was like I feel like sometimes does she have a clinical background? Because the way she was responding to the people, it was beautiful. As it pertains to clinicians specifically, several expressed their appreciation for the judge’s incorporation of their professional recommendations: “…when I mentioned my word being held for the weight that it has, that is very much modeled by him. It’s always he’s telling the coordinator or case management ‘what does the therapist say,’ always referring back to that.” Unlike the more unified perspective of judicial leadership, providers shared more varied experiences with community coordinators. Several spoke to the good job that their coordinator was doing for the team, ensuring appropriate linkages between the professional team members as well as the parents; however, several shared concerns. For example, one clinician desired more support from the coordinator (and case manager) when bringing up concerns related to the families. Another shared a need for improved coordination of the team: I don’t want to call anyone out but I think we have some issues with our community coordinator. I think it really should be someone who can be unbiased, really supports the child, supports – it’s my understanding that ECC is really about shortening the placement. Of course providing all the services that are necessary and making all those referrals and making sure everyone is on target with what their – with their treatment plan whether it be mental health substance abuse in the CPP portion of it. But it’s also making sure that the child – if that’s not working, if the parent is not compliant it’s making sure that that child is placed and that case to expedite the placement for that child. And that’s sort of a concern too. I think maybe perhaps the lack of experience for our coordinator is in play a little bit. And taking information in from so many different sources, determining which cases are appropriate and which are not. Clinicians’ relationships with case managers were similarly varied. Despite several challenges mentioned (e.g., boundary issues, disagreeing with recommendations, communication struggles), a few noted the importance of working closely with the case managers: “I try to cultivate a good relationship with caseworkers because it’s always – I mean, my only priority is really the child, and it’s always better if everyone’s on the same page.” Several clinicians expressed frustration that, from their perspective, case managers do not have an adequate understanding of the therapeutic aspect of ECC, both in terms of the time commitment and the content. One provider expounded on the latter: And for child welfare to also recognize that we all – we will all want the best for these families, but these cases cannot – if you’re looking from a therapeutic lens these cases cannot be triaged just like a regular dependency case. They need to be looked at from a trauma-informed lens…But those are the challenges when the child welfare doesn’t understand the connection between trauma and attachment. And I see that a lot. I see that a lot.

FLORIDA INSTITUTE FOR CHILD WELFARE

A few participants brought up similar concerns regarding attorneys. Several shared that they believe attorneys either do not trust the clinician’s judgement or, in some cases, lack understanding of ECC (e.g., “That one attorney is just – he does not get the ECC model at all and he just attacks us all the time. And it’s like – I’m exhausted with him”). Still, others shared more positive experiences working with attorneys and how ECC shed new light on their collaborative relationship: I do like the team-building meetings. Often, those are kind of fun because the attorneys sometimes aren’t attorney-ish in those. They are themselves and that’s nice. Even if they’re going to cross-examine me and I’m going to be irritated at them later [laughs], at least I know them differently, which is nice. Therapist’s Role In terms of their involvement in major team processes, most participants said they participate in family team meetings and that those meetings are the spaces where they provide updates to the team on their work with the families, emphasizing the importance of transparency. Although, as previously noted, some therapists felt disproportionately burdened with being the “bad guy” on the team when they bring up difficult conversations. In addition to providing family progress updates, therapists said they take on other responsibilities, such as advocating for children or parents or educating the other team members (e.g., “really educating the other team members of like, what are the priorities here, what are we looking for. How do we know that mom’s making progress or the child’s making progress?”). Still another said that they would act in whatever capacity the family or team needs: So I’m always there. I’m always available. Sometimes I have parents that are really worried about family team meetings ‘cause they don’t know in the beginning how they’re going to go. So sometimes, I play more of an advocate and support role for the parents, and sometimes I just am there to answer questions about level of engagement and/or barriers that I identify. Really just as like a family-by-family need. And it’s just kind of whatever that person needs or whatever the team needs, that’s the role that I play. But I always make myself available for those because I think they’re really important. In addition to family team meetings, the majority of participants shared that they attend court and actively participate or otherwise provide their direct input as part of the process. Several discussed the court reports they contribute prior to hearings, such as this participant: Again as a CPP therapist we send court reports like a week before the hearing. And of course they combine all the reports together. So before court we receive the big report where all the providers, everybody, case managers report, everybody’s report is there. So I did talk about number of sessions we did, mom’s attendance, if she missed any sessions what was the reason, if I missed any sessions I’m sure there’s a reason, something. And of course my experience about the sessions. Once in court, further input is sometimes assessed (e.g., “So, in the court hearing the judge is always – we have to do a monthly report but the judge is always asking ‘Hey, [provider name], what are your recommendations?’”). One provider noted that the COVID-19 pandemic decreased their involvement, despite being present virtually. This provider also shared that “there’s not very much asked of me at court. I think that has a lot do with the fact that we have pretty thorough court reports, and I think that the magistrate reads them.” Very few providers shared that they did not feel welcome in court (e.g., “Until the one judge said, ‘Yes, I would like for the CPP provider to be in court,’ nobody – I’d never gotten an invitation. I was

not wanted there. I was not needed or wanted”). The few providers who indicated they did not participate in court shared varying reasons for this. While one felt disinvited, others said they did not have the time (e.g., “when we added it up…that’s too many hours”) or that they had not yet had a chance because their ECC was newly formed. Related, providers reported they feel supported by their team when they have strong communication with responsive team members and feel they are an appreciated part of the team (e.g., “I think the biggest thing is that they want to hear from me. They’re interested to have everybody present, and they want to know what I have to say”). A provider expounded on both the need for ongoing communication and respect: Easy to communicate with. Respected. That’s something that’s big for me, is -- or that I feel that I enjoy about it, is as a clinician whether or not it’s the parent’s attorneys who might disagree with something or might not like it, they respect my input a lot. And so, because it’s the same attorneys. You know? It’s the same attorney for mom, same attorney for dad. And we work together. and both of those attorneys I’ve known for ten years. So I think that the respect is important there. But yeah. Overall, great. Great ECC community and connection. Notably, several participants said they specifically felt supported by the judge (e.g., “…[the judge] talked to me and she asked me about my concerns and everything which I’m very thankful for that”; “the judge is extremely supportive”). Conversely, when providers felt unsupported, they typically attributed it to differences in opinion or other team members “not getting” the therapeutic elements of the case. At times, this escalated to the point where providers felt their expertise was invalidated (e.g., “I feel unsupported when other team members question my judgment or perspective or opinions and they question them or challenge them or say that they don’t understand why I have that [opinion]”). Several participants discussed rapport-building techniques to buffer against these types of challenges (e.g., team building exercises, keeping communication lines open, leading team member trainings on therapeutic processes used in ECC). Working with Early Childhood Court-Involved Families Characteristics of Families Involved with Early Childhood Court Providers shared that “the families [I work with] are as varying as people are...the variety is huge.” They noted they serve a diverse range of families in terms of demographics, including age, race/ ethnicity, and socioeconomic status. A provider shared their experience of working with families of various backgrounds: They are all different, even socioeconomically. I’ve had from like trust fund kids to mom-is-homeless kids where there’s really no resources and Mom doesn’t have the resources to get the resources. So the same thing with like cognitive function of parent and child is from one end to the other end. Not all of them, but many of them, there’s some substance abuse… Corroborating this participant’s experience, history of substance use was frequently mentioned as a “common” characteristic among ECC-served families. Similarly, providers noted that a history of trauma, and intergenerational trauma in particular, was often present in families in ECC: That’s the number one factor. When they come in and then you start actually unraveling the layers of trauma you see this is intergenerational. But sometimes they don’t realize the connection between their intergenerational trauma and how they themselves parent. Providers shared specific reasons for referral that mirror some of FLORIDA INSTITUTE FOR CHILD WELFARE

the families’ common characteristics, notably substance abuse. However, other common reasons for referral include domestic violence and mental health issues. In many instances, providers spoke to a combination of two or all of these factors (e.g., “the big three: domestic violence, substance abuse, mental health”). As it pertains to domestic violence, “failure to protect” was noted by several participants. For example: And usually, even in the DV, it would be the failure to protect is why they would be removed, even from the victim, because usually, it’s not the first time. If it is the first time, we always offer the victim help. We always offer to support them through usually non judicial. But then, if there’s not compliance there, then they usually end up going the judicial route, and then, it’s usually the failure to protect. System and Family Challenges Given the complexity of the cases, providers shared that working with families in ECC is not without challenges. Some problems are systemic and usually centered around poverty—such as lack of transportation, housing, and employment. Because these families are expected to access or achieve stability in these areas, it can present numerous challenges at the individual family level. Several noted demanding case plans (e.g., “Like it’s a lot. It’s kind of like a part-time job for them to work their case and that’s hard.”), with one provider questioning whether the system is “setting [families] up to fail.” They explained: But the increased stressors about, “I don’t have any place to live” or “can’t get a job to take care of my family,” it’s lifelong. They all have criminal backgrounds too. That’s another thing. That’s another kind of, the more I talk, the more things I’m thinking about, but criminal background history, poor credit history, all of those issues make it impossible for them to really find adequate stable housing, as well as employment, and those are the things, the safety and the basic needs, that if those are not met, are we setting them up to fail? In addition to larger systemic issues, providers also noted more client-centered challenges, and “readiness challenges” in particular. Resistance or lack of participation is common, though some providers noted that an initial resistance might be expected given the mandatory nature of services (e.g., “anytime you have a court order to court mandated service you have to overcome that initial resistance”). In a similar vein, providers shared that when parents do not accept responsibility for their role in the case, it influences both their success in therapeutic services and the ultimate case outcome. For example, one provider said that a parent must “acknowledge the child’s trauma, acknowledge its affect, and then acknowledge [their] responsibility in it. And if I have a parent who’s not successful in CPP it’s almost 100-percent, because they don’t ever get that piece.” As previously noted, some providers experience difficulty in the duality of their role as both therapist and ECC team member, and fear of losing rapport with clients (e.g., “We’ve spent quite a few supervision meetings of staffing this case with the other clinicians and how are we holding this alliance with the family, and trying to engage mom, and build that rapport”). When asked how they maintain their rapport with clients, one provider said: Patience. To the best of my ability, as long as they’re clean and sober, I support their engagement with the child. I support the child engaging with them. I get out of the way when I need to be out of the way. Finally, as the above quote alludes, parents’ struggles with sobriety can present challenges when working with families. Parents are expected “to do such a hard thing to get their kids back but do it without what they’ve used as a coping skill.” Relapses 10

were described as setting back the case plan and the parent’s “appropriateness to be involved in their child’s therapeutic service.” Some providers do not engage with parents who are actively using substances (e.g., “they’re not going to be able to connect all the dots”). Another noted that if a parent is actively using substances and unable to have visits with their child, they cannot demonstrate the skills they learned in their therapeutic work. A provider summarized the conundrum of staying engaged with these parents: And so, getting them to be fully committed to a sober journey is a challenge sometimes to help them be in a space where, “How can I therapeutically work with you when you’re still actively using to the point that, you know, you’re not fully available to process these feelings that your kid is having right now?” But to still give the opportunity to be successful and not feel like, “Somebody else just gave up on me.” Despite challenges, providers shared several positive aspects of working with families in ECC. In general terms, several participants shared that they appreciate the supportive, wrap-around, traumainformed approach of ECC and the rewarding feeling of seeing the model work the way it is intended. They spoke of having numerous examples of “successful families” and improved parent-child bonds were noted by several (e.g., “I have seen beautiful moments”). One provider specifically appreciated how ECC allows them to support the parent-child dyad: “I feel like I’m good at holding the parent and the baby. That’s something that I think a mainstream counselor doesn’t get.” Still others shared the significance of setting families up for lifelong success (e.g., “Because we’re not just teaching to get their kids back. We’re teaching so that their kids never go back into care again as long as they are able to continue to do what they do”). A provider summarized finding professional meaning in the longterm successes of families: Oh my goodness. I wouldn’t do this if I didn’t have enough positive experiences to make it worth my while. Because this is not a lucrative field and it’s often a thankless task. So as far as positive experiences, when I get texts from people I worked with years ago and it showed little Johnny graduating from kindergarten, or the parent I haven’t heard from in years texts me to say, “Hey, I got my GED.” And all they need is acknowledgement to say, “Yay, proud of you” and then I may never hear from them again. Or it may be another year where they’ll tell me, “Oh, I got pregnant and we’re having a baby.” Those kinds of things where I see long-term success and stability with those families, that’s worthwhile. Therapeutic Modalities Circle of Security A few participants reported being trained in COS, which took place over a period of several days. COS providers described the program as a manualized, eight-part video series taking place over the course of two to three months, indicating use of the COS-P model. Participants described COS as a foundational piece of working with families. As an attachment-based parenting program, it aims to teach parents essential concepts around nurturing and help them identify their own triggers. For those who later implement CPP, it helps establish common terminology, which the therapist and parent can retrospectively refer to as their work together progresses. Among those who utilize COS with families, providers expressed high levels of comfortability with its implementation (e.g., “extremely comfortable,” “just seems to be routine”). In discussing the trajectory of COS, providers noted that it has a beginning and an end; although one noted that they deliver COS and CPP simultaneously (e.g., “I will try to get done with like two chapters of Circle of Security, and then I’ll go back to CPP. And so then, I’ll just kind of go a week here, a week there.”). Most providers gave a general overview of COS, though one provided a detailed FLORIDA INSTITUTE FOR CHILD WELFARE

accounting of each session (e.g., “[Chapter] 2 is an explanation of the circle…Chapter 5 is shark music…”). COS providers discussed using multiple assessments (e.g., ACES, PITA, Forms A and B of COS, Parenting Sense of Competency), though no singular assessment was discussed by all providers. The primary strength of COS was reported to be its accessibility for parents. The videos resonate with parents and send a message that “there are no perfect parents.” As one provider said, it gives the clinician the opportunity to tell a parent, “This is just another way of looking at your relationship with your child to help you understand what your child needs.” Despite the program strengths, providers also noted its weaknesses. For example, COS does not teach concrete skills (e.g., feeding, diapering) so additional programming is needed to address those skills, if necessary. Another COS challenge is the difficulty parents have breaking out of intergenerational parenting practices: “This is the way it ought to be. You need to mind. You need to shut up. You need to sit down.” And that’s like – that’s not cool. That is not going to help you keep a relationship with your child. And so, Circle of Security talks about switching the way you look at it, but it’s so hard for parents who – they’re not ready to give up their parenting legacy or their honor culture and “This is just the way it is.” And it’s a real struggle fighting that. COS providers shared examples of cases where COS did and did not work well, most of which centered around understanding the circle. For cases that went well, providers reported that parents recognized that their current parenting techniques were not working (e.g., “Oh, wow. Yeah. That’s true. I don’t want to keep doing it this way”) and they incorporated COS content in future therapeutic work and beyond (e.g., “So, [the parents have] grown tremendously. I really, no doubt – I mean, they still talk about Circle”). Conversely, the few examples of times COS did not work well revolved around the parents’ lack of adoption of the content. While one example discussed needing to use different programming due to parental cognitive impairments, another expressed a difference in participating in COS and implementing the tenets in everyday life: And I got to a place with her – she kept her appointments – I got to a place with her where she said, “Yeah, I shouldn’t have hit my kids. And I’m not going to do it again.” And I thought, “Wow, this is fantastic. This is – we’re going to get someplace.” And then, when she got reunified her attitude completely changed, which we know happens a lot, and that’s why I’m always – it’s like “This is a very vigilant time. We need to be very vigilant and this is why.” Because she came in and she was saying, “Oh, my kids are so bad, and they were making all this noise, and blah, blah, blah, blah, blah.” And I said, “Well, give me an example.” And she tells me something that happened. And I said, “Well, where is he on the circle?” And she said, “Circle? I don’t need no circle. I got my own circle.” I’m like “Ugh. We’re back to square one.” Child Parent Psychotherapy Most of the sample reported implementing CPP with ECC-involved families, with several indicating it is “definitely the primary modality to use.” CPP aims to help parents understand their own trauma, as well as that of their child, and improve parent-child attachments through dyadic work. A participant shared: Child/parent psychotherapy is all about the healing part of the relationship, the recognizing the trauma, and helping the child to make sense of the trauma. And that’s really through a dyadic process, helping the parent so that they can be the avenue for the healing as they should be. Providers described the 18-month CPP training as a combination of in-person training (or virtual if participating during the COVID-19 11

pandemic) and reflective supervision. All participants discussed implementation of CPP in practice, though training statuses differed: ● Five providers were or were very near being rostered ● Four providers were still working towards becoming rostered ● F our indicated they implement CPP with clients, including one who noted they completed training, but none specifically addressed their roster status Clinicians feel comfortable with implementing CPP: “Oh love it…because I believe in the core of the CPP…like whatever we have learned. I can see it in the families, so I love it…And I’m very comfortable in it, yes.” Very few suggestions were provided regarding improving clinician preparedness to implement CPP, though the importance of reflective supervision was noted. In general, providers reported that reflective supervision is a space for providers to informally staff cases and receive feedback from other clinicians and their supervisor. One provider shared that the space also allows CPP clinicians to process how their own identities or traumas might encroach on their work with families (e.g., “…Is it really that Mom is noncompliant, or a lot of your stuff was coming out in the case, so it actually shows up in the relationship? And the relationship rupture becomes a barrier”). Negative aspects of reflective supervision (e.g., desire for more direct feedback on cases) were rarely shared, and most perceptions were positive. For example: I would say that supervision piece is huge. And I think that that is for any clinician and for any, you know, treatment modality that that supervision makes a huge difference in regard to how confident and successful you are. Clinicians described CPP as occurring in three broad phases: foundation, intervention, and termination. The foundation phase, noted as lasting between four and seven sessions, is primarily conducted with the parent or caregiver (i.e., not with the child present), and involves a battery of assessments and rapportbuilding: “A typical session for foundational phase is very interviewesque, me asking mom lots of her own prior history, all sorts of assessments. Sometimes it teeters on the edge of feeling almost like a psychological evaluation, ‘Tell me everything.’” Clinicians described using numerous assessments, noting that the context of the case often dictates which assessments they implement with parents and children (e.g., “It kind of just depends on what their presenting things are”; “It depends on the child and the age of the child”). Nearly half of participants use the Adverse Childhood Experiences (ACES) assessment. Other specific assessments were mentioned sporadically, such as the Progress in Treatment Assessment, Parenting Stress Index, and the Traumatic Events Screening Inventory (TESI) or other trauma-related assessments. Again, providers emphasized the importance of flexibility in assessment based on client needs: I’ll do the ACE and angels in the nursery. And then after that, I will do a TESI, like that trauma screening. Usually, I do a TESI adult and a TESI on the kid one kind of at the same time. And then if there’s any other specifics that I’m like, “Okay. Maybe this would be helpful.” Or maybe I’m a little bit confused about if I feel like they need like a [posttraumatic stress disorder] screen, I’ll do that. If I’m concerned about post-partum depression, I’ll do one of those perinatal postpartum depression screens. The intervention phase, which can be “months long,” was described as “child-led” work in which the clinician observes parent-child interactions, finding “port[s] of entry” to engage the dyad in therapeutic attachment work by “join[ing] the child and parent’s perspectives.” Nearly all clinicians discussed how play is the primary vehicle for engagement during the intervention phase: “…We move to intervention phase, which for ECC cases would look like a lot of FLORIDA INSTITUTE FOR CHILD WELFARE

child-parent play therapy and observation and relational work and building the bond between mother-baby or parent-baby, whatever that is.” Another provider elaborated on play-based sessions: Generally, then I would ask the parent and child to play about something or to free play with whatever the child wants to and sit on the floor and direct that play – not direct it but narrate it like ‘Oh, we’re with the – there’s the daddy and there’s the mommy in the dollhouse’ so we would do some play therapy. The use of narration arose several times for clinicians, which looks different depending on the age of the child engaged in treatment. For example, while older children (e.g., 4- and 5-year-olds) can typically self-narrate (e.g., during play), some clinicians said they use a “speaking for baby” technique in which they narrate for a nonverbal child to help the parent identify the child’s cues. One provider explained: So if I do notice that baby’s reaching up to [mom] and maybe she’s distracted for a minute, me intervening and saying, “Okay, I think he’s trying to tell us something. What do you think he’s trying to tell us?” So lots of narrating for her “what are these points that we can kind of look for” and almost being a translator. Even though baby doesn’t have the words, what is he trying to tell us? Several clinicians also spoke of the “triangle of explanation,” a tool used to help comprehensively explain an experience, as useful during the intervention phase: But on the triangle, it’s you’re talking about the experience and then the feelings that come from the experience, and then the third side of the triangle is like what you’re going to do about it or how I’m going to help them or how Mom is going to help them, whatever. Finally, during the termination phase, which was not extensively discussed by any participant, the clinician discusses sustainability with the parent (e.g., “…just like how are you going to maintain this growth that you’ve had outside of these services?”). Clinicians shared CPP’s various strengths, such as the inclusion of infant mental health, attention to trauma in both parents and children, and accountability (e.g., “without acknowledgement [of harm], there is no intervention”). However, the most frequently noted strength was the support CPP offers caregivers, namely parents. CPP provides “a more positive, a more structured framework which allows [the parent] to always go back to making sure that they’re always looking at the needs of the child.” Another provider shared: To me, CPP is magical. That’s probably not a great thing – I probably wouldn’t say that in a courtroom, but there is, there’s this magic that happens when you just feel like the whole thing just kind of goes… and the parent is getting it. When my one little girl took the baby [doll] – where she had gotten hit with a domestic violence situation – and she took the baby and threw it down the hallway and the mom went, “Wow, she’s really upset.” There’s the magic…I’m like “Let’s talk about that.” So, the strengths are, I think, in so much of it being supportive, that there are people – that this is a gentle process…and if you trust the process you see great things happen. All CPP providers were able to identify examples of cases that went well, which typically centered on parents’ “lightbulb” moments and empowerment. CPP affords parents the time to focus on their attachment with their child, as well process their own and their child’s trauma, including their role in the latter. Several providers also noted examples of CPP working well when it built or improved upon the parent’s supportive network (e.g., co-parenting relationships with other caregivers).

Providers discussed several weaknesses of CPP as well, such as the number of assessments and challenges in implementing with disengaged parents. Regarding the latter, some providers shared CPP with a disengaged parent is not appropriate, even when it is a court requested service. Several participants also discussed challenges around the “longevity of services” and the concern that parents do not have enough time to adequately engage in CPP within the temporal confines of their case plan, particularly given the engagement challenges previously discussed. One provider expounded on this: I think potentially you’re utilizing this model, and potentially it can set someone up to fail because there’s still the need to be involved in participating in this treatment that they may not be successful with their child welfare goal. They have a year to permanency, and if they don’t jump on board immediately that their child is removed, it could be problems with them in the long run. Clinicians shared that when CPP cases do not go well, it is when CPP is inappropriate; primarily because the parent does not engage in treatment or otherwise refuses to accept responsibility. One provider shared: …some therapists have a hard time finding the line where they can say in their heart of hearts, “This is not going to work. There’s nothing I can do to make this parent safe.” Because regardless of how good a model is, it’s only going to work if you have engagement of clients. And I’ve seen that happen, unfortunately. Implementation Decisions In general, clinicians decide on therapeutic modalities to implement on a “case-by-case” basis, a process that can last the entirety of the therapeutic relationship. In the beginning, the decisionmaking process is typically informed by biopsychosocial and other assessments; though, assessments are sometimes utilized later in treatment. Many clinicians shared that they will not engage in parenting education or interventions when parents are unwilling or unable to engage, particularly when they are actively using substances. As one provider said, “…if they’re not submitting clean [urinalysis], then that’s usually a red flag for me because I don’t feel that they are going to be present and do any productive work.” Others noted similar sentiments for relapses or other challenges (e.g., parental mental health concerns, lack of understanding of content) during the course of treatment and the need to shift therapeutic services to something more individualized. A provider described working with a mother who is not recognizing her child’s stressors: I say, “How do you feel about – there’s another intervention called therapeutic infant massage. How do you feel about –?” And so, I explain to them what that is. “How do you feel about our next session we might try that, and then you try that at home and see how that works?” So, that’s how I do that. Participants spoke to the variability in implementation of CPP. Again, clinicians reported being trained in multiple therapies or modalities. When asked if any are implemented with ECC-involved families, the general sentiment was that therapy is always rooted in CPP principles, but other modalities may be woven in as needed. For example: “Well, I use child-parent psychotherapy [as] the overarching approach. But I use other things within the model of CPP to support and to facilitate the process.” Another corroborated this: So far, it’s been solely CPP. We’ll integrate here and there a little bit of trauma-focused CBT, just smaller things of certain psychoeducation activities or lots of safety, and just different tools from that, but it is very much the basis of CPP, very much FLORIDA INSTITUTE FOR CHILD WELFARE

that attachment lens. So less therapeutic clinical modality, more education… Importantly, a number of participants reported that they use both COS and CPP (e.g., “I love Circle of Security…We love CPP as well. And so we’ve incorporated fully into my program, and so, anyone that’s enrolled and meets the needs of my program we use those treatment modalities”). In some instances, they are delivered sequentially (e.g., “A lot of times, I always do Circle at the beginning and then, I’ll go into CPP as well”). However other clinicians indicated that they interweave the sessions of the different modalities. Notably, one provider expressed confusion or frustration with dual implementation of COS and CPP, namely a lack of clarity on appropriate sequencing when they were a newer clinician: There really in the beginning wasn’t a clear, objective, “This is where you start. You know, this is” -- as far as CPP and circle, you start with one, you start with the other. “Do you do all of circle of security before you start CPP? Do you do it however you think it’s best for the family?” There wasn’t like a really clear, “This is the path that you should follow,” in regard to services. Provider Needs When asked to describe what they need to better serve families, responses typically centered around support, both for the team as a whole and fiscally for therapeutic providers. In terms of the team, providers shared that more orientation would be helpful: I think maybe more additional training and support – not so much for CPP, but for the ECC part of therapy. Like, what does the therapist’s role look like in other circuits? What was that designed to look like? And maybe just like, more direct support that way. Further, team cohesion—including understanding team members’ roles as well as their strengths and weaknesses—might be beneficial: “I think we need to become a more unified team before we will be a successful ECC. I think we need more training on becoming a more unified team.” Another shared the ECC team dynamics sometimes mirror the relational dynamics of treatment: We’re still checking our boxes. We’re tweaking it a little bit, depending on who needs what and with which family, and just defining those things. “Where do those lines lie? Where are those struggles? How do we navigate that?” ‘Cause it’s very CPPesque. If we are not speaking that unspoken, we cannot do anything with it… In addition to support navigating team dynamics, providers noted challenges for clinician funding (e.g., “…and, you know, funding is a big thing for a lot of people too, is navigating the funding”). Participants spoke of a lack of compensation for ECC activities that are not therapeutic in nature, including attending family team meetings and court hearings, as well as hidden costs of service provision: “So when there’s something that I identify that they really, really need, if I can’t find a resource for it then I provide it. And that is expensive, which means some of my cases cost me money.” While some providers discussed the challenges of billable hours by setting (i.e., private practice versus agency-based work), there was not a discernable pattern in their concerns. DISCUSSION Summary of Findings Clinicians in the present sample represent diverse educational and therapeutic training backgrounds, though many reported a history of work within the child welfare system. There was also varied tenure within their ECC teams, as well as how consistently they 13

work with ECC-involved families. While some providers have been a part of their ECC team since or near inception, many reported being onboarded through their employer, often without adequate orientation to ECC and the team. While clinicians noted their appreciation for the intensive team approach—including benefits for clients as well as team members—they identified numerous challenges such as a lack of understanding of roles. Clinicians reported that their primary function on the team, in addition to direct work with families, is to provide family progress updates or reports at family team meetings and court hearings. Some shared it can be difficult to articulate their perspectives on cases given their typical complexity. For example, some felt that their fellow teammates did not understand therapeutic participation is not a simple checkbox and that parents’ implementation of skills is necessary. Expert consultant Meredith Piazza (personal communication, May 21, 2021) agreed this is an ongoing issue: “Why have the families participate in therapy if they are not going to have to implement it? This has been an ongoing issue utilizing these models and then reporting to Courts regarding the families’ engagement.” Consultant Piazza also agreed with the providers’ reports of challenges when working with attorneys: “Many times the parent attorneys are looking for holes in the therapist’s report. It can feel like the therapist is the one on trial.” In a similar vein, providers report feeling pressured to be the team member responsible for initiating difficult conversations about clients (e.g., bringing up “the elephant in the room”). As it relates to confidentiality barriers specifically, consultant Piazza noted the importance of the ECC team “[sharing] all of this out ahead of time so that the family and the providers fully understand the confidentiality limits and therapeutic process.” In terms of service provision, clinicians shared that their ECCinvolved clients represent diverse demographics, though did note some common characteristics and referral reasons (e.g., substance use, domestic violence, mental health). Working with families can be challenging given systemic barriers (e.g., transportation, poverty) as well as client factors (e.g., lack of engagement, struggles with sobriety). Expert consultant Diane Koch (personal communication, May 20, 2021) shared that parental lack of engagement may be due to fear of the system, which can complicate the therapeutic relationship: “Parents are in a legal system and are reluctant to say anything to be used against them. It requires quite an art to be able to carve out a therapeutic relationship in such a context.” Consultant Piazza concurred with providers that sometimes it seems like the system is setting up families for failure, especially for parents with substance abuse issues: So many families may make huge gains in the therapy component, but not have stability or be able to remain substance free. Then this does not seem fair or ethical… put them through weeks or months of parenting work to strengthen their relationship with their child and then have a different barrier prevent that reunification. That can be hard for the therapist. Expert consultant Diane Koch (personal communication, May 20, 2021) believes more tailored case planning is a potential solution: I think at least in part this stems from a one size fits all mentality. Basically everyone gets the same plan. The plan should be made by the team, which includes the family. In this way, things can be more individualized and specific to the family culture. Still, providers report witnessing “beautiful moments” in their work with families and generally reiterated their appreciation for the ECC model. Notably, expert consultant Dr. Kimberly Renk (personal communication, May 23, 2021) shared that provider-described needs center around an overarching need for reflective supervision or consultation and reflective practice-based team building.

In general, participants in the present sample described the COS-P essential elements very similarly to its intended structure and goals. They spoke to the short training;18 the usual group nature of the work, with optional individual work;62 the series of eight videos and subsequent reflection activities;17 and the goal of educating parents on attachment and improving their responses to children’s cues.14 Notably, Consultant Koch (personal communication, May 20, 2021) reiterated her concern regarding parental distrust of the system and how that can interfere with COS in particular: A prerequisite for learning according to COS is a feeling of safety. This is the basic message of COS. Therein lies the conflict. We are trying to give a learning opportunity to people who can’t trust a system that took away their child and is judging them. Most participants in the sample discussed their use of CPP and, similar to COS implementation, there is initial evidence of broad fidelity to the model. Participants described CPP as being rooted in concepts of attachment and trauma,27 and discussed the three major phases of treatment (i.e., foundation, core, termination).28 The core, or “intervention” phase, was noted as largely reliant on play-based work.28 While providers spoke about fidelity generally, their stories alluded to multiple fidelity strands, such as reflective practice, dyadic/relational, and trauma framework.38 Overall, without assessing specific points of fidelity, present findings indicate that providers are implementing CPP in alignment with its essential tenets, corroborating previous findings.42,36 Limitations The present phase of this evaluation is not without limitations, namely a lack of generalizability to all ECC clinicians in Florida. While approximately 25% of the sampling frame participated, which is only slightly lower than other qualitative workforce studies conducted at the Institute (e.g., 35% in Radey, LangenderferMagruder, & Wilke, 2020)63, the potential for self-selection bias is present. In addition, the sample size is small, though it does meet recommended minimums for saturation in qualitative analyses.64

Chapter 3: Clinician Survey Lisa Magruder, Taylor Dowdy-Hazlett, Colleen McBride, & Michae’ Cain.3 In June 2021, the Institute team launched a survey of therapeutic providers to assess their general fidelity to the therapeutic approaches. Survey items were primarily informed by the qualitative findings, expert consultation, and modality protocols. Qualitative findings indicated providers primarily use COS-P and CPP, often in conjunction with one another. Thus, we relied on a single survey to capture the use of one or both of the modalities, including nuanced closed- and open-ended items regarding implementation. The survey was reviewed by the expert consultants before finalization. In addition to fidelity-related items, the team used this data collection opportunity to ask targeted questions to further explore patterns noted in the qualitative data (e.g., impacts of COVID-19 pandemic). METHODOLOGY Sampling To establish a sampling frame, the OCI requested a list of ECC therapeutic providers whose work focuses on parent-child relationships from all community coordinators in Florida. Community coordinators responded directly to OCI or to Dr. Magruder, principal investigator, and provided a list of clinicians (N = 52). We removed

3 Suggested Citation: Magruder, L., Dowdy-Hazlett, T., McBride, C., & Cain, M. (2021). Chapter 3: Clinician survey. In L. Magruder, 2020-2021 Early Childhood Court Evaluation (pp. 14-19).

FLORIDA INSTITUTE FOR CHILD WELFARE

two clinicians: one duplicate and one who is serving as a consultant on this evaluation. In addition, two providers responded to the PI via email that they no longer provide ECC services. Following survey distribution, four individuals were unreachable (i.e., bounced emails), leaving a final sampling frame of 44. We recruited participants in June and July 2021 by sending Qualtrics e-mail invitations to all potential participants. Each received an initial invitation, with three reminder e-mails sent at weekly intervals. The final sample size included 6 clinicians (13.63% response rate). Importantly, only three providers completed the survey in its entirety. The remaining three completed between 14 and 65 percent of the survey. No incentive was offered for completion of the survey. Data Collection Using qualitative findings (see Chapter 1), expert consultation, and modality protocols, the Institute team developed a singular survey to assess procedural fidelity to the primary therapeutic models: COS-P and CPP. For CPP fidelity, the evaluation team relied on portions of the publicly available fidelity forms,28 focusing primarily on procedural fidelity strand given the variation in nuanced service delivery noted in the provider interviews. However, items were included to assess reflective practice and trauma framework. Providers who consented to participation were asked to provide demographics, details of service provision (e.g., circuit(s), time as an ECC clinician), and perceptions of preparation to join the ECC team. Providers were then asked to indicate if they provided COS-P, CPP, or something else, with an option to select multiple modalities. Based on their responses, participants were asked follow-up questions relevant to each modality, including items on training, supervision, fidelity, clients served, assessments, and implementation strategies. Given the ongoing COVID-19 pandemic, the survey included an item regarding impact of service delivery by modality. Though the survey was initially intended to collect more quantitative data, the research team opted to include short-response items. Given the small sampling frame, the team anticipated being limited in our ability to conduct inferential statistics with quantitative data; thus, supplemental short-response qualitative items could provide richer data in the event of a small sample size. An IRB amendment was submitted and approved for this phase of the evaluation. Among those who completed the entire survey, the average duration was slightly over one hour (62.67 minutes). Data Analysis The research team used SPSS to run frequencies and descriptive statistics as appropriate. Given the very small sample size, the researchers did not run inferential statistics. Short-response items were coded using thematic analysis techniques.57,58,59 However, the small sample size prevents establishment of well-supported patterns and themes and the results presented below should be considered tentative, though the findings are triangulated with additional data sources in Chapter 5. As a form of debriefing,61 team members reviewed the findings for validation. Results were also shared with consultants to provide contextual interpretation of the findings and assist in guiding relevant discussion points. RESULTS Background and Preparation The majority of participants identified as female (83%, n = 5), with half (50%, n = 3) identifying as White. Ages ranged from 41-70, with an average of 54-years-old (SD = 10.71). All providers held a graduate degree in mental health counseling, psychology, or social work and are licensed mental health providers in their fields. Two participants (33%) previously worked in child welfare. On average, participants have been therapeutic providers for 13.58 years

FLORIDA INSTITUTE FOR CHILD WELFARE

(SD = 7.66, Range: 2.20–20.0 years), and with ECC specifically for 3.19 years (SD = 1.69, Range: 1.5–5.0 years). Clinicians provide therapeutic services to ECC-involved families through private practice (66.7%, n = 4) or through their agency (33%, n = 2). Participants provided services in seven circuits in Florida in five disparate regions of the state (Northwest, Northeast, Central, Suncoast, Southwest regions). Satisfaction with orientation to both the ECC model and to their team was inconsistent across respondents (see Table 1) and descriptions of the process were similarly varied (e.g., “thrown in”; through meetings and reading materials; “multiple levels” of orientation, including workshops and court and agency assistance). One participant shared they would have appreciated “any orientation.” When asked what would have better oriented them in their role as a clinician, specific ideas included: ● P rovision of additional information, specifically about the typical dependency system (i.e., not ECC) and screening cases for fit with the model ● W orking toward improved understanding of team members’ roles and acknowledging role qualifications ● E stablishing a more cohesive understanding of ECC as a trauma-informed model Table 1. Early childhood court orientation satisfaction (n = 6) How satisfied are you with the

Very

orientation you

Dissatisfied

Neither Dissatisfied

satisfied nor

Satisfied

dissatisfied

received to...

Very Satisfied

…the ECC model?

16.7%

0.0%

33.3%

16.7%

33.3%

…your ECC team?

16.7%

33.3%

All six providers reported that they use CPP in their practice with ECC-involved families. Three providers (50%) indicated they use both COS-P and CPP. Of those who use both, two (66.7%) implement them concurrently, while one (33.3%) implements them sequentially (“…I usually implement COS then CPP…”). One provider who does not offer COS-P shared: I wish that COS-P could be used as a tool for triage, eight weeks to get to know folks and then identify what services might benefit the individual and child before engaging in CPP, a parent might need a cognitive oriented class in basic child care, another might benefit from individual psychotherapy before introducing trauma focused CPP and other’s might be ready to engage in trauma focused CPP after the eight weeks in COS-P and completion of the foundational phase. Circle of Security-Parenting Training Two COS-P providers completed the majority of the COS-P survey items. Both providers stated that they attended the four-day training and received six contact hours per day. One provider was currently receiving ongoing supervision for COS-P while the other had previously engaged in ongoing COS-P-specific supervision. However, neither provider had participated in any fidelity coaching. The participant currently receiving supervision noted they “normally consult with [their] trained COS-P supervisor to discuss treatment to ensure fidelity and best practices.” Delivery Both COS-P providers deliver services in an individual counseling format, though one provider conducts group and in-home COS-P. 15

Both reported they deliver COS-P to parents and foster parents, with one indicating they also work with prospective adoptive parents. One provider discussed the major differences in delivering COS-P to different types of caregivers: When I use this with people who have never parented before it requires much more discussion time than with people who are familiar with littles. Also, there is some difference in parents who have a generational history of abuse and those that just want to “be better” (private referrals usually). When implementing COS-P, the providers noted that their overarching objectives are relationship focused: “interrupt[ing] the processes of intergenerational trauma” and “…ensur[ing] that families who come into ECC are provided with the tools [they] need to clearly understand the importance of attachment relationship, thereby, providing them with the hope that they can make a difference with this knowledge.” Providers reported that their ECCs do not require pre- and posttest assessments for COS-P, though both use a version of AAPI in their practice, and one reported using ACES. In terms of chapter coverage, providers shared that it typically takes one session to cover any particular COS-P chapter, sometimes two “depending on [the] parent.” Providers agreed that some chapters take more time to cover than others (e.g., “The last three chapters take the longest. Shark music and a greater comfort discussing the issues account for the time difference”). One provider shared that they regularly integrate a trauma-informed approach into their COS-P work, noting that it “flows through all we do” and provided examples of specific practices (e.g., allowing clients to take a processing break when triggered). When it comes to integrating cultural responsiveness into practice, the same provider said: I always ask my clients to teach me their culture. I discuss with my clients how every family has their own culture that is unique to only them and that I want to know about it. By opening the door from the beginning we can have dialogue about what is culture and what is developmentally appropriate within that specific culture. When facilitating the first chapter, providers shared that they “follow the guidebook initially then recover/recap/process the material in CPP” or utilize “relationship engagement to build rapport.” They do not typically skip any content and reported no facilitation problems. Only one provider answered the remaining questions pertaining to COS-P. They shared that for each chapter facilitation, they “follow along the guidebook and then process it again in CPP.” They reported that they do not skip content or encounter facilitation issues. For some chapters, they reported additional strategies, including for the final chapter, where they “congratulate my clients on being SUCCESSFUL and talk about building on that success in the future.” This provider described the successful completion of COS-P as, “First, the completions of 10 sessions consisting of pretest, 8 COS lessons, posttest.4 Second, they need to be able to verbalize to me how the material is applicable to them in their own personal situation.” If there are ongoing concerns for any ECC-involved family, the provider will “recommend a more intensive skills-based curriculum if necessary that focuses on more concrete aspects of parenting.” Though COVID-19 impeded the facilitation of group sessions, the provider noted that using Zoom allowed for provision of COS-P to a “much broader audience.” Finally, they shared that, they “love” the COS-P model, explaining: It is so beautifully simple that my youngest parents and lower functioning parents can gain insight from it. While it is deep enough that parents who self-refer for pre parenthood therapy and hold master’s degrees gain useful insight as well!

Child-Parent Psychotherapy All respondents indicated that they provide CPP (n = 6), either alone (n = 2) or in combination with another modality (n = 4). Four CPP providers completed the majority of the CPP survey items. These providers shared that their overall objectives as a clinician when delivering CPP services to ECC-involved families included 1) improving parent-child relationships and building attachments, 2) addressing trauma, and 3) improving child safety. For example, one provider noted elements of all three objectives: “Education and processing of trauma, building attachments and bonds between children and caregivers, creating a sense of safety and assisting caregivers in understanding the purpose of behaviors.” Training All four providers completed the 18-month CPP training. Most reported attending three classroom trainings, with one provider attended virtually due to the COVID-19 pandemic. Among in-person trained providers, two shared they engaged in bi-weekly reflective supervision. Three of the four participants are currently rostered to provide CPP services in Florida. Delivery Participants provided details on how they deliver CPP, including setting and duration, engagement with clients, and phase-specific work. All four providers reported tracking their fidelity to the model using CPP fidelity forms. Setting and Duration Participants provide CPP services in a variety of settings, including adoptive homes (25%, n = 1); birth family homes (25%, n = 1); foster/ kinship care homes (50%, n = 2); outpatient clinics (50%, n = 2), and community-based settings (50%, n = 2). None of the participants indicated they provide services at school-based settings. They indicated that the range of time in which they engage families in CPP varies. Some had set minimums (e.g., “20 minimum,” “at least one session past reunification”), while some gave more explicit approximations (e.g., “25,” “approximately one year to 18 months”). Similarly, the “typical” number of sessions varied (e.g., “14 minimum,” “20,” “approx 50,” “one per week”). Sessions generally last one hour but can range from 45 to 90 minutes. Engagement with Clients In addition to the child, CPP providers work with the child’s parents (100%, n = 4), foster parents (75%, n = 3), siblings (75%, n = 3), or others (25%, n = 1, “grandparents or potential adoptive parents”). Some providers shared that session content has to be tailored to specific roles: Pre adoptive parents is a great deal of psychoeducation on trauma, attachment, and the specific child’s needs as well as work building the attachment and coaching. Relative caregivers who have parenting experience is about teaching them the differences in raising this child than their own bio children. Foster parents I work with a great deal on forming a coalition with the bio family regardless of how the case is going. Education, education, education! Bio parents identifying their intergenerational traumas and helping them to identify their own traumas in order to interrupt the cycle for their children. SUPPORT! When the bio family will have their rights terminated I believe that it is imperative that we honor their need to grieve and hold them through the process… Another provider concurred: “The type of psychoeducation and support may differ depending on the role of the adults. The child remains the focus for all though.” Most providers (75%, n = 3) complete sessions with more than one parent-child dyad per family (e.g., two parents and one child, one parent and three children) within the same setting. This practice was

4 Provider previously indicating using AAPI2 pre-form A and AAPI post-form B, as well as ACES.

FLORIDA INSTITUTE FOR CHILD WELFARE

typically described as a way to more holistically assess or support the family. For example: “Assessment of the dynamics of the greater family constellation initially then as the children move back into the home to assist with transitions and the dynamics of the entire family in the home.” CPP Phases Only three providers responded to items regarding specific CPP phases. Note, for items where ranges were provided (e.g., “6-16”), a midpoint was taken to calculate means.

their interviews with instruments such as the Posttraumatic Symptom Scale-Interview (PSSI), the Life Stressor Checklist (LSC), the Center for Epidemiological Studies – Depression (CES-D), and the Beck Depression Inventory. Providers reported they regularly use several other assessment tools when working with ECC-involved families. Two providers reported they utilize the Crowell and two reported they use the Parenting Stress Index. Other assessment tools reported by only one provider each include:

Foundational Phase

● Patient Health Questionnaire-9 (PHQ-9)

On average, the foundation phase includes 8.2 sessions (SD = 2.47) over 7.5 weeks (SD = 3.12), with an average session length of just over one hour (M = 65.0 minutes, SD = 8.66). While one provider indicated that only the caregiver is present during foundation phase sessions, the others shared that the child might be involved as well (e.g., “caregiver for sensitive material, child present for orientation to therapist and parent child observation,” “caregivers and parents possibly child depending on the age”). One provider shared: “I work with the bio parent, caregiver, and the child. Almost never just one side of the equation.” When assessing and engaging families during the foundational phase, providers indicated they usually or always complete the associated CPP procedural fidelity checklist items,40 which include:

● Traumatic Events Screening Inventory (TESI)

● Elicit caregiver perception of need for treatment ● E licit caregiver description of family circumstances, challenges, and strengths ● Provide a sense of positive expectations about improvement ● Share with caregiver rationale for screening child trauma

● Parenting Sense of Competency ● Parent-Child Relationship Inventory ● Osofsky Relationship Scale Assessment ● General Anxiety Potential Inventory (GAD7) ● Child Abuse Potential Inventory (CAPI) ● Adverse Childhood Experiences (ACES) ● PICCOLO ● Nurturingskills ● Adult Adolescent Parenting Inventory Following the assessment period, providers begin to engage parents in the process of developing a treatment plan, using this time to collaboratively determine goals, discuss expectations and next steps, and gathering parental perspective (e.g., “draw parallels between parent and child experiences and feelings and asking how they hope the child will understand the experience”).

● Assess child symptoms based on caregiver report

When considering potential sources of challenge in the foundational phase, providers shared which aspects of their capacity they take into consideration (see Table 2). One provider shared, “Fidelity is an interrated process, all strands are important and best practice dictates monitoring each for potential challenges.”

● A ssess child trauma symptoms based on standardized measure completed by caregiver

Table 2. Foundational phase reflective practice fidelity (n = 3)

● A sk caregiver to jointly complete a child trauma screening instrument ● Consider caregiver’s response to child’s trauma history

● Assess child developmental functioning ● T alk to caregiver about connection between child’s symptoms and history

Therapist Reflective Practice Capacity

Frequency of Yes (n)

● Discuss trauma reminders

Emotional process fidelity (e.g., caregiver or child is triggered or shut down)

66.7% (2)

● A ssess for child safety risks to engage in trauma-informed treatment

Dyadic-relational fidelity (e.g., caregiver and child have competing agendas, are trauma reminders for one another)

100.0% (3)

● Observe child and caregiver interaction

Trauma framework fidelity (e.g., caregiver or child’s history unknown)

66.7% (2)

Procedural fidelity (e.g., scheduling challenges, family structure complexity)

66.7% (2)

Your reflective practice capacity as a clinician

66.7% (2)

● Discuss the impact of child trauma treatment on caregivers ● Share rationale for asking about caregiver symptoms ● Introduce caregiver symptom measures ● Process information gathered during assessment/engagement All providers use the Ages and Stages Questionnaire (ASQ) to assess child development, though each uses different instruments to assess child trauma, including: ● Infant Toddler Social Emotional Assessment (ITSEA) ● Brief Infant Toddler Social Emotional Assessment (BITSEA) ● Strengths and Difficulties Questionnaire (SDQ) ● T raumatic Events Screening Inventory--Parent Report Revised (TESI-PRR) ● Trauma Symptom Checklist for Young Children (TSCYC) ● Ages & Stages Questionnaires--Social-Emotional (ASQ-SE) Caregiver depression and posttraumatic stress disorder are measured using clinical interviews; some providers supplement FLORIDA INSTITUTE FOR CHILD WELFARE

If CPP were contraindicated for an ECC case following the foundation phase, one provider shared they would “report that to the referral source and give my recommendations to the court.” Another shared, “I have not come across a case that would not benefit from CPP. I might not address issues related to trauma as the caregiver or child was not prepared to address the issues related to the trauma.” Finally, two providers shared that the foundational phase can be long, with one noting: “It is important to make known that not everyone will make it past the foundational phase. Some people are simpl[y] not ready for the work. Or may need to stay in the foundational phase much longer.” Core Phase Providers indicated a wide range for the typical number of sessions/ 17

weeks for the core phase, from 7 to 30 (M = 17.5, SD = 11.46), with sessions lasting between 60 and 90 minutes (M = 65.0, SD = 8.67). All providers (n = 3) indicated that the child and caregiver are present during this phase.

Only two providers reached a planned termination phase with a family and the third “cannot wait to successfully engage in a planned termination.” One provider described the termination phase as “a bittersweet celebration.”

Providers “usually” or “always” follow the recommended procedural fidelity checklist items for introducing the child to CPP:

The two providers who have reached this phase report “usually” or “always” following the fidelity strands for terminating treatment, which includes:

● Prepare for session/select toys for treatment ● Explain the reason for treatment to child (if age appropriate) ● Tracked child response to introduction to treatment

● Reflect on termination ● Plan termination with caregiver

● Support caregiver during introduction to treatment

● Plan treatment evaluation (posttests)

● E xplain to caregiver any negative reactions/behaviors of either child or caregiver

● Complete treatment evaluation (posttests) ● Tell child about termination

● Process sessions with supervisor or colleague

● J ointly plan termination with caregiver and child (if older than infancy)

Providers shared how they introduce CPP to the child. Two specifically use the CPP triangle, and all incorporate elements of establishing a sense of safety or control for the child (e.g., “…I give them permission to be ‘in charge’ and to let me know if they need to take a break or end a session,” “…grownups want to understand and help the child feel better”). During the course of CPP, providers include specific therapies and techniques. Two providers reported they use cognitive behavioral therapy (CBT) and two reported using play therapy. Other therapeutic techniques included, but are not limited to, dialectical behavioral therapy (DBT), motivational interviewing, and acceptance and commitment therapy (ACT). All providers noted that they assess a family’s progress by monitoring the child’s behavior changes and caregiver reports. Some look for changes in the child-caregiver relationship and use data and document review (e.g., “tools to assess symptoms,” “review planned objectives and goals established prior to the start of core phase”). There is variability in how frequently providers formally re-assess potential sources of challenge to engaging in CPP with a family (e.g., every session, every third session, and ad hoc “as the need arises”). There was similar variability in how often providers formally assess their own reflective capacity (e.g., “never,” “each session,” “twice per month”). When reflecting on their CPP practice, all three contemplate their awareness of emotional reactions and personal and cultural biases; two consider their ability to consider multiple perspectives. In cases for which CPP becomes contraindicated during the core phase, providers “drop back.” They “end the dyadic sessions” or otherwise shift the “focus on strengthening the relationship and reductio in addressing any issues related trauma.” Two providers noted they would share the change of course with the team/referral source.

● Process the goodbye ● Countdown the sessions with the caregiver and child ● D iscuss the course of treatment and the family’s treatment narrative ● Plan for future and discuss trauma reminders with caregiver ● Conduct final session Reflective Supervision All three providers receive reflective supervision in both group (n = 3) and individual (n = 2) formats. Supervision is provided by a CPP trained reflective supervisor though the supervisor’s position varied (e.g., agency supervisor, clinical infant mental health supervisor). The frequency of regularly scheduled reflective supervision meetings varies from weekly, to bi-weekly, to monthly, although all providers indicated that they can access reflective supervision outside of formal supervision sessions. Providers are satisfied with their reflective supervisors’ ability to meet their needs (see Table 3). They shared that reflective supervision “allows me somewhere to go with the weight I help the families carry” and that it “keeps hope alive.” None offered ways in which reflective supervision could better meet their needs in providing CPP to ECCinvolved families, though one expressed: It is a challenge identifying a reflective supervisor who has grown into the role from an initial reflective practitioner, and flowering into a reflective supervisor. It is one thing to understand and perform the mechanics of reflective supervision it is another to truly come from a reflective space as a way of being in relationship. Table 3. CPP reflective supervision satisfaction (n = 3)

Finally, providers shared that the foundational phase is complex. One participant said: “If a family stays with the process to the core phase there is usually growth, but it is never quick and the pace varies greatly.” Another shared:

How satisfied are you

Maintaining fidelity during the core phase can be challenging. The flow is very different from the structure of the foundational phase, it can become challenging to keep the trauma in mind; the work can be painful given either of the narrative being shared, parent and child’s.

Processing emotional responses

0.0%

33.3%

66.7%

Considering alternate perspectives

0.0%

33.3%

66.7%

Seeking new knowledge

0.0%

33.3%

66.7%

Seeking new skills

0.0%

33.3%

66.7%

Addressing cultural differences or biases

0.0%

33.3%

66.7%

Termination Phase Only two providers shepherded a case to this phase. These providers shared different timelines for the 60-minute termination sessions, with one sharing that this phase typically includes 3-4 sessions in as many weeks, and the other sharing it includes 5 sessions over 10-12 weeks. As in the core phase, child and caregiver are both present for termination. FLORIDA INSTITUTE FOR CHILD WELFARE

with your reflective supervisor meeting your needs in the

Very Dissatisfied

Neither Dissatisfied

satisfied nor

Satisfied

dissatisfied

Very Satisfied

following areas?

Three providers responded to questions about the impact of the COVID-19 pandemic on their delivery of CPP. Responses focused on the now-virtual nature of providing CPP; one provider mentioned that conducting virtual therapy “allowed for observation of home settings but was also challenging for modeling interactions.” One provider mentioned that conducting virtual sessions meant that “I am getting referrals from farther away than I have in the past because no one was having in person sessions. These referrals were from areas with poor coverage in infant mental health.”

understanding how therapeutic modalities are being implemented with ECC-involved families. The evaluation team recommends using the present findings to refine survey efforts, including a reduction in the time needed to complete and implementation of incentives for participation.

Chapter 4: Therapeutic Networks Laura Kihlström, Shanda Vereen, Caitlynn Carr, Joanna Mackie, & Jennifer Marshall.5

DISCUSSION Key Findings All providers held graduate degrees and received prescribed training in their respective modalities. Similar to findings from qualitative interviews with providers (Chapter 2), there was variation in both process of and satisfaction with orientation to the ECC model. In general, COS-P providers indicated fidelity to the model by viewing and debriefing the eight-video series with clients, helping them connect the material to their own parenting practices. Practices were described more similarly for COS-P, likely given its semi-structured approach. CPP providers were more varied in their approach with families, including service delivery setting, duration of treatment, use of assessment tools, and specific therapeutic techniques. This is congruent with the findings from qualitative interviews with providers, where CPP was described as a “framework” as opposed to a structured modality. That is, the primary focus remains on building parent-child attachments through a trauma-informed approach, though the nuance of session content and delivery is highly dependent on the needs of individual families. Despite this variation, CPP providers self-reported procedural fidelity with respect to certain foundational, core, and termination phase activities.28 Assessments used throughout CPP varied, though the CPP fidelity forms are not prescriptive in terms of required tools. For example, during the foundational phase, the TESI-PRR is specifically offered as an instrument to screen for child trauma history, though only one provider in the current sample indicated using this. However, there is an option on the fidelity form to indicate a different instrument’s use. Finally, reflective supervision among this small sample suggests it is a supportive mechanism toward clinician wellbeing and skill-building. Limitations The present survey findings are limited by a very small sample size, and even smaller subsample sizes by modality, which introduces the potential for non-response bias. While a response rate of approximately 14 percent is slightly lower than average web-survey estimates of 20-30 percent,65 recruitment efforts may have been hindered by a small sampling frame. In addition, the evaluation team was made aware of another recent evaluation survey regarding ECCs, which may have caused participants confusion regarding whether or not they already participated. The evaluation team attempted to ameliorate this with a preemptive e-mail to the sampling frame. Still, the research team is encouraged by the disparate geographical representation of participants. Moreover, one team of researchers found response rates declined during the COVID-19 pandemic,66 which is a relevant, but external factor to recruitment efforts for this evaluation. In addition, the findings are limited by potential self-selection bias. For example, while providers indicated high fidelity to the COS-P and CPP and satisfaction with reflective supervision, it is possible that the present participants are more experienced or otherwise invested in their work with ECC-involved families, thus impacting their practice and positive perceptions. Though they are limited, these findings supplement previous qualitative data toward better

This chapter presents findings from the qualitative subcomponent of the evaluation completed by the USF team. The findings of this chapter describe the therapeutic programs, approaches, methods, and strategies used within Florida’s ECCs to 1) promote parent mental health; 2) identify the ways that ECC supports relationships among caregivers and with other ECC team members; 3) share the perspectives of ECC parents/caregivers and providers on the strengths and weaknesses of ECC therapeutic programs/methods in relation to racial equity/inequity; and 4) provide site-specific results for program promotion, planning, and improvement. METHODOLOGY The evaluation drew from individual interviews with caregivers, parents, and providers. To be eligible, participants had to be part of a team that surrounds a child enrolled in Florida ECC between 2019-2021 (current or closed cases). Eligible individuals to take part in the evaluation included biological parents, foster parents, kinship caregivers, and their ECC team providers which included case managers, community coordinators, counselors, therapists, and other individuals identified by the parents/caregiver(s) as one who plays a therapeutic role in their ECC experience. Sampling Recruitment for the evaluation was conducted via email. Invitation flyers with the evaluation team contact information were distributed to potential participants through the ECC network (parents, foster parents, community coordinators, therapists etc.). Parents and caregivers were recruited first. Participants interested in completing an interview contacted the evaluation team via email or phone to schedule an interview. Once parents and caregivers were interviewed, they were asked to refer the name of providers and individuals in their care network that played a therapeutic role in their ECC experience. If the specific provider/individual’s name was given, those individuals/providers were then contacted via email and/or phone and asked if they would like to participate in an interview. If the parent/caregiver only named a role or agency, contact information was requested from the community coordinator. Furthermore, the names of the participants who referred the providers were not disclosed to providers; therefore, named individuals within roles were not directly connected with interviewed parents/caregivers and some providers worked with more than one of the interviewed parents/caregivers. Data Collection A total of 16 participants (8 parents/caregivers, 8 providers) were included in the evaluation. Interviews were completed with participants over the phone and followed a semi-structured interview guide focusing on the themes of therapeutic programs, approaches, methods and strategies used within Florida ECCs; caregiver and team member relationships in ECC; improvements for ECC therapeutic services and relationships; as well as considerations of racial and social equity. While recruitment was statewide, participants responded from two counties within one

5 Suggested Citation: Kihström, L., Vereen, S., Carr, C., Mackie, J. F., & Marshall, J. (2021). Chapter 4: Therapeutic networks. In L. Magruder, 2020-2021 Early Childhood Court Evaluation (pp. 19-38).

FLORIDA INSTITUTE FOR CHILD WELFARE

court circuit. Participants received a $25 electronic gift card following the interview. After completing the initial interview, participants received follow-up questions in the form of a short survey to gather demographic information such as age, gender identity, and education. Data Analysis Interviews were transcribed verbatim and reviewed by the evaluation team for accuracy. Data analysis was conducted for parent/ caregiver and provider interviews separately, with two research team members coding provider interview transcripts and two team members coding parent/caregiver interview transcripts. All data analysis was completed with MAXQDA, following a similar approach as described in Chapter 2.58 Coding was conducted using a thematic analysis and followed an iterative approach which consisted of developing an initial a priori codebook based on evaluation topics and questions, testing out the codebook by having two researchers code the same interview independently, calculating a kappa rate to establish inter-coder reliability, discussing additional changes for the codebook, coding another transcript, and re-calculating kappa. This process was repeated until the codebook was finalized and a kappa of >0.80 was reached. The remaining transcripts were coded using the finalized codebook. Intercoder reliability for the provider interviews was 0.90 and 0.86 for the caregiver interviews. Themes across the coded data were identified, reviewed, defined and summarized. Following the analysis of transcripts, a graphical representation of each parent/caregiver’s therapeutic network was created based on the information disclosed during the interview. While these graphical representations may not consist of everyone on the individual’s ECC team or community network, it does depict the roles of the individuals who participants mentioned were instrumental to their socioemotional health and overall well-being during their ECC experience. RESULTS Throughout this chapter, quotations by participants are included as part of the main findings, with “B” referring to biological parents, “F” referring to foster parents, and “P” referring to providers. The authors acknowledge that not all parents are biological parents; these labels were used to simplify the designation between parents and foster parents/caregivers and avoid confusion when attributing quotes to parents and providers. A total of eight parents and caregivers involved in ECC were interviewed. The sample included six biological parents and two foster parents. The biological parents and foster parents who participated in the interviews were not part of related cases. Seven of the participants identified as female and one participant identified as male. Six participants were involved in cases that were still open and two participants were involved in cases that were closed. Participant length of time engaged in ECC varied, with three participating for a year of less, four participating for one to two years, and one participant reported participating in ECC for two years. All eight participants spoke English as their primary language. A total of three parents/caregivers responded to the follow-up questions regarding race/ethnicity and self-identified as White. Providers included in this evaluation represented multiple roles and positions within ECC, including community coordinator, substance abuse case manager, assistant regional counsel, child advocate/ guardian ad litem, judge, psychotherapist, and domestic violence counselor. A total of five providers responded in the follow-up questions regarding demographic information. All five providers currently worked full-time in the ECC system. Among these five providers, the average age was 42 years (range 25-50 years) and all identified as White females. Regarding the highest level of education completed, three providers reported having a bachelor’s degree, while two had a master’s/graduate degree. When asked how satisfied they were overall with the ECC program, three FLORIDA INSTITUTE FOR CHILD WELFARE

providers responded “satisfied”, one provider responded “very satisfied”, and one provider responded “neutral.” In this section, findings are presented first in terms of parent/ caregivers’ impressions of ECC overall in the context of therapeutic approaches as well as their reported “therapeutic networks” followed by the summary of all responses related to each evaluation question proposed in the evaluation plan. Under each question, findings are presented as themes that emerged from interviews conducted with parents and caregivers followed by synthesis from the provider interviews. Parents’/Caregivers’ Overall Experience in ECC When asked to describe their overall experience within the Early Childhood Court system, most of the parents/caregivers responded with positive appraisals, stating that their experience was “a blessing,” “positive,” or “a good experience.” A small number of parents reported that their experiences in ECC were very different from what they expected because they previously heard negative comments about dealing with the court system in general. These participants felt that the negative appraisals were due to individuals not wanting to do the work required to resolve their case. For example, one participant stated: I’ve always heard a lot of negative things about DCF or just the whole system. Especially going to a drug treatment center, you hear the worst of the worst, but I always rely – I’m intelligent so I always knew that that’s their experience and that’s really the routine here. They’re kind of going based on how their case went and usually when things aren’t good it’s because you aren’t doing what you’re supposed to. So, I always try to remember to, just like anything else, to not judge. Just let my experience tell that for themselves and not listen to anybody, what they had to say. -B005 Another participant stated, regarding the Department of Children and Families (DCF): I mean people make it sound worse than it really is. It just seems like they didn’t really want to get going. - B004 Throughout the interviews, parents/caregivers further explained their personal experiences with ECC. They were also able to further elaborate on the specific experiences they believed were both supportive and unsupportive to their overall emotional and mental well-being. They also noted areas where ECC can make improvements to their therapeutic services. Therapeutic Networks Identified by Parents and Caregivers As shown in the network maps below (Figure 1), there were several commonalities among parent/caregivers’ descriptions of the individuals who helped them most or were supportive of their wellbeing from a therapeutic standpoint. Each network represents a parent or caregiver and their reported network roles. For illustrative purposes, green outlines denote the providers in those roles who were interviewed; however, not all providers were specifically connected to each parent due to anonymity. The figure below (Figure #) illustrates the roles of the individuals who participants mentioned were supportive of their well-being throughout the ECC process. Most participants mentioned the community coordinator (six participants mentioned) and therapist (five mentioned) as ECC team member supports. In addition, all participants noted family members, some of whom were kinship caregivers, as important members of their therapeutic networks. Substance abuse treatment counselors, case managers and peer counselors were mentioned by four of the six biological parents as important members of their therapeutic network. Three mentioned their judge, and one or two participants noted the attorney, guardian ad litem, DCF caseworker and domestic violence program counselor.

Two participants specifically noted that DCF was detrimental to their emotional well-being and one participant noted that therapy would have been helpful, but they had only been able to attend one session before there was a change in their case, noting that they “Didn’t even get a chance to take part in all [that] ECC offers.” The foster parents who were interviewed, one of whom was experienced and one new to the role, noted that there is no counselor, therapist, or activities to support caregiver emotional well-being. However one foster parent stated that “We have several different contacts that we can reach out to and so that is helpful…” and the other noted that “Every part of the team is equally helpful.” A foster parent also mentioned that there was support from formal and informal foster parent programs/groups. Figure 1. Roles of the individuals mentioned by participants as supportive during their ECC experience

Figure 2. Therapeutic network members identified by parents and caregivers Note: Garnet border denotes the role of a provider that was interviewed. Due to the nature of the referral system for this evaluation (sometimes roles, rather than names were provided and interview participant names were not disclosed to others), it is unknown whether the specific providers interviewed are linked to each specific biological or foster parent BIOLOGICAL PARENTS’ THERAPEUTIC NETWORKS

Substance abuse case manager/

Therapist

counselor/peer specialist

Aunt

GAL Community Coordinator

001

Attorney

002

Judge Judge

FLORIDA INSTITUTE FOR CHILD WELFARE

Community Coordinator

CPP would have been, only attended 1 session

BIOLOGICAL PARENTS’ THERAPEUTIC NETWORKS (CONTINUED)

DV program “somewhat” helpful

Therapist Mental Health Therapist

Community Coordinator 004

003 Community Coordinator

Substance abuse

Grandparents

case manager/

Grandparents

counselor/peer specialist

Methadone Clinic Counselor

Domestic Violence Counselor Case Manager

Judge

Substance abuse

Mental Health Therapist

Community Coordinator

005

case manager/ counselor/peer

007

Grandfather

specialist

Foster Parent

CPP Psychother apist

Substance abuse

Maternal Aunt

case manager/ counselor/peer specialist

FOSTER PARENTS’ THERAPEUTIC NETWORKS

Case Manager

Community Coordinator

006

Foster Care Organization

FLORIDA INSTITUTE FOR CHILD WELFARE

Guardian ad Litem

Foster Parent Social Media Group

008 Case Manager

Community Coordinator

Evaluation Question 1 What are the therapeutic programs, approaches, methods, and strategies used within Florida ECCs to promote parental mental health? Therapeutic Programs and Services Parent/caregiver descriptions of the therapeutic programs and strategies used in ECC extended beyond child-parent psychotherapy (CPP) and “parenting classes” with mention of substance abuse treatment programs and domestic violence classes as well. However, rather than describing how these programs are implemented, much of the parent/caregiver responses regarding therapeutic programs and strategies centered around whether they felt the programs and strategies, or individual practitioners, positively or negatively contributed to their personal mental or emotional wellbeing. Sentiments also reflected how supportive or unsupportive parents felt about ECC as a whole. Half of the parents/caregivers specifically mentioned that CPP and/or “parenting classes” were part of their therapeutic program within ECC. Circle of Security was not specifically mentioned. Some parents mentioned a nine-week parenting class, while another mentioned watching videos and discussing them. Several attended parenting classes as part of their substance dependence treatment program. All these participants acknowledged that the CPP and parenting classes were beneficial, especially for fostering the connection between the parent and child. One parent stated, “I guess that’s where the child-parent psychotherapist comes in, that’s what they’re there for to make sure you maintain a connection or relationship with your infant child…” In addition, these parents and caregivers mentioned their child-parent psychotherapist as being supportive during their ECC experience. However, due to changes in the way services were delivered during the COVID-19 pandemic, one parent believed they may not have fully benefited from therapeutic aspects of CPP and the parenting class. Parent comments about CPP and parenting classes include: Yes. I’m actually still - I don’t know how long it is, but I’m still attending - I attended a parenting class with us for nine weeks, and then I attend-I still attend, I guess they call it CPP. I don’t know what it stands for either. -B007 I was supposed to do the child-parent psychotherapy and I did do a session but, honestly, I didn’t continue them. I feel like they would have been very therapeutic for me, but I just was going through a lot at home and, like I said, I didn’t have that in-person contact, somebody from ECC lighting a fire under my butt and holding me more accountable and responsible than I was.-B002 I guess that’s where the child-parent psychotherapist comes in, that’s what they’re there for to make sure you maintain a connection or relationship with your infant child, but it’s just I guess how my personality is. I’m just very much the same on the phone as I am in-person. So, it was very easy and me and the foster mom started to really have a good relationship, we just really connected the first time we met on the phone. She kind of parented the way I did so it was very easy for me to stay connected to my son even though I only see him on video for nine months. I really owe that to my relationship with the foster mom, but I can see where it be very difficult for some people to remain connected to their child with not being able to touch now or see them, hold them. It’s very hard but I believe if you’re open to it, you can remain some kind of relationship as best you can with things going on, if you’re open to it. You have to put the time and effort in. -B005 I mean like I said, I’m part of the Child Welfare system so I’m very familiar. No, we’ve not had to have any counselling for our family or anything. We did do infant health evaluation for our little girl several months ago and she told us to follow up with CPP which we’re doing, again, to try to address some of the behaviors after visitations and to see what the cause of those FLORIDA INSTITUTE FOR CHILD WELFARE

issues might be. We did that evaluation through [infant mental health provider]. -F008 Yes. Pretty much everyone was very sensitive to what I was going through, from my caseworker, my child-parent psychotherapist. I’m also – I go to a methadone clinic because I am on years off of heroin. So, I have counselors that I get there, so everyone’s been very sensitive to my situation, what I was going through, and make sure that I’m heard out and get my mental health checked, you know and taking my medication that’s prescribed, and just kind of taken care of what I’m using there to be a whole healthy person. -B005 Yeah. Just the child-parent psychotherapy. Despite my women substance abuse counselor at the [parenting] group, I have a counselor at the [methadone treatment] clinic. I also have a therapist I see at the [mental health counseling] clinic that gives my depression medication, so just some of these people. They’ve always been very helpful to me. -B005 Honestly, with my experience, no. I honestly can’t think of something that was therapeutic for me, other than the classes and stuff that they provide, that they offered. I did do some parenting classes but, like I said too, I’m more of the in-person, hands-on type of person so I really didn’t – I don’t feel like I’ve benefited too much from the virtual classes. -B002 Many of the parents and caregivers felt the support from ECC providers was positive, encouraging, and genuine. In addition, some noted feeling as though they were not in ECC alone, but rather that there was a team and others had their best interest in mind. In terms of feeling supported, one of the prominent themes that emerged during the parent/caregiver interviews was feeling a sense of encouragement by the ECC providers. One of the biological parents mentioned feeling encouraged by ECC staff members (particularly the community coordinator) when they helped calm them down and ease their worries about traumatic life events: Just the way she spoke to me, and she just encouraged me and stuff, because we’ve never met in person, it’s all been over the phone, and she just always knew what to say and how to calm me down when I’d be really at the breaking point with a couple of things. -B002 Another biological parent participant shared how their community coordinator offered support while they were struggling with substance use: Like, you know, there was one time I did-I did slip up and you know, so I had relapsed or whatever. She was making sure that I was supported during that time. I actually ended up going to a doctor to find out if I [had a mental health condition] and get on certain medication, and that’s what - before I got them, it’s what caused me to I guess relapsed again, especially being at home with the father, that didn’t help either. -B007 One of the aforementioned biological parents expanded and said that they felt encouraged and reassured by their community coordinator, and that they helped her to not feel so “hard on herself:” She encouraged me, she gave me a lot of encouragement and then also reassuring that my son was in good hands and that I’m doing – not to be so hard on myself because I am being a good mother to him still. -B002 In addition to this same biological parent reported feeling encouraged and reassured by community coordinators, reassurance by the ECC judge also emerged during conversations: The judge assigned to my case was very caring. He was strict but, at the same time, he kept reassuring – he kept letting me know that he had a lot of confidence in me. He encouraged me

and he knows that I can do it, but he was also stern and let me know how serious everything was too.-B002 [There was a death in the family] so that kind of gave me some unusual circumstances that kind of threw my case in sort of different direction. So, they were really – everybody kind of stepped up and were really supportive and there for me. Definitely including my judge, [who] was very supportive and very positive.-B005 Another factor related to parents feeling supported is communication style: They were really engaged in the way they talked to me and just how they - they always offered help, but it just seemed a lot more genuine and there was more compassion in the beginning. -B002 Although social media is a source of social support outside ECC, one foster parent described her experience participating in a Facebook group for foster parents as a socially supportive experience. None of our parent participants mentioned participating in social media groups during their ECC experience: We joined a couple of foster parent Facebook groups one of them is specific to the county that we are in. One of them was for our county specifically and the other one was just with Embrace Florida. Being on there which was a mix of both foster parents in one group and then the other group had the people that work for the company that could answer questions pretty readily and help through the process. -F006 While the majority of parents/caregivers alluded to feeling supported during the ECC process, some participants also mentioned feeling unsupported at times. Participants described instances in which they did not feel as though they received the communication, assistance, and support that they needed from their ECC team. For example, one biological parent participant stated that they felt unsupported when miscommunication occurred with their case manager: My case manager, like I said, it definitely started between me and my aunt that I had never even – didn’t say. There was a lot of back and forth, “No, I didn’t,” “Yes, you did,” type of stuff. I just feel like she’s looked at me a certain way when I failed at drug test and sharing my relapse or something because I feel like, if you’ve never really been an addict - there are different addictions, but certain ones are a lot stronger than others. I feel they don’t really know. Like my aunt and uncle, I was told, “If you don’t stop doing drugs, your kid’s going to be taken from you,” that I should stop right there, no questions asked. It’s not that easy. -B002 Another participant shared how they felt unsupported by ECC staff while their family member was receiving a medical procedure: Because I mean I had to tell them I had to have overnights because my [family member] was [having a medical procedure] so I can lift my kids from my [family member], and they were trying not to give it to me, and they said four nights out of just a week. She needs a week to heal so I need to be over here every night. They ended up giving it to me, but I had to speak up. My lawyer didn’t speak up until I did. -B004 In addition, a biological parent mentioned feeling unsupported from a financial standpoint: Yes. Once. With child support, because I needed help getting child support from my child’s father because I pay for daycare by myself, because I’m by myself. I was like saying, “Can someone help me with child support?” Then, it was like, they said I had to ask my attorney. And she’ll say that she’s not over it and stuff. So, I was just feeling like it was, again, nobody was trying help FLORIDA INSTITUTE FOR CHILD WELFARE

me. So, I just tried talking to him again. Then he started helping me out a little more. -B001 Another biological parent participant stated they felt particularly unsupported in the domestic violence class offered by ECC. When asked to further describe this experience, they stated that the class conversations and activities were tailored for domestic violence perpetrators, even if some of the participants were domestic violence survivors. Because of this, they felt like the class was not particularly beneficial to their progress in the ECC system and general well-being, “…but I did learn stuff about relationships and stuff that you shouldn’t deal with and stuff like that.” -B004 Emergent Theme: The Impact of Current or Past Traumas on Engagement in the Therapeutic Process Many of the parents reported experiencing a number of different traumas that included past or current experiences of substance dependence, domestic violence, financial instability, unstable housing, the death or illness of a family member, and unemployment. In addition, participants experienced stressors related to the death of a loved one and the COVID-19 pandemic. The experience of being separated from their child(ren) was also noted as traumatic. Parents understood that addressing these traumas was essential to improving their mental health as well as their re-unification with their child(ren). Both parents and caregivers reported a number of ways in which these pre-existing and current traumas impacted both their mental health and their experiences in ECC. Some parents indicated that being separated from their child(ren) served as a trigger for some traumas such as substance use. A number of parents also described experiencing multiple external traumas that happened during their ECC case, which compounded with being separated from their child(ren) and may have impacted their mental/emotional well-being and participation in the program (e.g., attending classes or meetings). Conflicts with work schedules impeded parents’ ability to attend some of the in-person programs and services provided by ECC. As a consequence, non-compliance with the ECC plan may result in delays in reunification and program completion. Further, alluding to virtual programming, one participant stated, “I’m not employed right now myself; I wouldn’t have been able to take a cab half the time because I wouldn’t have the money for it.” -B002. Conversely, participants discussed how lack of employment can impact financial stability and proving they can provide financially for their child(ren) if reunified. Having the stress of not knowing whether they can prove their financial stability took a toll on some participants’ mental health: When you’re not able to provide or be financially responsible for yourself, you know. If you can’t do it for yourself, you can’t do it for you child, and so if you can’t show that, then you’re not going to get your son back. It kind of makes you feel like less than a person if you can’t take care of things, you know, so it does affect your mental health. -B005 Similar to employment and financial stability, housing instability was another major hurdle for parents to overcome. During COVID-19, one parent reported experiencing homelessness and due to the available resources, was unable to secure stable housing for a period of time. This biological parent stated: They kind of always, with COVID, there really wasn’t like shelters or anything that were really open, so it’s very hard for them to offer like kind of the services they usually would offer because of COVID. They really made some of the things they usually would recommend for housing and stuff were hard, hard to help me. So, I kind of really had to rely on friends to live. -B005 Regarding domestic violence, one participant discussed some of the differences in opinion they had with the ECC team about reuniting with a partner with whom there was documented intimate 24

partner violence. The parent felt like the situation may change since going through the program, however, the parent reported that the ECC team expressed some concerns: “I mean they said that they’re worried if I got back with [the other parent] which, I mean, I understand that, but for some people, it might be different.” -B004 Substance use was a stressor described by a number of parents, whether they experienced substance dependence currently or in the past. One participant detailed their experience with having their child taken away at birth and how it sparked continued drug use and the cycle of starting over multiple times: Again, they – I honestly feel that I didn’t even get a chance to take my son home with me from the hospital. I feel like I didn’t get the chance to show them I could be a good mom or that I can stop because having, especially my first child, not home with me, it really set me down that road mentally and then it made my drug use a little bit more. -B002 It could cause – it triggers people having something traumatic happen to them. Then they’re going through this process with the court, and something could trigger them to either relapse or just throw up or something. That would be a problem or issue with the therapeutic part of it because it’s starting you – there are constantly some people starting over again and again because they keep having setbacks from a traumatic event that they’re not dealing with, maybe. So, when the trigger comes, it sends them all the way back to the starting line. -B002 Parents did chronicle some ways in which ECC was able to help or provide referrals to help alleviate some of these stressors during their time in ECC. Some of the resources participants described included referrals for food assistance, housing assistance, and job preparation. Another prominent theme throughout discussions with parents and caregivers was the complexities associated with struggling with multiple challenges or traumas which compounded to affect parental socioemotional well-being and their ability to engage in ECC therapeutic processes. Below are participant accounts of how experiencing multiple traumas impacted their mental/emotional wellbeing and their journey in ECC: I think it affects it a lot, just not having stable-stable residence, I wouldn’t get my son back so it was always in the back of my mind. But then, the death of my [relative] kind of threw me through another loop, so it was really hard to go through the grieving process because there’s no time limit on grieving, but I mean my ECC case was one year, so just not being stable. That really kind of affect my case for a while because I knew that I wasn’t going to get my son back if I couldn’t show stability. So, that’s was probably the hardest thing for me to overcome, it’s just having that stability issue, stability. […] …food stamps, and the temporary cash assistance, and those things…They just kind of help guide me through all this kind of services and things like that. -B005 I’m going to be very honest, in some of our family team meetings and even in court and other staffing, the parents have gotten into altercations during the process. I believe there were times that they appear to be under the influence during some of these staffing as well. It’s one of those where the parents have to be willing to engage in the therapeutic process in order to be able to move forward and be able to even be more productive in those meetings but they’ve got to be able to take those first steps. -F008 Yes, it clouds your judgement and it’s a rollercoaster itself without having the depression, anxiety and things that the therapy and stuff would be therapeutic for you, or that’s what you’re trying to fix or get help for, the depression or anxiety. Having the substance abuse on top of that, it makes it a little bit

FLORIDA INSTITUTE FOR CHILD WELFARE

harder because you got to stay focused and stay encouraged because you are real hard yourself. It may be some stuff that’s a lot. I think that’s one of the things, or some of the things. -B002 Provider Feedback on Specific Therapeutic Approaches, Methods, and Strategies

Programs,

In this section, findings are provided for the therapeutic approaches used within Florida ECCs as reported by providers working in the system. Providers described therapeutic programs such as Circle of Security (COS), Child-Parent Psychotherapy (CPP), Women and Family Intervention Services (WFIS), Family Intensive Treatment (FIT), and other therapeutic approaches. The most commonly mentioned ECC therapeutic program mentioned by providers was child-parent-psychotherapy (CPP). CPP was defined by one provider: It’s kind of a parenting course combined with family therapy between an infant and a mother or father. The goal of childparent psychotherapy is to have parents understand basically developmental milestones and developmental needs of children in their infancy in order to be able to have the brain development that we’re looking for to be as healthy and normal as possible despite these abnormal circumstances that the parents find themselves in. Also, teaching them combined with basic parenting skills. So it’s not so much a mental health service for the parents but it’s a bonding approach that’s being taught between a parent and child so that can really successfully address how you interact with an infant, what you should do to educate yourself regarding their ages and stages questionnaires and determining whether or not they’re meeting milestones successfully but it’s not targeting a parent’s specific mental health diagnoses for instance to have a diagnosis of major depression or bipolar disorder, schizophrenia. That service is not made to address that in any type of way. So, any services where or any cases where we have allowed a parent who has mental health issues to come into the Early Childhood Court Program, therapeutic mental health services would be separate from that entirely. -P003 CPP was mentioned as a consistent part of ECC aimed at tackling the root causes of the issues children and families were facing: I think what’s working really well is our CPP approach. I say that because it’s so different than what I remember parents receiving when I was first in child welfare 20 years ago. It really is – in my opinion, CPP is really about like finding the root cause of what led a parent to the place that they’re currently in, that’s also led to the removal of their child and a fractured relationship. So, because behind every action, there’s a cause, we’re looking at the cause, how did we get here, and how do we address it, and not just kind of cover it up, but really dig down deep, so that when we address it, it’s addressed, and we’re not just putting a Band-Aid on it and hoping it never returns. We’re actually healing families, I think, because families are thinking not only about the current situation they’re in but it’s also like where they came from, like what’s been on their journey. How did they get here? So, I have parents who have been so receptive to CPP because it teaches them to be a parent to their specific child, as opposed to being a parent of any child, like you used to get with old-fashioned parenting classes. Like these therapies are designed for families specifically to their needs and their challenges, and so, it’s not cookie cutter. It’s very individualized. It’s very specialized. I think because of that, we’re seeing – I’m seeing like families change. -P001 This same provider added that CPP helps families to understand why something happened rather than just focusing on when something happened. Referring to CPP as “the gold standard,” this

provider stated that CPP is designed to address the trauma a child has experienced because of abuse and neglect and the removal episode itself, and it teaches parents how to respond to their children’s needs and wants and helps them learn how to manage their children’s behaviors. Another provider agreed that using trauma-informed methods such as CPP in ECC at an early stage was critical for both the child and the parent: Like I said, I believe in the child-parent psychotherapy. I also believe we are trauma-informed. Most of our therapeutic programs start with trauma that the children and parents have dealt with when these children are of young age. We’re working with that trauma and attachment aspect of trauma-informed care. It’s really intervening, it’s helping get the parents parenting skills that’s something the child – the parents know how to parent the child, and the child getting that attachment. -P007 Another provider added that parents’ engagement in CPP was typically a predictor of the parents’ long-term success in the program. They also remarked that the holistic approach used in CPP was one of its advantages, as opposed to other therapeutic approaches which tended to compartmentalize issues into different topics and categories to be addressed by different counselors: Certainly, the child-parent psychotherapy is the critical component of ECC court, infant mental health. ECC parents who successfully engage in that tend to do much better than parents who don’t, as it relates to their relationship. We have the FIT program that’s also state-wide, family-intensive therapy, that it’s kind of a collaborative approach or an overall holistic approach as it relates to mental health and substance abuse. Admittingly, I think the less providers parents work with, the more bang for their buck, so to speak, the better. Across the board, whether it’s ECC or just the regular juvenile docket, I cannot stand that kind of we only talk about your substance abuse issues or we only talk about your anger issues, or we only talk about your trauma. We’d have counselors who are willing to address all of those things. If someone’s struggling, it’s hard for them in their mind to compartmentalize the chicken and the egg. I get very annoyed with this approach that it’s prevalent throughout the dependence. -P006 In addition to CPP, other therapeutic programs offered in ECC mentioned by the providers included individual therapy, group therapy, Circle of Security, substance abuse advocacy, the WFIS program (Women and Family Intervention Services), as well as a women’s program on domestic violence: I think the biggest part that parents have identified in any case is the component – it’s called Circle of Security. It’s a theory of childparent interactions that focuses on healthy attachments. Healthy and strong attachments between parents and children. It’s basically in contrast to I guess helicopter parenting. It’s basically trying to strike a balance between the glut and hypervigilance in relation to raising an infant especially as a young parent…this is a generational issue. We don’t have lots of parents who come in the system who had healthy upbringings themselves. So it’s a generational issue and that Circle of Security Program really teaches new habits and new systems of thought that probably would never have been part of these parents’ lives were it not for the Early Childhood Court Program. -P003 We have the [program]. That’s a 12-week program on various topics of domestic violence educating women on what domestic violence is, safety planning, coping skills, grief and loss, different various topics on domestic violence. Then we offer individual counseling which is support counseling for the women that they can do weekly, biweekly, monthly. Then we offer support group which is counseling, which is once a week, where the women come together to support one another. We offer all of these services for the victims. P007 FLORIDA INSTITUTE FOR CHILD WELFARE

Other therapeutic programs and approaches used in ECC and mentioned by providers included a child abuse protection program designed for substance exposed newborns and the Family Intensive Treatment (FIT) program: We do have a [substance exposed newborn child protection program]… They wrap around – do all the stuff that we need them to do. They do the physical. They go to the doctor’s appointments and deal with any kind of mental health medications that the families need. They get them into therapy, whichever therapy they need. They deal with substance abuse issues and mental health issues and all that kind of stuff. Down the road, they can do check and security and CPP with our parents as well. -P008 Then we have other therapeutic approaches in terms of like substance abuse. We have a really great program here called FIT and [substance exposed newborn child protection program]. I can’t say enough good things about that program. It’s designed for families, zero to three, who have substance abuse diagnosis. It covers not only the parent’s substance abuse issues, but mental health. It addresses parent’s trauma as children themselves or trauma that they’ve experienced as adults that’s kind of unresolved and might be leading to mental health concerns or might be leading to substance abuse use. So, that program is fantastic. Of course, we have inpatient treatment options here. We have domestic violence providers in town for both victims and providers. -P001 As for additional therapeutic supports, one provider mentioned an informal book program which had been initiated to improve caregiver relations: We have started a two-for-two program here which I don’t know if you’re familiar about it. Basically, a parent picks out a book and they get a copy of it and then the caregiver gets a copy of it. We set up a time every single week that the parent will read the book. The child can follow along with the book but the parent can read the book to them either over the phone or FaceTime in addition to their face-to-face visits but the idea is that every – that they actually, again just being key, routine being key that every night, the mom, “We’re going to talk to mommy and then mommy is going to read the book and grandma is going to show me the book pictures while she’s reading,” that kind of thing. That program has had a lot of success. We have just recently started it obviously on this side of COVID. Yes, I’m hopeful for it though. The caregivers have all, as you know, given these that they will do their part on that. I know that I did have some parents at least every time we work, they do come. They change out their book. So, we’re doing that. I think that’s a positive way in which that we’re trying to engage visitation and caregiver relationships with the parent as well. -P006 In addition to the abovementioned therapeutic programs and methods, providers also mentioned how client-centered language and communication mattered in parents’ and caregivers’ perceptions of the degree to which ECC is therapeutic. Examples of communication as a therapeutic support included client-centered language, icebreakers, and comfort calls, as well as expressing care towards the parents and caregivers: So, all of our language is client-centered language for sure. Our particular ECC coordinator, [name redacted], takes this to whole another level. It’s kind of funny. Sometimes, I’ll talk to her, and I’m like, ‘You’re being too nice. They’re not hearing you if they can’t hear what you’re saying when you’re that nice,’ but we do make every effort to treat the clients respect, to speak to them as equal, not as somebody less. -P002 I can’t say enough about the icebreakers and comfort calls. Back when I started 20 years ago, we kept our foster parents separate from our parents. We were afraid something bad was going 26

to happen if they knew each other or if, god forbid, somebody assaults somebody or something happens of that nature, but I feel like those comfort calls and those icebreaker meetings especially with the foster parents, have been a big success. We’ve got a team of people called our Cares Team that makes those calls and schedules those icebreakers right off the bat so we’re not waiting. -P008 We’re getting those things set up as quickly as possible so those parents can – it’s not going to be ideal for them regardless – but if that takes a little bit off of them to know that “These people, you know, care enough about my child that they are willing to hear what I have to say about his likes and dislikes and his medical conditions, his allergies that they’re taking good care of them.” That goes a long way with our parents. So, I feel that’s been a big success and I’m happy that that’s been a part of the way we are doing things now. It’s part of our modus operandi, I guess, if you will say. -P008 Well, I’m not a therapeutic provider. I work with guardian ad litem as a child advocate manager and we advocate for the best interests of the children in a courtroom. So, that’s my role in the ECC and so when we have our team meetings and such, I’m involved to see how things – how the parents are doing on their case plan, if there’s anything, any information that we could provide based on a case, if we need anything from the parents, if we have any concerns about anything, that’s kind of my role in the ECC realm of things. I have a lot more interactions with the parents typically with ECC compared to normal dependency court, because we do meet on a monthly basis. -P004 Then in between the family meetings, I basically just make myself available to the parents. I do not do a lot of – cold calling isn’t the right word but I don’t do a lot of intensive follow-up with my parents. My general role for contact is if something’s going on in your case, contact your caseworker first. If there’s something that your caseworker can’t address or hasn’t addressed satisfactorily for you, contact me so I can help navigate that process. But because we’re in contact regularly through those family team meetings, I don’t necessarily have a lot of maybe office appointments or phone appointments with my ECC parents because we just have such more intense level of communication naturally due to the way that the Early Childhood Court Program is set up. -P003 Then, for the parents, my support includes reaching out to them by phone or text several times a month to see how they’re doing, to see if there’s anything specifically on their mind or their hearts that they want to discuss at family team meetings, to just kind of check in with them, little bits of – If I knew at a team meeting last month that a parent had a job interview, I might follow up a day or two after the interview or the day or two before the interview to just give them like words of encouragement. “Thinking of you today. I hope you rock the interview,”, or just checking in to see how things were going. I don’t know how specific to be here but we recently had a parent check into an inpatient substance abuse treatment program, and it was a huge step for her. You could kind of tell that she was nervous about being there. She was a little nervous about if this would work for her or if she could even kind of do it. So, I slipped a little note in the mail to her at the inpatient treatment program just saying, “The team is so proud of you for making this decision to focus on your sobriety. You can do this. We’re all pulling for you, and we can’t wait to hear about the journey.” Simple things like that might not seem like a lot but can sometimes mean a lot to someone who’s struggling or having certain challenges. - P001 According to the providers, parents who had strong social support systems tended to do better in the program overall: Parents with their own internal support system are – it’s a good sign. Parents who have their own whether it’s family or nonrelative or their own, maybe they have – they go to an AA group FLORIDA INSTITUTE FOR CHILD WELFARE

or an NA group. Parents who have other supports outside of the Early Childhood Court Program tend to perform better because the bigger your support system is, typically the better your outcomes are because when a stressor presents itself or there’s a barrier, maybe there’s a transportation issue or an issue with housing. The bigger that support group is and the bigger the group of informal safety service providers that are available, the better the outcomes tend to be. -P003 Finally, providers themselves also expressed that they needed therapeutic services to process the trauma affiliated with the work: Just 100 % honestly, I received my own therapeutic services for trauma-related mental health issues. So, I have that support already in place to offset the ECC program and it does – it helps to have an outlet where you can process not only your own trauma and grief but what you’re experiencing because of your participation in this program where you’re constantly having to process other people’s trauma and grief and maltreatment. I don’t know if I could be in ECC court if I wasn’t in therapy, to be quite honest with you. I don’t know if I would be able to do it because as an empathetic person, it’s very difficult to have to constantly be exposed to trauma. It’s hard. It’s very hard. It’s not an easy role for – even to be at such an arm’s length as I am naturally a parent’s attorney and not even being a provider in the traditional sense or in case management that has put me in sadness, be at that level but even to be at such arm’s length, it’s a challenge without proper counseling. -P003 Another provider mentioned they felt somewhat unprepared to deal with work-related challenges in ECC particularly when they had first started their job. While this provider recognized the importance of having time only for themselves outside of work, this was made difficult by the sheer number of cases in ECC: I was not totally oblivious to what I was walking into and I certainly was prepared for some sad stories. I think it was the volume I was I was unprepared for…the sheer amount of child abuse that never gets prosecuted or heard about in the media that became a trafficking aspect of this assignment. I think I was unprepared in the scene, handled. I kind of felt that was a [regional] issue. So, it’s a real thing, and it’s very traumatizing. It still is. I think it’s again, I get annoyed, because the answer to that is to have a webinar that I can’t attend because I’m in the docket. So, there’s all this pressure to move these cases and the volume of cases, and at the same time, too, like I think the answer is have more time off. Really – to have some down time. It feels very impossible to have down time sometimes, especially coming off the other side of COVID because we don’t really have to slow down, but there is still a backlog of forensic determination for a life…I think some of the bigger circuits, I think we should have a therapist of our own, some 1-800 number to call and talk to somebody, but really, when somebody comes that’s not on office hours a couple of times a month, we can go talk to, because again, that’s the other issue. You can’t have for judges to see a therapist. Therapists testify in all sorts of family law issues. It’s like you don’t want to be using off of those cases and everybody knows you see the therapist. Again, it’s very hard to maintain this neutral Switzerland-like approach. The judges are kind of – they don’t get involved. …I do think that there is – judge stress I think is coming to head, honestly. We see it sometimes in the news in the extreme situations, and there’s not a lot of sympathy for those judges. I think the longer I do this job, the more sympathy I have for them that are either overmedicating or coping through substance abuse issues or, again their PTSD and their anger and their explosiveness. I don’t think judges aren’t lined to be helped for it. -P006 Another participant mentioned that reflective supervision was key to dealing with the stress of the job:

Speaking with my supervisors if the case gets overwhelming, talking it out, always seek supervision and just self-care. Sometimes this work can be overwhelming but making sure that I relay if something’s overwhelming to my team and my supervisors.-P007 Evaluation Question 2 How does ECC support relationships among caregivers and with other ECC team members? Parents/Caregivers: Strategies and Support for Relationships with ECC Providers and Caregivers Communication proved a salient theme throughout the Provider/Caregiver interviews. By “communication,” we refer to communication between parents/caregivers and ECC and communication between parents/caregivers with foster parents. Communication was described by parents/caregivers in a variety of contexts. One of the prominent themes that emerged during our interviews is communication between the community coordinator and foster parent/biological parent: Our caseworker was phenomenal. She was very responsive and supportive. She has had this case since the beginning before he came to us and so she has been there the entire time I think she’s really happy with the outcome of this case and she’s always been very supportive and helpful. Our care coordinator [community coordinator] is another person that I can think of. I only recently have been involved with her in the past few months with the monthly ECC update and she has been wonderful in following up and making sure that she answered any questions that either I had or that came up over the course. – F006 Parents/caregivers also discussed how beneficial it was to have open communication with their team whenever they were having issues or concerns. Having ECC team members who were accessible and had availability to speak after conventional work hours contributed to parent/caregiver peace of mind. One participant stated: Knowing that I have communication whenever needed. We haven’t had too many times where I’ve needed like an answer late night or on the weekend, but I have several different contacts that I can reach out to and so that is helpful just knowing that if I have a question or need an answer that I can get it. -F006 When biological parents referenced their experiences communicating with their assigned community coordinator, several mentioned the importance of genuine communication, caring, and “checking in” with them as a tool to assist with the therapeutic process: The care coordinator [community coordinator] is very genuine and caring, always asking how I’m doing. She takes me throughout the month as well, checks on me. That’s definitely what I like, too. -B002

parent and their child: They were very, very helpful. When my own daughter came home, she had a toddler bed but I had ended up buying her a Toys R Us bed, and [ECC provider] Like I said, I don’t know her, but she’s just on the ECC before. And she was like, I had needed a mattress. I had needed a mattress, and she brought it to my house. She also brought her some Christmas toys. -B001 One foster parent explained how their case worker was instrumental in helping them have difficult conversations with the biological parent when they showed up late or missed visits with their child: Well, it was just like our experience with the parents that actually at the beginning we didn’t think that we were even going to have our child very long because he already had unsupervised visits and they were working towards overnight and things like that and then things just kind of totally fell off and the parent started coming late for visits or missing visits them. You know, I was trying to be supportive of them, talk with the mom about being responsible and respectful of our time and respectful of her child. It would help for a little while but then it would go back and even continue to get worse than – so, working with the parents was just disheartening and they were young, so I don’t blame them a lot. I just think they were not ready to have a child but other than that the caseworker was very helpful in having some of those difficult conversations that I didn’t feel comfortable having with the biological parents. She would call and just tell them how it was that seemed to be rectified and again, it would work for a little while, so I appreciate the caseworker in helping to have these conversations. -F006 In addition, foster parents provided their opinions of biological parents’ willingness to engage in the program. However, our general findings concerning the foster parents in this evaluation suggest that foster parents had limited information about the biological parent’s personal life and the array of obstacles and trauma-related experiences preventing parents from fulling engaging in the ECC program. One foster parent speculated as to whether a biological parent attended a team meeting “under the influence”: I’m going to be very honest, in some of our family team meetings and even in court and other staffing, the parents have gotten into altercations during the process. I believe there were times that they appear to be under the influence during some of these staffing as well. It’s one of those where the parents have to be willing to engage in the therapeutic process in order to be able to move forward and be able to even be more productive in those meetings, but they’ve got to be able to take those first steps. -F008 The same foster parent also mentioned that biological participants were given “every resource possible” by the ECC team, and that it was fully up to the parent to engage in the ECC program activities and requirements.

Another means in which ECC provided support for relationships was via communication with foster parents/caregivers. During interviews, foster parents mentioned that that the majority of this communication came from the community coordinator and case manager:

I know that the team and everybody involved, they’re trying to be supportive and they’re trying to help parents encourage them and give them every resource possible. I mean I know in our case they’ve given them every resource possible in the area so I don’t think that there’s anymore that they could try to do. -F008

I mean every part of the team is equally helpful. I feel I’m always in communication with the guardian ad litem. I’m in constant communication with the case manager and they relay all concerns, anything that’s doing well, basically everything to the coordinator. I mean I feel like everybody is equally helpful. -P008

These comments indicate that foster parent perceptions of biological parents’ experiences and attitudes are important for building caregiver relationships and promoting meaningful engagement in ECC among all caregivers.

In addition to supporting relationships through communication, findings suggest that the community coordinator was also instrumental in purchasing material items needed by the biological FLORIDA INSTITUTE FOR CHILD WELFARE

Providers: Strategies and Support for Relationships with ECC Parents and Caregivers This evaluation suggests there were several ways in which relationships among caregivers and other ECC team members were 28

supported in ECC, including through various team member supports such as monthly meetings, through promoting a non-judgmental culture and various forms of communication, as well as through legislative support and relationship-building. Regular team meetings were frequently mentioned as one of the core ways through which relationships were maintained. Since the COVID-19 pandemic, the meetings were organized virtually on Zoom, and were described as a “place” where parents and caregivers were invited once a month to meet with their case manager, their providers (such as the CPP provider or local domestic violence shelters represented), as well as attorneys: We advocate for the parents to have good relationships with the caregivers of their children. We hold family team meetings where all of them can come in at the same time. We want to promote that the caregiver’s a part of the reunification process is to have a good communication between a caregiver and the parents. We promote good relations, then we actually have them come together in meetings to discuss any issues, et cetera. We just promote the wellness of that relationship. -P007 They are invited to attend our family team meetings. I think that our caregiver participation in family team meetings could improve, that’s one way that we’re supporting relationships, I think, is giving them an opportunity to have a voice at the table. The caregivers that do attend, I think the team does a pretty good job, and the judge, especially, does a pretty good job of thanking caregivers, like taking a moment and recognizing what they’re sacrificing, for lack of better word, to care for someone else’s child. Especially a lot of our caregivers are older, so that’s even more difficult for them. They’re not that season of life anymore to raise small children, but I think that we do a good job of recognizing and thanking them and showing our appreciation for the role that they’re playing and stepping up to do something so selfless. -P001 However, providers also mentioned it was difficult to ensure caregivers attended meetings and that communicating during these meetings was not always easy: Typically, they (caregivers) are all invited to the family team meetings, and I try very hard in my role to encourage them to speak up, to contact the case manager if they’re having a problem and to join the team meetings. I think we do a pretty good job of that here, saying, “Welcome, come in.” We’re here. We want to hear from you. I do have one family that has just asked some children to be moved, and it’s very hard for me right now. I was trying so hard to keep them together and looking back on it, I realized that they have never once come to a family team meeting. I think, at the beginning, it was because they did not want to get angry or upset with the birth parents, but I think it’s been detrimental that they didn’t feel like their voices were heard even though I’d say, “Come to the meeting. Come to the meeting.” I think we still have some challenges with that. -P005 So, we try to get the information out as much as we can, but we try to – and the same goes with the fittest kin and with relative caregivers telling them that they are part of the process as well as the parents. That we try to talk about not being adversarial. A lot of times, especially, with family members, there has been bridges burned, and there’s hard feelings and things of that nature. We try to help them, engage them in some of these meetings, especially, and visitations to kind of help build those bonds so that the children aren’t put in the middle and kind of torn back-and-forth between that. We always talk with our parents about the concurrent planning right off the bat as well as our caregivers to let them know we’re not here to hide anything from you. What we want is what’s best for your children. If they can’t be with you, we want it to be somebody that’s going to love them and going to care for them. So, we talk about those things right in the open as early as we can, but we try to get everybody FLORIDA INSTITUTE FOR CHILD WELFARE

kind of involved in everything. When there’s a meeting, when there’s court, we want those caregivers there with the rest of the players on the team just to make sure that their voices are heard, and that we do everything we can to not burn them out. We don’t want to put everything on them. We’ve got a new case for the baby that there are five visits a week because it’s a newborn, we try to get our case managers and our families and coworkers involved so they can do some of those visits as well as so the foster parents and relatives aren’t having to do every single visit so that we don’t burn them out. We try to remember their feelings. They have families as well. So, we do everything we can just to make the caregivers and the parents feel equally values and know that they are both an important part in the process. -P008 A lot of times in the ECC courts, parents find that there’s a lot that’s not being said but I have to pull out, that we need to hash out that nobody really wants to be the one to say it but in body language, I think trying to just really read people’s body language, like what’s going on between moms, grandmoms today kind of thing sometimes happens. -P006 Providers also discussed how they themselves felt supported as part of a larger team in ECC. Many mentioned regular meetings, communication, and professional development activities as supports: Then we attend monthly Early Childhood Court. We call them a family team meeting which is where it’s just basically the same group of people who decide on whether or not cases should be considered for ECC. On top of this, specific providers for each family’s case, they get together in a 30-minute Zoom call. We go over conditions for return on every family team meeting to determine whether or not the family can – or whether or not the team can write the reunification with one or more parents and the child. We discuss case plan compliance and non-compliance for each parent, we discuss any visitation issues that may exist, any placement issues that may have been brought to someone’s attention since our last family team meeting the prior 30 days. The providers will give input on their recommendations regarding continuing contact, additional services for parents or children. -P003 The Office of State Court Administration has put on a couple of hour-long webinars of sorts as like an introduction to ECC. I attended our all-site training, which is all ECC Courts across the state, training for judges, case managers, and attorneys, as well as our providers, and then our judicial college course was taught by [the circuit’s] ECC judge who’s been doing this for a number of years now. Additionally, I think we get – I would say, ECC more than any other specialty courts that I have seen just flooded with emails, webinars, data. We have quarterly meetings. We have monthly meetings. We have meetings and meetings. Our ECC coordinators stay in touch with me with weekly extensive variety of topics and/or articles about the national ECC court data that may impact EC court. -P006 I love the statewide team. I know that at any time, if I have a question or concern or I just want to know more about how something’s done in a different part of the state, that I can pick up the phone. I can call any coordinator in the state. I can open my email and email any coordinator in the state and get an almost immediate response. I have endless support from the Office of the State Court Administrator, the Office of Court Improvement. They have been so quick. -P001 One provider mentioned they did not always feel like team members were equals, particularly in meetings where they felt like their voice was not always heard: Well, I have felt in the past that kind of show down my opinion on what’s going on, almost as if what I have to say doesn’t matter, because I’m just a guardian ad litem. I’ve felt, in a way, 29

that my – yes, what I had to say didn’t matter, so it didn’t bode well to speak up about anything, or that if I did speak up about something, then it might be taken in the wrong way or it might be seen as siding with the parent or not giving the parents enough credit kind of thing, and I’m all about giving credit where credit is due, but I don’t think that I should be giving credit for buying your child diapers, because that’s part of being a parent, which – and I’ve gotten backlash on that before. So, there have been times that I felt that I can’t speak up for the children or even the parents, because my main goal – my main concern during those team meetings or court is about the child or children. -P004 Participants stated communication and fostering a non-judgmental culture played a key role in supporting relationships between parents and caregivers as well as the larger team: So, I think that those relationships can be improved – I think there are so many ways, but some of the ones that hit me right off are being willing to hear the parent’s story. I think a lot of times, we reach shelter petition, and we get so caught up in what the allegations are in the story that’s in black and white on a piece of paper that we don’t really stop to ask the parent, “Hey, tell me about you.” Like getting to know parents I think from their perspective and their words and not the words of an investigator I think is really important to setting the tone for the beginning of a relationship with parents in ECC. -P001 What I do myself, because I only work with parents, I don’t work with the child or the caregiver. What I do with the parents is I try to talk about being patient. I try to talk about understanding how that caregiver may feel one way or another. If it’s a family member of my client who’s been using drugs for five years and they’ve had to bail them out of jail and they’ve had to give them Narcan and taken to the hospital, then that caregiver’s got a lot of feelings about my parent, and so, I do try to encourage the parent to be amenable, to pick their battles, that kind of thing. On the flipside of that, the child welfare case manager is talking to the caregiver and saying, “I understand that you’re protective of this baby, I understand that’s important that you want to keep the baby safe, but Mom has rights, and Dad has rights, and Mom and Dad are doing what they need to be doing, so you need to allow these visits to happen.” So, basically, that’s kind of what we do. We had a shelter – it’s about two months ago, and at the shelter, the two mothers were like, “Do not push this baby with,” – it was – the woman who gave birth to this child, her mom, and both mothers with this baby, they’re married, both mothers of the baby, were like, “Don’t put her there, don’t put her there, don’t put her there, she won’t let us see – we’ll never see this baby again,” and next month, that absolutely was what was happening, and so the judge moved the baby because the judge recognized in that moment that this relationship was not going to work, that it wasn’t conducive to visits, it wasn’t conducive to a working relationship, and that is what happens. So, they’ll move it if it can’t get fixed, but they certainly encourage harmony when they can. -P002 We start on the positive note. We try to end on a positive note, and try and put stuff that needs to be worked on in the middle. Admittingly, I just think we would talk about this, but I don’t feel like my ECC docket really followed the model. It’s just kind of a more intensive dependency docket. Unfortunately, I think that’s going to –must fit on resources that we are not following the model very well. We’re not having a lot of success in ECC Court as it relates to – if you consider the ultimate reunification, we’re not having a lot of success. Quite frankly, if you consider success to be permanency within 12 months, we’re not having a lot of success. So, it depends on what you believe if that is the defined time, but we don’t have a lot of parents that are really doing well right now on ECC court. Part of me trying to keep them motivated is try and offer a big case, from the big case first, just so that some of the parents that are doing well are doing it. I asked the parents to reach out to other parents and FLORIDA INSTITUTE FOR CHILD WELFARE

just try and offer some encouragement, maybe encouragement will come from their peers instead of me. Many of our parents are also ACEs and I’m cognizant of that obviously. I try and be sympathetic to that they’re overcoming their own struggle and their own battles and just trying to acknowledge that with them so that they don’t think I’m just sitting up in an ivory tower being judgmental. Additionally, especially with these parents who are fighting drug addiction, which is most of our parents, relapse is part of recovery and trying to differentiate those who are serious, that aren’t struggling, versus those who really aren’t motivated to be clean. -P006 Finally, participants mentioned legislative support: Well, I’ve been really excited that we have the Senate Bill 80 that passed recently that stuff that we’ve been doing for a while has actually become law now with the comfort calls and the icebreakers and transition plans and things like that. We always try to engage our caregivers right from the beginning. -P008 Evaluation Question 3 In what ways can ECC therapeutic services and relationship supports be improved? Parents/Caregivers: Barriers to Engagement in Therapeutic Services and Supports Parents/caregivers detailed events, unexpected life circumstances, and traumas that inhibited their ability to have a beneficial and therapeutically healing ECC experience. Specific barriers mentioned by parents and caregivers (and described above) included the following: virtual classes (instead of in-person), unexpected life events (e.g., the death of a relative or family member), lack of reliable transportation, mental health-related obstacles (e.g., depression, anxiety), and triggers/past trauma. Negative COVID-19 Experiences One barrier that emerged for some parent/caregivers was related to changes in the ways ECC delivered services due to the pandemic. These changes included offering virtual classes in lieu of in-person, “hands-on” classes. While some participants viewed virtual classes as more accessible and helpful to their therapeutic process, others stated that switching to virtual classes instead of in-person classes was detrimental to their progress in the ECC program. The onset of the COVID-19 pandemic also impacted parent/ caregiver interactions with the ECC team, how parent meetings were conducted, virtual court sessions, and visitation with children. Parents and caregivers had mixed responses as to whether virtual services positively or negatively impacted their ECC experience. Some parents did not like the virtual format of the services, while other parents believed that keeping some form of virtual services in place after the pandemic would be helpful due to work schedules and transportation/travel issues. Those parents/caregivers impacted negatively believed they would have fared better in the program if they were able to participate in in-person services: Honestly, when all this started is when COVID happened. A lot of the services and stuff that were provided were all virtual or over the phone. It wasn’t hands-on meetings anymore, like they did in the past. I feel like I didn’t really get the full – everything that they offered. - B002 I could see people that are very open or very outgoing would have a very hard time with doing therapy or counseling sessions via phone, you know, and not in person. Especially grieving counseling, it’s really hard to get intimate or still that kind of connection if you’re not in person, but umm luckily, I’m blessed, like my personality really just I can really overcome things like that but I could see where it would be a barrier for a lot of people that maybe aren’t that outgoing or personable. -B005 30

In addition, one participant with a history of addiction and substance abuse stated that virtual classes were particularly difficult because she felt she could hide the barriers and struggles she encountered behind the screen. She indicated she may not have been able to hide her struggles as much as she did during virtual encounters if she saw the ECC staff members in person. Me personally, being an addict and my personality, I just needed more hands-on or more in-person contact than I had because, like I said, everything was done over the phone. It was easy for me to say, “Yes, I’m okay, I’m doing okay,” but, really, if it was in – if I had been in front of them, they could have seen, “No, she’s not okay.” I feel like, maybe, they could have addressed it and something could have been done a little bit sooner. -B002 One caregiver described the negative impact virtual services had on parent accountability in the program, leading to missed meetings and court dates and the ability to hide substance use. The caregiver detailed: I feel like there’s a little bit less accountability for the parents. They’re able to use COVID-19 as an excuse and I’ve seen that numerous times. You don’t want to get a drug screen done that they, “Oh, I might have been exposed to COVID-19,” things like that. Not being held accountable to actually show up to court, show up to court appropriately, on time, not under the influence. To show up to these meetings, again, not under the influence, acting appropriately. I feel like a little bit of accountabilities have been taken away that the parents and what not should have. Even the Child Welfare professionals, they should have an accountability as well and I think that’s kind of been taken away. It’s a little bit more convenient and I think that the parents are being allowed to get away with more or not be seen as often, and to make sure that they’re taking the appropriate steps that they need. - F008 The COVID-19 pandemic affected parent-child visitation in a number of ways. Some parents reported that caregivers paused visitation for a period and recently resumed visits at the time of the present data collection. Parents described how the changes in visitation negatively impacted their emotional well-being, especially parents with young children. Some parents who were previously coping with not having their child(ren) every day before COVID-19, found it difficult and felt like they were losing the opportunity to interact with and see their children grow. One parent reported being unable to see their child for in-person visits for several months during COVID-19, while another caregiver reported that parent participation in virtual visitation decreased during COVID-19: I would say probably having negative impact regarding the visitation. You would think it would make it easier for the parents to have visitations but I think it actually – because we were doing it electronically and all they have to do is pop on their phone from wherever they were but I think it also made it very easy for them to just not care so I do think that impacted like not being able to physically see their child and hold their child. It made it easier for them to just disconnect from their child. Although, I do think it was heading that way for our case to begin with because of the number of visitations that they had missed prior to COVID. So, I don’t know if it would’ve changed the end result how his court case came out. But to me, it was frustrating because I felt like, “How hard is it just to get on your phone and have a visit with your child? -F006 Additional Barriers Another barrier to ECC therapeutic services included the assurance that the needs of both the parent and the children were being met throughout the duration of the program. One of the interviewed foster parents believed this was a potential barrier to therapeutic program outcomes and strategies. As mentioned in the quote below, FLORIDA INSTITUTE FOR CHILD WELFARE

they voiced their concern about drawing “the lines” between offering support to both parents and children and stated that it is “difficult to do both and do both well”: Well, other than what they do, I mean I know they both went to counseling. Both of the parents went to counseling and have to complete parenting courses and do urinalysis tests and I think they have things in place to help keep parents in check but I also understand that the focus is on the children and keeping the children safe so I don’t know how much ECC really – I don’t know where the lines are between what should ECC be doing to help the parents versus focusing on the child. Obviously, they’re related but it’s difficult to do both and do both well. You would definitely need more staff or like an entirely separate department towards just with the parents, so I don’t know otherwise how to answer that. -F006 Another notable barrier identified by parents/caregivers includes the times ECC activities/classes were offered. As mentioned below, one of the biological parent participants stated that most of the ECC classes were offered during “regular” work hours, meaning that most of them ended by 5:00 PM. The participant also stated that if parents work during the day and want to participate in these classes after “regular” work hours, they were required to pay money for the classes. This finding was particularly concerning because all ECC classes/activities are to be free of cost to participants: I mean for other people it might not be, because I had a job that they worked with me, like scheduling-wise so I can make my meetings and stuff for other people who have regular jobs. I work for a local company, so they work with me a lot, but if a regular company was working with somebody, it needs to be after 5:00 PM for everything. Normal people work until 5:00, and I was having to do meetings at 1:00 and stuff like that in the middle of the day. They said I could have meetings afterwards, but I have to pay, and I was like, “Whoa, I mean I volunteered to do this.” You know what I’m saying? I didn’t see how that was right, how you go from volunteering for this and then it gets court-ordered and then if you’re not following the schedule, you got to pay money. -B004 Transportation also proved to be a common obstacle/barrier for several of the interview participants. As one parent mentioned, not having reliable transportation was particularly challenging because if they had to rely on another person for transportation, that person would have to be approved by ECC to be in the presence of and/or transport their child: …Whoever is going to be around your child, regardless of if they’re just picking you up or taking you to pick them up and dropping you all off, they have to be approved. If they’re not, then it’s farewell. That’s still one of my things because, even though I have a way to get to visitations with my son, or whatever, because of the person that would be driving me isn’t approved, I can’t even go, period. -B002 Regarding the therapeutic process specifically, mental health conditions such as anxiety and depression were also mentioned as barriers to staying focused and encouraged while navigating the ECC program. Substance use was also mentioned as being linked to anxiety and depression: Yes, it clouds your judgment and it’s a rollercoaster itself without having the depression, anxiety and things that the therapy and stuff would be therapeutic for you, or that’s what you’re trying to fix or get help for, the depression or anxiety... -B002 In addition to anxiety, depression, and substance use, traumatic events and “triggers” were also identified by the same participant as barriers to the therapeutic process:

…it triggers people having something traumatic happen to them. Then they’re going through this process with the court, and something could trigger them to either relapse…. That would be a problem or issue with the therapeutic part of it because it’s starting you – there are constantly some people starting over again and again… So, when the trigger comes, it sends them all the way back to the starting line. -B002 Improvements to Therapeutic Programs and Services Parents/Caregivers Participants also discussed ways in which ECC could improve the therapeutic services and programs for parents and caregivers. Parents wanted to feel less judged and more understood by their ECC team. Participants suggested all ECC team members undergo increased trauma-informed care training to better understand the predicaments that their clients experience and to show more compassion and empathy for those they are serving. One participant suggested: I feel like – I think that, if people were working with people who have substance abuse issues, they should definitely have a class that they take to better understand how an addict feels, how they can feel attacked even though they might not be being attacked. Everybody’s perspective is different, and some people just need to be trained on how to recognize what somebody’s going through and assess the situation better. -B002 Another concern for participants was the excessive use of jargon. One participant explained that there were several acronyms and terms associated with ECC that they were not aware of when they first entered into the program. Being inundated with unknown terms was frustrating. One participant suggested explaining the terms and acronyms up front instead of assuming parents and caregivers understand the meanings of these terms. One participant described their experiences with encountering unknown terminology in the following way: I don’t know if harmful is the word. I think maybe confusing or frustrating that I think part of that is because we are new foster parents and so a lot of the terminology, we were not familiar with and so either we had to ask what the terminology was rather than it being explained to us at the forefront. I’m in a world where we use a lot of acronyms as well and I try to remember to always explain to parents or people that are coming in what those acronyms mean and that wasn’t something that I felt was readily done so either I would have to look up the acronym myself or just point blank ask, “What is this? What does that mean?” I would just say, “Don’t assume that people know what the terminology means. Just be more upfront with what it means, the timeline…” that was a frustrating thing too. I felt like that we were told that things were going to happen on a certain timeline and then of course, I know stuff happens, but the timeline would continue to get extended, extended, extended. -F006 Positive COVID-19 Experiences While work schedules and transportation issues were reported as barriers to parent engagement in ECC, some parents experienced increases in engagement due to the changes in how services were offered during the pandemic. Parents/caregivers reporting positive impacts and felt they were able to be more engaged in the program due to the accessibility of Zoom and virtual services. Other participants believed that offering virtual services was helpful for parents and caregivers who are working and may not be able to take time off to attend meetings and program services in-person. Below are positive participant reactions to the changes in ECC services to the virtual format during the pandemic: Well, I will say and I don’t know or it’s hard for me to remember, actually, how it was before COVID but one of the things that FLORIDA INSTITUTE FOR CHILD WELFARE

actually was I felt that made me feel more supported was because I was able to be more involved in the regular ECC meetings because of the use of Zoom. I don’t know if that’s something they’re going to continue to use but especially since I was in a different county, I was able to just hop on to Zoom and be involved and all of those ECC hearings with the judges and the monthly ECC reviews that they had, very, very easily. I don’t remember before COVID really being able to be a part of those at all. I just would kind of get an update after the fact because I wasn’t able to attend. I don’t know if that’s something you’re to consider doing but it was very helpful for me just being out of county. -F006 I think it’s very helpful online for people that work and stuff like that…Even after all this COVID stuff, I think they should be able to do it on Zoom or go in. -B004 The meetings on the phone to me were a lot better than doing them in person. -B003 It was also noted that some parents reported more positive experiences with virtual visitation during the pandemic, reporting that some caregivers allowed continued visitation, while others were able to conduct virtual visits in place of in-person visits. Two parents described their experiences with virtual visits in the following ways: For his grandmother, actually, it helped with her visitation. She had visitation every other week and she has been on-point every other week. We have visitation for an hour and that has been very helpful for her to be able to continue to have that connection with him. -F006 Providers To improve ECC therapeutic services and relationship supports, themes found from provider interviews included allocating more resources to have better access to mental health services, fostering a non-judgmental culture, having a quicker timeline for reunification or permanency, and improving communication with parents and caregivers. Allocating more resources for mental health services was mentioned by several providers: In order for ECC to be successful, there are a lot of factors outside of the influence of the judge and quite frankly, even the Department of Children and Families…if the legislature is not giving extra money to my case management system to that fund, ECC hits the spot. If we don’t have enough child psychotherapy providers to handle the case loads, if we don’t have intensive drug treatments for these parents to go to, it’s all for none. It can’t just be ECC money or justice…or structural issues. It has to be providers. It has to be who’s on the ground. I think that’s where we need that. I mean, we do. Otherwise, we can have this great model and weakness. It just won’t work. It won’t work unless everybody is given a specific fund. -P006 One of the things our particular court did – I was just having this discussion with a case manager about an hour ago – is limited mental health services that we have. We do have good representation from our local domestic violence shelter, but even those services are very disjointed, and to me, it’s something we really need to figure out how we’re going to tackle it from here on out because we’re getting people – we’re getting parents placed in wrong services, according to me anyway. The mental health services that we have are very, again, disjointed. Some people just go to a local community health center, and they get a screening, and they usually go get on medicine, and if they’re not really in counseling or they’re sent to a group, we don’t have a specific trauma treatment, so that’s a problem. -P005 I think there needs to be more mental health… I think there needs to be a stronger mental health focus with the parents. -P007

Needs to be improved? I mean it would be – like I said, we have this provider [redacted]. They are the provider for the lion’s share of mental health and substance abuse, parenting, all that stuff for this whole area, not just our cases in ECC, but throughout the whole dependency system and private folks that need assistance as well. There can be a backup with services. I think it’s gotten better. Like I said, with the telehealth, they’ve been able to do more and they’re able to engage more folks. Now that we’re going back in person, I just don’t want us to go back because sometimes we wait a couple of weeks to get that initial evaluation, or we would only be able to get somebody in for counseling once a month, their individual therapy once a month, and we need them more often than that. So, timewise, that’s been a sticking point with some of our providers. -P008 I do think we should have more targeted mental health services for parents. I think that those parents “deserve” the ability to participate in Early Childhood Court just like any other parent does. It’s not punitive but it’s just – I think it’s a function of the providers that are currently available to us in our county, in our circuit. So, I don’t think it’s necessarily that there’s some conspiracy or some – what word am I looking for? Guideline that we shouldn’t be providing ECC services to parents who have several mental health issues that we don’t have the resources we need to address that in such a way that this early permanency can really be achieved for those parents. So, if we could have – identify a provider or providers who would be able and willing to cooperate with the ECC model and provide mental health services to parents in light of the fact that they’re participating in this baby court, that would be really helpful for the population of parents coming into the system in general. Besides that, I think for us it really is just lack of providers for mental health of parents. We have the other maltreatment pretty well covered in Early Childhood Court in our area but mental health for parents is definitely, um, something that hopefully, it can be addressed going forward if providers can be identified. -P003 Providers also mentioned that tailored services to the parents involved were needed, including those dealing with complex trauma and substance abuse: What would need to be improved? For my particular area, I would really like us to see some more – I hate to say traumainformed, but anything that they understand that parents have been through trauma, and that most of our services are not geared towards parents who have been through significant complex trauma, and that is a problem. So, I would like to see more of that. I would like to see more services for the adults, and I would like to see our system of domestic violence evaluation being more streamlined. -P005 Relapse is part of recovery, and at the same time, to at what point this is not really getting off the ground, kind of thing, which again goes back to what we have to talk about the substance abuse issue. We need what we lack in services for those people who need to go in immediately as an intensive in-patient treatments, of having beds available, because if we can get that right when their child is born, that is right, when a child is sheltered right, when they come to see me and get them a bed, I think they would have a much higher success rate, because they’re motivated. They really, in that moment, want their baby back. They want to get clean, but they don’t have that. “We’ll have a bed for you in six weeks,” well they want to get clean right this minute. They may not want to get clean in six weeks, you know? So, it can be very difficult. -P006 Another suggestion for improvement referred to the ECC timeline: That’s my biggest problem with – my biggest frustration with Early Childhood Court is these cases typically are not being resolved any quicker than the traditional dependency case models. I think a big part of that is a basic failure to understand FLORIDA INSTITUTE FOR CHILD WELFARE

conditions for return on part of both the case management and the judiciary. I’d like to think that I’m not biased based on my current role but I think that my experience has been that parents who are meeting the statutory requirements for either expansion of visitation or reunification aren’t being granted those because of the fundamental lack of understanding of what the statute requires as far as successful reunification. So, the cases are still asking, permanency is still talking sometimes up to a year to be achieved. I have now case that’s been opened since I believe the end of 2018 and we’re still handling the case. So this model of accelerated permanency for the child isn’t really coming to fruition probably in – I don’t think 80 % of the cases would be an exaggeration to say that we’re not reaching this permanency in any more of a timely manner than we would on a traditional dependency case. -P003 The same provider added that due to the shorter timeline, parents sometimes get discouraged with the process and that it was bad for a parent’s morale if they fulfilled every expectation with the promise that it would bring reunification sooner, and then that would not happen: “As far as mental health, I think that’s probably what I would identify as the downside to this delayed permanency despite the stated intentions of the Early Childhood Court model.” Another provided mentioned that the way ECC was set up made it difficult to set up parents for success: So that’s the problem I think in general any of the – from my perspective, any of the counseling services, which require – that are solely relying on a self-evaluation or a self-assessment, do not tend to be very helpful, because in general, I think a lot of these parents lack insights into their issues. I don’t find those to be very helpful. Again, this is departmentalized. I can appreciate that there are some people who they come in with this singular issue. They just have this history of bad relationships with men, so yes domestic violence-type counseling for victims. Yes, maybe that’s all they need, but they have this. They were also molested as a child. They were also self-medicating, and just send them to all these various places. I find that to be very – I don’t think that that’s very helpful, because admittingly, that’s what these people lack, transportation. They lack motivation. They lack a positive support system. So, they might get motivated to do one, but it gets overwhelming to do all of it, especially when you have the pressure of the timeline each child lacks. -P006 Several providers mentioned one concrete improvement would be to foster an even more non-judgmental culture in ECC so providers would approach the cases without preconceptions: So, this might be a bit controversial, but the area that I see, as sitting in my chair, in my neutral chair, I really see that we need more parent support and encouragement, and to really be able to look at cases with a fresh perspective, without any preconceived thoughts or past experiences from other cases that cloud our ability to work with, support, encourage, and forgive families. I say forgive, because I think that we sometimes in child welfare can hold on to mistakes that parents have made, and really don’t give them a lot of grace to – while they’re working on making changes, I think our expectations sometimes are too high that everything’s going according to plan, and we don’t offer a lot of forgiveness when there’s a bump in the road. So I definitely would like to see us be more strength-based, where we’re focusing on what the parents are doing well and giving them that motivation that they need to really dig deep and do the really hard work that’s coming in order for reunification to happen. I think that’s where I see some of our like therapeutic approaches kind of falling short. I definitely think we could improve the relationship that we have with our ECC families. -P001 Such a non-judgmental approach would also support relationships between and among parents and caregivers:

There are so many different types of caregivers and there are so many different relationships, especially for relative and nonrelative caregivers that exist for maybe decades before we’ve even been in the picture that a lot of times there are dynamics that exists between the parents and the caregivers that can lead to a breakdown of communication, can lead to parents feeling like the caregiver’s being favored over them in certain circumstances. So it’s going to be very individual because there’s only so much, for instance, licensed foster parents and a natural parent are going to be able to do to have a relationship that really best serves the child’s development and the child’s best interests because there are just natural limitations built into being a foster parent that it’s just – there’s nothing I can think of necessarily that would help to improve that relationship other than foster parents just being willing to be an open line of communication because I have a lot of parents who don’t have any direct communication with their children’s caregiver because they’re licensed parents and they don’t want contact information to be included with the parent, which I completely understand. Understandable why that would be the case but it can be harmful and lead to a parent feeling a sense of disconnect from a child versus licensed foster parents who maybe have a separate line, a separate cellphone for parents to contact them or they could be speaking to the children in between visitation time or family time or being able to have that open line of communication to speak directly to the foster parent about medical issues for the child and mental health needs that are being noticed by the caregiver. So if they can have that partnership type relationship, I think it makes outcomes more successful ultimately for everyone who’s involved. -P003 Open communication, frequent family team meetings, coordination with the team between both of them, just collaborating as a whole with the team collaborating, having the parents and the caregivers collaborate, I guess. -P007 Another participant said that for ECC to be improved, parents need to feel like they had agency in the process. However, according to one provider, parents sometimes ended up in ECC due to the wrong reasons: The biggest problem I have with ECC is that sometimes, parent lawyers can kind of shoehorn the parent into ECC, and ECC is supposed to be a voluntary thing, but these attorneys understand that the parents get a lot more services a lot more quickly, they get in front of the judge a lot more often, and so they advise these parents: “Oh no, you want ECC,” and parents are not in a place where they are open to – or interested in listening to the fact that there was a problem to begin with, and that the answer is going to require work on their part. So, what is happening is we get parents in our program who are incredibly frustrated, and that frustration, there’s no way that the children are not taking up on that frustration. So, I think that that is detrimental to their mental health. I really do feel like – we had a judge in 2019 who would frequently tell our parents that ECC was a privilege, that it was not a right, and that is when you came to ECC, there was an understanding that you were agreeing that there was a problem and that you wanted help with that problem. When that happened, it was a beautiful, amazing, profound process. It’s the most amazing thing I’ve ever seen, but when they come in and they’re like “You took me kids and I wasn’t doing anything wrong,” then everybody’s frustrated. So, that’s my biggest problem with ECC, is that the attorneys, the defense attorneys can push to get people in ECC who are not in the space that is going to make it helpful. So, then, every month, we’re having that conversation, and I think that’s detrimental to everybody – well, to them, actually. It’s not helpful to them, and I think it’s detrimental. -P002

FLORIDA INSTITUTE FOR CHILD WELFARE

Another provider mentioned that there needed to be a more comprehensive system in place to ensure families were a match to ECC: If we could have a comprehensive assessment of the child, a comprehensive behavioral assessment of a child prior to the decision being made whether or not to accept a case into Early Childhood Court, I think that could be crucial in determining which families would be the best match for the program but I just don’t think we would be able to get assessors to speak to the families and observe the children in a timely enough manner to make the decision whether or not to allow the family to have the opportunity to participate in ECC. So, it’s very informal. It’s very quick. -P003 Another provider mentioned that communicating more effectively and efficiently—through sending fewer emails— would leave time for the daily work of ECC as well as professional development: Part of the issue I think is, and I have voiced my concern to the offices, OSCA, what we call it, as well as local court admin, is that there’s not a lot of time to do the training. There’s a lot of training offered, but not a whole lot of facilities to advantage every class. Every class that comes up that I may want to take because I have dockets. That’s part of the issue. Additionally, with the emails, the email situation I think has been out of control as it relates to – for juvenile judges in general, there’s like an email from every agency you can think of, from every community organization you can think that’s offering something, and it could be very difficult to sift through all those emails. -P006 Finally, the pandemic was mentioned as something that affected the therapeutic components of ECC and that it should be taken into account regarding potential improvements, particularly how the pandemic had an impact on how parents could access therapeutic services: I think the only thing is that COVID-19, just like with everything, has slowed down the process. A lot of places were shut down for quite some time before they either figured out how to do it virtually or until local governments kind of started opening back up. A lot of places were closed down and not accepting new clients or not doing in-person, and even just the simple fact of losing employees due to COVID-19. Right now, I’m sure it’s happening where you’re at too. Everywhere is hiring right now, because they lost so many employees during the pandemic, or at least during the worst of the pandemic. So, I think that’s affected it as well. Therapy services might have lost some of their therapists due to it, and making it harder for them to take on new clients. So, that then speaks to pushing the parents behind on their case plan, because they haven’t been able to start their mental health assessment, their substance abuse assessment, or any kind of assessment for therapy that is required in their case plan. -P004 I know it disrupted child-parent psychotherapy for several months because they were trying to figure out how can we take what’s supposed to be a hands-on bonding therapeutic session and translate it to electronic means. So, I think we had lots of families who weren’t having child-parent psychotherapy for extended periods of time because just practically there wasn’t a way to get it done. I think once it became more apparent that the pandemic was going to be an extended period of time that providers just had to work with what they had as far as getting families intakes done and evaluations done. With people losing work, having trouble with transportation to getting the service providers could be an issue. I have a lot of parents who don’t regularly have access to phone or data that rely on maybe just a phone that can only be used when it’s connected to Wi-Fi. That leads to

difficulties contacting case managements, contacting providers, contacting counsel. I feel like there’s a different feeling now that we’re back in-person in our monthly status conference hearings. There’s just a different dynamic that being in-person brings that I don’t know if I can quantify really or explain because I just don’t have the base of knowledge for that sort of sociological question but definitely being in-person in court has made the program feel more real again. -P003 So, everybody in this area went virtual last March or April, and they were all predominantly virtual for about six months. Some of the others started to do face-to-face, or CPP providers started doing face-to-face three or four months ago. Me and the domestic violence and the mental health are probably going back to face-to-face in June or July. So, two things happen. One is that services became more available to most clients – actually, three things happen. One, clients had to have a phone, and before, they could hop on a bus or get in the car and show up at your office, but they couldn’t do that this time. They had to have a phone, or they could not work for any of the services. They couldn’t attend ECC, they couldn’t do CPP, they couldn’t do anything. So, they had to have a phone, but if you had a phone, which most people do, the services were more readily available. It was easier for the clients to get staffing, it was easy for them to get to court, it was easy for them to attend their meetings. So, that was kind of interesting, and I know that we were all – the service providers were all kind of looking at how much we’re going to keep virtual versus face to face because of the ease of it for the individuals. Now, the third thing that happened is that there – the biggest thing that I saw in court is that the clients were considerably more disrespectful than they ever were in a faceto-face court. So, the judges had to tolerate a lot of behaviors that did not happen when they were sitting in the court room, and I know, from a substance abuse standpoint, that the more a client has to work to get what they want, the better those lessons stick. Before COVID, I wouldn’t do an intake unless the person came to my office, showed up in my office, and did the intake. So, if they missed appointment, then they never got services, but these clients, they call me on the phone, they’re in, they’re in services. I looked at my numbers last year, and I noticed that my success rate took a big hit between March and June. I went from 60% success rate to a 30% success rate, but then, over the next six months, it leveled out. So, either I or the clients got used to the video services, and it ended up working out better for them. So, those are the things that I saw. -P002 In addition, providers mentioned that they sometimes had difficulties keeping parents engaged in the therapeutic process. Canceling appointments was mentioned as a barrier to the therapeutic process in ECC. Potential improvements regarding this matter should also incorporate notions from parents and caregivers and the many reasons they may have for canceling: We have one case that’s been a severe problem that the parents constantly blame the Department for not bringing the children for visits, but they also cancel a lot of visits. So, this is a constant thing we have to keep coming back to that I can’t recommend that they get re-unified because they simply haven’t had enough CPP sessions. So, that is one of the barriers is just constantly cancelling, rescheduling, saying they can’t do a visit, or they can’t do a session. That’s the biggest problem. -P005 One potential improvement is to ensure ECC therapeutic activities align with the parents’ schedule. The same provider mentioned above stated: They are typically held during the week, and I can’t speak for the ECC coordinator, but I think they tried very hard to work it around the parents’ schedule. So, if somebody is working, we’ll ask what day they are off or something like that. -P005

FLORIDA INSTITUTE FOR CHILD WELFARE

Evaluation Question 4 To what extent do participating caregivers and providers perceive that the therapeutic components of ECC contribute to racial inequity or equity? Parents/Caregivers Participants were asked if they ever experienced or witnessed any bias, unfair treatment, discrimination, or microaggressions (defined/ explained to participants within the interview protocol) while in ECC. The majority of participants reported that they did not experience any bias, unfair treatment, or discrimination based on race/ethnicity, cultural background, sexual orientation, culture, or appearance. One participant declared, “I’ve always felt completely respected and I always felt like I was treated fairly and not less than. I guess that I’ve always felt very blessed to have a positive situation.” -B005 However, only a small number of parents/caregivers reported their racial or ethnic identity and those who did, identified as White. Due to the small number of parents and caregivers that reported race/ ethnicity information and the lack of diversity among participants, further evaluation regarding racial bias, unfair treatment, discrimination, or microaggressions while in ECC is warranted. Conversely, it was noted that sometimes the primary caregiver is male, yet the system and some providers seem to be geared towards assisting women. Other participants felt they may have been judged unfairly or stereotyped by either ECC team members or their child(ren)’s caregiver based on their experiences with substance use and domestic violence. For these participants, it made it harder to open up and fully engage in the program. One participant explained their experience with the ECC and the caregiver in the following ways: Just the way that they would look at me during the meetings and stuff…and I was honest every time with them, any time I had a slipup, I didn’t hide it. When I would be sharing why I slipped up and stuff like that, the looks that I was getting just wasn’t something that somebody like me sharing and opening up needed to see. I just feel like I was looked at a certain way when I failed at drug test and sharing my relapse or something because I feel like, if you’ve never really been an addict - there are different addictions, but certain ones are a lot stronger than others. I feel they don’t really know…It’s not that easy. -B002 Providers Providers were also asked about how they perceive that the therapeutic components of ECC contribute to racial and social inequity or equity: I mean, we, of course, have not as diverse a population as a lot of areas, but we do have some diversity in our population. We obviously, don’t base any of our admissions on anything related to anything but the circumstances of the case, the age of the children, and things of that nature. I know we’ve got a lot of people on our team. For instance, one of our guardian[s] ad litem, we had an African American child placed with a White family. The guardian ad litem is African American. She was able to kind of help that foster parents understand the child’s…as little as it sounds to us…their hair care needs. Things like that are big in the African American culture. -P008 One provider mentioned that foster parent training should address race more explicitly: Well, I’m not sure exactly what all is covered in the foster parent training. I haven’t sat through the entirety of their classes, but I think making sure that we have those different things to talk about, we have to talk about them to be able to address them, to be able to get kind of an understanding of them. So, that’s where I think we should start. It’s making sure that those things,

different cultures and different things like that are talked about and training these folks so that they understand going in if they get a child or a different race or a different culture that they have heard, “Hey, this is what I remember hearing about that particular culture and maybe we can seek out some folks to help us, make this child a little bit more comfortable in our home.” But always that even they’re just babies, making sure that we’re honoring the culture of their family of origin and making sure that if they are unified with their family that they haven’t completely forgotten about everything that was going on with their family, and the background of their family before they were removed. -P008 One provider mentioned they were, as a provider, seeking professional development opportunities to learn more about the racial and social inequity: As the provider, as a clinician, it’s part of my psychosocial [intake process]. I have to talk about cultural issues, and I’m always looking to make that better. I just went to one of those Zero to Three trainings where they were talking about assessments and culture because I feel like I can always learn something more about what I’m not thinking of. -P005 The same provider mentioned that race was not a topic they thought about regularly in the everyday of ECC work: I haven’t really thought about that too much, but that is something that I could add whenever I first evaluated the children, you know, ‘Here are some cultural things I’m seeing or that might be something we need to address. -P005 Another provider mentioned they would like to have access to more training on topics related to racial and social equity, as well as dealing with a diverse client base. They also highlighted the need to individualize therapeutic services and supports: I honestly don’t know if I’ve seen anything like that. I haven’t received any training regarding that. I haven’t seen any opportunities for training regarding that. Home studies do include questions regarding a caregiver’s or the parents’ ethnic background, their religious backgrounds, their – any specialized class of people whether the caregivers or parents be homosexual or trans or have any sort of special classification, special protections under the constitution regarding any of those demographics. Those are in-home studies and those are referenced in their family functioning assessment, but I don’t know of any way they’re being taken into account to really mold services provision in light of those cultural, ethnic, [gender identity] characteristics. I don’t know of anything that’s really being done to individualize services in that regard. -P003 Several providers mentioned they had not witnessed biases or discrimination in ECC. However, they also stated that they may have not been able to detect biases that others may have experienced: Anything bad in regard to that, though I have all different backgrounds in ECC, and my experience is that they’re all treated equally, and I don’t see a difference from one case to another based on race or event – or social standing. -P004 I recognize that I have a lot of privilege that a lot of families coming into the Early Childhood Court system don’t have. So, I could naturally – not naturally but just because of systemic factors, I could have blinders on and not be aware of any biases that may exist, but I haven’t noticed anything in particular that I could point to. -P003 I don’t think I’ve seen that. Our teams are very diverse. We have people from all backgrounds on our teams. So, I don’t recall ever having witnessed that with our teams. I don’t think it’s really

FLORIDA INSTITUTE FOR CHILD WELFARE

something that we – I mean, like I said, all of our cases we treat everybody equally. Everybody has access to the same services. We do everything for one parent that we do for the next parent. -P008 Well, I know we don’t discriminate. Right now, we have a couple who are both female and when we get the initial report, we don’t know the backgrounds of our clients, nor is that ever brought up. We will work with any race, any gender, any couple. We acknowledge homosexual, heterosexual, and any form of gender identity couple, and any race or ethnicity. We’re not biased to any of that. -P007 Another provider said they recognized how race mattered in the ECC system, despite earlier attempts to be “colorblind”: Well, it is a loaded question because again I think I come from a perspective where just to assess people blind for so long I thought the right answer was not pay attention to race, to treat people as people no matter what. Now I think there has been a big shift in that we are, so to speak, consciously looking at this issue and I think, again, not that that comes as positive because again, I do think when you look around the room, if you’re the only woman or if you’re the only Black person or the only Hispanic person, that that is something that’s very obvious to you and maybe it would be obvious for someone, for me. It’s a very touchy subject and I do think that there – again, you’re right that there are some cultural issues that create added strain in EC court. -P006 A similar remark was made by another provider who was cognizant of the fact that race mattered in ECC, and that this led to them finding ways to work around such biases: I know of all of the service providers; they have mandated diversity training. We all do the best that we can to try to be of assistance to make sure that we’re hearing the people that we’re working with. I’m trying to think. One of the things that I do – and it’s funny. I’m White, I’m Caucasian, and I have a coworker who’s African American, and in my county, it’s about 60% Caucasian, 35% African American, and about 5% of others, and what I have noticed in the 10 years that I’ve been working at this job is that the racial lines for my clients kind of leave out that way. I have very few Hispanic clients and no Asian clients. So, if I feel like I’m not connecting with an African American client, I will run something by my African American co-worker, and I feel like the other service providers in that room are doing things to that nature as well. I think it comes down to us making the effort to listen and be as attentive as we can. I would like to think that most of the time, what we react to, and we respond to is the client’s actions and not the color of their skin. I know that we have had – it’s predominantly – it’s probably – the staff and staffing – let me see all their faces. It’s probably 80% Caucasian, 20% African American in the staffing, and I know that we have had Caucasian clients that we – it was just all we could do to hang onto that client-centered book for the positive, do the best you can, and we’ve had African American clients that do so extremely well, but sometimes, we forget that we have them as clients because they’re doing so well. So, I hope that that is as true that it can be, but I’m not naïve to the fact that there is absolutely favoritism. Both of our judges are Caucasian; all of our judges are Caucasians, and we don’t have any people with color on the bench in [our] County that I know of – well, not in ECC anyway, only for like that one woman, but she’s pretty cool, I like her. -P002 This person added: I would assume that a Black client would feel more persecuted by a White judge than a Black judge, but that may not be true.

So, I don’t know. I’ve never had a Black judge. Some of my Black clients like their White judges, and some of them can’t stand them, but I don’t know if that’s because of that additional – I don’t see, and we’re in the South, too, so there’s a lot – I don’t know. -P002 In addition to racial equity, providers also mentioned examples of how approaches used in ECC were not always applicable to individuals from diverse backgrounds. For example, one provider mentioned that the ECC system assumes heterosexuality, so they needed to individually worked towards inclusivity with a same-sex couple: Again, from a systemic issue, the clerk called and said, ‘We aren’t set up to have two mothers or two moms. How am I supposed to put this in the system? Who is the mother and who is the father?’ We don’t have an option. This is just how the system is set up. Instead of parent one and parent two, it’s mother and father, even though they’re both legal mothers. That’s the conversation I’d have, and I just try to have that conversation with them. ‘How do you want me to refer to you as? What is the baby going to call each of you?’ She did have a biological mother. I mean, a mother who carried her. We had that conversation but sometimes I just think that’s – I mean again, especially when I’m feeling the teens are struggling with the identity and how they identify, it’s all those. Parents who don’t accept that and non-binary in terms of how they want to be called versus a boy or a girl and vice versa. I feel like across the board, it’s just I just try and say, ‘What is it you want me to call you? That’s what we’ll call you.’ Again, I’m sure I’ve gone through extensive training. Of course, it’s just different in “real life” in a class. -P006 Another provider mentioned how clients with a lower socioeconomic status were disadvantaged in the system: We make efforts to be sure that [clients with low socioeconomic status] have transportation assistance, that they are in contact with resources that can help them with food, that can help them with housing, almost all of the services that we provide are free. The only ones that aren’t are the batterer intervention courses, they have to pay for, but everything else is free. …the courthouse itself is situated two blocks away from a bus terminal. So, to just kind of aim anybody – so, if anybody can get on a bus, they can get to the courthouse for the staffing and for the hearings. So, I think every effort it made particularly for the lower socioeconomic classes. -P002 DISCUSSION Summary of Findings While the focus of ECC and the Child Welfare System is the safety and wellbeing of the child, ECC places emphasis on therapeutic components that support the parent’s mental health and parenting approaches, and the parent-child relationship. The results of our evaluation of parent-reported therapeutic networks were telling, illustrating those ECC and non-ECC members who play a critical role in parents’ and caregivers’ therapeutic processes. The therapeutic network and influences include the important role of the therapist but also extend to the community coordinator, intimate partner violence and substance use disorder providers, and also family members. As the most mentioned person in the therapeutic network, the community coordinator was seen as particularly essential to the parent/caregiver therapeutic network. They play a role in creating a welcoming and supportive environment for the parent/caregiver and serve as a connection between the providers in the therapeutic network as well. In addition, therapeutic networks often include other family members and/or caregivers, counselors from intimate partner violence and substance treatment programs, as well as child-parent psychotherapists. FLORIDA INSTITUTE FOR CHILD WELFARE

The interviews highlight the importance of understanding the complexities of providing services for individuals experiencing multiple traumas. Trauma-informed care was mentioned by both providers and parents/caregivers as a crucial component of the therapeutic process in ECC. Other stressors like substance use, violence, socioeconomic situation, and life circumstances, along with having the child removed, compound past traumas that can be triggered by the ECC experience—that should be recognized, acknowledged, and addressed. Substance use emerged as a significant theme for ECC program participants in the context of the trauma-related evaluation question. Parents formerly or currently experiencing substance dependence referenced the need for providers/ECC staff members to gain further training on substance dependence and recovery, suggesting that all ECC team members should receive training on what substance use is like for those experiencing it (empathy), as well as increased education on addiction in general. Evaluation Challenges The USF evaluation team faced some challenges during the conduct of this work. Even with full support of circuit court judges, OCI, and the ECC state lead who facilitated reminders to staff throughout the state (we were unable to contact parents directly), participant recruitment was challenging; participants from only one circuit responded. While in-person recruitment may have been preferred, potential participants and those helping with recruitment were contacted primarily through email due to the constraints of the ongoing COVID-19 pandemic. Since school, employment, and program services may have transitioned to a virtual format for some individuals, they may have been inundated with emails/virtual work requirements and may not have read the recruitment flyer, did not want to participate in a virtual interview, or may have forgotten to respond to the interview announcement due to their busy schedules. In addition, recruitment for other evaluations that involved ECC providers were simultaneously occurring, which may have resulted in provider fatigue with participation and/or confusion as to whether they previously participated in the evaluation. Another challenge related to recruitment included contacting and successfully interviewing members of the parent/caregiver network who were referred by other interview participants. The evaluation team made every attempt to contact all members of the therapeutic network mentioned by participants during the interview. Interview participants were invited to recommend others in their therapeutic network (including other caregivers) for an interview; parents/ caregivers were encouraged to share contact information with the evaluation team and to let any potential participants know that the evaluation team would contact them via text. Still, there was some difficulty in contacting referred parents/caregivers. To protect the anonymity of parents, providers were not informed nor asked the names of parents they worked with. Thus, as shown in the network map, providers in particular roles identified by parents as therapeutic were interviewed but may not have been connected to each particular identified parent. Some parents named particular providers who were recruited for interviews; other parents noted provider roles or agencies. Limitations While efforts were made to recruit participants across the state of Florida, the evaluation findings are reported for one court circuit across two counties. Therapeutic networks and ECC experiences may vary in other regions of Florida. However, the results of this evaluation help us understand the complexities of the parent and caregiver experience in ECC. In addition, part of the focus of our evaluation was to assess perceptions of racial equity and inequity in ECC. The reported racial/ethnic demographics of the evaluation participants are not diverse enough to fully elucidate the various perceptions of racial equity and inequity in ECC from individuals 37

of diverse cultural backgrounds. Some of the parents/caregivers did not disclose their racial/ethnic identities. Furthermore, the majority (63%) of Florida’s ECC participants identify as white, and these interviews also occurred with participants from a less racially and ethnically diverse region of the state. Future evaluations may need to further investigate racial equity and inequity within ECC programming. Overall, our findings suggest that parents involved in ECC value the therapeutic approaches offered. In addition, parents noted that qualities such as encouragement, genuineness, empathy, and reassurance are essential for ECC team members when working directly with parents and caregivers. Parents conveyed that in order to feel part of a team that has their child’s best interests in mind, they need to feel like the ECC team is “speaking up for them” when misunderstandings, misrepresentation, or bureaucratic issues occur within the broader child welfare system. While parents did not articulate that ECC was discriminatory, it was noted that some individuals in the system—particularly DCF—were less respectful, reliable, and supportive than others. These factors play a significant role in whether or not the parent/caregiver truly engaged in the program and essential therapeutic processes and influenced the parent/caregivers’ overall well-being.

Chapter 5: Secondary Data Analyses of ChildParent Psychotherapy Outcomes Michael Killian, Taylor Dowdy-Hazlett, & Lisa Magruder.6 To complement the provider data, the evaluation team conducted secondary data analyses of administrative data to assess outcomes by treatment modality. Availability of data were limited in that variables related to therapeutic modality are only standardly collected for CPP; there are no standardly collected variables on COS-P or other parenting or parent-child relationship variables. Thus, analysis of administrative data focused on CPP. Importantly, there were significant missing data; results should be interpreted conservatively. METHODOLOGY Data Collection The OCI, based on available administrative data and consultation with the evaluation team, provided de-identified data for all closed ECC cases between 2014 and 2020 (N = 805). Given that all cases were closed, all were eligible for inclusion in analyses regarding families’ CPP involvement (yes/no). In addition to CPP involvement, variables included demographics, case details (e.g., reasons for referral, substance use), and outcomes (i.e., re-removal, time to permanency, permanency type). Data Analysis The analyses in the next section tested for differences between those ECC-involved families receiving and not receiving CPP (hereinafter, CPP and standard services, respectively). Statistical analyses testing for differences between these groups of parents included chi-square (χ2) and independent samples t-tests. When a characteristic of the families is a category (e.g., gender with male and female, abandonment with yes and no, alcohol with yes and no), a χ2 test examines if there is a greater proportion of cases from one category in the CPP group or in the standard services group. The t-tests examine if the mean of a characteristic is statistically significantly higher in either group compared to the other (e.g., days from removal to reunification, number of days taken to close the

case). Statistical significance is provided in terms of the p-values for each analysis (* p < .05, ** p < .01, *** p < .001). For each test, the size of the difference between the two groups is provided as well. These effect sizes are phi (ϕ) for χ2 tests and Cohen’s d for t-tests. A test may be statistically significant (p < .05), but still, it may not demonstrate practical difference (i.e., a small overall effect or effect size). For both ϕ, 0.1 is considered a small effect, 0.3 a medium effect, and 0.5 a large effect. For Cohen’s d, effect sizes are considered a small (d = 0.2), medium (d = 0.5), and large (d = 0.8). RESULTS Generally, there were very few differences between those families participating in CPP (n = 128) and those receiving standard services (n = 677). These results are provided in Table 4. There were slightly more females in CPP (n = 74, 57.8%) than receiving standard services (n = 338, 49.9%), yet the overall proportion of females in both groups were not significantly different. Race and minority status were not significantly different between the two groups. Cases with no court approved case goals were in the majority (n = 464, 57.6%). While not a significant difference, CPP cases tended to include slightly more adoption cases and those with “maintain and strengthen” as the case goal compared to standard services cases. There were no significant differences between the CPP and standard services groups based on the number of re-removals during or after the CPP program period. Case Characteristics Across nearly all case characteristics, the two service groups did not significantly differ (Table 5). Reason for case referral were not significantly different between the groups except for reasons involving domestic or family violence and failure to thrive. CPP involved families had a significantly greater proportion (χ2=5.44, p = .020, ϕ=0.08) of domestic or family violence (n = 67, 52.3%) compared families receiving standard services (n = 279, 41.2%). Though very few cases, there were slightly greater proportions of families (χ2=5.55, p = .019, ϕ=0.08) that were considered Failure to Thrive in the CPP program (n = 4, 3.1%) compared to standard services (n = 5, 0.7%). Similar to the case characteristics, nearly all substance abuse and use categories were proportionally similar between the CPP and standard services groups (Table 6). The only observed difference (χ2= 8.17, p = .004, ϕ=0.10) was that standard services families had a greater proportion (I = 149, 22.0%) of other illicit substances when compared to the CPP families (n = 14, 10.9%). Scores for parents on various measures are reported in Table 7. Outcomes Permanency outcomes differed significantly between the groups (Table 4). A significantly greater proportion of CPP cases (χ2=7.10, p = .029, ϕ=0.09) achieved permanency with one or both parents (n = 79, 61.7%) when compared to the standard services group (n = 334, 49.3%). However, it should be noted that this analysis excluded cases that closed due to voluntary dismissal, unknown/ unspecified reasons, or death as there were so few of these case outcomes in the evaluation. There were no differences in the proportion of termination of parental rights (TPR) cases between the two services groups. While number of days from recent removal to reunification did not significantly differ between the groups, the number of days from removal to case closure was significantly more (t = 2.31, p = .022, d = 0.22) in the CPP group (mean = 632.92, SD = 247.91) compared to the standard services group (mean = 577.38, SD = 253.63).

6 Suggested Citation: Killian, M. O., Dowdy-Hazlett, T., & Magruder, L. (2021). Chapter 5: Secondary data analyses of child-parent psychotherapy outcomes. In L. Magruder, 2020-2021 Early Childhood Court Evaluation (pp. 38-42).

FLORIDA INSTITUTE FOR CHILD WELFARE

DISCUSSION Summary of Findings Few differences emerged between families that received CPP services compared to services as usual. The proportion of families referred to services for domestic or family violence or failure to thrive was higher in CPP-involved families, while proportionally more families with illicit substance use received standard services. In terms of outcomes, a significantly greater proportion of CPPinvolved families achieved reunification with one or both parents compared to those not involved in CPP. No differences existed between the groups for re-removals or an outcome of TPR. Although time to reunification did not differ between groups, time to case closure was significantly longer for those in the CPP group. Notably, there was significant missing data for parent and child assessments. Limitations The present analyses are limited by data availability and inconsistencies in reporting. Regarding data availability, treatment details (e.g., completion of services, assessment scores) are optional fields in the OCI’s data management system (Leigh Merritt, personal communication, March 11, 2021); thus, some cases had missing data on these variables. The greatest frequency of missing data was observed in assessment scores for both children and parents, nearing 100 percent for parental assessments. Standardly collected pre- and post-treatment assessment scores would be useful in better assessing family progress in therapeutic treatment as well as case outcomes. For example, assessment scores could be used in predictive modeling of permanency type. Inconsistencies were also noted. For example, while some cases were coded as having received CPP services, their corresponding number of CPP sessions at exit were listed as zero. In some—but not all—instances, this appeared to be due to the TPR status of the case. Expert consultant Diane Koch (personal communication, September 5, 2021) also questioned why relatively few cases had a primary goal of reunification, as in her experience as a clinician, “reunification is always the goal when there is a case plan. Sometimes there is a concurrent goal of termination, etc. but until TPR happens, reunification is on the case plan.”

FLORIDA INSTITUTE FOR CHILD WELFARE

Table 4. Case Characteristics CPP Involvement No

Yes

Overall

(N = 677)

(N = 128)

(N = 805)

χ2=2.63, ϕ=0.06

Gender Male

338 (49.9%)

54 (42.2%)

392 (48.7%)

Female

338 (49.9%)

74 (57.8%)

412 (51.2%) χ2=6.90, ϕ=0.01

Race White

416 (61.4%)

73 (57.0%)

489 (60.7%)

Black/African-American

194 (28.7%)

42 (32.8%)

236 (29.3%)

Asian

0 (0%)

1 (0.8%)

1 (0.1%)

Multi-racial

62 (9.2%)

10 (7.8%)

72 (8.9%)

Race unable to be determined

2 (0.3%)

0 (0%)

2 (0.2%) χ2=0.64, ϕ=0.03

Racial majority vs. minority Minority

258 (38.1%)

53 (41.4%)

311 (38.6%)

Majority

416 (61.4%)

73 (57.0%)

489 (60.7%) χ2=4.77, ϕ=0.08

Primary case goal Adoption

Test Statistic

97 (14.3%)

23 (18.0%)

120 (14.9%)

Maintain and Strengthen placement with parent(s)

95 (14.0%)

22 (17.2%)

117 (14.5%)

No Court Approved Goal

393 (58.1%)

71 (55.5%)

464 (57.6%)

Permanent Guardianship

49 (7.2%)

4 (3.1%)

53 (6.6%)

Reunification with parent(s)

35 (5.2%)

8 (6.3%)

43 (5.3%)

636 (93.9%)

124 (96.9%)

760 (94.4%)

Yes

41 (6.1%)

4 (3.1%)

45 (5.6%)

χ2=4.77, ϕ=0.08

Re-removals after ECC start and before close?

χ2=0.03, ϕ=0.01

Re-removals after closing? No

627 (92.6%)

118 (92.2%)

745 (92.5%)

Yes

50 (7.4%)

10 (7.8%)

60 (7.5%) χ2=7.10*, ϕ=0.09

Permanency Outcome Adoption

263 (38.8%)

39 (30.5%)

Guardianship – Non Relative/Relative

68 (10.0%)

8 (6.3%)

76 (9.4%)

Permanency - One or Both Parents

334 (49.3%)

79 (61.7%)

413 (51.3%)

302 (37.5%)

χ2=0.16, ϕ=0.01

TPR case? No

485 (71.6%)

91 (71.1%)

576 (71.6%)

Yes

192 (28.4%)

37 (28.9%)

229 (28.4%)

Mean (SD)

304.10 (230.85)

317.46 (174.66)

306.47 (221.78)

Median [Min, Max]

248 [2.00, 1810]

272 [82.0, 930]

251 [2.00, 1810]

Mean (SD)

577.38 (253.63)

632.92 (247.91)

586.20 (253.39)

Median [Min, Max]

517 [66.0, 1670]

568 [306, 1290]

524 [66.0, 1670]

Number of Days Recent Removal to Reunification t=0.579, d=0.06

Number of Days Removal to Closure t=2.31*, d=0.22

* p < .05, ** p < .01, *** p < .001

FLORIDA INSTITUTE FOR CHILD WELFARE

Table 5. Reasons for Referral and CPP Involvement CPP Involvement

Abandonment, n(%), yes

Yes

Overall

(N = 677)

(N = 128)

(N = 805)

1 (0.8%)

22 (2.7%)

χ2=2.18, ϕ=0.05

21 (3.1%)

Test Statistic

Bizarre Punishment, n(%), yes

2 (0.3%)

0 (0%)

2 (0.2%)

χ2=0.38, ϕ=0.02

Death of sibling, n(%), yes

3 (0.4%)

0 (0%)

3 (0.4%)

χ2=0.57, ϕ=0.03

279 (41.2%)

67 (52.3%)

346 (43.0%)

χ2=5.44*, ϕ=0.08

Emotional abuse/mental injury, n(%), yes

2 (0.3%)

2 (1.6%)

4 (0.5%)

χ2=3.50, ϕ=0.07

Failure to protect, n(%), yes

63 (9.3%)

7 (5.5%)

70 (8.7%)

χ2=2.00, ϕ=0.05

Failure to thrive, n(%), yes

5 (0.7%)

4 (3.1%)

9 (1.1%)

χ2=5.55*, ϕ=0.08

Domestic or Family Violence, n(%), yes

74 (10.9%)

17 (13.3%)

91 (11.3%)

χ2=0.59, ϕ=0.03

Human Trafficking, n(%), yes

3 (0.4%)

0 (0%)

3 (0.4%)

χ2=0.57, ϕ=0.03

Inadequate housing, n(%), yes

53 (7.8%)

6 (4.7%)

59 (7.3%)

χ2=1.56, ϕ=0.04

136 (20.1%)

22 (17.2%)

158 (19.6%)

χ2= 0.57, ϕ=0.03

Malnutrition/Dehydration, n(%), yes

8 (1.2%)

4 (3.1%)

12 (1.5%)

χ2= 2.77, ϕ=0.06

Medical neglect, n(%), yes

31 (4.6%)

6 (4.7%)

37 (4.6%)

χ2= 0.002, ϕ=0.01

Mental health issues (of parent) , n(%), yes

90 (13.3%)

12 (9.4%)

102 (12.7%)

χ2=1.49, ϕ=0.04

Neglect, n(%), yes

78 (11.5%)

13 (10.2%)

91 (11.3%)

χ2=0.20, ϕ=0.02

Physical abuse, n(%), yes

39 (5.8%)

3 (2.3%)

42 (5.2%)

χ2= 2.54, ϕ=0.06

Sexual abuse, n(%), yes

2 (0.3%)

0 (0%)

2 (0.2%)

χ2=0.38, ϕ=0.02

Substance abuse (of parent), n(%), yes

463 (68.4%)

84 (65.6%)

547 (68.0%)

χ2=0.38, ϕ=0.02

Substance-exposed newborn, n(%), yes

133 (19.6%)

24 (18.8%)

157 (19.5%)

χ2=0.05, ϕ=0.01

39 (5.8%)

8 (6.3%)

47 (5.8%)

χ2=0.05, ϕ=0.01

Hazardous conditions environmental hazards, n(%), yes

Inadequate supervision, n(%), yes

Other removal reason, n(%), yes

* p < .05, ** p < .01, *** p < .001 Table 6. Substance Use and CPP Involvement CPP Involvement

Alcohol, n(%), yes

Yes

Overall

(N = 677)

(N = 128)

(N = 805)

13 (10.2%)

96 (11.9%)

χ2=0.45, ϕ=0.02

83 (12.3%)

Test Statistic

Amphetamines, n(%), yes

6 (0.9%)

3 (2.3%)

9 (1.1%)

χ2=2.07, ϕ=0.06

Benzodiazepines, n(%), yes

46 (6.8%)

15 (11.7%)

61 (7.6%)

χ2=3.73, ϕ=0.07

203 (30.0%)

43 (33.6%)

246 (30.6%)

χ2=0.66, ϕ=0.03

3 (0.4%)

0 (0%)

3 (0.4%)

χ2=0.57, ϕ=0.03

140 (20.7%)

26 (20.3%)

166 (20.6%)

χ2=0.01, ϕ=0.01 -

Cannabis, n(%), yes Club Drugs, n(%), yes Cocaine, n(%), yes Hallucinogen/Dissociative, n(%), yes Inhalants, n(%), yes Methamphetamine, n(%), yes Nicotine, n(%), yes Opioids- Heroin, n(%), yes Opioids- Illicit and Prescription Medication, n(%), yes Other Illicit Substance Use, n(%), yes Over-the-Counter Medications, n(%), yes

0 (0%)

1 (0.1%)

0 (0%)

1 (0.1%)

121 (17.9%)

24 (18.8%)

145 (18.0%)

χ2=0.06, ϕ=0.01

1 (0.1%)

1 (0.8%)

2 (0.2%)

χ2=1.74, ϕ=0.05

131 (19.4%)

17 (13.3%)

148 (18.4%)

χ2=2.64, ϕ=0.06

10 (1.5%)

3 (2.3%)

13 (1.6%)

χ2= 0.51, ϕ=0.03

149 (22.0%)

14 (10.9%)

163 (20.2%)

χ2= 8.17**, ϕ=0.10

0 (0%)

Sedative Hypnotics, n(%), yes

1 (0.1%)

0 (0%)

1 (0.1%)

Synthetic Cannabinoids, n(%), yes

20 (3.0%)

2 (1.6%)

22 (2.7%)

χ2= 0.78, ϕ=0.03

Synthetic Cathinones (e.g. Bath Salts, Flacca), n(%), yes

1 (0.1%)

0 (0%)

1 (0.1%)

* p < .05, ** p < .01, *** p < .001

FLORIDA INSTITUTE FOR CHILD WELFARE

Table 7. Parent and Child Measures CPP Involvement (N = 128)

Child ACE Score Mean (SD)

3.45 (1.52)

Median [Min, Max]

3.00 [0, 7.00]

Missing

72 (56.3%)

Father ACE Score Mean (SD) Median [Min, Max]

8.00 (1.83) 8.00 [6.00, 10.0]

Missing

124 (96.9%)

Father PITA at Intake Mean (SD) Median [Min, Max]

22.3 (2.08) 23.0 [20.0, 24.0]

Missing

125 (97.7%)

Father PITA at Discharge Mean (SD) Median [Min, Max]

13.5 (17.7) 13.5 [1.00, 26.0]

Missing

126 (98.4%)

Mother ACE Score Mean (SD) Median [Min, Max]

6.33 (3.26) 8.00 [1.00, 9.00]

Missing

116 (90.6%)

Mother PITA at Intake Mean (SD) Median [Min, Max]

15.6 (9.32) 17.0 [1.00, 24.0]

Missing

123 (96.1%)

Mother PITA at Discharge Mean (SD) Median [Min, Max]

20.8 (11.4) 26.0 [1.00, 30.0]

Missing

Chapter 4). Notably, clinicians may be included as “care team members” as specific role identities were withheld for confidentiality purposes. SUMMARY AND TRIANGULATION OF FINDINGS Delivery of Therapeutic Services Overwhelmingly, providers reported using Child-Parent Psychotherapy (CPP) and Circle of Security-Parenting (COS-P) as primary modalities for ECC-involved families. While usually not explicitly named, some parents and caregivers described receipt of these services. Broadly, providers described the primary objectives of services as promoting attachment and addressing trauma. Though fidelity in service provision is desired, providers indicated that working with ECC-involved families involves taking a clientresponsive approach as opposed to following rigid protocols. Having flexibility to revise, pause, or otherwise deviate from a standardized course of treatment was described as a necessity, though participants did describe ways in which they follow “protocol,” including the general use of COS-P and utilization of CPP fidelity forms and reflective supervision. Generally, fidelity was more straightforward for COS-P given the semi-structured nature of intended implementation. That is, videos are standardly shown sequentially and involve post-viewing discussion, though the content of specific discussions might vary. CPP was described as more variable, even as a framework within which providers infuse their own practice-orientations. This incorporation of practice wisdom is in alignment with the transdisciplinary model of evidence-based practice,68 though does present challenges for highly structured evaluation of services. Indeed, the variation in CPP delivery could explain why few differences in outcomes were found between families that did and did not receive CPP (see Chapter 5). As a broader conceptualization of provision of therapeutic services, ECC as a whole emphasizes that there is “no perfect parent” and multiple parties recognized the need to encourage and support parents throughout the process:

119 (93.0%)

Chapter 6: Summary of Evaluation Findings

Clinician

So as far as positive experiences, when I get texts from people I worked with years ago and it showed little Johnny graduating from kindergarten, or the parent I haven't heard from in years texts me to say, “Hey, I got my GED.” And all they need is acknowledgement to say, “Yay, proud of you” and then I may never hear from them again. – On post-ECC support

Parent

The Care Coordinator is very genuine and caring, always asking how I’m doing. She takes me throughout the month as well, checks on me. That’s definitely what I like, too.

Care Team Member

If I knew at a team meeting last month that a parent had a job interview, I might follow up a day or two after the interview or the day or two before the interview to just give them like words of encouragement. “Thinking of you today. I hope you rock the interview,” or just checking in to see how things were going.

Lisa Magruder & Jennifer Marshall.7 The present evaluation aimed to evaluate the effectiveness of ECC therapeutic modalities related to parenting and the parent-child relationship toward the goal of ensuring quality, accountability, and fidelity of the programs’ evidence-based treatment.67 Given what little was known about how clinicians are providing therapeutic services to ECC-involved families, this evaluation focused heavily on the process of service delivery, particularly as it relates to COS-P and CPP. Outcome data for families is not uniformly collected, making child and family-level outcomes difficult to evaluate at this time. In addition to specific therapeutic modalities, the evaluation team expanded the conceptualization of provision of therapeutic services to include care team members identified by ECC-involved parents and caregivers as being integral to their overarching therapeutic experience as a participant in ECC. In this chapter, the evaluation team draws on multiple data sources to summarize the key findings of the evaluation. Please note, as described in this chapter, “clinicians” are those who deliver therapeutic services (i.e., COS-P, CPP; data derived from Chapter 2 and Chapter 3); “care team members” refers to broader team members, such as guardians ad litem (data derived from

Expert consultant Diane Koch (personal communication, September 5, 2021) reiterated how important this support is for not only the families served, but also for providers as a way of fully realizing the ECC model: …. It was summed it up very nicely in the write up when you said clients voiced that they needed encouragement, genuineness, empathy, and reassurance to be fully engaged. However, what still needs to be realized is that these qualities that people say they need are the same for clients and for providers and that if these can be accomplished then we have arrived at what the model is all about. This is the message underlying CPP and

7 Suggested Citation: Magruder, L., & Marshall, J. (2021). Chapter 6: Summary of evaluation findings. In L. Magruder, 2020-2021 Early Childhood Court Evaluation (pp. 42-46).

FLORIDA INSTITUTE FOR CHILD WELFARE

COS-P. Without this context of respect and openness, whatever specific information or treatment effort will be limited or not very meaningful. To assist in delivering quality services to ECC-involved families, most clinicians engage in and are satisfied with reflective supervision. It was described as helpful in improving confidence and success in service delivery as well as supporting the emotional needs of the provider. Outside of clinicians, other team members indicated a need for this type of support (e.g., “…some 1-800 number to call and talk to somebody…”), with at least one participant (a parent attorney) already engaged in therapeutic services to obtain emotional support related to their work. Parental Factors Influencing Delivery of Therapeutic Services Parents in the evaluation reported various trauma—past or present—such as substance dependence, intimate partner violence, and financial and housing instability. Across participants roles, intergenerational trauma was noted as particularly prevalent. As in a previous Florida ECC evaluation,1 primary issues of mental health concerns, intimate partner violence, and substance use disorders arose. Present participants focused heavily on substance use disorders as influencing services. Generally, participants expressed the need for empathy for parents experiencing substance use, with clinicians noting the necessity of balancing a parent’s need for services with their ability to meaningfully participate in treatment. In this vein, several clinicians indicated that sobriety is a condition of therapeutic services. Still, they recognized that progress is not linear, and that when parents relapse, offering support is essential:

Clinician

And so, getting them to be fully committed to a sober journey is a challenge sometimes to help them be in a space where, “How can I therapeutically work with you when you're still actively using to the point that, you know, you're not fully available to process these feelings that your kid is having right now?” But to still give the opportunity to be successful and not feel like, “Somebody else just gave up on me.”

Parent

It could cause – it triggers people having something traumatic happen to them. Then they’re going through this process with the court, and something could trigger them to either relapse or just throw up or something. That would be a problem or issue with the therapeutic part of it because it’s starting you – there are constantly some people starting over again and again because they keep having setbacks from a traumatic event that they’re not dealing with, maybe. So, when the trigger comes, it sends them all the way back to the starting line.

Care Team Member

Additionally, especially with these parents who are fighting drug addiction, which is most of our parents, relapse is part of recovery and trying to differentiate those who are serious, that aren’t struggling, versus those who really aren’t motivated to be clean.

More than any one specific parental issue, participants emphasized that parental readiness for change and general level of participation are indicative of “success” in ECC, particularly as it relates to the parent child relationship:

Clinician

Acknowledge the child's trauma, acknowledge its affect, and then acknowledge [their] responsibility in it. And if I have a parent who's not successful in CPP it's almost 100-percent, because they don't ever get that piece.

Parent

It’s one of those where the parents have to be willing to engage in the therapeutic process in order to be able to move forward and be able to even be more productive in those meetings, but they’ve got to be able to take those first steps.

Care Team Member

ECC parents who successfully engage in that tend to do much better than parents who don’t, as it relates to their relationship.

FLORIDA INSTITUTE FOR CHILD WELFARE

Still, it is important to consider parental engagement in therapeutic services—or lack thereof—within the context of the totality of their case. Expert consultant Diane Koch (personal communication, May 20, 2021) shared that parental lack of engagement may be due to fear of the system, which can complicate the therapeutic relationship: “Parents are in a legal system and are reluctant to say anything to be used against them. It requires quite an art to be able to carve out a therapeutic relationship in such a context.” ECC Factors Influencing Delivery of Therapeutic Services Clinicians shared varied and sometimes insufficient orientation to ECC and their team. Relevant to this concern is clinicians’ perceptions that there is a lack of both clarity in team member roles and unified messaging around the purpose of ECC. Notably, participants in multiple roles may feel this confusion about their place on the team. For example, some clinicians expressed that they felt “pressure” to be the team member to bring up difficult conversations regarding families, while one guardian ad litem shared that they felt they could not express certain opinions (i.e., support for the parent) based on their role. This suggests that despite a multidisciplinary team approach, there are varying comfort levels regarding bringing up an “elephant in the room.” Indeed, clinicians reported disparate experiences as to the extent to which their opinions are taken into consideration by the team. In addition to role-related issues, some participants—including a parent—expressed there is no uniform trauma lens among team members. Given the trauma history of parents, the lack of a traumainformed lens is made more challenging by the intensive nature of ECC, which lends to high expectations of both parents and clinicians. Participants in multiple roles, including parents, shared that the volume of meetings or tasks to complete can be difficult for families to manage, particularly given the intended 12-month timeline of ECC services (e.g., “…So, they might get motivated to do one, but it gets overwhelming to do all of it, especially when you have the pressure of the timeline…”):

Clinician

They need to be looked at from a trauma-informed lens… But those are the challenges when the child welfare doesn't understand the connection between trauma and attachment. And I see that a lot. I see that a lot.

Parent

I feel like – I think that, if people were working with people who have substance abuse issues, they should definitely have a class that they take to better understand how an addict feels, how they can feel attacked even though they might not be being attacked. Everybody’s perspective is different, and some people just need to be trained on how to recognize what somebody’s going through and assess the situation better.

Care Team Member

What would need to be improved? For my particular area, I would really like us to see some more – I hate to say trauma-informed, but anything that they understand that parents have been through trauma, and that most of our services are not geared towards parents who have been through significant complex trauma, and that is a problem.

For clinicians, these challenges emerge when the therapeutic goal timeline is longer than the “child welfare goal” timeline of parents’ cases (e.g., “They have a year to permanency, and if they don’t jump on board immediately that their child is removed, it could be problems with them in the long run”). As previously noted, some parents are not immediately motivated or ready to engage in services, or must first address concurrent issues (e.g., substance abuse) to meaningfully engage in COS-P or CPP. And although some participants indicate a rushed time frame for ECC, the data on CPP (Chapter 5) indicates that families receiving CPP have longer time to case closure than those not receiving CPP. While increased length of cases may be attributed to CPP, it is possible that other factors account for the longer timeline, from the need to first address 43

a parent’s substance use issues to scheduling conflicts. Regardless of the cause, the average length of time from removal to case closure for families both receiving and not receiving CPP is longer than the 12-month “accelerated” timeline noted by some participants. This suggests that case closure timelines are not rigid, and clinicians may have some recourse to advocate for additional therapeutic time if necessary. Still, extensions would need to be considered in the context of staying committed to timely permanency for children. Indeed, this balance of meeting both parents’ and children’s needs were noted by participants (e.g., “…Obviously, they’re related but it’s difficult to do both and do both well.”) While some parents may need more time to work their case plans, clinicians and care team members shared that not all clients referred to ECC are a match for services. Clinicians shared that those parents who do not accept their responsibility are not successful in therapeutic services, which influences the ultimate case outcome. One care team member shared similar sentiments, noting that when ECC-involved parents do not accept this responsibility, they experience frustration and “there’s no way that the children are not taking up on that frustration. So, I think that that is detrimental to their mental health.” Further complicating service delivery is a lack therapeutic providers for ECC-involved families. A care team member spoke of frustration with the lack of funding for the infrastructure of ECC-related services. Clinicians similarly shared frustration with the lack of appropriate compensation, as some are not paid for their time spent on nontherapeutic activities (e.g., court hearings, family team meetings) or reimbursed for providing families with necessary resources: Clinician

Care Team Member

So when there's something that I identify that they really, really need, if I can't find a resource for it then I provide it. And that is expensive, which means some of my cases cost me money. If we don’t have enough child psychotherapy providers to handle the case loads, if we don’t have intensive drug treatments for these parents to go to, it’s all for none. It can’t just be ECC money or justice or ECC money or structural issues. It has to be providers. It has to be who’s on the ground. I think that’s where we need that. I mean, we do. Otherwise, we can have this great model and weakness. It just won’t work. It won’t work unless everybody is given a specific fund. …Therapy services might have lost some of their therapists due to it, and making it harder for them to take on new clients. So, that then speaks to pushing the parents behind on their case plan, because they haven’t been able to start their mental health assessment, their substance abuse assessment, or any kind of assessment for therapy that is required in their case plan. – On the impact of COVID-19 on services

Relational Factors Influencing Therapeutic Services Participants across roles discussed relational factors that influenced therapeutic service delivery. Both clinicians and parents emphasized the importance of support from their judges. Three of six parents interviewed identified their judge as part of their therapeutic network and clinicians nearly uniformly spoke positively about the ECC judge in their circuit:

Clinician

Parent

The way she responded to the parents, to the team, to the lawyers, everybody. In my experience the judges are like very robotic, very clinical. Not clinical, very professional. But this judge was like I feel like sometimes does she have a clinical background? Because the way she was responding to the people, it was beautiful. …So, they were really – everybody kind of stepped up and were really supportive and there for me. Definitely including my judge, he was very supportive and very positive.

FLORIDA INSTITUTE FOR CHILD WELFARE

Parent

The judge assigned to my case was very caring. He was strict but, at the same time, he kept reassuring – he kept letting me know that he had a lot of confidence in me. He encouraged me and he knows that I can do it, but he was also stern and let me know how serious everything was too.

Participants described relationships with other team member roles as having more challenges. While relationships with child welfare workers and community coordinators were more varied (i.e., positive and negative experiences), interactions with attorneys appear to be the most strained. Clinicians shared stories of attorneys not understanding the ECC model or otherwise not respecting their professional perspective and issues with parent attorneys arose for both parents and care team members. Only one parent or caregiver identified their attorney as part of their therapeutic network. Previous research corroborates that differing policies and procedures on multidisciplinary child welfare teams can pose collaboration challenges.69

Clinician

That one attorney is just – he does not get the ECC model at all and he just attacks us all the time. And it's like – I'm exhausted with him.

Parent

I had to speak up. My lawyer didn’t speak up until I did.

Care Team Member

The biggest problem I have with ECC is that sometimes, parent lawyers can kind of shoehorn the parent into ECC, and ECC is supposed to be a voluntary thing, but these attorneys understand that the parents get a lot more services a lot more quickly, they get in front of the judge a lot more often, and so they advise these parents: “Oh no, you want ECC,” and parents are not in a place where they are open to – or interested in listening to the fact that there was a problem to begin with, and that the answer is going to require work on their part.

Another significant relationship to emerge was that of the parent and caregiver (i.e., the foster parent or kinship caregiver). Some care team members expressed that caregiver participation in the ECC process (e.g., attending family team meetings, knowing the families) could be improved. Notably, lack of participation in meetings may be due to scheduling conflicts as opposed to desire to contribute, as one foster parent shared that the virtual meetings during the COVID-19 pandemic allowed them to attend meetings they otherwise could not have. Several clinicians and care team members spoke to the importance of facilitating a positive relationship between parents and caregivers for the good of the case. The parent rarely exists in a vacuum; thus, ECC should consider the importance of supporting co-parenting and the other primary caregivers in the child’s life. Indeed, co-parenting was identified as an important aspect of ECC in the Institute’s previous evaluation.1 Still, some participants noted that caregiver resistance to this relationship is understandable, particularly with relative caregivers who may already have a strained relationship with the parent. Related, caregivers—regardless of relative status—may need tailored or additional supports throughout the life of the case. One clinician acknowledged this need for tailored psychoeducation during the foundational phase of CPP: Pre adoptive parents is a great deal of psychoeducation on

Clinician

trauma, attachment, and the specific child’s needs as well as work building the attachment and coaching. Relative caregivers who have parenting experience is about teaching them the differences in raising this child than their own bio children. Foster parents I work with a great deal on forming a coalition with the bio family regardless of how the case is going. Education, education, education!...

COVID-19 Factors Influencing Therapeutic Services The COVID-19 pandemic has impacted social and judicial service delivery,70,71 and Florida’s early childhood courts were no exception. While evaluation participants reported both positive and negative ECC experiences as it relates to changes in service delivery, parents appear more negatively impacted than providers. For example, clinicians shared how disruptions changed their way of work and how they had to adapt to virtual training and therapeutic sessions. Still, several participants discussed how their service reach had increased, including one who noted that they were now able to serve clients in a previously underserved area. Conversely, parents experienced more challenges. While virtual services were sometimes described as more convenient, some parents felt that they did not receive the full experience of ECC from behind a screen. Disruptions to in-person visitation presented new emotional challenges for parents who lost interaction time with their children. Participants felt that the virtual nature of services sometimes, but not always, increased parental participation. Specifically, some foster parents perceived that parents used COVID-19 as “an excuse” for not following their case plan (e.g., completing drug screens) or engaging with their children. Some participants in the present study reported how the challenges parents were already experiencing were exacerbated by COVID-19. This, coupled with high parental expectations in ECC, could contribute to parents being unable to fully engage in their case plan. For example, one parent shared:

Parent

They kind of always, with COVID, there really wasn’t like shelters or anything that were really open, so it’s very hard for them to offer like kind of the services they usually would offer because of COVID. They really made some of the things they usually would recommend for housing and stuff were hard, hard to help me. So, I kind of really had to rely on friends to live.

LIMITATIONS This evaluation of ECC-related therapeutic provision has several significant limitations. Although the qualitative data provides rich descriptions of experiences in ECC, it is not generalizable. While clinician representation was adequate for saturation and included participants from various regions of Florida, there were fewer parent, caregiver, and other professional team member participants, and their involvement was limited to one circuit. While this offers a unique case study perspective of that circuit, it may not represent other ECCs in the state. Low response rates were common across all primary data collection methods and introduces nonresponse bias. The research team surmises this could be due to a multitude of factors (e.g., simultaneous national evaluation of ECCs, lack of compensation for the clinician survey, COVID-19 impacts on willingness to participate). Future evaluation efforts should account for any malleable barriers to improve response rates (e.g., use of incentives). Related, there was a high frequency of missing data in the secondary dataset provided by OSCA, which indicates data related to therapeutic services are not being uniformly collected or entered into a centralized system. To provide an accurate assessment of outcomes, data collection efforts will need to improve. Finally, the evaluators were limited in their ability to assess fidelity to therapeutic programs for parents given the flexible nature of the therapeutic modalities, especially CPP. The present evaluation findings broadly indicate clinician fidelity to the general processes of COS-P and CPP, though future evaluators should consider more nuanced examinations of fidelity that still maintains client confidentiality. RECOMMENDATIONS Despite these limitations, the evaluation team developed several recommendations for OSCA, both specific to parenting-centered FLORIDA INSTITUTE FOR CHILD WELFARE

therapeutic provision in ECC, as well as for Florida’s ECCs overall: ● A llow the use of multiple therapeutic modalities within ECC. When considering which therapeutic modalities should be included in ECC services, clinicians have expressed an interest in “both/and” as opposed to “either/or.” Most clinicians are using CPP as the primary therapeutic service for ECCinvolved families and those who use COS-P tend to use it as a supplement to CPP—either before or throughout CPP services. For those implementing CPP, it was described as a framework for services wherein clinicians could bring in additional therapeutic modalities and techniques at their discretion. The ability to remain flexible, while maintaining a focus on the parent-child relationship, was considered a main tenet of therapeutic services with ECC-involved families. ● S tandardize system-wide data collection. A primary barrier to evaluating therapeutic fidelity is the lack of uniform data collection or entry from clinicians. OCSA should consider implementing data reporting requirements for all ECC clinicians that includes objective treatment details (e.g., therapist’s name, modalities used, number of sessions). Incorporation of more subjective treatment details (e.g., client level of participation) would be useful for evaluation analyses, though should be carefully considered with input from clinicians and families. At minimum, objective data should be entered for COS-P and CPP as the primary reported modalities to improve future analyses related to family outcomes. ● C larify processes regarding client eligibility for services. Clinicians shared that not all ECC-involved families are prepared to engage in intensive parenting therapeutic services. OSCA should explore the potential of including a standardized readiness for change assessment at the outset of services, the results of which could be shared with all team members. Expert consultant Meredith Piazza (personal communication, September 3, 2021), corroborated this recommendation, noting that standardizing assessments for issues that are consistent across ECCs (e.g., substance use) can better ensure fidelity to therapeutic models, but also increase consistency in operations across ECC teams. ● D evelop policies and procedures regarding the incorporation of therapists’ professional expertise. Clinicians expressed concerns regarding the appropriateness of clients for therapeutic services. They also reported having some reservations about being the team member who was responsible for bringing up “the elephant in the room.” Having policies and procedures in place for therapists to clearly articulate their recommended treatment plan or related adjustments to the ECC team could promote case transparency and establish more clarity around the role of the therapeutic provider. Adjustments to the treatment plan would not necessarily require a family’s removal from ECC, though could provide justification for an extended permanency timeline. Clarifying processes might also be helpful for situations in which families are unable to receive services due to the lack of available providers. ● R e-evaluate compensation practices for clinicians. While not a uniform concern, several clinicians expressed a lack of financial support for clinicians (e.g., non-billable hours for non-therapeutic time spent on ECC activities). While not a focus of this evaluation, this emergent issue merits further exploration. Clinicians and expert consultants noted a lack of therapeutic providers, which can contribute to delayed service connection for families. Incentivizing clinicians through compensation for non-therapeutic ECC activities may be one way to recruit and retain quality providers. Notably, concerns were not uniform, though setting (i.e., private practice versus agency-based work) does appear to impact pay practices. 45

● I mprove team member orientation to ECC. Clinicians expressed a desire for improved orientation to the ECC team and foundational principles, with an emphasis on role clarity and the use of a trauma-informed approach. While the small sample size of the present survey hindered more nuanced examination of specific improvement suggestions, the OSCA should explore how the ECC team onboarding process might be improved. Related, parents in the present evaluation perceive therapeutic support from a range of team members, not just therapists. This suggests a need for team messaging that ECC—at large—is a therapeutic intervention for families and that therapeutic benefits are not relegated to parents’ work with clinicians alone. Expert consultant Meredith Piazza (personal communication, September 3, 2021) also suggested that, given the high prevalence of substance use and need for relapse support, special attention be paid to how substance use disorder providers receive onboarding to ECC. Related, the evaluation team suggests onboarding for all team members include content on how substance-related issues impact families and their progress with ECC. ● E xplore expansion of reflective supervision to nonclinician team members. Given that professional team members contribute to families’ therapeutic networks, the OSCA should explore the possibility of arranging reflective supervision for them. Clinicians highlighted the value of their reflective supervision experiences in not only helping them process their experiences, feelings, and reactions, but also in understanding how those experiences, feelings, and reactions could impact their service provision. Providers in other roles also indicated they are receiving or need to receive therapeutic support for the emotional toll that this work can cause. Reflective supervision may be one way to support the team to provide optimal services for families. Expert consultant Diane Koch (personal communication, September 5, 2021) corroborated this recommendation: The [providers’] expressed needs for more orientation, ongoing training, mental health supports, and team cohesion can be readily addressed by [reflective supervision (RS)] being built in as a priority. RS is the key to the success of ECC. It needs to be recognized as the essential part of facilitating the cultural change or model change inherent in ECC. ● I mplement increased supports for caregivers. A primary component of ECC is supporting parenting and parenting capacities, which can extend to providing support for caregivers (i.e., foster parents and kinship caregivers) throughout the life of a case. While caregivers in the present sample spoke highly of their relationships with professional team members, they reported more challenges or frustrations when engaging with parents. Though participants expressed empathy for caregiver resistance to parental engagement, clinicians shared that a co-parenting relationship between all the child’s parents and caregivers is beneficial and can serve as a great support to the parents’ therapeutic networks. Caregivers in the present sample spoke of supports that are not ECC-specific (e.g., social media support groups). In addition to these informal support networks, ECCs should further explore with caregivers what additional formal supports would be helpful toward fulfilling their role as part of the ECC team. Recent Florida legislation (Ch. 2021-170) requires each community-based care lead agency to establish a kinship navigator program, which is intended to assist kinship caregivers through resource brokerage. Similar instrumental supports should be explored for non-kinship caregivers, and emotional supports should be explored for all caregivers. All available supports should be clearly communicated to caregivers. FLORIDA INSTITUTE FOR CHILD WELFARE

Next Steps In addition to the above recommendations, the evaluation team suggests future evaluations incorporate assessment of… ● … variation in therapeutic services related to both processes and outcomes. Parenting therapies are not “cookie cutter” and need to be client responsive. Being overly prescriptive might hinder practice wisdom, which is one of several elements of evidence-based practice.68 Still, there is a need for more in-depth examination of if and how fidelity is achieved despite the variation, as well as which variations in practice are most beneficial to families. The present survey’s small sample size precludes any definitive conclusions regarding fidelity. Though there appears to be general procedural fidelity, questions remain. For example, clinicians reported providing CPP over the course of seven to 30 weeks, which is misaligned with the current CPP evidence base (Kim Renk, personal communication, August 6, 2021). Although client-responsive practices can make evaluation challenging, evaluators could examine de-identified fidelity forms. They could also measure variation within service provision to isolate particularly impactful practices, such as the sequencing of COS-P and CPP or particular techniques used within the CPP framework (with attention to the possibility of the interventionist effect). ● …challenges in CPP implementation, specifically as it pertains to working with multiple family members in a single session. While not a widespread concern, it was noted that CPP—in its intended form—is meant to be implemented with a single parent-child dyad. This may present challenges for clinicians who are mandated to provide CPP within ECC, which typically involves complex family systems (e.g., multiple caregivers, older siblings). ● … how the COVID-19 pandemic impacts therapeutic services. Participants shared both positive and negative experiences with ECC during COVID-19. While providers tended to have more mixed experiences, parents and caregivers reported more negative impacts. This is significant as there is ongoing dialogue around how COVID-19 is transforming social service delivery, including in Florida. For example, a shift to virtual supervised visitation has resulted in both benefits (e.g., more frequent and more flexible visitation, parents being able to visualize their child’s foster home environment), and challenges (e.g., parental frustration with lack of physical interaction with their children, technology problems, monitors’ lack of control over other participants on the call).72 ● … the potential significance of clinicians having a professional background in child welfare. Particularly given some of the collaboration challenges noted with case management and other child welfare representatives, it is possible that a history in child welfare work might be beneficial for rapport and better understanding of roles (e.g., by empathizing with the stresses of case management).

References 1

1 Magruder, L., Tutwiler, M., & Pryce, J. (2019). 2018-2019 Early Childhood Court evaluation: Final report to the Office of Court Improvement. Tallahassee: Florida State University, Florida Institute for Child Welfare. Available from: https://ficw.fsu.edu/ecc 2 Office of the State Courts Administrator [OSCA]. (n.d.). Early childhood courts. Florida Courts. Retrieved May 4, 2021 from https://www.flcourts.org/Resources-Services/Court-Improvement/Problem-Solving-Courts/Early-Childhood-Courts 3 ZERO TO THREE [ZTT]. (2020). The Safe Babies Court TeamTM approach. Retrieved May 4, 2021 from https://www.zerotothree.org/resources/ services/the-safe-babies-court-team-approach 4 Supreme Court of the State of Florida. (2019). Florida early childhood court best practice standards. https://www.flcourts.org/content/download/682604/file/Early_Childhood_Court_Best_Practice_Standards.pdf 5 ZERO TO THREE [ZTT]. (2017). Final evaluation report of the quality improvement center for research-based infant-toddler court teams. Retrieved May 4, 2021 https://www.zerotothree.org/resources/2181-safe-babies-court-team-trauma-informed-care-that-s-changing-lives 6 ZERO TO THREE [ZTT]. (2021). HRSA infant-toddler court program sites. Retrieved May 4, 2021 https://www.zerotothree.org/resources/3115-hrsainfant-toddler-court-program-sites 7 Office of the State Courts Administrator [OSCA]. (2020). Florida’s early childhood court: Data analysis report. Retrieved May 31, 2021 from: https://www.flcourts.org/content/download/732711/file/ECC%20December%202020%20Data%20Analysis_Final.pdf 8 Florida Department of Children and Families [DCF]. (2021). Children & young adults entering out-of-home care statewide. Retrieved May 4, 2021 from https://www.myflfamilies.com/programs/childwelfare/dashboard/c-entering-ooh.shtml 9 Mackie, J. F., Foti, T. R., Agu, N., Langenderfer-Magruder, L., & Marshall, J. (2020). Early childhood court in Florida: Qualitative results of a statewide evaluation. Child Abuse & Neglect, 104. https://doi-org.proxy.lib.fsu.edu/10.1016/j.chiabu.2020.104476 10 McCombs-Thornton, K. L., & Foster, E. M. (2012). The effect of the ZERO TO THREE Court Teams initiative on types of exits from the foster care system—A competing risks analysis. Children and Youth Services Review, 34(1), 169–178. https://doi.org/10.1016/j.childyouth.2011.09.013 11 Shea, K., & Graham, M. (2018). Early Childhood Courts: The opportunity to respond to children and families affected by the opioid crisis. ZERO TO THREE, 38(5), 39–47. 12 Casanueva, C., Fraser, J. G., Gilbert, A., Maze, C., Katz, L., Ullery, M. A., Stacks, A. M., & Lederman, C. (2013). Evaluation of the Miami Child Well- Being Court model: Safety, permanency, and well-being findings. Child Welfare, 92(3), 73–96. 13 Stacks, A. M., Wong, K., Barron, C., & Ryznar, T. (2020). Permanency and well-being outcomes for maltreated infants: Pilot results from an infant- toddler court team. Child Abuse & Neglect, 101. 14 Bowlby, J. (1980). Attachment and Loss. Vol. 1: Attachment. Basic Books. 15 Dehghani, A., Malekpour, M., Abedi, A., Amiri S. (2014) The efficacy of circle of security on attachment and wellbeing in preschool children. Management Science Letters, 4(3), 607-612. doi: 10.5267/j.msl.2013.12.031 16 The Circle of Security International (2019b). Circle of Security Parenting with careers: Tools for childcare providers, foster parents, and classrooms. Retrieved May 28, 2021 from https://www.circleofsecurityinternational.com/circle-of-security-model/cosp-with-carers/ 17 Cassidy, J., Brett, B. B., Gross, J. T., Stern, J. A., Martin, D., Mohr, J., et al. (2017). Circle of Security Parenting (COS-P): A randomized controlled trial in Head Start. Developmental Psychopathology, 29(2), 651–673. doi: 10.1017/S0954579417000244 18 The Circle of Security International (2019a). Comparing COSP and COS-Intensive. Retrieved May 31, 2021 from https://www.circleofsecurityinternational.com/circle-of-security-model/cosp-vs-cos-intensive/ 19 Cassidy, J., Woodhouse, S., Sherman, L., Stupica, B., & Lejuez, C. (2011). Enhancing infant attachment security: An examination of treatment efficacy and differential susceptibility. Development and Psychopathology, 23, 131-148. doi:10.1017/S0954579410000696 20 Huber, A., McMahon, C. A., & Sweller, N. (2015). Improved child behavioral and emotional functioning after COS 20-week intervention. Attachment and Human Development, 17(6), 547–569. https://doi.org/10.1080/14616734.2015.1086395 21 Huber, A., McMahon, C., & Sweller, N. (2016). Improved parental emotional functioning after Circle of Security 20-week parent–child relationship intervention. Journal of Child and Family Studies, 25(8), 2526–2540. doi: 10.1007/s10826-016-0426-5 22 Ramsauer, B., Mühlhan, C., Lotzin, A., Achtergarde, S., Mueller, J., …& Romer, G. (2020). Randomized controlled trial of the Circle of Security- Intensive intervention for mothers with postpartum depression: Maternal unresolved attachment moderates changes in sensitivity, Attachment and Human Development 22(6), 705-726. https://doi.org/10.1080/14616734.2019.1689406 23 Coleman, M. J. (2014). An attachment-based parenting training program for opiate dependent carers. Journal of Addiction Research and Therapy, 5(4), 1-7. doi: 10.4172/2155-6105.1000201 24 Yaholkoski, A., Hurl, K., & Theule, J. (2016) Efficacy of the Circle of Security Intervention: A Meta-analysis. Journal of Infant, Child and Adolescent Psychotherapy, 15(2), 95-103. https://doi.org/10.1080/15289168.2016.1163161

25 Child Parent Psychotherapy [CPP]. (2018a). History. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/about/history/

26 Child Parent Psychotherapy [CPP]. (2018b). About CPP. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/about/

FLORIDA INSTITUTE FOR CHILD WELFARE

27 The National Child Traumatic Stress Network [NCTSN]. (2012, April). CPP: General information. Retrieved May 31, 2021 from https://www.nctsn.org/sites/default/files/interventions/cpp_fact_sheet.pdf 28 Child Parent Psychotherapy [CPP]. (2018c). CPP fidelity. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/providers/tools-and resources/fidelity/ 29 The California Evidence-Based Clearinghouse for Child Welfare. (2015). Child Parent Psychotherapy. Retrieved May 4, 2021 from https://www. cebc4cw.org/program/child-parent-psychotherapy/ 30 Child Parent Psychotherapy [CPP]. (2018d). CPP training. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/providers/training/ 31 Child Parent Psychotherapy [CPP]. (2018e). Child-Parent Psychotherapy learning collaborative. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/providers/training/lc/ 32 Child Parent Psychotherapy [CPP]. (2019). SF CPP 2019 cohort 7. Retrieved May 4, 2021 from https://www.ucsfcme.com/2020/MMH20030/2019CPPLCoverview.pdf 33 Illinois Child-Parent Psychotherapy Learning Collaborative. (n.d.). Frequently asked questions. Retrieved May 4, 2021 from https://illinoiscpp.org/faq 34 The Alaska Training Cooperative. (n.d.). Agency participant child-parent psychotherapy training learning collaborative application. Retrieved May 4, 2021 from https://www.aktclms.org/Training/Class/102922 35 The Self-Care Practice. (2021). NY/NJ Winter 2021 virtual CPP learning collaborative. Retrieved May 4, 2021 from https://static1.squarespace.com/ static/5be20b6cf93fd476a14eb4a7/t/5fe37a16a1e5f73bea11d60c/1608743447055/NY-NJ+Winter+2021+Virtual+CPP+Learning+Collaborative.pdf 36 Skale, G., Perez, H., & Williams, M. E. (2020). Factors influencing implementation of evidence-based mental health interventions for infants and young children. The Journal of Behavioral Health Services & Research, 47(4), 493–508. https://doi.org/10.1007/s11414-020-09694-5 37 Child Parent Psychotherapy [CPP]. (2018f). CPP Agency Mentorship Program (CAMP). Retrieved May 4, 2021 from https://childparentpsychotherapy. com/providers/training/camp/ 38 Child Parent Psychotherapy [CPP]. (n.d.). CPP fidelity compass: Overview. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/wp content/uploads/2020/03/fidelitycompass_overview_final.pdf 39 Child Parent Psychotherapy [CPP]. (2015a). CPP fidelity: Core intervention phase. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/ wp-content/uploads/2018/05/2.-Core-Intervention-Fidelity-2015.pdf 40 Child Parent Psychotherapy [CPP]. (2015b). CPP fidelity: Foundational phase. Retrieved May 4, 2021 from https://childparentpsychotherapy.com/ wp-content/uploads/2020/07/1.-Foundational-Phase-Fidelity-2015.pdf 41 Child Parent Psychotherapy [CPP]. (2015c). CPP fidelity: Recapitulation and termination phase. Retrieved May 4, 2021 from https:// childparentpsychotherapy.com/wp-content/uploads/2018/05/3.-Termination-Fidelity-2015.pdf 42 David, P., & Schiff, M. (2018). Initial clinician reports of the bottom-up dissemination of an evidence-based intervention for early childhood trauma. Child & Youth Care Forum, 47(1), 115–132. https://doi.org/10.1007/s10566-017-9422-1 43 Smith, A. B., Dopp, A. R., Perrine, C. M., Parisi, K. E., Vanderzee, K. L., John, S., Edge, N. A., & Kramer, T. L. (2020). Training providers at Children’s Advocacy Centers to treat early childhood trauma: Perspectives on barriers, facilitators, and sustainability. Children and Youth Services Review, 118, 105379. https://doi.org/10.1016/j.childyouth.2020.105379 44 Cicchetti, D., Rogosch, F. A., & Toth, S. L. (2006). Fostering secure attachment in infants in maltreating families through preventive interventions. Development and Psychopathology, 18(3), 623-649. https://doi.org/10.1017/S0954579406060329 45 Stronach, E. P., Toth, S. L., Rogosch, F., & Cicchetti, D. (2013). Preventive interventions and sustained attachment security in maltreated children. Development and Psychopathology, 25, 919–930. https://doi.org/10.1017/S0954579413000278 46 Toth, S. L., Maughan, A., Manly, J. T., Spagnola, M., & Cicchetti, D. (2002). The relative efficacy of two interventions in altering maltreated preschool children’s representational models: Implications for attachment theory. Development and Psychopathology, 14(4), 877–908. https://doi.org/10.1017/ S095457940200411X 47 Lieberman, A. F., Van Horn, P., & Ghosh Ippen, C. (2005). Toward evidence-based treatment: Child-parent psychotherapy with preschoolers exposed to marital violence. Journal of the American Academy of Child & Adolescent Psychiatry, 44(12), 1241–1248. https://doi.org/10.1097/01. chi.0000181047.59702.58 48 Lieberman, A. F., Ghosh Ippen, C., & Van Horn, P. (2006). Child-parent psychotherapy: 6-month follow-up of a randomized controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 45(8), 913–918. https://doi.org/10.1097/01.chi.0000222784.03735.92 49 Ghosh Ippen, C., Harris, W. W., Van Horn, P., & Lieberman, A. F. (2011). Traumatic and stressful events in early childhood: Can treatment help those at highest risk? Child Abuse & Neglect, 35(7), 504–513. https://doi.org/10.1016/j.chiabu.2011.03.009 50 Ghosh Ippen, C., Harris, W. W., Van Horn, P., & Lieberman, A. F. (2011). Traumatic and stressful events in early childhood: Can treatment help those at highest risk? Child Abuse & Neglect, 35(7), 504–513. https://doi.org/10.1016/j.chiabu.2011.03.009 51 Hagan, M. J., Browne, D. T., Sulik, M., Ghosh Ippen, C., Bush, N., & Lieberman, A. F. (2017). Parent and child trauma symptoms during child-parent psychotherapy: A prospective cohort study of dyadic change. Journal of Traumatic Stress, 30(6), 690–697. https://doi.org/10.1002/jts.22240 52 Bernstein, R. E., Timmons, A. C., & Lieberman, A. F. (2019). Interpersonal violence, maternal perception of infant emotion, and child-parent psychotherapy. Journal of Family Violence, 34(4), 309–320. https://doi.org/10.1007/s10896-019-00041-7

FLORIDA INSTITUTE FOR CHILD WELFARE

53 Kenny, M., Hassett, A., & Pae, L. (2017). Exploring how parents make sense of change in parent-child psychotherapy. Journal of Infant, Child, and Adolescent Psychotherapy, 16(1), 73–92. https://doi.org/10.1080/15289168.2016.1248200 54 Regional Judicial Opioid Initiative. (2020). Baby courts: A promising child protection court model to serve families with infants and toddlers. National Center for State Courts. https://www.ncsc.org/__data/assets/pdf_file/0018/16506/baby-courts_rjoi_final.pdf 55 Chinitz, S., Guzman, H., Amstutz, E., Kohchi, J., & Alkon, M. (2017). Improving outcomes for babies and toddlers in child welfare: A model for infant mental health intervention and collaboration. Child Abuse & Neglect, 70, 190–198. https://doi.org/10.1016/j.chiabu.2017.05.015 56 Hazen, K. P., Patnode, C. M., Fessinger, M. B., & Cole-Mossman, J. (2018, April). Child-Parent Psychotherapy in child welfare court: An evaluation [Poster session]. UNL Child Youth and Family Studies Summit on Research in Early Childhood, Lincoln, NE. 57 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101. doi: 10.1191/1478088706qp063oa 58 Clarke, V., & Braun, V. (2013). Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning. The Psychologist, 26(2), 120-123.

59 Guest, G., MacQueen, K. M., & Namey, E. E. (2012). Applied thematic analysis. Sage

60 QSR International Pty Ltd. (2020) NVivo (released in March 2020), https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home

61 Creswell, J.W. (2007). Qualitative inquiry & research design: Choosing among five approaches (2nd ed.). Thousand Oaks, CA: Sage Publications Inc.

62 Powell, B., Cooper, G., Hoffman, K., & Marvin, B. (2014). The Circle of Security intervention: Enhancing attachment in early parent– Child relationships. Guilford Press 63 Radey, M., Langenderfer-Magruder, L., & Wilke, D. (2020). How does client violence affect worker health? Understanding client violence among child protection services workers. Journal of the Society for Social Work & Research. https://doi.org.10.1086/712821 64 Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59-82. doi: 10.1177/1525822X05279903 65 Qualtrics. (2021). How to increase online survey response rates. Retrieved September 7, 2021 from https://www.qualtrics.com/experience management/research/tools-increase-response-rate/ 66 Rothbaum, J., & Bee, A. (2021). Coronavirus infects surveys, too: Survey non-response bias and the coronavirus pandemic. U.S. Census Bureau Working Paper. Retrieved from https://www.census.gov/content/dam/Census/library/working-papers/2020/demo/sehsd-wp2020-10.pdf 67 Early Childhood Court Programs, Fla. Stat. § 39.01304 (2020). 68 Satterfield, J. M., Spring, B., Brownson, R. C., Mullen, E. J., Newhouse, R. P., Walker, B. B., & Whitlock, E. P. (2009). Toward a transdisciplinary model of evidence‐based practice. The Milbank Quarterly, 87(2), 368-390. doi: https://doi.org/10.1111/j.1468-0009.2009.00561.x 69 Young, T. L., & Nelson-Gardell, D. (2018). A grounded theory study of collaboration in multidisciplinary teams. Journal of Public Child Welfare, 12(5), 576-595. https://doi.org/10.1080/15548732.2018.1436112 70 Office of the State Courts Administrator [OSCA]. (2021). COVID-19 update archive. Retrieved September 14, 2021 from https://www.flcourts.org/ Resources-Services/Emergency-Preparedness/COVID-19-Information-and-Updates/COVID-19-Update-Archive. 71 Seay, K. D., & Stafford McRell, A. (2021). Child welfare services response to COVID-19: Addressing face-to-face contacts. Journal of Child and Family Studies, 30, 2055-2067. https://doi.org/10.1007/s10826-021-02000-7 72 Oehme, K., O’Rourke, K. S., & Bradley, L. (2021). Online virtual supervised visitation during the COVID-19 pandemic: One state’s experience. Family Court Review, 59(1), 131-143. doi: 10.1111/fcre.12555

FLORIDA INSTITUTE FOR CHILD WELFARE

Appendices Appendix A—Initiative One Interview Schedule

involved families you have worked with? ● W hat are the major reasons ECC-involved families are referred to your services? ● W hat are some common challenges you face when serving ECC-involved families?

INTERVIEWER SCRIPT

• Related to families?

Good [MORNING/AFTERNOON/EVENING]. My name is [INTERVIEWER NAME] and I work at the Florida Institute for Child Welfare. I am part of a research team that is conducting an evaluation of the therapeutic modalities used with Early Childhood Court—or ECC—involved families. As part of that, we are interested in hearing about your experiences as a therapeutic provider. Our conversation should last about an hour. Remember, this is completely voluntary, and you can stop participating at any time. Additionally, you can choose to skip any question, for any reason, which you are not required to share with me. After completing your interview, our research team will e-mail you a $25 Amazon gift card to show our appreciation for your time.

• R elated to ECC broadly? (e.g., support, funding/ resources, team dynamics)

I know you have already read through the project information sheet online, but before we begin, I wanted to make sure that you did not have any questions. What questions can I answer for you about this study? [Interviewer should answer all questions before proceeding]. And just to be sure, is it OK if I record this interview? [If on Zoom, you can remind them that only the audio file will be sent for transcription, no video files].

● A re there any common positive experiences you have when serving ECC-involved families? • Related to families? • R elated to ECC broadly? (e.g., support, funding/ resources, team dynamics) 4) D escribe the therapeutic modality or modalities you use with ECC-involved clients. ● W hich of these approaches are evidence-based? [Do not ask if Circle of Security or Child-Parent Psychotherapy as they are evidence-based.] • [ If Circle of Security, ask if they implement Circle of Security – Parenting and/or Circle of Security – Intensive Model.) ● What training did you receive for these approaches? • A re you rostered and/or did you finish the whole training (if relevant)?

[If yes]: Great! I am going to start the recording now. [START RECORDING TO THE CLOUD]. This is [interviewer first name] on the phone with [participant first name], ID number [ECC_ID], for an interview on Early Childhood Court. Today is [DATE] at [TIME].

● H ow do you determine which modality/modalities are bestsuited for your ECC-clients?

[If no]: That’s OK – would you still like to participate? [If yes]: Great! I can take detailed notes instead. It may take some additional time to make sure I’m capturing what you are saying correctly. So if there are some extended pauses on my end, that’s why. Today is [DATE] at [TIME].

● H ow comfortable are you in implementing these approaches? [Interviewer can ask for details related to each major modality noted by the participant.]

INTERVIEW GUIDE *Note to Interviewer: Questions 1-7 are intended to be asked of all participants. Sub-questions are meant to guide interviewers if additional probes are needed. *Note to Interviewer: If the provider uses a specific job title (therapist, counselor, etc.), you can adjust the script to match. 1) F irst, can you start by telling me a bit about your professional background? ● How did you decide to become a therapeutic provider? ● What type of education or training did you receive related to your work? [Assess for credentials, majors/degrees] ● (If not mentioned) Have you done specific training in infant mental health? Trauma-informed care? 2) H ow did you come to be involved with providing services to families being served by Early Childhood Court? ● How long ago was that?

• A re there any qualifying or disqualifying factors in determining which clients you serve?

● H ow many families have you treated with each major modality? ● I s there anything that would help you feel better prepared to implement these approaches? ● D o you participate in reflective supervision/consultation? Can you tell me a bit about that? • I f not, who do you go to when you have treatment questions? ● A re there any therapeutic approaches you will not implement with ECC-involved families? If so, please tell me more. 5) [For each major modality noted] Walk me through the sessions for a typical case using [name of modality]. ● W hat are the strengths of this therapeutic approach? Weaknesses? ● D oes this therapeutic approach have a manual and/or fidelity measures that you use? Tell me about those. ● W hat kind of pre- and post- assessments, if any, do you complete with the parent or child? (e.g., parent interview, child-parent interaction observation)

● How consistently have you been involved?

● T ell me about a time that this approach worked particularly well.

● H ow were you oriented to ECC? (e.g., your role, the team, processes)

● T ell me about a time where this approach did not work well. What did you do next?

3) Describe the ECC-involved families you have worked with.

● [Repeat for each major modality noted]

● What are some of the common characteristics of ECCFLORIDA INSTITUTE FOR CHILD WELFARE

6) C an you describe for me your relationship with the larger ECC team(s) you work with? ● W hat role do you play in family team meetings? Court hearings? ● I n what ways do you feel supported by your fellow team members? Unsupported? • I n your experience, what is the judicial leadership like on your ECC team? ● W hat, if anything, would help you in better-serving your ECC-involved clients? 7) B efore we end our conversation today, is there anything that we have not covered that you feel is important for me to know? Thank you so much for your time. If you need to get in touch with the research team, please refer to your copy of the project information sheet. Have a great day.

FLORIDA INSTITUTE FOR CHILD WELFARE

Appendix B—Initiative One Provider Survey

● Circuit 5 – Magistrate Coleman

SECTION 1: BACKGROUND

● Circuit 6 – Judge Crane

First, we’d like to ask you a few questions about your background. 1) What is your gender identity? ● Female

● Circuit 6 – Judge Moore ● Circuit 13 – Judge Essrig ● Circuit 12 – Judge Dees ● Circuit 12 – Judge Riva

● Male ● Transgender/Nonbinary ● Other, please specify: 2) What is your race? Please check all that apply. ● White ● Black or African American ● American Indian or Alaska Native ● Asian ● Native Hawaiian or Pacific Islander 3) Do you identify as Hispanic, Latinx, or Spanish? ● No ● Yes 4) How old are you? Please provide your answer in years. ●

● Circuit 9 – Judge Latimore

years

5) In what field is your undergraduate degree? ● 6) Do you hold a graduate degree? ● No [Skip to #9] ● Yes 7) In what field is your graduate degree? ● 8) Are you a licensed mental health provider in your field? ● No ● Yes ● Licensure in progress 9) O utside of your clinician role, have you ever worked in the field of child welfare? ● Yes ● No [skip to #11] 10) What was your role in child welfare? Please check all that apply. ● Case management ● Investigations ● Other, please specify: 11) In which circuit(s) do you provide services for ECC-involved families? Please select all that apply. ● Circuit 1 - Judge Frydrychowicz ● Circuit 1 – Judge Burns ● Circuit 1 – Judge Polson ● Circuit 1 – Judge Ketchel ● Circuit 14 – Magistrate Lord ● Circuit 2 – Magistrate Pedroso ● Circuit 2 – Judge Miller ● Circuit 2 – Judge Walker ● Circuit 4 – Magistrate Strawbridge ● Circuit 7 – Judge Gaustad ● Circuit 7 – Judge McNeilly FLORIDA INSTITUTE FOR CHILD WELFARE

● Circuit 20 – Judge McFee ● Circuit 17 – Judge Bristol ● Circuit 15 – Judge Kroll ● Other, please specify: 12) How long have you been a therapeutic provider with any population? Please provide your answer in years and months. ●

years and

months

13) How long have you been a therapeutic provider with ECC specifically? Please provide your answer in years and months. ●

years and

months

14) How do you provide therapeutic services to ECC-involved families? ● As a private practitioner ● Through an agency ● Other, please specify: 15) How were you oriented to the ECC model? ● Open-ended 16) How satisfied are you with the orientation you received to the ECC model? ● 5-point Likert scale (1=dissatisfied, 5=very satisfied) 17) How satisfied are you with the orientation you received to your ECC team? ● 5-point Likert scale (1=dissatisfied, 5=very satisfied) 18) What, if anything, could have helped better orient you to ECC in your role as a clinician? ● Open-ended 19) Based on our previous evaluation findings, when working with ECC-involved clients, most providers use Circle of Security-Parenting (COS-P), Child-Parent Psychotherapy (CPP), or both. Which modalities do you use when working with ECC-involved families? Please select all that apply. Note: If you incorporate multiple therapeutic techniques within COS-P or CPP delivery, we will ask about those later in the survey. ● C ircle of Security-Parenting (COS-P) [If selected, complete section 2] ● C hild-Parent Psychotherapy (CPP) [If selected, complete section 3] ● Other, please specify [If selected, complete section 4] 20) [Display only if multiple modalities selected in #19] You indicated you utilize more than one modality with ECC-involved families. When you do this, which of the following best describes the implementation: ● S equentially. I implement one to completion, followed by another. • Please provide an example. ● Concurrently. I implement multiple modalities at the same time. • Please provide an example. 52

SECTION 2: CIRCLE OF SECURITY—PARENTING Now, we’d like to ask you some specific questions about your experience with Circle of Security-Parenting (COS-P). As a reminder, please consider your work with ECC-involved families specifically. 1) Did you complete the 4-day training for COS-P? ● No (Skip to #3) ● Yes 2) Did you receive 6 contact hours per day during training? ● No ● Yes 3) D o or did you ever you participate in ongoing supervision for COS-P? ● No (Skip to #5) ● Yes, currently ● Yes, previously 4) P lease describe your experience with COS-P supervision. (If no experience with supervision, write “N/A.”) ● Open-ended 5) D o or did you ever participate in Fidelity Coaching with a certified COS-P Fidelity Coach? ● No (Skip to #7) ● Yes, currently ● Yes, previously 6) Please describe your experience with Fidelity Coaching. ● Open-ended 7) I n what format(s) do you typically facilitate COS-P? Note: If you began providing ECC services during the COVID-19 pandemic, how would you typically facilitate COS-P? Please select all the apply. ● Groups ● Individual Counseling ● In-home services ● Other, please specify: 8) T o whom do you typically provide COS-P? Please check all that apply. ● Parents ● Foster parents ● Other caregivers, please specify: ● Other, please specify: 9) How do you describe COS-P to participants? (If you do not do this, write “N/A.”) ● Open-ended 10) [Display only if multiple individuals selected in #8] You indicated you provide COS-P with multiple types of caregivers. Please describe any major differences in COS-P implementation between these groups. ● Open-ended 11) Please describe your overall objectives as a clinician when facilitating COS-P to ECC-involved families. ● Open-ended 12) Does your ECC require pre- and post-assessments for COS-P? ● Yes ● No ● I don't know FLORIDA INSTITUTE FOR CHILD WELFARE

13) Which assessment tools do you use as part of COS-P? ● Adverse Childhood Experiences (ACEs) ● Parenting Stress Index (PSI) ● Crowell ● Child Behavior Checklist (CBCL) ● Working Model of the Child Interview (WMCI) ● Other, please specify We’d like to ask you questions about how you implement COS-P chapters. 14) How many sessions do you typically need to cover any particular chapter of COS-P? ●

sessions

15) Do the COS-P chapters take approximately the same amount of time to complete? ● No ● Yes 16) [Display only if no to #13]: Which chapters typically take more time than others? ● Open-ended 17) [Display only if no to #13]: Which chapters typically take less time than others? ● Open-ended Now, we’d like to ask you a few questions about each chapter. 18) Chapter 1: Welcome to Circle of Security Parenting. ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 19) Chapter 2: Exploring Our Children’s Needs All the Way around the Circle ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 20) Chapter 3: “Being With” on the Circle ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 21) Chapter 4: “Being With” Infants on the Circle ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 22) Chapter 5: The Path to Security ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 53

23) Chapter 6: Exploring our Struggles ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 24) Chapter 7: Rupture and Repair in Relationships ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 25) Chapter 8: Summary and Celebration ● Briefly describe how you implement this chapter. ● I s there content that you typically skip or do not include? If so, which content? If not, write “no.” ● W hat challenges, if any, do you encounter when implementing this chapter? 26) How do you determine a client’s successful completion of COS-P? ● Open-ended 27) Following COS-P, if you have concerns or think there is a need for follow-up services, how is this communicated to the ECC team? ● Open-ended 28) How do you integrate a trauma-informed approach when facilitating COS-P? (If you do not do this, write “N/A”.) ● Open-ended 29) How do you integrate cultural responsiveness when facilitating COS-P? (If you do not do this, write “N/A”.) ● Open-ended 30) Please describe how the COVID-19 pandemic has impacted your facilitation of COS-P with ECC-involved families? (If there has been no impact, please write “N/A”.) ● Open-ended 31) Is there anything else you’d like to share about COS-P? ● Open-ended SECTION 3: CHILD-PARENT PSYCHOTHERAPY Now, we’d like to ask you some specific questions about your experience with Child-Parent Psychotherapy. As a reminder, please consider your work with ECC-involved families specifically. 1) Did you complete the 18-month training for CPP? ● No (Skip to #5) ● Yes ● Currently in training 2) What entity provided or is providing your CPP training? ● Open-ended 3) Briefly describe the format of your training. ● Open-ended 4) [ Display only if response to #1 is yes) Are you rostered to provide CPP services in Florida? ● No ● Yes FLORIDA INSTITUTE FOR CHILD WELFARE

5) W here do you typically deliver CPP? If you began providing ECC services during the COVID-19 pandemic, how would you typically deliver CPP? Please check all that apply. ● Adoptive Home ● Birth Family Home ● Foster/Kinship Care ● Outpatient Clinic ● Community-based Agency / Organization / Provider ● S chool Setting (Including: Day Care, Day Treatment Programs, etc.) ● Other, please specify: 6) How many weeks do you typically engage in CPP with a given family? ●

weeks

7) H ow many sessions do you typically hold with clients in that time period? ●

sessions

8) H ow long are your typical sessions? Please provide your response in minutes. ●

minutes

9) I n addition to the child, to whom do you typically provide CPP? Please check all that apply. ● Parents ● Foster parents ● Other caregivers, please specify: ● Child’s siblings ● Other, please specify: 10) [Display only if multiple selections for #9] You indicated you provide CPP with multiple types of clients. Please describe any major differences in implementation between these groups. ● Open-ended 11) Do you ever complete sessions with more than one parentchild dyad per family? For example, with two parents and one child or one parent and three children. ● Yes, but in separate settings ● Yes, within the same setting ● No 12) [Display only if 11b is selected]: In what instances would you incorporate more than one parent/caregiver and/or child in single sessions of CPP? ● Open-ended 13) Please describe your overall objectives as a clinician when delivering CPP to ECC-involved families. ● Open-ended 14) Do you track your CPP fidelity? ● Yes ● No 15) How do you track your CPP fidelity? ● CPP Fidelity Forms ● Other, please specify: 16) Please describe how the COVID-19 pandemic has impacted your implementation of CPP with ECC-involved families? (If there has been no impact, please write “N/A) ● Open-ended

Subsection A: Foundational Phase We’d like to ask you a few questions about each phase of CPP. We’ll start with the foundational phase. We’ll be asking you about assessment/engagement, feedback, and therapist reflection. 17) How many weeks do you typically engage in the CPP foundational phase? ● Open-ended 18) How many sessions do you typically hold with clients in that time period? 19) Open-ended: How long are your typical assessment and engagement sessions? Please provide your response in minutes. 20) Open-ended: Who is typically present at the assessment and engagement?

● Strengths and Difficulties Questionnaire (SDQ) ● Eyberg Child Behavior Inventory (ECBI) ● Other, please specify: 23) How do you typically assess a child’s developmental functioning? ● I do not assess child’s developmental functioning. ● Ages and Stages Questionnaire (ASQ) ● Clinical observation ● Other, please specify: 24) 24. How do you typically assess caregiver PTSD? Please check all that apply. ● I do not assess caregiver PTSD. ● PSSI ● Clinical Interview

● Caregiver only

● Other, please specify:

● Caregiver and child ● Other, please specify: 21) We’d like to understand how you assess and engage with ECCinvolved clients during the foundational phase. In a typical ECC case, how frequently do you complete the following activities as part of the foundational phae? ● Elicit caregiver perception of need for treatment ● E licit caregiver description of family circumstances, challenges, and strengths ● P rovide a sense improvement

positive

expectations

about

● S hare with caregiver rationale for screening for child trauma ● A sk caregiver to jointly complete a child trauma screening instrument ● Consider caregiver’s response to child’s trauma history

25) How do you typically assess caregiver depression? Please check all that apply. ● I do not assess caregiver depression. ● Center for Epidemiological Studies – Depression (CES-D) ● Clinical interview ● Other, please specify: 26) Are there any other instruments/tools that you regularly use in your CPP practice with ECC-involved families? Please check all that apply. ● Adult Adolescent Parenting Inventory ● Adverse Childhood Experiences (ACEs) ● Child Abuse Potential Inventory (CAPI) ● Crowell ● General Anxiety Disorder Questionnaire (GAD7)

● Assess child symptoms based on caregiver report

● Osofsky Relationship Scale Assessment

● A ssess child trauma symptoms based on standardized measure completed by caregiver

● Parent-Child Relationship Inventory ● Parenting Sense of Competency

● Assess child developmental functioning

● Parenting Stress Index

● T alk to caregiver about connection between child’s symptoms and history

● Parental Stress Scale

● Discuss trauma reminders

● Traumatic Events Screening Inventory (TESI)

● A ssess for child safety risks to engaging in traumainformed treatment ● Observe child and caregiver interaction ● Discuss the impact of child trauma treatment on caregivers ● Share rationale for asking about caregiver symptoms ● Introduce caregiver symptom measures ● P rocess information gathered during engagement with supervisor/colleague

● Progress in Treatment Assessment ● Patient Health Questionnaire-9 (PHQ-9) ● Other, please specify: 27) Following the assessment period, how do you engage parents to help develop your treatment plan? Please include specific activities, where needed. ● Open-ended

assessment/

28) When considering potential sources of fidelity challenges to engaging in CPP, what do you typically consider? Please check all that apply.

22) Which instrument(s) do you typically use to assess child trauma?

● E motional process fidelity (e.g., caregiver or child is triggered or shut down)

● Other, please specify:

● I do not assess child trauma. ● D evereux Early Childhood Assessment for Infants and Toddlers (DECA-I/T) ● D evereux Early Childhood Assessment Clinical Form (DECA-C) ● Infant Toddler Social Emotional Assessment (ITSEA) ● B rief Infant (BITSEA)

Toddler

Social

Emotional

FLORIDA INSTITUTE FOR CHILD WELFARE

Assessment

● D yadic-relational fidelity (e.g., caregiver and child have competing agendas, are trauma reminders for one another) ● T rauma framework fidelity (e.g., caregiver or child’s history unknown) ● P rocedural fidelity (e.g., scheduling challenges, family structure complexity) ● Your reflective practice capacity as a clinician ● Other, please specify: 55

29) When CPP is contraindicated for an ECC case following the foundation phase, how do you proceed? ● Open-ended 30) Is there anything else you’d like to share about the CPP foundational phase? ● Open-ended Subsection B: Core Phase Now, we’d like to ask you a few questions about the core phase of CPP. 31) How many weeks do you typically engage in the CPP core phase? ● Open-ended 32) How many sessions do you typically hold with clients in that time period? ● Open-ended 33) How long are your typical assessment and engagement sessions? Please provide your response in minutes. ● Open-ended 34) Who is typically present during the core phase CPP sessions? ● Caregiver only ● Caregiver and child ● Other, please specify: 35) We’d like to understand how you engage with ECC-involved clients during the core phase. In a typical ECC case, how frequently do you complete the following activities as part of the foundational phase? (0 = Never, 4 = Always) ● Prepare for session/select toys for treatment ● Explain the reason for treatment to child (if age appropriate) ● Tracked child response to introduction to treatment ● Support caregiver during introduction to treatment ● E xplain to caregiver any negative reactions/behaviors of either child or caregiver ● Process sessions with supervisor or colleague ● Other, please specify: 36) How do you introduce CPP to the child? (If you do not do this, write “N/A”.) ● Open-ended 37) During the course of CPP, which other therapies or techniques do you incorporate? Please select all that apply. ● Cognitive Behavioral Therapy (CBT) ● Dialectical Behavioral Therapy (DBT) ● Eye Movement Desensitization and Reprocessing (EMDR) ● Gestalt ● Motivational Interviewing ● Parent-Child Interaction Therapy (PCIT) ● Play Therapy ● Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) ● Trust-Based Relational Intervention (TBRI) ● Other, please specify: 38) How do you assess whether your work with a family is progressing toward meeting CPP objectives? (If you do not do this, write “N/A”.) ● Open-ended

FLORIDA INSTITUTE FOR CHILD WELFARE

39) Approximately how frequently do you formally re-assess potential sources of challenge to engaging in CPP with a family? (If you do not do this, write “0”.) ● every

sessions

40) Approximately how often do you formally assess your reflective practice capacity? (If you do not do this, write “0”.) ● every

sessions

41) When reflecting on your CPP practice, which of the following do you typically take into consideration? Please check all that apply. ● Awareness of emotional reactions ● Personal and cultural biases ● Ability to consider multiple perspectives ● Other, please specify: 42) When CPP becomes contraindicated during the core phase, how do you proceed? ● Open-ended 43) Is there anything else you’d like to share about the core phase? ● Open-ended Subsection C: Termination Phase Now, we’d like to ask you a few questions about the termination phase of CPP. 44) How many weeks do you typically engage in the CPP termination phase? ● Open-ended 45) How many sessions do you typically hold with clients in that time period? ● Open-ended 46) How long are your typical assessment and engagement sessions? Please provide your response in minutes. ● Open-ended 47) 1. Who is typically present during the termination phase CPP sessions? Please check all that apply. ● Caregiver only ● Caregiver and child ● Other, please specify: 48) In a typical ECC case, how frequently do you complete the following activities as part of a planned termination? (0 = Never, 4 = Always) ● Reflect on termination ● Plan termination with caregiver ● Plan treatment evaluation (posttests) ● Complete treatment evaluation (posttest) ● Tell child about termination ● J ointly plan termination with caregiver and child (if older than infancy) ● Process the goodbye ● Count down the sessions with the caregiver and child ● D iscuss the course of treatment and the family's treatment narrative ● P lan for future and discuss trauma reminders with caregivers ● Conduct final session ● Other, please specify: 56

49) Is there anything else you’d like to share about the termination phase of CPP? ● Open-ended Subsection D: Reflective Supervision Finally, we’d like to ask you several questions about your experience with reflective supervision as part of CPP. 50) Do you receive reflective supervision? ● Yes ● No [If selected, skip to end of block] 51) In which formats do you receive reflective supervision? Please check all that apply. ● Group ● Individual

into your work with ECC-involved families? ● Open-ended 2) In what instances would you implement [insert other specified modality/modalities] with ECC-involved families? ● Open-ended 3) Is there anything else you’d like to share about your implementation of [insert other specified modality/modalities] with ECC-involved families? ● Open-ended We appreciate your time spent on this survey. If you have any additional feedback regarding the therapeutic parent-child services you provide to ECC-involved families, please provide that here. [open ended essay box]

52) Who provides reflective supervision? Please check all that apply. ● CPP trained reflective supervisor ● Agency supervisor ● Someone else, please specify: 53) How frequently do you participate in regularly scheduled reflective supervision? ● Weekly ● Biweekly ● Monthly ● Quarterly ● Other, please specify: 54) Are you able to access reflective supervision outside of regular meetings if specific needs arise? ● Yes ● No 55) How satisfied are you with your reflective supervisor meeting your needs in the following areas? [5 point Likert scale; 0 = Very dissatisfied, 4 = very satisfied] ● Processing emotional responses ● Considering alternate perspectives ● Seeking new knowledge ● Seeking new skills ● Addressing cultural differences or biases 56) In what other ways does reflective supervision benefit your CPP practice with ECC-involved families? (If none, please write “N/A”) ● Open-ended 57) In what ways could reflective supervision better meet your needs in providing CPP to ECC-involved families? (If none, please write “N/A”) ● Open-ended 58) Is there anything else you’d like to share about your reflective supervision experience? ● Open-ended Section 4: Other Modalities You indicated you also implement [insert other specified modality/ modalities] with ECC-involved families. We’d like to ask a few more questions about how you implement these. 1) How do you integrate [insert other specified modality/modalities] FLORIDA INSTITUTE FOR CHILD WELFARE

Appendix C—Initiative Two Interview Schedules

FLORIDA INSTITUTE FOR CHILD WELFARE