Limitations

have the knowledge or skills needed to help youth. This is notable given caregivers’ lack of confidence or inconsistency in assisting youth with developing most life skills (by domain). CBC lead agency representatives, professionals, and caregivers all noted a lack of available resources and programming. Further, assessing for life skills mastery can be challenging with current assessments, as they typically do not capture demonstration of skills. Youth’s lack of engagement is also challenging. At times, participants attributed this to typical youth behavior, though it was also attributed to inconsistent relationships between youth and caregivers and professionals (e.g., due to placement and workforce turnover). Some professionals perceive caregivers to be disengaged, though caregivers reported workers are not adequately familiar with relevant resources. The evaluation team also identified two important issues that present challenges within Florida’s approach to developing life skills among foster youth for self-sufficiency. First, vague policies create difficulty in translating requirements into practice. While some requirements are clearly outlined in policy and statutes (i.e., caregivers are required to complete life skills logs and discuss life skills assessment with youth), and easy to translate into specific procedures and practices, some are vague. This can make it difficult to translate requirements into concrete practices and procedures. For example (from CFOP 170-17): “Caregivers are required to take the lead in a number of activities demonstrating quality parenting such as assisting youth in mastering age-appropriate life skills.” This statement requires caregivers to demonstrate “quality parenting,” and while there is a definition of what this term means in Florida Statute 409.145, this definition is not included in CFOP 170-17 and may not be clearly conveyed to caregivers. Similarly, caregivers should assist youth in “mastering age-appropriate life skills." While Florida Statute 409.14515 and CFOP 170-17 provide some clarification, questions remain: What are age-appropriate life skills? Are these standard skills, or are they youth-specific? If youth-specific, how does one determine what the skills are? How is mastery demonstrated? Florida’s privatized system provides a great deal of autonomy to CBC lead agencies, which results in variability in some practices. In the context of life skills development, data variability, including the use of multiple different assessment tools and inconsistent uploading of data into the FSFN, create challenges in pulling representative statewide data as well as CBC-level data for comparisons. At a practical level, this might create challenges in consistent assessment and monitoring of life skills for individual youth. For example, a caregiver in the current evaluation noted that paperwork was lost in the multiple shifts in independent living specialists assigned to her foster youth due to turnover. Increased consistency in data collection processes can improve continuous monitoring of life skills provision and development, at the individual, CBC lead agency, and statewide levels.

The findings of this evaluation should be considered within the context of several important limitations. First, despite multiple attempts, the evaluation team was unable to obtain participation from all CBC lead agencies in the state. Related, not all CBC lead agency representatives were able to comprehensively speak to independent life skills provision as it related to their agency. For example, given Florida’s privatized structure, specific responsibilities varied by contact person, i.e., the age ranges of youth in which the representatives work with varied (e.g., 14-21, 18 and older). Similarly, child welfare professional, caregiver, and youth sample sizes were small for both surveys and interviews. Upon review of initial recruitment efforts, the evaluation team expanded the approach, resulting in a modest increase in participation. It is important to note that it was particularly challenging to recruit current foster youth. In general, minors are a federally protected class of research participant and it is common practice to obtain both parental consent and youth assent for participation in studies. Foster youth are further vulnerable to power differentials in research given their status as wards of the state, and there is variation in how researchers have (or have not) obtained guardian consent in previous minimal risk studies (Greiner et al., 2018). The present evaluation took a conservative approach by utilizing a tiered sampling approach requiring professionals and caregivers to provide minor foster youth with evaluation materials to determine interest in participation, versus obtaining a waiver of parental consent from the FSU IRB. This may have hindered recruitment efforts given the level of gatekeeping. In addition to small sample sizes, all participants self-selected into the evaluation, introducing the possibility of self-selection bias. Specifically, participants may represent individuals who had particular experiences. For example, most caregiver survey respondents were non-child specific foster parents, and their experiences may differ from those of kinship caregivers. All former foster youth interviewed were relatively engaged in the range of aftercare services available to foster youth, omitting the experiences of former foster youth less or not as connected to services. Therefore, small sample sizes and self-selection bias across primary data sources reduces confidence in the representativeness of each sample type. Related to sample limitations, it seems not all child welfare professionals in this sample work directly with transition age youth. The evaluators attempted to obtain accurate data on professionals’ life skills-related practices by including eligibility criteria and skip patterns for certain items (e.g., frequency of assessment was limited those who reported conducting assessments), but these data should be interpreted with caution. In addition, it is important to point out that—given our recruitment strategies—there is no clear link between the professionals, caregivers, and youth in this evaluation. That is, when perspectives differ, this could be attributed to the unique experiences of those participants and self-selection bias, as opposed to discrepant perspectives of a particular case. Despite these limitations within the primary data sources, the evaluators were able to triangulate data from multiple sources, which increases credibility of findings contained within this report. Importantly, the evaluation timeline was expedited given the direction of the mandate. Given the complexity of Florida’s system, the desire to incorporate youth voice, and pre-existing burdens on the workforce, a longer evaluation period could have offered opportunity for longer or enhanced recruitment efforts. Finally, there were several limitations related to youth outcome data. First, despite the evaluation team’s efforts to obtain data from both CBC lead agencies and the DCF, consistent data were unavailable. To circumvent this data challenge, the evaluators obtained federal NYTD data, though due to NYTD-specific confidentiality procedures, the evaluators were unable to identify youth who transitioned to independence from all CBC lead agencies in Florida. Improved state-level data collection could address this limitation in the future. Related to the sample, there was also significant attrition between waves 1 and 2, reducing confidence that outcomes at wave 2 are reflective of all youth in the 2017 cohort. Finally, the evaluation team opted to utilize the 2017 cohort of youth to balance the need for at least one wave of outcome data and remain mindful of updates to CF170-17 made in 2017. Thus, outcomes are limited to those at age 19. Future analyses could incorporate 2021 data (i.e., outcomes at age 21) when NYTD makes those data available. Longitudinal analyses can provide a more comprehensive examination of the relationship between receipt of independent life skills development and self-sufficiency outcomes. To this end, the Institute is currently