Evaluation of the Florida Child Welfare Pre-service Training EXECUTIVE SUMMARY
SEPTEMBER 21, 2020 Principal Investigators: Amy Vargo, Ph.D., University of South Florida Pam Menendez, University of South Florida Full Report Available Here
Introduction The child welfare pre-service training evaluation was designed to: 1) assess the readiness of case managers and CPIs to begin their job responsibilities, 2) determine whether pre-service training is at the level it should be, and 3) identify both environmental factors and workers’ individual coping strategies that facilitate and hinder knowledge acquisition and skill development while in the roles of case managers and child protective investigators. Five of the six regions in Florida were represented by nine study sites, comprised of six case management organizations, two DCF Circuits, and one sheriff’s office. This report concludes a two and a half year, in-depth look at the child welfare pre-service training and the firstyear trainees in the field. It is comprised of five core evaluation activities: 1) a knowledge assessment; 2) an observational checklist and focus groups; 3) a self-report survey; 4) a review of the Family Functioning Assessment (FFA) proficiency; and 5) an organizational culture and climate assessment.
Project Overview KNOWLEDGE ASSESSMENT The Knowledge Assessment component was developed to examine transfer of knowledge that occurred as a result of preservice curriculum training, in addition to knowledge retained over time. Findings indicate that knowledge was gained as a result of the CORE curriculum training as well as the Specialty Track training. Although some scores decreased by the 6-month followup assessment and again at 12-month assessment, overall, scores were still greater than those observed at pre-test. It is likely that some items included in the knowledge assessments are more applicable to case managers’ and CPIs’ roles than others, which may explain the fluctuation over time of correct responses. Participants also stated there may be a gap between what is learned in pre-service training and what supervisors support and prioritize once workers are in the field. CASE MANAGER AND CPI FOCUS GROUPS The purpose of the case manager and CPI 6- and 12-month focus groups was to examine how helpful the child welfare pre-service training was to case managers and CPIs in the day-to-day work they do. Overall, participants felt that their pre-service training was beneficial to developing the necessary skill set for their daily work as child welfare professionals. They indicated that information provided on identifying and understanding maltreatment was adequately
covered. Areas of the pre-service training they found most helpful were role playing exercises, shadowing experienced child welfare professionals, and field days. Case managers felt that the CORE training was geared more toward CPIs and proposed several areas where additional training was necessary, such as navigating all aspects of the judicial process, case and safety plan updates, working in FSFN, aspects of adoption work, and time and stress management. Case managers also asked for more field days scheduled at times when experienced case managers were conducting home visits. Other suggestions included more role-playing exercises; jointly carrying an active case during pre-service training with an experienced child welfare worker; and providing a peer advocate for case managers to go to with questions once in the field. Some case managers expressed a reluctance to ask for help from supervisors who had indicated that pre-service training should have provided the necessary skill set. Case managers also reported that their understanding of FFA-Os and safety planning, two critical functions of their daily responsibilities, were not taught and/or practiced at a level that gave them enough knowledge or confidence once they were working directly with children and families. CPIs additionally asked for more practice with assessments and data systems that are required for daily use; and to make the training more comprehensive and realistic in addressing the wide variety of individuals and situations CPIs can expect to encounter in their work. CPIs reported that their understanding of FFA-Is and safety planning, two critical functions of their daily responsibilities, required more opportunities to observe and practice with “real” families in order to fully understand. In addition, first year turnover rates were high. Participants named several factors in their decision to remain employed in child welfare, such as loving the work they do, feeling that they made a difference, having a supportive supervisor and supportive co-workers, and that some case management areas are perhaps easier than others to stay in. Case managers, specifically, pointed mainly to the quality and accessibility of supervisors and the support of coworkers as major factors in their desire to remain in their positions. Alternately, reasons provided as to why peers left their case management position included emotional strain, an unsupportive supervisor, turnover as a domino effect, and low salaries/lack of recognition. SELF-ASSESSMENT SURVEY Case managers and CPIs completed an electronic, self-assessment survey at baseline (approximately 1 month), 6 months, and 12 months after their unit start dates. Overall, the responses by case managers on the 6-month self-assessment indicated ongoing challenges with key areas in transfer of knowledge that were also reported at the 1-month interval. These included: difficulty obtaining enough hands-on experience in pre-service training; a stressful work environment with unrealistic expectations for caseloads; and some discrepancies between expectations during pre-service training and actual practice, which may, in part, be due to different approaches to the work by superiors. With regard to supports in retaining knowledge and skills, case managers and CPIs consistently pointed to the benefit and importance of internal support by supervisors and other colleagues and mentors.
FAMILY FUNCTIONING ASSESSMENTS A review was conducted of FFA-Is that were completed by CPIs to make initial case decisions and FFA-Os by case managers to make ongoing case decisions. Overall, reviews indicated a lack of understanding of how to accurately assess and document decisionmaking regarding child safety and risk, as well as insufficient documentation and evidence of information collection needed to make informed decisions in most areas and domains. In addition, only 35 percent of cases documented supervisory consultation before case plan approval and only 38 percent had evidence of regular consultation between supervisors and case managers; this declined further at 12 months. Although pre-service training appears to teach, within a classroom setting, the purpose of the FFA, assessment, engagement, and information collection and documentation, these skills and knowledge do not appear to translate to the field. ORGANIZATIONAL CLIMATE AND WORKPLACE CULTURE ASSESSMENT Participants completed the Organizational Climate and Workplace Culture Assessment survey at 3 and 12 months after pre-service training. It was commonplace for trainees to view a new child welfare job as only temporary until a better opportunity became available, with respondents expressing little aspiration for promotion within the field. The majority planned to leave their job to look for employment outside of child welfare. Staff remaining in their positions for 12 months or longer were more satisfied overall with their work than the 3-month respondents. Specific job areas that saw the highest gains in satisfaction as time progressed were satisfaction with intellectual challenge, organizational climate, and opportunities for advancement. However, satisfaction over time regarding salary and benefits dropped, and emotional strain and burnout were common at both data points.
Policy Recommendations
•
Incorporate time management or disaster planning/coping strategies into pre-service training and consider bringing in a more experienced case managers/CPI to a pre-service training day in order to talk about what strategies they developed to cope.
•
Continue emphasizing hands-on learning strategies in preservice training such as mock cases and trials, role plays, and FSFN practice.
•
Consider increasing the number of field days during preservice training, as well as a more formalized process for assuring that field days are scheduled on days and times that provide new case managers and CPIs with the ability to observe and learn from relevant activities. Participants stressed the need to learn under less scripted conditions than what the current role-playing exercises offered during preservice.
•
Consider scheduling field days directly after related training content, so as to better match training content with field experience. For example, if FFA-Is had just been reviewed in class, participants felt it would be helpful, then, to see a case manager complete one in the field.
•
Develop strategies across regions to provide realistic opportunities for interacting with families with diverse scenarios during training (e.g., “field house” model or role plays with outside personnel or volunteers).
•
Provide clarification during pre-service training on the function and importance of FFAs for CPIs and ensure dedicated inservice training on FFAs is available on a continuous basis.
•
Increase training content and duration on interviewing skills, specific to interviewing both parents and young children.
•
Increase training content on case management functions. Case manager trainees suggested spending more time on specific aspects of case management rather than the frontend maltreatment that caused the investigation or aspects of CPI work. Participants were interested in more detail on dayto-day activities, processes, and forms case managers need to complete.
•
Integrate supervisors into pre-service training processes to create a more fluid transition between knowledge gained during training and expectations in the field.
•
Consider ways to decrease the caseload size of new workers during their first year to prevent the cascading effects of turnover.
The following recommendations are made to improve the training experience and first year of employment for new case managers and CPIs: •
Provide additional information to trainees regarding what types of challenges workers can expect to face during their first year (e.g., viewing a day in the life of an experienced case manager/CPI), so they may better determine whether or not the job is a good fit for them.
•
Consider what formal supports, such as field trainer positions, can be provided during a trainees’ first year to help staff better cope with emotional strain.
•
Consider additional support positions for supervisors that would assist with observations and feedback once trainees are back in units (e.g., Mentor, Field Training Coach, Advocate).
Questions & Comments FOR MORE INFORMATION, CONTACT:
Amy Vargo, Ph.D.
Assistant Professor University of South Florida Department of Child & Family Studies avargo@usf.edu
Pam Menendez
Program Director University of South Florida Florida’s Center for Child Welfare pmenendez@usf.edu
Lisa Magruder, Ph.D.
Program Director of Science & Research Florida Institute for Child Welfare lmagruder@fsu.edu
Marianna Tutwiler
Program Director of Administration Florida Institute for Child Welfare mtutwiler@fsu.edu