Roadmap to Peace: Process Evaluation Report 2017

Roadmap to Peace A community-defined approach to improve the health, safety and economic opportunities of Latinx youth

Process Evaluation Report 1: The Service Network in the Pilot Year

Elizabeth Brown Paula Fleisher Sheldon Gen Belinda Reyes

CĂŠsar E. ChĂĄvez Institute College of Ethnic Studies San Francisco State University

Community Engagement & Health Policy Program Clinical and Translational Science Institute University of California, San Francisco

February, 2017

1

Introduction

p.2

1.1

RTP and its mission

1.2

The Healing Wheel as conceptual logic model

1.3

Purpose of the formative evaluation

2

Building a community initiative to impact young people’s lives

2.1

Funding and resources

2.2

RTP backbone

2.3

Community service partners

2.4

Community partners: Staffing and programs for RTP youth p.13

2.5

The RTP Service Model

3

The service network in the pilot year

3.1

RTP youth in the pilot year

3.2

The recruitment and referral Process

3.3

Care management and individual service plans

3.4

Services provided to RTP youth in the pilot year

3.5

Data collection and maintenance

3.6

Service utilization

3.7

Beyond Connection

3.8

Successes and challenges in service provision

4

The RTP holistic model of care

4.1

Adopting collective impact

4.2

Restorative justice

4.3

Youth-informed

4.4

Trauma-informed/Recovery

4.5

Timely service provision

5

Recommendations for RTP service network

5.1

Expand network and organizational capacities

5.2

Prioritize and address youth service gaps

5.3

Prioritize and implement personnel training

5.4

Improve information and communications systems

p.3 p.4

1

p.6

p.8

p.8 p.9

p.14 p.16

p.17 p.19 p.19 p.21

p.22

contents

p.7

p.24 p.25 p.28

p.31 p.32

Appendices

p.33

Figure A - Service Network

p.36

Figure B - RTP Process

p.38

p.39 p.41 p.41

p.43 p.43 p.44

p.48

p.47

1 Introduction 2

I

n 2012, six Latinx youth were murdered within a 5-week period in San Francisco. The Mission Peace Collaborative, a coalition of community members and organizations, decried this violence and convened meetings and town halls to promote peace and address this surge in Latinx youth violence. Over a two year period, members met weekly, organized committees and working groups and engaged in a comprehensive, bottom-up development of a community approach to end violence experienced by Latinx youth. A member of the collaborative recalls that time: Because of the commitment that everyone has – and putting that time – it wasn’t easy to set that time aside. There may be some tough conversations that we had to have. But because of people’s commitment to knowing what this meant for the community and for these young people, people really have set that dedication and that time and to continue to come to the table. I wouldn’t say that it’s smooth, but I think it’s been an inclusive process. It’s been a community process. People have been committed to making it work.

This commitment to “making it work” resulted in a comprehensive, culturally rooted, community-based plan for violence prevention called the Roadmap to Peace (RTP), which is now in its first year of implementation, after completing their pilot year. This report provides a formative evaluation of the RTP approach in the pilot year.

1.1 RTP and its mission Built on several decades of violence prevention work by Mission-based community providers and grounded in a collective impact model, RTP aims to address the unique experiences of Latinx youth, who along with African American youth, have the highest rates of poverty, lowest rates of academic achievement, and highest rates of involvement in the juvenile justice system in San Francisco1. Living in a city with the most expensive housing in the nation, Latinx and African American youth also face displacement and housing instability as a result of severe gentrification and family evictions. The Mission district, a historically Latinx neighborhood in San Francisco, was 50% Latinx in 2000, but only 38% in 20132. The City also adopted a series of gang injunctions, increasing police intimidation, and bolstering an already punitive justice system. A recent report by a Department of Justice investigation of the San Francisco Police Department found “numerous indicators of implicit and institutionalized bias against minority groups” among San Francisco’s police officers (p.10)3. The RTP initiative brings together eight community based organizations to address this reality in the lives of Latinx youth.

3 barriers to healthy development for young people by connecting them to a continuum of support services to improve health, safety and economic outcomes. To achieve this, the eight partner organizations are committed to the following approach: • Community-based knowledge & solutions informed by trauma recovery and restorative justice approaches • Providing timely services, rather than lengthy wait-times • Shared care conferencing through an integrated, coordinated network rather than isolated services • Advancing policy reforms that invest in the capacity building and development of Latinx youth RTP consists of three interrelated strategies that inform one another to produce a longterm decrease in violence: a comprehensive

RTP is a five-year comprehensive, communitybased, collaborative approach to youth violence prevention grounded in a collective impact model. They engage Latinx youth, ages 13-24 who are disconnected from school and/or employment, in-risk of violence or system-involved. RTP aims to remove Mission Analytics. (2015). Analysis of Racial and Ethnic Disparities in SF Juvenile Justice. Mission Analytics, as quoted by DCYF’s 2016 Community Needs Assessment, A Snapshot of San Francisco Children and Families, http://dcyf.org/Modules/ShowDocument.aspx?documentid=4470 1

Full report about gentrification in the mission is available at http://medasf.org/redesign/wp-content/uploads/2015/08/1429-03252015_MPNNALCAB-Mission-Housing-Assessment_v151.pdf 2

See the full report from the Department of Justice at https://ric-zai-inc.com/Publications/cops-w0817-pub.pdf 3

4

social support network, policy advocacy and community building. The RTP network aligns services and funding, client intake, and tracking mechanisms to provide wrap-around care for Latinx youth. Furthermore, RTP aims to develop public policy and increase the

capacity of the community to generate longterm solutions to violence.

1.2 Healing Wheel as a conceptual logic model The RTP’s initiative primary goal is to strengthen the resilience of in-risk and violence-involved youth. RTP’s innovative approach is informed by the deficiencies of traditional service provision where youth often work with several service providers, without any communication between them.

Services are disconnected from one another.

This disjointed and bureaucratic system often Figure 1 - Healing Wheel

left youth underserved and frustrated. As one person from the network explained, “there was a lot of pathologizing of our communities.

Everything becomes individualized and pathologized… And there’s no connection to all the other things around the environment.“ In response, RTP developed a holistic set of coordinated services to strengthen youth “resiliency” (see Figure 1).

Resilient youth are defined by the philosophical healing wheel that guides RTP efforts as those who have: • Healthy Lives (social/emotional support, health care, and engaging in healthy behaviors) • Lifelong Learning (educational support, pathways towards a degree or vocational training) • Community Connections (mentors and family support, youth leadership and community advocacy) • Self-Sufficiency (job training and workforce development opportunities, housing, literacy, employment status) The RTP healing wheel philosophy inverts the pathologizing mechanisms of traditional services, as one member of the network explained:

“Instead of thinking about Latinx youth as dangerous to the public, the RTP wants to protect the safety of youth [and limit] social barriers that affect them. The idea is to understand the vulnerability of these kids….and the deeper you look, and the more you know them… the more you’ll see the many macro and micro aggressions they receive every day.”

The healing wheel guides an approach to youth violence that understands the vulnerability of youth, addresses the trauma they and their families experience, and provides viable alternatives to violence involvement for young people in culturally and rooted ways.

Providing services from arts to counseling to care management, RTP aims to restore young people’s safety and empowerment.

5

1.3 Purpose of the formative evaluation 6

To help in the refinement of the initiative, SF State’s César E Chávez Institute (CCI) and UCSF’s CTSI Community Engagement and Health Policy Program were contracted to design, implement, analyze, and disseminate a formative evaluation of RTP’s pilot year, concentrating on the service network. The external reviewers worked closely with the Roadmap to Peace (RTP) Data and Evaluation (D&E) Work Group and RTP Steering Committee in the design, formulation, and analysis of the evaluation results.

In their pilot year (July 1 2015 –June 30, 2016), RTP concentrated on hiring the needed personnel, establishing the service network, and strengthening the partnership by aligning practices, increasing communication and solidifying the backbone structure of RTP. Stakeholders also refined RTP’s restorative justice, timely, trauma and youth informed model of care. In the second year of the initiative, and the first year of implementation (July 1, 2016-June 30, 2017), RTP continues this work, while also developing RTP’s policy and capacity building agenda.

organizations, we interviewed service providers (e.g., care managers and counselors), program managers, and executive directors (see table 1). Six of the interviewees were also voting members of the Steering Committee. Each of the interviews were assessed: (1) individually as compared to others in the RTP collaborative; (2) we compared responses from respondents from the same organization; and (3) we compared responses of individuals with the same position (all care managers, for example). This information provided an in-depth understanding of how interviewees view the developing collaboration, and gave us an opportunity to assess and identify convergent and divergent views and practices. We drew additional context and impressions from our participation in RTP network activities, including Data and Evaluation Work Group and Service Network meetings. As the evaluation got underway, instruments and protocols were vetted by the Data and Evaluation Workgroup, the part of the RTP “backbone” that will continue to build and implement RTP’s overall internal accountability structure in collaboration with the evaluation team.

This formative evaluation examines the development of RTP’s service network in the pilot year with respect to: 1. operationalization of the collaborative service mode;

Table 1: Service Provider Sample

2. development of the collective impact structure; 3. implementation of the philosophical model of care, This report is based on 30 semi-structured interviews conducted between April and September 2016 (protocols in Appendix) with service providers (26 interviewees) and RTP’s core staff (4) and descriptive analysis of program data from RTP agencies and DCYF. At each of the 8 partner

Note: Some interviewees hold more than one role.

2 Building a community initiative to impact young people’s lives “I’m tired of seeing people buried...”

RTP’s organizational structure is the result of community efforts to build a strong, coordinated multisector system of care that could also sustainably effect large scale systems change for disconnected Latinx youth. Early on in the development of the initiative, RTP planners and leaders looked to the collective impact model to help shape and refine that structure. First coined in 2011, “Collective Impact” requires five conditions to be successful (see table 2)4 . RTP providers at all levels describe RTP’s structure as a uniquely effective way to “wrap youth in a blanket” and keep them from “falling through the cracks.” Latinx youth without such a network of support face a constant risk of disconnection and resulting violence involvement.

Table 2: Conditions for Collective Impact

Collective Impact | Stanford Social Innovation Review”. July 2011. ssireview.org.

4

7

2.1 Funding and resources 8

Initial funding from SF Department of Health supported the planning for the initiative.

Community advocacy led the mayor of San Francisco to fund a five-year grant from the City of San Francisco’s Department of Children, Youth and Families (DCYF) to launch the initiative. The collaborative was also successful in securing the

Google’s Impact Challenge Grant and funding from Jobs for the Future Opportunity Works grant, supported by the Corporation for National and Community Service’s Social Innovation Fund (SIF)5. Two primary organizations lead each grant: IFR administers the DCYF grant and BACR the other two.

2.2 RTP Backbone Collective impact literature defines the “backbone” organization as the people and processes that fulfill the administrative, strategic planning and leadership/champion functions of the collaborative. RTP’s “backbone” is created by several different groups. First, RTP is overseen by the Steering Committee which guides the agenda of the initiative and is comprised of members from 8 social sectors, plus a coalition of community-based organizations (Mission Peace Collaborative) and transitional age youth.6 Currently, the director of RTP and an organizational partner program manager serve as co-chairs of the Steering Committee. Second, RTP administrative functions are overseen by five core staff: Director, AJ Napolis; Community Builder and Planner, Angela Gallegos-Castillo; Service Coordinator, Chris Reyes; Youth Organizer, Ashley Rodriguez, and recently hired Administrative Assistant, Brandie Bowen.

Third, RTP also organizes other groups to strategize about specific aspects of the RTP process (see figure 2). The “Service Network” is the collective of all RTP providers. The “Care Development meetings” is a subset of the “Service Network” and consists of network providers from various organizations who meet on a biweekly

basis to advance services planning and support self-care for providers. The meeting is facilitated

by IFR’s mental health specialist with backbone support from the Service Connector. The “Data and Evaluation Work Group” meets biweekly to ensure that RTP services and processes are well-documented and evaluated. The “Fund Development Work Group” is in charge of seeking external funding for the initiative.

Meanwhile the Youth Leadership Group, “RTP Warriors”, is in charge of youth engagement and development. Finally, RTP is in the process of convening a “Policy Advocacy Work Group” and an “Education Work Group” to direct RTP’s policy transformation initiatives and educational advocacy.

The role of Instituto Familiar de la Raza is key to the strength and functioning of the RTP “backbone.” IFR has decades of grounding and expertise in the social, emotional and spiritual needs and strengths of the community served by RTP. IFR is also fiscally strong, and thus provides a stable institutional base for the initiative. Core staff credit IFR’s executive director with orienting them to RTP and providing ongoing mentoring and guidance.

http://www.jff.org/initiatives/back-track-designs/opportunity-works The sectors included are: workforce, the Mission Peace Collaborative, housing, legal/immigration, health, arts and culture, academia, education, community, and transitional age youth (TAY) 5

6

Figure 2: RTP Initiative Operational Structure 9

2.3 The Community Service Partners During the pilot year, RTP was composed of eight social service organizations -- Instituto Familiar de la Raza (IFR), CARECEN of San Francisco, Mission Neighborhood Health Center (MNHC), Mission Neighborhood Centers (MNC), Homies Organizing the Mission to Empower Youth (H.O.M.E.Y.), Bay Area Community Resources (BACR)/Communities in Harmony Advocating for Learning and Kids (CHALK), Five Keys Schools and Programs (Five Keys), and Horizons, Unlimited. To ensure smooth collaboration, RTP created “Articles of Collaboration” which govern their collective work together.

RTP organizations are longstanding anchor institutions in the Mission District, San Francisco, and the Bay Area region (see table 3). The

organizations are well-integrated throughout the region,7 and all have prior experience with community partnerships. The youngest organization has been serving youth for over 13 years (HOMEY), and four organizations have budgets over $10 million. These organizations have been at the forefront of youth leadership development, violence prevention, and family support in San Francisco for many years. All were also part of the Mission Peace Collaborative and the community dialogues that led to the formation of RTP. The partner organizations share similar values of providing culturally sensitive community based services to their clients. Organizations aim to develop or transform individuals, families, and/ or communities through sustained leadership

BACR has programs in San Francisco, Napa, Contra Costa, and Alameda counties. MNC has 11 sites throughout San Francisco and provides a wide range of services with community partners. And about a third of IFR’s funding is coming from 3 different community partnerships they are currently leading.

7

10

Table 3: RTP Service Network Partner Organizations: Overview

CHALK is a project run out of BACR since 1996 that employs youth as service providers for their peers and aims to provide transformative employment and leadership development opportunities.

8

development and empowerment approach that builds the capacity of their clients. Components of the RTP model are found within the strategic visions and missions of each organization: community and/or youth engaged, trauma recovery and/or culturally based programs, and restorative orientations.

RTP organizations provide a wide range of services to the community, as shown in Table 4. Three of the organizations specialize in youth services, such as workforce and leadership development (CHALK, HOMEY, and Horizons). Three other organizations focus on providing culturally responsive services to Latinx children and families (MNHC, IFR, and CARECEN). MNHC has two clinics that provide culturally sensitive health care and wellness to Latinx children, teens and adults and also runs a homeless center. IFR provides trauma recovery, culturally-based wellness and mental health to primarily Latinx clients.

CARECEN serves primarily Latinx youth and families providing trauma recovery and mental health services, immigrant legal services and help for youth to leave behind violent pasts through their Second Chance Tattoo removal program. MNC provides support to low-income families and has a wide range of programs for children, teens, adults and seniors. Five Keys is a public high school for adults and transitional aged youth that provides educational opportunities for individuals with barriers to their education including those who are incarcerated in the SF County jails. This wide range of services creates opportunities to continue to expand the services provided to RTP youth.

11

12

Table 4: RTP Service Network Partner Organizations: Services and Staffing

9

Patients served for medical services

2.4 Community Partners: Staffing and programs

for RTP youth Agencies, in the pilot year, have specialized in providing particular set services to RTP youth (as described in Table 5). The primary providers focus on care management while providing a specialized service in workforce, trauma recovery,

and education10. Secondary providers offer other supportive services: a fatherhood program, safe passage rides for youth, substance abuse and medical care.

Table 5: RTP Service Network Partner Organizations: Services and Staffing

11

In the first year of implementation, RTP has moved away from the primary and secondary labels. But the structure itself has not changed in terms of the number of organizations providing care management for youth and inter-agency referrals in the network.

11 See Figure A in the Appendix for a visual representation of the network.

12 Horizon is currently searching for a counselor for RTP. 10

13

14

As this was the pilot year, most of the staff involved with the program are new – two-thirds of the staff and all the care providers. RTP agencies hired staff with significant experience, but some positions are still vacant – IFR has not been able to hire a mental health expert.

This builds delays into the referral process as providers must learn about each other’s services and people on an ongoing basis while developing the network.

contributing their time in-kind. This is always necessary as a new initiative is launched, particularly one as complex as RTP, but it may not be sustainable in the future and may impact the agencies’ capacity.

As the initiative develops and organizations adjust to a new program and new staffing some of the work is falling in the hands of middle managers and administrators who are

2.5 The RTP Service Model Youth who join RTP go through a process that members hope efficiently connects them to needed services. The process involves recruitment and referral, care management placement and life plan development, and referrals throughout the system and beyond (Figure B in the Appendix shows the process graphically).

RTP’s partners share in the recruitment of youth for the initiative. There are two different referral processes for youth who are in custody and those who are being referred from the community. Those who come from the community are referred to RTP from: community-based organizations, schools, and direct community referrals. Youth go through a referral process (online shared referral link) and fill out a referral form. The form goes to the service connector. The service connector filters all the referrals and direct youth to one of four agencies that were assigned care managers in the pilot years (primary providers) – IRF,

13

CARECEN, BACR/CHALK, or Five Keys13. If the youth is in custody, Five Keys’ care manager cares for them while they are in custody and help set them up for reentry. A care manager describes the process for youth recruited in custody: I help get them prepared for reentry.

So what we do is we meet with the individual, and we develop a reentry plan. So the reentry plan also spells out, then, who's the agency that they're going to be handed off to in the community. So whether it's BACR, Instituto Familiar de la Raza, or CARECEN, for primary case management in the community. At that point, they get handed off of there.

At the primary organization the youth are assigned a lead care manager. The lead care manager holds the young person through the network. First they fill out the global intake form (partnering organizations developed a

Since we completed our analysis a fifth organizations is now providing care management, MNC.

global intake form that covers the different areas and certain key questions to ask in the first initial global intake). They do an assessment and develop a care plan with the youth that will identify different services that the network offers through mostly secondary providers, and then monitor the youth’s progress throughout the network.14 As is necessary for mutually reinforcing activities, the care connector and the primary care provider need to know about the other programs in the network to know what best to provide for each youth. But they also track the progress of the youth through the programs.

It's really important to understand what everybody's providing. Then when they're with the young person, they can negotiate that. They can see when to make a referral, what referral to make, and then doing that real warm handoff and being able to... Then it's important to be able to keep these secondary providers updated and connected on what they need to know.

The care manager needs to know what progress they are making in each of the programs and the services they're going to – and also being able to communicate back out to the secondary providers anything that we need to know that affects the work that we're doing, as well… Their role is to know what's out there, who's out there, and to make those connections in a very meaningful way and be able to still keep everybody on the same page, and that's working with the young person.

In order to facilitate this continuous communication and mutually reinforcing activities, a key part of network development is building trust amongst service providers through frequent interaction and collaboration.

Many providers are understandably protective

of “their youth”, and thus, they must trust other providers to provide high-quality care when they refer their youth to another organization.

To facilitate building this trust requires significant information across the network and coordination. Providers come together once a month for service network meetings and biweekly for care conferencing. They also come together for workshops, trainings, community forums and RTP events.

Coming together on a regular basis, people also just start to get to know what people are doing. They hear more. They get to know each other, too. People continuously coming together every other Monday or twice a week. That helps, but there’s more that we can do for care managers to know very intimately what the different providers are offering.

Capacity building in the network is thus an ongoing process. Further, this capacity building has also been challenging as RTP’s philosophical model is built on agencies coming together with the same goals and ends. Yet, most agencies had their own way of doing things prior to coming into RTP, so part of the pilot year has been spent making sure that all in the network are working towards the same goals.

There may be some inefficiencies, but the delivery system you build on the skills of your partner, but also the opportunity to share, cross learning, is a good thing and builds morale.

I’m learning a lot about how other organizations do things. Trying to get everyone to do things in the same way has been a challenge. We would want more uniformity, but everyone has their own way to do things. Everyone is very serious about what they do.

RTP is currently developing an orientation for the youth that join the program to learn about the entire network and all the programs available to them as part of RTP. This was not taking place during the pilot year.

16

15

3 The Service Network in the Pilot Year 16

Next we describe the results of our analysis.

We examine the development of RTP’s service network in the pilot year with respect to: (1) operationalization of the collaborative service mode; (2) development of the collective impact conditions; and (3) implementation of the philosophical model of care. This would give RTP an understanding of the implementation of the model during the pilot year from the perspective of the people implementing the initiative.

RTP leaders recognize that the power of the RTP collaborative to change both people and systems is greater than the sum of its parts, even if that power is defined by interdependence and loss of significant autonomy of any single partner. The promise of RTP as a collective impact initiative beginning with the service network - is grounded in a shared belief in the potential of disconnected Latinx youth to thrive and the credibility of the organizations doing the work. Many RTP staff articulated that the currency of Collective Impact ultimately is

relationships, and for the Service Network, the trusting relationships between front line staff of different organizations are critical to serving RTP youth. Relationships developed in RTP support the creation of a web of support and safety for the youth. This web of support, however, requires that organizations extend themselves beyond the bounds of coordination and collaboration.

“How do you coordinate coordination and not just become on paper a collaborative? But then in reality, you don’t even know what each other’s doing, you’re not...you’re not interdependent. I think that is the goal of RTP in my mind, value-wise, is to become very interdependent.”

This interdependence is supported and fostered by the Service Network processes described above; in this section, we examine how this interdependence has worked in practice over the pilot year.

3.1 RTP youth in the pilot year A priority of RTP in the pilot year was to develop a targeted strategic outreach approach to reach the youth most at need. They wanted to reach youth that are system involved, youth referred from schools, agencies and community partners, but they also wanted to target youth in the streets.

In the Pilot Year, RTP clients reflect RTP’s intended population (see Table 6). Of the 88 participants,15 99% were Latinx, most identifying as Central American or Mexicano, and 9% identifying as Multiracial/ Multiethnic. The overwhelming majority of RTP youth (75%) were male, while 22 youth (25%) were female. About half (51%) have experience with the juvenile justice system -- 12 youth have been to juvenile hall, 12 have been to jail and 21 have been to both. Moreover, a significant number of them are reporting violence and vulnerability -- 59 youth (67%) reported having been bullied or harassed; 14 youth have been removed from their family by police or social worker or have lived in a foster home; 11 youth were current or previously homeless; and 9 had or currently has a parent in jail or prison. Sixty five youth (74%) were unemployed. at time of referral.

The majority of the youth’s home language was English, or 55% of youth (48); while 45% spoke Spanish at home. Most RTP youth are concentrated in two zip codes: 94110 and 94112. These two zip codes cover two of the neighborhoods with the highest concentration of Latinx youth in San Francisco: the Mission 17

This is the number of RTP youth in the program by October 1st, 2016.

District and Excelsior (or Outer Mission).

When asked about the primary needs of RTP youth, responses varied widely with no

obvious clustering. Responses included a sense of belonging, housing (particularly for undocumented youth), employment, trusting relationships, legal representation, care management, mentorship, education, advocacy, substance abuse counseling, basic needs (e.g., clothing, food, shelter), community connection, spiritual connection, and healthcare. The perceptions were clearly related to the services the respondents provide, or their training.

17

18

Table 6: Participant Demographics at time of enrollment

Source: Researcher’s calculations from CMS data.

3.2 Recruitment and referral process Respondents mostly shared a consistent uInterview respondents mostly shared a consistent understanding of the processes for recruiting youth into the RTP program. That shared understanding was clearest among the care managers, while there was some vagueness among secondary service providers and some managers.

Respondents described multiple points of outreach to the community and access to the program. RTP staff present the program at community meetings, local high schools, and city departments, to encourage them to refer eligible youth to the program. Those efforts have resulted in actual referrals coming from all those sources, including the district attorney’s office, the public defender’s office, juvenile probation department, schools, and community based organizations. Of course,

referrals also come from the RTP partners themselves as youths seek services from each individual organization. Some youths who are aware of RTP have even self-referred themselves to the program, particularly when they are incarcerated.

RTP is a young initiative, but it has already established a presence in the community. One respondent noted, “I’m surprised that a lot of people know about RTP. I thought because it was a brand new initiative, a lot of people wouldn’t know, but especially the schools they know about RTP!” Even so, MNC respondents noticed that fathers are underrepresented in the youths who have been recruited. They expect more fathers, and others with parental roles, who are RTPeligible than the numbers they are currently serving.serving.

3.3 Care manangement and individual service plans There is broad agreement among care managers and other direct service providers on the processes of care management in RTP. Once a care manager is assigned to a youth, that care manager contacts the referring agency, then meets with the youth (with their parents if they are under 18) to establish the relationship.

The care manager conducts assessments of the youth’s conditions and needs, and together they develop a service plan, drawing from the resources and opportunities of the RTP partners (See Figure B in the Appendix for a flowchart and timeline for RTP process). Sometimes, when the youth’s referral comes from another RTP partner, the referring partner will consult on the service plan as well. Once the plan is agreed upon, the care manager and youth make appointments for services, and the care manager introduces the youth to RTP partners, through what is often referred to as ‘warm handoff.’

These organizational and interpersonal transitions for the youth, from one service provider to another, are facilitated by a strong sense of network and family among the RTP

service providers.

There is a perception among some providers that servicers are underutilized, despite a known level of need. As reflected by the interviews, there were challenges during the Pilot year in service planning and referral. For example, providers felt that while the assessments at intake might identify a high need for substance abuse counseling, or trauma services, but only a fraction of those in need proceed to engage those services. Respondents observing this phenomenon offered three distinct explanations: (1) youth unwillingness to receive a particular service, (2) how care manager handles tensions between rival youth groups and (3) centralization of services creating bottlenecks in service delivery.

TThe first reason why a youth may not receive services may be the youth’s lack of readiness to address those needs. Care managers might prescribe services for substance abuse, mental health, or trauma recovery, for example, but

19

20

the youth may not show up because they are not ready to address those needs. This opting out may even be encouraged by the program’s approach to care. As one respondent noted, “The whole vision was that we would meet you where you are,” meaning the youths drive their participation in RTP services, rather than RTP imposing them.

Still, another respondent noted the disservice that can arise from youth-driven service engagement: the professional advice of RTP service providers may be ignored, leaving the youth underserved.

“There is a definite emphasis… on being clientfocused, client-driven, but in my opinion… it should not ignore your expertise.” She went on to illustrate the problem, “If a person wanted an Ebola vaccine, would we offer them that instead of STI screening?

You have to provide your expertise as well.”

Another service provider offered his compromise to this tension. In order to be youth-focused while taking expert advice, he negotiates with the youths. “The client gets two goals and [we] get one,” in order to start addressing underlying barriers to success that might otherwise be ignored by the youth.

“The kid might say ‘I want a job, and I want a parenting group.’ And so we help them accomplish those goals,” but then they set a third goal and say “We need to deal with that substance abuse issue or that trauma issue.”

Second, some tensions that exist between RTP youths (e.g., rivalries) may affect whether referrals are made to partner service providers or if youth follow up with the referral. There is a real concern to not place the “wrong mixture of youth in the same room,” as one service provider described. But others noted that they are able to deal with those tensions between youths, and the RTP partners should trust that ability. One respondent noted that the RTP initiative is young, and even the RTP partners are still building trust and understandings of their capabilities among themselves.

Third, some see the centralization of service referrals with the care managers as creating a “bottleneck” in service delivery. They note that

care managers are necessary to connect RTP youth to the continuum of care, but they may actually be “choking” service delivery, perhaps through the sequencing of services, or through the youths’ varied interests in services on their plans.

Regardless of the explanation, the upshot of this phenomenon is not only underserved RTP youth, but also underutilized services for the community. When service capacities are reserved for RTP youth, and RTP youths do not use those reserved services, those services sometimes go unused. This is the situation with tattoo removal, for example, where the service providers perceive high demand, but low service utilization. Some service providers suggested there would be “two tiers [of clients], there’s core and there’s secondary,” in order to fully utilize RTP services. Another idea was to develop more accurate predictions of RTP service utilization, and allocate resources accordingly. Right now, some perceive that too much service supply is reserved for RTP youth and going unused.

Individual service plans developed with their Individual service plans developed with their care managers guide the youths through their formal RTP experiences. Examining CMS data on Table 7 we see that most RTP youth (57) that joined RTP in the Pilot year were referred at least once. The average youth is referred 1.6 times, but there were a number of youth that were referred for more than one service. Employment, school and mental health were the largest categories of referrals. The clients who experienced the most referrals – 5 or more – received a wide range of services from the network, for example ESL courses, health, housing and immigration services or another client that received cash assistance, housing, legal, out-patient substance abuse treatment and school guidance. Given this is the Pilot year is encouraging that care managers are doing a number of referrals throughout the system of support that is being created for youth. Periodic information about how youth are moving through the network would be important to keep partners inform. This would help better manage resources at each network.

3.4 Services provided to RTP youth in the pilot year A challenge identified by respondents focused since most of these youths would be eligible on the status and role of the “secondary” service for those services anyway. As a result, when providers in the partnership, those who are not asked about services provided to RTP youth, providing care management. The “secondary” some participants went beyond what they were designation clearly frustrated some of those specifically designated for RTP. They are listed partners, inferring that they are less than full parenthetically in table 7. Notice inconsistencies partners in the services for RTP youth. Some between this and the programs we described in expressed that their organizations’ full range of Table 5 on section 2. Together, the “primary” services is comparable to those of the “primary” and “secondary” service designations, combined partners, even if they are not utilized by the with vagueness of actual RTP-designated services, network. Some of them view their contributions raises the question of how RTP services should as passive, waiting for RTP youth and care best be determined. managers to engage their services, even when they know of high demand.

Table 7: Referral of RTP clients FY2015-16 Also, HOMEY staff noted that it is not always clear whether the youths they are serving are RTP or not. RTP partners request their safe passage service for both RTP and non-RTP youths. Administratively, this compromises the network’s efforts to accurately track services provided.

Most of the partners’ service offerings to RTP represent a subset of their respective organization’s total services.

Even so, some respondents noted that RTP youth could still receive their organizations’ regular services that are not specifically designated for RTP,

Source: Researcher’s calculations from CMS data.

21

3.5 Data collection and maintenance 22

From the initial referral to RTP, to the completion of services, a plethora of records document the activities and progress of RTP youths. These include: • the RTP global intake; • the referral intakes of the RTP partner organizations; • psychosocial assessment and risk assessment: to identify needs such as housing, legal and immigration services, food, clothing, etc.; to gauge readiness for treatment; • forms to enroll in insurance for medical services, and to gain parental consent for minors; • case files • individual service plans • the “client management system” or CMS database (the accountability data of DCYF) Of these, the assessments are ideally repeated every 3 months to track progress, and the case files are routinely updated to narratively describe that progress. CMS data are updated to track activities and units of service. Service partners generally accept the need to collect these data, and understand their utility to service provision and accountability. “If it’s not documented, it never happened,” acknowledged one care manager. There is also general agreement on the purposes of different data sets: • In-house case files are the records that truly track the progress of RTP youth, particularly in the progress notes; they are subject to subpoenas, so they must be a very precise and accurate; • Data entered into CMS mostly focus on activity levels, not youth progress.

That said, one respondent noted that CMS is advantageous for care managers operating at multiple sites, because it can be accessed from anywhere. So he enters his case notes in CMS

(which isn’t required), rather than keep separate case files. Still, he notes CMS won’t cover other information that some of the partners need, like criminal record.

The discussions on data maintenance identified several specific challenges that RTP partners face.

First, the proliferation of forms is frustrating to some. “The damn thing has just gotten so complicated, and I feel like developing joint tools is important [like] common intake forms.” Relatedly, the high demand for data collection has led to inconsistencies in assessments and data entry.

One respondent described a 2-hour assessment as too demanding for the youth and care manager to be implemented regularly. He sometimes skips it. Another noted that undocumented youths can be placed at risk by some of the requested information (e.g., employment), so some care managers may be intentionally vague to protect their youths. Some care managers have developed their own personal case files (e.g., on Google Drive), on top of the organizational case files and CMS, to keep personal notes not entered in official records, and to have mobile access to those notes. Lastly, some respondents noted specific shortcomings of CMS: it is not set up for Safe Passage services and its multiple trips for clients, and it is not clear about dates to enter (e.g., referral date or actual service).

The CMS data is also critical to distribute the funding. The grant from DCYF provides funding on a reimbursement basis, meaning that organizations cover their expenses ahead of time, enter services provided into the CMS database and provide evidence of the expenses to IFR – canceled checks, receipts, etc. There were a number of comments in the interviews about the cumbersome data and reimbursement process.

If we don’t provide adequate information, everybody else is affected, and that can be extremely challenging for small organizations because they don’t have the working capital to put those dollars

in advance. They’re waiting for Instituto to reimburse, and if I didn’t submit the information correctly, the check is not going to be forthcoming for Instituto because DCYF is going to have questions on one invoice.

Multiple respondents expressed a need to access real-time data on service utilization by RTP youths, to coordinate service delivery more efficiently, and to better track progress. The RTP service connector wants to address this need by developing an information management system— including a database and website—that allows partners to seamlessly share meaningful data while satisfying CMS data requirements.

means of communicating that a partner is actively participating in the network. There is a tension between data entry as a demonstration of collaboration and responding to centralized monitoring.

No, it’s not about monitoring you, because it’s not just about you, it’s about the whole initiative. I have to think about the initiative. You think about yourselves, I can’t. We’re trying to build initiative so we think about each other, because otherwise we should just get funded individually.

What makes this initiative special if we can’t work together?

Service providers and Core staff expressed consistent views that measurement is about the accountability of partners to the whole Service Network, initiative partners more broadly, and ultimately the youth served. A great deal of effort has been made during the pilot year of the RTP Service Network to set up and begin using personto-person and technology systems to track and measure activities of Service Network partners. In interviews providers recognize that while Core staff and leadership are developing the skills, credibility and trust to tread carefully to achieve a centralized accountability, this is also challenging for the network as a whole.

We’re still building trust because what we also are doing here that’s very unique is building collective accountability. And people were not used to having to be accountable to each other. It’s like you did your thing. I did mine.

In some cases where communication between providers and the Core is minimal, as can happen when staffing problems arise at a partner organization and staff are unable to regularly attend Service Network or Care Development meetings, digital data entry is the primary

The primary tools for shared data entry, analysis and sharing for the RTP Service Network are the RTP’s Global Intake Form and the SF Department of Children Youth and Families’ CMS. Individual partner organizations share their screening, service planning and assessment tools, but to date there is no global RTP Service Network assessment for youth progress and individual outcomes. Data on youth demographics and service activities are entered by RTP service providers at each service interaction with youth.

Service providers and Core staff expressed the

23

24

importance of Service Network meetings for data reporting (monthly from CMS) and explaining, training and troubleshooting on the data systems and developing new protocols - like the regular submission of narratives about the experience of RTP youth - that deepen RTP’s ability to know itself and grow based on that commonly held knowledge.

The hope at RTP is that the existing digital and person-to-person systems for data collection, analysis and sharing will be developed to be easy to use, relevant and useful to all partners in the Service Network and the Initiative

“backbone”. Privacy provisions and guidelines are in place in the Service Network and Care Development Work Group, and in the Core as well, but reinforcement through training of those rules should accompany the development of any new data tool. Serious concerns about making the youth more vulnerable to punitive public policies limits options for data usage and sharing, so safety concerns pose significant constraints on the systems that will work for an initiative building trust with the politically vulnerable youth they are serving.

3.6 Service utilization The total RTP youth served was estimated at that time of the interview at about 80, putting RTP on target for its enrollments overall. The primary challenge, as previously described, is the process and sequence of service utilization within the network. BACR’s employment services are highly desired, while it is more challenging to engage youths in Horizon’s

substance abuse services. Also, while 80 youth were enrolled in RTP at that time, not all were actively engaged in services at the same time.

One respondent estimated that perhaps only 20 or so youths are actively engaged at any given time. Underutilized RTP services may be idle, or delivered to non-RTP clients.

Table 8: Respondents’ description of services provided to RTP youth

Source: Researcher’s calculations from CMS data.

3.7 Beyond connection Analysis of interviews showed the importance of two closely related conditions of the collective impact approach: continuous communication and mutually reinforcing practices. Together they move RTP services from connected to networked. Continuous communication across the service organization partners was often mentioned in relation to the time and effort it requires and the ultimate benefits it offers, ranging from simple logistics to strategic development and implementation of services that are mutually reinforcing.

Continuous communication was also noted as key to attempting to provide timely services and coordination between primary and secondary providers.

Logistical communications are the purview

of the core staff, and are relied upon for the efficient coordination of the many meetings required to set up and run the service network.

Interview participants emphasized how just “showing up” to meetings and events is critical to ensuring open lines of communication between service partners. Showing up sometimes requires overcoming a mindset in which all providers are not equally important.

This is a challenge interviewees acknowledge and aim to address by continuing to value and practice open and direct communication, even when it’s challenging to do so.

So now we are all a part of the club.

We’re all part of this. In order for us to serve these youth well, we have to communicate. That’s an issue, still.

25

26

Given the level of coordination necessary to build the network, it is not surprising that one of the key concerns of managers was the potential overburdening of staff. Extensive coordination and communication requires significant staff time. In one case, a program was also concerned about potential disconnection of the staff member from the rest of the organization as the staff member focuses exclusively on RTP. The frequent communication necessary to keep the network cohesive can also exhaust committed providers that are serving a lot of clients. One interviewee suggested RTP core staff should communicate more streamlined and targeted information, as opposed to blanket emails that go to everyone in the network.

Communication facilitates the connectedness of the service network, which is key to delivering expert care to RTP youth. One interview participant put it succinctly: “We’re each doing our work, but we’re [also] connecting it.” Yet core staff expressed concern that RTP youth do not realize they are in a holistic, wraparound network different from standard service provision. Front line service providers seem to lack sufficient information about the work of their peers in other parts of the network to achieve connection and even further, mutual reinforcement. The complicated process of setting up a referral and care management system and identifying slots and/or availability of programs has made it difficult to refine RTP activities to the point that they mutually reinforce one another. Making these processes more coordinated and efficient will help make RTP collaborators a network.

The problem is that’s where the knowledge spectrum and perception and myth come in because I think a lot of people are perceiving it as like, ‘Well you’re not taking my referral. This is not meeting my need….’ We (as leaders of RTP) have to message that this is not about people’s

lack of willingness or lack of disposition.

It’s really a design … we’re learning. We over-planned this. We wanted to plan for everything before we ever tested it.

Given the many types of stakeholders and competing demands on their time and attention, effective, reliable, targeted and efficient communications practices are seen as vital to well-coordinated, mutually reinforcing, service provision. It takes time and trust by consistently communicating program priorities, reliable data findings, agreement among care managers to a plan for a challenging youth to ensure that partner activities in one sector reinforce activities of partners in other sectors.

Shared measures across the network - still in development in RTP - will also contribute to mutual reinforcement.

RTP’s “backbone” is set up to support the collective impact condition of substantive mutual reinforcement, rather than just connection. When decisions need to be made or new information incorporated into the service network’s practice, it can be vetted through the “backbone” structure, with core staff or other backbone members putting the matter on a Work Group agenda and passing it from one Work Group to another and/or to the Steering Committee for input and reinforcement. This structure runs the risk of being too complex and inefficient, however.

“I think that we’ve gone a little too… program wonky. We have too many charts. It’s just got to be way more simple.

It’s complex, but it has to be translated into a more simple approach because… it doesn’t resonate… It has to be so simple that you can remember. It’s that elevator speech. We need to know ‘what’s the point of entry,’ ‘what’s the process that the youth navigate,’ and ‘where do they end?’ It has to be that simple… We have a design challenge.”

A noteworthy method of mutual reinforcement mentioned by members of the service network is the explicit appreciation of best practices of individual organizational partners and the willingness of those partners to offer those practices for the initiative as a whole, to be “branded� as RTP practices. On the other hand, a service provider mentioned that her organization was refraining from modifying a program because staff there felt RTP was not ready for that change. In this case, the need for mutual reinforcement in a bigger network of partners was deemed too cumbersome and has slowed down programmatic innovation. Over time the backbone and service partners will

develop processes that promote and measure mutual reinforcement without being overly complex and burdensome.

Some interview participants mentioned that someday the service network might expand to include new partners, though all agreed that RTP is not ready to take that step16. Lessons learned by the backbone in this pilot year about the structural and practical challenges of achieving mutually reinforcing activities should inform decision-making about the timing of service network expansion as well as orientation and integration of new network partners.

Even though they are not not ready to fund new organizations to join the service network, they are working to build partnerships with other organizations to expand the services and programs available to youth.

16

27

3.8 Successes and challenges in service provision 28

When asked about the network’s care management so far, two distinct narratives surfaced. On one hand many respondents had very positive perceptions, particularly among the primary partners themselves. All of them reported strong relationships of trust built with the RTP youth, as well as with the other primary service providers. The word “family” was frequently used in this discussion. “What’s good is that everybody that’s dedicated to Roadmap to Peace is like a family. And [the youth] feel that support,” said one program manager. Another said the service providers “now work closer together so that, say for example, we have a youth who needs assistance, there is no waiting list. We can call each other, like we’re already friends. Not just colleagues, but family. That’s how it feels. We’ve got your back, we support, we have a slot.”

One credited IFR, and Dr. Estela García in particular, for setting “a tone of respect and love and accountability” among the partners.

And early perceptions of the outcomes of this familial network are positive.

Youths are applying for jobs, having their felonies reduced to misdemeanors, and not reoffending. Five Keys noted that their incarcerated RTP youths are doing particularly well with their case plans, in part because their time is easier to manage while incarcerated.

On the other hand, the perceptions of care management from the secondary partners were more varied. The previously mentioned inefficiencies in how youths are connected to secondary services were echoed in this discussion. Some secondary service providers feel underutilized and

underappreciated by the network. From the perspective of a primary service provider, it is challenging to get the youths to follow through with their service plans. Clients can be overwhelmed by the processes and services prescribed. There is often initial excitement that slowly erodes over the weeks. As one service provider described, The division of perceived effectiveness of care management was echoed in the perceived effectiveness of secondary services. In general, care managers perceived greater effectiveness, while secondary service providers perceived greater challenges.

Also the time commitment for some services is substantial, and perhaps conflicting with obligations that the youths have to work and family. “We have to be realistic about the things we have them do,” said one care manager. Services have to be prescribed, but also self-driven by the youths. “We can’t be gatekeepers of services.” Again, respondents mentioned that it is particularly difficult to get youths to commit to services for mental health and substance abuse. To overcome this barrier, one care manager uses the illustration of the RTP service wheel to discuss holistic health with the youths, which helps them to see all that they have to address.

Another described a problem with care management capacity. A requirement of “intensive case management” is to meet with each client at least 3 hours per week. For a care manager, that means no more than 10 clients at a time, totaling 30 hours per week (while allowing for 10 hours per week for case conferencing, administrative duties, etc.). Yet, some care managers already exceed 10 clients.

Respondents also described challenging populations that require further attention by the network. One noted that undocumented RTP youths are difficult to employ, and the network does not yet have a strategy for addressing that circumstance. Five Keys identified some special challenges that incarcerated youth present to RTP. Youths transitioning out of incarceration sometimes drop out of touch for 1 to 6 months because of homelessness, and youths with pending cases (e.g., unsentenced, fighting cases) are not well served because of their temporal incarceration. Mission Neighborhood Center noted the lack of fathers, and others with parental roles, being enrolled in RTP, relative to the known demand from that population.

Lastly, one manager cautioned that care managers can “over-identify” with their youths and their families, feeling responsible for them. Healthy boundaries need to be maintained in this profession. This was reinforced by a theme that reappeared about the wellbeing of care providers. Divisions between the Norteños and Sureños was

mentioned a number of times in the interviews as impacting the service network.

It particularly impacted access for youth to services, as it is often difficult for some youth to safely move throughout the city, highlighting the importance of providing safe passage as a service in the network. It also highlights, however, how the providers of services are often from the same communities as the RTP youth, and thus must live and deal in the same environments as their youth.

Providers often mentioned being able to be there for the youth, while balancing their own life and knowing the balance.

The RTP network is working on articulating the RTP care manager approach to creating more uniformity on care management throughout the network and to use to onboard new care managers.

29

30

You don’t do everything for them. You’re not the one to deal with this or deal with that. You’re a good care manager. You know the different places that will do these things very well. Your role is to connect them to that – and to connect them very carefully, very – the warm handoff and all that, and support them a little through that at first until they feel comfortable with that person. Your role is to really connect them to that. Then you’re building their circle of care.

4 The RTP Holistic Model of Care RTP is a collective impact initiative with a holistic model of care. This philosophical orientation to violence prevention is informed by a collective impact approach to youth violence that is built upon restorative practice, trauma-recovery, and youth informed services delivered in a timely manner. In order to move towards a common agenda, however, RTP participants have had to learn and adapt as the network has developed. Examples include: moving from “on-demand” to “timely” services; agreement not to use the term “gang” when describing the services, use of terms like “victims” or “perpetrators” (from the youth perspective, they are intertwined) to “violence-touched.” These shifts reflect the philosophical commitment of the network towards collective impact and a holistic model of care. As one interviewee noted, this flips the mainstream language about Latinx youth: What we did is we turned the concept around. Instead of saying right now the concept is those youth are dangerous to the public, we said we want to protect the safety of those youth. That’s a very different position to take. That’s not a suppression. At all. That is about understanding the vulnerability of these kids. Because they’re often extremely exploited...And the fact that they end up with a gun in their hand is not what anybody wants.

It’s not at all what we want. But if we don’t attend to their safety, and their protective factors, we will never have them drop that gun. They will not drop that gun. That’s the only thing that [can make that happen]...that there’s a semblance of safety for them.

In this section, we assess directly how the RTP holistic model of care has impacted the initiative and service network.

Model adoption is proceeding amongst the service providers in the RTP network, with varying levels of saturation of these core concepts throughout the network. Model components that preexisted in organizations are also the places that are most fully using the philosophy; agencies with these models less integrated are eager to adopt more fully, but have had some challenges in adapting the model to their own work.

One other important finding was that the model principles of restorative justice and trauma-informed were seen not only as something important to provide to youth in the RTP network, but also was utilized in collective governance of the network.

For instance, some respondents noted that the importance of trauma-informed was something directed not just at RTP youth, but also at the RTP collective in general as a mechanism for self-care. Importantly, many in the RTP network have experienced trauma themselves, and thus, saw trauma informed as a way of recognizing how one’s own trauma impacts their work. Restorative practice was also used in resolving conflicts amongst service providers that inevitably arise in collaborative work.

Traditional Medicine Wheel symbol

31

32 30

4.1 Adopting Collective Impact Service providers, core staff and initiative leaders were asked to describe how RTP is beginning to operationalize the collective impact model.

Their responses to this direct query about collective impact model adoption were analyzed using the definition of the “Five Key Conditions for Shared Success” in Table 2. Our analysis assesses how the RTP Collective Impact model is understood and implemented by RTP partners and how the groundwork is being laid for the operationalization of the conditions of collective impact.

“Collective Impact” is a phrase that is often discussed in strategy and program development meetings by RTP leaders, managers and core staff. In the formative evaluation interviews we heard the following articulations of what “Collective Impact” means to them, but the most enthusiastic voices about the model were those of the executive directors. While technical terms associated with collective impact hold significant meaning to the leadership and program planners, collective Impact is understood and

articulated by the front line providers as a uniquely strong and united community effort.

Collective impact is...a methodology...

It’s how do you imbue [the work] into a system. Whereas the thinking of some of my colleagues is we just want a strong community – we want a strong community effort. They go hand in hand. I don’t see a contradiction.

Using the “Collective Impact” model has allowed RTP leaders to organize a potentially impactful network in the context of a system they experience as segmenting and alienating. Doing so has meant translating abstract concepts into practical terms providers can unite around.

How could we come together and put our minds and our thinking into the needs of these young people? And who are these young people? And what are they faced with? And what do they have to do? So there was like a thinking about this collective impact process. And I don’t

know if everybody even conceptualized it that way, and it doesn’t really matter, it’s really...I think of it...It’s community.

While they don’t use the term “collective impact” to describe their RTP work, most of RTP’s front line providers say they think about and discuss RTP’s collective impact elements, mentioning that referrals must “make sense to youth” (common agenda) and that intake forms and assessments reflect all RTP services (mutually reinforcing activities), that care providers, their supervisors and core staff have the same understanding of RTP aims and procedures (continuous communication and backbone function), in practical terms with colleagues and with youth. Front line staff did speak about the implementation of collective impact activities, but did not apply the “collective impact” label to the work.

Developing a common agenda has been the most fundamental condition of collective impact RTP has addressed in the pilot year. Partners are overwhelmingly supportive of the goals and model of RTP, but are still working together to build trust and learn from one another what those goals are, what it takes to achieve them, and how to measure those achievements. RTP’s common agenda is most formally codified in the

Articles of Collaboration and MOUs between partner organizations. Interview participants mentioned that the following elements have been key to laying the groundwork to support a common agenda: • Common connections to and legitimacy with community served; • A demonstrated history of organizational and individual commitment to the community and the population served; • Willingness of individual staff and organizations to make an extraordinary time commitment to the initiative; • Cultural and linguistic competence of individuals and organizations; • Commitment of the community to help the community; • Familiarity of partner agency staff to one another; • Legitimacy of partner agency staff to community served and to one another; and • Humility and willingness of staff to learn from one another

4.2 Restorative justice Restorative justice was the model component with the greatest consistency amongst service providers.

When asked what restorative justice meant, fifteen different respondents (50%) defined restorative justice in opposition to traditional, punitive criminal justice practice—i.e. police, courts, juvenile justice, and prison. Adopting a restorative justice framework is clearly, amongst service providers, necessary to confronting what many saw as the dehumanizing logics of the traditional criminal court process. As one respondent put it:

So restorative justice, as I have began to learn it, really – and, actually, what I know about it is that it’s actually an indigenous, very indigenously based philosophy. And it has to do with the idea of, you know, when people do wrong in a community, or in a family, or in a society, they have to make amends. Okay?

But the questions is: Do they have to make amends in a cell? You know? Is the answer that you are incarcerated for 20 freaking years? Or that you never get to see your family

31 33

34

again? Or that your family never knows that you’re sorry as hell? Or that you’re...you’ve learned something of this terrible tragedy that has come upon you and others, and that you take responsibility? And so...So there is alternative, and more compassionate, and more just ways for people to deal with injury.

Whether that’s a personal injury, or that’s a public offence.

Restorative practice as in opposition to traditional criminal justice system was the most common answer to the question of how restorative practice shapes the RTP network. However, as a definition that is opposition to traditional court practices, it does not illuminate exactly what restorative justice is in practice.

In terms of how restorative justice is actually practiced, there was variation amongst RTP service providers. Some, like the quote above, defined it as a culturally specific or indigenous practice (6 respondents). Others defined it as restoring the relationship between an individual and community, or “balancing” community and individual needs, as one respondent put it (14 respondents). As one respondent noted, in their organization “When we say restorative, we’re saying community.” Respondents defining restorative justice in the affirmative typically described components of restorative justice that are found throughout the RJ literature: mediation (4), restoring harm (2), creating relationships that are youth-centered (4), and individual accountability (3). Others defined restorative justice as youth conference meetings (1), youth voice (3), and finding a solution to the problem (1). Care managers did not have answers that were distinct from other organizational roles, but did discuss how care management often meant taking youth

centered approaches.

Of particular note was the number of respondents whose embrace of restorative justice came from the opportunity for training by the RTP network. Five respondents noted that it was because of RTP training that they saw their work as connected to the issues of restorative justice, and two respondents noted that they would like more opportunity for training and enhance their skills in this area. A handful of individuals did indicate that RTP adoption of restorative justice did not change their approach to the work, as their organizations had already embraced restorative practice. This was the particularly the case for those organizations (e.g. Five Keys, BACR) that already practice restorative approaches.

Despite congruence on defining restorative justice in opposition to punitive structures, there was some struggle amongst respondents to define exactly how it was implemented in the RTP process. Five respondents noted that restorative practices were not yet fully implemented in the RTP process, although 3 of these respondents came from the core staff interviews. This likely suggests that while individuals may have an understanding of what restorative justice is, and some even verbalize how this impacts their practice (e.g. burning sage, practicing healing circles), there continues to be struggle on exactly how the initiative itself is restorative.

Importantly, it is the purposeful infusion of indigenous cultural practices across service areas that serves as a link across the service network, and those practices were identified as part of restorative practice. As a Core staff member put it, quoting a colleague, “That’s the secret sauce of RTP. The culture piece is what makes RTP so unique.”

There are different forms of culture, that can be like a protective factor. Because I think a lot of them were lost...It’s like you almost don’t have an identity so you turn to somebody else to tell you who you are, what group you’re supposed to belong to, or something like that. So I think bringing in that culture like, ‘Oh, I belong to something. I have a family, you know like generations of ancestors that have my back.’

Bolstering the connection between restorative practice language and the indigenous based cultural practices of the network may be an important step in defining the specific restorative practices of the RTP network.

35

4.3 Youth-informed 36

RTP strives to create service plans and policy solutions that are youth-informed. In pursuit of this goal, RTP currently provides several places where youth directly influence the RTP process. First, four youth sit on the RTP steering committee. Second, RTP employs a transitional-age youth (TAY) advocate in its administrative structure (Ashley Rodriguez).

Third, RTP holds regular “Youth Development” meetings, which are convened by the youth advocate (a youth herself) and attended by youth steering committee members. These meeting are an opportunity for youth to meet outside the confines of adult-space and consider the development and practice of RTP. Finally, RTP has created a set of youth called “RTP Warriors” who will be directing youth development activities, creating a youth summit, and spearheading a youth-led participatory evaluation project.

All respondents indicated support for the youth-informed philosophy of RTP, but there was variation about what this actually meant in practice. Two definitions emerged. First, several respondents (8) defined youth informed as youth providing information to service providers about their lives or RTP as being influenced and informed by youth, but not led. As example, one respondent answered in response to how youth informed manifests in their organization the relationship between youth informed as fostering resiliency: Being we sort of hold ideas, like we believe part of it is the resiliency model. Everyone has problems, but also everyone has strengths. And so you sort of need to listen and understand where they’re coming from. And build your delivery around young people giving you the information to understand where they’re at and where they want to go.

This included such things as reporting trauma, medical history, perceptions of needs, and reporting other challenges to the RTP network.

Here, 4 different respondents defined youthinformed as meeting youth needs.

The second definition of youth-informed that emerged emphasized greater leadership opportunities for youth. This included youth leadership over service plans and agency goals, with some noting that youth played roles determining content and structure of agency planning. One responded who echoed this view replied to the question of what youth informed meant with: Youth-informed. So for us, it’s about looking at having youth voice and really looking at not getting bogged down with what we think is right. A lot of times, as adults, we’re great about being visionaries and thinking that we understand trends. It’s important to look at all that information that’s generated, but also taking youth voice and asking them, is this true, or what parts of this are true, or how does this play out in your community?

How does this information play out in your neighborhood? How does this information play out within yourself? So I think youth informed is about that, and I think that we’re doing a great job of writing that. It’s not shying away from our expertise but really using youth voice to really check in on the expertise Respondents noted that youth led meant youth had directions over agency goals and programs (6), that part of service provision was empowering leaders (3), and that youth should be viewed as experts on their own lives (1).

There was some variation between these two different definitions, with some noting that youth should have direction over their service plans (7), but 5 of these respondents did not go as far to suggest that youth should have control over agency goals. For instance, one respondent noted this hybrid definition as:

Youth-informed for us is to talk with the youth and see where their priorities are, and see what they want to do. And then we fit it into what we have in terms of a structure. But we always let the youth lead.

And so they lead discussions, they lead all the discussions around what we do. We make it into a partnership. There’s no staff verses youth, it’s not like that, it’s equal.

Variance in this definition again fell along agency lines, with organizations that practice youth-led service provisions prior to joining the RTP network (CHALK, Five Keys) having the most far-reaching definitions. In both of these organizations, members discussed allowing youth to shape their experience with the service, and tailoring the services to the youth-identified needs.

Several respondents, all of whom embraced the youth-led definition, reported that RTP could bolster its use of youth voices. One respondent remarked that youth were “tokenized” in the RTP process, and several others noted that more could be done to integrate youth. Some respondents also questioned the extent to which RTP could be youth led. As one respondent noted, it was quite hard to include youth in custody and thus even if RTP included youth in the process, it was a select group of youth. Another respondent noted that youth led actually conflicted with the legal construction of the family, where parental consent was often necessary for youth intervention services.

Based on the totality of responses, RTP does practice the prevailing definition of youthinformed but does not rise to the level of youth-led that is practiced in many partner organizations. Given that several respondents noted that RTP could do more to incorporate youth voice, the network may want to consider other ways for this to happen. Even further, the productive tension around the definition revealed some important fault lines that may be ameliorated with greater dialogue about when and how it’s appropriate to incorporate youth into the RTP process. In particular, RTP may want to

decide where youth informed is most robust, and when it should be relegated to the least expansive definition. For instance, RTP may want to promote youth-led policy development, but only youth informed service provision.

In particular, the role of adult expertise and its place in the network should be considered, as at least one respondent felt that by increasing youth voice, the role of adult expertise is sidelined. At least one member of the network also wants RTP to be a place where youth find it okay to

challenge adults, so the network might also consider how to build in provisions to account for the unique ways that adults can foster this ability amongst youth. This may be a place where those with greater experience integrating youth into service provision could inform the network about strategies that both enable greater youth participation but do not challenge the role of adults and community leaders in the process.

To improve effectiveness, core staff suggested a need to focus more on engaging the RTP youth, rather than just serving them. They plan to develop programming (RTP Warriors) that entices fuller participation in RTP. A better branding and marketing plan was also suggested for fuller engagement of RTP youth in all the network’s services.

37

4.4 Trauma-informed / Recovery 38

In contrast to restorative justice and youthinformed, trauma-informed was the least saturated of the philosophical components.

When asked to define trauma-informed and how it impacted their work, 4 different respondents stated outright that they did not know what it was and/or were unclear about how it impacted their work. Importantly, all four of these respondents were care managers with the greatest connection to youth in the RTP process. This lack of clarity is evidenced in the following response: I’m not too sure if it’s exactly what the question is asking. But we have suicide trainings, for youth too, to be able to pick on these certain things. Maybe that’s trauma-informed training. I understand when someone is having a traumatic event or going through some sort of stressful situation. So we’ll teach them how to point out someone’s being suicidal… So these are some of the trainings that we put some of our – not just RTP kids – through but all the youth that go through our doors…

That also used to be our suicide hotline.

People would call, and our youth would be the ones picking up the phones and asking questions.

The lack of understanding about how traumainformed impacted RTP service provision directly also meant that not a single respondent noted that they had changed their service provision in response to the trauma-informed philosophy of RTP. Some service providers were certainly more aware of it as a result of RTP, but that awareness had not permeated through to their practice. Respondents who did practice it with RTP youth came from two organizations that were more closely related to health and counseling: IFR and MNHC. Despite lack of consistency in the definition and practice of trauma-informed care, many respondents did note that trauma-informed had permeated amongst the service network.

For instance, six respondents defined traumainformed as service providers practicing self-care.

In recounting a training on trauma-informed, one respondent recalled:

I know we were talking about healthy boundaries. They talked about how it affects relationships. I’ll give you an example. Let’s say a youth calls you at 8:00 or 10:00. They say you shouldn’t give out your personal number and this stuff. Sometimes – they’re not saying about us, but in the past there have been people who had relationship problems because of that. There have been people who were kept away from their families because they were too busy, always working, with work and stuff – you’re never – you need self-care.

One additional respondent noted that knowing one’s limitations was a trauma-informed approach.

While trauma informed may not have permeated down to the service network, it certainly was something embraced by RTP service providers as a useful construct for thinking about their own trauma.

When trauma-informed was defined, its definition varied. Two definitions of trauma-informed were dominant: trauma-informed as recognizing

the multiple traumatic experiences of youth (6) and trauma informed as cultural competency (5). Other definitions included trauma-informed as crisis management (1), warm handoffs (1), providing safety (2), and as oriented toward youth strengths (1). One respondent also questioned whether trauma-informed could be used in all service provision. Two respondents also noted that they supported trauma-informed approaches, but that this approach was primarily limited to mental health specialists.

Given the variance in defining trauma-informed care, and the lack of permeation to practice in the RTP network, trauma-informed seems an important place to develop additional trainings.

In particular, these trainings should not only provide an introduction to the practice of traumainformed, but should also articulate how this philosophical approach can be embedded within other types of service provision than mental health. Employment development in particular is one place where a potential conflict between service provision and trauma-informed approach was noted.

4.5 Timely service provision Over the course of the evaluation, RTP transitioned from using the phrase “on-demand” to “timely” to describe the time period between initial referral and RTP service provision. However, at this point, the bulk of service provider interviews were already completed. Below, most of the results refer to the phrase “on demand” and how it is practiced within the RTP network. Even with this transition, the results below again demonstrate considerable variance in how this was defined, suggesting that greater clarity is needed about what timely service provision means from the perspective of different service providers.

Despite the transition to the language of “timely” service provision, RTP respondents expressed enthusiasm for quicker connection of youth to services. As one respondent put it, “I think on-

demand is what sets us apart right now, I think, from other initiatives or other programs.” Another respondent described on demand as: Like right away, which we have access to.

So if you’re an RTP client, you have access to on demand mental health services. You don’t have to wait on their wait list at IFR. You have priority. If you want to get a tattoo removed, you’re a former gang member, or you tattooed a girl’s name on you or a guy’s name, you can go to Carecen and get it removed. If you’re an RTP client, you don’t have to get on a waiting list. You will be seen relatively soon. If you need health concerns, we have a clinic through Mission Neighborhood Health Center where they have priority to go there. They have Rachel

39

40

who’s there. We’ll go meet with Rachel, and she’ll be able to do everything. I just got a client Medi-Cal last week. We also offer safe passage rides from HOMEY. If they need to get home and it’s 9:00 at night they’ll go and pick them up and bring them home. Those are all on demand services.

Three different respondents did question what “on-demand” meant, with one noting that likely confusion about what it meant amongst service providers stemmed from “miscommunication.”

This miscommunication may have arisen from the wide variance in what service providers expressed as “on-demand”: a time period ranging from “immediate” to up to 4 weeks.

5 respondents described “on-demand” as immediate, or within 24 hours; 3 described it as within 48 hours, and 5 respondents noted that the care management process and health care processing both introduced delays into service provision that could be as long as a month.

Further, others described it as “no waitlist” (4),

as being there for youth (1), and as complicated by particular inattentive service providers (1).

This variance speaks to the need to communicate amongst service providers the difference in demands for particular services. For instance, HOMEY noted that Safe Passage can be provided on-demand, and this service seems appropriate for that. However, some care managers noted that it can take upwards of 2-4 weeks to fully integrate a client into services, which makes on demand less apropos for this type of activity.

Whether service provision is called timely or on-demand, respondents generally felt that RTP services should be available when a youth needs them. One respondent talked about on-demand as “preventative” of further violence, while another noted that it was difficult to be ondemand when what we actually need is “youth readiness.” Further, one respondent noted that, like the other philosophical approaches of RTP, on demand was in reality for the service provider, such that they could reach out to others in times of need.

5. Recommendations for RTP service network RTP is an evolving initiative that is adjusting as they learn about issues. They are already responding to many of the challenges we described in this report. For example, they eliminated the use of primary and secondary providers, responding to complaints from network partners. They are developing an orientation for RTP youth to help youth understand they are part of an entire network and to increase the connection of youth with all the service providers, hopefully easing some of the bottle neck in supportive services.

Our analyses above identified the successes

and challenges of the RTP service network. These challenges lead to a set of recommendations. Moreover, providers made a number of recommendations. We gave priority to recommendations made by multiple respondents and clustered the recommendations around four major themes: expand network and organizational capacities, prioritize and address youth service gaps, prioritize and implement personnel training, and improve information and communication systems. Each of these themes include several specific recommendations.

5.1 Expand network and organizational capacities hese recommendations aim to increase the capacities of the network itself as well as the individual partner organizations, in order to improve services while preparing for growing RTP enrollment in coming years.

Deepen the engagement of community partners, including those with government agencies and businesses.

Government agencies, community based organizations, and neighborhood businesses should not only be referring agencies, but also vested partners in RTP’s efforts to deliver services, reform policies, and develop neighborhood capacities. RTP needs to directly involve and engage these stakeholders to address systemic issues that may have contributed to RTP youths’ system-involvement or created barriers to their success. For example, RTP may need business partners to offer paths to meaningful employment to disadvantaged youth. It might also need school partners to develop educational supports tailored toward youths coming from a history of violence. Expanding and deepening engagement with government agencies and businesses is consistent with a collective impact approach to building a common agenda.

There is already widespread recognition in the RTP service network of the need to engage a wider range of types of “community” partners in the RTP initiative and they are already beginning to engage in this broader community outreach.

Consider alternatives for financial sustainability.

Other service networks have employed development staff or charged minimal fees for service. The RTP Development Working Group needs to develop a sustainability strategy for RTP. RTP should consider augmenting current funding sources with, flexible planning/infrastructure grants as well as outcomesbased funding for collective impact. This recommendation falls under the administrative “backbone” function of mobilizing funding mechanisms that are most appropriate for collective impact.

41

42

Expand partners’ capacities for care management, and discontinue the distinctions between primary and secondary service provision.

The analysis identified stark differences in the perspectives of primary and secondary service providers, not only in the effectiveness of the service network, but also in their perceived partnerships in it. Secondary service providers sometimes felt underutilized, while services were bottlenecked with care management.17 Simultaneously, some service providers noted that the network’s capacity for care management will soon be exceeded by demand, if it has not already.

To address all these issues, consider installing and expanding care management capacity with all eight service network partners. Doing so would not only help elevate the partnerships with the secondary service providers, but also help meet the demands for services in the near future. With increased enrollment in coming years, new staff hires to the network could be focused on care management with HOMEY, Horizons, MNC and MNHC. Some respondents proposed creating two tiers of RTP youth: one group with a care manager and another without a care manager, but with accesses to supporting services. But we believe it is critical to engage youth with a care manager that guides and cares for the youth while in the initiative. Others recommended making some services required to participants. For example, every RTP participant must visit MNHC for health care and Horizon for a substance abuse assessment. But making to many things mandatory may make the program unappealing to youth.

Include supervisory/managerial staff and maximize core staff functions in care manager meetings.

The interview data revealed that staff members’ understandings of the network’s structure, function and approaches to service varied by rank, with the most consistently comprehensive views coming from the immediate managers of the direct service providers. These supervisors generally understood both the intricacies of service delivery and the broader goals and approaches of the network. To ensure that the network initiatives and reforms are translated to service provision and thereby improve the administrative backbone function in the collective impact model - these supervisors should periodically participate in the care manager meetings. Likewise, core staff attendance at care manager meetings should result in more efficient and thorough implementation and communication of programmatic developments within the service network and between the service network and other areas of the broader RTP initiative.

Practice restorative justice with network partners.

Of all the approaches to service delivery that RTP has adopted, restorative justice is the most resonant among RTP partners. At the same time, the complexity of the network’s functions periodically produces friction between organizations and individuals in the partnership. For example, the perceived unequal partnerships between primary and secondary service providers was a repeated source of friction. RTP can take advantage of partners’ commitment to restorative justice by practicing it in their own partnership. RTP partners can continue to support risky, “courageous conversations” internally, and with staff of other RTP agencies, in order to strengthen community within the network, as part the administrative backbone function.

We recently learned that RTP is no longer using the “primary” and “secondary” categories, but to our knowledge they are still working on addressing the inefficiency problem and possible bottlenecks.

17

5.2 Prioritize and address youth service gaps Service providers identified a few areas of need of RTP youth that are not currently well-addressed by the network. These recommendations may be considered tentative, until they are corroborated by upcoming data from RTP youth themselves.

Deepen engagement of youth in RTP. Core staff identified a need to focus more on engaging the RTP youth, rather than just serving them. Current engagement largely consists of participation in RTP standing committees (e.g., the Youth Development Committee/ Youth Warriors and Steering Committee), but there is some concern that those roles border on tokenism because of the disproportionate power distribution in those committees (e.g., few youths and many adults). RTP should consider ways to fully engage the youths in the initiative’s operations without jeopardizing the important role of adult expertise.

Some suggestions included engagement in RTP outreach activities, branding and marketing, development of a score youth leadership group (RTP Warriors), and upcoming plans to engage youth in a youth summit and RTP’s outcome evaluations.

Develop programming for specific underserved sub-groups.

RTP is already designed to fill a public service gap for system-involved Latinx youths. Within that population, RTP is already identifying sub-groups of youths with additional service gaps.

Multiple respondents specifically identified housing support as a key need for RTP youth, particularly those coming out of incarceration.

Similarly, employment services for undocumented youths was mentioned multiple times as a critical area of need.

Respondents also requested more programming specifically for girls/young women and LGBT youths.

5.3 Prioritize and implement personnel training Service providers identified several training needs to help them better serve the youths. Some of these trainings will also help make referral process more efficient, and better engage all service partners in the network.

Cross-train in network partners’ areas of service, and clearly articulate RTP services by each partner.

Service providers did not have consistent understandings of other partners’ RTP services. This impeded care managers’ full utilization of the service network when developing service plans with the youths. This cross-training might also enhance the sense of community in the network, particularly among the secondary service partners. The objective of cross-training would be for each RTP staff member to fully understand all the services available in the network, and to improve their own services by the approaches adopted by the network (e.g., trauma-informed and restorative justice, as described further, below).

43

44

Train RTP staff in adolescent development focused on RTP youths.

Even staff who have had coursework in adolescent development requested additional human development education focused on adolescents who are system-involved, traumatized, homeless, undocumented, or otherwise matching the experiences of RTP youth.

Train RTP staff on service provision that is trauma-informed.

Complementing the above request, staff also requested training on how their service provision can be trauma-informed, with particular attention on how it applies to non-mental health services. Such training can also help care managers negotiate priority needs with presenting needs when developing service plans with their youths.

Train RTP staff in service provision advancing restorative justice.

Similarly, staff requested training on how restorative justice practices can be implemented across services, to more proactively advance restorative justice. This will also address the varied understandings of restorative justice among service providers.

Train “backbone” members in best practices according to collective impact model.

RTP aspires to deliver services and develop the initiative as a whole according to the collective impact model. That model relies on an “administrative backbone” to provide specific critical support to the network. Interview participants often referred only to administrative (Core) staff as the RTP “backbone” and cited the importance of the “backbone” for communication and building relationships. According to the collective impact literature, RTP’s “backbone” actually is comprised of the core staff, steering committee and working groups that include service providers serving in a backbone capacity in this context. Such a complex backbone structure understandably requires focused time to solidify their roles and responsibilities. Formal training on collective impact and the role of the “backbone” will improve that function for the initiative. This training would cover the role of a complex backbone in supporting and growing the Service Network and the initiative as a whole, particularly in the areas of maintaining a common agenda and developing a shared measurement system, mutually reinforcing activities, and continuous communication.

5.4 Improve information and communications systems The interview data revealed several issues of data collection, data access, and information communications that may be impeding RTP services. These recommendations aim to develop shared systems of data measurement, dissemination, and communications to support the collective impact model.

Explore the development of an online youth service information management system.

Service providers expressed the need for current information that would identify all RTP youth, track their progress and status in their service plans, and be available to all service providers. Ideally, it would also integrate with CMS reporting requirements. The RTP service connector has already begun an investigation into this kind of an information management system.

Consolidate data collection efforts for RTP. Some service providers observed a proliferation of unique forms and data collection requirements among the service providers, on top of the data that RTP collects itself. There may be opportunities to consolidate data collection for RTP youth, and lessen the data collection efforts of each partner. This will be further explored by the evaluation team as it develops a proposal for performance measurement in the coming months.

Develop specific measures related to staffing.

In the meantime, service providers also requested the development of estimated times for each partner’s services, in order to better sequence timely services. Managers also requested a measure of staff time devoted to RTP (e.g., including administrative and planning time), rather than just services provided, to more accurate capture the efforts invested.

Develop a suite of regular communications to specific audiences.

Regular updates on the status and progress of RTP services would help executive directors and steering committee members keep a pulse on the service network. Similar information could also be repackaged for external stakeholders and community members. Other forms of regular communications recommended by respondents include social media, newsletters, and network branding and marketing..

Articulate a common language for RTP. RTP is committed to using language that does not reinforce mainstream stereotypes of disconnected Latinx youth and reframes the problems they experience. Related to the above recommendation on communications, and to the earlier recommendations on training, the network should explicitly articulate its nomenclature and train RTP staff in it. For example, “care management” versus “case management”, restorative justice, and trauma-informed services should all be commonly understood by staff because they are central to the initiative’s approach. This recommendation might best be addressed in the service network manual currently under development.

45

appendices 46 34

Figure A - The Service Network

47

48

Figure B - Participant Flow

c ĂŠ s a r

e.

chĂĄvez

in st it ut e

college of ethnic studies s.f. state university