EXECUTIVE SUMMARY

EXECUTIVE SUMMARY As an organization, 4POINT0 Schools (4.0) believes in supporting community-centered education innovation, rooted in equity. Education Innovators, called fellows, receive coaching, curriculum, community and monetary support for their ventures. Over the past ten years, more than a million students have been impacted by 1,200 leaders.

In May of 2020, a partnership was established between 4.0 and the University of Delaware’s Center for Research in Education and Social Policy (CRESP) in order to take a closer look at 4.0’s impact and to help build capacity to collect and use data to inform program operations. This report summarizes findings from a series of evaluation efforts to date, where evaluation questions are nested within 4.0’s four focus areas (Talent Development, Idea Development, Ecosystem Development, and Research Development). Below is the 2021-2022 evaluation question framework.

1

Evaluation Questions Focus area 1: Talent Development – Increased leadership capacity within communities ● How often do 4.0 alumni continue applying 4.0 mindsets/skills in their professional lives? (T1) ● In what ways do fellows apply what they learned post-program? (T2) ● To what extent do fellows become leaders in the education field? (T3) Focus area 2: Idea Development – Increased identification of promising educational innovations in key focus areas through 4.0 funding and support ● What early-stage ideas have been supported by 4.0? (I1) ● How have fellow ventures contributed to education innovations? (I2) ● What can be learned from ideas that fail? (I3) ● How have early-stage ideas grown and pivoted? (I4) Focus area 3: Ecosystem Development – Increased development of promising ventures in key focus areas through facilitation of fellow-funder partnerships and alumni networks ● How does 4.0 foster networks and collaboration within the alumni network? (E1) ● How does 4.0 facilitate connections of funders to promising ventures? (E2)

1 Evaluation questions noted in italics are largely planned for evaluation in 2022-2023. 5

● How is 4.0 engaged with the broader community of education funders? (E3) Focus area 4: Research Development – Increased sharing of lessons learned and best practices for entrepreneurship in education ● What are 4.0’s key focus areas and how do they align with best practices in education? (R1) ● What are 4.0’s strengths and needs around applied data collection and evaluation skill building, for both ventures it supports and as an agency? (R2) ● What lessons were learned regarding small-scale pilot education innovations that can be shared with the broader research and education communities? (R3)

Findings The Alumni Survey, administered in 2021 and 2022, was used to address the first Talent Development evaluation question, how often do 4.0 alumni continue applying 4.0 mindsets/skills in their professional lives? (T1). Based on responses from 574 participants across the two years: • An average of 94.2% of 4.0 alumni consistently apply the mindsets, skills, and values that they learned at 4.0 in their professional lives, regardless of venture status (see Table 2). In order to answer T2, in what ways do fellows apply what they learned post-program? the evaluation team examined alumni interview data to learn about the specific ways that fellows applied what they learned. Interviews were conducted with a subset of 31 alumni in the spring of 2021, in which alumni highlighted the value of questioning assumptions and being open to feedback from peers, as well as coaches. • Alumni apply what they have learned regarding the ways that 4.0 values community input and testing assumptions. • Openness and peer feedback are highly valued skills learned as part of 4.0, and frequently applied. • Alumni appreciate and apply the “questioning of assumptions” mindset they learn as part of the 4.0 process later in the development of their ventures and leadership.

o Allowing ventures to pivot and accepting that change is good are elements of the 4.0 program that alumni highly value. One alumnus noted, “you can’t be so attached to your idea that you’re not willing to let it morph and change.” In relation to T3, to what extent do fellows become leaders in the education field? • The alumni survey revealed that the overwhelming majority of respondents (83.3%) are considered leaders in this field. • Many report that 4.0 was instrumental in their development as leaders (see Table 6). Based on these findings, it was determined that 4.0 is successful at building increased leadership capacity within communities.

The first evaluation question for focus area 2, Idea Development, aims to understand what type of early-stage ideas have been supported by 4.0. Findings from the Graduation Survey revealed that 4.0 supports a variety of ventures, with the most common type being a program or service. • Specifically, the program or service categorization made up nearly one-half of ventures between the years of 2021 and 2022 (average of 48.31%). • Furthermore, results show that 4.0 continued to have an impact during the pandemic, with nearly 6,000 participants taking part in pilot and pop-up project tests between 2019 and 2021. • Additionally, it is known from the Alumni Survey (2022) that 87.7% of ventures are continuing in some way and more than of half of them have received additional funding from a source other than 4.0, indicating that fellow’s earlystage ideas are experiencing growth after their participation in 4.0 programming. I2 focuses on education innovations and asks How have fellow ventures contributed to education innovations? • A review of the portfolio including 4.0 venture data and data from the fellows show that 4.0 ventures include a broad range of focus areas including those that are addressing innovation in increased academic and socio-emotional learning

(SEL), improved education, improved quality of life, and improved communitybased, culturally relevant educational programming. I3 asks, what can be learned from ideas that fail? This is closely related to the final evaluation question for the idea development focus area, I4, which explores a topic known as pivoting. • Only a relatively small proportion of ventures (12.3%) are no longer continuing. • Interviews with alumni (2021) teach us that ideas fail for a variety of reasons including: o funding cuts, o lack of technical expertise, and o life circumstances. • Additionally, interviews suggested the ways in which 4.0 can support alumni through transition periods, including: o creating opportunities to extend the role of coaches to ongoing advisement or, o establishing thought partners with coaches and other alumni. o During one interview an alumnus said, “we learned to pivot and we pivoted constantly,” which highlights the importance of being flexible and cultivating dynamic ventures.

While much of the evaluation for Focus Area 3: Ecosystem Development is planned for 2022-2023, the 4.0-CRESP team did begin investigation into E1, how does 4.0 foster networks and collaboration within the alumni network? • The Alumni Survey asks respondents, “How often do you keep in touch with other fellows you met or collaborated with through 4.0? And Have you participated in any of the following 4.0 Communities?” Results show that over the past year, participation in events was limited. o While 75% reported participating in the alumni network, about one-third of those participated annually or a few times a year (“rarely”).

o Participation in communities was a bit more limited, where 40% of alumni in 2022 reported some level of participation while 60% did not participate at all.

Evaluation efforts related to Focus Area 4: Research Development will continue through the next evaluation cycle. Findings to date have revealed that, in response to R1, what are 4.0’s key focus areas and how do they align with best practices in education? two key practices were found to be foundations for the 4.0 approach. • The first is design thinking, a creative process for solving problems that keeps humans at the center. • The second is the science of pivoting, which is a plan designed to test a new hypothesis in the face of failure, real or potential. Several significant activities have been undertaken to begin evaluation in relation to R2, what are 4.0’s strengths and needs around applied data collection and evaluation skill building, for both ventures it supports and as an agency? These include monitoring our RPP and the development of a logic model in concert with the program teams which reflects both evaluation objectives as well as current activities undertaken as part of fellow-programming. • Based on the Quick Check administered early this year, the overwhelming majority of those involved in the 4.0-CRESP RPP feel positively about their experience in the partnership. Specifically, 100% of respondents reported that yes, their opinions were taken into account “to a great extent” in the partnership.

Recommendations Based on these findings, the researchers from CRESP propose several recommendations for consideration by 4.0: 1. Consider an in-person alumni meeting for those who completed the program during COVID. a. Alumni from this time would like to have some in-person experience. A gathering, follow-up refresher or similar attempt to invite them to network and celebrate their accomplishments would be welcome. 9

2. Assist alumni in connecting to 4.0 in new ways and through existing channels they might not be aware of. a. If not done already, consider informing graduates before they leave the program of the alumni opportunities available to them, and sign them up at that time. Additionally, other opportunities to increase outreach to alumni communities should be explored, as alumni are mixed in terms of their familiarity with what communities exist and how they can get involved with them. b. Create additional supports for alumni around funding, which is an important reason that they would like to stay connected. Alumni would like ongoing opportunities to learn about where to find resources, how to apply for funds, and how to write grants and structure proposals. 3. Explore the ways in which 4.0 training could tailor efforts to best support entrepreneurs with similar interest areas, perhaps offering content expertise and evidence-based ideas, as well as examples of effective strategies to foster fellows’ understanding of current best practices. a. Additional needs described during the application stage may also be a framework for common instruction, such as app development and technology infrastructures. i. A venture self-assessment completed by fellows periodically throughout 4.0 participation, as well as the opportunity for alumni to pick workshops that are themed around where their idea is now, including topics such as pivoting may be of use. This could particularly support those that are re-designing ventures/pivoting. b. Furthermore, teambuilding during and after the fellowship for common venture ideas could be expanded upon. c. Finally, consider how fellows are educated about the variety of activities, theory and content options used in similar programs. For example, creating a mix of activities that include opportunities for physical activity and de-stressing, especially if programming extends the school day for youth.

4. Ascertain fellows’ incoming knowledge and utilization of mindsets and skills such as joy, anti-racism, equity and determination. This is critical so that programming is not redundant with their existing knowledge. Rather, programming should expand upon the areas where there are gaps or dive deeper into areas where they already have background knowledge. 5. Continue developing evaluation tools and measures that are as targeted and efficient as possible. a. Consider a process that occurs annually to review survey tools from the prior year. As part of this strategy, reflect on the alignment of tools with program components and priorities. Maintain consistency whenever possible and update the tools as needed for the subsequent year, eliminating questions when they are not needed. b. Furthermore, a strategic review of the graduation survey in concert with the alumni survey and data collected on fellow applications may be of use. Questions such as those that focus on participant records of their assumptions, for example, may be lengthy for respondents to complete and difficult to use in reporting.

The full report provides a detailed accounting of evaluation findings and recommendations. Researchers from CRESP are available to answer questions regarding analyses presented in this report or to assist in their interpretation. For more information, please contact Allison Karpyn at karpyn@udel.edu.