Evaluation of the Girls Inc. of Greater Philadelphia and Southern NJ Operation SMART Initiative Interim Report, April 2016
Prepared by Jessica Lucas, Evaluation Professional for XYZ Corporation participating in the Step- by- Step Evaluation of Girls Inc. Operation SMART
Abstract Girls Inc. of Greater Philadelphia and Southern New Jersey runs Operation SMART to expose and encourage girls in the fields of science, technology, engineering and math (STEM). While the three programs under the Operation SMART initiative have seen success in the form of feedback and observation, no one has run a formal, in-depth evaluation of the programs ability to achieve the proposed objectives.
Killer Paragraph The twenty first century has seen a great rise of jobs in the domain of science, technology, engineering and math (STEM). However, these lucrative fields are often dominated by male professionals as girls subtlety discouraged from pursuing STEM interests on a societal level. Girls Inc. of Greater Philadelphia and Southern New Jersey is a non-profit organization dedicated to empowering young girls in all aspects of life and their initiative, Operation SMART, focuses on STEM topics and activities. Currently the three programs running under the Operation SMART banner have used traditional evaluation tools such as surveys, feedback and observation to receive information on the programs, but has yet to run a formal, in-depth evaluation against the proposed objectives. Taking the same participatory action research (PAR) used by Chen, Weiss, & Nicholson1 in their 2010 case study of five Girls Inc. programs, I recommend applying this method to Operation SMART, and to the Built It program in particular. Built It has girls build technology such as websites and blogs. PAR train the girls in the program to perform the evaluation and the Built It platforms offer a way for the girl evaluators to execute the formal evaluation.
1 Chen, P.Y., F.L. Weiss, and H.J. Nicholson. "Girls Study Girls Inc.: Engaging Girls In Evaluation Through Participatory Action Research." American Journal Of Community Psychology 46.1 (2010): 228-237.
2
Background Girls Inc is a non-profit organization dedicated to supporting the emotional and educational growth of girls through various programs, events and clubs. The organization’s tagline “inspiring all girls to be strong, smart and bold” neatly encompasses their vision. Girls Inc. is an established program with a long history in this country going back over 100 years. It is also very transparent with many well-known sponsors and partners. The board of directors is filled with women executives and entrepreneurs. According to the 2015 Annual Report, 84% of expenses go directly to programs, which translates into $7.3 million dollars. Funding is accomplished through events and both individual and corporate sponsorship. It appears to be a financially well run organization, with a cash reserve for continued programming support 60% higher than the industry standard. One of the main focuses of Girls Inc. is a program encouraging girls in the STEM fields (science, technology, engineering and math) called Operation SMART. Women are underrepresented in all STEM industries, holding only one in five STEM jobs. Girls Inc. believes that encouraging girls in math and science will lead to more girls entering these lucrative and important fields. The key values of Operation SMART are: • Counteract the societal message that math and science or not for girls. • Encourage girls to make take risks and make mistakes in their STEMs explorations • Resist the gender-stereotypes and the idea that STEMs is not feminine. • Expect success – now and as they continue to higher education and the workforce. To achieve these key values, the regional Girls Inc. of Greater Philadelphia and Southern New Jersey offers three unique programs. They are: • STEM GEMS- Designed to introduce girls ages 9-11 to a variety of STEM fields (chemistry, engineering, biology, physics, computer science, environmental science) through hands-on activities, games and field trips. Participants work closely with Lockheed Martin employees to discover STEM as an interesting field. This program meets two Saturdays a month from 10-2PM. Activities include: oil spill experiment, designing a wind mill, archaeology dig, rubber band car, egg drop parachute, squishy buddies, bristel bot, etc.
3
•
Build IT- Designed to engage middle schools girls (6th-8th) to explore computer science/information technology. Participants work on activities centered around the Design Process. Example activities include: design a clubhouse for a specific user, developing blogs, and designing websites using html. In addition, girls learn about internet safety and how the computer communicates. Evaluation includes: pre/post tests and performance tasks throughout the unit.
•
PECO Energizing Education "NEED"- This is an energy focused curriculum designed to educate girls ages 5th-8th grades about nonrenewable/renewable energy sources, transformations, conservation and efficiency. Activities include: designing a windmill. Evaluations include: observations, feedback from students, pre/post tests.
The methodology suggested for this evaluation is similar to the participatory action research (PAR) that was used as an evaluation strategy to research five Girls Inc affiliate programs in the United States. i The researchers posited two questions: “Can Participatory Action Research (PAR) be an effective developmental strategy to involve girls in evaluation? and (2) Can PAR be an effective evaluation strategy to gain insights about and improve Girls Inc. environments?� (Chen et al., 2010). Their idea was to have the girls themselves evaluate the program based on data that indicates youth-led evaluation of youth programs yield better results (Chen et al., 2010). Participatory action research engages the objects of the research (in this case the girls) as the researchers. The girls were guided to come up with research questions, interview and collect data, analyze and then present the results. The research was conducted over the course of three years and involved five Girls Inc. affiliates. The results of the study indicated that using the PAR approach yields insightful information that can be used to plan and design future programs. There were several lessons that were reveled when conducting this study. One was that technology was a factor in keeping the girls interested. Another involved making sure that girls were paired appropriately for their skill set. A main lesson is that the girl researchers need adequate training in order – all five groups cited that the data collection proved the hardest task. Better training would help this process. This study as it suggests an optimal way to evaluate a youth-based programs. The qualitative data gathered in this type of research serves two purposes that both offer insights into the evaluation of the Girls Inc. program. The first is the data gathered by the girls as an evaluation of the program that they are participating in. This gives evaluators information on how the program is perceived by the very population it aims to help. The second is the quality or types of data the girls choose to gather or the types of research questions they ask. This can also offer information on where they may be gaps in the program, if the data is skewed toward some topics and away from others. Researchers can use this to create programs that interest the girls. The participatory action research approach provides a meaningful evaluation tool and fits in with the aims and mission of Girls Inc. programs. I suggest that the Girls Inc. of Greater Philadelphia and Southern New Jersey use this approach to evaluate their Operation Smart program. It would work particularly well for their BuildIt program, which engages 6th through 8th graders in the STEM design process and includes activities such as designing blogs and websites. The PAR evaluation can be woven into the activities as part of the BuiltIt process. For example, when they girls create a website through the program, they can include tools such as surveys on their site about the program. In this way the activity and evaluation blend into one process.
4
i Chen, P.Y., F.L. Weiss, and H.J. Nicholson. "Girls Study Girls Inc.: Engaging Girls In Evaluation Through Participatory Action Research." American Journal Of Community Psychology 46.1 (2010): 228-237.