address is 58 Victoria Embankment, Visit: www.challengeprizecentre.org London, EC4Y 0DS. Finalists

3. Evaluation of the Results Introduction

Evaluation is a core aspect of every challenge prize. In running an evaluation, we aim to share results in a transparent way, learn from evidence and engage stakeholders by allowing them to contribute in the observation and assessment of the change generated.

“We are just absolutely thrilled at the success of this project that students have been working on, and to actually have that recognition of winning it is just superb, and we’re just very excited as a school because of it.” Teacher “This was an amazing opportunity for the students. They have developed their tech skills, teamwork, working under pressure, Confidence has improved, planning and communications skills.” (Teacher)

The Longitude Explorer evaluation was run during the implementation of the challenge prize, and includes an impact assessment and a process evaluation of the prize journey. The impact assessment aimed at capturing the results of the activities against its initial objectives, to state the changes that the prizes created. The process evaluation focused on the prizes activities to understand participants’ feedback on the prize structure and whether the activities were well tailored for such participants.

The prize was designed to reach six main objectives: • Provide a practical education opportunity to improve skills such as long-term planning, as well as data collection and analysis. • Increase awareness of young people about the Internet of Things technology, its possibilities and how it can affect their lives. • Increase awareness of young people about health issues faced by people nowadays and how they can be prevented. • Engage young people in and support them to develop practical STEM skills linked to IoT to enhance their learning and practical understanding of how they can utilise the technology for social good. • Develop young people’s understanding of the relevance of IoT to entrepreneurship and UK industry. • Inspire young people to consider continuing development of their skills in technologies The overall evaluation shows that the Longitude Explorer Prize was positively received by all key stakeholders, and was able to achieve all of its objectives as well as meeting overall satisfactions with the challenge prize experience. Below are some of the key highlights of the prize: • All students (100%) enjoyed their experience of participating in LEP • The majority of finalists (80%) felt they had developed or gained STEM skills and knowledge through the Prize. • Over a quarter of finalists (36%) who participated in LEP were able to develop additional STEM skills through the Prize. • Most (92%) students believed they had increased their awareness about health issues facing people nowadays and how they can be prevented. • All teachers (100%) thought the Challenge

Prize was a suitable tool for young people, and would participate in the prize again.

“I have really enjoyed the experience and I feel like I have been given a really good opportunity to create programmes in a way I have not done before.” -Student

Methods

In order to collect the necessary data to evaluate the prize we ran three different surveys, as listed in the table below. All surveys were run in parallel of the prize main events, in order to be able to engage with the highest number of stakeholders and minimize the resources time and cost around evaluation.

When Method Used What Who we engage Respondents

Beginning Semi-structured survey Entry forms evaluation class to assess the baselines data from beneficiaries Students Teachers 64

During Semi-structured survey

End Semi-structured survey Assessment of Students workbooks - to judge the quality of their development Students Finalists 36

Assessment of the innovation, capabilities and ecosystems changes achieved via the prize. Also, assessment or prize process. Students Finalists & Teachers

Impact Assessment

Longitude Explorer Prize has been a useful tool in introducing innovation and entrepreneurial skills to young people in schools.

The evaluation proves this as both students and teachers reported very positively on the topic choices, activities and learning outcomes of the Longitude Explorer Prize journey. The evaluation is divided according to three impact categories, as the different outcomes of the prize meet a certain category: • Innovation as an innovative tool, which brings unusual skills • Capabilities is the focus on specific hard skills

• Ecosystem as serving as a strong model and inspiration to raise awareness and engage students further on relevant topic and skills.

Innovation Impact

“I feel compassionate about this project as I can see how it could have really helped me. I have had the opportunity to develop my knowledge around computing but most of all my confidence has really improved”- Student The innovation impact in this challenge is seen through providing a practical education opportunity for students. In meeting this objective, 81% of students thought the prize had provided this opportunity to improve skills such as long-term planning, as well as data collection and analysis. Figure 1 provides a more comprehensive picture of what skills were improved and developed.

1. Has the Longitude Explorer Prize helped you to develop following skills

Communication Planning Maths Engineering Science Computing Coding Problem Solving

Not at all Probably not Maybe Yes, a little Yes, a lot

Capabilities Impact

“I have acquired many skills in programming and technology with the Raspberry PIs. I have also learnt a lot market research and what to do with the data. In making the prototype design of the app, I invented many of the ideas behind the app as well has how we could apply them”. - Student The prize met its objective of engaging and supporting young people to develop STEM skills linked to IoT to enhance their learning and practical understanding of how they can utilise the technology for social good. It managed to engage and support 80% of students, as figure 2 displays. Also, all teachers felt their students had developed practical STEM skills which would aid them in the future.

2. Do you feel you’ve developed any additional STEM (Science, Tech, Engineering, Maths) skills by taking part? 25 respones

Not at all

Maybe Probably not

Yes, a little Yes, a lot

16% 6 32%

48%

“As a programmer, I have learnt 3 more languages which benefited me a lot and I really liked it as three of them are all really useful.” “This experience has been an amazing opportunity for me. I have learnt so much: how to code, how to create and channel my ideas to produce a result and how to work efficiently within a team, so that that work/ project/assignment gets completed. But most importantly, I have enjoyed every single minute of it.” - Student Of those who responded, 93% of students found they had developed an understanding of the relevance of IoT to entrepreneurship and UK industry, which met one of the prize objectives. Additionally, all students grasped a better understanding of what it takes to bring an idea into the market.

“The Longitude Explorer prize has really made me understand the creation process of products and the amount of work that goes into seemingly simple ideas. I previously thought that IoT, or even basic products, were fairly easy to make and produce, but by going through the process of creating a prototype and attempting to make it work, I have learned that it is much harder than previously thought. I really enjoyed learning about how IoT devices work, and all the software and thinking that goes behind it, as well as having learned some knowledge about how hardware such as an Arduino is set up.”

“I thought doing this project would really help with my college application and my future career prospects.” - Student

Ecosystem Impact

3. Has participation in the Longitude Explorer Proze helped you better understand the concept of the Internet of Things? Students judgement: Extent to which each activity was useful to learn

The topic of IoT was very well received, as 60% found it interesting, while 40% found it fascinating. This is very significant in increasing the awareness of young people about the IoT technology its possibilities and how it can affect lives. 92% of students had increase in their awareness, as figure 2 shows. All teachers felt their students had increased their awareness about the IoT through the Challenge.

Not at all

Maybe Probably not

Yes, a little Yes, a lot

“The experience has taught me a lot about the Internet of Things and I have also learnt about myself. I enjoyed learning about how to make a product that is most suited to its target market.” - Student

8%

12% 80%

96% of students felt the prize encouraged them to develop a better understanding and awareness about health issues faced by people nowadays and how they can be prevented. 71% of teachers thought students improved their awareness of health issues. “I hope to see our idea blossom to its full potential and help make children fitter and healthier, improving their quality of life, as well as showing the wide potential of the Internet of Things in all parts of life, including fitness and gaming” The Prize aimed to act as a source of inspiration for students to consider continuing the development of their skills in technologies, which 62% of students felt the Prize did for them. Some highlighted they would take STEM subjects at GCSEs or A-Levels, attend clubs or partake in competitions similar to LEP. While 71% of teachers believe their students were also inspired by the prize in continuing STEM related activities.

Evaluation of the process

Both students and teachers gave overall positive feedback on the Prize activities. The majority of students expressed positive remarks on the main Challenge activities, as shown in the below figure. In particular: • 96% of students state the Challenge allowed them to have a positive experience in working with their teams. • 92% of them said completing the workbook was a very useful activity. • 75% valued the digital element of the prize, as using IBM BlueMix proved to be an invaluable activity for them.

100%

75%

50%

25%

0%

Using IBM Bluemix

Using SODAQ Board from Digital Catapuilt Working with my team Completing the workbook Positive

Neutral

Negative

Teachers and students provided feedback on the logistical aspects of the Prize. Most notable was the issue of time in participating in the Prize, as 37% of students mentioned this.

Students had to balance working on their ideas for the Prize as well as focusing on school studies and exams. Teachers also suggested that more technical help from experts during the various Challenge stages would have been very useful, as these experts would have provided better guidance and knowledge on STEM related topics.

Following the successful delivery of Longitude Explorer Prize, we were able to reach over 200 students, all of whom reported as having enjoyed the experience and 80% felt their STEM skills had improved.

Building on the success of the first two Longitude Explorer Prizes, as well as growing interest from schools and pupils we intend to run the Prize again launching in September 2018.

This time we aim to engage more schools based across the UK (including Scotland and Northern Ireland). We will build on the existing network of interested schools and increase the number of schools participating, and will support schools to encourage more female students to take part.

In 2018, the Longitude Explorer Prize will continue to aim to inspire young people to develop their STEM skills alongside important skills such as teamwork, presenting skills and entrepreneurial skills. Crucially, the 2018 programme will continue to challenge young people to consider the ethical implications of advancing technology, and encourage them to be creative and ambitious in designing STEM innovations that have a positive impact on the world.

Objective Evaluation

Key

100% positive results From 80% to 99% positive results From 60% to 79% positive results From 40% to 59% positive results Less than 39% positive results

Objective

Key Performance Indicators Stakeholders Interviewed Who we engage Key

1. Provide a practical education opportunity to improve skills such as long-term planning, as well as data collection and analysis. Prize as a learning tool Teachers 100% of teachers considered a challenge prize to be a suitable learning tool for young people

Attracting new talents Students 48% thought the prize experience was good and 44% said it was very good.

Willingness to participate in Prize Teachers 100% said they would participate in the prize again

Interest in IoT technology Students

2. Increase awareness of young people about the Internet of Things technology, its possibilities and how it can affect their lives. Improvement in knowledge of IoT Students

Use of IoT topic to support STEM education Teachers 40% found IoT fascinating, while 40% found it interesting and 20% thought it was quite interesting

80% said they improved a lot. 12% said they improved a little and 8% said they may have improved. No negative comments were expressed

100% of teachers thought the use of IoT helped students in developing STEM skills