Impact Report - Norfolk Music Hub & NSFT
Prepared by: Damien Ribbans, Operations Director
Date: September 2021
5
Impact Measurement & Reporting
5
Overall results
5
Quantitative impact data
6
Demography
6
Attendance and completion
7
Wellbeing data
8
Social Return on Investment
9
Qualitative impact data
10
And finally…
15
Objective
5
EXECUTIVE SUMMARY
4
Covid-19
4
Understanding success
3
Why well-being?
3
The evidence base
3
Our approach
3
About Noise Solution
1
Impact Report - Norfolk Music Hub & NSFT
Table of contents
About Noise Solution Noise Solution is a social enterprise which delivers 1:1 music mentoring sessions that have been independently proven to improve well-being amongst a population labelled ‘hardest to reach’. Uniquely, mentoring is enhanced through a digital platform that engages participant, family and professionals. As well as contributing to wellbeing impacts, the platform also captures and evaluates well-being impact both qualitatively and quantitatively. This duality of approach solves issues of rigorous impact capture often experienced by third sector organisations.
Our approach Our approach is two-pronged. We have developed a digital sharing platform (re ective for participants and families of everyday social media experiences) which has been designed speci cally to engage, compliment and reinforce an individualised 10-week mentoring program currently focused on music technology and beat-making (though we believe the options for widening the pedagogy to include other art forms or activities other than music are strong). We utilise both the bene ts of personalised mentoring and a digital platform to connect and embed the impacts that the approach has by sharing participants success and encouraging external validation from those whose opinions the participant values. Additionally, all aspects of every interaction are measurable and reportable within the platform. Importantly, the participants voice in the form of comments and video re ection is front and foremost driving the process, rather than being an added-on evaluation exercise postintervention. The platform inclusively engages participants (often refusing or struggling with engagement), their families and professional keyworkers around a story of the participants success (strengths based). This digital sharing scaffolds re ection for participants and family and reinforces successes whilst also allowing us to measure the impacts on well-being. The platform builds in rigorous statistical analysis and comparison against nationally benchmarkable well-being data (SWEMWBS), developed by the NHS and Warwick and Edinburgh Universities.
The evidence base Our methodology arises from £100k investment in research and development over the last 5 years, plus a twoyear Research Masters at Cambridge University. This research looked at the evidence underpinning the psychological needs which have to be supported to increase engagement, understand and increase well-being, and how to measure change meaningfully. For a deeper understanding of the theoretical underpinnings of the work, Simon Glenister MEd (CEO, Noise Solution) has published about Noise Solution’s work and the evidence base. You can read more here. In 2016 The Social Investment Consultancy carried out a year-long external evaluation report, looking at our impacts. The evaluation found that Noise Solution was highly statistically signi cant in positively impacting on well-being amongst its population. These results have been consistently replicated since.
Why well-being? Whilst Noise Solution presents as a music project, our focus is on positively impacting on well-being. We focus on well-being because signi cant empirical evidence proves that if we improve well-being we get better health, social, educational and engagement outcomes. By focusing on well-being we can effect change in all of these
fl
fi
fi
fi
fi
fl
fl
areas, which means our approach is applicable across a wide range of people.
Understanding success Success is individual, and it is important to recognise the small achievements alongside the transformational ones. Noise Solution does not impose pre-determined agendas on participants. Instead, Noise Solution creates a window of opportunity within which other organisations and individuals around a participant can re-engage with each other and acts as a catalyst for change to refocus negative trajectories into positive ones. Progression is important to us. Through the Digital Story, we are able to engage those around a participant with the participants voice at the centre. Around session ve we start looking with the participant at what they would like to do post-Noise Solution. The participants preferred progression routes are fed to the supporting professionals through weekly Session Reports. Overall, Noise Solution sees about 71% completion and 88% attendance amongst its population of proli c nonattenders, phobics and the hardest to engage. All of the population data is only drawn from those who have completed as it relies on bivariate data and clearly those who don’t complete don’t provide end data. The 29% of those who don’t complete is made up of those who Noise Solution has not managed to engage with, but also those who move out of area or those who move on to other things before they have completed.
Covid-19 In March 2020 Covid-19 changed the face of the world and forced organisations such as ourselves to reach our target populations remotely. As a predominantly digital organisation anyway, Noise Solution was able to develop a 100% online offer within three or four days. The underlying pedagogy remains the same, but we are now using a combination of: •
Zoom video calling - with unique links appearing within our Participant Community shortly before each session.
•
Browser-based music-making software - with unique logins for each individual to their own workspace, including lots of bitesize tutorials on a wide range of music genres encouraging selfdirected learning.
•
The Noise Solution Participant Community - containing all the information needed for each participant within a secure, cloud-based platform.
To date, Noise Solution has delivered over 2,500 hours of online delivery to participants across our current footprint of the East of England but also across East London and northern England. We are currently engaged with a researcher from the University Of East Anglia who is working with us to understand the impacts that our work creates when it is delivered solely online. Anecdotally there is lots of interesting feedback around it being less intimidating for some participants who feel one step removed through the computer screen and therefore more comfortable, and also feeling more of a sense of ownership over the process because the participant is using their own device rather than the Musicians equipment.
fi
fi
fi
It must be noted that, as valuable as the online experience is for some, the move to digital delivery has exposed a signi cant seam of digital poverty. This has meant that there is now a segment of our target population which we are unable to reach digitally. Whilst we have lent laptops to participants to enable them to engage this doesn’t help if the participant does not have access to the digital infrastructure needed to access the internet. We are able to deliver face-to-face sessions to these individuals, with all the Covid-secure protocols in place.
EXECUTIVE SUMMARY Objective The intention of this piece of work was for Noise Solution to deliver music mentoring programmes to those young people felt most ‘in need’ from either Norfolk Music Hub or Norfolk Music Hub. We intended to see the same impacts on engagement and well-being as we see across the rest of our work.
Impact Measurement & Reporting Noise Solution measures its impact on well-being using the Shortened Warwick Edinburgh Mental Well-Being Scale (SWEMWBS). The SWEMWBS data is not designed to be used as an individual diagnostic tool, but a way of measuring impact at a population level. The sample size is too small to be able to extrapolate meaning from here but when combined with our other work (delivered in exactly the same way) we see high statistical signi cance of well-being impacts across our population. The data captured from the participant at the start and end of a set of sessions, and the ‘distance travelled’ is statistically analysed on individual and population levels (any change over 2 is considered to be meaningful). Alongside these quantitative data, Noise Solution captures an equal amount of qualitative data in the form of the Digital Stories which is created by each participant as well as re ective feedback sought from the Participant and other involved third parties.
Overall results
fl
fi
Broadly we see similar patterns in SWEMWBS data for this contract as we see across our other services. We have seen reductions in low and moderate well-being being translated into increases of the same, or moving up into the next banding.
Quantitative impact data
The demographic data below includes all participants referred to Noise Solution through this contract, regardless of whether they started or completed. There may be more participants than contracted as some didn’t start or complete the full programme, so these hours were reallocated elsewhere.
Demography Referred participants were focused around Norwich, unsurprisingly given the rurality of Norfolk.
The age distribution of the referred participants peaks around the early teenage years (12-16 years old).
Age 8 6 4 2
Over 16
12-16
Under 12
Male
Female
21%
79%
Attendance and completion Across the entirety of the contract we saw the majority of people attending more than 90% of the time, which is slightly higher than our overall average(85%-88%).
Less than 60% 60-75%
75-90%
Over 90%
Attendance %
21% 36%
43%
Still to complete
No
fi
Yes
The gender distribution is signi cantly more female. This is unusual, as Noise Solution overall sees a gender split of somewhere around 60/40 male/female. Interestingly our data shows we are much more impactful with young women.
We saw fairly low levels of completion from all those who were referred to us compared to our overall non-completion rate (25%-29%). This is likely to have been compounded by the pandemic and participants situations changing dramatically.
Wellbeing data Low
Moderate
High
Low
47.11% 49.78% 3.11%
Moderate
25.78%
START SWEMWBS BAND
60%
High
14.22%
END SWEMWBS BAND
The sample size of this contract is too low to do any statistical analysis. The data above shows our all-time wellbeing data, and the individual scores are comparable to these. The SWEMWBS data above shows a halving of low well-being, a slight increase in moderate well-being and a more signi cant increase in high well-being. This shows Noise Solution acting as a catalyst to improve feelings of well-being at population levels. No Within meaningful range At or above
No Within meaningful range At or above
17% 37%
39% 22%
60% 24%
NATIONAL WELL-BEING AVERAGE (START)
NATIONAL WELL-BEING AVERAGE (END)
Whilst the changes in population scores are important, what is more important is that there is an increase and that this increase is ‘meaningful’. The literature de nes a ‘meaningful’ increase of two points or more using the SWEMWBS scale. With these samples, we saw 52% of participants achieving a meaningful change in their wellbeing scores of two points or more. Additionally, we see 63% of participants nish at or within a meaningful range of the national well-being average (an increase of 23%). We carry out statistical analyses on our data to statistically determine whether or not what we are doing is likely to be successful. Our latest statistical analysis shows:
A p-value of <.0001 (using a Wilcoxon Signed Rank Test). This means that it is massively statistically probable that our work had an impact; we have removed the element of chance.
An ‘R’ number of 0.586 (using Cohens scale). This means there was a ‘large’ range effect; we made a ‘large’
fi
fi
fi
difference.
In 2019 a team of health economists from Costello Medical carried out an eight month pro bono investigation into the Social Return On Investment of Noise Solution’s work. The team were given a brief to be conservative in their assumptions, and only claim what could be robustly evidenced. We also are not claiming things which didn’t happen (for example we know anecdotally that we have diverted a number of young people away from acute mental health services, but because it didn’t happen we haven’t claimed it as a saving). Their work has shown that Noise Solution produces a 334% return on investment. So for every £1 spent, we return £3.34 in savings to ‘the system’.
Our SROI data shows this contract returned over £73,000 back to ‘the system’
Social Return on Investment
The Noise Solution platform allows us to pull together quantitative data (above) with qualitative data (below) in the form of video re ections, photos and blog posts. This allows us to drill down into each datapoint (or person!) above and show their story, told from and by them to people that are important to them. What follows below are some anonymised highlights from the digital stories of participants who were funded through this contract. These highlights demonstrate the three key pillars of our work; autonomy, competence and relatedness.
USING MUSIC TECHNOLOGY TO CREATE ‘QUICK WINS’
USING THE DIGITAL STORY TO CAPTURE AND SHARE SUCCESSES
fl
PROVIDING AN APPROPRIATE LEVEL OF CHALLENGE AND RECOGNISING SUCCESS
Qualitative impact data
GIVING PARTICIPANTS CONFIDENCE IN THEMSELVES
USING THE DIGITAL STORY TO AID PRE-ENGAGEMENT
USING A PEDAGOGY BUILT AROUND QUICK WINS TO EMBED COMPETENCE QUICKLY
EXTERNALISING AND VALIDATING SUCCESSES
USING IN-BUILT BITESIZE TUTORIALS TO PROMOTE BETWEEN SESSION ENGAGEMENT
USING THE DIGITAL STORY AS A SPACE TO RECOGNISE AND REFLECT ON SUCCESSES
GOING TO WHERE THE PARTICIPANT IS AT AND ENGAGING THEM IN THEIR SPACE
CREATING ENCOURAGING AND SUPPORTIVE STRENGTHS-BASED NARRATIVES
USING BROWSER-BASED MUSIC-MAKING TOOLS TO ENABLE MUSIC CREATION QUICKLY AND EASILY
PRESENTING AS A ‘NON-THREATENING’ MUSICIAN TO AID ENGAGEMENT
And finally… As already stated, Noise Solution is not actually a music organisation. We aren’t trying to create musicians. For us, the act of making music is as important as the end result. Having said that, music and lyric writing is an excellent way for some people to tell their story. This track was put together by a participant funded under this contract, and is simply beautiful. Enjoy.
CLICK ABOVE TO PLAY THE TRACK
Not my fault. I was in my bed thinking but it made no sense. Was I dreaming, was it true or didn't I know? I rubbed my eyes and I pinched myself. And started asking myself questions, But then hope came and I pushed the bad away, I couldn't believe what happened that day. I got out of bed and told my mom straight away. It took a lot of courage but I did it anyway. I did it because that was all I had left. I knew it was either this or that again. And if I can do it, I know you can too. So when I told my mom she was so supportive, I had no doubt that she wouldn't be. It de nitely took a weight o my shoulder. And then I realised that it wasn't my fault. But then hope came, and I pushed the bad away. I couldn't believe what happened that day. I got out of bed and told my mum straight away. It took a lot of courage but I did it anyway. I did it because that was all I had left. I knew it was either this or that again, and if I can do it, I know you can too.
ff
fi
So just remember that it wasn't your fault. It was not your fault, it was not your fault.
The following is some feedback from the mother of a participant referred under this contract:
My son [participant] was referred to Noise Solution by the mental health team. [Participant] suffers with autism, Tic/Tourette disorder, selective mutism, severe anxiety, low moods, episodes of self harm etc. [Participant] plays both the guitar and piano and really does get lost In his music so when we were contacted by Noise Solution we were delighted to hear [participant] had been referred to this service for 10 sessions. [Participant] was assigned to a lovely young man called [Musician] and shortly afterwards [participant] had his rst session at home. [Musician] arrived and he and [participant] clicked straight away (this is unheard of with [participant]). After the 1st session [participant] was absolutely buzzing and couldn’t stop talking about it! He couldn’t wait for his next sessions and would eagerly be waiting for [musician] to arrive. As [participant]’s mum I have to help and support him with a great deal of his day to day life including communicating on his behalf when he self mutes, to our astonishment this never once happened with [musician], he made [participant] feel at ease from day one and gave him a con dence we had never before witnessed! Even with the last 4 sessions being away from his home and in a recording studio was no problem, [participant] was happy and con dent to attend (one of his biggest trigger points is leaving the safety of his own home) He was absolutely gutted when the sessions came to an end but was blown away by how lucky he was to be given the opportunity to do these sessions and work with a genuinely amazing team of people. As a family we can not thank [musician] and Simon enough, they went above and beyond to accommodate [participant] and his needs and between them gave [participant] a chance to be himself and succeed without any judgment. Without these guys [participant] wouldn’t of come so far in the last 10 weeks with his mental health, his con dence, his self esteem and his general well-being after a terrible time during lockdowns of completely shutting down and hiding himself away and not talking to anybody. With the thought of his impending transfer to a college he didn’t know where he don’t know anybody hanging over him had sent his anxiety through the roof! I genuinely have never seen [participant] as happy and relaxed as he was doing these sessions with you, something that we very rarely see with [participant]. [Participant] isn’t one to express his worries or concerns or to even talk freely with many people other then me or his grandparents but during these sessions he was able to talk about college and the worries he was facing and [musician] took time to listen and help [participant] at every opportunity and being done in a music environment [participant] was really relaxed, enabling these conversations to happen. They helped him more than words can explain and for that as a family thank you doesn’t even come close to covering it.
fi
fi
fi
fi
fi
I hope this service is made available to more children and if your child is offered the opportunity grab it with both hands because the difference it made to [participant] has been amazing and I know how they could have such a bene cial impact to another young persons life and that of their families too.
