SROI Model:
Executive Summary and Methodology
0
Costello Medical UK | US | Singapore | China www.costellomedical.com
SOLUTION APRIL 2024
NOISE
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 1 Contents List of Figures................................................................................................................2 List of Tables.................................................................................................................2 Executive Summary........................................................................................................3 Introduction ..........................................................................................................................................3 Methods................................................................................................................................................3 Results..................................................................................................................................................3 Key Takeaways......................................................................................................................................3 Model Methodology........................................................................................................6 Social Return on Investment (SROI)................................................................................6 Calculating SROI............................................................................................................6 Objectives .....................................................................................................................7 Scope and Stakeholders .................................................................................................8 Time Horizon.........................................................................................................................................8 Stakeholders .........................................................................................................................................8 Population Inputs...........................................................................................................9 Investments ................................................................................................................10 Activities and Outcomes ............................................................................................... 11 Impact Map......................................................................................................................................... 11 Evaluating and Monetising Outcomes.................................................................................................... 13 Impact on Participants’ Education......................................................................................................... 14 Re-entry into education, employment or training................................................................................ 14 Reduced behavioural issues/school conduct problems......................................................................... 15 Reduced/avoided school exclusion..................................................................................................... 16 Reduced need for additional support in education and related services................................................ 17 Impact on Participants’ Health.............................................................................................................. 19 Increased social capital/reduced isolation........................................................................................... 19 Diverted from acute mental health admission..................................................................................... 20 Disengagement from mental health services ...................................................................................... 21 Impact on Participants’ Family.............................................................................................................. 22 Reduced stress and burden for family members/guardians.................................................................. 22 Increased labour productivity from guardians..................................................................................... 24 Adjustment ......................................................................................................................................... 25 Attribution........................................................................................................................................... 25 Drop-Off ............................................................................................................................................. 25 Sensitivity Analyses......................................................................................................27 Assumptions and Limitations......................................................................................... 28
List of Figures
List of Tables
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 2 General Limitations of SROI Methodology.............................................................................................. 28 Limitations of the Noise Solution SROI Model ........................................................................................ 28 References ..................................................................................................................29
Figure 1. Summary of SROI analysis methodology ......................................................................................4 Figure 2. Per participant costs associated with each outcome included in the SROI analysis, with and without Noise Solution, across two years................................................................................................................5 Figure 3. Impact map.............................................................................................................................. 12
Table 1. Seven principles of the SROI method1 6 Table 2. Summary of outcomes valued in the SROI analysis 13 Table 3. Estimated proportion of Noise Solution participants with NEET status 14 Table 4. Age of Noise Solution participants 14 Table 5. Financial proxy for NEET status 14 Table 6. Estimated behavioural problems in non-NEET participants in school 15 Table 7. Financial proxy for managing school behavioural problems ........................................................... 15 Table 8. Estimated school exclusions of participants.................................................................................. 16 Table 9. Financial proxy for school exclusions, per exclusion...................................................................... 16 Table 10. Education and related service use, as reported in literature ........................................................ 17 Table 11. Estimated use of support in education and related services by participants .................................. 18 Table 12. Financial proxy for additional support in education and related services....................................... 18 Table 13. Estimated loneliness of participants........................................................................................... 19 Table 14. Financial proxy for loneliness .................................................................................................... 20 Table 15. Estimated in-patient mental health admission ............................................................................ 20 Table 16. Financial proxy for in-patient mental health admission................................................................ 21 Table 17. Estimated engagement of participants with mental health services.............................................. 21 Table 18. Financial proxy for community mental health services................................................................. 22 Table 19. Estimated stress and burden for siblings 22 Table 20. Estimated stress and burden for guardians 23 Table 21. Financial proxy for stress and burden for family members/guardians 23 Table 22. Estimated loss of productivity of guardians per participant 24 Table 23. Financial proxy for labour productivity 24 Table 24. Annual drop-off for each outcome 26
Executive Summary
Introduction
Noise Solution is an innovative organisation harnessing the power of music alongside technology to boost and build relationships around those that they work with, while positively impacting upon the well‑being of participants. Their 1:1 music mentoring programme gives individuals facing a variety of challenging circumstances time to work on and direct a creative project that uses a combination of music technology and instruments. The whole process is securely documented by participants using video, photos and audio on a cloud-based platform, creating a private ‘digital story’ of success that develops with each weekly session. Each participant can share their success with friends, family and professional key workers in a way that looks and feels like a social media feed, inviting their network to comment on the digital story. In this way each feed mirrors success and creates a cycle of positive affirmation from people whose opinions matter to the young person. You can find out more about this here.
The programme’s approach is designed to focus on instilling feelings of autonomy, competence and relatedness; evidenced within Self Determination Theory as the basic psychological needs vital for well-being to flourish. Through measuring well-being using a National Health Service validated scale (Short Warwick‑Edinburgh Mental Well-being Scale [SWEMWBS]), it was found that 61% of participants showed a meaningful increase in well-being through attendance of Noise Solution’s programme. Furthermore, qualitative analyses suggested that the programme was often transformational for participants, with positive changes including re‑engagement with education or key workers, reduced isolation and increased hopes and aspirations for the future.
This study aimed to bring together insights gathered from participants engaged with the Noise Solution programme to quantify the financial value that the programme brings to society. A social return on investment (SROI) analysis was conducted, which indicates the social and economic value (£) generated per £1 of investment. An SROI greater than 1 indicates that more value is generated by the programme than that which was initially invested.
Methods
The SROI analysis methodology is summarised in Figure 1 and presented in the Model Methodology section.
Results
The investment per participant attending the programme was valued at £2,075.00. This was estimated to generate £26,071.87 of social and economic value, meaning that for every £1 invested into the Noise Solution programme, there is a return of £12.56 in social and economic value. This indicates that resources are used effectively to result in transformative impact for participants in the programme.
The costs per participant associated with each outcome in the SROI analysis, before and after participation in the programme, are presented in Figure 2. The majority of outcomes considered in the analysis were experienced by the participants of the programme themselves and fell into two broad categories of education and health. Outcomes experienced by participants’ families were also considered. Other individuals engaged with Noise Solution, such as supporting professionals and musicians involved in the programme, were not considered in the analysis due to difficulty robustly quantifying the associated outcomes.
Key Takeaways
The analysis indicates that the Noise Solution programme uses resources effectively to achieve transformative impact for participants. The SROI analysis captured outcomes that could be robustly modelled and applied factors to ensure they reflected the real-world impact of Noise Solution as closely as possible. However, the exclusion of certain outcomes and people who benefit from Noise Solution means that these results are likely to be an underestimate, and the impact of the Noise Solution programme is anticipated to extend beyond that captured here.
SROI MODEL METHODOLOGY 3
Impact Map
An impact map captured the resources invested into the Noise Solution programme, the activities that took place, and the changes (outcomes) experienced by participants engaged with the programme. These included participants, their families, professionals, musicians and the Noise Solution team. These changes were gathered through interviews and feedback.
Indicators
For each outcome, it was evaluated whether there were robust data that evidenced the change taking place (an indicator).
Noise Solution's internal records were then used to identify the proportion of people who experienced each outcome. In some cases, online sources and assumptions were also used.
Financial Proxies
A monetary value (financial proxy) was assigned to each outcome. As cost data were not available from Noise Solution's records, they were obtained from online sources.
Adjustment
To ensure that the analysis reflected the real-world impact of the programme, adjustment factors were applied to the outcomes. These included deadweight, which considers the proportion of outcomes that would have happened without the programme, and attribution, which considers the impact of other services on the outcomes.
SROI Calculation
The data collected were used to quantify the social and economic benefit experienced by participants engaged with the Noise Solution programme. To calculate the SROI ratio, this was compared to the investment required to run the programme.
Abbreviations: SROI, social return on investment.
SROI MODEL METHODOLOGY 4
Figure 1. Summary of SROI analysis methodology
Re-entry into education, employment or training
Abbreviations: SROI, social return on investment.
Noise Solution With Noise Solution
SROI MODEL METHODOLOGY
| 5
Copyright © Costello Medical Consulting Ltd
Figure 2. Per participant costs associated with each outcome included in the SROI analysis, with and without Noise Solution, across two years
↓£256.31 ↓£18.46 ↓£1,976.99 ↓£690.68 ↓£19,980.93 ↓£634.33 ↓£208.41 ↓£1,336.44 ↓£969.32 £1 £10 £100 £1,000 £10,000 £100,000
Reduced/avoided school exclusion Increased social capital/reduced isolation Diverted from acute mental health admission Disengagement
mental health services Reduced need
additional support in education
Reduced stress
Increased labour
Without
Reduced behavioural issues/school conduct problems
from
for
and related services
and burden for family members and guardians
productivity from guardians
Model Methodology
Social Return on Investment (SROI)
An SROI analysis is an outcomes-based framework that enables organisations to understand, quantify and monetise the social, environmental and economic value created through their services. To assign a financial value to social and environmental outcomes that lack a well-defined market value, financial proxies, derived from published literature, are used. The final output of an SROI analysis is an SROI ratio, indicating the value of an organisation’s services (expressed in cost units, £) delivered per £1 investment.
There are seven principles behind the SROI analysis method, as set out by the SROI Network.1 These are described in Table 1.
Table 1. Seven principles of the SROI method1
Principle
Involvement of stakeholders
Understand what changes
Value the things that matter
Only include what is material
Do not over-claim
Be transparent
Verify results
Description
The consultation of well-defined stakeholder groups throughout the analysis ensures that the value of the organisation’s outcomes, and the way in which they are measured, are informed by those directly affected.
Robust analyses include an evaluation of how both positive and negative change may be created by an organisation, including those which are unintentional.
Financial proxies are used to assign a value to outcomes generated by the organisation, for outcomes that do not have a well-defined market value or financial worth.
Critical selection of the inputs and outcomes to be included in the model is crucial to give a true and fair representation of the value generated by an organisation.
A correct evaluation of outcomes involves recognition of the other factors which may contribute to the outcome, beyond the organisation’s services.
An honest account should be provided of when the SROI analysis may be considered accurate. The methodology and results of the analysis should be reported to and discussed with stakeholders.
Independent validation by appropriate parties may help to remove subjectivity.
Abbreviations: SROI: social return on investment.
Calculating SROI
The SROI ratio is calculated as a ratio of the net benefits generated through an organisation’s activities and the net investment required to run these activities, and represents the value of their services over a defined period of time. The simple equation applied to calculate the SROI is as follows:
SROI= Totalbenefit(net) Totalinvestment(net)
Any SROI ratio with a value greater than one indicates that more value is being generated than invested. However, there is currently no agreement for what constitutes a “good” SROI ratio, as methodological differences prevent robust comparison of SROI analyses conducted by different parties. In particular, the SROI ratio estimates yielded by these analyses may vary based on the type of outcomes being included and excluded, the considered time horizon, and the level of conservatism applied while making subjective assumptions.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 6
Objectives
The objective of this SROI analysis was to identify the value generated by Noise Solution services and activities. This report reflects the methodology of the updated SROI analysis, which builds on the original SROI analysis carried out in 2019 by incorporating an updated impact map and alternative inputs for quantifying outcomes. Details of the original SROI model can be found in a separate report. It is hoped that the identified evidence can be used, in conjunction with qualitative evaluations, to support future funding of Noise Solution.
The SROI ratio produced indicates the value generated by Noise Solution per £1 of external investment. The focus of this ratio is placed on social value, as opposed to purely financial. In addition, whilst the conversion of social value to monetary units is analytically useful and helps to convey social value in a widely accepted and tangible format, it was not possible to monetise all the anticipated social outcomes of Noise Solution’s work. Furthermore, whilst the most appropriate model inputs were used wherever possible, some assumptions were required due to limitations in the available data.
The SROI ratio presented in this report therefore provides a conservative and provisional estimate, rather than a comprehensive outlook, of Noise Solution’s impact. The ratio is designed to complementan evaluative assessment of the social value generated by Noise Solution, as well as to identify gaps in Noise Solution’s internal evaluative data.
This report describes the methodology of the updated SROI analysis conducted and details all financial proxies used to monetise the socioeconomic outcomes produced by Noise Solution’s work. The seven principles underpinning the SROI framework, described above, have been adhered to throughout the analysis and this report:1
⬧ Involvement of stakeholders: The impact map of outcomes generated by Noise Solution were primarily informed by discussions with participants, immediate family/guardians and mentors in the original analysis, with support from other stakeholders within Noise Solution. Where possible, data used in the model were informed by responses from users of Noise Solution services
⬧ Understand what changes: Key outcomes captured by the model were determined, where possible, by looking at relevant changes in users of Noise Solution services. Expert opinions from Noise Solution stakeholders were also used to inform the outcomes
⬧ Value the things that matter: Informed by published financial proxies, the SROI model considers not only the benefits and costs to Noise Solution service users, but also to the wider population through governmental-related costs
⬧ Only include what is material: It is crucial to accurately establish impact in SROI analyses, by estimating how much of an outcome would have happened anyway and what proportion of an outcome experienced by a stakeholder can be isolated as stemming from Noise Solution activities.1 Model results were therefore adjusted for deadweight, attribution and temporal drop-off. Please see the section on Adjustment for further details on these three adjustment factors
⬧ Do not over-claim: Where estimates and assumptions were made as part of this SROI analysis, these were conservative and clearly stated
⬧ Be transparent: All sources used to inform both outcomes and financial proxies were recorded and are detailed in this report. All decisions (and the accompanying rationale) likely to affect the outcome of the analysis are also detailed in the SROI model
⬧ Verify results: The SROI framework is subjective, and as a result multiple sources were used to justify the inclusion and exclusion of all outcomes and costs. Where possible, discussions with Noise Solution were held to verify outcomes and their association with Noise Solution services
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 7
Scope and Stakeholders
Time Horizon
The SROI focuses on participant outcomes that are generated within two years of completing the programme. Longer-term outcomes that occur beyond two years of interacting with Noise Solution are not explored in the analysis. This approach is rationalised by the fact that many model outcomes depend on participation in Noise Solution services (e.g. reduced behavioural problems, increased social capital). Complex interactions between internal and external factors mean that estimates of impact made over longer time horizons are unlikely to be accurate. In addition, a single time horizon across the model facilitates interpretation, compared with a model using varying time horizons across inputs. A duration of two years therefore balanced these requirements to maximise both accuracy and interpretability of estimates.
It should be noted that some outputs are estimated to deliver a different impact in the first and subsequent years after Noise Solution participation. For example, the impact of Noise Solution on productivity loss of guardians includes a partial impact in the year of Noise Solution and a subsequent full impact the following year, due to the time allocated by guardians to attend sessions with participants. The total impact is therefore quantified as an average of the two years.
The model also includes the functionality to account for ‘drop-off’ in value from Noise Solution services throughout the year (see section on Adjustment; see the Model User Guide for practical considerations). For example, it would be reasonable to assume that the impact of the programme on a participant would be lower after 12 months of completion than after one month. This functionality allows for more granular and conservative estimates of the social value produced by Noise Solution.
Stakeholders
Several stakeholder groups are considered in the SROI model. Discussions with Simon Glenister, Director of Noise Solution, and Damien Ribbans, Operations Director, assisted in directing the scope of the analyses. Stakeholder groups include:
⬧ Programme participants
⬧ Immediate family members (e.g. guardians and siblings)
⬧ Other related services: private and government-funded health/social/educational services (including employees)
Musicians tutoring Noise Solution participants and Noise Solution were also considered in the impact mapping stage (see Impact Map); however, it was decided that they were not relevant for inclusion in the SROI model. This was due to a lack of outcome materiality, with some outcomes relating to these stakeholder groups being difficult to quantify/monetise and other outcomes difficult to accurately attribute to Noise Solution; for example, the supporting of the local community and economy by hiring musician tutors. It was therefore decided that their inclusion may overclaim the benefits of Noise Solution’s programme.
In the original SROI analyses, interviews with one person from each of the key stakeholder groups were conducted. The interviews aimed to identify and confirm the key outcomes of interest for the SROI model, as well as identify outcomes to consider for future data collection and any relevant stakeholders who were not originally considered in the model. In the updated analyses, the original research was complemented with qualitative analyses of feedback received from Noise Solution participants, conducted by Noise Solution employees.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 8
Population Inputs
Noise Solution participants were defined as individuals who had completed the programme. Data on 374 previous participants of Noise Solution were obtained and used in the model to inform age demographic inputs, including median ages and the proportion of participants under 18. Well-being data from previous participants were also incorporated into the model; during the Noise Solution programme, questionnaires reflecting the Short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS) are completed by participants to measure their well-being. A significant improvement in well-being is defined by an increase of at least one in the SWEMWBS score, as per Noise Solution’s suggestion and the previously reported minimum important difference.2 Further analyses of the SWEMWBS data can be found in a separate report.
Population inputs can continue to be updated by Noise Solution to increase representation of the participant population.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 9
Investments
The investment per participant for Noise Solution has been calculated as the cost of participating in the 10-week programme (£2,075). This was deemed an appropriate valuation of the investment, as Noise Solution is costed on a cost-recovery basis, and therefore covers all necessary investment costs incurred.
SROI MODEL METHODOLOGY 10
Activities and Outcomes
Impact Map
An updated impact map for the programme was developed collaboratively by Noise Solution and Costello Medical. The impact map details the relationship between the resources (inputs) that are used by Noise Solution to deliver their activities, and the direct and tangible results of these activities (outputs). Outcomes are the consequences of the activities on relevant stakeholder groups. Relevant outcomes are listed on the impact map, separated into categories based on the timepoint at which they are expected to be achieved:
⬧ Immediate: Achieved immediately up until three months of engaging with Noise Solution, and when maintained, may result in medium-term or sustained outcomes. Outcomes in this category include developing new skills and improvements in self-esteem and self-confidence
⬧ Short-term: Achieved within three to six months of engaging with Noise Solution, and when maintained, may result in a sustained outcome. Outcomes falling into this category include increased educational attainment and willingness to try new activities and tasks
⬧ Medium-term outcomes: Achieved in six to twelve months, and possibly sustained for a longer period. Outcomes in this category include increased labour productivity from guardians and reduced need for additional support in other areas through increased resilience and independence
⬧ Long-term outcomes: Achieved from 12 months, for example reduced financial burden on families and reduced isolation
This relationship is called a theory of change and aims to summarise the impact Noise Solution has on society. The finalised impact map is presented in Figure 3.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 11
Abbreviations: FTE: full-time equivalent; NEET: not in education, employment or training; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 12
Figure 3. Impact map
Evaluating and Monetising Outcomes
The outcomes presented in the impact maps were subsequently assessed to determine whether they could be modelled robustly. Attempts were made to capture and quantify the vast majority of these outcomes; however, this was not possible in all cases for a number of reasons:
⬧ Some shorter-term outcomes overlapped with longer term outcomes, such as ‘Increased labour productivity of families’ and ‘Reduced financial burden’. In this case, modelling both outcomes would have led to double counting in the value generated
⬧ Some long-term outcomes affected broad and ill-defined groups of stakeholders, such as ‘Increased awareness of self’. Monetising these outcomes would introduce considerable uncertainty to the findings
⬧ Despite attempts to monetise all outcomes, suitable financial proxies could not be found for some, such as ‘Improved sense of hope’
The following sub-sections detail the outcomes that have been included in the SROI model. For each outcome considered, the parameters and the respective data sources are described, including the indicators and financial proxies used in the model. An indicator is used to determine whether change has taken place. A proxy is a value that is deemed to be close to the desired indicator, for which exact data might be unavailable.1 A summary of the outcomes valued in the SROI analysis is provided in Table 2
For each outcome, multiple estimates and assumptions have been made. These have been described below. To decrease the uncertainty in the model, inputs could be replaced by data collected by Noise Solution in the future. Inputs for which this is applicable have been highlighted throughout the input tables using orange shading.
Outcome
Re-entry into education, employment or training
Impact on participant’s education
Impact on participant’s health
Impact on participant’s family
Reduced behavioural issues/school conduct problems
Reduced/avoided school exclusions
Reduced need for additional support in education and related services
Increased social capital/reduced isolation
Diverted from acute mental health admission
Disengagement from mental health services
Reduced stress and burden for family members/guardians
Increased labour productivity from guardians (increased/returned to work)
Abbreviations: SROI: social return on investment.
For all financial proxies, the costs have been adjusted for inflation using the consumer price index (CPI). The calculation for adjusting the costs has been provided below:
Inflatedcost= CPI(currentyear) CPI(yearoffinancialproxy)xfinancialproxy
Costs were inflated to the 2023 price due to the 2024 CPI having not yet been published at the time of the SROI analysis. All costs provided in this report are those from original sources and are unadjusted for inflation; please check the model for inflated costs.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 13
Table 2. Summary of outcomes valued in the SROI analysis
Impact on Participants’ Education
Re-entry into education, employment or training
“HehasmadegreatprogresssinceworkingwithNoisesolution,heisnowenrolledontoafull-time Musiccourse.ThecommonassessmentframeworkIhadopenforhimhasnowclosedasthingsat homehavegreatlyimproved.HisworkwithYouthJusticehasbeencompletedandtherehavebeenno moreincidentswiththepolice.Ithasbeengreattoseehisjourneyandthemusichehasproduced.
Thankyouformovinghimforwardwithhislife!”– Referring Professional
As participants show re-engagement with education, training or employment, there is an expected reduced burden on government social services that would otherwise be spent to support these youth. The proposed indicator for this outcome was the proportion of Noise Solution participants who are not in education, employment or training (NEET) prior to Noise Solution, and those who have shed their NEET status following Noise Solution. It was assumed that individuals with NEET status would not be in any form of education, employment or training for the entirety of a given year. Data on truancy of Noise Solution participants are currently not collected. Therefore, estimates from Noise Solution were used; these are presented in Table 3
Table 3. Estimated proportion of Noise Solution participants with NEET status
Input Value
Proportion of all participants with NEET status, before Noise Solution 40.0%
Proportion of NEET status holders shedding their NEET status, after Noise Solution 30.0%
Abbreviations: NEET: not in education, employment or training.
Source/Justification
Noise Solution estimate for proportion of participants with NEET status
Noise Solution estimate for proportion of NEET participants shedding their NEET status
The financial proxy used for this outcome was the weekly Jobseekers Allowance (JSA) for individuals aged over 16. Allowances vary for individuals aged 16–24 and over 25, therefore the proportion of participants in each age category was calculated using a Noise Solution dataset of past participants (Table 4) and different costs assigned to each category (Table 5).3 The proportions of participants moving between age groups during the time horizon were not considered. It was assumed that individuals under 16 years old would not be receiving JSA if they are NEET, therefore they were not accounted for in this outcome. Other costs included in the model, including guardian labour productivity, were assumed to already capture the associated costs of individuals aged under 16 having NEET status.
Table 4. Age of Noise Solution participants
Input
Proportion of all Noise Solution participants aged 16–24 years
Proportion of all Noise Solution participants aged 25+ years
Table 5. Financial proxy for NEET status
Financial Proxy
Job Seeker’s weekly allowance for 16–24 years
Job Seeker’s weekly allowance for 25+ years
Abbreviations: NEET: not in education, employment or training.
Source/Justification
Noise Solution dataset
Source/Justification
Department for Work and Pensions3
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 14
Value
22.9%
5.3%
Value
£67.20
£84.80
Reduced behavioural issues/school conduct problems
As participants develop sense of competency in a new area, and increased engagement in pro-social activities and behaviours in the short term, there is expected to be a reduction in school conduct problems, such as detentions. These outcomes from the impact map have been calculated by estimating the proportion of Noise Solution participants that would receive behavioural complaints that disrupt classroom teaching both pre- and post-participation in Noise Solution. As this outcome is specific to school behaviour, it is applied to schoolaged participants (under 18) with non-NEET status only.
In the absence of any data identified in the literature or collected by Noise Solution, the reduction in school conduct problems has been based on assumptions. It has been assumed that before Noise Solution, 80% of school-aged, non-NEET participants receive behavioural complaints at school. Post Noise Solution, it is assumed that participants who show a significant improvement in well-being have a 65% reduction in receiving behavioural complaints, whilst all other non-NEET participants receive the same number of complaints These inputs are summarised in Table 6 Participants who shed NEET status and return to school are also considered following Noise Solution. It is assumed that these individuals return to school with the same reduction in behavioural problems as individuals who were in school.
Table 6. Estimated behavioural problems in non-NEET participants in school
Input
Proportion of school-aged participants with behavioural problems, before Noise Solution 80.0%
Proportion of school-aged participants with previous behavioural problems, who showed reduced behavioural problems post Noise Solution
Proportion of reduction in behavioural problems due to Noise Solution
61.0%
65.0%
Source/Justification
Noise Solution estimate for number of school-aged non-NEET participants
Proportion of participants experiencing increased (>1 score increase) SWEMWBS score assumed to experience reduced incidence of behavioural issues
Noise Solution estimate for reduction of behavioural complaints in school-aged, nonNEET participants who have a significant improvement in well-being
Abbreviations: NEET: not in education, employment or training; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.
The financial proxy used to calculate the value of this outcome was the cost of a school-based social and emotional learning programme, as reported in the PSSRU 2021, in addition to the annual cost of teaching time salary that is lost to management of classroom behavioural issues per disruptive child.4-6 These values are summarised in Table 7.
Table 7. Financial proxy for managing school behavioural problems
Financial Proxy Value
Cost of a school-based social and emotional learning programme
Annual cost of teaching time (salary) that is lost to management of classroom behavioural issues per disruptive child
Abbreviations: PSSRU: Personal and Social Services Research Unit.
£181.00
£157.23
Source/Justification
PSSRU 20214
Cost per child per year
The Time and Cost of Classroom Behavior Management (Gormley, 2021)5
Assuming 3 seconds of disruption per occurrence, teachers spend 10.95 hours responding to a single referred child’s behaviour each year. Based on average England (excluding London/fringe) teaching salary of £28,0006 (assumed 37.5 hours/week)
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 15
Value
Reduced/avoided school exclusion
“WithoutNoiseSolution,thestudentwouldhavebecomefurtherdisengagedfromlearningand ultimatelysociety,ashefeltundervaluedanddidnotseehimselfasalearnerorsomeonewhocouldbe taughtbyteachers.NoiseSolutionisabsolutelybrilliant!Phenomenalwaytoengagestudentsin learning.Disengagedbecomeengaged.” – Referring Professional
Outcomes relating to young people re-engaging with education, training, employment or voluntary placements are included throughout the impact map. As participants are expected to gain confidence, competency and achievement in the short-term, and hence re-engagement in the medium-term, there is an expected re-engagement (either in mainstream schooling, or other means) of a proportion of those participants who were previously excluded from schooling. This outcome has been calculated as the proportion of participants who have been excluded before Noise Solution, and the proportion of participants who remain excluded after Noise Solution, and has only been applied to school-aged participants. These inputs have been estimated by Noise Solution and are presented in Table 8. It has been estimated that of the ~45% of participants that are expelled/suspended from school, there is a 1:1 split between exclusions and suspensions. This estimate has been used due to a lack of appropriate input data, however should be updated to better reflect the proportion of Noise Solution participants who are excluded from school based on data captured at Noise Solution referral.
Table 8. Estimated school exclusions of participants Input
Proportion of school-aged participants who have been excluded, before Noise Solution 22.5%
Proportion of school-aged participants who remain excluded, after Noise Solution 13.5%
Source/Justification
Noise Solution estimate – ~45% of participants have been expelled/suspended from school (1:1 split between exclusions and suspensions)
Estimate from Noise Solution – ~40% of participants with prior exclusions re-engage with education, employment or voluntary placements
To estimate associated costs, a report published by New Philanthropy Capital that estimates the total cost of school exclusions in the UK was reviewed.7 Specifically, costs to the education system and social services were extracted. Other costs included in the report that were not considered in the SROI model were impact on earnings, cost to the NHS and cost of higher crime. These were not included as they were either captured elsewhere in the model (e.g. mental health treatment costs), or likely to come to bear outside the duration scope of the model (e.g. physical health costs). The cost of managing the process of an exclusion was applied in the model as a one-off cost in the first year and not considered in subsequent years. A summary of the costs associated with school exclusions is provided in Table 9
Table 9. Financial proxy for school exclusions, per exclusion
Financial Proxy Value
Cost of managing the process of an exclusion (one-off cost)
Net annual cost of alternative educational provisions
Annual cost of increased use of social services
£831.00
£7,181.00
£1,169.00
Source/Justification
Misspent Youth Report: Cost of Truancy and Exclusion, 20077
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 16
Value
Reduced need for additional support in education and related services
“Ifoneofourincrediblevulnerablechildrenisnotengagingwitheducationorneedssupportthrougha difficulttime…thenNoiseSolutionisthefirstportofcall.Regardlessastowhethertheyhavemusicas abackground.…Ican'tthankNoiseSolutionenoughfortheworktheydowithourmostvulnerable childrenandyoungpeople.”– Referring Professional
Another outcome identified in the impact map is the reduced need for additional support in education and other related services. This outcome has been calculated by estimating the use of educational support services pre- and post-participation in Noise Solution. As this outcome is specific to schools, it is only calculated for school-aged, non-NEET participants.
Three educational services were captured in the model; educational psychologist, special needs support and classroom assistance, based on sources cited in literature.8, 9 The proportion of participants in contact with an educational psychologist, and/or with special educational needs (SEN), is based on the Mental Health of Children and Young People 2017 survey, stratified by type of behavioural disorder (see Table 10).8 In the absence of estimates for classroom assistants, this was assumed equal to the proportion of children accessing SEN support. Of all Noise Solution participants, it was assumed that 80% had either an emotional, conduct or hyperactive disorder, distributed equally. The remaining 20% were assumed to not have a disorder. It is assumed that the prevalence of disorders is irrespective of participant age. Mean use of each service was derived from Romeo 2006 (assumptions of frequency [that the time recurred weekly] were made due to a lack of their being reported), a paper estimating the cost of severe antisocial behaviour in children, which collected service use data for children who had been referred to mental health services in the UK (Table 10).9
Following participation in Noise Solution’s programme, it was assumed that participants who had a meaningful improvement in well-being, measured through SWEMWBS, reduced their use of each service by 65% (estimate provided by Noise Solution). Those without a meaningful improvement in SWEMWBS continued with the usage of services as before. The impact of individuals who shed NEET status are also considered postNoise Solution, and are assumed to follow the same distributions in terms of disorders and service use as individuals who did not have NEET status at the start of the programme. A summary of the inputs derived from literature are presented in Table 10. The resulting estimated proportion of Noise Solution participants receiving support before and following Noise Solutions is presented in Table 11.
Table 10. Education and related service use, as reported in literature
Input
Proportion of all Noise Solution participants with disorder
Proportion of children who sought educational psychology services
Proportion of children with SEN
Proportion of children who sought classroom assistance
Emotional disorder: 20.1% Conduct disorder: 31.9%
disorder: 1.9%
Emotional disorder: 26.8%
Conduct disorder: 42.4%
62.9%
disorder: 6.1%
Emotional disorder: 26.8% Conduct disorder: 42.4%
62.9%
disorder: 6.1%
Educational psychologist annual use (hours) 2.00
Source/Justification
Noise Solution estimate
Mental Health of Children and Young People in England, 20178
Mental Health of Children and Young People in England, 20178
Assumed equal to SEN
Romeo 20069
Based on mean use of 2 visits (one-to-one service) in 12 months
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 17
Value
Emotional disorder: 26.7% Conduct disorder: 26.7% Hyperactivity: 26.7% No disorder: 20.0%
Hyperactivity:
No
32.1%
No
Hyperactivity:
Hyperactivity:
No
Special
Abbreviations: SEN: special educational needs.
Adapted from Romeo 20069
Assumes a mean use of 40 minutes per week across a period of 3 months
Adapted from Romeo 20069
Assumes a mean use of 2 hours per week across a period of 5 months
Table 11. Estimated use of support in education and related services by participants
Proportion of school-aged participants receiving educational psychologist support, before Noise Solution
Proportion of school-aged participants receiving special education needs support, before Noise Solution
Proportion of school-aged participants receiving classroom assistance, before Noise Solution
Proportion of school-aged participants receiving educational psychologist support, after Noise Solution
Proportion of school-aged participants receiving special education needs support, after Noise Solution
Proportion of school-aged participants receiving classroom assistance, after Noise Solution
Abbreviations: NEET: not in education, employment or training.
Combination of the proportion of individuals who receive selected support according to behavioural disorder, as reported in Table 10
Reduction in support services is applied at the cohort level, so that of participants who have a significant improvement in wellbeing, 65% cease to use support services. This is equivalent to each of these participants reducing use of services by 65% each, if service use has a 1:1 ratio with disorder intensity. Participants who shed NEET status have the same reduction in use of education support and related services
Costs for each service were derived from the PSSRU and GOV.UK.4, 10 The cost of an educational psychologist was assumed equal to a paediatric occupational therapy visit. Special needs support and classroom assistance hourly rates were calculated based on the mean wage of a teaching assistant in the UK in 2023 (£23,000), assuming 195 working days and 6.5 hour days.11
Table 12. Financial proxy for additional support in education and related services
Abbreviations: PSSRU: Personal and Social Services Research Unit; SEN: special educational needs. Financial Proxy
PSSRU, 20214
Paediatric occupational therapy, cost per hour
Cost per hour based on average annual salary of SEN teaching assistant (£23,000)11 with 195 working days per year, and 6.5 hours per day
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 18
needs support annual use (hours) 8.70
Classroom assistance annual use (hours) 43.48
Input Value
Source/Justification
11.7%
18.7%
18.7%
8.5%
13.0%
11.7%
Value
Educational psychologist £160.00
Source/Justification
Classroom assistance
£18.15
and SEN
Impact on Participants’ Health
Increased social capital/reduced isolation
“TheyoungpersonIamworkingwithlovedthesessions.Hehadnotbeeninanyformofeducationor learning/socialactivityforaroundayear...Thesessionsgavehimtheconfidencetointeractwithother peopleandtheknowledgethathecouldachievesomethinggiventherightenvironment.”
– Referring Professional
A further outcome identified in the impact map is the increased social capital as a result of Noise Solution participation and thereby reduced isolation. The value of this outcome was approximated by calculating the proportion of individuals experiencing loneliness based on data published by the ONS.12 This assumes that all participants who join Noise Solution have the same experience of loneliness as someone with a probable mental health disorder. The proportion of individuals experiencing loneliness was reported in individuals aged 11–16 and 17–22, as presented in Table 13. The proportion of individuals experiencing loneliness before Noise Solution was calculated assuming all participants aged 16 and under have the same likelihood of loneliness as one another, and that all participants aged 17 and over have the same likelihood of loneliness as one another. The proportion of participants who no longer experience loneliness following Noise Solution has been estimated by Noise Solution based on discussions with participants who took part in the programme.
Table 13. Estimated loneliness of participants
Input
Proportion of individuals aged 16 and under often or always experiencing loneliness, before Noise Solution
Proportion of individuals aged 17 and over often or always experiencing loneliness, before Noise Solution
Total proportion of participants experiencing loneliness, before Noise Solution
Proportion of participants experiencing loneliness who are no longer experiencing loneliness, after Noise Solution
Abbreviations: ONS: Office for National Statistics.
29.5%
ONS, 2023; Proportion of individuals experiencing loneliness by mental health of child, 2023; aged 11–1612
Assumes that all participants in Noise Solution have the same experience of loneliness as someone with a mental health disorder and participants aged 16 and under have the same likelihood of experiencing loneliness
ONS, 2023; Proportion of individuals experiencing loneliness by mental health of child, 2023; aged 17–2212
Assumes that all participants in Noise Solution have the same experience of loneliness as someone with a mental health disorder and participants aged 17 and over have the same likelihood of experiencing loneliness
A weighted average of the above inputs based on the age distribution of Noise Solution participants
Noise Solution estimate based on communication with families
The financial proxy associated with this outcome was based on an analysis by the Department for Digital, Culture, Media and Sport which estimated the costs associated with loneliness stemming from the impact on subjective well-being, health and productivity.13 For individuals aged 17 and over, the total cost considered the impact of health, productivity and well-being for individuals with moderate to severe loneliness. The report did not consider the cost for individuals under 16, therefore it was assumed that the cost of mild
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 19
Value
Source/Justification
16.6%
27.8%
30.0%
loneliness, which does not consider productivity loss, would be an appropriate proxy for individuals 16 and under. The costs have been summarised in Table 14.
Table 14. Financial proxy for loneliness
Financial Proxy Value
Cost of moderate to severe loneliness in individuals aged 17 and over
Cost of moderate to severe loneliness in individuals aged 16 and under
£9,976.00
£6,429.00
Source/Justification
Loneliness monetisation report, 202013 Assumes loneliness impacts subjective wellbeing, health and productivity
Loneliness monetisation report, 202013 Assumes cost of moderate to severe loneliness in individuals aged 16 and under is proportionate to mild loneliness in individuals aged 17 and over, to account for the lack of impact on productivity
Diverted from acute mental health admission
“TwooftheactsbookedforlastSaturday'sgigweremadeofNoiseSolutiongraduates.Theirsecond gigs.Theonegirlgavealong,emotionalspeechonstageabouthowitsavedherlifeandshewouldn't beheretodaywithoutit.”– Referring Professional
In-patient admission to mental health services was identified as an outcome impacted by Noise Solution. The proportion of patients with previous in-patient mental health admission prior to Noise Solution was estimated by Noise Solution, as was the proportion diverted from in-patient admission following the intervention. These proportions were assumed to apply equally to participants irrespective of age.
Table 15. Estimated in-patient mental health admission
Input
Proportion of all participants with previous in-patient mental health admission, before Noise Solution
of all participants with in-patient mental health admission, after Noise Solution
Source/Justification
Noise Solution estimate
This outcome has been monetised by valuing the cost of in-patient admission for children, which is under the responsibility of CAHMS, and for adults. For children, a report in 2017 reported the average cost of CAMHS in‑patient admission per person (£61,000.00).14 For adults, the average cost per person was estimated using the average cost in-patient admission per day (£446.00), the average length of in-patient admission (49 days) and the costs following discharge from acute medical units (£2,431.00).15-17 The costs have been summarised in Table 16.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 20
Value
30.0%
Proportion
5.0%
Table 16. Financial proxy for in-patient mental health admission
Financial Proxy Value
Cost of average CAMHS in-patient admission
£61,000.00
Cost of average adult in-patient admission
£24,285.00
Source/Justification
Children's Commissioner: Children's Mental Healthcare in England14
Average cost of CAMHS in-patient admission from 2017
East London NHS Foundation Trust, 2020; daily cost of in-patient mental health admission (£446.00)16
The strategy unit, 2019; average length of mental health in-patient admission (49 days)17
PSSRU, 2022; post-discharge costs (£2,431.00)15
Abbreviations: CAMHS: Child and Adolescent Mental Health Service; NHS: National Health Service; PSSRU: Personal and Social Services Research Unit.
Disengagement from mental health services
In addition to in-patient mental health admissions, it is also expected that individuals who attend Noise Solution will disengage from community mental health services as a result of improved well-being. In the absence of any data identified in the literature or collected by Noise Solution, it was estimated that 35% of participants use community mental health services prior to Noise Solution. Following Noise Solution, it is estimated that this reduces to 15%. These inputs have been summarised in Table 17
Table 17. Estimated engagement of participants with mental health services
As with in-patient mental health admissions, different providers are used for children and adult services. For children, the outcome was monetised by valuing the cost of CAHMS community support per child per year, as reported by the NHS.14, 18 For adults, the cost was estimated using the average cost per attendance of adults requiring mental health support to day care, provided by local authority social services (£58.00) or the private and voluntary sector (£41.00), with the assumption that the adult would attend one or the other, with equal likelihood of each, for the average length of recovery (three sessions). The cost of delivering behavioural activation, a treatment for depression provided in a group setting, was also considered, with the assumption that the adult would attend a complete programme (12 sessions) once a year (£228.00) in addition to local authority/voluntary services.15 These costs have been summarised in Table 18.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 21
Input Value Source/Justification Proportion of all participants using community mental health services, before
35.0% Noise Solution estimate Proportion of all participants using community mental health services, after Noise Solution 15.0% Noise Solution estimate
Noise Solution
Table 18. Financial proxy for community mental health services
Financial Proxy Value
Cost of CAMHS community mental health services, per year
Cost of adult community mental health visits, per year
£2,518.00
£376.50
Source/Justification
NHS Benchmarking Network: Benchmarking Mental Health18
Average cost of a year of CAMHS community mental health services from 2017
PSSRU, 202215
Cost per attendance to day care, provided by social services (£58.00) or voluntary or private sector (£41.00); assumed attendance of three sessions
Cost per attendance to behavioural activation programme (12 sessions), once a year (£228.00)
Abbreviations: CAMHS: Child and Adolescent Mental Health Service; NHS: National Health Service; PSSRU: Personal and Social Services Research Unit.
Impact on Participants’ Family
Reduced stress and burden for family members/guardians
“Anawfullotoftutorsinthepasthaven’tbeenabletocopewithhimbecauseofhisdisabilitiesandhave givenuponhim.Heusedtobeabig,bighandful!Towatchhimnowisajoy.I'vegivenhimsomuch praise.”– Family of Participant
An outcome identified in the impact map is the reduction in stress and reduced burden experienced by family members, resulting from improvements in participants’ behavioural problems. This outcome estimates the proportion of guardians and siblings experiencing increased stress/caregiver burden, before and after Noise Solution, and applies the cost of a counselling intervention, to represent the value with and without Noise Solution. It is assumed that guardians and siblings are affected by the participant’s behaviour, regardless of the participant’s age.
Separate calculations were made for guardians and siblings, each of which are described in turn. The proportion of siblings affected by the participant’s behaviour pre Noise Solution was based on reported data from the Mental Health of Children and Adolescents 1999 study.19 Specifically, the answers to ‘child’s problems cause difficulties with other family members’ for children with emotional, conduct and hyperactive disorders was considered. The proportion of siblings affected ranged from 19–35% for individuals with behavioural disorders. Given that not all Noise Solution participants are anticipated to have behavioural disorders, a conservative estimate was used whereby 20% of all siblings are disrupted by the participant’s behaviour. After Noise Solution, it was anticipated that individuals who have a significant improvement in well-being, measured through SWEMWBS, would have a reduction in behavioural problems that would remove the stress and burden experienced by siblings. The average number of siblings per participant was calculated using data reported by the ONS.20 The inputs used to value the reduced stress and burden for siblings are described in Table 19.
Table 19. Estimated stress and burden for siblings
Input
Proportion of siblings affected by participant's behaviour, before Noise Solution 20.0%
Source/Justification
Conservative estimate based on proportion of siblings affected by child’s behaviour according to different behavioural disorders; reported in Mental Health of Children and Adolescents, 199919
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 22
Value
Proportion of siblings affected by participant's behaviour, after Noise Solution 7.8%
Average number of siblings per participant 0.75
Proportion of participants experiencing increased (>1 score increase) SWEMWBS score assumed to experience reduced incidence of behavioural issues
All siblings of participants who do not show a significant improvement in well-being remain affected, if already in an affected state
ONS, 2017; Families with dependent children by number of children20 Equal to the average number of children per family (1.75) minus one (the participant)
Abbreviations: ONS: Office for National Statistics; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.
For guardians, it was assumed that prior to Noise Solution, all guardians showed increased stress resultant of the child’s behaviour, in the absence of relevant data. Stress and caregiver burden post-Noise Solution was calculated in the same way as for siblings, with a proportional reduction based on Noise Solution participants showing a meaningful improvement in well-being. The average number of guardians per participant was calculated based on the proportion of families with one or two guardians using data published from the ONS.21 Table 20 summarises the described inputs used to calculate the proportion of guardians who are impacted by the participant’s behaviour, pre- and post-Noise Solution.
Table 20. Estimated stress and burden for guardians
Average number of guardians per participant 1.85
Proportion of participants experiencing increased (>1 score increase) SWEMWBS score assumed to experience reduced incidence of behavioural issues
The model assumes that all guardians of participants who do not show a significant improvement in well-being maintain affected, if already in an affected state
ONS, 2023; Calculated from the proportion of families with one or two guardians21
Abbreviations: ONS: Office for National Statistics; SWEMWBS: Short Warwick-Edinburgh Mental Wellbeing Scale.
The financial proxy associated with this outcome was the average cost per counselling intervention, as reported in the PSSRU 2021.4 Specifically, the cost applied was for interventions ranging from 9 to 12 counselling sessions (i.e. short- to medium-term) over 12 hours of intervention, at a value of £1,217 per counselling intervention (i.e. per family member affected). It should be noted that this cost is applied as a representative approximation of the ‘value’ of reduced stress and caregiver burden, rather than the actual cost that would be expected to be incurred by family members/guardians experiencing increased stress. The cost has been summarised in Table 21.
Table 21. Financial proxy for stress and burden for family members/guardians
Financial Proxy Value
Cost per counselling intervention
£1,217.00
Abbreviations: PSSRU: Personal and Social Services Research Unit.
Source/Justification
PSSRU, 20214
Short to medium term intervention, ranging from 9 to 12 counselling sessions (12 hours per case)
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 23
Input Value Source/Justification
of guardians affected by participant's behaviour, before Noise Solution 100.0% Assumption Proportion of guardians affected by child’s behaviour, after Noise Solution 39.0%
Proportion
Increased labour productivity from guardians
As participants show increased engagement in pro-social activities and behaviour in the short-term, there is anticipated increased labour productivity from guardians due to increased/return to work. The value of this outcome was approximated by calculating the average number of days absent from work pre, during and post Noise Solution, and applying a pro-rata average wage for each day absent, across both guardians. This outcome is assumed only to impact participants who live with their guardians and/or need adult supervision. Therefore, the outcome was only considered for participants aged 18 and under.
In the absence of data from Noise Solution, it was assumed that pre Noise Solution, one guardian per participant was absent one day per month, resulting in an annual estimate of 12 days. For the year including and immediately after the 10-week duration of Noise Solution, in which participants attend 10 two-hour sessions, it was assumed that a guardian attended the first five sessions with participants and that this time represented lost productivity in addition to the 12 day background rate. This additional productivity loss was pro-rata adjusted to reflect the proportion of sessions that participants failed to attend (15.8%), assuming that non-attendance was equally likely between guardian-attended and guardian-non-attended sessions In the case of non-attendance, it is assumed that there is no additional loss of productivity for the guardian above the base rate of productivity loss, i.e. the guardian would have been absent from work on the given day irrespective of a planned Noise Solution session and that the attendance time is absorbed in the 12 day background rate
For both the year of the programme, and the years following that, it was assumed that the guardians of participants who show a significant improvement in well-being no longer have any absent days from work related to caring for the participant, while guardians of other participants remain at pre-Noise Solution levels. These inputs are summarised in Table 22 Productivity loss relating to care of the participant was applied in addition to productivity loss due to attendance of sessions with participants in the year of the programme. In subsequent years, only productivity loss relating to the care of the participant is considered, as guardians no longer attend the course with the participant
Table 22. Estimated loss
participant, before
Annual
absent from
caring for participant, after Noise Solution
Noise Solution estimate of one guardian
Assumes guardians attend half of Noise Solution sessions and that non-attended sessions do not result in loss of productivity
Assumes that guardians of participants who show a significant improvement in wellbeing no longer have any absent days from work, while other guardians of participants remain at pre-Noise Solution levels
Assumes that guardians of participants who show a significant improvement in wellbeing no longer have any absent days from work, while other guardians of participants remain at pre-Noise Solution levels
This outcome has been monetised by valuing the loss of productivity resulting from guardians being absent from work. The financial proxy for this outcome is the national average weekly wage in the UK (Table 23).22
Table 23.
202322
Median weekly earning of full-time employees in the UK
Abbreviations: ONS: Office for National Statistics
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 24
Input Value Source/Justification Annual numbers of days
work, caring for
Noise Solution 12.00
absent
numbers of days absent from work, caring for participant, during Noise Solution 6.02
of productivity of guardians per participant
absent from
from work one day per month Average
numbers
days
4.68
of
work,
Financial
Value Source/Justification
average weekly
£682.00 ONS,
Financial proxy for labour productivity
Proxy
National
wage
Adjustment
To avoid overestimating the impact of Noise Solution, the following adjustments were applied in the SROI model:
⬧ Deadweight: Deadweight involves the recognition that some stakeholders might experience outcomes of interest, even if Noise Solution did not intervene. Deadweight was built into the final calculations for each outcome modelled, by considering the number of stakeholders who would have experienced the outcome due to other influences, in a situation where Noise Solution had not provided the service
⬧ Attribution: Attribution involves the recognition that outcomes achieved may not entirely be due to Noise Solution’s services alone. Service users, for example, may be engaged with several organisations, providing overlapping support. Through use of a percentage, attribution estimates the proportion of impact created entirely by Noise Solution, after deducting the contribution of other services. Only this is accounted for when calculating the final value of the service in question. Please see Attribution for further information
⬧ Drop-off: Drop-off involves the recognition that outcomes may not last indefinitely. The impact of an outcome may decrease over time, or benefits may be more likely to derive from other factors. Drop-off was accounted for by applying monthly percentages to each outcome; please see Drop-Off for further information on how drop-off was calculated, and for drop-off factors
⬧ Discounting: Discounting involves the recognition that outcomes that occur in the future are valued less than those incurred immediately. In the base case, a 3.5% discount rate is assumed across all variables for year two onwards, based on the HM Treasury Green Book for discounting in the public sector.23 The model includes the flexibility for the user to vary this to 5% (more conservative) or 0% (less conservative) for all of the outcomes, or alternatively, the user can adjust it for each outcome individually
Attribution
The default values for attribution included in the model were informed through discussion with stakeholders. From the interviews conducted in the original SROI analysis, it was understood that many participants engaging with Noise Solution simultaneously receive additional support (such as counselling or therapy). However, Noise Solution is understood to act as an enabler to the effectiveness of these alternative interventions, and therefore remains a key component to the overall success of the participant. Furthermore, the majority of participants who are referred to Noise Solution have already tried alternative interventions that have not delivered the satisfactory results. As a result, it is estimated that the proportion of outcomes observed attributable to other interventions is expected to be relatively low.
In the absence of data, 25% of each outcome has been attributed to another intervention, and therefore not included in the final base case SROI calculation. The model includes the flexibility for the user to vary this to 50% (more conservative) or 0% (less conservative) for all of the outcomes, or alternatively, the user can adjust it for each outcome individually.
Drop-Off
All outcomes included in the model are expected to last two years. Estimates for drop-off after one year have been based on long-term data reported in the literature where possible. For outcomes where no data have been identified for estimating drop-off, it has assumed to be 50%.1 The model includes the flexibility for the user to vary this to 50% or 75% for all inputs (more conservative), or 0% (less conservative). Alternatively, the user can adjust this input for each outcome individually. In addition, the model also includes the flexibility to calculate the SROI assuming each outcome lasts for one year only. If this option is selected, drop-off is no longer relevant, and therefore not applied in the model. The drop-off values used in the base case are described in Table 24.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 25
Drop-off based on Impetus 2017 where 52% of NEET people remained NEET the following year.24 In the absence of data, this outcome has been conservatively assumed to only last two years.
for additional support in education and related services
Abbreviations: NEET: not in education, employment or training; SROI: social return on investment.
In absence of alternative data, dropoff has been aligned with a SROI publication of a UK school-based wellbeing programme, which assumes 50% each year.1
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 26
Outcome Annual Drop-Off Source Re-entry into education, employment
training 48%
Table 24. Annual drop-off for each outcome
or
Reduced behavioural issues/school conduct problems 50%
Reduced/avoided school exclusions 50% Reduced
50% Increased social capital/reduced isolation 50% Diverted from acute mental health admission 50% Disengagement from mental health services 50% Increased labour productivity
50% Reduced stress and
members/guardians 50%
need
from guardians (increased/returned to work)
burden for family
Sensitivity Analyses
A one-way sensitivity analysis was conducted on the base case results to identify the inputs which the SROI was most sensitive to. Sensitivity analysis was conducted through varying the inputs used to calculate each outcome one at a time, with an upper and lower value. The impact on the SROI ratio was subsequently recorded, in order to identify the top ten variables that had the greatest impact on the SROI. The results of the sensitivity analysis are presented in a tornado plot in the model.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 27
Assumptions and Limitations
General Limitations of SROI Methodology
Every economic model is a simplification of events happening in the real world, and thus is subject to assumptions and limitations. However, while assumptions and limitations are an inherent part of economic modelling, the model itself can still produce useful and reliable results if limitations are addressed, or results are interpreted within the constraints of those limitations.
SROI analyses are subject to several inherent limitations. First, SROI methodology attempts to quantify and monetise the non-tangible impact of an organisation. The process of assigning financial values to nonfinancial outcomes is inevitably associated with a degree of uncertainty.1 Therefore, it is important to be cautious when drawing detailed conclusions from the analysis. Moreover, the lack of a universally accepted approach to SROI modelling means that few meaningful conclusions can be drawn from the comparison of different organisations’ final SROI ratios. Nonetheless, when used in the right context, SROI analysis is a valuable tool to highlight the breadth and depth of impact of the activities of Noise Solution on its stakeholders in a granular and semi-quantitative way.
Limitations of the Noise Solution SROI Model
There was a lack of sufficient evaluative data to fully capture the impact of the Noise Solution programme. In most cases, exact inputs were not available and input-specific assumptions had to be made. Where assumptions have been made for model input values, there were informed by comparable figures, published evidence and discussions with Noise Solution. Therefore, any model input that relies on assumptions is presumed to be close to the true value and any impact on model results is anticipated to be minor. Furthermore, conservative values were used to minimise the risk of inflating the SROI results, and assumptions were validated by Noise Solution. Noise Solution are currently working to update the information captured about their participants in referrals and their customer relationship management platform, in order to better understand the journey of the individuals that join their programme. Once these data becomes available, the SROI model can better reflect the population.
The lack of sufficient data also meant that certain outcomes and stakeholder groups had to be excluded from the analysis. For example, a significant improvement in well-being has been observed from participation in the programme. However, the paucity of available quantitative data and a lack of a robust way to incorporate this in the SROI model led to exclusion of this outcome from the analysis. Consequently, these limitations mean that the final SROI values are likely to be an underestimate due to the use of conservative assumptions and the exclusion of certain outcomes.
Other limitations of the model include its limited ability to capture the value that beneficiaries of the programme were able to pass on. For example, participants of Noise Solution may have been able to share their experience with peers facing a similar situation and therefore introduce them to the programme. It is expected that impact generation occurred beyond that captured in the model, and this impact is likely to continue to grow beyond the model time horizon as knowledge and expertise continues to spread. However, quantification of the value of these relationships comes with inherent uncertainty and would also require an element of forecasting, which was beyond the scope of this analysis.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 28
References
1. Nicholls J, Lawlor E, Neitzert E, et al. A Guide to Social Return on Investment. Available at: https://socialvalueuk.org/resources/a-guide-to-social-return-on-investment-2012/ Last accessed: Feruary 2024: The SROI Network, 2012.
2. Shah N, Cader M, Andrews WP, et al. Responsiveness of the Short Warwick Edinburgh Mental WellBeing Scale (SWEMWBS): evaluation a clinical sample. Health and Quality of Life Outcomes 2018;16:239.
3. Department for Work and Pensions. Benefit and pension rates 2023 to 2024. Retrieved from: https://www.gov.uk/government/publications/benefit-and-pension-rates-2023-to-2024/benefit-andpension-rates-2023-to-2024. Last accessed: February 2024, 2023.
4. Personal Social Services Research Unit (PSSRU). Unit costs of health and social care 2021. Retrieved from: https://www.pssru.ac.uk/project-pages/unit-costs/unit-costs-of-health-and-social-care-2021/ Last accessed: February 2024, 2021.
5. Gormley MJ. The Time and Cost of Classroom Behavior Management. Presented at the National Association of School Psychologists Annual Convention. 2021.
6. Gov.uk. National pay scales for eligible teaching and education leadership occupation codes. Retrieved from: https://www.gov.uk/government/publications/national-pay-scales-for-eligibleteaching-and-education-jobs/national-pay-scales-for-eligible-teaching-and-education-leadershipoccupation-codes. Last accessed: February 2024, 2023.
7. Brookes M, Goodall E, Heady L. Misspent Youth - The Costs of Truancy and Exclusion. Available at: https://www.thinknpc.org/wp-content/uploads/2018/07/Misspent-youth-report.pdf Last accessed: Feruary 2024. New Philanthropy Capital, 2007.
8. Collinson D. The mental health of children and young people in England. Retrieved from: https://digital.nhs.uk/data-and-information/publications/statistical/mental-health-of-children-andyoung-people-in-england/2017/2017. Last accessed: February 2024. In: National Centre of Social Research, Office for National Statistics, eds, 2017.
9. Romeo R, Knapp M, Scott S. Economic cost of severe antisocial behaviour in children - and who pays it. British Journal of Psychiatry 2006;188:547–553.
10. Education Endowment Foundation. Teaching and learning toolkit. Available at: https://educationendowmentfoundation.org.uk/education-evidence/teaching-learning-toolkit Last accessed: February 2024, 2018.
11. National Careers Service. Special educational needs (SEN) teaching assistant. Available at: https://nationalcareers.service.gov.uk/job-profiles/special-educational-needs-(sen)-teaching-assistant. Last accessed: February 2024, 2023.
12. Office for National Statistics. Mental Health of Children and Young People in England, 2023: Wave 4 follow up to the 2017 survey. Retrieved from: https://digital.nhs.uk/data-andinformation/publications/statistical/mental-health-of-children-and-young-people-in-england/2023wave-4-follow-up/part-1-mental-health. Last accessed: February 2024, 2023.
13. Department for Digital, Culture, Media & Sport. Loneliness monetisation report. Available at: https://assets.publishing.service.gov.uk/media/602fcb91d3bf7f72154fabc3/Loneliness_monetisation_r eport_V2.pdf Last accessed: February 2024, 2020.
14. Children's Commissioner. Briefing: Children’s Mental Healthcare in England. Available at: https://www.childrenscommissioner.gov.uk/resource/briefing-childrens-mental-healthcare-in-england/ Last accessed: February 2024, 2017.
15. Personal Social Services Research Unit (PSSRU). Unit costs of health and social care 2022. Retrieved from: https://www.pssru.ac.uk/project-pages/unit-costs/. Last accessed: February 2024, 2022.
16. East London NHS Foundation Trust. FOI DA3620. Available at: https://www.elft.nhs.uk/sites/default/files/2022-01/anon_response_-_foi_da3620.pdf Last accessed: February 2024, 2021.
17. The Strategy Unit. Exploring Mental Health Inpatient Capacity across Sustainability and Transformation Partnerships in England. Available at: https://www.strategyunitwm.nhs.uk/sites/default/files/201911/Exploring%20Mental%20Health%20Inpatient%20Capacity%20accross%20Sustainability%20and %20Transformation%20Partnerships%20in%20England%20-%20191030_1.pdf Last accessed: Feruary 2024, 2019.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 29
18. National Health Service (NHS). Benchmarking Mental Health. Available at:
https://static1.squarespace.com/static/58d8d0ffe4fcb5ad94cde63e/t/58ecf71de58c62adea37fa27/149 1924766551/BenchmarkingMentalHealthCard2017FINAL.pdf Last accessed: Feruary 2024, 2017.
19. Meltzer H, Gatward R. The mental health of children and adolescents in Great Britain. Available at: https://www.dawba.info/abstracts/B-CAMHS99_original_survey_report.pdf Last accessed: February 2024. In: Office for National Statistics, ed, 1999.
20. Office for National Statistics. Families with dependent children by number of children, UK, 1996 to 2017. Retrieved from:
https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/families/adhocs/0 08855familieswithdependentchildrenbynumberofchildrenuk1996to2017. Last accessed: February 2024, 2018.
21. Office for National Statistics. Families and households: 2022. Retrieved from:
https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/families/bulletins/ familiesandhouseholds/previousReleases. Last accessed: February 2024, 2022.
22. Office for National Statistics. Employee earnings in the UK: 2023. Retrieved from: https://www.beta.ons.gov.uk/employmentandlabourmarket/peopleinwork/earningsandworkinghours/b ulletins/annualsurveyofhoursandearnings/2023#:~:text=Median%20weekly%20earnings%20for%20f ull-
time%20employees%20was%20%C2%A3682,highest%20growth%20since%20comparable%20recor ds%20began%20in%201997. Last accessed: February 2024, 2023.
23. HM Treasury. The Green Book: Central Government Guidance on Appraisal and Evaluation. Retrieved from:
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1 063330/Green_Book_2022.pdf Last accessed: February 2024, 2018.
24. Impetus Private Equity Foundation. Youth Jobs Index. Available at: https://impetusorg.files.svdcdn.com/production/assets/publications/Report/Youth-Jobs-Index-II-report-final.pdf Last accessed: February 2024, 2017.
SROI MODEL METHODOLOGY Copyright © Costello Medical Consulting Ltd | 30