Analyses of 2024 SWEMWBS scores by Costello Medical by noisesolutionuk

Analyses of SWEMWBS Scores from Noise Solution

Costello Medical UK | US | Singapore | China www.costellomedical.com NOISE SOLUTION 11 APRIL 2024

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA Contents List of Figures................................................................................................................3 List of Tables.................................................................................................................3 Glossary.........................................................................................................................4 Context..........................................................................................................................5 Objective.......................................................................................................................6 Methods.........................................................................................................................6 Data Sources 6 Analysis Approach..................................................................................................................................6 Software 6 Data Cleaning and Transformation.......................................................................................................6 Data Summaries.................................................................................................................................8 Analyses.............................................................................................................................................8 Results.........................................................................................................................10 Summary Statistics...............................................................................................................................10 Demographics...................................................................................................................................10 Analyses..............................................................................................................................................10 Change in SWEMWBS scores.............................................................................................................12 Meaningful Changes in SWEMWBS Scores 14 Transitions in Well-Being Bands.........................................................................................................14 Fujiwara Social Value Change 16 Discussion....................................................................................................................17 Limitations and Cautions.......................................................................................................................18 References...................................................................................................................19 Appendix......................................................................................................................20 Methodology 20 Statistcal Results Summary...................................................................................................................21 Additional Results 23 Copyright © Costello Medical Consulting Ltd |

List of Figures

Figure 1. Individual changes in SWEMWBS score after participation in the Noise Solution programme, ranked by magnitude of changea..........................................................................................................................13

List of Tables

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Figure 2. SWEMWBS scores at the start and end of Noise Solution participation, by gendera,b 13 Figure 3. Transitions between high, medium and low well-being categoriesa,...............................................15

4. SWEMWBS scores as a function of age, at the start and end of the Noise Solution programmea 23

5. Change

SWEMWBS score as a factor of

agea

6. Change

score

as a

of SWEMWBS score at the start of the

Figure 7. The number of participants in each well-being band at the start and end of the Noise Solution programme, by age and gendera, b............................................................................................................25 Figure 8. Transitions in Fujiwara (2017) SWEMWBS value categories8 26

Figure

participant

...........................................................23 Figure

in SWEMWBS

a following participation in the Noise Solution programme

factor

programme........................................................................................24

Table 1. Research questions.......................................................................................................................6 Table 2. SWEMWBS well-being band definitions..........................................................................................7 Table 3. Fujiwara (2017)8 categories with associated SWEMWBS scores and social values..............................8 Table 4. Summary of analyses performed 9 Table 5. Ages of Noise Solution Participants, by gender.............................................................................10 Table 6. Summary of analyses results 11 Table 7. SWEMWBS scores at the start and end of the Noise Solution programme, and change in SWEMWBS scores, by gender 12 Table 8. Proportions of participants showing positive, negative and no meaningful change in SWEMWBS scores, and their respective changes in score 14 Table 9. Changes in value for participants, by increase type and gender.....................................................16 Table 10. R package used for research questions 1–6 20 Table 11. Results of analyses...................................................................................................................21 Table 12. Transitions between high, medium and low well-being SWEMWBS bandsa 24 Copyright © Costello Medical Consulting Ltd |

Glossary

Technical terms used throughout the report are defined here. Please also refer to footnotes in individual tables and figures to support interpretation.

Term Definition

Alpha

Centring

Intercept

Linear model

Link function

Log10

Logistic model

Mean

Median

Mixed model

Multinomial model

P value

Paired t test

Quartile 1 and Quartile 3

Standard Deviation (SD)

Variable

Pre-specified statistical significance threshold

Adjusting data values by subtracting the mean (average) from each value, shifting the average to zero. This technique simplifies analysis by aligning the data around a common starting point

The value of the dependent variable when all other variables are set to zero

A statistical model that presents the relationship between an outcome (or outcomes) of interest (dependent variable[s], such as change in SWEMWBS score) and other factors that might be associated with the outcome of interest (independent variables, such as age and gender)

A tool used in some statistical models to ensure that the model’s predictions stay within a range reflective of the real-world. Different link functions, such as logit or identity, are used depending on the type of data/their values

Data may be transformed using this logarithm to make them more suitable for analysis using linear models. To convert from Log10(value) to the absolute value, calculate as absolute value = 10Log10(value)

A statistical model that estimates the likelihood of a binary outcome (such as yes/no) by fitting data to a curve instead of a straight line (as in linear models)

The most commonly used type of average-of-all-values in a dataset, calculated as the sum of all the datapoints divided by the number of datapoints. This type of average is affected by extreme values

The middlemost value in a dataset, or half-way between the two middlemost values when there is an even number of values or entries in a dataset. This type of average is not biased by extreme values or outliers and so is often used when these are present in a dataset

A statistical modelling approach (applied in some instances here to both linear and logistic models) that includes both fixed factors (effects which are consistent across individuals or groups) and random factors (effects which vary across individuals or groups, representing natural variability or differences not explained by the fixed effects)

A statistical model that predicts outcomes where there are more than two possible categories, such as SWEMWBS meaningful change category (e.g. low, medium and high)

The number of datapoints in a given group. For instance, the overall N of participants in the SWEMWBS analysis dataset was 374

The number of datapoints in a subgroup, such as the number of male or female participants in the analysis

Represents the probability that a pattern in the data (i.e. the observed effect of a given variable) is due to chance. For instance, whether differences in SWEMWBS scores between genders are likely to be ‘real’ or due to random chance. Higher p values mean a real effect is less likely; a value <0.05 is considered statistically significant, whereby we interpret the observed effect as real

A statistical test used to determine if there is a significant difference between the means of two related groups, such as participants’ SWEBWBS scores before and after Noise Solution participation

The values under which 25% and 75% of datapoints are found, respectively, when they are arranged in increasing order. These values provide a measure of variation that is not biased by extreme values or outliers and is usually presented alongside the median

A measure of how much data are spread out around the mean, alongside which it is usually reported

Anything that can take on different values, such as age, gender or starting SWEMWBS score (independent variable), on which a measured outcome of interest (dependent variable, such as change in SWEMWBS score) might be dependent upon

Context

Noise Solution’s programme aims to instil feelings of autonomy, competence and relatedness; evidenced in Self Determination Theory as the basic psychological needs for well-being to flourish.1 Their 1:1 music mentoring programmes gives individuals facing a variety of challenging circumstances time to work on and direct a creative project that uses a combination of music technology and instruments. The participant can share their success with friends, family and professional keyworkers in a way that looks and feels like a social media feed, inviting their network to comment on the digital story. In this way, the programme creates a cycle of positive affirmation from people whose opinions matter to the young person.

The result of programme participation is typically improved participant well-being, quantitatively measured using a National Health Service validated scale (Short Warwick-Edinburgh Mental Well-being Scale [SWEMWBS]), a questionnaire which participants are encouraged to self-complete at the start and end of the programme. The SWEMWBS uses seven statements regarding thoughts and feelings, which focus primarily on functioning, and are positively worded with five response categories ranging from ‘none of the time’ to ‘all of the time’.2, 3 Using these scales, children and young people are asked to describe their lived experiences over the past two weeks. Importantly, improved well-being has been empirically shown to improve education, engagement, social and health (both mental and physical) outcomes, ultimately impacting individual quality of life and reducing the financial burden on local and governmental services.4-6 A minimum important change threshold of ±1 point in SWEMWBS score has been reported and may be considered to constitute a meaningful change (positive or negative),3 whilst scores ±1 standard deviation either side of the UK national average may be considered to represent categories of ‘high’ and ‘low’ well-being, respectively.2, 7

Over time, Noise Solution have accrued a sizeable SWEMWBS dataset. This dataset was used in the social return on investment (SROI) model, developed by Costello Medical for Noise Solution, to inform various model estimates (further details are available in a separate report). Furthermore, based on the work of Fujiwara et al. who developed a method for estimating the social value created by interventions which improve people’s mental health and subsequent well-being (as measured via SWEMWBS), it is possible to ascribe an estimated financial value to the improvements in well-being demonstrated by Noise Solution participants.8 These latter estimates, however, were not considered appropriate to integrate within the SROI model, since the methods of Fujiwara et al. produce an indirect estimate of ‘how much additional money or income would be required to have the same impact on well-being as a change in the SWEMWBS score’ and not an actual estimate of cost saving based on the downstream effects of such a change. Instead, these values can be calculated separately to provide an alternative financial indicator of the beneficial effects of Noise Solution’s programme on well-being.

Given the richness of Noise Solution’s available dataset, there existed an opportunity to pursue further analyses, including statistical interrogation of the observed effects of Noise Solution’s programme and application of the Fujiwara method to estimate an alternative financial indicator for changes in subjective well-being. Therefore, the purpose of this report is to present the results and underlying methodology of a suite of statistical analyses of Noise Solution’s available SWEMWBS dataset.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Objective

The objective of the analyses was to answer the research questions outlined in Table 1

Table 1. Research questions

Question

1 Are male and female participants of similar average age?

2 How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

3 How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

4 How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

5 How does age and gender affect the well-being band of participants, and does the well-being band that participants occupy at the start of the programme differ from the band that they occupied at the end?

6 For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the programme?

7 What is the social value change associated with change in SWEMWBS scores between the start and the end of the programme?

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Methods

Data Sources

The core dataset comprised data collected from 374 individual Noise Solution participants who completed the programme. Age and gender (male; female; non-binary; other; prefer not to say) were collected from participants, who could withhold information if so desired. Participants also completed the standard SWEMWBS questionnaire,7 from which pre- and post-Noise Solution programme participation SWEMWBS scores were derived.

Analysis Approach Software

All data manipulation, visualisation and analysis was conducted in R, version 4.3.0.9

Data Cleaning and Transformation

A small number (n=11) of participants included in the dataset had attended the Noise Solution programme multiple times (maximum rounds: 3); for each of these participants, only the first round of participation was retained and subsequent rounds were excluded. This removed dependence between points and any effect of multiple rounds of participation on scores. One participant was additionally removed as the recorded age (4 years) was confirmed by Noise Solution to be erroneous and the validity of other data for this individual was therefore uncertain.

A small number of participants declined to provide information on gender (n=1), or stated ‘other’ (n=3) or ‘non-binary’ (n=1). These groups were too small to treat separately in analyses and all such responses were therefore replaced with ‘not applicable’ in order to group them alongside those participants who provided no information on gender.

For all statistical analyses, where age and SWEMWBS scores were used as predictors, these variables were centred. Doing so ensured that the intercept for models was at the mean value, not at zero. This was

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

considered more meaningful, because neither age nor SWEMWBS scores would ever equal zero in this dataset and thus centring prevented the model from predicting zero values.

Based on the previously reported minimum important difference, a change in SWEMWBS score (ΔSWEMWBS score) of at least one point increase or decrease between the start and end of the Noise Solution programme was considered a meaningful change.3 Participants were thus categorised according to the meaningful change category into which they fell:

 Up: a meaningful increase in score; ΔSWEMWBS score ≥1

 Down: a meaningful decrease in score; ΔSWEMWBS score ≤1

 None: no meaningful change in score; 1> ΔSWEMWBS score <1

Participants were also mapped to well-being bands at the start and end of the Noise Solution programme, based on their respective SWEMWBS scores. Following the methods of Fat et al. (2016), these bands represent functionally important levels of well-being and were defined using the national average statistics for SWEMWBS scores (mean: 23.5; SD: 3.9).2, 7 These data were used so that Noise Solution participants could be put into the context of broader society. Bands were defined as follows in Table 2.

Footnote: Mean and SD refer to the national mean (23.5) and national SD (3.9).2, 7

Abbreviation: SD, standard deviation; SWEMWBS, short Warwick-Edinburgh mental well-being scale. SWEMWBS scores were mapped to social value changes, following the methods of Fujiwara (2017), who used linear modelling to attribute social value to categories formed from discrete ranges of SWEMWBS scores (see Table 3).8 Participants’ start and end SWEMWBS scores were rounded down (to be conservative) to the nearest whole number, allocated to the Fujiwara categories and attributed the appropriate social value; changes in value were then calculated as the starting value (pre Noise Solution) subtracted from the ending value (post Noise Solution).

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Band Definitiona SWEMWBS Score Threshold Low SWEMWBS score < (Mean – SD) ≤19.5 Medium (Mean – SD) > SWEMWBS score < (Mean + SD) >19.5–<27.5 High SWEMWBS Score > (Mean + SD) ≥27.5

Table 2. SWEMWBS well-being band definitions

Table 3. Fujiwara (2017)8 categories with associated SWEMWBS scores and social values

Abbreviation: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Data Summaries

Summary statistics were calculated from the dataset; missing values were excluded from calculations of averages and associated variation.

Analyses

The analyses conducted on the dataset are outlined in Table 4. A paired t-test was used to compare ages between male and female participants. For the remaining analyses, more complex models were constructed to incorporate estimates for the effects of multiple variables at once, and to accommodate different response variable statistical distributions. Model types and link functions were specified based on the expected statistical distribution of the response variable. Across all research questions, an alpha (significance threshold) of 0.05 was used; consequently, any p value below this value was considered to represent a ‘statistically significant’ observed effect. Further details of the analyses’ methodology are available in the Appendix.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Category SWEMWBS Score Social Value (£) 1 7–14 0 2 15–16 9,639 3 17–18 12,255 4 19–20 17,561 5 21–22 21,049 6 23–24 22,944 7 25–26 24,225 8 27–28 24,877 9 29–30 25,480 10 31–32 25,856 11 33–34 26,175 12 35 26,793

Table

Summary of analyses

1 Are male and female participants of similar average age? Age

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

How does age and gender affect the well-being band of participants, and does the well-being band that participants occupy at the start of the programme differ from the band that they occupied at the end?

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the programme?

SWEMWBS Scores (start and end of programme)

Change in SWEMWBS Scores (from start to end of programme)

SWEMWBS meaningful change category (up, down or none)

SWEMWBS wellbeing band (low or non-low; from start to end of programme)

SWEMWBS wellbeing band after programme for participants in the low band at the start

Gender, age,a start versus end of Noise Solution programme

Gender, age,a starting SWEMBWBS score

Gender, age,a starting SWEMWBS score

Gender, age,a start versus end of Noise Solution programme

Linear mixed model (identity link)

Linear model (identity link)

Multinomial model (logit link)

Logistic mixed model (logit link)

Gender, agea Multinomial model (logit link)

Footnote: Log10 participant age was used, as this ensured that the distribution is suitable for linear modelling. Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

performed Research Question Dependent Variable Variables Model Fitted/Test Performed

Gender

t-test

Paired

Results

Summary Statistics

Demographics

The distribution of Noise Solution participants by age and gender is presented in Table 5. The majority (67.1% of all participants) of participants were male. The median age of all participants was 14 years, ranging between 8 and 60 years old. Although the median age of females was higher than males, this was not statistically significant (p=0.13). The vast majority were legally children, being less than 18 years old (85.6% of all participants), and 71.7% of all participants were less than 16 years old.

Table 5. Ages of Noise Solution Participants, by gender

Footnote: a This category includes participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1); sample size was too small to produce meaningful analyses for these groups; b i.e. legally considered children and of school age (% adult can be calculated as 100 – %<18 years old).

Analyses

A summary of the analyses results are presented in Table 6. Further details of the results of each research question are provided thereafter, and detailed results of the statistical tests are provided in the Appendix

Gender N Proportion of N (%) Age (Years) Proportion (%) Median Minimum Maximum Quartile 1 Quartile 3 <18b <16 All 374 100.0 14 8 60 13 16 85.6 71.7 Female 85 22.7 15 9 60 13 17 82.3 62.0 Male 251 67.1 14 8 53 12 16 86.1 73.0 Othera 38 10.2 14 11 25 13 15 89.5 84.2

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

Gender

Age

Starting score

Gender

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the Noise Solution programme?

Age

Starting score

Start versus end of programme

Gender

Age; start versus end of programme

Gender

Age

No evidence for gender affecting change in SWEMWBS score between the start and end of the programme, i.e. males and females increased by a similar amount, on average (p=0.08)

Older participants have significantly larger positive differences in SWEMWBS scores between the start and end of the programme than younger participants (p=0.02)

Participants with higher-than-average starting SWEMWBS scores are likely to have smaller differences, or even reductions, in SWEMWBS scores between the start and end of the programme compared with participants with average or lower-than-average starting SWEMWBS scores (p<0.0001)

No evidence for gender affecting meaningful change category between the start and end of the programme (negative change: p=0.42; no meaningful change: p=0.80)

No evidence for age affecting meaningful change category between the start and end of the programme (negative change: p=0.36; no meaningful change: p=0.06)

The higher the starting SWEMWBS scores of participants, the more likely they are to have no meaningful change (p<0.001), or a negative meaningful change (p<0.0001) compared with a positive meaningful change between the start and end of the programme

Participants are significantly more likely to be in the low well-being band than the non-low well-being bands at the start of the programme compared with the end of the programme (p<0.001)

Females are significantly more likely than males to be in the low well-being band compared with the non-low well-being band (p<0.001)

The older a participant, the more likely they are to be in the low well-being band than the non-low well-being band at the start of the programme, but this relationship is not present at the end of the programme (p<0.0001)

No evidence for gender affecting transitions from the low well-being band to the medium (p=0.52) or high (p=0.14) well-being bands between the start and end of the programme

No evidence for age affecting transitions from the low well-being band to the medium (p=0.26) or high (p=0.22) well-being bands between the start and end of the programme

Footnote: a Green signifies the variable having a statistically significant effect on the result; yellow signifies a lack of statistical significance.

Abbreviations: SE, standard; SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Table 6.

Research Question Variable Resulta

Were male and female

of similar average age? Age No evidence for a difference in age between sexes (p=0.13)

Summary of analyses results

participants

Start versus end of programme Significant increase in SWEMWBS scores after programme compared to the start of the programme (p<0.0001) Gender Females have significantly lower SWEMWBS scores than males at the start and end of the programme (p<0.001) Age; start versus end of programme Older participants have significantly lower SWEMWBS scores than younger participants at the start of the programme, but not at the end of the programme (p=0.01)

Ltd

Costello Medical Consulting

Change in SWEMWBS scores

The average change in SWEMWBS score following Noise Solution participation was an increase of 2.5 points (SD: 4.32); this varied between 1.83 and 2.80 across genders (Table 7).The individual changes in SWEMWBS score are presented in Figure 1, which demonstrate that whilst the majority of individuals experienced an increase in SWEMWBS score, some participants experienced no change or a decrease in score. The overall change in SWEMWBS score after Noise Solution participation across all participants was shown to be statistically significant (p<0.0001; Figure 2).

At both the start and end of the programme, average participant SWEMWBS scores were lower than the national average score of 23.5.7 Females had significantly lower SWEMWBS scores, on average, at both the start and end of the programme, than males (p<0.001; Figure 2). However, the changes in SWEMWBS scores between the start and end of the programme were similar for males and females. Those in the ‘other’ gender category had the lowest SWEWMBS scores but represented a mixed group with a small sample size, limiting interpretability of this finding.

Table 7. SWEMWBS scores at the start and end of the Noise Solution programme, and change in SWEMWBS scores, by gender

Abbreviations: SD: standard deviation; SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Gender Start or End of Programme SWEMWBS Score Change in SWEMWBS Score Mean SD Mean SD All Start 20.34 3.93 2.50 4.32 End 22.83 4.41 Female Start 18.76 4.13 2.80 4.50 End 21.56 4.15 Male Start 20.95 3.86 2.49 4.42 End 23.44 4.55 Othera Start 19.85 2.84 1.83 3.05 End 21.68 3.05

Figure 1. Individual changes in SWEMWBS score after participation in the Noise Solution programme, ranked by magnitude of changea

Footnote: a The broken orange lines indicate thresholds for meaningful changes in SWEMWBS scores, where a change of at least 1 SWEMWBS point (increase or decrease) was considered meaningful.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Figure 2. SWEMWBS scores at the start and end of Noise Solution participation, by gendera,b

Footnote: a the central, horizontal line in the box plot indicates the median value, the box horizontals indicate the first and third quartiles, and the whiskers indicate the range (excluding outliers). Outliers (closed circles on figure) are defined as values greater than 1.5 times the extent of the third quartile, or less than 1.5 times the extent of the first quartile; b participants for whom gender was not known, as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1), were excluded from analyses; sample size was too small to produce meaningful analyses for these groups.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Older participants had significantly lower scores at the start of the programme compared to younger participants (see Appendix, Figure 4). However, at the end of the programme, there was no correlation between age and SWEMWBS score, meaning that older participants had a larger positive increase in score than younger participants (see Appendix, Figure 5).

Participants with a higher SWEMWBS score at the start of the programme had a smaller increase in their score following the Noise Solution programme compared with participants with lower SWEMWBS scores. For individuals who experienced a decrease in SWEMWBS score following the Noise Solution programme, participants with a higher initial SWEMWB scores had a larger decrease in score compared with participants with a lower initial SWEWMBS score (see Appendix, Figure 6).

Meaningful Changes in SWEMWBS Scores

The majority (61.0%) of participants showed a meaningful increase in SWEMWBS score, whilst 17.6% experienced a meaningful decrease and 21.4% showed no meaningful change (Table 8). Individuals with higher-than-average starting SWEMWBS scores had a statistically significant increase in likelihood of seeing a negative meaningful change (p<0.0001) or no meaningful change (p<0.001) compared with a positive meaningful change. Thus, participants with lower starting SWEMWBS scores were more likely to experience meaningful increase in SWEBWBS score after participation in the Noise Solution programme. Neither gender nor age was a significant predictor of meaningful change category.

Table 8. Proportions of participants showing positive, negative and no meaningful change in SWEMWBS scores, and their respective changes in score

Footnote: a A change of at least 1 SWEMWBS point was considered meaningful. Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Transitions in Well-Being Bands

The transitions in well-being bands between the start and end of the Noise Solution programme have been presented visually in and numerically in the Appendix (see Table 12). At the start of the Noise Solution programme, the majority of the population were in the low and medium well-being SWEMWBS bands (43.42% and 53.74%, respectively). Following the programme, 59.36% of participants did not change well-being band and 34.76% experienced an improvement in their band, resulting in a general shift towards higher well-being bands and away from the low band. However, some participants did decrease band (5.88%), including 0.27% moving from high to low and 4.81% from medium to low. Overall, participants were significantly less likely to be in the low well-being band at the end of the programme compared to the start (p<0.001).

Well-being bands were also compared between genders and ages (see Appendix, Figure 7). Females were significantly more likely than males to be in the low well-being band compared with other bands (p<0.001) Older participants were also more likely to be in the low well-being band compared to other ages (p<0.0001), however this was only found at the start of the programme. Among those participants who started in the low well-being band, age and gender did not affect the band which they were in at the end of the programme.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Value Meaningful Change Categorya Positive Negative None Proportion of population (%) 61.0 17.6 21.4 Number of individuals 228 66 80 Mean SWEMWBS score change 4.96 −3.06 0.07 Standard deviation of SWEMWBS score change 3.42 2.42 0.58 Mean SWEMWBS starting score 19.24 23.11 21.17 Median SWEMWBS starting score 19.25 23.21 20.73

Figure 3. Transitions between high, medium and low well-being categoriesa,

Footnote: a Box size is proportional to the number of participants in each category before and after the Noise Solution programme, and arrow width is proportional to the number of participants experiencing a given transition (see Table 12); b methodology based on Fat et al. (2016).2

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Fujiwara Social Value Change

Across all participants, the average Fujiwara SWEMWBS value change per participant was +£3,478.48 (Table 9), corresponding to transitions to nearby categories (see Appendix, Figure 8). However, some participants experienced extreme value changes with increases as great as +£25,856.00 and declines of as much as –£21,049.00, corresponding to extreme positive or negative transitions. The average value change was also found to be higher for females (+£4,645.51) than males (+£3,146.61).

Table 9. Changes in value for participants, by increase type and gender

Footnote: a Defined as the amount of additional money or income required to have the same impact on well-being as the corresponding change in the SWEMWBS category, following methods by Fujiwara 2017;8 b defined as all participants showing a change in SWEMWBS score >0; c this category includes participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1); sample size was too small to produce meaningful analyses for these groups.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Increase Type Gender Value Change (£)a Mean SD Median Quartiles Maximu m Minimum 1 3 All All 3,478.4 8 6,057.8 0 2,912.0 0 0.00 7,316.00 25,856.00 −21,049.0 0 Female 4,645.5 1 6,878.6 7 3,488.0 0 0.00 10,689.0 0 24,877.00 −12,255.0 0 Male 3,146.6 1 5,900.9 3 2,536.0 0 0.00 5,383.00 25,856.00 −21,049.0 0 Otherc 3,060.1 1 4,815.8 2 2,896.0 0 0.00 5,306.00 15,238.00 −5,306.00 Positive score change onlyb All 6,391.3 5 4,898.8 7 5,306.0 0 2,912.0 0 9,639.00 25,856.00 0.00 Female 7,699.0 3 4,985.4 0 8,358.0 0 3,254.0 0 11,229.7 5 24,877.00 0.00 Male 5,761.7 1 4,950.9 5 3,838.5 0 2,536.0 0 7,922.00 25,856.00 0.00 Otherc 6,118.9 1 3,917.6 8 5,306.0 0 3,488.0 0 8,576.00 15,238.00 0.00

Discussion

SWEMWBS data collected by Noise Solution from a total of 374 participants who had participated in their programme were analysed to investigate possible patterns and trends with regards to the effectiveness of the programme. Overall, the majority of participants identified as male (67% of all participants versus 22% identifying as female), however the age of participants was balanced between male and female, with an average age of approximately 14 years. 85.6% of all participants were under 18 years of age when they first started participating. At the start of the programme, mean SWEMWBS scores were typically markedly lower than the national average (20.34 versus a national average of 23.5),2, 7 with males having significantly higher SWEMWBS scores than females (average: 20.95 versus 18.76, respectively; p<0.001). In population-level reports of well-being, there does not seem to be the same discrepancy between males and females, 10 suggesting that females attending the Noise Solution programme differ from the wider population in typically having lower starting well-being than males who attend the programme. Starting SWEMWBS scores were also typically lower in older participants.

Overall, participants had significantly higher SWEMWBS scores at the end of the programme than at the start, such that they were only slightly below the national average (overall average end score: 22.83; mean change: 2.50; p<0.0001). Of all participants, 61.0% experienced a meaningful improvement of at least one SWEMWBS point, whilst 21.4% did not experience a meaningful change; only a small number of participants showed meaningful decreases in SWEMWBS score after participation (17.6%). Akin to the difference seen at the start of the programme, at the end of the Noise Solution programme, males still had significantly higher SWEMWBS scores than females (average: 23.44 versus 21.56, respectively; p<0.001), however the average level of improvement was similar between genders (mean score change in males: +2.50; females: +2.80). Older participants experienced significantly greater increases in SWEMWBS score than younger participants (p=0.02), offsetting their lower starting values, such that there was no significant effect of age on SWEMWBS score at the end of the programme.

The magnitude and direction of change in SWEMWBS score was significantly linked to starting score (p<0.0001), such that those with higher starting scores tended to have smaller positive, and occasionally negative, changes in SWEMWBS by the end of the programme compared with individuals with lower starting scores. Applying bands of well-being (‘high’, ‘medium’ or ‘low’ based on participants having SWEMWBS scores greater than, within, or lower than one standard deviation of the national average), overall, participants tended to increase (34.8%) or maintain (59.4%) their pre-Noise Solution well-being band, with only a small proportion experiencing a negative band change (5.9%). This led to an overall substantial increase in the proportion of participants in the ‘high’ well-being band by the end of the programme, and a reduction of those in the ‘low’ band, whilst the proportion in the ‘medium’ band had a slight increase. There was no effect of age or gender on the ability of participants to transition away from the low well-being band.

When applying the Fujiwara well-being financial proxy to the dataset, on average, the Noise Solution programme provided substantial net value gain (+£3,478.48), and this was higher for females (+£4,645.51) than males (+£3,146.61), however some participants experienced a negative value change. Participants typically transitioned to nearby well-being value categories, however some individuals experienced more extreme changes. Consistent with the findings for well-being bands, the greatest increases in value categories were most often seen in those who started in the lower categories, whilst those who started in the highest categories more often did not transition or transitioned in a slightly lower category.

The present analyses demonstrate a marked and statistically significant overall effect of the Noise Solution programme on participant well-being, as measured by the SWEMWBS. Furthermore, application of the Fujiwara well-being financial proxy complements the findings of the SROI analysis, demonstrating an alternative, but similarly favourable, monetary-equivalent value of the programme. Future investigation may be warranted to try and understand why older participants seem to respond better to the programme.

Limitations and Cautions

The data analysed in this suite of analyses were not from a randomised sample and no control group or condition exists. Thus, all results presented within this report must be interpreted with caution and without extrapolation to wider populations. Most importantly, it should be reminded that observed relationships (associations) may not be causal in nature and may instead be attributed to unmeasured factors outside of the Noise Solution programme. Specifically, it is important to note that only limited demographic data were available for participants; other key factors known to affect SWEMWBS scores and associated major categories, such as ethnicity and diet (amongst many other possible factors)2 were not available and hence could not be included in the analyses. This may affect the results surrounding the variables we do have, as effects may be dependent on unavailable characteristics, which could mask true effects, or create false effects.

Furthermore, caution must be applied in the interpretation of findings. For instance, reported trends are average trends; there is substantial variation around such data, so they are not deterministic (e.g. a participant with a low starting score is more likely to see a larger increase in end score, but this cannot be said with certainty). Without a matched control group or condition, the overall effect of Noise Solution cannot be separated from any background processes. For example, some participants showed reduced SWEMWBS scores at the end of the programme, but it is impossible to know whether this was related to Noise Solution participation (i.e. the programme was detrimental to the individual), whether participation failed to stop a reduction (i.e. the programme had no effect on the individual) or mitigated some but not all reduction (i.e. the programme had a positive effect, but this was not enough to completely reverse the effects of external challenges). Finally, it must be recalled that participants who did not complete the programme, and thus did not have a post-Noise Solution SWEMWBS score, were not included in the dataset, which may bias results towards demonstrating positive changes.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

References

1. Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being.

2. Ng Fat L; Mindell J B, Stewart-Brown Evaluating and establishing national norms for the short Warwick-Edinburgh Mental Well-being Scale (SWEMWBS) using the Health Survey for England. Quality of Life Research 2016;26:1129-1144.

3. Shah N, Cader M, Andrews WP, et al. Responsiveness of the Short Warwick Edinburgh Mental WellBeing Scale (SWEMWBS): evaluation a clinical sample. Health and Quality of Life Outcomes 2018;16:239.

4. Boehm JK, Kubzansky LD. The heart's content: the association between positive psychological wellbeing and cardiovascular health.

5. Glenister S. A study of stakeholder perceptions of Noise Solution's practices: measuring impact on the well-being of youth facing challenging circumstances. Faculty of Education: University of Cambridge, 2017.

6. Gutman LM, Vorhaus J. The Impact of Pupil Behaviour and Wellbeing on Educational Outcomes, 2012.

7. Warwick Medical School.

8. Fujiwara ea. HACT, MentalHealth_and_LifeSatisfaction_web.pdf, 2017.

9. R_Core_Team. R: A Language and Environment for Statistical Computing_. R Foundation for Statistical Computing, Vienna, Austria. <https://www.R-project.org/>. 2023.

10. Office for National Statistics. Personal well-being in the UK, quarterly: April 2011 to September 2021., 2022.

11. Hartig. F. DHARMa: Residual Diagnostics for Hierarchical (Multi-Level / Mixed) Regression Models. R package version 0.4.6., 2022.

12. Fox J WSARCtAR, Third edition. Sage, Thousand Oaks CA., 2019.

13. Douglas Bates MM, Ben Bolker, Steve Walker. . Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software 2015;67:1-48.

14. Yee TW. Vector Generalized Linear and Additive Models: With an Implementation in R. New York, USA: Springer, 2015.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Appendix

Methodology

In order to conduct the paired t-test to address research question 1, data were first investigated for normality to confirm the test was valid. Normality is a method of evaluating how data are distributed around a central point (the average); normally distributed data have most values spread around the centre, with fewer values occurring further away from that centre, forming a ‘bell curve’ when plotted on a graph. Simulated residuals of other models were also inspected with respect to their assumptions of normal distribution and homogeneity of variance using the Dharma package.11 In order to meet the requirements of a paired t-test, age, which was right-skewed (meaning many participants were younger than the average and only a few were older), was subject to a Log10 transformation prior to conducting analyses, to allow the data be handled appropriately in the analysis.

Model selection followed the process of backwards elimination. Initial models included all pairwise interactions between included predictors; three-way interactions were not included as they were not underpinned by an expected mechanism and resulted in over-parameterisation of models. The Car::Anova function in R was then used to test for significance of terms using type 3 sums of squares.12 Non-significant interaction terms were removed sequentially in descending order of p value. Non-significant main variables were not removed from the model. Where an interaction term was significant but one of its constituent main effects was not significant, the interaction was still retained for investigation as it was likely caused by a crossover or similar effect. P values from regression analysis, rather than the analysis of variance (ANOVA) used to select models, were used to assess significance of terms in the final models as they are relevant at a category level for discrete variables, which was of greater interest. For the multinomial regressions, which cannot support missing values in independent variables, only individuals with complete data for relevant characteristics were included in these analyses.

The R package::function used for research questions 1–6 are described in Table 10

Table 10. R package used for research questions 1–6

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Research Question R Package::Function 1 Are male and female participants of similar average age? Stats::t.test9 2 How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme? Lme4::lmm13 3 How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme? Stats::lm9 4 How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants? VGAM::vglm14 5 How does age and gender affect the well-being

and does the well-being

the

they

the

Lme4::lmm13

For

band

participants,

band that participants occupy at

start

the programme differ from the band that

occupied at

end?

those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the programme? VGAM::vglm

Statistcal Results Summary

Table 11 provides results of the statistical analyses completed for research questions 1–6.

Table 11. Results of analyses Research

How does age and gender affect SWEMWBS scores, and do participants’ scores at the end of the programme differ from those at the start of the programme?

3 How does age, gender and starting SWEMWBS score affect participants’ change in SWEMWBS score between the start and end of the programme?

How does age, gender and starting SWEMWBS scores influence the category of meaningful change of participants?

Interceptc NA

Gender[Male]

Odds ratio

Down: 0.28 (1.41) NA NA

None: 0.37 (1.35) NA

meaningful increase

Reference Level Estimate Type Estimate (SE) Test statistic (df) P Valued 1

Log10(Age) NA NA 0.026 (0.017) t = 1.52 (126.95) 0.13 2

Interceptc NA Response (change in SWEMWBS scores) 19.02 (0.42) NA NA Gender[Male] Female 1.90 (0.47) t = 4.09 (320) <0.001 State[End] Start 2.50 (0.24) t = 10.6 (321) <0.0001 Log10(Age) Mean age −5.95 (1.83) t = −3.26 (518.65) 0.001 Log10(Age) × State[End] Mean age; Start 6.79 (1.86) t = 3.66 (321) 0.01

Interceptc NA 1.87 (0.45) NA NA Gender[Male] Female 0.9 (0.52) t = 1.74 (319) 0.08 Log10(Age) Mean age 4.23 (1.75) t = 2.41 (319) 0.02 Starting Score Mean start score −0.43 (0.06) t = −7.60 (319) <0.0001

Question Variablea,b

Are male and female participants of similar average age?

0.72

z = −0.82

0.42

Female;

Down:

(1.50)

(NA)

z =

Mean Log

0.27

z = −0.92

0.36

0.08

None: 0.92 (1.42)

−0.25 (NA) 0.80 Log10(Age)

10(Age); meaningful increase Down:

(4.16)

(NA)

None:

(3.82) z

−1.90 (NA)

How does age and gender affect the well-being band of participants, and does the wellbeing band that participants occupy at the start of the programme differ from the band that they occupied at the end?

For those participants who started in the low well-being band, how does age and gender influence their well-being band at the end of the Noise Solution programme?

Footnote: a Both age and starting score are centred around the mean; b Contents in square brackets describe subgroups; c the intercept is the value of the estimate when all variables are at their reference level; d this represents the p value from the regression analysis, not the ANOVA used to select models; statistically significant p values are in bold – p values for categorical variables show the likelihood that the variable is not different from the reference, whilst values for continuous variables show the likelihood that the slope (i.e. the strength of association) of the relationship is 0. If a significant interaction is present, this should be used for interpretation over the main effects; e a z-value is not reported here due to detection of Hauk-Donner effect, which, due to low data availability, artificially inflates the z-value; participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1) were excluded from analyses; sample size was too small to produce meaningful analyses for these groups.

Abbreviations: SE, standard error; SWEMWBS, short Warwick-Edinburgh mental well-being scale. Copyright

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Reference Level Estimate Type Estimate (SE) Test statistic (df) P Valued Starting Score Mean starting score; meaningful increase Down: 1.32 (1.05) z = 5.71 (NA) <0.0001 None: 1.16 (1.04) z = 3.48 (NA) <0.001

Research Question Variablea,b

Interceptc NA Odds ratio 1.56 (1.41) z = 1.29 (NA) NA Gender[Male] Female 0.25 (1.51) z = –3.35 (NA) <0.001 State[End] Start 0.21 (1.32) z = –5.67 (NA) <0.001 Log10(Age) Mean age 409.59 (5.69) z = 3.46 (NA) <0.0001 Log10(Age) × State[End] Mean age; Start 0.003 (7.98) z = –2.86 (NA) <0.01

Interceptb NA Odds ratio High: 0.04 (2.82) NAe NA Medium: 0.99 (1.35) z = -0.03 (NA) NA Gender[Male] Female High: 4.99 (3.00) z = 1.47 (NA) 0.14 Medium: 1.27 (1.45) z = 0.64 (NA) 0.52 Log10(Age) Mean Log10(Age) High: 18.13 (10.59) z = 1.23 (NA) 0.22 Medium: 4.81 (4.08) z = 1.12 (NA) 0.26

Additional Results

Figure 4. SWEMWBS scores as a function of age, at the start and end of the Noise Solution programmea

Footnote: a The teal lines represent the age component of the linear model. Log10 participant age is shown, as this ensures that the distribution is suitable for linear modelling; to convert to absolute age, from Log10(age), calculate as age = 10Log10(age), where Log10(age) is a value on the bottom (x) axis. Closed circles represent individual participant datapoints.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Figure 5. Change in SWEMWBS score as a factor of participant agea

Footnote: a The teal line represents the age component of the linear model and the orange dashed lines show the thresholds for meaningful change (score change at least 1 point). Log10 Participant age is shown, as this ensures that the distribution is suitable for linear modelling; to convert to absolute age, from Log10(age), calculate as age = 10Log10(age), where Log10(age) is a value on the bottom (x) axis

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale. Closed circles represent individual participant datapoints.

Figure 6. Change in SWEMWBS scorea following participation in the Noise Solution programme as a factor of SWEMWBS score at the start of the programme

Footnote: a The teal line represents the age component of the linear model and the orange dashed lines show the thresholds for meaningful change (score change at least 1 point).

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

Table 12. Transitions between high, medium and low well-being SWEMWBS bandsa

Footnote: a SWEMWBS has a mean of 23.5 and a standard deviation of 3.9 in UK general population samples and the cut-point for the high wellbeing band was set at 27.5 (mean + 1 SD) and for the low band at 19.5 (mean – 1 SD), with the medium band being between these points, following methodology of Fat et al. (2016) and using mean and SD values from the Warwick Medical School website.2, 7

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Start Band End Band Number of Participants Percentage of Population (%) High High 7 1.87 Medium 3 0.80 Low 1 0.27 Medium High 34 9.09 Medium 149 39.84 Low 18 4.81 Low High 11 2.94 Medium 85 22.73 Low 66 17.65 Change: Up 130 34.76 Change: Down 22 5.88 No Change 222 59.36

Footnote: a To convert to absolute age, from Log10(age), calculate as age=10Log10(age), where Log10(age) is a value on the bottom (x) axis; b participants for whom gender was not known as well as the small number of participants (n=5) who declined to provide this information (n=1), or who stated ‘other’ (n=3) or ‘non-binary’ (n=1) were excluded from analyses; sample size was too small to produce meaningful analyses for these groups.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Figure 7. The number of participants in each well-being band at the start and end of the Noise Solution programme, by age and gendera, b

Footnote: Grid headings show starting category, as defined by Fujiwara (2017)8 (no participants started in categories 10 or 11), the size of the purple point indicates the proportion of all participants in that starting category, which is also denoted by the value (n) underneath the point. The thickness of the teal line indicates the percentage of participants who started in that category and transitioned to each Fujiwara category, also denoted by the value at the end of each line.

Abbreviations: SWEMWBS, short Warwick-Edinburgh mental well-being scale.

NOISE SOLUTION | ANALYSES OF SWEMWBS DATA

Figure 8. Transitions in

Fujiwara (2017) SWEMWBS value categories