Whole school interventions addressing multiple adolescent risk behaviours: a feasibility study by Scottish Collaboration for Public Health Research & Policy (SCPHRP)

Chief Scientist Office Form 4 Final report form

CSO reference number: SCPH/04 (SCPHRP ref no. 04-09/10)

Please complete this form in Verdana 10 point font size Project title: Whole school interventions addressing multiple adolescent risk behaviours: a feasibility study

Start date: May 2010

Finish date: March 2012

Investigators: Professor Lyndal Bond

MRC/CSO Social & Public Health Sciences Unit

Professor Daniel Wight

MRC/CSO Social & Public Health Sciences Unit

Dr Marion Henderson

MRC/CSO Social & Public Health Sciences Unit

Paul Ballard

Deputy Director of Public Health (and Honorary Senior Lecturer, Dundee University Medical School)

Summary

The school environment can affect young people beyond their capacity to learn and succeed academically. There is evidence of “school effects” in secondary schools for substance use and mental health, and evidence suggesting that changing school ethos can reduce substance abuse and anti-social behaviour. These effects are not currently well understood, particularly within the Scottish educational context, and there is a need to evaluate whole school approaches to improving school ethos. This study aimed to assess current approaches to improving school ethos in Scottish secondary schools to clarify their intended mechanisms and contribute towards the development of further interventions and their evaluation. The study found a strong policy and education context within Scotland for the development of approaches to improve ethos and a wide and varied range of promising current practices. Many of these practices are closely related to ideas in the literature on school connectedness, engagement and attachment but none had a sufficient theoretical basis to warrant immediate rigorous evaluation.

2 Aims The aim of this study was to map and assess the range of current interventions in Scotland that have the capacity to change school ethos. Our objectives were to contribute towards the development of further interventions and the design of future evaluations by clarifying: 1. intended mechanisms behind interventions and their likely impact on health inequalities 2. if an existing programme warrants rigorous evaluation, or whether it is necessary to develop a new intervention that integrates the best features of existing projects 3. what should and can be measured, which methods are feasible and, by gaining a better understanding of the institutional contexts, what might facilitate or impede evaluation. We aimed to assess the feasibility of implementing and robustly evaluating interventions focused on modifying schools’ environment, social relationships or culture to address multiple adolescent risk behaviours, in particular reduce violence and substance use, which previous research suggests are the most likely impacts. The study addressed eight research questions with respect to current practice and attitudes to evaluation of school-based activities to promote ethos.

3 Methodology 3.1 Participants Twenty four interviews were held with 40 informants from three levels: national, local authority and school level. Table 1 outlines interviewees’ affiliations and roles. 3.2 Analysis Interviews were recorded and transcribed verbatim or listened to several times and notes taken. A sample of interviews was coded and compared with the research questions to develop an analytic framework for coding subsequent data and development of main themes (Ritchie & Spencer, 1993; Seale, 1999). NVivo 9 was used to aid this process.

Table 1. Interviewees’ affiliations and titles Respondent affiliation (number of respondents)

Role

National Level Interviews Scottish Government’s Positive Behaviour Team (PBT)* (1)

Regional Team Member

Her Majesty’s Inspectorate of Education (HMIE)* (2)

National Specialist for Health and Wellbeing and Health and Nutrition Expert

Learning and Teaching Scotland (LTS)* (1)

(Health and Wellbeing Team Leader)

Pupil Inclusion Network Scotland (PINS) (1)

Voluntary sector worker with extensive experience in supporting the health and wellbeing of children and young people in Scotland

* as of July 2011, PBT, HMIE and LTS have been brought together under the umbrella of Education Scotland Local Authority Level Interviews Local Authorities were identified through the national level interviews as conducting particularly good work on improving the health and wellbeing of their pupils. Dumfries and Galloway (2)

Health and Wellbeing Officer and Health Improvement Lead for Education in Young People

Highland (1)

Health Promoting Schools Manager

East Renfrewshire (3)

Quality Improvement Officers

Midlothian (2)

Director of Education and Communities and Education officer for schools

Perth and Kinross (1)

Development Officer, Education Services

City of Edinburgh (1)

Manager of a Social and Emotional Wellbeing primary school programme School Case Study Level Interviews

Four secondary schools were selected following recommendations from national and local authority level interviews. School names are pseudonyms. Dumfries and Galloway school: “Copperhill High” (10) Highland school: “Glenabbey High” (4) Perth and Kinross school: “Riverfield Academy” (7)

Roles included Senior Management, classroom teachers, Principal Teachers with responsibility for Support or Guidance and support staff.

East Renfrewshire school: “Devongate Academy” (4)

4 Results 4.1 National Level Interviews Throughout the interviews, it was clear that policymakers believed that getting the culture, values and ethos “right” was important and promoting emotional, physical and wellbeing of all was fundamental. There was a strong sense that secondary schools are not just places to learn, but are part of a wider community with a responsibility to work with children to embrace, include and encourage them. Changing school climate and ethos however, requires a commitment to a whole school approach which recognises that change occurs as integral to the school/council improvement planning. Terms such as initiative or intervention were regarded as unhelpful as they implied activities additional rather than central to the business of learning. The relationship between health and wellbeing and attainment Those at a policy or local authority level endorsed the idea that focusing on emotional, physical and mental wellbeing has the potential to improve academic outcomes through building self-esteem, confidence, motivation for learning and raising aspirations, and through attachment to school. However, academic attainment was seen as the ultimate goal and while most schools accepted that good health is needed to learn, there remained a tension in practice between literacy and numeracy on one hand and health and wellbeing on the other. Scottish Policy Commitment to health and wellbeing Policymakers consistently referred to school ethos and climate when discussing improvement of health and wellbeing and this is clear in policy documents (e.g., Getting It Right For Every Child (GIRFEC) (Scottish Government, 2012) and 2010 Scottish Curriculum for Excellence (CfE)). CfE was prominent in interviews as the principle policy driver for improvement in health and wellbeing in schools: “There is something positive about working with the Government at the moment because it's increasingly about a social justice agenda in education, CfE and GIRFEC, rather than all about attainment.” [PBT Respondent (Education Scotland)] “Curriculum for Excellence contains a commitment to ensuring young people are included in the process rather than having things done to them. [It’s] less about the rules of the school than attitudes, responsibilities and relationships” [LTS respondent (Education Scotland)] However we found while school ethos and health and wellbeing were aligned at the policy level, in practice they were presented as distinct areas for improvement for school inspections, with these audits possibly reflecting traditional health promotion practices. Many schools reported feeling daunted by the non-prescriptive nature of CfE and felt that there was insufficient time set aside to engage in discussions of how best to address health and wellbeing in their institutional context. Further, addressing health and wellbeing, and changing ethos, in secondary schools was seen as particularly challenging due to schools’ academic structure and large size. Evaluation At the national level, most believed that health and wellbeing practices should be evaluated, and that evaluation should be part of the whole process, not added at the end of policy implementation. Despite this, there was little evidence of rigorous evaluations of existing programmes nor of evaluation having been considered in the development of CfE. Evaluation and feedback was commonly a retrospective process rather than an element built into the foundations of developing and implementing a new strategy. Plans of evaluation early in the process seemed to imply an expectation of early effects: “If we put in [a new whole school approach] you have to evaluate … but people will say it takes between three and five years before it has a significant impact so if you're going to evaluate it, don't be too soon, too premature” [PBT Respondent]

4.2 Case study schools The four schools were (pseudonyms are used): 1. “Copperhill High School” (CH) (small school, mixed rural and urban, predominantly high deprivation, lower than average attainment) 2. “Glenabbey High School” (GH) (small school, predominantly rural, average to low deprivation, high attainment) 3.

“Riverfield Academy” (RA) (large school, mixed inner city urban and rural, mixed high and low deprivation, mixed attainment)

4. “Devongate Academy” (DA) (large school, predominantly suburban, predominantly low deprivation, high attainment) From all the schools we found that: 

Health and wellbeing was seen as a high priority and a foundation, with schools acknowledging that good health was needed to achieve, and vice versa.

●

There was a fundamental concern for the whole child emotionally, physically and mentally, not just on academic development and staff were motivated to take on health and wellbeing.

●

We found no stand alone programmes with a primary focus on improving school ethos however all schools participated in a range of activities focusing on improving social and emotional health and developing the school community.

How schools improve ethos Good practice was identified at three levels: 1. Whole School Level: i.

Shared values centred on mutual respect, appreciation, warmth/caring and trust between and among staff and pupils. “[It] only takes a minute of your time to let them know you care about them” [CH3, PT Pupil Support] “I mean we have to look after them while they’re here. You know, I feel we do so well for every child in the school, we know them all and everyone is an individual, and I would say, everything’s tailored to them” [GH1, Youth Development Officer]

ii.

Raising pupils’ self-esteem and developing an ethos of achievement and providing opportunities to succeed in extra-curricular activities. “So much of the success of our school is about building ambition and showing kids there's more to life than this scheme” [CH4, PT Curriculum] “It’s about that feeling of being better, being confident, being able to go out, they’ve not got a problem now going out and being Riverfield Academy, they’re very proud of it whereas before they would have disappeared under the table” [RA1, Head Teacher]

iii.

Building a strong, safe community environment and fostering connection with the school. “The success of Copperhill High is dependent on good will and collaboration and collegiality of the staff - this is unique at Copperhill High” [CH4, PT Curriculum] “We have a strong ethos of a close-knit community among staff - teachers feel free to dip into each others' classes and come and go. This atmosphere of community among staff is now feeding through the rest of the pupils and wider community” [CH3, PT Pupil Support] “And I think … yes we are known as a school that produces excellent results and things like that but it’s because the kids feel you know, confident and they feel

comfortable … they feel safe here, you know it’s, it’s a good school to work in for that reason as well” [DA2, Maths Teacher] iv.

Establishing fair and transparent discipline policies based on respectful relationships including restorative approaches. “The other thing in terms of the ethos of the school, which I believe most staff are really strong on, is a fresh start every day. You know, you've screwed up, right, it's a fresh start, let’s continue, let’s carry on from where we left off. It took me a while to get into that” [GH3, English Teacher]

Strong staff leadership and provision of Continuing Professional Development and effective partnerships with outside agencies. “But I find in a school, especially of this size, if you are constantly giving staff the same message, it’s that drip-drip effect, I think that’s what works. If I just did a big presentation and talked about health and wellbeing ... they would forget, cos they’re busy people” [DA3, DHT, Support for Pupils]

2. Classroom and Curricular Level: i.

Using initiatives such as Eco Schools and Being Cool in School, buddying and mediation. “The Eco-schools, green flag group, that was quite an initiative to keep going, that’s very much linked into the [CfE] Health and Wellbeing Outcomes and … citizenship and being: thinking about what they’re doing” [GH2, DHT]

ii.

Building flexibility into the curriculum to respond to pupils’ needs and abilities. “Focusing on what's best for pupils gives direction if there is ever professional disagreement about the best thing to do. We have flexibility in the school to help individual children, we can't be too rigid” [CH4, PT Curriculum]

3. Individual Level: i.

Balancing universal approaches with an ability to identify and respond to individual pupils’ needs in a targeted way. “I believe we're working our way towards a situation where we can target and pick up on the pupils that need that specific support and help, I believe that ... The whole CfE thing, what's good about it is it's turning education into an umbrella and I mean that in a positive way. A school doesn’t, isn't a collection of boxes … and the mentality should be such that everything we're doing in here is producing someone who is going to go out and exist after this” [GH3, English Teacher]

Discussion

The findings from the interviews are interpreted with respect to our research questions. 5.1 What activities, programmes and practices are being implemented by Scottish secondary schools that address, or have the potential to address, adolescent health risk behaviours, health and wellbeing? (RQ1) This study has identified a broad range of activities at individual, classroom, whole school and community level. Scotland is in a good position in terms of how its education system is structured as its emphasis on health and wellbeing, especially in secondary schools, provides a strong context for improving ethos. There was consistency throughout all interviews on the importance of addressing health and wellbeing at a whole school level. The local authorities and schools in this study were selected as examples of good practice. We cannot therefore say how typical our case study schools are, however the overarching policy aims underpinning education in Scotland are encouraging for the development and sustaining of a focus on health and wellbeing and improving school ethos. 5.2 What are the causal mechanisms by which the most promising programmes are intended to work and to what extent are these prescribed activities or emerging processes? Which programmes might also specifically address those factors that lead to students’ low attendance and disengagement from school? (RQ2&3) Intended mechanisms were rarely clearly articulated by schools but there was evidence in the data of a theory base to many of their practices. The initiatives and activities that schools identified addressed a number of protective factors shown to be associated with positive outcomes for young people e.g. fostering connectedness and attachment to school, engaging pupils and creating a safe, inclusive environment. Connectedness to school School connectedness has been defined as the belief that adults within a school care about a young person’s learning and about them as individuals (Centres for Disease Control and Prevention, 2009). School connectedness has been shown to be related to school retention, better attainment, mental health and decreased risk of substance use (Bond et al., 2007; McNeely, 2003; Henderson et al, 2008), and schools can provide a protective function for those pupils who are not connected to resilient families (Rutter, 1984; Werner and Smith, 1988). Our study schools were doing many of the things that have been identified in the literature as being important for fostering good school ethos such as: 

Encouraging shared values, especially those of co-operation, inclusion and effective resolution of conflict



Creating a safe, caring, inclusive culture



Giving pupils opportunities to contribute and take initiative



Placing a strong emphasis on the development of positive relationships between staff and pupils and among pupils



Pupil well-being as a high priority within the school



School leadership team with a continuing commitment to working with all staff to implement a vision based around safety, well-being and personal growth



Explicit programme of behaviour management implemented at a whole school level (Whitlock, 2003, McGrath and Noble, 2010, Bond & Butler, 2009)

Engaging Students In addition to the above, our schools undertook a number of activities to explicitly address potentially disengaged students, which again align with the literature in this area (National Research Council, 2004). For example: 

fostering personalised and continuous relationships between teachers and students



guidance and counselling responsibilities diffused among school staff, including teachers, who are supported by professionals



improved communication, coordination and trust between the school and the community



greater coordination with social and health services within the community



making the curriculum more meaningful, varied and appropriately challenging to engage students cognitively, emotionally and behaviourally

Restorative approaches Discussion of restorative justice approaches did not generally occur in the interviews, however restorative values are consistent with the restorative conflict resolution that was evident in the assertive discipline and “Fresh Start” policy in Glenabbey. 5.3 Is there a programme that has a clear theoretical basis, has been adequately piloted, and has sufficient evidence of likely effectiveness (either from its own evaluation or from the literature) to warrant a large scale trial? (RQ4) No such programme was identified. Our study was small, so it might be possible that other schools are implementing such programmes, however, given who we interviewed and how we sampled our case studies, we think it unlikely. Perhaps what is of greater interest and relevance to schools is not implementing a programme but rather ensuring their practices and procedures follow best practice for creating a supportive environment for their students and staff. We found that this was the case, albeit, as stated above, such practices were not necessarily explicitly related to theory or evidence. Perhaps we as researchers need to rethink how we can evaluate such practice rather than being confined to thinking these come wrapped up in a programme. This is also important as schools were quite clear that the terminology of interventions, etc., was not helpful and made such activities external to core business. 5.4 What are stakeholders’ attitudes to evaluating these school-based activities in terms of assessing health and wellbeing outcomes for young people? (RQ5) There was agreement on the need for evaluation of practices and positive reaction to researchers undertaking this work. There was little evidence for effectiveness of current practices within Scottish schools, including Curriculum for Excellence. The lack of articulated theory base, variability in practices between and within schools, the complex causal pathways and a lack of knowledge or experience in designing evaluations may all contribute to a general reluctance to engage in evaluating activities in any rigorous manner. Again, as researchers we need to engage here, acknowledging that these are complex interventions in complex settings (e.g. Bond & Butler, 2010) but there are research methods that can accommodate such complexity. 5.5 What should or could be measured with respect to health and wellbeing outcomes? (RQ6) Respondents were reluctant, or unable, to identify outcomes that they would like to measure. The Curriculum for Excellence places a large responsibility on schools to provide evidence that children are meeting Health and Wellbeing Experiences and Outcomes and they are also evaluated through HMIE, so many perceived themselves to be overevaluated. There is scope here for further work to determine what evidence or outcomes schools are expected to provide for that component of the Curriculum for Excellence and

what schools might consider as indicators of success such as reductions in disciplinary incidents and indicators of student engagement with school broadly. 5.6 Given the intended mechanism (RQ2), what process data should be collected to investigate the extent to which the programme operates as intended and the contextual factors that facilitate or hinder delivery? (RQ7) As we were unable to identify a programme suitable for evaluation, we cannot link recommendations to a specific process. However, our study suggests not focusing on programmes developed externally to schools, rather on building schools’ capacity to develop reflective practices thus allowing them to determine how they might best promote a positive school ethos. This study and others (e.g. Gatehouse Project) indicate that schools need assistance in choosing and evaluating evidence-based practices, help in making explicit why they choose to implement them, and processes and tools to assist in reflective practice. 5.7 What data is or could be collected to assess whether subgroups of pupils are differentially affected by school practices (with reference to RQ3)? (RQ8) This study did not identify any routinely collected data that would answer this question. We propose that schools consider, from the administrative data they collect (e.g. discipline incidents, suspensions, and post school destinations), how what they do in terms of whole school activities differentially affects their pupils.

6 Conclusions 

There is a recognised need for schools to prioritise health and wellbeing, and the role of the school climate or ethos in contributing to this.



Scotland’s Education policies provide a strong framework to address health and wellbeing although many find the non-prescriptive nature of the new curriculum difficult to work with.



We identified a broad range of good practices including whole school, curricular and individual based approaches. These have generally evolved by committed staff rather than being in response to specific need or strategic planning.



While not necessarily aligned to explicit theories, many current practices align with the evidence about school connectedness and student engagement. No individual programme was identified that was ready for rigorous evaluation.



This study, as with others, indicates that we may need to rethink or move away from considering the development and evaluation of discrete programmes or packages to improve ethos, to developing approaches that facilitate schools’ capacity to explicitly choose, implement and reflect on evidence-based practices or practices aligned to relevant theories.

7 Importance to NHS and possible implementation Evidence suggests that improving school ethos, and thus improving the social and emotional wellbeing of pupils, has benefits in preventing a wide range of other problems that may occur in later adolescence or adulthood such as depression, substance abuse, poor sexual health and smoking. These incur considerable cumulative costs to the NHS (as well other societal costs of crime, additional education and state benefits) indicating the cost effectiveness of prevention in schools (Scott, Knapp, Henderson and Maughan, 2001).

Future research 

The SCPHRP funded development of Social and Emotional Education and Development (SEED) intervention.



Dissemination   

NIHR funded a five year cRCT of SEED commencing in Autumn 2012 (http://www.phr.nihr.ac.uk/funded_projects/10_3006_13.asp)

Tweedie, S., Henderson, M., Wight, D. & Bond, L. (2011).Whole school approaches to reducing risk behaviour and promoting social and emotional development. Poster presented at SCPHRP Reflections and Next Steps, Edinburgh, November 21 We plan one academic paper The study contributed significantly to the development of SEED

Research workers

Sarah Tweedie, Investigator Scientist at MRC/CSO SPHSU, and co-applicants.

Financial statement

To be forwarded by the MRC SPHSU Finance Office within one month of submission of this report. Word Count: 3,497

Executive summary (Focus on Research)

Background/Aim The school environment can affect young people beyond their capacity to learn and succeed academically. There is evidence suggesting that changing a school’s climate or ethos can affect a number of health and wellbeing outcomes including substance abuse and anti-social behaviour. However, these effects are not currently well understood, particularly within the Scottish educational context. Our study aimed to assess current approaches to improving school ethos in Scottish secondary schools at a whole school level and examine the feasibility of evaluating such approaches. Project Outline/Methodology In order to better understand the contextual factors surrounding school ethos and health and wellbeing we conducted qualitative interviews with 15 people who have responsibility for implementing and supporting education policy within Scotland, including members of Education Scotland and education officials in six Local Authorities. From these interviews we identified four promising secondary case study schools. In these schools we interviewed a further 25 respondents including classroom teachers, senior management and support staff. Key Results School ethos was seen as fundamental not only to health and wellbeing but also to academic attainment. We found a strong policy and education context within Scotland for the development of approaches to improve ethos and a wide and varied range of promising current practices. However, there were challenges for schools due to the nonprescriptive nature of the Scottish curriculum. Evaluation was accepted as necessary and desirable although little evidence was found to suggest that evaluation is routinely an integral part of educational developments. Case study schools demonstrated a range of promising practices at whole school, classroom and individual level. These practices and activities focused on improving relationships, engaging pupils in school and fostering a strong, safe community environment rather than implementation of stand alone health and wellbeing programmes.

Conclusions A wide range of approaches to improve school ethos was found along with a strong need to prioritise health and wellbeing in secondary schools. Practices in schools were focused on creating a supportive environment for their students and staff rather than implementation of programmes. No individual programme was identified that was ready for rigorous evaluation. What does this study add to the field? Many of the practices identified in this study are closely related to the literature on school connectedness, engagement and attachment although schools themselves may not clearly articulate this link between practice and theory. Implications for practice or policy This study, as with others, indicates that we may need to rethink or move away from considering the development and evaluation of discrete programmes or packages to improve ethos, to developing approaches that facilitate schoolsâ&#x20AC;&#x2122; capacity to explicitly choose, implement and reflect on evidence-based practices or practices aligned to relevant theories. Where to next This study has led to the development, and funding, of a five year trial of Social Emotional Education and Development (SEED) in Scottish Primary schools which focuses on providing schools with local data about their students and supporting schools to choose activities and build on their practices to address the needs identified in the data at the whole school, classroom and individual level. Further details from Professor Lyndal Bond MRC/CSO Social and Public Health Sciences Unit 4 Lilybank Gardens G12 8RZ Email: lyndal@sphsu.mrc.ac.uk

References

Bond, L. & Butler, H. (2009). The Gatehouse Project: a multi-level integrated approach to promoting wellbeing in schools. In: Killoran A, Kelly M, Eds. Evidence-based Public Health: effectiveness and efficiency. Oxford: Oxford University Press: 250-69 Bond, L., Glover, S., Godfrey, C., Butler, H. & Patton G. (2001). Building capacity in Schools: Lessons from the Gatehouse Project. Special issue on Policy and System Level Approaches to Health Promotion in Australia. Health Education & Behavior; 28(3): 368383 Bond, L., Butler, H., Thomas, L., Carlin, J.B., Glover, S., Bowes, G. & Patton, G. (2007). Social and school connectedness in early secondary school as predictors of late teenage substance use, mental health and academic outcomes. Journal of Adolescent Health 40(4) 357.e9â&#x20AC;&#x201C;357.e18 Centres for Disease Control and Prevention (2009). School Connectedness: Strategies for Increasing Protective Factors Among Youth. Atlanta, GA: U.S. Department of Health and Human Services Henderson, M., Butcher, I., Wight, D., Williamson, L. & Raab, G. (2008). What explains between-school differences in rates of sexual experience? BMC Public Health 2008; 8:53 McGrath, H. & Noble, T. (2010). Supporting positive pupil relationships: Research to practice. Educational & Child Psychology, 27(1), 79-90 McNeely, C. (2003). Connections to School as an Indicator of Positive Development. Paper presented at the Indicators of Positive Development Conference, Washington, DC

National Research Council (U.S.) Committee on Increasing High School Studentsâ&#x20AC;&#x2122; Engagement and Motivation to Learn (2004). Engaging Schools: Fostering High School Studentsâ&#x20AC;&#x2122; Motivation to Learn. Washington, D.C: The National Academies Press Ritchie, J. & Spencer, L. (1993). Qualitative data analysis for applied policy research, in: Bryman A. and Burgess R. (Eds) Analysing qualitative data. London: Routledge; pp. 173194 Rutter, M. (1984). Resilient Children. Why some disadvantaged children overcome their environments, and how we can help. Psychology Today, March, 57-65 Scott, S., Knapp, M., Henderson, J., and Maughan, B. (2001). Financial costs of social exclusion: Follow-up study of antisocial children into adulthood. British Medical Journal, 323(7306), 191 Scottish Government (2012). A Guide to Getting It Right For Every Child. Available at: http://www.scotland.gov.uk/Resource/0039/00394308 Scottish Government (2009). Curriculum for Excellence: Health and Wellbeing Across Learning: Responsibilities of All. http://www.educationscotland.gov.uk Seale, C. (1999). The Quality of Qualitative Research. London: Sage Werner, E. & Smith, R. (1988). Vulnerable but Invincible: A Longitudinal study of Resilient Children and Youth. Adams, Bannister and Cox