A measurable impact: Helping young people to be and stay well

Page 1

HELPING YOUNG PEOPLE TO BE AND STAY WELL


REACHOUT AUSTRALIA ReachOut is the most accessed online mental health service for young people and their parents in Australia. Its trusted self-help information, peer-support program and referral tools save lives by helping young people be well and stay well. The information it offers parents makes it easier for them to help their teenagers, too. ReachOut has been championing wider access to mental health support since it launched its online service more than 20 years ago. Everything it creates is based on the latest evidence and is designed with experts, and young people or their parents. That’s why ReachOut is a trusted, relevant service that’s so easy to use. Accessed by more than 2 million people in Australia each year, ReachOut is a free service that’s available anytime and pretty much anywhere. AUTHORS Bianca Kahl Hilary Miller Kathryn Cairns Hayley Giniunas Louisa Welland Mariesa Nicholas ACKNOWLEDGEMENT We would like to acknowledge the contribution of Associate Professor Phil Kavanagh, University of Canberra, for peer-reviewing the results and interpretation of this research. SUGGESTED CITATION Kahl, B. L., Miller, H., Cairns, K., Giniunas, H., Welland, L., & Nicholas, M. (2020). A Measurable Impact: Helping Young People to Be and Stay Well. Sydney: ReachOut Australia.

We acknowledge the traditional owners of Country throughout Australia and recognise their continuing connection to lands, waters and communities. We pay our respects to Aboriginal and Torres Strait Islander cultures, and to Elders both past, present and emerging. We recognise connection to Country as integral to health and wellbeing.


Contents Foreword 1 Executive summary 3 Introduction 5 Young people’s mental health and wellbeing: A growing concern 5 The service system and policy landscape 6 The role of digital services in transforming mental health care 6 Capitalising on a captive audience 8 Circumventing the barriers to help-seeking 9 Evidence of promise 10 A longitudinal cohort study of ReachOut users Research aims Methods and participants Who is using ReachOut?

13 13 13 15

Findings Mental health Suicide risk Help-seeking How does ReachOut help?

17 18 20 20 22

Actions taken after visiting ReachOut 23 Young people’s reflections on the impact of using ReachOut 24 Recommendations 26 Conclusion 34 References 36



FOREWORD. ASHLEY DE SILVA, CEO REACHOUT AUSTRALIA

FOREWORD For the last two decades, ReachOut has been there for young people, helping more than 2 million people in Australia each year. ReachOut is a digital mental health service that provides trusted self-help information, a peer support program and referral tools that save lives and help young people to be well and stay well. Our earlier research and program evaluations have shown we are effective in helping young people to understand and deal with issues; connect with others and feel less alone; and feel more positive about themselves. With the explosion of digital mental health programs, apps and tools, it has never been more important to understand whether these services are improving the mental health of young people. For this reason ReachOut embarked upon the largest research study it has ever conducted. The study, which has been independently peer reviewed, aimed to understand the impact of ReachOut, an unstructured digital mental health service, and measured impacts over time on mental health and wellbeing, and on help-seeking behaviour. While we were confident the work we do has a meaningful impact, this study has confirmed that. Young people using ReachOut experienced a significant reduction in their symptoms of depression, anxiety and stress, and in suicide risk. There was also an increase in help-seeking behaviour. For some young people ReachOut was the help they needed, while for others with more significant mental health challenges we provided pathways to other services, matched to their needs. The growing burden of mental ill-health in Australia’s young people deserves our urgent attention. While some young people will seek help, others continue to suffer in silence. We know there are significant problems in our mental health system that will take some time to fix. Now and for the future, digital services like ReachOut are there, so that young people get the help they need to be well and stay well.

With the explosion of digital mental health programs, apps and tools, it has never been more important to understand whether these services are improving the mental health of young people.

Ashley de Silva Chief Executive Officer ReachOut Australia

1


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Danny’s story

I’m a 20-year-old student studying Psychology at Sydney University. I’ve always dreamed of being a psychologist and even though I’ve got a long way to go, it’s been a journey I’ve thoroughly enjoyed. It’s been challenging and fun balancing my social life, work, uni assignments and family time, but there have definitely been moments when it’s felt like too much. Being a part of a big family of six is a huge part of my life, and I’ve always had this dream of being able to help my family and friends. I thought that I should be the one to carry the weight of the people I care about and be the rock of my friendship group. But after I finished school, some of my closest friends and family went through incredibly hard times and I started to feel that always trying to be there for them was draining all of my energy. Everything I thought I knew and had learnt didn’t prepare me for when my friend tried to take his own life one night. It really affected me. I felt completely lost, and didn’t know what to do or what to say to him.

‘ReachOut encouraged me to start doing more of what I love, and to give myself some time to reflect and think about where I wanted to be.’ I found ReachOut just after this, while looking for ways to help my friend. Reading articles about other people who went through the same thing helped me to understand and get closer to my friend. We started connecting and having a lot of fun together, and I feel like it made such a difference for both of us. However, as I kept reading on ReachOut, I realised I had forgotten to look after myself along the way. I hadn’t been eating or sleeping properly 2

for months, and I was always anxious and overwhelmed by stress that I had been bottling up. It was really hard for me to realise that I’d been ignoring myself and that I needed to look at the problems I’d been brushing aside. ReachOut encouraged me to start doing more of what I love, and to give myself some time to reflect and think about where I wanted to be. I started writing poems to vent, and spent a lot of time relaxing while listening to music. I also started to meditate whenever I felt overwhelmed, and took to the forums to read about how other people have gone through their own journeys. It’s been a long process and something that I’m still going through now, but I always feel glad that I know where to find the right support to help me understand what I need to do and how to do it. If my experience has taught me anything about online support, it’s that it is a great first step for young people who really need some direction. Having a space with so many different kinds of support that are so easily accessible can make a world of difference. Moving forward, I know that ReachOut and its community will continue to play a part in my life and my future.

Danny, Sydney ReachOut Volunteer


EXECUTIVE SUMMARY

The National Mental Health Commission’s latest report (NMHC, 2019) indicates that while there has been progress on mental health reform, considerable work still needs to be done to transform how mental health services are delivered, and to reduce the personal, social and economic burdens of these illnesses in the community. Australia’s mental health-care system is fragmented and complex, with responsibility dispersed across the private, government and non-government sectors. And while the number of people accessing clinical services is growing, many people are being turned away from care, due to a lack of resources. Investing in digital mental health services such as ReachOut can help both to relieve this burden and to achieve the key recommendations made in the 2019 National Report. The most notable of those recommendations address the social determinants of mental health and integration, and shift the focus to prevention and early intervention, where there has been a funding imbalance.

About ReachOut ReachOut is an online mental health service that offers young people trusted information, as well as digital self-help and peer support. ReachOut provides resources and strategies for: •

being well and staying well

connecting with others and sharing stories

building confidence to seek help from a mental health professional

coping between treatment sessions

building ongoing healthy behaviours.

Importantly, ReachOut offers practical self-help strategies that young people can use anywhere, at any time, as well as pathways to other support services if required. In 2018–19, ReachOut reached more than 2.6 million young people in Australia. As a user-centred organisation, ReachOut involves young people in the design of everything that it creates. Co-design methods provide young people with opportunities for meaningful involvement, and for sharing their experiences, thoughts and opinions within a structured and safe environment. This ensures that ReachOut provides services that are relevant, meaningful and helpful in the everyday lives of our users. ReachOut also works with a Clinical Advisory Group, a diverse team of clinicians who advise on best practices in mental health service delivery.

3


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Aims

Recommendations

This report presents findings from a longitudinal cohort study that surveyed approximately 2000 ReachOut users over a three-month period. The study explored how young Australians engage with a digital, unstructured intervention and how this engagement impacts on their mental health status and help-seeking behaviour. Based on these findings, this report will:

Based on the findings from this study, our recommendations are as follows: •

To invest in programs and interventions that promote wellbeing in young people.

To harness the potential of digital technology to improve service availability and accessibility.

highlight the critical role of digital mental health services in the stepped-care model to support effective and efficient service integration and delivery

To clarify and solidify the role of digital services in a stepped model of care, and build service readiness.

demonstrate how digital services can be used to respond to several challenges facing the mental health sector, by providing matched care and onward referral where appropriate

To recognise the importance of participatory design to ensure that services meet the needs of young people.

outline the role of technology and digital mental health in reaching young people and at-risk populations.

To invest in research and program evaluation to drive quality assurance and effectiveness of mental health services, and to facilitate better treatment outcomes across the mental health sector.

Findings The findings of this study showed that ReachOut provides young people with a relevant, engaging and accessible resource that helps them to understand and cope with their experiences of tough times and mental health difficulties. The findings also showed that young people who used ReachOut experienced: •

improved mental health status

reduced suicide risk

increased help-seeking behaviour.

Users also told us that ReachOut had helped them to: •

better understand their own and others’ experiences of mental health and wellbeing

access practical strategies and tools for supporting their mental health

connect with others, and to feel more positive about themselves.

4


INTRODUCTION

INTRODUCTION

Young people’s mental health and wellbeing: A growing concern Young people experience a high burden associated with mental ill-health, with approximately one in four people aged 15–24 experiencing a mental health problem in a given year (ABS, 2018). Mental health concerns are also the largest contributors to the non-fatal health burden among young people (AIHW, 2019). Furthermore, while intentional self-harm accounts for a relatively small proportion (1.9 per cent) of all deaths in Australia, it accounts for a much higher proportion of deaths among younger people. In 2018, intentional self-harm accounted for 40.3 per cent of deaths among people aged 15–19, and for 37.2 per cent of deaths for people aged 20–24 (ABS, 2019). Unfortunately, the prevalence of psychological distress in Australia’s youth population appears to be increasing (Hall et al., 2019), and suicide remains the leading cause of death for young people (ABS, 2019).

Given the growing prevalence of mental ill-health, novel approaches are needed to reduce the incidence of these problems, to enable young people to flourish during this critical developmental period and to reduce the burden in the community.

Mental health problems such as depression and anxiety typically have their first onset by adolescence or early adulthood (Kessler et al., 2007). If left untreated, they can disrupt developmental pathways and result in more severe and frequent episodes in adulthood, as well as poorer treatment outcomes (Jorm, 2012; Kessler & Wang, 2009; Woodward & Fergusson, 2001; Fergusson, Boden & Horwood, 2007). Prevention and early intervention are therefore critical during this developmental window to reduce the burden of mental ill-health and suicide in young Australians. While there have been some improvements in help-seeking for mental health problems, the level of unmet need remains unacceptable (Lawrence et al., 2016). Given the growing prevalence of mental ill-health, novel approaches are needed to reduce the incidence of these problems, to enable young people to flourish during this critical developmental period and to reduce the burden in the community.

5


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

The service system and policy landscape

The role of digital services in transforming mental health care

The Australian mental health-care system has been undergoing notable change in recent years with the introduction of several new reforms and suicide prevention activities. Despite considerable reform action, significant improvements in population mental health in Australia have not been observed (Jorm, 2014), and the current infrastructure of the mental health sector remains ill-equipped to cope with the burden of mental ill-health in the community. The National Mental Health Commission’s 2019 National Report (hereafter ‘National Report’) highlights several shortcomings related to the current structure and resourcing of the mental health sector, as well as the continued neglect of preventative action.

Digital technology offers an effective method of delivering a variety of scalable, evidence-based and cost-effective mental health programs and interventions in a highly accessible manner. The E-Mental Health Strategy for Australia, introduced in 2012, proposed that new technology be utilised to develop high-quality digital mental health services that can be effectively integrated into the broader health-care system and offered to those in need in a timely and efficient manner (Australian Government, 2012).

The effective implementation of a stepped-care model, a focus on prevention and early intervention, and investment in digital mental health services are critical in addressing these gaps and transforming the future of mental health care.

Since 2012, the Australian Government has maintained that digital mental health, prevention and early intervention services remain key priorities. Increasingly, there is a push for an integrated stepped-care approach, and for the implementation of more integrated digital solutions within the primary health-care system (Australian Government, 2019; COAG, 2017; Department of Health, 2015; Department of Health, 2019; NMHC, 2014; Productivity Commission, 2019). This model would enable consumers to be better matched with the level of support they need, according to their symptoms, personal circumstances and preferences, and in a timely way, with a view to reducing the reliance on costly ‘downstream’ services (NMHC, 2014). While we have seen some progress, such as the launch of Head to Health in 2017 (Department of Health, 2018), there remains considerable work to be done to transform the delivery of mental health services, to address imbalances in government expenditure across the spectrum of mental health intervention, and to increase the awareness of digital services among consumers and health professionals. Digital services are well positioned to facilitate these goals, and will be critical in addressing several of the recommendations identified in the current proposed reforms. Digital services present a unique opportunity to address various shortcomings within the existing mental health system, and to overcome a number of barriers both service providers and consumers face within the sector. Given this, the government should remain focused on the provision of new and emerging digital technologies to optimise mental health services as we move into the new decade.

6


Mental health problems such as depression and anxiety typically have their first onset by adolescence or early adulthood.

7


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Capitalising on a captive audience While digital services are important more broadly, their potential for tackling the challenge of youth mental health is significant. Technology provides an avenue for reaching and delivering programs to young people who require support in their natural environment, and allows them to take control of their own mental health and wellbeing. Given the central role that technology plays in the lives of young people, there is an urgent imperative to harness the potential of digital technology to engage them in the treatment and management of their mental health. The widespread uptake of technology and social media among young people presents a unique opportunity for digital mental health services. There is overwhelming evidence that young people are highly engaged with technology and various digital channels, with teens spending approximately 3.3 hours per day on social media (APS, 2017). Young people largely use the internet to connect with others and for entertainment; however, there is growing evidence that they are also turning to digital platforms to access information and support (Hall et al., 2019).

8

Digital services offer novel and flexible modes of delivery to provide mental health support in ways that best suit a young person’s needs and personal circumstances. Digital mediums such as apps (Wang, Varma & Prosperi, 2018), websites (O'Kearney et al., 2009), interactive games (Shepherd et al., 2015), social media (Rice et al., 2018; Santesteban-Echarri et al., 2017), text messaging services (Gibson & Cartwright, 2014), and online counselling or therapy (Andrews et al., 2018) all provide innovative means of reaching both youth and their support networks, and of offering evidence-based resources to foster mental wellbeing. Digital services are therefore playing a progressively larger role in the delivery of mental health services and in supporting young people. These roles include providing information, prevention, assessment, diagnosis, peer-support, and counselling and treatment programs that address a range of conditions of varying severity (Kazdin, 2019; Lal, 2019).


INTRODUCTION

... poor mental health literacy represents a considerable barrier to timely receipt of care, as young people will often fail to recognise that their symptoms are serious enough to warrant support, hence preventing them from seeking help.

(Hall et al., 2019). Participants in the 2018 youth survey who were experiencing psychological distress reported that they would potentially go online to find information about specific issues, use an online quiz or assessment tool, chat to someone who has had a similar experience, access personal stories or testimonials, and find out about available services. Smaller proportions indicated they would access online counselling, participate in an online forum or support group, or use an online course or program (Hall et al., 2019). The potential for young people to remain anonymous when using digital services is also a significant facilitator to their engagement with these offerings. There is considerable evidence that, despite extensive public efforts to destigmatise mental health issues, young people often experience fear and embarrassment when contemplating whether to disclose that they are experiencing a mental health problem (Gulliver, Griffiths & Christensen, 2010; Hall et al., 2019). The barrier of stigma can be particularly pronounced in rural areas. ReachOut’s own research with young people, coupled with the wider literature, attests to the pervasive stigma attaching to mental illness and help-seeking in rural communities (Ivancic et al., 2018; Aisbett et al., 2007; Kõlves et al., 2012). Digital services can be accessed discreetly and anonymously, allowing young people to get the help they need without fear of judgement or embarrassment.

Circumventing the barriers to help-seeking In addition to their novelty and acceptability, digital services assist in overcoming the barriers to help-seeking that young people often report, including lack of awareness and understanding of mental health issues, financial limitations, limited service accessibility/ availability (e.g. geographical remoteness and lack of services), concerns about confidentiality and stigma, and a preference for self-reliance (Gulliver, Griffiths & Christensen, 2010; Hall et al., 2019). First and foremost, poor mental health literacy represents a considerable barrier to timely receipt of care, as young people will often fail to recognise that their symptoms are serious enough to warrant support, hence preventing them from seeking help (Gulliver, Griffiths & Christensen, 2010). Digital services can play a critical role in helping young people to build their awareness and understanding of mental health issues and mental health literacy. Mission Australia’s annual youth survey consistently demonstrates that when young people are experiencing an issue or a problem, one of the first places they turn to for help, particularly for stigmatised issues such as mental health, is the internet

Digital services also have the potential to reach large numbers of young people, particularly in areas where services may be limited or lacking. Service accessibility and availability is vital in ensuring that young people can access appropriate support in a timely manner. Unfortunately, long waiting lists, restricted opening hours, and the lack of easily accessible clinical services in some remote regions all impede young people’s ability to access help (Aisbett et al., 2007; Gulliver, Griffiths & Christensen, 2010). Fortunately, digital services help to overcome this barrier by being accessible whenever and wherever young people need them. Digital services such as ReachOut can provide young people with a sense of immediate relief from their emotional load, as well as resources to help them cope with challenges they may be facing in their lives. Digital self-help services like ReachOut may also provide alternatives to face-to-face care when it is not required and/or readily available, and might serve as useful tools for those waiting for or being transitioned out of face-to-face services (March et al., 2019). Furthermore, digital interventions can assist in overcoming the barrier of affordability of services. Despite the high demand for face-to-face counselling

9


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

and psychological treatment, these services can be financially inaccessible, and many Australians often forgo seeking professional help due to the associated financial burden (Callander, Corscadden & Levesque, 2017). The Australian Psychological Society (2019) recommends that the standard psychologist consultation fee be $251. While Australians suffering from a mental health problem may be eligible to receive up to ten rebated sessions of therapy, rebated sessions can still incur out-of-pocket costs. Moreover, ten sessions may be inadequate for those with more chronic and/or complex mental disorders. There are many Australian evidence-based digital mental health services that are free to use (e.g. Mood Gym, ReachOut, etc.) and can provide mental health support for young people and their families who may otherwise be forced to go without, or that may be used as an adjunct to face-to-face services (Andrews et al., 2018; March et al., 2019). Additionally, they can be an affordable solution for professionals and other gatekeepers who may be required to provide a young person with mental health support (e.g. teachers, sports coaches or social workers). The affordable, and often cost-free, delivery of digital services can facilitate access to mental health support for youth, families and their wider communities.

Finally, an attitude of self-reliance has been a notable barrier to help-seeking, with many young people feeling that they would prefer to, or should be able to, solve their own problems without external assistance or support (Gulliver, Griffiths & Christensen, 2010). These attitudes are likely related to normal developmental processes of individuation and autonomy, which have a particularly strong influence during late adolescence and emerging adulthood (Wilson, Deane & Ciarrochi, 2005; Wilson et al., 2011). Unfortunately, this can serve as a barrier to accessing traditional mental health services (Wilson et al., 2011; Wilson & Deane, 2012). However, it is important that young people in need are still able to access support that fulfils their need for autonomy. User-driven digital services such as ReachOut enable young people to take control of their own mental health and wellbeing by engaging with effective self-help strategies and connecting anonymously with peers in a safe, stigma-free, supportive space, as they develop their confidence and readiness to access formal services where needed (Gibson & Cartwright, 2014). Unfortunately, numerous barriers prevent young people from seeking the help they need. Given the growing prevalence of mental ill-health, it is important to explore innovative methods of service delivery to overcome these barriers. Not only do digital services have the potential to mitigate them, but there is growing evidence for the effectiveness of digital interventions in supporting mental health and wellbeing outcomes.

Evidence of promise The body of evidence for the effectiveness of digital mental health services is growing rapidly, with a number of Australian digital mental health resources demonstrating benefits for the mental health of Australian young people. For those experiencing mental illnesses such as anxiety or depression, self-directed internet interventions based on cognitive behavioural therapy (CBT) have been associated with symptom reduction following intervention (e.g. Mood Gym: O’Kearney et al., 2009; Calear et al., 2009). Furthermore, Australian online wellbeing interventions based in positive psychology have also been linked to increased wellbeing and decreased 10


Given the growing prevalence of mental ill-health, novel approaches are needed to reduce the incidence of these problems, to enable young people to flourish during this critical developmental period and to reduce the burden in the community.

stress in healthy youth (Manicavasagar et al., 2014), suggesting that online mental health resources can also support and maintain the mental health of youth who are not experiencing distress. In addition, apps designed to prevent suicide in vulnerable populations have been linked to reductions in distress (e.g. Ibobbly: Tighe et al., 2017), highlighting the potential of mobile technology for helping those in crisis. Further, online interventions may also help prevent young people from experiencing a relapse in mental illness, with research suggesting that structured online social media programs can provide support and maintain mood for those in remission

from depression (Rice et al., 2018). Given the versatility of digital mediums, digital services are well placed to support the broad range of unique mental health issues that young people, both well and unwell, experience. In addition to the growing evidence for the effectiveness of digital interventions, the emerging evidence base also indicates substantial economic benefits associated with adopting a digital approach. Research has shown that digital interventions can be cost-effective when compared to other help-seeking methods, other recognised treatment, and waitlist control groups (Donker et al., 2015; Le et al., 2019). 11


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Furthermore, delivery of preventative mental health programs through these channels is also likely to cost significantly less than the expense accrued over life from lack of or delayed treatment (Knapp, McDaid & Parsonage, 2011). For governments, digital mental health may be an avenue for national health-care savings: while the Australian economy loses an estimated $53 billion per year to mental health (Productivity Commission, 2019), research indicates that investment in preventative mental health interventions and digital services can create substantial savings in the long term (Knapp, McDaid & Parsonage, 2011; Le et al., 2019; Mihalopoulos & Chatterton, 2015). In summary, there is a clear rationale to invest in the development and evaluation of evidence-based digital services for youth mental health, based on the evidence of their acceptability, effectiveness and cost-effectiveness. Although there is substantial evidence for structured and clinician-led digital interventions, there has to date been limited research investigating the effectiveness of unstructured digital mental health interventions, such as that which ReachOut provides. The unstructured, multicomponent nature of ReachOut enables it to cater to a wide variety of mental health needs that are experienced by young people at different ages and stages – from everyday stress, through to learning to manage more complex mental health issues. Further, ReachOut gives a service experience that is tailored to each individual young person, offering flexibility and choice of a range of resources, which responds to young people’s need for autonomy. The ReachOut service attracts over 2.6 million people annually, indicating that this unstructured online service modality holds appeal for young people. The wider literature also supports the acceptability of unstructured online mental health services such as

12

ReachOut (Kauer, Mangan & Sanci, 2014; Manicavasagar et al., 2014; Povey et al., 2016), but evidence regarding their effectiveness in promoting the mental health and wellbeing of young people is lacking. ReachOut is committed to conducting an ongoing program of innovative and engaging research and evaluation with its users to evolve the evidence for digital mental health services for young people, and to drive continuous improvement of the service. Preliminary data from past cross-sectional evaluations of ReachOut shows promise. These evaluations found that the majority of young people ReachOut engages present with high levels of psychological distress and are not otherwise accessing any mental health care at their first visit (Collins et al., 2011; Metcalf & Kauer, 2013). Encouragingly, over a third of these users report increased intentions to seek help after visiting ReachOut. The results of the study detailed in this report will build on these earlier evaluations, employing a longitudinal study design and following users over a three-month period to understand the impact of engaging with ReachOut over time.


A LONGITUDINAL COHORT STUDY OF REACHOUT USERS Approximately 2000 ReachOut users participated in a longitudinal study in 2015 and 2016 that followed participants over a three-month period.

Research aims

Methods and participants

The study aimed to understand how young people experiencing symptoms of depression, anxiety and/or stress engage with an open access, unstructured digital intervention, as well as how this intervention impacts on mental health and help-seeking behaviour over time.

Participants were ReachOut users who were recruited through a pop-up on the ReachOut site. This was a rolling sample, with participants being asked to complete surveys carried out over a three-month period. Participants were eligible to enrol in the study if they met the following criteria:

The overarching research question was: ‘What happens after a young person uses ReachOut?’ Specifically, the study aimed to examine whether young people who were experiencing symptoms of depression, anxiety or stress would: •

experience changes in their symptoms after using ReachOut go on to access appropriate sources of help after using ReachOut.

The study also explored young people’s experience of using ReachOut, as well as their reflections on the impact of using the service.

Aged 16–25 years

Living in Australia

Had visited the ReachOut website to get information or support for themselves or someone they know.

Surveys were conducted at: •

baseline

1 week post baseline

5 weeks post baseline

3 months post baseline.

13


The surveys comprised several standardised scales and bespoke closed- and open-ended questions. The study was approved by the University of Melbourne Human Ethics Committee. The analyses were peer-reviewed in 2019 by an external academic from the University of Canberra. A number of participants were excluded from the analysis for age ineligibility. This resulted in a final sample of 1982 participants at baseline; however, the sample was reduced to 1638 participants at three months post baseline, due to attrition. The overall study retention from baseline to follow-up was 82.6 per cent. Fifteen participants also participated in a follow-up interview. The interviews provided more in-depth insights into ReachOut users’ service experience.

14


A LONGITUDINAL COHORT STUDY OF REACHOUT USERS

Who is using ReachOut? ReachOut appeals to a range of different audiences and there is diversity among our user base. However, the majority of young people who took part in the study were female, and many lived in metropolitan areas. ReachOut has a high level of engagement with the lesbian, gay, bisexual, transgender, queer, intersex and asexual (LGBTQIA+) community, and this was reflected in the sample of this study. A large proportion of participants identified as LGBQ+, accounting for 35 per cent of the sample, compared to an estimate of 11 per cent representation nationally (AHRC, 2014), and a further 4.6 per cent of participants were transgender and/or gender diverse.

Gender A large proportion of participants were female (n=1657), followed by male (n=233), and transgender or gender diverse (n=92).

83%

12%

5%

Female

Male

Transgender or gender diverse

47%

27%

26%

16–18 years

19–21 years

22–25 years

65%

35%

Heterosexual

LGBQ+

Age A large proportion of participants were within the 16–18 year-old age bracket (n=915). Smaller proportions were within the 19–21 (n=523) and 22–25 year-old age brackets (n=509).

Sexual orientation Most participants identified as heterosexual (n=1268). A large proportion also identified as gay or lesbian (n=103), bisexual (n=254), unsure or questioning (n=166), and other sexual identities (n=159).

Location A large proportion of participants reported living in a metropolitan area (n=1378). The remainder were in an inner regional (n=331), outer regional (n=119), or remote or very remote area (n=17).

75%

18%

6%

1%

Metro

Inner regional

Outer regional

Remote

15


16


FINDINGS ReachOut’s research and program evaluations have always indicated that the organisation has a unique and meaningful impact on the mental health and wellbeing of young people. However, the findings of this study provided even stronger evidence of ReachOut’s impact on young people in distress and further insight into the role it plays in young people’s lives. Overall, the results showed an improvement in mental health and wellbeing, and a reduction in suicide risk. There was also an increase in help-seeking behaviour among participants who were experiencing symptoms of depression, anxiety and/or stress.

17


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Mental health To examine and monitor the mental health and wellbeing of our users, participants completed the Depression, Anxiety and Stress Scale (DASS-21; Henry & Crawford, 2005) on three occasions: at baseline, at five weeks post baseline, and at the three-month follow-up (three months post baseline). The DASS-21 comprises three sub-scales that measure levels of depression, anxiety and stress. Each sub-scale is scored and classified as ‘normal’, ‘mild’, ‘moderate’, ‘severe’ or ‘extremely severe’ levels of depression, anxiety and stress. At baseline, a large proportion of users (68 per cent) scored in the ‘severe’ or ‘extremely severe’ range on symptoms of depression, anxiety and/or stress. Self-reported mental health status was also evaluated at baseline. While there were a number of participants who positively rated their mental health status, the vast majority of young people that participated in this study were highly distressed. At the beginning of the study,

60%

Anxiety

Week 5

Depression

20%

10%

0

Stress

24

Average scores on DASS symptoms subscales

30%

34.6%

40%

40%

44.5%

30%

22

20

18

16

14

12

Depression

Anxiety

Stress

Figure 1. Severe and extremely severe depression, anxiety and stress scores at baseline, week 5 and the three-month follow-up

18

Furthermore, there were no significant differences in mental health improvements between age, gender and sexuality. That is, improvements were observed across the spectrum for users with diverse characteristics.

Baseline

51.5% 38%

41.5%

40%

The mental health status of ReachOut’s users was measured over time, to examine the impact of using the service. We saw a significant reduction in the percentage of young people who were classified as ‘severe’ or ‘extremely severe’ on depression, anxiety and stress from baseline to follow-up (see Figure 1). The results of the study showed not only a reduction in the proportion of participants experiencing severe symptoms, but also an overall significant reduction in symptom scores of depression, anxiety and stress over the period of the study (see Figure 2).

Follow-up

55.2%

50%

71.7 per cent of participants rated their mental health as either ‘fair’ or ‘poor’.

Baseline

Week 5

Follow-up

Figure 2. Average depression, anxiety and stress scores across the three-month study


19


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Suicide risk

Help-seeking

To capture risk for suicide, young people completed an abbreviated version of the Suicidal Ideation Questionnaire (SIQ-8; Reynolds, 1987) at baseline, five weeks post baseline, and at the three-month follow-up. At the start of the study, 12.3 per cent of participants were at high risk of suicide.

We also examined the help-seeking behaviour of our users, including intended and actual help-seeking, previous experiences of seeking help from a mental health professional, barriers to help-seeking, as well as how help-seeking behaviour changed over the course of the study.

Suicide risk was also examined in relation to the severity of participants’ depression symptoms. At the beginning of the study, over a quarter of the participants who were classified as ‘extremely severe’ on the depression sub-scale were at high risk of suicide. As shown in Table 1, participants’ risk of suicide decreased with reduced severity of depression symptoms.

At the beginning of the study, 75 per cent of participants had previously seen a mental health professional. However, of those participants who had sought help, only 54.5 per cent said they found the help they received from that professional to be helpful. A smaller proportion of participants (14.6 per cent) had previously been admitted to hospital for a mental health issue. Although a large number of participants had a history of help-seeking, a substantial proportion (37.5 per cent) reported that they did not seek help despite needing it in the three months prior to the study. Given that this is not uncommon, we explored a number of barriers to help-seeking. We found that self-reliance – specifically, the belief that ‘if I had a problem I would solve it myself’ – was the most commonly endorsed barrier to help-seeking. Other salient barriers included concerns about what they may be expected to do if they were to seek help from a mental health professional, as well as the affordability of that help. See Figure 3 for the full list of barriers.

It was encouraging to see a significant reduction in the proportion of participants who were at high risk of suicide over the three-month period of the study, with a decrease from 12.3 per cent at baseline to 8.6 per cent of participants by the three-month follow-up.

Table 1. Depression severity scores and high-risk of suicide at baseline

Depression severity

20

High risk

Normal

0.3%

Mild

0.0%

Moderate

2.5%

Severe

5.4%

Extremely severe

29%

For some young people, ReachOut is the help they need; however, for others with more significant mental health challenges, we aim to facilitate their onward referral to mental health professionals who are better matched to their needs. Over the period of the study, it was encouraging to see an increase in help-seeking behaviours among participants who were experiencing symptoms of depression, anxiety and/or stress, with 11.6 per cent going on to seek help from a mental health professional and an additional 5.7 per cent seeing a doctor or GP. Interestingly, while a mental health professional was the most common source of intended help-seeking, the three most common sources that young people reported having sought help from within the past three months were friends, online sources, and parents. Although it was encouraging to see that participants had listed several likely places to seek help, and that a number of participants had been seeking help from various people and places, 5.8 per cent of participants stated they didn’t know where to go for help, and 5 per cent said they would not seek help for a mental health issue.


FINDINGS

For some young people, ReachOut is the help they need; however, for others with more significant mental health challenges, we aim to facilitate their onward referral to mental health professionals who are better matched to their needs.

If I had a problem I would solve it myself

66%

If I got professional help I may have to do or say something I dont want to

57%

I think I should work out my own problems

51%

I’d never want my family to know I was getting professional help Even if I had a problem, I’d be too embarrassed to get professional help for it Older adults really can’t understand the problems that young people have If I got professional help, I might find out I was crazy Even if I wanted to, I wouldn’t have time to get professional help

10%

20%

30%

Not at all helpful

50% 43%

50%

60%

70%

22% 26%

4%

0%

10%

37%

36% 33%

15%

Feel more able to deal with the issue

36%

32%

17%

38%

40%

21%

10%

Think more positively about yourself

35%

30%

13%

Feel like you are less alone

Figure 3. The proportion of participants who agreed or strongly agreed with potential barriers to seeking professional help

Fairly helpful

21%

31%

0%

Slightly helpful

Worry less about what other people would think about the issue

56%

I could not afford to get professional help even if I wanted to

Very helpful

20%

30%

48%

40%

50%

Figure 4. Participants’ helpfulness ratings of ReachOut across four domains

21


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

How does ReachOut help? Participants were asked about their experience of using ReachOut, including their main reasons for coming to ReachOut, their experience of the website, whether ReachOut was helpful to them, and if they would recommend it to others. A large proportion (71.3 per cent) of participants reported that they had visited ReachOut for help during a tough time, highlighting the important role ReachOut plays in young people’s lives. Participants reported that the most common reasons for visiting ReachOut were depression and anxiety, followed by self-harm, general mental health issues and suicide. Participants provided very positive experience ratings of ReachOut, with a large majority (90–95 per cent) rating their overall experience of ReachOut as ‘good’ or ‘excellent’. Furthermore, the majority of participants agreed that ReachOut is relevant (82 per cent), available and accessible (86.7 per cent), and that they ‘can see the benefits of ReachOut’ (78.2 per cent). Nearly 80 per cent of participants also agreed that ReachOut had helped them to understand the mental health experiences of others (78.1 per cent), as well as their own experiences 22

(71.6 per cent), and stated that it had given them a range of practical help strategies and tools (68.6 per cent). Young people also felt that ReachOut equipped them with coping strategies, and helped them to feel less alone in their experience, to feel more positive about themselves, and to worry less about what other people would think about the issue they were experiencing. As shown in Figure 4, the statements with the highest levels of endorsement were: ReachOut helps me to ‘feel more able to deal with the issue’ and ‘feel like you are less alone’.


Actions taken after visiting ReachOut Here is a selection of actions young people in the study took

Went to see my doctor

I talked to a friend about my problems

Worked hard at positive self-talk I sought further help from a health professional and began to improve my coping skills

Set goals that were more reachable. Separated my goals into several steps

I read about and looked into some resources. I also booked another appointment with a psychologist I have seen before

Talked to a trusted mentor

Got a pet, morethe exercise, ateIbetter Write down on did paper things am feeling

Talked to my parents about how I was feeling

I downloaded some apps that they suggested might help with some of my stress

Went to a school counsellor

Distanced myself from the person giving me trouble

Coping with self-harm

with relaxation like yoga

and meditation. Thought

about talking to someone

I followed recommendations

Made a suicide safety plan Got a pet, did more exercise, ate better

Learnt to focus attention on the root cause of my stresses and to seek additional help in the form of counselling

Breathing techniques

I learnt to think more positively about myself

Contacted Kids Helpline and Lifeline

Thought about the issue differently and opened up to a couple of friends about it

Visited other recommended websites and followed the advice on things to think about 23


YOUNG PEOPLE’S REFLECTIONS ON THE IMPACT OF USING REACHOUT Fifteen participants took part in follow-up interviews that explored how ReachOut users experienced the service. They mentioned the many benefits of ReachOut for their wellbeing and mental health. Young people spoke of feeling a decreased sense of isolation in terms of what was going on for them, which helped them to realise that many other people felt the same way they did, and offered hope that their experience may improve.

‘It’s great to get the info and know that it’s not just you, that you’re not “broken” and that there is a way through it, even when it does seem like there really is no hope or any options left.’

‘I read the articles and blogs related to the concerns that I had. It helped me see that I was not alone in the issue and that other people were going through the same thing I was. It gave me reassurance.’

Young people also felt that using ReachOut helped them to understand what they were going through and gave them the language to articulate this for themselves.

‘It feels like you’re stuck on a leaking boat in the ocean but you don’t know which land you should paddle towards… But from the site even just finding the name for a disorder and that it’s something so many people have… That was like being given the map and compass to sort out life.’ Young people also spoke about how ReachOut had increased their confidence in their ability to enact strategies to improve their mental health and wellbeing, and to put them into practice.

‘I suppose the most important thing I actually did was put my own help as a priority. I started thinking about what would help me be better as well as the people around me. Mostly realising that one’s health, be it emotional or physical, has a great impact on their current and future lives. And making sure you are in good health is always important.’

‘I made sure I was regularly exercising and eating well, and also maintaining a healthy sleep pattern – and I actually found that these things helped greatly.’

24


FINDINGS

Young people also told us how using ReachOut encouraged them to seek additional help by recognising the importance of doing so, both informally through friends and family and formally through speaking to counsellors.

‘[After visiting ReachOut] I spoke to my school counsellor, tried out some calming strategies and talked to a few friends about [things] and essentially “reached out” as best as I could while still feeling comfortable.’

Interviews with participants provided valuable insights as to the potential mechanisms by which ReachOut improves the mental health and wellbeing of young people. By helping young people to understand and talk about what they are going through, and to realise that they are not alone in that experience, and by equipping them with the knowledge, tools and confidence to undertake self-help or to access help from a mental health professional, ReachOut can be seen to provide a critical service for young people experiencing challenges to their mental health and wellbeing.

‘I went to see a counsellor at headspace for a while after I looked up some things [on ReachOut] and it helped me speak about it to my family a bit more. [ReachOut] did help with understanding that I did need to go talk to someone about things and that it was okay to do that.’

25


RECOMMENDATIONS

The study aimed to understand the impact of ReachOut, an unstructured digital mental health intervention, by exploring the possible impacts of ReachOut over time on mental health and wellbeing outcomes, and on help-seeking behaviour. The findings demonstrate that over a three-month period, young people using ReachOut experienced a significant reduction in mental health symptoms such as depression, anxiety and/or stress, and in suicide risk, and an increase in help-seeking behaviour. Follow-up interviews with participants provided further insight into the role that ReachOut plays in the lives of young people, and how it can benefit their wellbeing. Young people spoke about experiencing a decreased sense of isolation, increased understanding and ability to articulate their lived experiences, as well as increasing confidence in implementing strategies to improve their mental health. The burden associated with mental ill-health is continuing to grow in Australia, and remains the largest contributor to the non-fatal health burden among our young people (AIHW, 2019). Yet, there are large numbers of young people who are not seeking help despite needing it (Hall et al., 2019). Although we have seen some progress, there remains considerable work to be done to transform how mental health services are delivered, and to reduce the burden of mental ill-health in the community.

Technology, in its many forms, including digital interventions, is making the fastest gains in delivering health care and is recognised for its scalability and reach (Kazdin, 2019). It can provide interventions to large numbers of young people in a user-friendly and highly accessible manner. Digital mental health services also offer a unique opportunity to access hard-to-reach groups, overcome barriers to accessing traditional mental health-care services, and provide onward referral when needed (Kazdin, 2019). ReachOut provides an online platform where young people can conveniently and discreetly access relevant and helpful resources and support through a device in the palm of their hand. ReachOut is designed to help young people to navigate the challenges they may face, from everyday issues to tough times and mental health problems. Although ReachOut’s ongoing research and program evaluations have helped us to understand the benefits of digital mental health services and the subjective impact of ReachOut on the wellbeing of young people, this study has provided a more robust understanding of how ReachOut helps young people who are experiencing distress. While the reductions in symptoms observed in this study were relatively modest, these improvements should not be discounted, particularly given the aforementioned burden associated with poor mental health during this developmental stage and the brief, unstructured nature of the intervention (AIHW, 2019). We observed medium effect sizes for reductions in symptoms of anxiety (np2 =0.05) and stress (np2 =0.05), and a large effect size for improvements in symptoms of depression (np2 =0.11). Reviews of prevention programs targeting all young people irrespective of symptoms (i.e. universal interventions), and young people with some level of risk or symptoms (i.e. selective or indicated prevention), have typically reported small to moderate effects in reduction of


RECOMMENDATIONS

symptoms and diagnoses of depression and anxiety (e.g. Farrer et al., 2018; Hetrick et al., 2016; Stockings et al., 2016; Werner-Seidler et al., 2017). Notably, the majority of these trials are delivered face-to-face, most commonly in the school setting. Digital platforms may provide a more cost-effective platform from which to deliver effective prevention interventions to reach

more young people. Although these interventions are not all directly comparable, it is interesting to note the overall effects of ReachOut, a brief, self-directed and relatively inexpensive intervention, alongside more resource-intensive programs. The effects highlight the value that ReachOut and other digital services can deliver. 27


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

It is also noteworthy that similar levels of improvement in mental health symptoms were observed across various groups, including by age, gender and sexuality. We hypothesise that this is related to the involvement of diverse young people in the development of service resources and the flexibility of the intervention. Young people from all backgrounds are involved in the co-creation of ReachOut information, support and tools. In addition, the unstructured nature of ReachOut allows young people to navigate and select information and/or support on a wide variety of topics and from diverse perspectives. The result is a flexible intervention that is adaptable to the individual, and yet can be scaled up to reach large numbers of young people in need. Furthermore, it is well established that young people are often reluctant to seek help for mental health problems (Rickwood, Deane & Wilson, 2007), with the main barriers to help-seeking being stigma, embarrassment and a preference for self-reliance (Gulliver, Griffiths & Christensen, 2010). A review by Kauer, Mangan and Sanci (2014) identified that although young people’s experiences of online mental health services are typically very positive, to date there has been little evidence to suggest that these services facilitate help-seeking. It was therefore encouraging to see, among those young people using ReachOut who had not sought help from a mental health professional, an increase in help-seeking intentions and behaviours over the three-month period of the study. This finding highlights the considerable potential of digital services to mitigate common barriers to help-seeking. Interestingly, the cohort of young people that participated in this study had fairly significant mental health needs, and many had sought help from a mental health professional previously. Of concern, however, is that many did not rate their experience with this professional as helpful. It was encouraging that these young people with unmet mental health needs had experienced improvements in their mental health symptoms after engaging with ReachOut, and they rated the support they received from ReachOut in a more positive light. Although it is primarily intended as a prevention/early intervention service, the findings from this report highlight that ReachOut is supporting young people across the spectrum of intervention and all the way through their mental health journey, in a way that other services are unlikely to be able to do. The findings of this study also demonstrate that, although many young people intended to seek help from a mental health professional, they more frequently relied on friends, parents and online resources. The Mission Australia youth survey reports similar findings, with friends, parents/caregivers and the internet consistently being the most commonly cited 28

sources of support (Hall et al., 2019). This finding not only highlights the reliance on digital resources, but also speaks to the importance of services such as ReachOut and ReachOut Parents, which offer resources and support to people, such as friends or parents, who may find themselves providing informal support to a young person. Given the wealth of evidence regarding the relationship between social support and increased wellbeing (e.g. Hakulinen et al., 2016; Ronen et al., 2016; Thoits, 2011), these informal supports should not be discounted. However, it is important that these individuals are equipped with the skills to navigate conversations sensitively, and in ways that will facilitate receipt of appropriate care, be it self-help or seeing a mental health professional. It is also important that family and friends are supported to prioritise their own self-care while caring for the young person in their lives. Overall, the study provided valuable insights into the benefits of digital interventions; however, it is not without its limitations. First, participants self-selected into the study and there were minimal eligibility criteria other than age, location and current use of ReachOut. Furthermore, while the results indicate significant improvements in mental health, suicide risk and help-seeking among young people, the lack of a control group precludes the definitive attribution of these outcomes to the use of ReachOut.


It is well established that young people are often reluctant to seek help for mental health problems, with the main barriers to help-seeking being stigma, embarrassment and a preference for self-reliance

The growing burden of mental illness in Australia’s young people warrants urgent attention. While some young people seek help, the level of unmet needs among this population remains unacceptable. Accessing appropriate and timely support is imperative in improving the wellbeing of young people. As such, we need to ensure that services are easily available and accessible, relevant and affordable, to ensure that young people who need help are not prevented from obtaining it. The findings of this study highlight the benefits of digital interventions and point to the need for further investment in these services. Digital mental health services will play a large role in overcoming barriers of traditional mental health care for young people, as well as more broadly addressing several gaps within the current mental health-care landscape.

29


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Recommendation 1: Invest in programs and interventions that promote wellbeing in young people. As many experiences of mental illness have their origins in childhood and adolescence, there is a clear imperative to resource programs that build protective factors in young people, such as equipping them with problem-solving skills, coping strategies, and information about everyday issues, as well as providing them with a safe and supportive environment, to help mitigate future experiences of mental illness. As such, taking a preventative approach is expected to yield both social and economic benefits while also reducing the burden on the mental health-care system (Arango et al., 2018; Mihalopoulos & Chatterton, 2015; Rice, Purcell & McGorry, 2018).

30


RECOMMENDATIONS

Recommendation 2: Harness the potential of digital technology to improve service availability and accessibility. Technology offers an efficient method of delivering scalable, evidence-based and cost-effective programs and interventions. Not only do digital platforms allow young people to engage with relevant services in a discreet and convenient manner, but they also aid in delivering services to hard-to-reach groups, including those residing in isolated areas with limited access to face-toface services. Digital approaches also enable services to be responsive and to harness ongoing technological developments, whether this is producing timely and relevant content, such as support for young people after experiencing a natural disaster, or artificial intelligence or machine learning, as a method of detection, prevention and program delivery (Larsen et al., 2015; Merry et al., 2012). Furthermore, digital interventions are available whenever and wherever. Their user-driven approach provides a young person with a vital sense of autonomy in their recovery journey and control over their own mental health and wellbeing. Digital services promote self-management, and provide a space to find and share hope, connection and emotional support (Gibson & Trnka, 2020; Gibson & Cartwright, 2014).

An example of this is ReachOut’s NextStep tool, which is designed to gather minimal information from the user to generate personalised and tailored self-help and help-seeking recommendations. This intervention has been found to be cost-effective (Le et al., 2019) and effective in reducing negative affect and improving satisfaction in young people with the help-seeking process up to three months after using the tool (Sanci et al., 2019). We hypothesise that a greater investment in personalising the ReachOut service offering would drive further improvement in the user experience and mental health impacts associated with engaging with ReachOut. For example, personalisation would allow ReachOut to better identify users in distress and who require urgent help, ensuring that young people at risk of suicide are identified more quickly and offered a referral and/or one-on-one specialist support.

One of the main advantages of digital modes of service delivery is the flexibility they offer to assess the characteristics and needs of the user, and to personalise and tailor the intervention accordingly. ‘Personalisation’ relates to the use of specific program content or features based on the personal characteristics of users, while ‘tailoring’ refers to the presentation of information or features that are relevant to the user’s needs or wants (Ritterband et al., 2009). Personalisation and tailoring are key facilitators of engagement with, and effectiveness of, digital self-help interventions (Ritterband et al., 2009).

31


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Recommendation 3: Clarify and solidify the role of digital services in a stepped model of care, and build service readiness. Digital mental health services offer an incredibly flexible platform that can guide and direct young people experiencing mental health challenges to service delivery options that are best matched to their needs and preferences. The mental health-care system is stretched and is struggling to meet the demand for services in the community (NMHC, 2019; Lawrence et al., 2016), particularly early in life where there is a peak in the incidence of mental health problems (Kessler et al., 2005). Anonymous, unstructured interventions that are delivered online are likely to have a low threshold for access, given their potential to circumvent many of the established barriers to face-to-face help-seeking (Gulliver, Griffiths & Christensen, 2010), which makes them an optimal entry point to support the implementation of a stepped-care model. Furthermore, digital services and tools have considerable potential to re-distribute the burden of care, enabling young people with mild to moderate symptoms to be matched with less intensive care online, thus freeing up space in more resource-intensive, face-to-face services for young people with more severe symptoms (Kauer et al., 2017).

‘Digital services and tools have considerable potential to re-distribure the burden of care.’ For some young people, digital self-help resources and peer support may be sufficient to bring about an improvement in their mental health and wellbeing, while others facing more severe mental health challenges may require more intensive faceto-face intervention. For those requiring a more intensive intervention, digital services can act as a first port of call, building their mental health and

32

emotional literacy, reducing self-stigma and feelings of isolation, and readying them to engage with a mental health service that is better matched to their level of need. This might involve clarifying expectations about what engagement with a mental health professional involves, educating them about different service options and their rights as a consumer, sharing personal stories from other young people about their experience of getting support for a mental health problem, and building their sense of agency and motivation to seek additional help. Digital mental health services may also be a valuable complement to face-to-face services as young people’s symptoms improve and they are transitioned out of formal treatment services, providing a less intensive and more easily accessible form of support (Santesteban-Echarri et al., 2017).


RECOMMENDATIONS

Recommendation 4: Recognise the importance of participatory design to ensure that services meet the needs of young people. It is important that services are welcoming and accessible to all young people, recognising the unique barriers and mental health challenges faced by the LGBTQIA+ community, Aboriginal and Torres Strait Islander young people, rural and regional youth, as well as young people who are culturally and linguistically diverse. By co-designing services with diverse groups of current and/or prospective users, as well as adopting user-led experience goals and principles, services can ensure they are tailored for people with different needs. Participatory approaches have been shown to enhance uptake of interventions and services, as they are more accessible and responsive to the needs of communities and individuals (Nicholas et al., 2012). The inclusion of users throughout the cycle of design, development and evaluation of tools and programs recognises that the end users are the experts in their lives, and in turn increases the likelihood that the end product is engaging and useful (Hagen et al., 2012; Blake et al., 2016). Furthermore, there is good evidence that when digital self-help interventions are co-designed with these groups and tailored to their specific needs, there are tangible benefits in terms of uptake, engagement and impact (e.g. Tighe et al., 2017; Shepherd et al., 2015; Lucassen et al., 2015; Povey et al., 2016).

Recommendation 5: Invest in research and program evaluation to drive quality assurance and effectiveness of mental health services, and to facilitate better treatment outcomes across the mental health sector. As highlighted in the National Report, it is important to remain innovative in our approaches to addressing the complexities of mental health and suicide prevention (NMHC, 2019). Research will play a significant role in improving mental health outcomes by identifying and monitoring the prevalence of risk and protective factors, evaluating the effectiveness of various programs and service delivery models, and translating research findings into evidence-based practices. The potential for digital interventions for youth mental health is widely promoted, yet there is increasing recognition that there is a need for more rigorous research and evaluation to support the development of interventions guided by an evidence base (Rickwood, 2012). While there is emerging evidence for digital services and resources, the limited evidence base to date is largely focused on structured and guided self-help. Although this research has provided some preliminary evidence of the effectiveness of unstructured, self-guided interventions, and of the appeal of these resources to young people, further research is required. The impact of program evaluation to support evidence-based implementation and translation can be maximised by careful planning and resourcing from the outset of a project. Ideally, 5–15 per cent of a program’s budget will be allocated to evaluation (Better Evaluation, 2019); however, this is rarely made available. To maximise this investment in research and evaluation, it is also important that the findings are made available to a broad range of potential users, which can be facilitated via dissemination through different channels including peer-reviewed journals, subject matter conferences, social media, and media releases.

33


CONCLUSION Overall, the findings of this study illustrate the impact that digital prevention and early intervention services can have for youth mental health. This research provided evidence of the effectiveness of an unstructured digital mental health service in reducing young people’s symptoms of depression, anxiety and/or stress, and risk of suicide, as well as mitigating common barriers to help-seeking. While more intensive, face-to-face treatment service modalities clearly have their place, it is important that early intervention and prevention are prioritised to help reduce the incidence of common mental health disorders early in the lifespan, and to maximise the efficiency of the limited resources available (Scott, Thomas & Erskine, 2019). This report has also highlighted the value of digital services, exploring how they can aid in the achievement of key recommendations made in the National Report, and ultimately assist in reducing the burden of mental ill-health in Australia. The research presented in this report is helping ReachOut to understand who uses its services, what role it plays in their lives, and how it helps them to take control of their mental health and wellbeing. ReachOut is committed to conducting an ongoing program of innovative and engaging research and evaluation with its users, to help design and deliver a service that best meets the needs of all young people.

34


ReachOut is committed to conducting an ongoing program of innovative and engaging research and evaluation with its users, to help design and deliver a service that best meets the needs of all young people.

35


REFERENCES Aisbett, D., Boyd, C., Francis, K., & Newnham, K. (2007). Understanding barriers to mental health service utilization for adolescents in rural Australia. Rural and Remote Health, 7, 624. Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018). Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated metaanalysis. Journal of Anxiety Disorders, 55, 70–78. https://doi.org/10.1016/j. janxdis.2018.01.001 Arango, C., Díaz-Caneja, C. M., McGorry, P. D., Rapoport, J., Sommer, I. E., Vorstman, J. A., ... & Carpenter, W. (2018). Preventive strategies for mental health. The Lancet Psychiatry, 5(7), 591– 604. https://doi.org/10.1016/s22150366(18)30057-9 Australian Bureau of Statistics (2018). National Health Survey: First Results, 2017–2018. Retrieved from: https:// www.abs.gov.au/ausstats/abs@.nsf/ mf/4364.0.55.001 Australian Bureau of Statistics (2019). Causes of Death, 2018. Retrieved from: https://www.abs.gov.au/Causes-ofDeath Australian Government (2012). E-Mental Health Strategy for Australia. Canberra: Department of Health and Ageing. Australian Government (2019). Australian Government Response to the Senate Community Affairs References Committee Inquiry Report: Accessibility and Quality of Mental Health Services in Rural and Remote Australia. Retrieved from: https://www.health.gov.au/sites/ default/files/response-accessibility-andquality-of-mental-health-services-in-ruraland-remote-australia.pdf

36

Australian Human Rights Commission (2014). Face the Facts: Lesbian, Gay, Bisexual, Trans and Intersex People. Retrieved from: https://www. humanrights.gov.au/sites/default/files/7_ FTF_2014_LGBTI.pdf Australian Institute of Health and Welfare (2019). Australian Burden of Disease Study: Impact and Causes of Illness and Death in Australia 2015 – Summary Report. Australian Burden of Disease Study Series No. 18. Cat. no. BOD 21. Canberra: AIHW. Australian Psychological Society (2017). ‘Digital Me’. Retrieved from: https:// apo.org.au/sites/default/files/resourcefiles/2017/11/apo-nid118961-1180226. pdf

Collin, P. J., Metcalf, A. T., StephensReicher, J. C., Blanchard, M. E., Herrman, H. E., Rahilly, K., & Burns, J. M. (2011). ReachOut.com: The role of an online service for promoting help-seeking in young people. Advances in Mental Health, 10(1), 39–51. Council of Australian Governments Health Council (2017). The Fifth National Mental Health and Suicide Prevention Plan. Canberra: Australian Government Department of Health. Retrieved from: http://www.coaghealthcouncil.gov. au/Portals/0/Fifth%20National%20 Mental%20Health%20and%20 Suicide%20Prevention%20Plan.pdf Department of Health (2015). Summary of

Australian Psychological Society (2019). Psychologist fees. Retrieved from: https://www.psychology.org.au/for-thepublic/about-psychology/what-it-costs

Actions within the Australian Government

Better Evaluation (2019). Identify what resources are available for the evaluation and what will be needed. Retrieved from: https://www.betterevaluation.org/en/ node/5286

www1.health.gov.au/internet/main/

Blake, V., Buhagiar, K., Kauer, S., Nicholas, M., Sanci, L., & Grey, J. (2016). Using participatory design to engage young people in the development of a new online tool to increase help-seeking. Journal of Applied Youth Studies, 1(3), 6–83. Calear, A. L., Christensen, H., Mackinnon, A., Griffiths, K. M., & O’Kearney, R. (2009). The YouthMood Project: A cluster randomized controlled trial of an online cognitive behavioral program with adolescents. Journal of Consulting and Clinical Psychology, 77(6), 1021–32. Callander, E. J., Corscadden, L., & Levesque, J. F. (2017). Out-of-pocket healthcare expenditure and chronic disease: Do Australians forgo care because of the cost? Australian Journal of Primary Health, 23(1), 15–22.

Response to Recommendations to the Review of Mental Health Programmes and Services. Retrieved from: https:// publishing.nsf/Content/EDB3AA47E060 9E3ACA257F06007F794D/$File/actions. pdf Department of Health (2018). Department of Health 2017–2018 Annual Report. Canberra: Australian Government Department of Health. Retrieved from: www.health.gov.au/internet/ main/publishing.nsf/ Content/AnnualReport2017-18 Department of Health (2019). Australia’s Long-term National Health Plan to Build the World’s Best Health System. Retrieved from: https://www.health.gov. au/sites/default/files/australia-s-longterm-national-health-plan_0.pdf Donker, T., Blankers, M., Hedman, E., Ljótsson, B., Petrie, K., & Christensen, H. (2015). Economic evaluations of internet interventions for mental health: A systematic review. Psychological Medicine, 45(16), 3357–76. doi:10.1017/ S0033291715001427


REFERENCES

Farrer, L. M., Walker, J., Harrison, C., & Banfield, M. (2018). Primary care access for mental illness in Australia: Patterns of access to general practice from 2006 to 2016. PLoS ONE, 13(6), e0198400. https://doi.org/10.1371/ journal.pone.0198400 Fergusson, D. M., Boden, J. M., & Horwood, L. J. (2007). Recurrence of major depression in adolescence and early adulthood, and later mental health, educational and economic outcomes. British Journal of Psychiatry, 191(4), 335–42. Gibson, K., & Cartwright, C. (2014). Young people’s experiences of mobile phone text counselling: Balancing connection and control. Children and Youth Services Review, 43, 96–104. https://doi. org/10.1016/j.childyouth.2014.05.010 Gibson, K., & Trnka, S. (2020). Young people’s priorities for support on social media: ‘It takes trust to talk about these issues’. Computers in Human Behavior, 102, 238–47. https://doi.org/10.1016/j. chb.2019.08.030 Gulliver, A., Griffiths, K. M., & Christensen, H. (2010). Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry, 10(1), 113. https://doi. org/10.1186/1471-244X-10-113 Hagen, P., Collin, P., Metcalf, A., Nicholas, M., Rahilly, K., & Swainston, N. (2012). Participatory Design of Evidencebased Online Youth Mental Health Promotion, Intervention and Treatment. Melbourne: Young and Well CRC Publication. Hakulinen, C., Pulkki-Råback, L., Jokela, M., Ferrie, J. E., Aalto, A. M., Virtanen, M., ... & Elovainio, M. (2016). Structural and functional aspects of social support as predictors of mental and physical health trajectories: Whitehall II cohort study. Journal of Epidemiological Community Health, 70(7), 710–15. Hall, S., Fildes, J., Perrens, B., Plummer, J., Carlisle, E., Cockayne, N., & Werner-Seidler, A. (2019). Can We Talk? Seven Year Youth Mental Health Report, 2012–2018. Sydney: Mission Australia.

Henry, J. D., & Crawford, J. R. (2005). The 21-item version of the Depression Anxiety Stress Scales (DASS–21): Normative data and psychometric evaluation in a large non-clinical sample. British Journal of Clinical Psychology, 44, 227–39. https://doi. org/10.1348/014466505x29657 Hetrick, S. E., Cox, G. R., Witt, K. G., Bir, J. J., & Merry, S. N. (2016). Cognitive behavioural therapy (CBT), third-wave CBT and interpersonal therapy (IPT) based interventions for preventing depression in children and adolescents. Cochrane Database of Systematic Reviews, (8). https://doi. org/10.1002/14651858.CD003380.pub4 Ivancic, L., Cairns, K., Shuttleworth, L., Welland, L., Fildes, J. & Nicholas, M. (2018). Lifting the Weight: Understanding Young People’s Mental Health and Service Needs in Regional and Remote Australia. Sydney: ReachOut Australia and Mission Australia. Jorm, A. F. (2012). Mental health literacy: Empowering the community to take action for better mental health. The American Psychologist, 67(3), 231–43. Jorm, A. F. (2014). Why hasn’t the mental health of Australians improved? The need for a national prevention strategy. Australian & New Zealand Journal of Psychiatry, 48(9), 795–801. https://doi. org/10.1177/0004867414546387 Kauer, S. D., Buhagiar, K., Blake, V., Cotton, S., & Sanci, L. (2017). Facilitating mental health help-seeking by young adults with a dedicated online program: A feasibility study of Link. BMJ Open, 7(7), e015303. https://doi.org/10.1136/ bmjopen-2016-015303 Kauer, S. D., Mangan, C., & Sanci, L. (2014). Do online mental health services improve help-seeking for young people? A systematic review. Journal of Medical Internet Research, 16(3), e66. https://doi. org/10.2196/jmir.3103 Kazdin, A. E. (2019). Annual research review: Expanding mental health services through novel models of intervention delivery. Journal of Child Psychology and Psychiatry, 60(4), 455–72. https://doi. org/10.1111/jcpp.12937

Kessler, R. C., Amminger, G. P., AguilarGaxiola, S., Alonso, J., Lee, S., & Ustün, T. B. (2007). Age of onset of mental disorders: A review of recent literature. Current Opinion in Psychiatry, 20(4), 359–64. doi:10.1097/ YCO.0b013e32816ebc8c Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV Disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry, 62(6), 593–602. https://doi. org/10.1001/archpsyc.62.6.593 Kessler, R. C., & Wang, P. S. (2009). Epidemiology of depression. In I. H. Gotlib & C. L. Hammen (eds.), Handbook of Depression (pp. 5–22). New York: The Guilford Press. Knapp, M., McDaid, D., & Parsonage, M. (2011). Mental Health Promotion and Mental Illness Prevention: The Economic Case. Retrieved from: http://eprints.lse. ac.uk/38227/1/Knapp_et_al_MHPP_ The_Economic_Case.pdf Kõlves, K., Milner, A., McKay, K., & De Leo, D. (2012). Suicide in Rural and Remote Areas of Australia. Brisbane: Australian Institute for Suicide Research and Prevention. Lal, S. (2019). E-mental health: Promising advancements in policy, research, and practice. Healthcare Management Forum, 32(2), 56–62. https://doi. org/10.1177/0840470418818583 Larsen, M. E., Cummins, N., Boonstra, T. W., O’Dea, B., Tighe, J., Nicholas, J., … Christensen, H. (2015). The use of technology in suicide prevention. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), (August), 7316–19. https://doi. org/10.1109/embc.2015.7320081 Lawrence, D., Hafekost, J., Johnson, S. E., Saw, S., Buckingham, W. J., Sawyer, M. G., ... & Zubrick, S. R. (2016). Key findings from the second Australian child and adolescent survey of mental health and wellbeing. Australian & New Zealand Journal of Psychiatry, 50(9), 876–86. https://doi. org/10.1177/0004867415617836 37


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Le, L. K.-D., Sanci, L., Chatterton, M. L., Kauer, S., Buhagiar, K., & Mihalopoulos, C. (2019). The cost-effectiveness of an internet intervention to facilitate mental health help-seeking by young adults: Randomized controlled trial. Journal of Medical Internet Research, 21(7), e13065. https://doi.org/10.2196/13065 Lucassen, M. F. G., Merry, S. N., Hatcher, S., & Frampton, C. M. A. (2015). Rainbow SPARX: A novel approach to addressing depression in sexual minority youth. Cognitive and Behavioral Practice, 22(2), 203–16. https://doi.org/10.1016/j. cbpra.2013.12.008 Manicavasagar, V., Horswood, D., Burckhardt, R., Lum, A., Hadzi-Pavlovic, D., & Parker, G. (2014). Feasibility and effectiveness of a web-based positive psychology program for youth mental health: Randomized controlled trial. Journal of Medical Internet Research, 16(6), e140 https://doi.org/10.2196/ jmir.3176 March, S., Donovan, C. L., Baldwin, S., Ford, M., & Spence, S. H. (2019). Using stepped-care approaches within internet-based interventions for youth anxiety: Three case studies. Internet Interventions, 18, 100281. https:// doi.org/https://doi.org/10.1016/j. invent.2019.100281 Metcalf, A., & Kauer, S. (2013). ReachOut. com National Survey 2012. Sydney: Inspire Foundation. Merry, S. N., Stasiak, K., Shepherd, M., Frampton, C., Fleming, T., & Lucassen, M. F. G. (2012). The effectiveness of SPARX, a computerised self help intervention for adolescents seeking help for depression: Randomised controlled non-inferiority trial. British Medical Journal, 344, e2598. https://doi. org/10.1136/bmj.e2598 Mihalopoulos, C., & Chatterton, M. L. (2015). Economic evaluations of interventions designed to prevent mental disorders: A systematic review. Early Intervention in Psychiatry, 9(2), 85–92. https://doi.org/10.1111/eip.12156 National Mental Health Commission (2014). The National Review of Mental Health Programmes and Services. Sydney: NMHC. 38

National Mental Health Commission (2019).

Rice, S. M., Purcell, R., & McGorry, P. D.

Monitoring Mental Health and Suicide

(2018). Adolescent and young adult

Prevention Reform: National Report

male mental health: Transforming

2019. Sydney: NMHC.

system failures into proactive models

Nicholas, M., Hagen, P., Rahilly, K., & Swainston, N. (2012). Using participatory design methods to engage the uninterested. In Proceedings of the

of engagement. Journal of Adolescent Health, 62(3), S9–S17. https://doi. org/10.1016/j.jadohealth.2017.07.024 Rickwood, D. (2012). Entering the

12th Participatory Design Conference:

e-spectrum: An examination of new

Exploratory Papers, Workshop

interventions for youth mental health.

Descriptions, Industry Cases, Vol.

Youth Studies Australia, 31(4), 18–27.

2 (pp. 121–24). Association for

Rickwood, D. J., Deane, F. P., & Wilson,

Computing Machinery. https://doi.

C. J. (2007). When and how do young

org/10.1145/2348144.2348183

people seek professional help for mental

O’Kearney, R., Kang, K., Christensen, H., & Griffiths, K. (2009). A controlled trial of a school-based internet program for reducing depressive symptoms in adolescent girls. Depression and Anxiety, 26(1), 65–72. Povey, J., Mills, P. P., Dingwall, K. M., Lowell, A., Singer, J., Rotumah, D., … Nagel, T. (2016). Acceptability of mental health apps for Aboriginal and Torres Strait Islander Australians: A qualitative study. Journal of Medical Internet Research, 18(3), 1–12. https://doi. org/10.2196/jmir.5314 Productivity Commission (2019). Mental Health: Draft Report. Canberra. Reynolds, W. M. (1987). Suicidal Ideation Questionnaire (SIQ). Odessa, FL: Psychological Assessment Resources. Rice, S., Gleeson, J., Davey, C., Hetrick,

health problems? Medical Journal of Australia, 187(S7), S35–S39. https:// doi.org/10.5694/j.1326-5377.2007. tb01334.x Ritterband, L. M., Thorndike, F. P., Cox, D. J., Kovatchev, B. P., & Gonder-Frederick, L. A. (2009). A behavior change model for internet interventions. Annals of Behavioral Medicine, 38(1), 18–27. doi. org/10.1007/s12160-009-9133-4 Ronen, T., Hamama, L., Rosenbaum, M., & Mishely-Yarlap, A. (2016). Subjective well-being in adolescence: The role of self-control, social support, age, gender, and familial crisis. Journal of Happiness Studies, 17(1), 81–104. Sanci, L., Kauer, S., Thuraisingam, S., Davidson, S., Duncan, A. M., Chondros, P., ... & Buhagiar, K. (2019). Effectiveness of a Mental Health Service Navigation Website (Link) for

S., Parker, A., Lederman, R., Wadley,

Young Adults: Randomized Controlled

G., Murray, G., Herrman, H., Chambers,

Trial. Journal of Medical Internet

R., Russon, P., Miles, C., D’Alfonso, S.,

Research Mental Health, 6(10), e13189.

Thurley, M., Chinnery, G., Gilbertson, T.,

Santesteban-Echarri, O., Rice, S.,

Eleftheriadis, D., Barlow, E., Cagliarini,

Wadley, G., Lederman, R., D’Alfonso,

D., Toh, J., McAlpine, S., Koval, P.,

S., Russon, P., … Álvarez-Jiménez, M.

Bendall, S., Jansen, J. E., Hamilton,

(2017). A next-generation social media-

M., McGorry, P., & Alvarez-Jimenez, M.

based relapse prevention intervention

(2018). Moderated online social therapy

for youth depression: Qualitative data

for depression relapse prevention in

on user experience outcomes for social

young people: Pilot study of a ‘next

networking, safety, and clinical benefit.

generation’ online intervention. Early

Internet Interventions, 9, 65–73. https://

Intervention in Psychiatry, 12, 613–25.

doi.org/https://doi.org/10.1016/j.

doi:10.1111/eip.12354

invent.2017.06.002


REFERENCES

Scott, J. G., Thomas, H. J., & Erskine, H. E. (2019). Improving Australia’s population mental health: An ounce of prevention is worth a pound of cure. Australian & New Zealand Journal of Psychiatry, 53(5), 470–71. Shepherd, M., Fleming, T., Lucassen, M., Stasiak, K., Lambie, I., & Merry, S. N. (2015). The design and relevance of a computerized gamified depression therapy program for Indigenous Māori adolescents. Journal of Medical Internet Research Serious Games, 3(1), e1. https://doi.org/10.2196/games.3804 Stockings, E. A., Degenhardt, L., Dobbins, T., Lee, Y. Y., Erskine, H. E., Whiteford, H. A., & Patton, G. (2016). Preventing depression and anxiety in young people: A review of the joint efficacy of universal, selective and indicated prevention. Psychological Medicine, 46(1), 11–26. https://doi.org/10.1017/ S0033291715001725 Thoits, P. A. (2011). Mechanisms linking social ties and support to physical and mental health. Journal of Health and Social Behavior, 52(2), 145–61. Tighe, J., Shand, F., Ridani, R., Mackinnon, A., De La Mata, N., & Christensen, H. (2017). Ibobbly mobile health intervention for suicide prevention in Australian Indigenous youth: A pilot randomised controlled trial. BMJ Open, 7(1), e013518. https://doi.org/10.1136/ bmjopen-2016-013518

Wilson, C. J., & Deane, F. P. (2012). Brief report: Need for autonomy and other perceived barriers relating to adolescents’ intentions to seek professional mental health care. Journal of Adolescence, 35(1), 233–37. https://doi.org/10.1016/j. adolescence.2010.06.011 Wilson, C. J., Deane, F. P., & Ciarrochi, J. (2005). Can hopelessness and adolescents’ beliefs and attitudes about seeking help account for help negation? Journal of Clinical Psychology, 61(12), 1525–39. Wilson, C. J., Rickwood, D. J., Bushnell, J. A., Caputi, P., & Thomas, S. J. (2011). The effects of need for autonomy and preference for seeking help from informal sources on emerging adults’ intentions to access mental health services for common mental disorders and suicidal thoughts. Advances in Mental Health, 10(1), 29–38. https://doi.org/10.1111/ j.1751-7893.2010.00239.x Woodward, L. J., & Fergusson, D. M. (2001). Life course outcomes of young people with anxiety disorders in adolescence. Journal of the American Academy of Child & Adolescent Psychiatry, 40(9), 1086–93.

Wang, K., Varma, D. S., & Prosperi, M. (2018). A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. Journal of Psychiatric Research, 107, 73–78. Werner-Seidler, A., Perry, Y., Calear, A. L., Newby, J. M., & Christensen, H. (2017). School-based depression and anxiety prevention programs for young people: A systematic review and metaanalysis. Clinical Psychology Review, 51, 30–47. https://doi.org/10.1016/J. CPR.2016.10.005

39


A MEASURABLE IMPACT: HELPING YOUNG PEOPLE TO BE AND STAY WELL

Š ReachOut Australia 2020 abn 27 075 428 787 dgr 442 641 Level 2, Building B 35 Saunders Street, Pyrmont NSW 2009 Call us on +61 2 8029 7777 or email info@reachout.com

ReachOut is the most-accessed online mental health service for young people and their parents in Australia. Our trusted self-help information, peer-support program and referral tools save lives by helping young people be well and stay well. The information we offer parents makes it easier for them to help their teenagers, too.


Turn static files into dynamic content formats.

Create a flipbook
Issuu converts static files into: digital portfolios, online yearbooks, online catalogs, digital photo albums and more. Sign up and create your flipbook.