#MakeItHomeNZ
Scaling Success MEASURING THE IMPACT OF TRAMPING SAFETY VIDEOS
PREPARED BY THE NZ MOUNTAIN SAFETY COUNCIL
“I found the video the single most valuable resource in gaining information about the walk and what to do to be prepared. Videos such as this should be standard for any major or backcountry tracks so trampers can be well informed about what they can expect.” – RESEARCH PARTICIPANT
2
Scaling Success | Tramping Video Series
Contents Introduction
5
Insights 6-9 Partnerships 10-13 Promotion and Engagement
14-15
Measuring Impact
18-33
Where to from here?
35
Thank you and acknowledgments
35
New Zealand Mountain Safety Council Level 1 Harbour City Centre, 29 Brandon Street | Wellington 6011 info@mountainsafety.org.nz mountainsafety.org.nz All images copyright Mountain Safety Council 2018 Cover photo: LoĂŻc Lassueur, Travers Saddle, Nelson Lakes National Park Inside cover photo: Bevan Smith, Kepler Track, Fiordland National Park
NZ MOUNTAIN SAFETY COUNCIL | 2019
3
4
Scaling Success | Tramping Video Series
Above: LoĂŻc Lassueur, Milford Track, Fiordland National Park
Introducing our tramping video project During 2017/2018 the NZ Mountain Safety Council (MSC) developed a series of track-specific tramping videos. The purpose of the video series was to provide a specific planning resource that could be used in the ‘active tripplanning’ phase of trip preparations. The videos aimed to provide highly engaging content utilising techniques such as animations, 3D mapping and drone footage. Each video focuses on content relevant to that track. They cover topics such as the terrain, key weather factors to consider, walking times, key decision making points, common risks and hazards as well as any specific skills required to safely navigate a particular section/s of the track. Ultimately the videos aim to help trampers make it home safely.
You can use the camera on your phone to watch the videos online. Open camera and hold over this QR code, tap the link. Or visit www.bitly.com/AlpTramp
This publication provides a detailed account of the video-based prevention initiative. It has been developed specifically to showcase the key elements which were critical to the success of this video series. Most notably, the impact research which was conducted over the summer of 2018/2019 provides clear evidence that the video series has had a significant positive impact on user behaviour. While the impact research may be the key success measure of this prevention initiative there are many factors that contributed to this outcome. The success of this initiative provides a blueprint for future prevention tactics.
NZ MOUNTAIN SAFETY COUNCIL | 2019
5
Insights
Above: LoĂŻc Lassueur Aoraki/Mt Cook National Park
6
Scaling Success | Tramping Video Series
Evidence-based decision making During 2017 the MSC team was working on an insights project focussed on the activity of tramping. Combining both incident (safety) data and participation data, the project used 1 eight different data sources . The results produced the most comprehensive dataset of tramping safety incidents available in New Zealand.
As MSC explored these track-specific incidents in greater detail several factors become apparent:
The project was guided by an insights roadmap; a list of key questions that MSC sought answers to. A simple example of these questions was ‘How many trampers are involved in a Search and Rescue (SAR) each day/month/year?’. The project had over 150 research questions with some requiring highly complex coding, analysis and statistical modelling.
2. I n almost all situations safety incidents were concentrated to specific sections of a track. For example, on the Milford Track the vast majority of Search and Rescues occur between Pompolona Lodge and Dumpling Hut. This is the alpine section over MacKinnon Pass. While this may not be a surprise given the nature of the terrain, typical weather and arduous nature of the ascent/descent, it does highlight the fact that it is this section of the multi-day walk where more people are experiencing safety incidents.
In total, the findings numbered into the hundreds of Excel sheets, each focussed on answering one of the insight roadmap questions. This project eventually resulted in the publication of A Walk in the Park? which is available on the MSC website.
A Walk in the Park? contains approximately 10% of the wider project findings. The remaining findings are used internally by MSC for its prevention work, and are made available to partner organisations to support their evidence-based decision making. Through this insights project it became clear there were a number of key tramping tracks in which higher numbers of safety incidents were occurring.
You can use the camera on your phone to read A Walk in the Park?’ online. Open camera and hold over this QR code, tap the link. Or visit www.bitly.com/awitp
1. T he majority of safety incidents were believed to be entirely preventable (we had specifically removed the small number of medical events). E.g. Heart attacks
3. One reasonably common factor amongst the top tracks (especially in terms of Search and Rescue events) was the theme around the ‘alpine section’ of the walk. For example, Poukirikiri/Travers Saddle includes reasonably straightforward valley travel either side of an alpine saddle (pass) crossing. It’s on this section that most safety incidents occur. By the time MSC’s business planning was in full swing around May 2017 the concept of an ‘Alpine Tramping Video Series’ had begun to take shape. It was clear to MSC that there were a number of tracks which could feature in an initial series, and that these videos could utilise both our newly developed track level insights and local expert knowledge to craft highly specific content. After securing funding via the MSC’s major funder, the New Zealand Lottery Grants Board, work began on making the series a reality.
1. A CC Injury Claims, NZ Police Category 1 SAR, Rescue Co-ordination Centre Category 2 SAR, Coronial Services Unit Fatalities, Sport NZ Active NZ Participation Survey, Department of Conservation participation; track counters and booking data, Auckland Council participation and visitation data and MBIE International Visitor Survey.
NZ MOUNTAIN SAFETY COUNCIL | 2019
7
Why these tracks? A number of factors were used to finalise the 12 tracks that feature in the series, the primary one being to focus on tracks that have the highest number of safety incidents. The list on the following page features each of the 12 tracks and their ranking on a national list of tracks by tramping Search and Rescue count over a seven-year period between 2010 to 2017. In addition to the Search and Rescue numbers we also considered tramping fatalities and, where known, the location of moderate to severe injuries. While some of the tracks featured in the series are lower down the Search and Rescue list they were included for other reasons. Two examples of this were Waiau Pass and Gertrude Saddle. Waiau Pass serves as the first true test of a South Island alpine pass for walkers completing the Te Araroa Trail. While there’s no evidence that these walkers are at a higher risk than others, or that they account for more safety incidents, the feeling was that with the increasing popularity of the Te Araroa Trail it would be wise to have this resource available for anyone who intends giving this section a go in the future. Gertrude Saddle Route has experienced a significant increase in popularity in recent years, partially attributed to user-generated visitor (domestic and international) promotion via social media platforms like Instagram. Its ranking on the list changes considerably when we added in the two fatalities which occurred during 2016 and 2017 (a further fatality occurred in April 2018 after we had already filmed the video but before the video was released). RANK OF TRACKS BASED ON TRAMPING SEARCH AND RESCUE EVENTS (2010-17) Tongariro Alpine Crossing
1st
Milford Track
2nd
Mt Taranaki Summit Route
3rd
Ben Lomond Track
5th equal
Routeburn Track
5th equal
Mueller Hut Route/ Sealy Tarns
8th
Poukirikiri Travers Sabine Circuit
9th
Robert Ridge Route/Angelus Hut
10th
Kepler Track 19th Gertrude Saddle Route
20th
Waiau Pass 22nd Cascade Saddle Route
Left: Bevan Smith, Routeburn Track, Fiordland National Park
8
Scaling Success | Tramping Video Series
Outside the top 30 for SAR but a known fatality hotspot
TONGARIRO ALPINE CROSSING
MT TARANAKI SUMMIT ROUTE
POUKIRIKIRI/TRAVERS SADDLE
ROBERT RIDGE ROUTE Used for research
WAIAU PASS
MUELLER HUT ROUTE Used for research
ROUTEBURN TRACK Used for research
CASCADE SADDLE TRACK
BEN LOMOND TRACK
GERTRUDE SADDLE
MILFORD TRACK
KEPLER TRACK Below: Bevan Smith, Cascade Saddle Route, Mt. Aspiring National Park
NZ MOUNTAIN SAFETY COUNCIL | 2019
9
Partnerships
10
Scaling Success | Tramping Video Series
Above: Bevan Smith, Ben Lomond, Queenstown-Lakes District
“Your videos are fantastic and provide essential and easy to digest information for tourists and less experienced trampers. Please continue making them and hopefully more people will be better prepared for the wilderness!” – RESEARCH PARTICIPANT
We didn’t want to do this alone Involving partners was a key contributing factor to the success of this series. As each of the 12 tracks were managed by the Department of Conservation (DOC) it was essential that they were onboard early. DOC was a major supporter throughout the entire project. Critically, their regional operations staff were heavily involved as ‘local experts’ helping to shape and contextualise the video content. In addition, DOC waived all concession fees, approved helicopter access and allowed for drone filming in otherwise highly restricted areas. For two of the locations, Tongariro Alpine Crossing and Mt Taranaki Summit Route, local iwi were also significant contributors and great enablers of these videos. While the videos focussed on safety, it was important to also include elements of cultural expectations and we were pleased to be able to focus on these points, most notably around protocols associated with not standing on the summit and the importance of specific features like Emerald Lakes. Six other Mountain Safety Council member organisations also played critical roles. MetService, NZ Police, NZ Alpine Club, NZ Outdoor Instructors Association, NZ Mountain Guides Association and LandSAR all contributed time and expertise through the content development phases and post-production video reviews. Like DOC and iwi, their networks of members were essential in engaging ‘local experts’. Further partners were involved at other stages throughout the project, notably Litmus Marketing for their masterful data analytics and statistical work, Research NZ for their ability to develop and deliver a very successful research project, and Quite Nice Films, who were with us every step of the journey through rain, hail and sunshine in the hills and countless hours of editing work. The success of the series is in large part due to the magic they delivered.
Top to bottom: Photos by Bevan Smith - Kepler Track, Milford Track, Cascade Saddle, Milford Track.
NZ MOUNTAIN SAFETY COUNCIL | 2019
11
Content development There were two primary elements which ensured the successful development of each video’s unique content: track level insights (data) and local expert knowledge. The importance of both elements working symbiotically cannot be overstated. At every key stage of the content development local experts (and partners) were involved. Content development for the videos followed a five-step process. Pre-script planning > script writing > storyboards > filming > post-production.
Above: Behind the scenes and screenshots from the Tramping Video Series
VIDEO PRODUCTION
LOCAL EXPERT KNOWLEDGE
12
Scaling Success | Tramping Video Series
TRACK INSIGHTS (DATA)
“As a mountain tramping lover, I am so happy to see these series of videos!!! That’s what I really would love to check before I go tramping. You are really doing a great job! Please make more videos, for example, Dusky Track, Rees- Dart Track, Tongariro.” – RESEARCH PARTICIPANT PRE-SCRIPT PLANNING This step provided the initial opportunity for each of the local experts to contribute their knowledge. This knowledge was considered alongside the track-level insights MSC had developed. A detailed document covering the key points for each video was developed through these conversations and this was used to draft the scripts and storyboards. SCRIPTING & STORYBOARDS The pre-script documents were then used to create a draft script, this was the content that would feature in the voice-over dialogue. With a draft script in place an initial storyboard could be developed. These storyboards allowed those reviewing the content to see the visual style and features such as animations and maps alongside the content. The storyboards were a pivotal tool that needed to be in place before filming began. FILMING From mid December 2017 to late March 2018, MSC undertook seven dedicated filming trips covering all 12 video locations. Each filming trip had three or four people, with each person playing a role of some description. The team had to carry a lot of gear between them and other than a couple of the day-trip videos, usually had between 20-30kg gear each. Most of this weight was attributed to the vast quantity of camera equipment and accessories. With no power supply for up to four days of filming a lot of batteries were required to keep the two cameras and drone operational. POSTPRODUCTION With so many videos involved, postproduction was a complex process. Between March and September 2018 editing of the 12 videos took place. The local experts and partners who had contributed to the earlier content development were once again involved in reviewing and providing feedback. 3D mapping (where available) was added into the videos after filming as we felt this provided a more effective way of displaying information than just using a 2D topographical map. Animations were added after the bulk of the footage had been selected and placed and this became a key feature of the series as it supported the engaging content and helped to convey key information.
NZ MOUNTAIN SAFETY COUNCIL | 2019
13
Promotion and Engagement
14
Scaling Success | Tramping Video Series
Above: Bevan Smith, Cascade Saddle Route, Mt Aspiring National Park
99% Positive reactions on Youtube
Reaching the participants Over several weeks during October 2018, 9 of the 12 videos were released. Each video was initially promoted via the MSC Facebook Page and supported by many of the partners sharing the content to their own pages and audiences. Additionally, the videos were uploaded to the MSC YouTube channel which gave partners the ability to embed the videos into their own websites.
REACH METRICS OCTOBER 2018 TO 31 AUGUST 2019
DOC was a key channel for reaching people in the active trip planning phase and each of the videos was embedded on their relevant website pages.
FACEBOOK VIDEO VIEWS
712,576
TOTAL FACEBOOK REACH
2,300+ FACEBOOK SHARES
Three videos; Robert Ridge Route, Mueller Hut Route and Routeburn Track were withheld from public release so MSC and Research NZ could conduct impact research.
6,000+ FACEBOOK LIKES
The following metrics are counted from when each of the videos were released, therefore they represent a slightly different length of time for each video. They also include the three videos that weren’t released until after the impact research (April 2019) so the metrics underrepresent a true reflection of the series popularity.
YOUTUBE METRICS PER VIDEO
252,314
3,500+ FACEBOOK COMMENTS 198,697 YOUTUBE VIEWS
TOTAL VIEWS
TOTAL EXTERNAL VIEWS
AVERAGE VIEW LENGTH (OF TOTAL)
% THAT WATCHED UNTIL THE END
Tongariro Alpine Crossing
44,070
21,061
57.3%
27.5%
Mt Taranaki Summit Route
21,346
8,989
58.4%
26.0%
Robert Ridge Track*
2,646
978
59.6%
35.3%
Poukirikiri/Travers Saddle
6,268
2,190
60.1%
31.2%
Waiau Pass
5,348
863
56.0%
33.0%
Mueller Hut Route*
4,204
1,700
60.8%
26.9%
Cascade Saddle
12,300
3,223
54.9%
24.9%
Gertrude Saddle Route
16,796
7,984
55.3%
8.6%
Routeburn Track*
9,323
5,048
62.4%
26.8%
Ben Lomond
17,974
9,524
55.1%
24.9%
Milford Track
35,491
17,492
59.4%
27.6%
Kepler Track
28,226
15,798
61.4%
22.0%
*Released April 2019 N.B. External views include: DOC Website, other partner websites, media/PR.
NZ MOUNTAIN SAFETY COUNCIL | 2019
15
What did they have to say? – QUOTES FROM RESEARCH PARTICIPANTS
‘ What a great resource! Nice job DOC. If I was attempting a more challenging walk here again I would search for a similar video.’
‘ Impressed with the quality and clarity of the information - hope there are ways we can encourage more people to watch them, as they seem effective at [conveying] a lot of important messages in an accessible way.’
‘ [I] Shared the video on a blog so more people can benefit from it, and to support more videos like this being produced. Can’t beat seeing real footage of the conditions you might come across, e.g. climbing rocks, using at least three points of contact.’
‘ The content and production quality are outstanding. Please come over to Australia to train Australian National Parks on how to do [this] well.’
‘ Definitely a good idea! It’s hard to make out some of the more finer details of the walk when just looking at the standard DOC PDF pamphlets of the walks.’
‘ Great idea and very informative - should be compulsory viewing for anyone attempting these tracks.’
16
Scaling Success | Tramping Video Series
Above: Loïc Lassueur, Queenstown-Lakes District
‘ I think they’re fantastic, generally. I’m not the target audience (I don’t think - lots of mountaineering experience, sub-alpine and alpine tramping for years, now starting to do overnighters with my kids...) but they WERE really good for getting the kids [ready].’
‘ I think they are a valuable resource, I am an experienced hiker and trailer runner and have lived in NZ for a while but still found them very useful and helpful for planning my hike. Especially since I was traveling with someone less fit than I.’ ‘ I will buy wet weather rain pants. Was going to try and get by with shorts.’
‘ I really enjoy how informative and helpful the information in the videos is, especially for more dangerous walks with potential safety hazards. Thanks DOC!’
‘In my opinion, these videos are very helpful for a person who is new to the region or even to the activity. It informs them of the important aspects of the track and help them make decisions on what to carry with them and what to expect on the track.’
‘It’s extremely helpful to give an idea about what to expect when you are not in New Zealand country, to make preparation efforts appropriate.’
NZ MOUNTAIN SAFETY COUNCIL | 2019
17
Measuring Impact
18
Scaling Success | Tramping Video Series
Above: LoĂŻc Lassueur, Lake Angelus, Nelson Lakes National Park
“In my 35 plus years’ experience, I have never ever seen behavioural results as strong as these that can be attributed to one single initiative. There is no doubt, the videos have had a significant positive impact on trampers’ safety-related behaviour.” – EMANUEL KALAFATELIS, DIRECTOR RESEARCH NZ AND FELLOW OF THE RESEARCH ASSOCIATION OF NEW ZEALAND (RANZ)
Research objectives and methodology We knew this project was unlike anything we had attempted before, and that this would require some form of dedicated research to determine the video’s impacts. To answer ‘How will we know the videos have been a success?’ MSC turned to Research NZ for their expertise. Through mid-2018 the details of this project were developed and the decision to delay the public release of three videos was made. This would allow Research NZ to conduct a comprehensive, controlled, research methodology without participants seeing the video in a public environment.
Research participants were recruited via a co-branded email invitation. The email encouraged lead bookers to register for the research and then to share the research registration with other members of their walking party (the DOC booking system only captures lead booker contact details, so we had to request lead bookers to share with others). After participants had registered for the research, they were randomly divided into either a ‘Test’ or ‘Control’ Group. The Test Group were exposed to the video prior to their walk, whereas the Control Group were not. Both groups saw the video after their walk. Both groups completed a survey seven days prior to their walk and seven days after their walk ended.
The research objectives were aligned to the purpose of the videos. Specifically, the research set out to measure:
Participation in the research project was incentivised. Six prize packs were available through a random draw, available to all participants who completed the full research programme. Each of the winners were randomly selected and received their prizes in May 2019.
»» Were the videos increasing awareness and knowledge? »» Were the videos increasing trip preparedness? »» Were the videos enabling more effective decision making? To achieve this the research utilised a randomised control experimental survey design. This methodology required that no participants had seen the videos outside of the research. By selecting the Routeburn Track, Mueller Hut Route and Robert Ridge Route Research NZ was able to access booking contact details for the relevant track huts, over the 2018/19 summer booking season. DOC supplied those contact details as a key project partner, through a secure and confidential portal, only accessible by the Research NZ team.
An additional 278 trampers completed the pre-walk survey but did not finish the post-walk survey. Their results have not been included in the findings. Total N=924
Test Group N=466
Control Group N=458
RESEARCH PROCESS REGISTRATION
CONTACT DETAILS VIA DOC
SURVEY 1
SURVEY 2
CONTROL GROUP
CONTROL GROUP
ALL POTENTIAL RESPONDENTS
TRAMP TEST GROUP
TEST GROUP
* Watched the video NZ MOUNTAIN SAFETY COUNCIL | 2019
19
“I understand more about where to make decisions about the walk. Will listen carefully to the info available at the St Arnaud DOC office. Will be more prepared to turn back if weather is not good.” – RESEARCH PARTICIPANT
Research findings context 47% 41%
6%
5% BEGINNER
INTERMEDIATE ADVANCED
Walkers could self-select their tramping experience and ability. They were grouped into four categories. There was very little difference across the three tracks or between the Test and Control Groups.
EXPERT
Most people were completing the walk for the first time, there was very little difference across the three tracks or between the Test and Control Groups.
84% Were doing their walk for the first time
New Zealander
There was a reasonably even spread between International Visitors and New Zealanders. This was a self-selection question based on identifying your home country.
46% Home country of Test and Control Group Respondents
54%
International Visitor
Total Respondents N=924
20
Scaling Success | Tramping Video Series
“You’re doing a great job. The video helped us a lot to re-plan everything. Keep doing what you’re doing and I think people will take it more seriously and will be much more safety conscious.” – RESEARCH PARTICIPANT
Almost finished
42% Middle stage
26% 25% Early stage
Finished
6%
95% Watched the video right to the end
When asked what stage of planning their trip they were in, most responded they were somewhere between the middle stage and finished, with the highest category somewhere in-between those two stages. This question was answered approximately seven days prior to their trip starting and reflects the fact that most people were still in a period of ‘active preparation’.
When the Test Group were shown the video as part of the research almost all people watched the whole video to the end. Only 5% skipped some parts of the video. The ability to ‘fast forward’ parts of the video was consciously left enabled so we could measure how many people watched the whole thing. A very small number of those who skipped sections was due to internet issues.
Neutral Not enjoyable
4% Overall, 95% of all trampers stated they found their walk enjoyable. Only 1% didn’t enjoy their trip.
67%
25% 41%
Response to how they enjoyed their walk
95% Enjoyed their walk
Total Respondents N=924
Test Group N=466 NZ MOUNTAIN SAFETY COUNCIL | 2019
21
“I will make sure to write down the information about the important decision points and the left turn in the saddle that can be easy to miss.” – RESEARCH PARTICIPANT
Did they learn anything? Having watched the video prior to departing, some participants felt more, or less, prepared.
Less prepared
6%
About the same
54%
40%
Felt more prepared
95%
Learned something new
SPECIFICALLY....
15%
18%
Increased knowledge of hazards along the track
Increased overall understanding of the track
85%
82% Yes
Test Group N=466
22
22%
Scaling Success | Tramping Video Series
Yes
Places to make key decisions
78% Yes
“I was a bit surprised at the average summer temperatures quoted and might take an extra layer of clothing.” – RESEARCH PARTICIPANT
Would it change their behaviour?
76%
85%
of Test Group respondents said they would make changes to their plans as a result of the video
of International Visitor Test Group respondents said they would make changes to their plans as a result of the video
What changes would the Test Group respondents make in their planning after watching the video?
41%
Check the weather forecast ahead of walking
34%
Speak to a DOC staff member
28%
Research where key decisions may be made
26%
Plan the right food and water Research hazards
24%
Visit MSC website or social media
24%
Check have right equipment
23%
Research alternative routes for if weather turns bad
23% 20%
Check safety equipment
15%
Visit DOC website or social media page Leave outdoor intentions Change departure time to earlier Talk to others with experience
14% 13% 12%
Make none of the above changes
Test Group N=466
24%
International Visitor Test Group N=256
NZ MOUNTAIN SAFETY COUNCIL | 2019
23
“Will watch this video again with the whole group and will check in at the DOC centre in Mt Cook before starting walk.” – RESEARCH PARTICIPANT
Did people talk about the video?
89%
Of Test Group talked about the advice in the video
SPECIFICALLY....
59%
Talked about the advice before the walk
Test Group N=369 (they had to be part of a walking group)
24
Scaling Success | Tramping Video Series
6%
Talked about the advice on the walk
24%
Talked about the advice before and during the walk
“I’ll take poles for walking and definitely wear hiking boots, I was potentially going to wear trail trainers.’’ – RESEARCH PARTICIPANT
Net Promoter Score Net Promoter Score is a method used to gauge the loyalty of a customer to a brand or product. In this case it represents the research participants likelihood to recommend the videos to another person.
43
The Test Group respondents gave the videos a Net Promoter Score of 43. This is an incredibly high score.
Net Promoter Score (NPS)
48% gave the videos a rating of 10. The NPS is calculated by taking the Promoters (57%) and subtracting the Detractors (14%) to leave a score of 43. 48%
17% 11% 6% 2% 0
DON’T KNOW
2%
1%
1%
1%
1
2
3
4
NOT AT ALL LIKELY
DETRACTORS 14%
5
NEUTRAL
9%
3% 6
7
8 PASSIVE 28%
9
10
EXTREMELY LIKELY
PROMOTERS 57%
Test Group N=466
NZ MOUNTAIN SAFETY COUNCIL | 2019
25
“They’re a fantastic resource - please continue to produce them!?” – RESEARCH PARTICIPANT
How did they make them feel?
Test Group N=466
26
Scaling Success | Tramping Video Series
74%
MORE INFORMED
74%
EXCITED
28%
CONFIDENT
23%
NERVOUS
15%
INTRIGUED
3%
FEARFUL
1%
CONFUSED
3%
NONE OF THESE
0%
DISINTERESTED
84%
Of Test Group respondents rated the videos as interesting and engaging
Above: LoĂŻc Lassueur, Lake Constance, Nelson Lakes National Park
NZ MOUNTAIN SAFETY COUNCIL | 2019
27
“Brilliant idea to have a series of videos - they distil critical information into a few minutes. I’d spent time researching the Mueller Hut Route online and while all the necessary information is available, video format is much easier to digest.” – RESEARCH PARTICIPANT
Comparing the groups after their walks Comparisons between the Test Group, who saw the video prior to their trip, and the Control Group, who never saw the video before their trip, reveals further success.
After walking the tracks, both groups were asked if the video was valuable for the planning process
54%
87%
57%
32% 30%
Of the Control Group rated the video as valuable for planning future walks
10% 9% 1%
0%
3%
1%
2
1
NOT VALUABLE AT ALL
3
4
5
VERY VALUABLE
After walking the tracks, both groups were asked if they would look for similar videos for future walks
55% 55%
78%
Of the Control Group would be likely to search for similar videos when planning future walks
21% 23% 15% 4%
Test Group N=466
4%
1
NOT AT ALL LIKELY
Control Group N=458
28
Scaling Success | Tramping Video Series
5% 2
10%
6% 3
4
5
VERY LIKELY
“I think they are very helpful and I would definitely watch one again prior to a walk. More informative than YouTube videos as the terrain/course shots are clear and easy to understand.” – RESEARCH PARTICIPANT
How was their experience? Comparing the two groups based on their expectations of how challenging each walk would be shows the difference watching the video made. Overall, the Test Group found their trips less challenging than they expected and the Control Group found their trips more challenging than expected. Remember, the only difference between these two groups was the exposure to the video prior to their walk (Test Group).
TEST GROUP
CONTROL GOUP
Less challenging than I expected
22%
16%
About the level I expected
65%
61%
More challenging than I expected
13%
23%
Test Group N=466
Control Group N=458
NZ MOUNTAIN SAFETY COUNCIL | 2019
29
“Very valuable video! It made a big difference to my knowledge and confidence while tramping. I had previously walked down Robert Ridge about 30 years ago, so I thought I knew enough, but the video provided a timely reminder for me and allowed me to inform my group (who were first timers).” – RESEARCH PARTICIPANT
What did they experience? Overall, the most common safety-related involvement was ‘getting cold/wet/hungry and fatigued’; experienced by 12% of the Control Group and 7% of the Test Group. Across the ten other factors there was no difference in the two groups. Only 3% of both groups suffered a physical injury, that’s about 27 of the 924 people. Both Test and Control Groups were asked if they experienced any of the following on their walk:
7% Got cold/wet/hungry and fatigued
12% 6%
Difficulty in attempting to cross a section of the track
9% 6%
Needed to turn around/ change your plans
6% 4%
Underprepared for the conditions
Suffered a phyisical injury (sprain/fracture)
5% 3% 3%
Overdue and/or out in the dark when you didn’t expect to
74%
Of the Test Group did not experience any safetyrelated incidents
2% 1%
Got left behind/separated from your group
2% 2%
Experience an equipment failure
1% 2%
Suffered a medical event (allergy/sting/illness)
1% 1%
No mobile phone coverage and needed to make an emergency call
0% 0%
Lost and unsure of where you are and what to do
0% 0%
Test Group N=466
Control Group N=458
30
Scaling Success | Tramping Video Series
67%
Of the Control Group did not experience any safetyrelated incidents
“These videos are great and will save lives. Most importantly, show people the real range of conditions, including things like, winter versus summer, the length of time a hiker can expect to be climbing a very steep slope, as this will motivate people to [be more prepared].”
What would they do differently?
– RESEARCH PARTICIPANT
Even after their walk, keeping in mind the low involvement in safety-related issues and very high enjoyment factor, more than half (51%) of the Control Group respondents stated they would have made changes to their planning had they seen the video pre-walk.
Research where key decisions may be made
20%
Research hazards
20% 16%
Research alternative routes for if weather turns bad
12%
Visit MSC website or social media Change departure time to earlier
7%
Speak to a DOC staff member
7%
Check safety equipment
7% 6%
Check the weather forecast ahead of walking Leave outdoor intentions
5%
Plan the right food and water
5%
Visit DOC website or social media page Would have done something else Make none of the above changes
51%
Of Control Group respondents would have made changes
3% 2% 49%
Control Group N=458
NZ MOUNTAIN SAFETY COUNCIL | 2019
31
95%
Of Test Group respondents found the changes they made because of the videos were useful
32
Scaling Success | Tramping Video Series
Above: LoĂŻc Lassueur, Travers Saddle, Nelson Lakes National Park
76%
90%
Said they would make changes to their plans
A measurable change in behaviour was recorded across 8 different behavioural factors Despite the clear success of the videos, the most significant outcome was confirmation that the videos were creating measurable behaviour change. Using a special analytical test called ‘pair-wise comparison’ we can examine the results of each and every respondent, rather than just looking at them as part of a big group. Specifically, this allows us to measure what the Test Group said they would do after watching the video, and then compare that to what they actually did do. As previously showcased earlier in the findings, 76% of the Test Group said they’d make changes to their plans after watching the video. An incredible 90% of them actually followed through on those changes.
Of them actually made these changes
Said they would...
of them actually did do it
41%
Check the weather
92%
34%
Speak to DOC Staff
84%
28%
Research key decision making points
67%
26%
Plan right food and water needed
82%
24%
Research hazards along the track
65%
24%
Visit MSC Website or social media
42%
23%
Check equipment, clothing, footwear
88%
23%
Research alternative routes if weather turned poor
43%
Test Group N=466
NZ MOUNTAIN SAFETY COUNCIL | 2019
33
Where to from here?
34
Scaling Success | Tramping Video Series
Above: Bevan Smith, Routeburn Track, Fiordland National Park
We will continue to develop video resources with our partners The evidence is clear: these videos are working. Based on the success of series one, MSC has committed to producing more prevention focussed video content of this nature. ACKNOWLEDGEMENTS None of this would have been possible without the support of many key partners, thank you. Direct project funder: Lottery Grants Board Insights funders: Lottery Grants Board Sport NZ Project Partners: Department of Conservation MetService New Zealand NZ Police NZ Alpine Club NZ Outdoor Instructors Association NZ Mountain Guides Association LandSAR Te Korowai o Ngāruahine Trust Ngāti Hikairo Ki Tongariro Filming & Production Quite Nice Films Impact Research Research NZ Data Analysis & Statistics Litmus Marketing
Ngāti Hikairo Ki Tongariro
NZ MOUNTAIN SAFETY COUNCIL | 2019
35
#MakeItHomeNZ
mountainsafety.org.nz
36
Scaling Success | Tramping Video Series