STF 2015 Final Evaluation

2015 Shaping the Future Evaluation

By: Lisa Tink & Bethan Kingsley

The goal of the 6th annual Shaping the Future conference was to INSPIRE, CONNECT and CELEBRATE healthy school communities by bringing together leaders in health, education and active living for two days of learning, networking and discussion.

To better measure the outputs, outtakes and immediate outcomes of the 2015 conference, a 3-­‐phase evaluation process has been implemented. The three phases of the evaluation included session evaluations, a post-­‐conference evaluation and an analysis of the information posted on the conference app and Twitter.

Both quantitative and qualitative data were collected through a number of multiple choice, Likert items and open-­‐ended questions. Of the 497 delegates, a total of 210 responded to the post-­‐conference survey, 360 concurrent session evaluations were completed (a complete analysis of the concurrent sessions can be found in report titled 2015 Shaping the Future Session Evaluation) and 1265 conference app and Twitter posts were analyzed. The analyzed data is presented in 5 sections:

1. Delegate Information

2. Quality and Relevance

3. Immediate Outcomes

4. Recommendations

5. Conclusion

2

DELEGATE INFORMATION

A total of 497 delegates attended the 2015 conference, which offered 4 full-­‐day preconference workshops, 41 concurrent sessions, 4 interactive workshops and 4 keynote speakers.

Delegate Representation

In an attempt to determine which sectors were represented, delegates were asked to identify which sector they best represent in both the session evaluations (completed using the conference App) and the post-­‐conference survey.

While there was representation from Education, Health, Active Living, Recreation, Post-­‐ Secondary and Research, the majority of the delegates reported that they worked within the Education Sector.

Post-­‐ Other Active Recreation Research Post-­‐

Research Secondary 5% Living Secondary 2% 2% 2%

6% 3% 3%

Recreation 2%

Health

Health 12% 8%

Education Education 77%

78%

Delegate Representation-­‐ Conference App Delegate Representation-­‐ Post-­‐Conference Survey

Further analysis revealed that 94% of those who work within the Education sector work directly in schools (e.g. teachers, educational assistants, principals, vice-­‐principals).

Delegate Motivations

In addition to knowing the sector representation, we also wanted to know how delegates found out about Shaping the Future, if they attended in the past and what their reasons were for attending.

The questions on the next page were included in the Post-­‐Conference Survey and the information can be used when developing a marketing and promotion plan for the 2016 conference.

3

Q: How did you find out about STF?

Colleague Alberta Healthy School Community Wellness Fund Ever Active Schools Staff Ever Active Schools Newsletter Ever Active Schools Website 0

Q: Have you attended STF in the past?

No 60%

10

20

30

40

50

60

70

80

Yes 40%

Q: What was your primary reason for attending Shaping the Future?

Q: To what extent was this met?

Professional/Personal Development

Don't know

13% Mostly

Networking 4%

Fully Present/Share Information

83%

Explore New Avenues for Collaboartion and Partnerships

0 20 40 60 80 100 120 140 160

4

QUALITY AND RELEVANCE

Quality

In an attempt to gauge the quality of the conference, the Post-­‐conference survey asked participants about the suitability of information presented at the conference and the extent to which speakers were knowledgeable and informative.

As can be seen in the charts below the majority of delegates “Agreed” or “Strongly Agreed” that the information presented was suited to the conference and that the speakers were knowledgeable and informative. This is consistent with the data collected in the session evaluations, which can be viewed the 2015 Shaping the Future Session Evaluation report.

Q: The information presented was suited to the conference.

Concurrent Session Strongly Disagree

Average= 4.32/5

Disagree Interactive Workshop

Undecided

Average= 4.06/5

Agree Keynotes

Strongly Agree

Average= 4.57/5

0%

20%

40%

60%

80%

100%

Q: Overall the speakers were knowledgeable and informative.

Concurrent Session Strongly Disagree

Average= 4.32/5

Disagree Interactive Workshop

Undecided Agree

Average= 4.18/5

Strongly Agree

Keynotes Average= 4.61/5

0%

20%

40%

60%

80%

100%

5

Other information relating to quality of the conference program:

4.6 -­‐ 4.9 The average sa[sfac[on for the keynote speakers fell between 4.6 and 4.9 out of 5.0

All Sessions

79% 79% of delegates indicated that the level of interac[on in the Interac[ve Workshops met their expecta[ons.

All sessions were referenced when delegates were asked which sessions they were "most interested in"

Relevance

In an attempt to determine if the conference was relevant for individuals from the various sectors, delegates were asked if the information presented in the concurrent sessions, interactive workshops and keynote speakers was relevant to their role. In an attempt to see which sectors the information was most relevant for, a filter was applied allowing us to separate the responses. The numbers shown in the diagram below provide an average relevance score for each sector out of 5.0.

Education

Health

• Concurrent Sessions 4.15 • Interactive Worshops 3.96 • Keynote Speakers 4.45

• Concurrent Sessions 4.27 • Interactive Worshops 3.14 • Keynote Speakers 4.41

Active Living • Concurrent Sessions 4.33 • Interactive Worshops 4.50 • Keynote Speakers 4.50

Recreation • Concurrent Sessions 4.0 • Interactive Worshops 4.0 • Keynote Speakers 4.0

Post-­‐ Secondary • Concurrent Sessions 4.40 • Interactive Worshops 4.67 • Keynote Speakers 4.5

Research • Concurrent Sessions 3.25 • Interactive Worshops 2.67 • Keynote Speakers 3.75

As shown in the diagram, Research is the sector that had the lowest relevance scores.

This could be due to the fact that the majority of ‘Researchers’ who attend are there as speakers and don’t find that the other information presented contributes to their work.

Additionally, the Interactive Workshops were scored slightly lower than the concurrent sessions and the keynote speakers for some of the sectors. This could be due to the fact that some people felt the level of “interaction” was relatively low for an “Interactive Workshop.”

6

Qualitative Comments

In addition to quantitative data, the quality, relevance and variety of the content were demonstrated through qualitative comments. Some of these comments are provided below:

“The sessions I attended were awesome. Relevant and meaningful to the work I do.”

“The choice of sessions was phenomenal, we never felt like nothing applied to us.”

“My impression was that the sessions offered this year were of a very high calibre compared to my experience in previous years…”

“I liked the variety of the workshops that you provided. I was torn about which workshop to attend. They all looked great.”

“I love how inspirational [the Keynote Speakers) all were. They were engaging and well worth the event.”

Quality and Relevance Summary

Based on the information collected related to the quality and relevance of the program it can be stated that the majority of delegates were satisfied with the information presented, the knowledge of the speakers and the variety in the presentations.

Recommendations about how to improve the quality and relevance for some delegates were shared and can be found in the Recommendations section.

7

IMMEDIATE OUTCOMES

In an attempt to determine the immediate outcomes the qualitative data collected through the session evaluations, the app and Twitter posts and the post-­‐conference survey were analyzed using a thematic approach. The themes and their associated sub-­‐ themes, based on the delegates perspectives, are provided below:

1. Learning a. New Knowledge and Ideas b. Implementation Strategies c. Practical Resources d. Research/ Evidence-­‐based Practice 2. Connections a. Networking b. Strategies for Partnerships and Collaboration 3. Inspiration

Using the information collected, an outcome model (presented below) has been developed. To better understand the outcomes, descriptions of each along with some quotes from the delegates are provided in the tables that follow.

8

Learning

Delegates indicated that the conference provided opportunities for learning from other regions, schools, projects, students and staff. Learning occurred in the form of new ideas or knowledge, implementation strategies, practical resources and research/evidence-­‐based practice.

New ideas or knowledge: New awareness or understanding of a specific concept as a result of the information presented during the sessions.

These were unique across the majority of the sessions.

“Ideas on including environmental education across curriculum.”

“Lots of great ideas for elementary student leadership.”

Implementation Strategies: Specific tools or processes for executing activities discussed in the sessions. Delegates reported learning about a variety of strategies.

“Terrific hands on ideas to use at our school.”

“Practical solutions to some common challenges in implementing healthy initiatives.”

Practical Resources: Specific resources were shared in a number of sessions and delegates felt that these resources would positively impact and aid their day-­‐to-­‐day activities.

Resources that delegates found useful included websites, analysis tools, grant programs, toolkits, and services/initiatives offered by specific organizations.

Research/Evidence-­‐based practice: Learning about relevant research and the value of having data to support activities and programs were significant to delegates.

“Nice to see some data to back up his thoughts (i.e. throwing interventions at our kids didn't make a big change whereas involving the kids does!)”

“The statistics blew my mind!”

“I learned a lot about research being done in the field of youth PA. I hope to apply it to the research I'm doing.”

9

Connections

Delegates discussed developing connections and relationships with other delegates through networking, and learning about strategies to develop partnerships and collaborations.

Networking: Having valuable conversations with a variety of other delegates.

“Made some GREAT connections. People are so willing to help.”

Strategies for partnerships and collaborations: Learning about the steps required to make a connection and develop a relationship around a common purpose or goal.

“Great networking session.”

“Language to use when forming partnerships with schools. How to structure a question.”

“The information on the process of collaborating effectively is invaluable.”

Further support to show that Connections were an outcome of the 2015 conference was observed in the responses to the following survey questions, to which most of the delegates “strongly agreed” or “agreed”:

The 2015 Program provided opportunities for me to identify new opportunities for partnerships and collaboration.

Strongly Disagree Disagree

Average = 4.35 / 5.0

Undecided

The 2015 program provided opportunities for me to connect with and learn from other delegates Average= 4.29 / 5.0

0%

Agree Strongly Agree

20% 40% 60% 80% 100%

10

Inspiration

In addition to learning and making connections, delegates were inspired during the 2015 conference. Specifically, delegates indicated that certain sessions and speakers were inspirational with their messages and identified a number of quotes they found inspiring.

These quotes focused on having a vision, creating change, challenging yourself, dreaming big, setting goals, and finding happiness. Some of these quotes are listed below.

What will your Nobel peace prize be for?

"Life is about finding what makes you happy and doing it," Shay Esposito, student keynote at #stf2015

“No one said it would be easy” This is so important and too often forgotten Great words of wisdom! Inspiring

"The most pathetic person in the world is someone who has sight, but no vision" -­‐ Helen Keller what's your vision? #stf2015

While these outcomes (Inspiration, Leaning and Connections) were limited to the Delegates in attendance, it should also be noted that the information does have the ability to transfer to other audiences and environments further increasing the impact of the conference. When asked how likely they were to use the information presented in the next 5 months the majority of delegates answered “likely” or “very likely.”

very unlikely unlikely

How likely are you to use the information presented in the next 5 months?

Unsure likely

0%

20% 40% 60% 80% 100%

very likely

*A survey will be sent to delegates in May of 2015 to see what information has been used and transferred directly into a practical setting.

11

Recommendations

Although the majority of the data collected highlighted how the 2015 Shaping the Future Conference was high quality, relevant to most delegates and resulted in immediate outcomes there were also a number of recommendations that were shared by delegates.

Program Recommendations

1. Include more critical concepts and ideas It was suggested that critical presentations that challenge our ideas about health in schools are needed. Additionally, it was suggested that generally accepted terms like “health”, “comprehensive” and “wellness” need to be further explored and challenged.

“Although understanding mental illness is important, clearly many people are not understanding that positive mental health is NOT about mental illness.”

2. Ensure all presentations provide specific examples and practical resources Delegates highlighted the importance of having simple and effective ways of using the information and learnings that are shared throughout the conference. It was suggested that specific tools, resources, policies and best-­‐ practices be shared in every session.

“In a school just beginning its wellness journey, I need to be able to provide teachers with simple and effective ways they can address wellness in their curricular instruction…I would have liked to hear more about how teachers are doing this currently so that I could pass it on.”

3. Allow for more opportunities to connect, share and learn from each other While the majority of delegates felt that the conference provided an opportunity to connect with others, it was suggested that formal opportunities might be more effective. Suggestions to increase connections included: •

•

More open-­‐space technology -­‐ “I am not a fan of going to a conference to hear people speak at me. I would love to see the conference start with an “open space” so that people can share, discuss and learn about topics that are most relevant to them. The 2 sharing sessions I attended left me wanting more.”

More connections/networking through play – “I would like to see more opportunities for connections to other delegates through play. I really enjoyed Lindsay Wright’s presentation for this reason. She ran an activity during her presentation that instantly broke the ice with other delegates. Maybe there could be more opportunities during breaks, not for sport (e.g. hockey) necessarily, but less conventional forms of play that are more inclusive.”

12

•

More free time and informal opportunities to connect-­‐ “I wonder if there should be more free time and perhaps incentives to connect with others.”

•

Include Networking opportunities based on a specific topic or area– “It was hard to connect with so many people. I had no idea who to even talk to about what I was most interested in…try to identify people’s interests/expertise (e.g. Physical Education, Mental Health) somehow to increase effectiveness of networking.”

4. More Interaction…especially in the ‘Interactive Workshops’ A number of delegates indicated that they would have like the Interactive Workshops to be more interactive in future. It was stated that learning and connection happen more naturally when people are able to engage with the information and that this should be the primary focus of ‘Interactive Workshops’.

“A little bit less lecture and more interaction.”

“[Need more] interaction! With the presenters and each other.”

5. Ensure session descriptions are accurate Many delegates felt that the session descriptions did not accurately reflect the information that was presented. Suggestions to ensure delegates make informed decisions on the sessions they attend included:

•

•

Be clear on what the session is about, the take away messages, and resources being shared-­‐ “I found the description/title of the session to be misleading as the majority of the work was spent on competencies…which is relevant to a degree but I had thought we would be learning more about implementing wellness in specific subject areas and provided with specific examples of how teachers are doing it.”

Clarify the primary and secondary audience for every session-­‐ “One thing I think would be helpful is to mark the sessions better. Maybe mark who it is geared for (e.g. teacher, administrator). Also a couple key words that describe the session. We had a couple of sessions where the title was misleading and what we thought we were going to attend was totally different.”

6. Walk the Talk A number of delegates felt that the sessions were very sedentary given that it was a conference focused on health and activity. It was therefore suggested that more activity be built into the concurrent sessions and the interactive workshops.

“There was an extreme and unnecessary amount of sitting during the conference. For an active conference it was not very active.”

13

7. Provide more time for sessions but ensure they are interactive.

A number of delegates felt that some sessions were too short and that more time should be given to allow for examples, ideas and resources to be shared.

“The sessions in the afternoon were too short. Difficult to gain knowledge in a 20 minute timeframe.”

“There was not enough time to her from more than 2 of the presenters in the room and there were so many good ideas to hear.”

8. Offer repeat sessions a. Delegates often had to choose between multiple sessions, all of which were of interest and relevant to their role. It was therefore suggested that some sessions be offered multiple times throughout the conference.

This would ensure delegates get to attend all relevant sessions and could prove to be more beneficial for the presenters because they would ultimately reach more people.

“I wish there would have been repeat sessions. I missed going to some sessions because they were only offered once and I had to choose between so many good ones.”

Other valuable ideas that were shared related to the conference program include:

• Increase the diversity of the delegates by advertising to specific schools or groups and ensuring presentations are more diverse (e.g. FNMI perspective).

• Send out a needs assessment prior to the conference to ask what information participants are interested in.

• Have a workshop on the value of having research done in schools and the value of evidence based practice.

• Do more “project” based workshops where delegates work through problems or scenarios together and get feedback from the group.

• Have more movement workshops (e.g. yoga) • More examples of how to foster student leadership.

• Examples of successful fundraising activities • Host a session or a social that introduces and explains the multiple groups across Alberta that support healthy schools.

Other Recommendations

In addition to comments about the program, delegates also shared some thoughts about the conference food, venue, location, speaker rates and the App.

14

Food There were a number of comments that suggested the delegates were not happy with the food that was available this year. Delegates commented about: • The type of food: “Hard boiled eggs and porridge…more options please” • The amount of food: “Each time I went for food they ran out” • The wait for food: “It took over 1.5 hours for us to eat (we were the last table) and when we got to the buffet line, there was little left.”

Venue Given that the conference has grown so much, a number of delegates commented on the Delta’s ability to accommodate the large number of people. Comments suggested:

• The Delta may be too small given the crowds: “Though the venue is secluded and intimate it is now too small to host so many.”

• The need to move seamlessly from meals to sessions: “Needs to be a dedicated main hall-­‐ not one that has to double for sessions and meals.”

• There was a lack of hotel rooms: “Finding a venue that all people can stay at. If the maximum number of participants at the conference is 500 people then have enough rooms blocked off at the venue for people.”

Location While there were some concerns about the Delta’s ability to accommodate the conference in future years there were a number of positive comments about the conference being hosted in Kananaskis.

“Kananaskis is a great location and generally brings more delegates to the conference.”

Speaker Rates It was suggested that ‘speaker rates’ should be something that is considered in future years, especially for those presenters who are only attending one day of the conference.

“It would be nice if there was some compensation for speakers towards the conference registration, or other fees.”

App While some delegates found the app useful others expressed frustration with the app and its functionality.

“The conference app was frustrating. I found last year’s app more user friendly.”

“The APP was not functional to all delegates. There were people in the delegates category that did not have phones or iPad or other pieces of technology compatible to the app…”

15

Conclusion

Based on the data collected a few conclusions can be made about the 2015 Shaping the Future Conference.

1. The information presented was relevant to the majority of delegates who attended.

2. The majority of delegates felt the speakers and sessions were of high quality.

3. The majority of delegates were presented with new ideas and acquired new knowledge, resources and tools.

4. The conference provided an opportunity for delegates to network and explore possibilities for potential partnerships and collaborations.

5. Delegates provided Ever Active with relevant recommendations to improve the experience and outcomes for those attending the 2016 conference.

In addition, based on their interpretation of the data the evaluators have provided their recommendations for enhancing the impact of future conferences.

1. While conference delegates represented a number of sectors, the audience was primarily comprised of individuals from Education. As such, the program content was largely geared towards a teacher and school-­‐based audience, which may have reduced the relevance for delegates from other sectors.

Therefore in order to maintain a cross-­‐sectoral approach and ensure the information presented is relevant for ALL delegates, it is suggested that:

a. Speakers, keynotes and the program content remain broad (i.e. from all sectors) to continue to ensure a range of learning opportunities while, b. Marketing Shaping the Future specifically to 1) those from the Education sector or 2) those interested in connecting, sharing and collaborating with individuals in the Education Sector.

2. A number of delegates shared that the session descriptions did not accurately reflect the presentations and/or that the sessions required more practical examples and resources.

To improve the alignment between expectations and delivery, it is suggested that a more detailed call for presentations is provided based on the Shaping the Future conference goals.

A more detailed abstract would help organizers to create a balanced program and aid delegates in making decisions about which presentations to attend. It would also provide the information necessary for staff to make specific changes based on what delegates shared in the Recommendations section.

16

An example call for presentations is provided below:

The goal of Shaping the Future is to provide opportunities for delegates to learn, connect and be inspired. In an attempt to ensure all sessions are meeting the needs of our audience members, we ask you to directly respond to the following questions:

• Who is your target audience?

• What new knowledge or ideas will your presentation introduce? (Are these ideas research and/or practically based?)

• What resources or implementation strategies will be shared?

• How does your session provide opportunities for delegates to interact with and learn from each other?

3. As a final point, we suggest that it would be beneficial to review the outcome model provided in this report (p.8) with Ever Active staff and funders to ensure it aligns with the goals of the organization and the Shaping the Future conference itself. If it is consistent with the overall outcomes of Ever Active Schools, it is recommended that the future conference programs intentionally reflect the outcome model. This will ensure it can be used as a reference point for evaluating future conferences and mapping outcome trends over time.

17