Best Practice in Performance Measurement of Fall Prevention Training: Toronto Public Health's Step Ahead Program for Health Care Providers and Caregivers to Older Adults Toronto Public Health Susan Plante and Margaret de Wit Public Health/Primary Health Care Conference June 4-5, 2014 1 of 20
Foundations for this Work
Mandates
• OPHS • Accountability Agreements • City Priorities
Partnerships
• LHINs; PHUs • Research Organizations • Service Providers/Agencies
Best Practice Guidelines
• RNAO - BPG • International • Performance Measurement
2 of 20
Step Ahead Program Goals • Improve health outcomes of older people and reduce burden of falls on the healthcare system in partnership with all healthcare providers serving older adults in Toronto
Objectives • Facilitate adoption of best practice fall prevention strategies for healthcare agencies • Improve access to fall prevention services for older adults
3 of 20
Program Target and Reach • Targeted to agencies and staff in Toronto serving healthcare needs of elderly, and to other personal caregivers • 400 agencies currently being reached through marketing, outreach, other strategies • 60 agencies participated in the training in the last 10 months (timeframe of this study)
4 of 20
Step Ahead Module 1: Fall Prevention Education
A Step Ahead to Fall Prevention in Older Adults Training Program for Health and Allied Health Care Providers
5 of 20
Step Ahead Module 2: Exercises (HSEP – CCAA)
6 of 20
Use of Best Practice Performance Measures •
Funders and other stakeholders now requiring tangible evidence of program efficacy, the capacity to produce a desired result
•
Consequently, a growing emphasis on measures of quality and impact in addition to activity (e.g., counts of activities or numbers served)
•
Performance measurement – focus on outcomes and results about efficiency and effectiveness of a program
7 of 20
Methods •
Various theories used to frame questions about acquisition, intention and practice of skills/knowledge
Between July, 2013 and April, 2014 • Self-administered questionnaire following completion of training by 140 participants
•
Electronic or telephone survey of 47 consenting participants at 3 months post training
8 of 20
Methods (continued) Key measures Initial post-training survey: (Activity) How many agencies/people have we served? (Quality) 1. Did you acquire knowledge/skills? 2. Do you intend to use/share them? 3 months post-training survey: (Impact) Have you used/shared your knowledge/skills with clients?
9 of 20
Methods (continued) Additional measures Initial post-training survey: ďƒ˜ (Quality) How satisfied were you with the workshop/training? 3 months post-training survey: ďƒ˜ (Impact) With how many clients have you used the exercises?
10 of 20
Survey Completion Rates
11 of 20
Results - Quality Knowledge Acquisition (initial survey) • 97% reported knowledge increase as a result of the training, with 68% stating substantial knowledge increase • 98% reported new skills learned
12 of 20
Results – Quality (continued) Intention (initial survey) • 97% reported intention to share workshop information with clients • 95% reported intention to use training resources with clients
13 of 20
Results – Impact Practice (3 months post training) • 77% with the opportunity (“yes”/”no”) reported knowledge practice • 41% overall reported “no use” of knowledge or “no opportunity to use”
14 of 20
Results – Impact (continued) Practice (3 months post training) • 68% with the opportunity (“yes”/”no”) used HSEP • 55% overall reported “no use” or “no opportunity to use” HSEP exercises with clients
15 of 20
Results – Additional Measures Quality Satisfaction (initial survey) 96% satisfied/very satisfied
Impact Clients served (3 months post training) Over 200 trained by 21 caregivers
16 of 20
Key Findings and Conclusions • Survey measures of quality (knowledge/skill, intention) and impact (practice following training) appear useful for collecting meaningful performance data • Step Ahead provides participants with high quality practical knowledge and training • Many participants are also integrating Step Ahead knowledge and training into practice
17 of 20
Challenges and Limitations Challenges (3 months post training) • Among those implementing HESP, 21% reported challenges
Limitations • Survey instruments collected information about self-reported knowledge acquisition, intentions and practice, not actual acquisition, intentions and practice • Self-selecting nature of participants studied does not facilitate generalization of results
18 of 20
Next Steps •
Explore possible challenges with implementation and reasons for non-use, despite initial intentions better identify in-scope agencies and appropriate target audiences to maximize relevance and uptake of training assess organizational readiness to support the use of Step Ahead
•
Continue with performance measurement and expand program reach to include all healthcare providers of older adults
19 of 20
THANK YOU Questions?
20 of 20