A Critical Review of Measures and Instruments for Quality Improvement Maribeth Bersani, Senior Vice-President, Argentum Sheryl Zimmerman, Professor, University of North Carolina at Chapel Hill Pat Giorgio, President/CEO, Evergreen Estates
The Importance of Quality Improvement in
Assisted Living
A Critical Review of Measures and Instruments for Quality Improvement in Assisted Living Funded by the Center for Excellence in Assisted Living
Aims
Conduct a comprehensive scan and review of tools (measures and instruments) suitable for quality improvement in assisted living and related residential care settings Include tools that have been used in assisted living and other health and long-term care settings, because … … few tools have been developed specifically for assisted living … tools developed in other settings may potentially be used or adapted for assisted living
Key Domains for Quality Improvement ď ś
Person-centered care
Key Domains for Quality Improvement ď ś
Person-centered care
ď ś
Medication management
Key Domains for Quality Improvement
Person-centered care
Medication management
Care coordination/transitions
Key Domains for Quality Improvement
Person-centered care
Medication management
Care coordination/transitions
Resident/patient outcomes
Key Domains for Quality Improvement
Person-centered care
Medication management
Care coordination/transitions
Resident/patient outcomes
Workforce
Key Areas Within Each Domain Person-centered care
Resident/patient outcomes
• Resident-direction
• Physical function (activities of daily living)
• Homelike atmosphere
• Psychosocial well-being (cognition, affect, quality of life)
• Close relationships
• Satisfaction
• Staff empowerment (e.g., collaborative decision making)
• Medical events (e.g., falls)
Medication management
Workforce
• Medication risk (prescribing)
• Turnover
• Medication self-administration
• Consistent assignment (e.g., staffing models, caregiver time)
• Medication errors (administration)
• Stress, burnout
• Medication reconciliation
• Satisfaction
Care coordination/transitions • Information transmission (e.g., timeliness) • Tracking/response to information (e.g., proactive)
• •
Efficiency (e.g., unnecessary readmissions, duplication) Patient experience
Sample Key Words for Scan • Tools: tool, measure, instrument, survey, interview,
•
•
•
• Settings of care: long-term care, assisted living, residential
inventory, questionnaire, scale, index, profile, toolkit,
care, board and care, senior housing, home care, home and
protocol, evaluation, assessment, test
community based, other
Person-centered care: resident direction (e.g., autonomy);
•
Resident/patient outcomes: function (e.g., mobility);
homelike atmosphere (e.g., home); close relation-
psychosocial well-being (e.g., depression);
ships (e.g., familiarity); staff empowerment
satisfaction; medical events (e.g., falls)
Medication management: risk (e.g., prescribing);
•
Workforce: turnover (e.g., separation); consistent
self-administration (e.g., capacity); errors
assignment (e.g., ratio); stress, burnout
(e.g., preparation); reconciliation (e.g., orders)
(e.g., burden); satisfaction
Coordination/transitions: information transmission (e.g., timeliness); tracking/response (e.g., communication); efficiency (e.g., readmission); patient experience (e.g., distress)
Literature Search and Advisory Panel
Research librarian: created every possible combination of search terms related to tools, settings, and domains/key areas, and searched numerous peer-reviewed databases, grey literature reports, Google sites, and websites Panel: fourteen experts provided input on grey literature, Google sites, websites, and overall format Research team: reviewed the information from all sources; those that identified relevant tools resulted in search of information about the tool, and synthesis and critique
Inclusion and Exclusion Criteria Inclusion Criteria “Tools” include measures and instruments that generate a score:
Measure: Not based on scales or indices; includes a numerator and a denominator
Instrument: Based on scales or indices; includes more than two items that are aggregated in some way
The tool measures a key area within the five domains Tools that measure end-of-life care and outcomes are included because they relate to transitions Eligible tools were developed or used in the last ten years in at least one target setting in the United States Exclusion Criteria Tools used for screening Tools used for care planning, including those to assess resident preferences
Tools used for clinical care Tools exclusively relevant to hospital and/or home care Tools related exclusively to outcomes for informal (family) caregivers Tools used in qualitative studies, editorials, dissertations, conference abstracts, and op-ed pieces
Synthesis and Critique Quantitative Scoring Strategy SCORING FOR INSTRUMENTS Component
Scoring Reliability 2 (good) if reliability coefficients generally ≥ 0.80 (test-retest, inter-rater, internal consistency) 1 (fair) if reliability coefficients generally 0.60–0.79 0 (poor) if reliability coefficients generally <.60 or no information Criterion validity 2 (good) if reliability coefficients generally ≥ 0.60 (convergent, discriminant, predictive, concurrent) 1 (fair) if reliability coefficients generally 0.40–0.59 0 (poor) if reliability coefficients generally <.40 or no information Interpretability/utility 2 (good) if range of scale is used and reflects potentially actionable items 1 (fair) if range of scale is used or reflects potentially actionable items 0 (poor) if range of scale is not used and does not reflect potentially actionable items SCORING FOR INSTRUMENTS AND MEASURES Component Scoring Ease of use 2 (good) if not time intensive to train/administer and has simple scoring 1 (fair) if not time intensive to train/administer or has simple scoring 0 (poor) if time intensive to train/administer and scoring is not simple Benchmarks for long- term care 2 (good) if benchmarks existed 0 (poor) if no benchmarks existed
Results ď ś
ď ś
The peer-reviewed literature search generated 9,048 nonduplicative citations; the grey literature search generated 361 sources in addition to websites of 51 organizations Reviewing all sources, assuring that the tool met eligibility criteria, and omitting duplications, resulted in 254 tools: 136 measures and 118 instruments
Results Number of Tools (Measures and Instruments) Identified For Each Domain
Domain
Tools
Measures
Instruments
Person-centered care
22
1
21
Medication management
24
20
4
Care coordination/transitions
32
28
4
Resident/patient outcomes
69
28
41
Workforce
107
59
48
Considerations to Recommend Tools
Relevance
Efficacy
Cost
Time demands
Person-Centered Care: 6 tools Name
Description
Source of Information, Process to Obtain
Culture Change Scale (CCS)
48 items assessing six sub-scales: 1) system-wide culture change; 2) resident choice; 3) organizational design; 4) empowering supervision; 5) job design; and 6) decision-making. Sample items include "the environment encourages new ideas; how often can residents eat what they really want; my job duties allow me to enough time to do my job properly." Scores are provided on a 5 point Likert scale and mean scores are derived.
STAFF INTERVIEW
Toolkit for PersonCenteredness in Assisted Living (PC-PAL Resident)
49 items to assess person-centered practices in assisted living and similar long-term care settings, addressing the domains of well-being and belonging (18 items), individualized care and services (12 items), social connectedness (10 items), and atmosphere (9 items). Items are scored on a 4 point Likert scale. Scores for each area and overall are obtained, with higher scores indicating more person-centeredness.
RESIDENT QUESTIONNAIRE
Toolkit for PersonCenteredness in Assisted Living (PC-PAL Staff)
62 items to assess person-centered practices in assisted living and similar long-term care settings, addressing the domains of workplace practices (23 items), social connectedness (16 items), individualized care and services (8 items), atmosphere (8 items), and caregiverresident relationships (7 items). Items are scored on a 4 point Likert scale. Scores for each area and overall are obtained, with higher scores indicating more person-centeredness.
STAFF QUESTIONNAIRE
Toolkit for Person-Centeredness in Assisted Living (PC-PAL) Sample Items (of 49 that assess four domains) I. Well-Being and Belonging These questions are about things that relate to your feelings of well-being and belonging in this assisted living community.
To what extent do you disagree or agree thatâ&#x20AC;Ś
Strongly disagree Disagree
Agree
Strongly agree
Donâ&#x20AC;&#x2122;t know
1
I feel a sense of belonging here.
1
2
3
4
DK
2
My needs for private space are met.
1
2
3
4
DK
3
It feels like home.
1
2
3
4
DK
4
Outdoor areas are easy to get to, and are safe, attractive, and have comfortable seating.
1
2
3
4
DK
5
The activities here are meaningful to me.
1
2
3
4
DK
6
There are opportunities and a place here for me to practice the religious or spiritual beliefs I choose.
1
2
3
4
DK
Medication Management: 10 tools Name Medication Tracking Tool (Advancing Excellence) Nursing Home Compare Percent of Residents Receiving Antipsychotic Medication (long-stay) Nursing Home Compare Percent of Residents Who Are Newly Administered Antipsychotic Medications (short-stay)
Description
Source of Information, Process to Obtain
Six rates related to antipsychotic use in nursing homes: rate of residents on PRNs, for those with dementia, more than one antipsychotic, gradual dose reduction (GDR) attempted, GDR with dose reduction, GDR with medication discontinued.
CHART ABSTRACT
As per name
CHART ABSTRACT
As per name
CHART ABSTRACT
Medication Tracking Tool (Advancing Excellence) www.nhqualitycampaign.org/goals.aspx
Information about each resident who is prescribed an antipsychotic
List of clinicians prescribing antipsychotic Details about each antipsychotic prescription
Care Coordination/Transitions: 17 tools Name
Description
Care Transitions Measure (CTM-15, CTM-3)
15 items assessing care transitions in relation to goals, potential health care needs, site of care, information, understanding, warning signs and symptoms, written plan of care, selfcare, confidence, and purpose, side effects, and administration of medications. Items are scored on a 4 point Likert scale. A 3 item version is available and recommended for public reporting.
Family Perception of Physician-Family Caregiver Communication (FPPFC)
7 items assessing family perceptions of communication between physicians and family caregivers of individuals who spent their last month of life in long-term care. Sample items include “the doctor always spoke to you, other family caregivers, or the resident about [his/her] wishes for medical treatment at the end of life, and the doctor always kept you or other family caregivers informed about the resident’s condition.” Items are scored on a 4 point Likert scale and a mean score is derived.
Avoidable Re-hospitalization The number of unplanned readmissions to any hospital divided by the number of Rate, 30-day Adjusted hospitalizations in that period, adjusted for patient characteristics. (and annualized)
Source of Information, Process to Obtain RESIDENT INTERVIEW
FAMILY INTERVIEW
CHART ABSTRACT
Care Transitions Measure (CTM-3) Transition Following Hospitalization
Strongly disagree Disagree
Agree
Strongly agree
1
The hospital staff took my preferences and those of my family or caregiver into account in deciding what my health care needs would be when I left the hospital.
1
2
3
4
2
When I left the hospital, I had a good understanding of the things I was responsible for in managing my health.
1
2
3
4
3
When I left the hospital, I clearly understood the purpose for taking each of my medications.
1
2
3
4
Resident/Patient Outcomes: 35 tools Name
Description
Source of Information, Process to Obtain
Nursing Home Compare Percent of Residents Who SelfReport Moderate to Severe Pain (long, short-stay)
As per name
CHART ABSTRACT
Nursing Home Compare Percent of High Risk Residents With Pressure Ulcers (long, short-stay)
As per name
CHART ABSTRACT
Nursing Home Compare Percent of Low Risk Residents Who Lose Control of Their Bowels or Bladder (long-stay)
As per name
CHART ABSTRACT
Percent of Residents Who Self-Report Moderate to Severe Pain Nursing Home Compare, Short Stay Version Denominator Description All short-stay residents with an assessment where the information was recorded and pain was present Numerator Description Short-stay residents with an assessment where the assessment meets either or both of the following two conditions: 1. Resident reports daily pain with at least one episode of moderate or severe pain; both of the following conditions must be met:
â&#x20AC;˘ Almost constant or frequent pain â&#x20AC;˘ At least one episode of moderate to severe pain 2. Resident reports very severe/horrible pain of any frequency
Workforce: 28 tools Name
Description
Consistent Assignment Tracking Tool (Advancing Excellence)
Tracks the number of caregivers each resident has for the month and calculates the percent of residents meeting a nursing homes target number.
Eaton Instrument for Measuring Turnover
Number of new employees (full or part time) divided by the number of employees in that category over a 12-month period.
National Nursing Assistant Survey (Management/ Supervision; Percent of respondents reporting perceptions of 10 items related to management/ Organizational supervision; 29 related to organizational commitment/job satisfaction; 14 items in workplace Commitment/Job environment. Satisfaction; Workplace Environment sections only)
Source of Information, Process to Obtain
CHART ABSTRACT
RECORD REVIEW
STAFF INTERVIEW
Consistent Assignment Tracking Tool (Advancing Excellence) www.nhqualitycampaign.org/goals.aspx
List of residents
List of caregivers Record of each residentsâ&#x20AC;&#x2122; caregivers, separately for long- and short-stay, during the first and second half of the month
Implementation: Plan, Do, Study, Act Continuous quality improvement
Administer the instrument/measure
Modify efforts (if necessary)
Meet with staff, residents, families
Re-administer the instrument/measure
Jointly plan for change
CONCLUSIONS AND GAP ANALYSIS
Resident acuity in relation to staffing levels
Overall measurement of quality
FINAL REPORT and TOOLS Center for Excellence in Assisted Living: http://www.theceal.org/ Sheps Center for Health Services Research University of North Carolina at Chapel Hill http://www.shepscenter.unc.edu/
THANK YOU FOR ATTENDING