9 minute read
U.S. News & World Report Releases Proposed New Ranking
from AMRPA Magazine | March 2020
by AMRPA
U.S. News & World Report Releases Proposed New Ranking Methodology for Rehabilitation Hospitals for 2020-2021 AMRPA to Provide Substantive Comment on Proposed Methodology
Kate A. Beller, JD, AMRPA Executive Vice President for Policy Development and Government Relations
In March 2019, U.S. News & World Report announced its intent to shift from relying solely on expert opinion in its rehabilitation hospital rankings and incorporate certain types of objective data in its methodology. As part of this effort, U.S. News has participated in months-long engagement with the American Medical Rehabilitation Providers Association (AMRPA) and other stakeholders to determine what other inputs should be considered as it works to revise its methodology, as well as the appropriate weights for those inputs. The revised rankings will initially focus on what U.S. News views as “high acuity” rehabilitation care.
U.S. News (working in conjunction with its contractor, Research Triangle Institute, Inc. [RTI]) also indicated interest in identifying data sources that could be incorporated in future ranking years, including all-payer registries. Over the course of numerous meetings, an AMRPA workgroup has been providing the U.S. News/RTI team with feedback on: 1) the current data sources and specific measures that U.S. News has identified for consideration; 2) other data sources that should be factored into the rankings in either the 2020-2021 rankings or future years; or 3) operational issues for U.S. News’ consideration, such as the fact that that certain patient populations are not reflected in some of the datasets under consideration. U.S. News expressed strong interest in a number of recommendations set forth from the AMRPA workgroup, such as including stroke as one the conditions that should be considered as part of the hospital’s “high acuity care” (in addition to traumatic brain injury and traumatic spinal cord injury).
Additionally, the U.S. News/RTI team expressed interest in further discussion of some inputs that could be included in current and future ranking editions, such as condition-specific CARF accreditation and teaching status. There was also discussion of implementation/operational issues tied to some of the inputs under consideration, such as taking into account the relationship of a hospital’s patient complexity and number of interrupted stays and other issues involving risk adjustment. U.S. News expressed interest in continuing to work with AMRPA as it finalized the upcoming 2020-2021 methodology, as well as future refinements to the inputs and weights included in its work.
U.S. News’ Proposed 2020-2021 Methodology Following AMRPA’s engagement, U.S. News & World Report released a draft methodology on January 28, with the following proposed measures and weights: Expert opinion (50%) Select Outcomes-Based Measures on IRF Compare (20%) Volume Conditions for Stroke, TBI and TSCI as reported on IRF Compare (10%) Certain Patient Services Reported on the American Hospital Association Annual Survey (6%) Certain Advanced Technologies Reported on the American Hospital Association Annual Survey (6%) Select Patient Safety Process Measures Reported on IRF Compare (5%) Designation as a NIDILRR Model System for SCI, TBI or Burns (2%) CARF International Accreditation (2%)
Additional information on the datasets/inputs, as well as the rationale provided by U.S. News, are detailed more extensively in the sections below:
Eligibility All facilities listed in the AHA annual survey database will automatically qualify for consideration in U.S. News & World Reports Best Hospitals rankings, with the exception of military and federally owned facilities. However, there are eligibility requirements to be considered in the rehabilitation rankings. Rehabilitation hospitals can qualify through 1) appearing in both the AHA annual survey database and CMS IRF Compare or 2) have an expert-opinion score of 1% or higher based on the three most recent years of U.S. News national physician surveys. Of the eligible hospitals, only those achieving what U.S. News determines to be the “highest scores” will receive a ranking. U.S. News also proposed that in addition to its traditional expert rankings, it will also use data from the annual AHA survey, IRF Compare, and NIDILRR model system designation from eligible hospitals.
Structural Measures In its designated “structure” portion of its ranking, U.S. News has proposed to put an emphasis on volume, with volume of patients treated with stroke, traumatic brain injury (TBI) and traumatic spinal cord injury (TSCI) each making up 3.33% of the rankings (10% total). U.S. News also proposes to examine patient services reported in the AHA survey, including whether hospitals offer services like patient representative services, translators, case management and several others. The patient management score will be weighted at 6% of the total score. Another 2% of the score will be determined by whether the hospital is designated as a model system by NIDILRR. U.S. News proposed methodology also includes an Advanced Technology ranking worth 6% of the score, which accounts for hospitals’ utilization of robot-assisted walking therapy and electrodiagnostic services, among other technologies. Finally, U.S. News will use CARF accreditation as 1% of the total score. In total, structure measures will be weighted at 25% of the total score.
Process Measures The “process” portion of the rankings as proposed will incorporate patient safety measures from IRF compare and its annual expert opinion rankings. U.S. News proposes to use the influenza vaccination rate measures for both health care personnel and patients. Each of these measures will be separately weighted at 2.5%, and calculated using data from IRF Compare. U.S. News acknowledges that the patient influenza rate was recently removed from the IRF Quality Reporting Program (QRP), and will therefore be retired after its inclusion in the 2020-2021 methodology.
Outcomes Measures The “outcomes” measures used in the proposed methodology will make up 20% of the total hospital score. CMS proposes to weight each of the following measures at 6.67% of the total score: Preventing potentially avoidable 30-day hospital readmissions after IRF discharge; Preventing potentially avoidable hospital readmissions during rehabilitation care; and Successful discharge to home and community. All of these measures will be calculated using data from IRF Compare.
Expert Opinion Under the proposed methodology, the annual expert opinion survey (which previously was 100% of the score) would be weighted at 50% of the score. For 2020-2021, U.S. News proposes to base a hospital’s expert opinion score on the average number of nominations from the three most recent annual surveys of board certified physicians conducted for the Best Hospitals rankings. In completing the survey, physiatrists are specifically asked to nominate hospitals in the field of rehabilitation medicine that “they consider best for patients with serious or difficult conditions,” and they can nominate as many as five hospitals. The sample of surveying physicians for 2020 are drawn from the Doximity Masterfile, and U.S. News expects that the survey will be completed in March.
Anticipated Industry Impact In its proposal, U.S. News provided some insight into how the new methodology likely will (or will not) effect hospital’s current rankings. It calculated scores using the current methodology and currently available data, and provided a comparison between previously ranked and unranked hospitals. This analysis found that previously ranked hospitals “performed better than previously unranked hospitals on nearly every measure that will be used in the new methodology,” particularly with respect to volume. The only exceptions were health care personnel and patient vaccination rate measures and the successful discharge to home and community measure. U.S. News also found the proposed methodology resulted in a two-fold difference in median overall score between previously ranked hospitals and all other hospitals.
Potential Future Refinements to the Methodology During discussions between U.S. News and AMRPA about potential future improvements to the rankings, numerous AMRPA members expressed interest in developing condition-specific rankings. While U.S. News expressed interest in moving to this type of rankings system, it “pragmatically” views this as being “many, many years off.”
U.S. News also expressed interest in incorporating other all-payer datasets into its methodology in future years. AMRPA will be engaging with U.S. News on this issue in the coming months to whether and how this type of data could be effectively captured in the rankings methodology.
*** AMRPA staff has closely reviewed the current draft of the U.S. News methodology following its late January release, and will be reconvening a workgroup to develop the Association’s comments and recommendations to U.S. News for the 2020-2021 methodology and future years. As the table shows, hospitals that were previously ranked in Rehabilitation performed better than previously unranked hospitals on nearly every measure that will be used in the new methodology. For example, as a group, the previously ranked hospitals treated a median of 59 traumatic brain injury cases and 293 stroke cases among Medicare beneficiaries, compared to medians of 19 and 107, respectively, among previously unranked hospitals. Differences such as these contributed to a two-fold difference in median overall score between previously ranked hospitals and all other hospitals. On the measure of overall score, even the lowest-scoring previously ranked hospital outranked all but a handful of the previously unranked hospitals in our preliminary 2020 analysis.
Input Source Total Weight
Expert Opinion Average Number of Nominations Received in the 2018, 2019 and 2020 Annual Survey of Board-Certified Physicians Conducted for U.S. News Best Hospital Rankings Outcomes-Based Measures: • Preventing potentially avoidable 30- day hospital readmissions after IRF discharge • Preventing potentially avoidable hospital readmissions during rehabilitation care • Successful discharge to home and community IRF Compare
50%
20% (each measure counted for 6.67%)
Patient Volume for Stroke, Traumatic Brain Injury and Traumatic Spinal Cord Injury Patient Services • Cardiac rehab • Case management • Enabling Services • Translators • Neurological services • Occupational health services • Pain management program • Palliative care program • Patient-controlled analgesia • Patient representative services • Physical rehab outpatient services • Psychiatric services • Social work services • Wound management services • Health research • Hemodialysis IRF Compare 10% (each measure counted for 3.33%)
AHA Annual Survey (Services are counted as “present” if they are available at the facility or health system, or via a partnership, as indicated on the AHA survey)
6%
Patient Safety Process Measures • Influenza Vaccination Coverage Among Health care Personnel • Percent of Residents/Patients Assessed & Appropriately Given Influenza Vaccine
IRF Compare
5% (each measure counted for 2.5) Note: the patient influenza rate will be retired next year in the U.S. News Rankings, as it was recently removed from the IRF QRP
NIDILRR Designation • Designation as a Model System in Rehabilitation in the area of Spinal Cord Injury, Burns, or Traumatic Brain Injury NIDILRR-Reported Data
CARF International Accreditation AHA Annual Survey
2%
1%
JOIN TODAY!
Advocating. Educating. Connecting.
AMRPA: Working Together To Preserve Access To Medical Rehabilitation Maggie Ramirez · VP of Membership Services · 347-573-3732 · mramirez@amrpa.org AMRPA: Working Together to Preserve Access to Medical Rehabilitation Elizabeth Katsion, AMRPA Member Services Coordinator, ekatsion@amrpa.org, 202-207-1102.
Transforming Medical Rehabilitation
