Mixed Messages to Consumers From Medicare: Hospital Compare Grades Versus Value-Based Payment Penalty

ABSTRACT

Objectives: To (1) compare the 2015 hospital grades reported on Medicare’s Hospital Compare website for heart failure (HF) and acute myocardial infarction (AMI) readmissions with the HF- and AMI-specific scores for excess readmissions used to assess Medicare readmission penalties and (2) assess how often hospitals were penalized for excess readmissions in only 1 or 2 conditions, given that hospitals received a penalty impacting all Medicare payments based on an overall readmission score calculated from 5 conditions (HF, AMI, pneumonia, chronic obstructive pulmonary disease, and total hip/knee arthroplasty).

Study Design: Retrospective secondary data analysis.

Methods: Descriptive analyses of hospital-specific, condition-specific grades and excess readmission scores and hospital-level penalties downloaded from Hospital Compare.

Results: Of the 2956 hospitals that had publicly reported HF grades on Hospital Compare, 91.9% (2717) were graded as “no different” than the national rate for HF readmissions, which included 48.6% that were scored as having excessive HF admissions, and 87% received an overall readmission penalty. Of 120 (4.1%) hospitals graded as “better” than the national rate for HF, none were scored as having excessive HF readmissions and 50% were penalized. AMI data yielded similar results. Among 2591 hospitals penalized for overall readmissions, 26.6% had only 1 condition with excess readmissions and 27.5% had 2 conditions.

Conclusions: Many hospitals with an HF and AMI readmission grade of “no different” than the national rate on Hospital Compare received penalties for excessive readmissions under the Hospital Readmissions Reduction Program. The value signal to consumers and hospitals communicated by grades and penalties is therefore weakened because the methods applied to the same hospital data produce conflicting messages of “average grades” yet “bad enough for penalty.”

Am J Manag Care. 2018;24(12):e399-e403Takeaway Points

Hospital grades for readmissions seen by consumers on Hospital Compare are often out of line with the readmission penalties assessed to hospitals. Although the same readmission data are used to calculate penalties and grades, hospital grade assignment conservatively takes into account measurement uncertainty, leading to most hospitals receiving an average grade of “no different than the national rate.” The threshold for a readmissions financial penalty does not account for uncertainty, however, and is additive across many conditions, leading to the majority of hospitals (even with good grades) receiving a financial penalty for readmissions.Risk-adjusted rates of unplanned hospital readmissions are used for 2 purposes: (1) assigning “grades” to hospitals on the patient-facing CMS Hospital Compare website and (2) assigning financial penalties to hospitals by the Hospital Readmissions Reduction Program (HRRP).^1,2 In 2015, the conditions targeted by the HRRP included heart failure (HF), acute myocardial infarction (AMI), pneumonia, chronic obstructive pulmonary disease (COPD), and total hip/knee arthroplasty; coronary artery bypass graft was added in 2017. The cost of readmissions for HF, AMI, pneumonia, and COPD for Medicare patients totaled $5.2 billion in 2013—one-third of the estimated $15 billion for Medicare readmissions annually.³

On Hospital Compare, CMS reports the grades assigned by computing a risk-adjusted 95% CI estimate for the hospital’s readmission rate and comparing the interval estimate with the national 30-day observed unplanned readmission rate. The readmission rates are risk adjusted for characteristics available in claims data that make an unplanned readmission to the hospital more likely, including age and comorbidities known at the time of the original admission. Hospitals with a 95% CI estimate including the national rate for the condition are graded as “no different than the national rate”; if their 95% CI estimate is entirely below the national rate, they are graded as “better than the national rate”; and if the entire 95% CI is above, “worse than the national rate.”¹ If fewer than 25 cases are available, the grade is listed as “not available” because there are too few cases to allow calculation. The use of 95% CIs for grade assignment on Hospital Compare was not specified by legislature; the rationale for the grading methodology as displayed on Hospital Compare is not reported, but it is a conservative approach that ensures a high degree of certainty that hospitals are correctly categorized on Hospital Compare as performing better or worse than the national average.

The HRRP was established in response to Section 3025 of the Affordable Care Act (ACA), which requires CMS to reduce payments to Inpatient Prospective Payment System (IPPS) hospitals with excess readmissions, effective for discharges beginning on October 1, 2012.² The ACA specified the initial conditions chosen, the maximum percentage penalty by year of program rollout, public reporting of readmission performance on Hospital Compare, and that the penalty was applied to the hospital’s entire Medicare IPPS payments; it also gave the secretary of HHS discretion to expand to more conditions. Financial penalties under HRRP² are assessed using the same exact readmission data used for the condition-specific grades on Hospital Compare; however, the penalty assessment for HRRP only uses the risk-adjusted readmission rate, rather than the 95% CI, for each condition. As detailed in the fiscal year 2002 IPPS final rule, although the penalty percentages are small (0%-3%), they apply to all Medicare revenues at the hospital, not just readmissions, so financial impact varies by the Medicare volume at an individual hospital. For the HRRP, the excess readmission ratio (ERR)^2,4 is calculated for each condition targeted by the policy (ie, 5 conditions in 2015) using a risk-adjusted “predicted” number of readmissions in the numerator and the expected readmission number at an average hospital with similar patients (ie, patients with similar risk factors for readmission, such as age and comorbidities) in the denominator. An “average” hospital is identified by its mathematical average performance. With “excess” defined as any value greater than 1.0, an overall ERR greater than 1.0 across all included conditions will prompt a penalty, but the dollar amount of each specific hospital’s penalty is determined by both the number of excess readmissions for all conditions included in the HRRP and the hospital’s excess cost for the readmissions.

Despite the same risk-adjusted readmission rates being used for assigning grades and financial penalties, the algorithms for assigning grades and penalties differ. By incorporating measurement uncertainty, Hospital Compare’s publicly reported grades categorize most hospitals’ readmission rates as “no different” than the national rate and far fewer as “better” or “worse” than the national rate.^1,5 In contrast, readmission financial penalties are calculated without respect to measurement uncertainty. Because of this difference, we hypothesized that many hospitals receive financial readmission penalties despite having Hospital Compare grades of “no different” or “better” than the national rate. In this analysis, we examined condition-specific ERRs and overall penalties assigned to hospitals graded as “no different” or “better” than the national rate for HF and AMI readmissions on Hospital Compare. We also assessed how often hospitals were penalized for excess readmissions in only 1 or 2 targeted conditions.

METHODS

This retrospective secondary data analysis combined publicly reported 2015 Hospital Compare grades data for readmissions within 30 days⁶ with the Medicare financial penalty data assigned for readmissions over the same time period.^7-9 (Data are available by download from CMS websites.^6,7) Measures studied included the condition-specific grades assigned by Hospital Compare for HF and AMI, AMI- and HF-specific ERRs, and whether the hospital received an overall readmission penalty. Methods for calculating grades and penalties have been previously published.^1,2,5,10 Per CMS policy, 2015 readmissions grades and penalties were based on hospital stays from July 1, 2010, to June 30, 2013, and required the use of a minimum of 25 cases to calculate a hospital’s ERR for each applicable condition.

Given our focus of contrasting penalized hospitals with their corresponding Hospital Compare grade, hospitals not found in both the penalties and grades files were excluded from analysis. By policy, Veterans Affairs and critical access hospitals were excluded, as were Maryland hospitals (which are excluded from the HRRP, as they have been exempted from CMS’s Acute Prospective Payment System since 1977) and Puerto Rico hospitals. Descriptive analyses examined the number of hospitals graded as “better,” “no different,” or “worse” than the national rate for HF and AMI and their corresponding penalization under HRRP. We also examined the range of ERRs associated with each Hospital Compare grade for each of the 5 conditions (ie, HF, AMI, pneumonia, COPD, total hip/knee arthroplasty) and the number of those conditions with ERRs greater than 1 for hospitals receiving an overall readmission penalty. Institutional review board approval was not required because this study used publicly available data not linked to individual human subjects.

RESULTS

There were 4748 hospitals that had readmission data available. The application of exclusion criteria to generate the analytic data set for the HRRP analysis is detailed in the study flow diagram (eAppendix [available at ajmc.com]).⁴ There were 3134 hospitals that had grade and penalty data available on at least 1 of the 5 readmission measures. For the 2 conditions studied in this analysis (HF, AMI), we provide a graphical representation of 3 types of readmission performance data downloaded from the CMS files for each hospital: (1) the publicly displayed grade for readmission rates on Hospital Compare on the x-axis, as “worse,” “no different,” or “better” than the national rate; (2) the ERR for the condition on the y-axis, with values greater than 1.0 categorized as excessive readmissions; and (3) whether the hospital received an overall financial penalty for readmissions across the 5 targeted conditions, indicated by the hospital’s dot color (light blue indicates penalized; dark blue, not penalized).

HF

Figure 1 displays the ERRs (y-axis) and Hospital Compare grades (x-axis) for HF. There were 2956 hospitals that had publicly reported HF grades on Hospital Compare. Of these, 2534 (85.7%) hospitals were penalized for overall readmissions. Of all 2956 hospitals, 2717 (91.9%) were graded against the national rate as “no different,” 119 (4.0%) were graded as “worse,” and the remaining 120 (4.1%) as “better.” ERRs for hospitals graded as “no different” for HF readmissions ranged from 0.81 to 1.19, with 1321 (48.6%) hospitals having an ERR greater than 1.0. Of the 2717 hospitals, 2355 (86.7%) received an overall readmission penalty. For hospitals graded as “better” than the national rate for HF, 0 had an ERR greater than 1.0 for HF (range, 0.699-0.914), and 60 of 120 (50%) were penalized. All 119 hospitals graded “worse” had HF ERRs greater than 1.0 (range, 1.089-1.389), and 100% were penalized.

AMI

Figure 2 displays the ERRs (y-axis) and Hospital Compare grades (x-axis) for AMI. There were 2178 hospitals that had publicly reported AMI grades on Hospital Compare. Of these, 1895 (87.0%) hospitals were penalized for overall readmissions. Of all 2178 hospitals, 2130 (97.8%) were graded against the national rate as “no different,” 23 (1.1%) were graded as “worse,” and the remaining 25 (1.2%) as “better.” ERRs for hospitals graded as “no different” for AMI readmissions ranged from 0.75 to 1.21 and included 1046 (49.1%) hospitals with ERRs greater than 1.0 for AMI. Of the 2130 hospitals graded as “no different,” 1863 (87.5%) received an overall readmission penalty. For hospitals graded as “better” than the national rate for AMI, 0 had an ERR greater than 1.0 for AMI (range, 0.724-0.879), and 9 of 25 (36%) were penalized. All 23 hospitals graded as “worse” had AMI ERRs greater than 1.0 (range, 1.120 to 1.254), and 100% were penalized.

Penalties as a Function of Number of Conditions With Excess Readmissions

Of the 2591 (82.7%) hospitals that received penalties for overall readmissions, more than half were penalized for having just 1 or 2 of 5 condition-specific ERRs greater than 1.0; 689 (26.6%) had an ERR greater than 1.0 for just 1 condition, and 713 (27.5%) had ERRs greater than 1.0 for just 2 conditions. The largest single ERR driver for penalties was total hip/knee arthroplasty, which accounted for 215 of 689 (31.2%) of the single-condition penalties. For 141 hospitals penalized for readmissions with only an HF ERR greater than 1.0, 140 (99.3%) were graded against the national rate as “no different” for HF readmissions, and 1 (0.7%) was graded as “worse.” There were 82 hospitals that were penalized for excess AMI readmissions only; all 82 were graded as “no different” from the national rate.

DISCUSSION

This study highlights the important differences in hospital performance regarding readmissions as reported to the public in grades on Hospital Compare and communicated to hospitals by financial penalties. Many hospitals are penalized for readmissions despite having publicly reported grades of average performance. The HRRP program has 2 goals: to save Medicare dollars and to motivate care improvement. Readmission reporting and penalty is part of CMS’ goal to move 90% of payments from activity-based to value- and quality-based payment by 2018.¹¹ The HRRP results in payment reductions required by legislation, yet it is unclear how closely the readmission scores are associated with avoidable readmissions. Consumers and hospitals should expect that the application of value-based strategies to our nation’s hospitals be predicated on 3 fundamental assumptions. First, hospitals providing better quality of care to patients should be financially rewarded by not receiving financial penalties. Second, by using tracer conditions like readmissions to proxy “value” in its value-based programs,¹² CMS is able to identify hospital deficiencies modifiable by the hospital. Third, methods used by CMS to identify quality of care are sufficiently robust to send the appropriate “value signal” back through the health system by public grades and financial penalties to motivate care improvement.

The definitions used in public reporting of grades and assigning financial penalties are both driven by the underlying statistical approach of defining poor care, which uses different statistical methods to score hospitals. Both of these methods use a normative-based assessment (ie, grading on the curve) as opposed to criterion-based assessment. Contrasts between normative- and criterion-based assessment have been extensively studied in education for more than 40 years, and the issues raised are somewhat analogous to the methods used here.¹³ Publicly reported grades for readmissions are assigned based on a modification of a normal distribution of performance, as detailed above,^1,5 whereas the risk-adjusted point estimate used for assigning financial penalties is normalized to an expected value. A rule such as this, which assigns a financial penalty to any hospital with a point estimate above the mean of an assumed normal distribution of outcomes, will necessarily result in half of the hospitals being penalized per measure. Because hospitals can be identified as having greater than expected readmissions in 1 or more categories, this approach’s net effect is to label most hospitals as having lower outcomes than expected. In contrast, if the methods for setting the threshold for qualifying for a financial penalty were modified to be similar to the grading criteria on Hospital Compare, in that there was a lower limit of a confidence interval for each hospital estimate to be above the mean before assigning a penalty, then fewer than half of the hospitals would be penalized.

What, then, can CMS do to more directly align its goal of improving value while decreasing cost? The most direct value signal that CMS could provide, which would serve to enhance value and decrease cost, is to stop paying for clinically inappropriate care and pay less for clinically marginal care.¹⁴ This does occur within the Hospital Value-Based Purchasing Program,¹⁵ which incorporates criterion-based assessments (eg, measures of timely and effective care, such as the percentage of patients provided treatment to prevent blood clots on day of or day after admission or surgery), in addition to patient outcomes of hospital-acquired complications and mortality. It is important to balance the selection of process and outcome measures when comparing hospital performance. Comparing by process measures rewards hospitals for providing appropriate care and ensures less impact on hospitals when patient outcomes are poor despite the hospital doing everything right. Comparing by risk-adjusted outcome measures focuses on the most important result for patients while allowing hospitals to try different processes to improve care, particularly when the protocol to improve the outcome is not well established. Similar approaches should also be considered for incorporation in the HRRP to include appropriate process measures in addition to outcome measures. Thus, if a hospital coordinates a home visit after discharge for a high-risk patient, this is evidence of appropriate care—regardless of whether or not this patient eventually requires a readmission. It is critical that clinicians enter the value conversation directly and embrace initiatives such as Choosing Wisely—which seeks to start stratifying the clinical value of care—to inform the most appropriate care. This clinically defined approach needs to be aligned more directly with payment redesign for CMS to truly deliver on its promise to change its payment paradigm from volume to value.

Limitations

The major limitation of this study was that hospitals had to have both grades and penalties reported for at least 1 measure to be included for analysis. However, because CMS manages the HRRP, we presume that hospitals excluded from reporting readmission data for grades and penalties were not subject to the HRRP.

CONCLUSIONS

Hospital grades for readmissions seen by consumers on Hospital Compare are often out of line with excessive readmission scores used to assign readmission penalties to hospitals. Discordant systems for grading and penalizing performance are confusing to consumers and hospitals and highlight persistent uncertainty in how best to identify and link value to payment. Although the legislation requiring financial penalties predetermines the cost reduction to be achieved by requiring penalties for the bottom half of the outcome distribution, the relationship to improved quality is tenuous. Penalizing 81% of hospitals suggests that there is little value in the penalty. This ambiguous value signal, currently evident in the various CMS programs and websites, undermines the value message. 

Acknowledgments

The authors thank Heidi Reichert, MA, for assistance with data analysis.Author Affiliations: Department of Internal Medicine (JM, SNS, TPH, MAMR, LP, LFM), Department of Pediatrics and Communicable Diseases (JM), and Department of Psychiatry (SNS), University of Michigan Medical School, Ann Arbor, MI; Center for Clinical Management Research, Veterans Affairs Ann Arbor Healthcare System (JM, TPH), Ann Arbor, MI; University of Michigan School of Public Health (LFM), Ann Arbor, MI.

Source of Funding: This work was funded by the Agency for Healthcare Research and Quality (2R01HS018334, 1K08HS019767).

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JM, SNS, TPH, LFM); acquisition of data (JM, SNS, LFM); analysis and interpretation of data (JM, SNS, TPH, MAMR, LFM); drafting of the manuscript (JM, SNS, LP); critical revision of the manuscript for important intellectual content (JM, SNS, TPH, MAMR, LP, LFM); statistical analysis (SNS); obtaining funding (JM, LFM); administrative, technical, or logistic support (MAMR, LP, LFM); and supervision (JM).

Address Correspondence to: Jennifer Meddings, MD, MSc, University of Michigan, 2800 Plymouth Rd, Bldg 16, Room 430W, Ann Arbor, MI 48109-2800. Email: meddings@umich.edu.REFERENCES

1. 30-day unplanned readmission and death measures. Medicare.gov website. medicare.gov/hospitalcompare/Data/30-day-measures.html. Accessed March 17, 2017.

2. Readmissions Reduction Program (HRRP). Medicare.gov website. cms.gov/medicare/medicare-fee-for-service-payment/acuteinpatientpps/readmissions-reduction-program.html. Updated September 27, 2018. Accessed March 17, 2017.

3. Fingar K, Washington R. Trends in hospital readmissions for four high-volume conditions, 2009—2013 [HCUP statistical brief no. 196]. Agency for Healthcare Research and Quality website. hcup-us.ahrq.gov/reports/statbriefs/sb196-Readmissions-Trends-High-Volume-Conditions.pdf. Published November 2015. Accessed November 29, 2018.

4. McIlvennan CK, Eapen ZJ, Allen LA. Hospital readmissions reduction program. Circulation. 2015;131(20):1796-1803. doi: 10.1161/CIRCULATIONAHA.114.010270.

5. Infections. Medicare.gov website. medicare.gov/hospitalcompare/Data/Healthcare-Associated-Infections.html. Accessed March 17, 2017.

6. Readmissions complications and death — hospital.csv. Hospital Compare data archive: 2015 annual files. Medicare.gov website. medicare.gov/download/HospitalCompare/2015/January/HOSArchive_Revised_Flatfiles_20150122.zip. Published January 22, 2015. Accessed March 31, 2016.

7. Hospital-acquired condition reduction program. Medicare.gov website. medicare.gov/hospitalcompare/HAC-reduction-program.html. Accessed April 3, 2017.

8. FY 2015 IPPS Final Rule Readmissions PUF-Oct CN.xls. Files for FY 2015 final rule and correction notice. CMS website. cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Downloads/FY2015-FR-Readmit-Supp-Data-File.zip. Accessed April 13, 2015.

9. Fiscal year 2015 results for the CMS Hospital-Acquired Condition Reduction Program and Hospital Value-Based Purchasing Program [news release]. Baltimore, MD: CMS; December 18, 2014. cms.gov/newsroom/fact-sheets/fiscal-year-2015-results-cms-hospital-acquired-condition-reduction-program-and-hospital-value-based. Updated January 22, 2015. Accessed March 17, 2017.

10. Hospital-Acquired Condition Reduction Program (HACRP). CMS website. cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/HAC-Reduction-Program.html. Updated July 30, 2018. Accessed December 2, 2018.

11. Better care. smarter spending. healthier people: paying providers for value, not volume [news release]. Baltimore, MD: CMS; January 26, 2015. cms.gov/newsroom/fact-sheets/better-care-smarter-spending-healthier-people-paying-providers-value-not-volume. Accessed March 17, 2017.

12. CMS’ value-based programs. CMS website. cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/Value-Based-Programs.html. Updated July 25, 2018. Accessed December 2, 2018.

13. Luyten H, Visscher A, Witziers B. School effectiveness research: from a review of the criticism to recommendations for further development. Sch Eff Sch Improv. 2005;16(3):249-279. doi: 10.1080/09243450500114884.

14. McMahon LF Jr, Chopra V. Health care cost and value: the way forward. JAMA. 2012;307(7):671-672. doi: 10.1001/jama.2012.136.

15. The Hospital Value-Based Purchasing (VBP) Program. CMS website. cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/Value-Based-Programs/HVBP/Hospital-Value-Based-Purchasing.html. Updated August 2, 2018. Accessed October 23, 2017.

Download PDF: Mixed Messages to Consumers From Medicare: Hospital Compare Grades Versus Value-Based Payment Penalty