Using a Modified Next Generation ACO Benchmark Can Improve the MSSP

ABSTRACT

Objectives: The current benchmark method used by Medicare Shared Saving Program (MSSP) accountable care organizations (ACOs) favors historically high-cost ACOs. To address this problem, the new Next Generation ACO model modifies the benchmarking methodology by incorporating regional and national relative efficiency discounts. We analyzed the effect of a benchmark on ACO savings under the original benchmark and these 2 adjusted benchmarking techniques.

Study Design: We simulated different benchmarking methodologies and their effects on ACO savings.

Methods: Using 2012 to 2014 Medicare spending data and MSSP ACO performance data, we applied county-level and national-level benchmark adjustments following the Next Generation ACO benchmarking method. We implemented both adjustments simultaneously and varied the adjustment caps between 0% and 5% to observe how the adjustment magnitude would affect savings for each ACO. We computed the average change in the newly calculated saving rate and earned shared savings for each $1000 change of the original benchmark, adjusting for ACO characteristics.

Results: The Next Generation benchmark adjustments can reduce the correlation between ACO earned savings and historical spending, but the proposed adjustment levels are not large enough. By simulating different adjustment levels, we conclude that a combined national and regional discount cap of 3.5% best removes the correlation between benchmark and MSSP ACO performance.

Conclusions: Under the current MSSP, a higher benchmark level is associated with a better chance to save and larger earned savings. Although the Next Generation ACO benchmark adjustment is effective, the adjustment should be larger to completely remove the correlation between prior performance and an ACO’s savings.

Medicare offers 3 main types of accountable care organization (ACO) programs: Pioneer ACOs, initiated in 2011; Medicare Shared Savings Program (MSSP), initiated in 2012; and Next Generation ACOs, initiated in 2016. The great majority of Medicare ACOs are MSSP ACOs; as of 2016, there are 434 active MSSP ACOs serving over 7.7 million Medicare beneficiaries, but only 9 Pioneer ACOs and 21 Next Generation ACOs.¹

MSSP ACOs earn shared savings if the realized spending is less than a required percentage, ranging from 2% to 3.9% of the target spending.² This benchmark, which is established at the start of a 3-year period, adjusts for patient risk scores and increases annually following a national spending trend, but makes no adjustment based on the relative historic spending level. As a result, providers who have achieved cost efficiency may be discouraged from entering or continuing to participate in an ACO payment model due to its reliance on historical performance. Several previous studies show that under the current benchmarking methodology, ACOs that earned more savings were located in high spending regions and had high spending relative to other ACOs in the same region.^3-5

In an attempt to fix this problem, the new Next Generation ACO model modifies the benchmarking methodology by incorporating regional and national relative efficiency discounts, so ACOs that have already attained cost efficiency compared with their region will have a more favorable discount.⁶ The Next Generation ACO also includes a standard discount of 3% and an adjustment based on ACOs’ quality scores. Since the inclusion of the 3% standard discount would not affect the relationship between saving and benchmark, and the quality adjustment has a phase in period in which it is not active, these features are not included in our analysis.

We used 2012 to 2014 Medicare spending data and MSSP ACO performance data to analyze the effect of a benchmark on ACO savings under 3 benchmark schemes: current MSSP benchmark and 2 benchmarks following the Next Generation ACO model, one with regional adjustments and another with national spending adjustments. The light purple bars in the Figure (panel A) show that the higher the current benchmark level, the higher the ACO saving rates. The darker colored bars in the Figure (panel A) show that this positive relationship weakens when incorporating the benchmarking adjustments. However, the regional and national adjustment levels are not large enough to remove the correlation. By simulating different adjustment levels, we concluded that a combined national and regional discount cap of 3.5% best removes the correlation between benchmark and MSSP ACO performance.

METHODS

MSSP ACO public use files, from 2012 to 2014, contain information on ACOs’ spending in the first 2 performance years, as well as characteristics of assigned beneficiaries, including county of residence, Medicare enrollment type, and risk score.^7,8 We combined this data with county-level fee-for-service (FFS) Medicare expenditures to determine each ACO’s spending level relative to the national and region levels. Our final dataset contained 329 ACOs. We dropped ACOs that stopped participating in MSSP and Golden Life Healthcare LLC—an outlier with a 44% saving rate; including this outlier does not substantially change results.

Outcomes are 2 ACO-level saving measures: 1) saving rate, defined as benchmark expenditures minus actual expenditures as a percentage of benchmark expenditures; and 2) earned shared savings per capita, defined as per-capita shared savings for ACOs that simultaneously exceeded their minimum saving rate and met the program’s quality standard.

Following the Next Generation ACO benchmarking method, we create 2 adjusted benchmarks: 1) county-level efficiency adjustment and 2) national efficiency adjustment. Specifically, if an ACO’s risk-adjusted historical per-capita baseline is higher (or lower) than the risk-adjusted county-level FFS baseline, the ACO’s county-adjusted benchmark is equal to the original benchmark decreased (or increased) up to 1%. Similarly, if an ACO’s historical baseline is higher (or lower) than the national level, the national adjusted benchmark is equal to the original benchmark decreased (or increased) up to 0.5%. We separately analyzed whether either adjustment can remove the relationship between historical spending and savings. Separating regional and national adjustments allowed us to compare them explicitly.

We then implemented both adjustments simultaneously, as in the Next Generation ACO model, and varied the discount caps between 0% and 5% to observe how the adjustment magnitude affects savings for each ACO. We computed the average change in the newly calculated saving rate and earned shared savings per capita for each $1000 change of the original benchmark, adjusting for ACO characteristics, including number of beneficiaries served in the ACO, average county spending level, the ACO’s patient demographic and enrollment characteristics, starting date indicators, and year indicators. Through this simulation, we determined the discount levels that best remove the positive relationship between benchmark and performance.

RESULTS

Higher Benchmark, Higher Savings

The purple bars in the Figure show the average saving rates of all ACOs and average earned shared savings per capita among ACOs that achieved saving by decile of the original benchmark amount. The higher the benchmark level, the larger the savings, confirming that the level of the benchmark has a large effect on both the extensive margin (whether or not an ACO saves) and the intensive margin (how much an ACO saves).

Regional and National Efficiency Adjustments Are Somewhat Effective

We recalculated the saving rate and earned shared savings per capita for each ACO under the 2 adjusted benchmarks. The first adjusted benchmark takes into account only the regional efficiency adjustment, while the second benchmark takes into account only the national efficiency adjustment. The Figure (panel A) shows the average saving rate by decile of the original benchmark when the saving rate is calculated using the original and the 2 adjusted benchmarks. The Figure (panel B) presents the relationship between these benchmarks and earned shared savings per capita. Both adjustments slightly mitigate the correlation between the original benchmark and the recalculated outcomes. The regional efficiency adjustment is more effective than the national efficiency adjustment, but this seems to be caused by the larger magnitude of the regional adjustment. Under both adjusted benchmarks, the positive relationships between the recalculated outcomes and the original benchmark are still present.

The Adjustments Should Be Larger

Controlling for ACO characteristics, the correlation between the original benchmark and the saving rate was 0.14 (P = .00). After incorporating the national and regional adjustments, the correlations decreased to 0.13 (P = .00) and 0.10 (P = .02), respectively. All the evidence suggests that the adjustment level currently being used in the Next Generation ACO model is effective, but not large enough to remove the correlation between benchmark and ACO performance.

To see what level of adjustment best removes the correlation, we varied the discount caps of these 2 adjustments and displayed the average change in the newly calculated savings rate for each $1000 change of the original benchmark in the Table (panel A) and newly calculated earned savings per capita in the Table (panel B). In the Table, both adjustments are implemented simultaneously. The darker cells correspond to scenarios with a stronger and more statistically significant relationship between recalculated outcomes and benchmarks, while the whiter cells indicate a weaker relationship with less statistical significance. At the current Next Generation ACO implementation level—1% cap for regional adjustment and 0.5% cap for national adjustment—an increase of $1000 in the original benchmark is related to a 0.52% increase in the savings rate and a $19.17 increase in earned savings per capita. This is not trivial, considering the average savings rate is 0.54% and the average shared savings per capita is $79.80 in our sample.

The analysis reveals that a combined national and regional discount cap of 3.5% best removes the correlation between benchmark and ACO performance under an ACO model. At this level, the absolute value of correlation is the smallest and is statistically insignificant. If the combined cap is too small, the program rewards ACOs with high benchmarks; if the combined cap is too large, the relationship becomes negative and saving becomes harder to achieve for high-benchmark ACOs. Finally, as shown from the diagonal shape of the white cells in the Table, both adjustments have similar effects on the correlation, and the combined adjustment level is more important than the relative size of each adjustment.

DISCUSSION

The benchmark calculation can greatly influence realized savings for ACOs. The current MSSP benchmark is set to be ACO-specific and is based on the prior 3 years’ performance in order to encourage participation from high-cost ACOs.

It is essential to have a program that gives all ACOs a fair chance to succeed. Under the current benchmark system, the incentives to join MSSP are weaker for ACOs with a lower benchmark, as it is harder for them to achieve savings. Under a 2-sided risk model, which MSSP was originally scheduled to transition to, the incentive to participate in an ACO program is even weaker because ACOs face a larger risk. Moreover, the current MSSP benchmark is scheduled to be recalculated every 3 years, so previous high-cost, high-performance providers will eventually have a low benchmark. Before the recalculation occurs, attrition may be the best choice for ACOs that would receive a low updated benchmark.

Our study explores 2 benchmark adjustments that are implemented in the Next Generation ACO model and designed to remove the relationship between the benchmark level and an ACO’s success. The regional efficiency adjustment makes savings harder to achieve for ACOs that are performing poorly in their region. Similarly, the national efficiency adjustment makes savings harder to achieve for ACOs that spent more than the national spending average in the current performance year, lowering the correlation between high-cost regions and MSSP success.

At the current Next Generation ACO adjustment level, both adjustments can partially remove the relationship between an ACO’s spending level and the realized savings. However, a stronger adjustment is needed to completely remove the correlation. Our analysis revealed that the level of the combined national and regional discount cap is more important and the respective level of each adjustment does not greatly affect results. It is not surprising that regional and national efficiency adjustments have similar effects, given that more than 80% of the ACOs are affected by these 2 adjustments in the same way: if an ACO spent more than the regional average, it is also more likely to have spent more than the national average. We recommend implementing a benchmark adjustment with a combined cap of 3.5%. At this level, the positive relationship between historical spending and savings is completely removed, making savings equally attainable for all ACOs, independent of their previous spending level.

CONCLUSIONS

In summary, the MSSP benchmark is calculated based on the ACO’s previous 3 years’ claims data, making savings hard to achieve for historically low-cost ACOs. We note that a higher benchmark level is associated with a better chance to save and larger earned savings per capita. We also show, when adjusting benchmarks using the adjustment methods in the Next Generation ACO model, that the relationship between benchmark and saving is weakened; however, a benchmark is still an important determinant of saving. We tested different levels of regional efficiency and national efficiency adjustments and determined that a combined adjustment cap of 3.5% best removes the positive relationship between benchmark and ACO performance. Although the Next Generation ACO benchmark adjustment is effective at reducing the correlation between prior poor performance and an ACO’s savings, the adjustment should be larger to completely remove the positive relationship.

Acknowledgments

Lulu Liu, MA, and Richard Svoboda, MA, made equal contributions to the article.

Author Affiliations: Department of Economics (LL, RS) and Department of Health Policy and Management (YZ), University of Pittsburgh, Pittsburgh, PA.

Source of Funding: Yuting Zhang gratefully acknowledges funding from the Commonwealth Fund (Grant 20150380). The sponsors played no role in the study conduct, data analysis, or report.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (LL, RS, YZ); acquisition of data (LL, RS, YZ); analysis and interpretation of data (LL, RS, YZ); drafting of the manuscript (LL, RS, YZ); critical revision of the manuscript for important intellectual content (LL, RS, YZ); statistical analysis (LL, RS); obtaining funding (YZ); administrative, technical, or logistic support (LL, RS, YZ); and supervision (YZ).

Send Correspondence to: Yuting Zhang, PhD, Department of Health Policy and Management, University of Pittsburgh, 130 De Soto St, Crabtree Hall A664, Pittsburgh, PA 15261. E-mail:

ytzhang@pitt.edu.

REFERENCES

1. Next Generation Accountable Care Organization Model. CMS website. https://www.cms.gov/Newsroom/MediaReleaseDatabase/Fact-sheets/2016-Fact-sheets-items/2016-01-11.html. Published November 1, 2016. Accessed June 1, 2016.

2. Methodology for determining shared savings and losses under the Medicare Shared Savings Program. CMS website. https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/ACO_Methodology_Factsheet_ICN907405.pdf. Published April 2014. Accessed June 1, 2016.

3. Heiser S, Colla C, Fisher E. Unpacking the Medicare shared savings proposed rule: geography and policy. Health Affairs website. http://healthaffairs.org/blog/2015/01/22/unpacking-the-medicare-shared-savings-proposed-rule-geography-and-policy/. Published January 22, 2015. Accessed May 1, 2016.

4. McClellan M, Kocot SL, White R. Early evidence on Medicare ACOs and next steps for the Medicare ACO program (updated). Health Affairs website. http://healthaffairs.org/blog/2015/01/22/early-evidence-on-medicare-acos-and-next-steps-for-the-medicare-aco-program/. Published January 22, 2015. Accessed August 25, 2016.

5. McWilliams JM, Song Z. Implications for ACOs of variations in spending growth. N Engl J Med. 2012;366(19):e29. doi: 10.1056/NEJMp1202004.

6. Next Generation ACO model benchmarking methods. CMS website. https://innovation.cms.gov/Files/x/nextgenaco-methodology.pdf. Published December 15, 2015. Accessed June 1, 2016.

7. Shared savings program accountable care organizations (ACO) PUF. CMS website. https://www.cms.gov/Research-Statistics-Data-and-Systems/Downloadable-Public-Use-Files/SSPACO/index.html. Updated March 8, 2016. Accessed June 1, 2016.

8. Aggregate expenditure and risk score data on assignable beneficiaries by county. CMS website. https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/Assignable-Beneficiaries-by-County-for-CMS-1644-P.zip. Updated June 29, 2016. Accessed August 25, 2016.

Download PDF: Using a Modified Next Generation ACO Benchmark Can Improve the MSSP