ACO Quality Over Time: The MSSP Experience and Opportunities for System-Wide Improvement

ABSTRACT

Objectives: To investigate accountable care organization (ACO) quality improvement over the first 4 Medicare Shared Savings Program (MSSP) years.

Study Design: Fixed-effects analysis examined associations of within-ACO MSSP quality metric changes with key time-variant ACO traits: changes in postacute care (PAC) expenditure and size (attributed beneficiaries). Fixed-effects subgroup analyses and linear regression were used for key time-invariant traits: ACO taxonomy (physician-led, hospital-led, or co-led), risk-bearing maturity, commercial contract presence, and rurality.

Methods: The sources of data were secondary MSSP public use files linked to the Leavitt Partners ACO Database (ACO panel: n = 528; 2013-2016).

Results: Confirming early federal findings, MSSP ACOs, on average, improved most quality measures. Larger ACOs had higher quality, but ACOs grew rapidly for the first 3 years, bringing “growing pains” in quality measures related to clinical care for at-risk populations, before plateauing in size in the fourth year. By comparison, PAC expenditures increased in the first year but then decreased in all remaining years, and PAC spending changes were inversely associated with quality, especially in quality measures related to care coordination and patient safety. Successes and challenges varied most notably by ACO taxonomy, risk-bearing maturity, and rurality.

Conclusions: MSSP ACOs improved quality despite their sicker, older population, suggesting that the model might work in other settings and populations and could shift to more advanced risk and payment models (eg, population-based prospective payment). Continued ACO infrastructure development funding, better relationships with PAC facilities, and opportunities for diverse ACOs to share their learnings would maximize quality improvement.

The American Journal of Accountable Care. 2018;6(1):e1-e15Accountable care organizations (ACOs) receive shared savings by establishing a local healthcare delivery collaboration to coordinate across the full care continuum while improving care quality and reducing care costs below a specified benchmark.¹ They are a promising approach to address care fragmentation and achieve the Triple Aim in the US health system.^2-6

Increasingly widespread, ACOs cover about 10% of the US population.⁷ Congress established the Medicare Shared Savings Program (MSSP) as 1 of several ACO programs implemented by CMS to incorporate ACOs into Medicare.⁸ As of early 2018, there are 561 MSSP ACOs representing 10.5 million lives,⁹ making it the largest of Medicare’s ACO programs. CMS tracks MSSP ACO performance and reports that these ACOs have achieved reductions in cost while improving, and outperforming Medicare fee-for-service (FFS) providers on, most quality measures.^10,11

Given the widespread nature of this delivery model, there is a need for evidence of its impact on healthcare quality. Longitudinal examinations of ACO quality have found that ACOs were associated with reduced utilization of postacute care (PAC), length of skilled nursing facility (SNF) stays,¹² mortality,¹³ readmissions,^14,15 and chronic obstructive pulmonary disease or asthma admissions¹⁶, and improved patient experience,¹⁷ chronic disease management, and preventive and pediatric care, especially among populations with low socioeconomic status.^18,19 Cross-sectional study findings further indicate that beneficiary race/ethnicity,²⁰ ACO size and leadership style,²¹ provider mix,²² and urban/rural county²³ are associated with quality.

Existing longitudinal literature focuses on comparing the ACO model with a counterfactual of other care models (namely, FFS), which is valuable and needed evidence given that the literature is still young and mixed. By comparison, given the expansion and prevalence of the MSSP, we examined key factors affecting within-ACO quality improvement in the program (ie, the counterfactual is an ACO compared with itself at an earlier time; we did not compare ACOs with non-ACOs), an area needing better evidence,²¹ so that we may understand how ACOs function maximally within the MSSP to improve quality, a key program goal.

METHODS

Data Sources and Study Population

We utilized 2 data sources. First, we used all publicly available MSSP ACO data files, containing quality scores, financial performance, and descriptive data on ACOs, their providers, and beneficiaries.^24-27 Performance year 1 contains 220 ACOs from when they began (at certain points in 2012 or on January 1, 2013) through the end of 2013, and performance years 2, 3, and 4 contain 333, 392, and 432 ACOs, respectively, during calendar years 2014, 2015, and 2016. Second, we used proprietary data on ACO taxonomy type, geography, and risk-bearing arrangements from the Leavitt Partners ACO Database, which has tracked ACOs since 2010 using news releases, public reports, industry directories, surveys, and interviews.^15,28-30 These were merged by MSSP ACO identifiers to construct a panel of ACOs from 2013 to 2016.

Measures

The objective was to examine how quality metrics changed within ACOs over time. This is represented by MSSP ACO quality performance metrics across 4 quality domains: patient/caregiver experience, care coordination/patient safety, clinical care for at-risk populations, and preventive health. Across 2013-2016, there were 44 measures (42 single measures plus 2 composites) that came from claims, patient survey, or ACO-reported data.³¹ In any given year, there were 33 to 34 single measures (some changed or were replaced over time). After careful review of measure definitions (see Table 1), in regression analyses we omitted certain years of measures before or after significant specification changes in a way that maximized data-years included but avoided spurious findings, a potential issue with the federal government’s early MSSP quality examinations.

We were primarily interested in 6 key independent variables related to ACO structure, function, relationships, and geography discussed or hypothesized in the literature as important to ACO care quality and success: 1) taxonomy (primarily led by a hospital, physicians, or both),^32,33 2) percentile of risk-bearing maturity compared with other ACOs, 3) presence of a commercial contract,³⁴ 4) expenditures on PAC,^12,32,35,36 5) size (attributed beneficiaries), and 6) rurality.²³ These were drawn or derived from the Leavitt Partners ACO Database (except size and PAC expenditures, which are publicly available).

Other variables can confound these relationships, so we included the following as covariates (ACO-level): beneficiaries’ age, sex, dual-eligible status, and Hierarchical Condition Category risk scores (to attempt case-mix adjustment); provider mix; patient-to-provider ratio; per capita benchmark; and market context (ratio of market cost relative to national cost).

Statistical Analyses

First, we used linear regression to examine cross-sectional adjusted associations between average quality performance and covariates (between-ACO associations), and then we used fixed-effects linear regression to longitudinally examine quality improvement (within­ACO associations). Fixed-effects methodology mechanically controls for time-invariant traits (measured or not).³⁷ We regressed each quality measure onto all time-variant independent variables, covariates, and fixed-effects dummies for ACO and year. Only 2 key independent variables, PAC expenditures and size, had time-variant data. However, we incorporated key time-invariant traits by examining how significant PAC and size coefficients from fixed-effects models differ when stratified by ACO taxonomy, commercial contract presence, rurality, and risk-bearing maturity (“subgroup analyses”). All analyses were conducted in Stata/SE 15.1 (StataCorp LP; College Station, Texas) using heteroskedasticity-robust standard errors.³⁷

RESULTS

Table 2 presents descriptive statistics of quality metrics and independent variables by year and change scores (ie, differences in scores) between years. Almost all quality measures improved across the 4 years, except that roughly half of the patient/caregiver experience measures showed small decreases until year 4, when they increased. As many quality measures are assessed on different scales, our analysis also calculated relative change scores to mimic early CMS methods.³⁸ We found similar but more conservative results than did early federal government reports,^10,11 likely due to our study only focusing on trends involving no measure definition changes. The largest, most consistent quality improvements were in preventive health. Larger improvements were notable in heart failure admission rates, screening for fall risk, pneumonia vaccination, and screening/follow-up for clinical depression.

Table 3 shows key independent variable coefficients from linear regressions of each quality measure onto all covariates (comparison groups: ACO vs other ACOs). Table 4 shows statistically significant key time-variant independent variable coefficients (PAC expenditures change and ACO size change) from fixed-effects regressions of each quality measure with 2 or more years of data onto all tim­e-variant coefficients and subgroup analyses, decomposing these coefficients by key time-invariant independent variables (comparison groups: ACO vs only itself at different times). Below, we summarize key stories from these tables.

First, cross-sectionally, there were significant positive associations between size and quality, particularly in clinical care for at-risk populations (Table 3). However, within-ACO change in size was inversely associated with quality changes, also notably among clinical care for at-risk populations (Table 4). Subgroup analyses showed that this inverse association varies substantially by ACO taxonomy and maturity, the highest magnitude being among physician-led ACOs and those less experienced in terms of program time and risk-bearing maturity. Figure 1 provides an illustrative example using 2013-2014 change scores in the coronary artery disease composite: Although most ACOs succeeded in improving quality, ACOs in the highest quartile of size change (+2289 to +40,091 beneficiaries) had much smaller improvements than those in the lowest quartile (—8209 to –294), varying substantially by ACO taxonomy and maturity.

Second, there is a clear relationship between PAC expenditures and quality. Cross-sectionally, there are consistent inverse associations between PAC expenditures and most quality measures, except for patient/caregiver experience measures (Table 3). Within-ACO changes in PAC expenditures had inverse associations with changes in care coordination/patient safety, notably all measures related to unplanned hospital admissions and readmissions (Table 4). Subgroup analyses showed this inverse association to generally be mitigated among physician-led ACOs and ACOs more experienced in terms of time in the program and risk-bearing maturity. Figure 2

provides an illustrative example using 2014-2015 change scores in all-cause readmissions: Although most ACOs decreased readmissions, ACOs in the highest quartile of PAC expenditure change (+0.5% to +10.5%) had much smaller decreases than those in the lowest quartile (—10.4% to –1.5%), varying substantially by ACO taxonomy and maturity.

The presence of commercial contracts or rural settings showed inconsistent findings in subgroup analyses, but we examined them and other time-invariant traits in cross-sectional results (Table 3). First, hospital-led ACOs had higher average quality in many measures of preventive care and clinical care for at-risk populations, although provider-led ACOs often had higher patient experience scores. Second, risk-bearing maturity had a consistent positive association with quality. Last, ACO rurality was generally associated with better quality, except in care coordination and patient safety.

DISCUSSION

This study examined in depth the first 4 years of MSSP data, finding 4 main quality improvement conclusions: 1) ACO quality appears to be broadly improving, highlighting the potential success of the MSSP model; 2) although ACO size was positively correlated with quality, ACOs experienced some quality challenges while growing; 3) PAC expenditures increased then decreased, which was associated with quality changes; and 4) these findings vary by key organizational traits. We discuss these findings below in the context of theory and prior work, then conclude with implications for practice, policy, and quality improvement in the broader healthcare system.

First, our findings generally mirror CMS’ broad conclusion that MSSP ACOs have improved most quality measures (with largest improvements in screening for falls risk, pneumonia vaccinations, and screening/follow-up for depression and blood pressure).^10,11 Additionally, we found large relative improvement in preventable heart failure admissions. Why the largest gains were seen in these measures, notably preventive health, is not entirely clear, although others have found ACOs to outperform non-ACOs in preventive care disparities^18,19 and the ACO model is designed to improve care coordination¹ (many preventive care measures require coordinated screenings). Lastly, the very small decreases in patient experience (likely clinically insignificant) observed in the first 3 years may be due to high average scores (topped-off measures), reinforcing prior calls to develop cross-cutting patient-reported outcomes that better reflect patient experience. Regardless, others have found that ACOs outperform non-ACOs in patient experience measures that are thought to be within a provider’s ability to control,¹⁷ and we found patient experience measures to improve in year 4.

Second, MSSP ACOs experienced large growth in beneficiaries in the first 3 years of the program (plateauing in the fourth), which was associated with quality. Larger size in any given year was associated with higher quality in more than one-fifth of measures (nonsignificant measures trended similarly), most in clinical care for at-risk populations, likely via larger economies of scale and more well-developed infrastructure and referral networks to handle complex patients. The process of ACO growth, however, was negatively associated with clinical care for at-risk populations. This could be explained by multiple startup cost mechanisms, including increased demand for resources to engage, attribute, and manage new beneficiaries; increased beneficiary-to-provider ratio; and any provider consolidation that growth may bring. The magnitude of this effect was generally largest among ACOs led by physicians and those with less program experience and risk bearing, potentially due to smaller average size (fewer economies of scale and less ability to absorb startup costs).

Third, the average ACO experienced increased initial PAC expenditures, presenting quality improvement challenges, but was able to later reduce these expenditures. Changes in PAC expenditures in any given year were associated with inverse changes in quality for two-thirds of measures. Within-ACO changes in PAC spending tended to affect preventable admissions and all-cause readmissions. ACO affiliation has been linked to reduced PAC utilization relative to non-ACOs,¹² but nonetheless, PAC is a large source of cost variation, and close coordinated care and referral partnerships with SNFs and other PAC facilities have been flagged as a potential locus for ACO quality improvement and cost reduction.^12,32,35,36 From subgroup analyses, we suggest that this opportunity may be the biggest for hospital-led ACOs.

Last, although limited to cross-sectional analyses, we gleaned insight into how ACO taxonomy, risk-bearing maturity, and rurality are associated with average quality. First, hospital-led ACOs performed better on measures of preventive health and clinical care for at-risk populations, perhaps because many require on-site screening and specialists to which smaller provider-led ACOs may not have direct access or because hospital-led ACOs may have more formal quality control programs reminding providers of preventive health requirements. We found that provider-led ACOs often had higher patient/caregiver experience scores. Although the reasons for this are unclear, provider-led ACOs had the smallest median size, and smaller practices are theorized to create a more personal setting that patients may prefer.³⁹ Second, higher quality was consistently associated with ACOs that were more mature in terms of total contracts, program time, and risk bearing. Although this finding is likely partly influenced by survivorship of more advanced or experienced ACOs, it also could indicate the impact that having more advanced contract arrangements has on quality. ACOs with 1 or more commercial contracts are documented as having higher quality than those with only public contracts, which may be due to greater use of disease-monitoring tools, patient satisfaction data, and quality improvement activities, and larger provider compensation mechanisms for improved quality,³⁴ although we observed that commercial contracts alone did not generally affect quality. Last, rural ACOs often had higher quality, which early evidence supports,²³ although underlying mechanisms are unclear. This finding is promising, considering the unique challenges faced by rural ACOs, including a smaller number of covered lives spread out over large geographies (which limits ability to absorb cost variation) and infrastructure and data analytic capabilities less equipped to effectively implement population health management.²³ Additional research is needed.

Limitations

These findings must be interpreted within this study’s limitations. First, our analyses did not compare ACOs with non-ACOs but instead compared ACOs with either other ACOs or with themselves at different time periods within the MSSP. Thus, our findings cannot make assertions about how quality functions in the ACO model versus other models, such as FFS. Instead, given the increasingly widespread prevalence of accountable care models over the last several years, our goal was to examine how quality changed for organizations within the MSSP over time in order to understand how to capitalize on successes and avoid challenges moving forward.

Second, although incomplete quality reporting occurs in ACOs, the MSSP tracks and strongly disincentivizes incomplete quality reporting by withholding shared savings for ACOs that do not meet the quality reporting standard. We examined this in a sensitivity analysis and found that it was very uncommon for ACOs to fail to meet this standard and that failure became increasingly rare in each subsequent program year, so we suspect minimal bias here.

Third, bias could be introduced by virtue of which ACOs are likely to enter and exit the program, although we performed a sensitivity survival analysis and found no association between quality scores and program survival.

Fourth, although we cannot definitively assert the causal direction of our main findings, changes in our key time-variant variables (ACO size and PAC expenditures) more likely precede changes in quality and not vice versa. Regarding growth, we examined data over the first 4 years of the MSSP when ACOs were ramping up (plateauing in year 4). PAC expenditure presents possible endogeneity given that some quality measures are related to PAC (eg, readmissions), although many have argued that PAC precedes costs and patient outcomes, including recent work using instrumental variable analysis to circumvent selection bias and directionality issues.⁴⁰ Nonetheless, further research is needed into how and why ACOs are growing and changing PAC expenditures and how these changes subsequently affect quality. Finally, although time-invariant confounding is ruled out in fixed-effects models, and in all models we have taken care to include both time-variant and time-invariant controls known or hypothesized to affect quality, we cannot definitively rule out confounding.

CONCLUSIONS

Most MSSP ACOs improved most quality measures over the first 4 years of the MSSP. The MSSP is a prevalent and expanding program that is, on average, achieving its quality improvement goals. Further, our companion study found that MSSP ACOs are achieving savings by shifting expenditures away from costly inpatient and long-term services to primary care provided in physician offices.⁴¹ That MSSP ACOs serve the primarily older and more chronically ill Medicare population indicates a success, suggesting that the MSSP model would likely be successful if expanded to other settings (public and private). Given this quality improvement success, the MSSP could more strongly incentivize more advanced risk­‑sharing (beyond 1-sided, moving to MSSP “Track 3”). Moreover, the finding that most ACOs improved quality shows how much room for improvement there is: Although the shared savings model offers major improvements over traditional/previous FFS models, it is still an alternative payment model (APM) built and dependent on FFS architecture. Continuing to move down the advanced APM path toward population-based payments (ie, care is prospectively contained within a single payment over a fixed time, linked to quality, value, and patient-centeredness) may further maximize value-

based healthcare.⁴²

There exist clear opportunities for ACO quality to further improve. First, ACOs experienced tremendous growth in the first 4 years of the MSSP, and although larger size was associated with higher average quality, growth presented minor quality challenges in caring for at-risk patients; in other words, growing cautiously as resources permit, not aggressively, may be a good strategy for ACOs. Further, concerns over economies of scale and administrative burden associated with becoming or expanding an ACO could accelerate provider consolidation, previously raised as an unintended consequence of recent health reform. Consolidation can increase shared resources and care coordination, but it has been associated with increased prices without improved quality due to reduced market competitiveness.^43-45 The number of ACOs continues to grow, and ACO size is just beginning to plateau, so continued public and private funding for ACO infrastructure development will be beneficial to maintaining quality gains and minimizing potential negative consequences, especially for ACOs serving rural or underserved populations (eg, the ACO Investment Model⁴⁶). Second, developing stronger, more coordinated partnerships with SNFs and other PAC facilities (bringing them more formally into ACO networks) likely would improve care quality and cost. Finally, ACOs could benefit from capitalizing on their diversity (quality challenges and strengths varied by ACO leadership type, risk­-bearing ability, and rurality), presenting shared learning opportunities.

Taking advantage of these opportunities could maximize ACO model expansion and quality improvement nationwide. More work still needs to be done, however, to better understand how and why the above trends, findings, and implications play out in ACOs and to share learnings with the broader system, as ACOs, advanced APMs, and the health system as a whole evolve.

Acknowledgments

The authors thank Lia Winfield, PhD, of Leavitt Partners, for her assistance in reviewing the submitted manuscript.

Dr Pham’s work on this manuscript was done while she was a Senior Policy Fellow at Duke University’s Robert J. Margolis, MD, Center for Health Policy (Washington, DC).Author Affiliations: Robert J. Margolis, MD, Center for Health Policy, Duke University (WKB, RSS, MBM), Washington, DC; Leavitt Partners (DBM), Washington, DC; Leavitt Partners (SQM), Salt Lake City, UT; Anthem, Inc (HHP), Washington, DC.

Source of Funding: Support for this research was provided by Duke University’s Robert J. Margolis, MD, Center for Health Policy, and Leavitt Partners.

Prior Presentation: An earlier version of this analysis was presented on June 27, 2017, as part of the “Quality, Spending, and Long-Term Viability of Accountable Care Organizations” panel at the AcademyHealth Annual Research Meeting in New Orleans, LA.

Author Disclosures: Dr Bleser, Dr Saunders, and Dr McClellan are employed by Duke University, which has an ACO that was not involved with this research, nor did it fund the research in this paper. They have attended meetings of the Health Care Payment Learning and Action Network and the Accountable Care Learning Collaborative. They have a grant unrelated to this manuscript from the Gordon and Betty Moore Foundation to study ACOs. Dr Muhlestein and Mr Morrison are employed by Leavitt Partners, which consults about ACOs and works with providers and ACOs. Dr Muhlestein has given speeches that included ACO research, some of which provided travel expenses or small honoraria, and has attended multiple ACO conferences. He has received grants from the Commonweath Fund and the Moore Foundation for work on ACOs. Dr McClellan is co-chair of the Health Care Payment Learning and Action Network and co-chair of the Accountable Care Learning Collaborative, and he has a grant unrelated to this manuscript from the Commonwealth Fund to study global best practices in accountable care. The remaining author reports no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (WKB, RSS, DBM, SQM, HHP, MBM); acquisition of data (WKB, RSS, DBM, SQM); analysis and interpretation of data (WKB, RSS, DBM, SQM, HHP, MBM); drafting of the manuscript (WKB, RSS, DBM, SQM); critical revision of the manuscript for important intellectual content (WKB, RSS, DBM, SQM, HHP, MBM); statistical analysis (WKB, RSS, DBM, SQM); administrative, technical, or logistic support (WKB, RSS); and supervision (RSS).

Send Correspondence to: William K. Bleser, PhD, MSPH, Robert J. Margolis, MD, Center for Health Policy, Duke University, 1201 Pennsylvania Ave NW, 5th Fl, Washington, DC 20004. Email: william.bleser@duke.edu.REFERENCES

1. Lowell KH, Bertko J. The Accountable Care Organization (ACO) model: building blocks for success. J Ambul Care Manage. 2010;33(1):81-88. doi: 10.1097/JAC.0b013e3181c9fb12.

2. Institute of Medicine. The Healthcare Imperative: Lowering Costs and Improving Outcomes: Workshop Series Summary. Washington, DC: The National Academies Press; 2010. nap.edu/catalog/12750/the-healthcare-imperative-lowering-costs-and-improving-outcomes-workshop-series. Accessed February 22, 2017.

3. Shortell SM, Colla CH, Lewis VA, Fisher E, Kessell E, Ramsay P. Accountable care organizations: the national landscape. J Health Polit Policy Law. 2015;40(4):647-668. doi: 10.1215/03616878-3149976.

4. Fisher ES, McClellan MB, Bertko J, et al. Fostering accountable health care: moving forward in Medicare. Health Aff (Millwood). 2009;28(2):w219-w231. doi: 10.1377/hlthaff.28.2.w219.

5. McClellan M, McKethan AN, Lewis JL, Roski J, Fisher ES. A national strategy to put accountable care into practice. Health Aff (Millwood). 2010;29(5):982-990. doi: 10.1377/hlthaff.2010.0194.

6. Fisher ES, Staiger DO, Bynum JPW, Gottlieb DJ. Creating accountable care organizations: the extended hospital medical staff. Health Aff (Millwood). 2007;26(1):w44-w57. doi: 10.1377/hlthaff.26.1.w44.

7. Muhlestein D, Saunders R, McClellan M. Growth of ACOs and alternative payment models in 2017. Health Affairs Blog. June 28, 2017. healthaffairs.org/blog/2017/06/28/growth-of-acos-and-alternative-payment-models-in-2017. Accessed June 29, 2017.

8. Accountable Care Organizations (ACO). CMS website. cms.gov/Medicare/Medicare-Fee-for-Service-Payment/ACO/index.html. Updated May 12, 2017. Accessed February 12, 2018.

9. Medicare Shared Savings Program fast facts. CMS website. cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/SSP-2018-Fast-Facts.pdf. Published January 2018. Accessed January 19, 2018.

10. Medicare Accountable Care Organizations 2015 performance year quality and financial results [news release]. Baltimore, MD: CMS; August 25, 2016. cms.gov/newsroom/mediareleasedatabase/fact-sheets/2016-fact-sheets-items/2016-08-25.html. Accessed February 22, 2017.

11. Office of Inspector General, HHS. Medicare Shared Savings Program Accountable Care Organizations Have Shown Potential for Reducing Spending and Improving Quality. HHS website. oig.hhs.gov/oei/reports/oei-02-15-00450.pdf. Published August 2017. Accessed January 19, 2018.

12. McWilliams JM, Gilstrap LG, Stevenson DG, Chernew ME, Huskamp HA, Grabowski DC. Changes in postacute care in the Medicare Shared Savings Program. JAMA Intern Med. 2017;177(4):518-526. doi: 10.1001/jamainternmed.2016.9115.

13. Chukmaitov A, Harless DW, Bazzoli GJ, Carretta HJ, Siangphoe U. Delivery system characteristics and their association with quality and costs of care: implications for accountable care organizations. Health Care Manage Rev. 2015;40(2):92-103. doi: 10.1097/HMR.0000000000000014.

14. Winblad U, Mor V, McHugh JP, Rahman M. ACO-affiliated hospitals reduced rehospitalizations from skilled nursing facilities faster than other hospitals. Health Aff (Millwood). 2017;36(1):67-73. doi: 10.1377/hlthaff.2016.0759.

15. Muhlestein D, Tu T, de Lisle K, Merrill T. Hospital participation in ACOs associated with other value-based program improvement. Am J Manag Care. 2016;22(7):e241-e248.

16. McWilliams JM, Hatfield LA, Chernew ME, Landon BE, Schwartz AL. Early performance of accountable care organizations in Medicare. N Engl J Med. 2016;374(24):2357-2366. doi: 10.1056/NEJMsa1600142.

17. McWilliams JM, Landon BE, Chernew ME, Zaslavsky AM. Changes in patients’ experiences in Medicare Accountable Care Organizations. N Engl J Med. 2014;371(18):1715-1724. doi: 10.1056/NEJMsa1406552.

18. Song Z, Rose S, Chernew ME, Safran DG. Lower- versus higher-income populations in the Alternative Quality Contract: improved quality and similar spending. Health Aff (Millwood). 2017;36(1):74-82. doi: 10.1377/hlthaff.2016.0682.

19. Song Z, Rose S, Safran DG, Landon BE, Day MP, Chernew ME. Changes in health care spending and quality 4 years into global payment. N Engl J Med. 2014;371(18):1704-1714. doi: 10.1056/NEJMsa1404026.

20. Lewis VA, Fraze T, Fisher ES, Shortell SM, Colla CH. ACOs serving high proportions of racial and ethnic minorities lag in quality performance. Health Aff (Millwood). 2017;36(1):57-66. doi: 10.1377/hlthaff.2016.0626.

21. Muhlestein D, Saunders R, McClellan M. Medicare Accountable Care Organization results for 2015: the journey to better quality and lower costs continues. Health Affairs Blog. September 9, 2016. healthaffairs.org/blog/2016/09/09/medicare-accountable-care-organization-results-for-2015-the-journey-to-better-quality-and-lower-costs-continues. Accessed February 20, 2017.

22. Albright BB, Lewis VA, Ross JS, Colla CH. Preventive care quality of Medicare Accountable Care Organizations: associations of organizational characteristics with performance. Med Care. 2016;54(3):326-335. doi: 10.1097/MLR.0000000000000477.

23. Zhu X, Mueller K, Vaughn T, Ullrich F, MacKinney AC. Medicare Accountable Care Organizations: quality performance by geographic categories. Rural Policy Brief. 2016;(6):1-4.

24. 2013 Shared Savings Program Accountable Care Organizations (ACO) PUF. CMS website. data.cms.gov/Special-Programs-Initiatives-Accountable-Care-Orga/2013-Shared-Savings-Program-Accountable-Care-Organ/faep-t7cf. Published November 14, 2016. Accessed February 23, 2017. Metadata updated September 21, 2017.

25. 2014 Shared Savings Program (SSP) Accountable Care Organizations (ACO) PUF. CMS website. data.cms.gov/Special-Programs-Initiatives-Medicare-Shared-Savin/2014-Shared-Savings-Program-SSP-Accountable-Care-O/888h-akbg. Published October 30, 2015. Updated November 19, 2015. Accessed February 23, 2017. Metadata updated September 21, 2017.

26. 2015 Shared Savings Program (SSP) Accountable Care Organizations (ACO) PUF. CMS website. data.cms.gov/Special-Programs-Initiatives-Accountable-Care-Orga/2015-Shared-Savings-Program-SSP-Accountable-Care-O/7rrf-3gxr. Published October 19, 2016. Accessed February 23, 2017. Metadata updated September 21, 2017.

27. 2016 Shared Savings Program (SSP) Accountable Care Organizations (ACO) PUF. CMS website. data.cms.gov/Special-Programs-Initiatives-Medicare-Shared-Savin/2016-Shared-Savings-Program-SSP-Accountable-Care-O/3jk5-q6dr. Published October 27, 2017. Updated November 7, 2017. Accessed January 19, 2018.

28. Tu T, Caughey W, Muhlestein D. Accountable Care Organizations and Risk-Based Payment Arrangements: Strong Preference for Upside-Only Contracts. Salt Lake City, UT: Leavitt Partners; 2016. leavittpartners.com/wp-content/uploads/2016/11/ACO_risks_whitepaper_v1.pdf. Accessed February 22, 2017.

29. Muhlestein D, McClellan M. Accountable care organizations in 2016: private- and public-sector growth and dispersion. Health Affairs Blog. April 21, 2016. healthaffairs.org/blog/2016/04/21/accountable-care-organizations-in-2016-private-and-public-sector-growth-and-dispersion. Accessed February 20, 2017.

30. Muhlestein D, Gardner P, Merrill T, Petersen M, Tu T. A Taxonomy of Accountable Care Organizations: Different Approaches to Achieve the Triple Aim. Salt Lake City, UT: Leavitt Partners; 2014. leavittpartners.com/wp-content/uploads/2014/06/A-Taxonomy-of-Accountable-Care-Organizations.pdf. Accessed February 22, 2017.

31. RTI International. Accountable Care Organization 2016 Program Quality Measure Narrative Specifications. Baltimore, MD: CMS; 2016. cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/Downloads/2016-ACO-NarrativeMeasures-Specs.pdf. Accessed February 23, 2017.

32. Shortell SM, Wu FM, Lewis VA, Colla CH, Fisher ES. A taxonomy of accountable care organizations for policy and practice. Health Serv Res. 2014;49(6):1883-1899. doi: 10.1111/1475-6773.12234.

33. Wu FM, Shortell SM, Lewis VA, Colla CH, Fisher ES. Assessing differences between early and later adopters of accountable care organizations using taxonomic analysis. Health Serv Res. 2016;51(6):2318-2329. doi: 10.1111/1475-6773.12473.

34. Peiris D, Phipps-Taylor MC, Stachowski CA, et al. ACOs holding commercial contracts are larger and more efficient than noncommercial ACOs. Health Aff (Millwood). 2016;35(10):1849-1856. doi: 10.1377/hlthaff.2016.0387.

35. Lage DE, Rusinak D, Carr D, Grabowski DC, Ackerly DC. Creating a network of high-quality skilled nursing facilities: preliminary data on the postacute care quality improvement experiences of an accountable care organization. J Am Geriatr Soc. 2015;63(4):804-808. doi: 10.1111/jgs.13351.

36. Berkowitz SA, Pahira JJ. Accountable care organization readiness and academic medical centers. Acad Med. 2014;89(9):1210-1215. doi: 10.1097/ACM.0000000000000365.

37. Allison PD. Fixed Effects Regression Models. Thousand Oaks, CA: SAGE Publications, Inc; 2009.

38. 2014 Star Ratings technical notes, fact sheet, LPI list and medication lists: Medicare 2014 Part C & D Star Rating technical notes. CMS website. cms.gov/Medicare/Prescription-Drug-Coverage/PrescriptionDrugCovGenIn/Downloads/2014_Star_Ratings_and_Display_Measures.zip. Published April 2, 2014. Accessed February 12, 2018.

39. Page L. Do small practices provide better patient care? Medscape website. medscape.com/features/content/6006322. Published November 19, 2014. Accessed May 31, 2017.

40. Doyle JJ Jr, Graves JA, Gruber J. Uncovering waste in US healthcare: evidence from ambulance referral patterns. J Health Econ. 2017;54:25-39. doi: 10.1016/j.jhealeco.2017.03.005.

41. Muhlestein DB, Morrison SQ, Saunders RS, Bleser WK, McClellan MB, Winfield LD. Medicare accountable care spending patterns: shifting expenditures associated with savings [published online February 12, 2018]. Am J Accountable Care. 2018;6(1). In press.

42. Health Care Payment Learning & Action Network. Alternative payment model (apm) framework: refreshed for 2017. HCP LAN website. hcp-lan.org/workproducts/apm-refresh-whitepaper-final.pdf. Published 2017. Accessed July 25, 2017.

43. McClellan M, McStay F, Saunders R. The roadmap to physician payment reform: what it will take for all clinicians to succeed under MACRA. Health Affairs Blog. August 30, 2016. healthaffairs.org/blog/2016/08/30/the-roadmap-to-physician-payment-reform-what-it-will-take-for-all-clinicians-to-succeed-under-macra. Accessed May 30, 2017.

44. Cooper Z, Craig SV, Gaynor M, Van Reenen J. The price ain’t right? hospital prices and health spending on the privately insured [NBER working paper no. 21815]. National Bureau of Economic Research website. nber.org/papers/w21815. Published December 2015. Accessed February 12, 2018.

45. Casalino LP, Pesko MF, Ryan AM, et al. Small primary care physician practices have low rates of preventable hospital admissions. Health Aff (Millwood). 2014;33(9):1680-1688. doi: 10.1377/hlthaff.2014.0434.

46. ACO Investment Model. CMS website. innovation.cms.gov/initiatives/ACO-Investment-Model. Published March 27, 2017. Accessed May 24, 2017.

Download PDF: ACO Quality Over Time: The MSSP Experience and Opportunities for System-Wide Improvement