ACOs With Risk-Bearing Experience Are Likely Taking Steps to Reduce Low-Value Medical Services

ABSTRACT

Objectives: Accountable care organizations (ACOs) are groups of healthcare providers responsible for quality of care and spending for a defined patient population. The elimination of low-value medical services will improve quality and reduce costs and, therefore, ACOs should actively work to reduce the use of low-value services. We set out to identify ACO characteristics associated with implementation of strategies to reduce overuse.

Study Design: Survey analysis.

Methods: We used the National Survey of ACOs to determine the percentage of responding ACOs aware of the Choosing Wisely campaign and to what degree ACOs have taken steps to reduce the use of low-value services. We identified characteristics of ACOs associated with implementing low-value care—reducing strategies using 3 statistical models (stepwise and LASSO logistic regression and random forest).

Results: Responding executives of 155 of 267 ACOs (58%) were aware of Choosing Wisely. Eighty-four of those 155 ACO leaders said that their ACOs also actively implemented strategies to reduce the use of low-value services, largely through educating physicians and stimulating shared decision making. All 3 models identified the presence of at least 1 commercial payer contract and prior joint experience pursuing risk-based payment contracts as the most important predictors of an ACO actively implementing strategies to reduce low-value care.

Conclusions: In the first year of implementation, just one-third of ACOs had taken steps to reduce the use of low-value medical services. Safety-net ACOs and those with little experience as a risk-bearing organization need more time and support from healthcare payers and the Choosing Wisely campaign to prioritize the reduction of overuse.

Am J Manag Care. 2018;24(7):e216-e221Takeaway Points

Collective risk taking in financial contracts is the most influential determinant for accountable care organizations (ACOs) in taking steps to reduce unnecessary care. Safety-net ACOs are not likely to take steps like educating physicians on low-value medical services and encouraging shared decision making. ACOs with less experience in risk bearing will likely start to prioritize overuse when they acquire more risk as an organization.

• ACOs with little experience in risk bearing and safety-net ACOs should be specifically stimulated to reduce overuse with targeted advocacy efforts of healthcare payers and the Choosing Wisely campaign.
• Research should focus on identifying efficient strategies for waste reduction with specific attention to audit-and-feedback mechanisms on overuse and underuse.

Accountable care organizations (ACOs) are voluntary groups of provider organizations that are collectively held accountable for both quality of care and total spending for a defined group of patients through payment contracts. A promising strategy to improve quality and financial sustainability involves the reduction of low-value medical services. Indeed, prior research shows a modest decrease in the use of low-value care, and thus in spending rates, for ACOs compared with non-ACO providers with predominantly fee-for-service payment models.¹ While utilization and related spending have decreased in ACOs, quality scores and care satisfaction have remained similar or improved compared with other organizations.^2-4 However, it is not clear what strategies ACOs deploy to lower unnecessary care, nor what features predict a commitment toward overuse reduction.

One way to tackle low-value care is to embrace the Choosing Wisely campaign.⁵ Choosing Wisely aims to reduce the delivery of low-value medical services by promoting conversations between patients and physicians on the appropriateness of care. More than 70 US specialty societies have defined concise lists of 5 to 10 wasteful interventions that “physicians and patients should question.”⁵ The synergies of ACOs and Choosing Wisely regarding care improvement and overuse reduction suggest that ACOs committed to reducing low-value care should be aware of this campaign and also work toward actively lowering the utilization of these medical services. In this study, we analyze data from the National Survey of ACOs (NSACO) to determine which strategies are used to reduce low-value care and identify the ACO characteristics that predict the use of such methods.

METHODS

NSACO

The NSACO is an online survey designed by researchers at the Dartmouth Institute for Health Policy and the University of California, Berkeley. It questions ACOs (Medicare Shared Savings Program [MSSP] ACOs, Medicare Pioneer ACOs, state Medicaid ACOs, and commercial payer ACOs) on their composition, characteristics, contracts, and capabilities.^6,7 A total of 752 ACOs were identified through public documents, provider surveys, scientific literature, and certification by the National Committee for Quality Assurance and invited to participate. Senior ACO executives, including chief executive officers, executive directors, and chief medical officers, filled out the survey. At the time of our analysis, the survey was fielded in 3 consecutive waves, with each wave questioning newly formed ACOs (wave 1, October 2012-May 2013; wave 2, September 2013-March 2014; wave 3, November 2014-May 2015). The median duration between the implementation of the ACO contract to the time of the survey was 11.6 months (interquartile range, 7.1-13.2 months).⁷ Over all 3 waves, 64% of ACOs filled out the survey. Waves 2 and 3 of the survey were more elaborate, and therefore, ACOs from wave 1 were approached with a follow-up survey asking additional questions during the fielding of wave 3. Questions about Choosing Wisely were not asked in wave 1, excluding from our analysis 93 ACOs that participated in the first wave but not the follow-up survey. Survey questions related to Choosing Wisely included: (1) “Are you aware of the Choosing Wisely program?” and, if the response was positive, (2) “What steps have you taken to reduce the use of Choosing Wisely tests and procedures?”

Multivariate Statistical Modeling

We divided our sample into 2 groups: (1) ACOs not aware of the Choosing Wisely campaign or aware but not taking steps to support it and (2) ACOs taking steps to actively reduce the use of low-value medical services.

We compiled an a priori list of 62 survey responses that could be associated with the decision to take steps to support Choosing Wisely (the eAppendix Table [eAppendix available at ajmc.com] lists these characteristics). Based on existing hypotheses about how these characteristics might affect an ACO’s decision to take steps to reduce overuse, as well as on simple pairwise significance tests, we then selected a subset of 22 variables from this list. We excluded ACO characteristics on quality behavior to prevent potential reverse causality with waste-reducing efforts. To identify the main drivers behind the decision to take steps to reduce overuse within those 22 variables, we used both logistic regression (stepwise regression and LASSO regression) and classification techniques (random forest).⁸ Stepwise logistic regression was performed both backward and forward. LASSO imposes shrinkage constraints on the variables, resulting in an optimal model with only those characteristics that have a coefficient greater than 0. Random forest stratifies the predictor space in regions with nonlinear boundaries between variables, producing multiple decision trees that are combined into a single consensus prediction. The 3 statistical approaches identified 3 sets of prediction variables, and we subsequently assessed consistency of associations across these 3 models. Furthermore, we evaluated the relative predictive merits of each model by comparing their receiver operating characteristic (ROC) curves and confusion matrices on the basis of their implied misclassification rates (fraction of false positives and false negatives).

Savings and MSSP Quality Scores

CMS publicly reports shared savings payments and the outcomes of 33 quality measures per performance year for each participating ACO. The quality scores are in 4 domains: patient experience (including a measure on shared decision making), care coordination, at-risk measures, and preventive care. We calculated an overall quality score and a quality score per domain for the first 2 performance years of each ACO (the year in which the ACO filled out the survey) using the CMS sliding scale approach, as described elsewhere.^7,9 We compared these quality scores and the savings per beneficiary attributed to the ACO according to CMS in the first 2 years with reference to historical expenditure benchmarks, between ACOs taking steps to reduce overuse and ACOs not taking steps, using a 2-sample t test.

RESULTS

Characteristics of ACOs Taking Steps to Reduce Overuse

Of 305 potential ACOs (wave 1 follow-up, 82 ACOs; wave 2, 95 ACOs; wave 3, 128 ACOs), survey respondents for 267 ACOs answered the question “Are you aware of the Choosing Wisely program?” Of these, 58% (155) reported awareness of Choosing Wisely, but only 31.5% (84) said they had also taken steps to reduce the use of low-value services (the eAppendix Table characterizes these ACOs). Consequently, 183 ACOs (68.5%) did not take such steps, partly because they were not aware of the Choosing Wisely campaign.

Compared with ACOs not taking steps, ACOs that had implemented strategies to reduce waste included more hospitals in their largest contract (P <.01) and were significantly more likely to consider themselves an integrated delivery system (P <.01) (the eAppendix Table compares these 2 groups). Provider organizations within ACOs implementing those strategies had more often jointly pursued risk-based payment contracts in the past (P <.01), and this group of ACOs had previously participated in a higher number of payment reform efforts than ACOs that did not take steps (P <.01). More ACOs taking steps to reduce waste had at least 1 commercial contract (P <.01), and Medicare contracts were less prevalent (P = .03). Although their commercial contracts were more often characterized by both bonus and downside risk (P = .02) than by a bonus only (P = .03), this bonus was less often contingent on quality metrics (P = .02). In addition, ACOs that actively reduced low-value services were more likely to allocate shared savings bonus payments across participating members (P = .04), compensate physicians based on clinical quality measures (P = .01), and share cost measures among their physicians (P = .02). A larger proportion of ACOs that were not aware of Choosing Wisely or not actively reducing waste were safety-net organizations (P <.01), defined as having more than 25% uninsured or Medicaid beneficiaries.

Of the 84 ACOs that reported using strategies to reduce low-value care, just 57 (68%) were asked to specify the steps they had taken, because this question was added to the survey after wave 2. On average, these ACOs took 3 different steps. The most frequently reported waste-lowering strategies included educating physicians on low-value tests and procedures (82.5% of ACOs), encouraging discussions between physicians and patients about appropriate care (73.7%), and disseminating Choosing Wisely materials (68.4%) (Table 1). Other strategies included handing out decision guides for patients, pop-ups in the electronic health record reminding physicians of the low value of certain tests and procedures, and audit and feedback on individual physician performance. Few ACOs (3.5%) reported taking steps to reduce waste by changing physician payment incentives.

Predictors for Implementing Strategies to Reduce Low-Value Care

Backward and forward stepwise logistic regression identified the same 6 variables and similar importance for predicting the use of low-value care—reducing strategies (Table 2). The presence of at least 1 commercial contract and prior participation of all provider organizations within the ACO in risk-based payment contracts were the most important factors. These were also the 2 most significant predictors in the LASSO model (eAppendix Figure 1). This model gave only 1 negative predictor, which predicted that an ACO would not actively seek waste reduction. The characteristic, more than 25% uninsured or Medicaid patients, was fourth in order of importance, and also 1 of the 6 characteristics in the stepwise regression (Table 2). In the consensus prediction tree from the random forest, the third model we used, the same 2 variables (at least 1 commercial contract, prior joint experience with risk-based payment contracts) were the 2 highest branches in the decision tree (Figure). Similarly, having more than 25% uninsured or Medicaid patients was the most influential negative predictor in this model. In conclusion, our working model to predict which ACOs would deploy strategies to reduce low-value care consisted of 2 positive predictors indicating joint experience in risk taking in the form of financial models, as well as 1 negative predictor (namely, a large contingent of safety-net patients).

The 3 models exhibited similar minimum misclassification rates on their ROC curves. The lowest possible misclassification rate for both stepwise regression and LASSO was 24% (eAppendix Figure 2 shows the true- and false-positive rates for the minimum misclassification of each model), whereas it was 26% for random forest (data not shown). This attributes to these models a power to identify 33% to 39% of true positives, or ACOs that take steps to reduce low-value care.

Relationship Between Taking Steps and CMS Data on Quality and Savings

Waste-reducing strategies were not associated with differences in CMS quality measures or savings per beneficiary for the 158 ACOs for which CMS data were available in performance year 1 (46 taking steps, 112 not taking steps) (Table 3⁹). Furthermore, changes in these parameters from performance year 1 to year 2 were similar (32 and 70 ACOs, respectively; data not shown).

DISCUSSION

Based on 3 waves of the NSACO, we show that just one-third of all ACOs are taking steps to reduce low-value care. The best predictors for an ACO to deploy strategies for waste reduction were consistent across the 3 models with various statistical approaches. ACOs with a commercial contract and those whose provider organizations have jointly pursued risk-based payment contracts in the past are more likely to actively reduce overuse. Our finding is consistent with those of a recent study wherein ACOs with commercial payer contracts were more actively implementing efficiency measures compared with ACOs with public payer contracts only.⁷

We found that ACOs with a large contingent of uninsured or Medicaid patients probably do not take steps to reduce overuse. However, the delivery of low-value care is as common among uninsured or Medicaid patients as among the privately insured.¹⁰ In prior research, minorities and those with poor and fair health were notably at a higher risk of receiving wasteful medical services.^11,12 Both groups are overrepresented in the Medicaid population.^13,14 Therefore, safety-net ACOs should pay attention to overuse of medical services. However, with relatively fewer resources for quality improvement, these ACOs may be prioritizing tackling the underuse of high-value practices over limiting low-value practices.

Furthermore, we did not find correlations between taking steps to reduce overuse and CMS quality measures (including the use of shared decision making) or between such steps and overall savings for ACOs participating in the MSSP in the first 2 performance years. Indeed, although waste in healthcare is ubiquitous,¹⁵ quality measures and savings are determined by many factors other than waste reduction alone. In addition, the effect of strategies other than clinical decision support and performance feedback to lower low-value care may be limited, certainly if they do not address both patient and provider roles.¹⁶ For example, audit-and-feedback methods are efficient in reducing unnecessary antibiotic prescriptions, but only if the intervention is perpetuated.¹⁷ Decision aids—but not shared decision making or the dissemination of educational materials&mdash;have moderate effect on the use of discretionary surgery.¹⁸ Although not yet extensively studied, certain pay-for-performance models have resulted in only modest improvements in care processes and outcomes.¹⁹ In our study, just half of ACOs aware of Choosing Wisely took active steps to reduce low-value care, and in most of those ACOs, the interventions were low impact and did not include the most promising strategies for waste reduction (namely, decision aids, computer decision support, and audit and feedback on individual physician behavior). To be meaningful, increasing awareness of Choosing Wisely should go hand in hand with practical advice for provider organizations on how to enhance appropriate use of care efficiently. Therefore, more research is needed to determine the correct design of strategies to reduce low-value care, the potential to lower utilization of such care in ACOs, and the numeric effect of reductions on quality and savings in the long run.

Limitations

Several limitations to studies involving NSACO data have been recognized,^6,7 notably, the fact that survey questions were answered by ACO executives who may not have been aware of efforts to reduce low-value care in their organization and the short period (less than 1 year) between the start of the ACO contract and the survey. Within this time frame, ACOs may not yet have been able to initiate strategies to reduce low-value care. Of note, previous comparisons of ACOs filling out the NSACO survey with those that failed to respond have not shown a significant nonresponse bias in terms of beneficiary or provider composition, organizational structure, overall quality, or savings distribution.^6,20,21

Physicians are an important source of low-value care utilization. It explains the focus chosen by Choosing Wisely on conversations between physicians and patients on appropriate care. However, collective risk taking in financial contracts influences ACOs to actively reduce low-value care more than physician leadership does. It is also possible that other unobserved characteristics in linkage with risk-bearing experience are influencing an ACO’s decision to seek a reduction in low-value care. In addition, reverse causality (ACOs most confident in their capacity to reduce waste signing risk-bearing contracts) cannot be excluded. However, this would not negatively affect the relationship we detected between collective risk taking and waste reduction.

CONCLUSIONS

The positive influence of risk bearing on efforts to reduce low-value care may be explained by a combination of resources, incentives, and opportunity. First, ACOs contracting with commercial payers may have more stringent contracting requirements that force them to prioritize waste reduction and more resources dedicated to quality improvement than do ACOs involved exclusively in public payer contracts. Second, ACOs with prior experience in risk-based contracting may have been able to develop the culture, systems, and technical know-how to tackle challenging issues such as overuse over time. Consequently, ACOs with less experience in risk bearing and safety-net ACOs will likely start prioritizing the reduction of overuse as they acquire more risk. In the meantime, those ACOs should be otherwise stimulated to reduce overuse and specifically targeted by advocacy efforts of healthcare payers and the Choosing Wisely campaign. Furthermore, researchers, policy makers, and the Choosing Wisely campaign should focus on defining waste-reducing efforts that are efficient and practical in use, with special attention to audit-and-feedback mechanisms on individual physician performance for both underuse and overuse. With US healthcare spending as high as 18% of gross domestic product, and healthcare outcomes in the United States lagging behind in comparisons with other high-income countries,²² all ACOs should use available levers for waste reduction, including Choosing Wisely materials, and implement strategies with proven efficacy in reducing low-value care to decrease costs and increase the quality of healthcare.

Acknowledgments

Dr Haverkamp and Dr Peiris, both Harkness fellows 2015-2016, gratefully acknowledge their sponsor, the Commonwealth Fund (New York), which has also supported the work with grant 20160616. In addition, Dr Haverkamp recognizes the Dutch Ministry of Health Welfare and Sport for sponsoring her fellowship. Furthermore, this project is supported by Agency for Healthcare Research and Quality (AHRQ) grants R01HS023812 and U19HS024075. The latter grant falls under AHRQ’s Comparative Health System Performance Initiative, which studies how healthcare delivery systems promote evidence-based practices and patient-centered outcomes research in delivering care.Author Affiliations: Department of Health Policy and Management, Harvard T.H. Chan School of Public Health (MHH, DP, MBR), Boston, MA; The George Institute for Global Health, University of New South Wales (DP), Sydney, Australia; The Dartmouth Institute for Health Policy & Clinical Practice (AJM, CHC), Lebanon, NH; Scientific Center for Quality of Healthcare (IQ healthcare), Radboud University Medical Center, Radboud Institute for Health Sciences (GPW), Nijmegen, Netherlands; Division of General Medicine, Brigham and Women’s Hospital (TS), Boston, MA; Department of Health Care Policy, Harvard School of Medicine (TS), Boston, MA.

Source of Funding: This study was funded by the Commonwealth Fund (New York) through grant 20160616, and personal grants to Dr Haverkamp and Dr Peiris (both Harkness fellows 2015-2016). In addition, the Harkness fellowship of Dr Haverkamp was sponsored by the Dutch Ministry of Health Welfare and Sport. Furthermore, AHRQ supported the work with grants R01HS023812 and U19HS024075.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (MHH, DP, GPW, MBR, TS, CHC); acquisition of data (MHH, CHC); analysis and interpretation of data (MHH, DP, AJM, MBR, TS, CHC); drafting of the manuscript (MHH, AJM); critical revision of the manuscript for important intellectual content (MHH, DP, AJM, GPW, MBR, TS, CHC); statistical analysis (MHH); provision of patients or study materials (CHC); obtaining funding (MHH, CHC); administrative, technical, or logistic support (AJM, CHC); and supervision (GPW, TS, CHC).

Address Correspondence to: Margje H. Haverkamp, MD, PhD, Department of Health Policy and Management, Harvard T.H. Chan School of Public Health, Kresge Bldg, Rm 431, 677 Huntington Ave, Boston, MA 02115. Email: mhhaverk@hsph.harvard.edu.REFERENCES

1. Schwartz AL, Chernew ME, Landon BE, McWilliams JM. Changes in low-value services in year 1 of the Medicare Pioneer Accountable Care Organization program. JAMA Intern Med. 2015;175(11):1815-1825. doi: 10.1001/jamainternmed.2015.4525.

2. Nyweide DJ, Lee W, Cuerdon TT, et al. Association of Pioneer Accountable Care Organizations vs traditional Medicare fee for service with spending, utilization, and patient experience. JAMA. 2015;313(21):2152-2161. doi: 10.1001/jama.2015.4930.

3. McWilliams JM, Chernew ME, Landon BE, Schwartz AL. Performance differences in year 1 of Pioneer Accountable Care Organizations. N Engl J Med. 2015;372(20):1927-1936. doi: 10.1056/NEJMsa1414929.

4. Kelleher KJ, Cooper J, Deans K, et al. Cost saving and quality of care in a pediatric accountable care organization. Pediatrics. 2015;135(3):e582-e589. doi: 10.1542/peds.2014-2725.

5. Choosing Wisely: promoting conversations between providers and patients. Choosing Wisely website. choosingwisely.org. Accessed November 11, 2016.

6. Colla CH, Lewis VA, Shortell SM, Fisher ES. First national survey of ACOs finds that physicians are playing strong leadership and ownership roles. Health Aff (Millwood). 2014;33(6):964-971. doi: 10.1377/hlthaff.2013.1463.

7. Peiris D, Phipps-Taylor MC, Stachowski CA, et al. ACOs holding commercial contracts are larger and more efficient than noncommercial ACOs. Health Aff (Millwood). 2016;35(10):1849-1856. doi: 10.1377/hlthaff.2016.0387.

8. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. New York, NY: Springer; 2009.

9. Medicare Shared Savings Program quality measure benchmarks for the 2014 reporting year. CMS website. cms.gov/Medicare/Medicare-Fee-for-Service-Payment/sharedsavingsprogram/downloads/MSSP-QM-Benchmarks.pdf. Published February 2015. Accessed November 11, 2016.

10. Barnett ML, Linder JA, Clark CR, Sommers BD. Low-value medical services in the safety-net population. JAMA Intern Med. 2017;177(6):829-837. doi: 10.1001/jamainternmed.2017.0401.

11. Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing Wisely: prevalence and correlates of low-value health care services in the United States [erratum in J Gen Intern Med. 2016;31(4):450. doi: 10.1007/s11606-015-3420-5]. J Gen Intern Med. 2015;30(2):221-228. doi: 10.1007/s11606-014-3070-z.

12. Schpero WL, Morden NE, Sequist TD, Rosenthal MB, Gottlieb DJ, Colla CH. For selected services, blacks and Hispanics more likely to receive low-value care than whites. Health Aff (Millwood). 2017;36(6):1065-1069. doi: 10.1377/hlthaff.2016.1416.

13. State health facts: Medicaid enrollment by race/ethnicity, FY 2013. Kaiser Family Foundation website. kff.org/medicaid/state-indicator/medicaid-enrollment-by-raceethnicity/?currentTimeframe=0&selected&#8203;Distributions=white--black--hispanic--other--total. Accessed January 13, 2017.

14. LaPar DJ, Bhamidipati CM, Mery CM, et al. Primary payer status affects mortality for major surgical operations. Ann Surg. 2010;252(3):544-551. doi: 10.1097/SLA.0b013e3181e8fd75.

15. Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307(14):1513-1516. doi: 10.1001/jama.2012.362.

16. Colla CH, Mainor AJ, Hargreaves C, Sequist T, Morden N. Interventions aimed at reducing use of low-value health services: a systematic review. Med Care Res Rev. 2017;74(5):507-550. doi: 10.1177/1077558716656970.

17. Gerber JS, Prasad PA, Fiks AG, et al. Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial. JAMA. 2013;209(22):2345-2352. doi: 10.1001/jama.2013.6287.

18. Stacey D, Légaré F, Lewis K, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2017;4:CD001431.

19. Bardach NS, Wang JJ, De Leon SF, et al. Effect of pay-for-performance incentives on quality of care in small practices with electronic health records: a randomized trial. JAMA. 2013;310(10):1051-1059. doi: 10.1001/jama.2013.277353.

20. Albright BB, Lewis VA, Ross JS, Colla CH. Preventive care quality of Medicare accountable care organizations: associations of organizational characteristics with performance. Med Care. 2016;54(3):326-335. doi: 10.1097/MLR.0000000000000477.

21. Wu FM, Shortell SM, Lewis VA, Colla CH, Fisher ES. Assessing differences between early and later adopters of accountable care organizations using taxonomic analysis. Health Serv Res. 2016;51(6):2318-2329. doi: 10.1111/1475-6773.12473.

22. Schneider EC, Sarnak DO, Squires D, Shah A, Doty MM. Mirror, mirror 2017: international comparison reflects flaws and opportunities for better U.S. health care. The Commonwealth Fund website. commonwealthfund.org/interactives/2017/july/mirror-mirror. Published July 2017. Accessed August 2017.

Download PDF: ACOs With Risk-Bearing Experience Are Likely Taking Steps to Reduce Low-Value Medical Services