Inappropriate Wrist MRI: Did Guidelines Have an Impact?

ABSTRACT

Objectives: To assess the national prevalence and cost of inappropriate MRI in patients with wrist pain prior to and following American College of Radiology (ACR) guideline publication.

Study Design: We used administrative claims from the IBM MarketScan Research Databases to evaluate the appropriateness of wrist MRI in a national cohort of patients with commercial insurance or Medicare Advantage.

Methods: Adult patients with a diagnosis of wrist pain between 2016 and 2019 were included and followed for 1 year. We made assessments of appropriateness based on ACR guidelines for specific wrist pain etiologies. We tabulated the total costs and out-of-pocket expenses associated with inappropriate MRI studies using weighted mean payments for facility and professional fees. We performed segmented logistic regression on interrupted time series data to identify predictors of receiving inappropriate imaging and the impact of guideline publication on MRI use.

Results: The study cohort consisted of 867,119 individuals. Of these, 40,164 individuals (4.6%) had MRI, of whom 52.6% received an inappropriate study. Inappropriate studies accounted for $44,493,234 in total payments and $8,307,540 in out-of-pocket expenses. The interrupted time series found an approximately 1% monthly decrease in the odds of receiving an inappropriate study after guideline dissemination.

Conclusions: MRI as a diagnostic tool for wrist pain is often inappropriate and expensive. Our findings support interventions to increase guideline adherence, such as integrated clinical decision support tools.

Am J Manag Care. 2024;30(3):e65-e72. https://doi.org/10.37765/ajmc.2024.89517

Takeaway Points

• Using the American College of Radiology (ACR) Appropriateness Criteria for wrist pain, we found that 53% of patients with wrist pain who underwent MRI received an inappropriate study.
• Most commonly, imaging was inappropriate due to a lack of a preceding x-ray, suggesting MRI is potentially used as a screening tool to evaluate wrist pain.
• Results of our interrupted time series analysis found ACR guideline publication had only a subtle impact on ordering practices.
• These findings indicate that efforts to reduce inappropriate imaging should employ active strategies, such as clinical decision support in electronic health records, to lessen low-value care.

The risks and benefits of MRI have been the focus of discussions over low-value care and its possible impact on initiatives to reduce unnecessary health care. Wrist pain is prevalent in approximately 19.1% of adults.¹ Although medical imaging is a useful tool for diagnosis and treatment, unnecessary imaging contributes substantially to overall health care spending, costing an estimated $30 billion annually in the US.²

Despite published guidelines aimed at reducing advanced diagnostic imaging, MRI represents the greatest expenditure compared with other imaging studies.^3-5 In 2018, the American College of Radiology (ACR) expert panel on musculoskeletal imaging released revised and more extensive appropriate imaging guidelines for wrist pain that characterize clinical scenarios in which MRI is appropriate. Guidelines were developed via a literature review of research published in peer-reviewed journals, graded on quality of evidence, supplemented by expert opinion, and reviewed annually.⁶ However, it is unclear whether the guideline development aided in reducing use and waste associated with inappropriate imaging.

It is critical to understand the impact of appropriateness criteria in reducing inappropriate advanced imaging to highlight where additional interventions are needed without compromising access and quality. This study applies the appropriateness criteria to patients presenting with multiple etiologies of wrist pain and identifies rates of inappropriate MRI use. We also evaluate the association between MRI use and patient characteristics. Additionally, we estimate the total payment and out-of-pocket (OOP) expenses associated with inappropriate studies. Lastly, we characterize the impact of ACR Appropriateness Criteria publication through an interrupted time series (ITS) analysis to compare MRI use before and after publication. We hypothesized that the rate of inappropriate use would vary by wrist pain etiology and no significant change in MRI use would occur in the postguideline period.

METHODS

Data Source and Cohort Selection

We used claims data from the IBM MarketScan Research Databases (formerly known as the Truven Health MarketScan Research Databases) from 2015 to 2020 for this retrospective cohort study. These databases contain deidentified outpatient data for patients with employer-sponsored and Medicare Advantage or supplemental insurance.⁷ This study adheres to the Strengthening the Reporting of Observational Studies in Epidemiology reporting guidelines for cohort studies. The study cohort included adults 18 years and older with at least 12 months of continuous enrollment prior to and following a wrist pain diagnosis with an index visit between January 1, 2016, and December 31, 2019. We excluded patients who had prior claims for wrist pain management by identifying wrist pain–related claims 12 months prior to the index visit identified in the study period. Further, we excluded patients with acute fractures or traumatic injuries, as these are not captured by the appropriateness criteria. An administrative database, such as MarketScan, is advantageous for achieving the study aims because it permits a large sample size and longitudinal observation of enrollees.⁸ Although administrative databases lack detailed clinical and causal information, analysis of such data facilitates timely study of real-world practices and trend analyses.^9-14

We identified patients with nonacute wrist pathology via International Classification of Diseases, Ninth Revision and International Statistical Classification of Diseases, Tenth Revision (ICD-9 and ICD-10, respectively) diagnosis codes and grouped them into the following categories based on etiology¹⁵: (1) scaphoid-related pathology; (2) Kienböck disease; (3) nonscaphoid malunion or nonunion; (4) inflammatory arthritis and gout; (5) neuropathic pathology; (6) posttraumatic arthritis and osteoarthritis; (7) instability, triangular fibrocartilage complex injury, sprain, and ligamentous injury; (8) cysts and soft tissue pathology; (9) muscle and tendon pathology; (10) nonspecific pain; and (11) miscellaneous. We assigned each patient to a definitive diagnosis category by the most specific diagnosis category identified throughout the 1-year follow-up, for which we applied the corresponding appropriateness criteria. Although the ACR Appropriateness Criteria guide imaging for prior scaphoid and nonscaphoid fractures in addition to suspected Kienböck disease, inflammatory arthritis, and carpal tunnel syndrome, they do not provide guidance when more than 1 of these specific etiologies are suspected. Thus, we excluded patients with diagnoses in at least 2 of the following more specific diagnosis categories: (1) scaphoid-related pathology, (2) Kienböck disease, (3) nonscaphoid malunion or nonunion, (4) inflammatory arthritis and gout, and (5) neuropathic pathology. We did not exclude patients who had multiple diagnoses comprising multiple less specific wrist pain etiologies. In the experience of the senior author (E.D.S.), patients often present to clinic initially with a diagnosis of nonspecific wrist pain, only to have this condition represented by a more specific code at a subsequent visit. Detailed information on the ICD-10 codes used and ACR Appropriateness Criteria can be accessed in eAppendix Tables 1 and 2, respectively (eAppendix available at ajmc.com).

Outcome Variables

We applied the ACR appropriateness variants to our wrist pain categorization schema to evaluate the following MRI studies, which were identified by Current Procedural Terminology codes: wrist MRI without intravenous contrast, wrist MRI with and without intravenous contrast, and wrist MR arthrography (eAppendix Table 2). Among those receiving more than 1 MRI, patients were categorized into the inappropriate MRI subgroup if they received at least 1 inappropriate MRI; those with only appropriate MRI were categorized into the appropriate MRI subgroup.

We recorded the total payment and OOP expenses (coinsurance, co-payment, and deductibles) associated with each MRI claim. The total payment represents the total amount paid by the insurer plus patient OOP expenses for the claim.¹⁶ We used professional and facility fees for each MRI to calculate the weighted mean payment.¹⁷ The weighted mean spending of total payments and OOP expenses for wrist MRI was $1522 and $395 (including both professional and facility components), respectively. We multiplied the weighted mean spending by the number of inappropriate MRI claims to obtain the inappropriate cohort total payments and OOP expenses. All cost variables were inflation adjusted to 2020 US$ using the Consumer Price Index for all urban consumers for medical care services.¹⁸

Explanatory Variables

We recorded sociodemographic characteristics, including sex, age at index visit, US geographic region of residence, median household income, place of residence, insurance plan type, and comorbidities. We used the metropolitan statistical area code to link to the 2019 US Census to determine median household income and place of residence.¹⁹ Insurance plan type included preferred provider organization (PPO), health maintenance organization (HMO), comprehensive, point of service, or other plan. The other category included basic plans, exclusive provider organization plans, consumer-driven health plans, and high-deductible health plans. As a proxy for health status, we calculated an Elixhauser Comorbidity Index score for each patient using ICD-9/ICD-10 codes.^20,21

Statistical Analysis

Descriptive statistics (mean and SD for continuous variables; frequency and percentages for categorical variables) were used to characterize the study cohort. To assess significant differences between the appropriate and inappropriate MRI subgroups, we used Mann-Whitney U tests for continuous variables (age and comorbidity score) and Pearson χ² tests for categorical variables. We used bivariate analysis to assess the relationship of each explanatory variable to receiving an inappropriate MRI; relevant variables were included in multivariable models to control for confounding. We used listwise deletion to account for patients with missing plan type, which resulted in exclusion of 16,420 patients, or 1.9% of the entire cohort, from our regression models.

We performed segmented regression analysis of ITS data to determine the change in level and the difference in changing rates (slopes) prior to and following ACR guideline release.²² The model contained 3 time periods: preguideline (January 2016-May 2018), dissemination (June 2018-October 2018), and postguideline period (November 2018-December 2019). To account for time needed to take effect and change physician behavior, we used a 5-month dissemination period based on similar methodology.²³ We modeled the monthly rate of inappropriate MRI studies to evaluate the impact of guideline publication on inappropriate MRI use. This approach permitted us to study an appreciable gradual change in ordering practice.²⁴

The primary aim was to evaluate inappropriate MRI use before and after guideline publication in May 2018. We modeled the probability of receiving inappropriate MRI by index visit month. We performed a piecewise multivariable logistic regression to estimate the association of index visit timing (preguideline or postguideline period) with the odds and predicted probability of a patient receiving an inappropriate MRI, allowing for a 5-month dissemination period while adjusting for the covariates. We conducted a sensitivity analysis using a 3-month dissemination period. All hypothesis tests were 2-sided and used a significance level of 0.05. Statistical analyses were completed from February to September 2022 using IBM SPSS Statistics 28.0 (IBM Corporation), SAS 9.4 (SAS Institute Inc), and R 4.1.2 (R Foundation for Statistical Computing). This study was considered exempt from review by the institutional review board at the University of Michigan because of the deidentified nature of the data.

RESULTS

Our study cohort included 867,119 patients with wrist pain (Figure 1), was 37.1% male, and had a mean age of 48.6 years (Table 1). Nearly half of patients (46.2%) had 1 to 3 Elixhauser comorbidities at the index encounter. Approximately 4.6% of patients (n = 40,164) received at least 1 MRI during the 12-month follow-up period after the index encounter. Of patients who received MRI, approximately half of them (52.6%) had an inappropriate study. Most commonly, imaging was deemed inappropriate because of the lack of a preceding x-ray (67.9%), and 32.1% of inappropriate MRI was due to an incorrect type of study obtained according to the ACR guidelines, such as MR arthrography against recommendations. Within the inappropriate-MRI subgroup, 58.3% of patients had more than 1 comorbidity compared with 50.7% of patients in the subgroup with only appropriate imaging (P < .001).

Total MRI use in the study cohort declined from 4.85% in the preguideline period to 4.02% in the postguideline period. Inappropriate MRI use decreased from 2.54% in the preguideline period to 2.15% in the postguideline period. The ITS found a significant decline in the predicted probability of receiving an inappropriate MRI following guideline publication (Figure 2). In the preguideline period, we observed no significant change in the odds of receiving an inappropriate MRI by month (OR, 1.000; 95% CI, 0.999-1.002; P = .601). In the postguideline period, the odds of receiving an inappropriate MRI decreased by approximately 1% every month (OR, 0.991; 95% CI, 0.985-0.998; P = .007). There was a significant difference in the slope of the preguideline period and the dissemination period and between the preguideline and postguideline periods. There was no significant difference between the dissemination and postguideline periods. Our sensitivity analysis using a 3-month dissemination period showed similar findings (eAppendix Figure). Predictors of receiving an inappropriate study are demonstrated in Table 2. Among patients who received MRI, patients older than 65 years had decreased odds of receiving an inappropriate study compared with patients aged 55 to 64 years (OR, 0.63; 95% CI, 0.53-0.76; P < .001). Patients enrolled in comprehensive and HMO health plans had decreased odds of inappropriate MRI compared with patients in PPO plans (OR, 0.82; 95% CI, 0.76-0.88; P < .001; and OR, 0.82; 95% CI, 0.78-0.86; P < .001, respectively). Patients with 4 to 7 comorbidities had increased odds of inappropriate imaging compared with patients with fewer (1-3) comorbidities (OR, 1.09; 95% CI, 1.04-1.14; P < .001) (Table 2).

Between 2016 and 2020, inappropriate MRI use resulted in $44,493,234 in total payments and $8,307,540 in OOP expenses. Patients with inappropriate MRI incurred a mean OOP expense of $495.11 compared with $219.08 in patients without MRI for wrist pain–related care during the 1-year period after index visit. During this period, patients with inappropriate MRI incurred a mean total payment of $2497.52 compared with $1028.61 in patients without MRI.

DISCUSSION

MRI is overused in many musculoskeletal conditions,^25-29 but its extent in patients with wrist pain was not previously characterized. We found that despite evidence-based guidelines endorsed by numerous medical societies,^6,30 more than half (52.6%) of patients with wrist pain who received MRI had at least 1 inappropriate study. Most commonly, MRI was inappropriate due to lack of a preceding x-ray (67.9%). We found that patients in HMO plans were significantly less likely to receive inappropriate imaging compared with patients in PPO plans. Patients with multiple (4-7) comorbidities also were more likely to receive inappropriate imaging compared with patients with fewer or no comorbidities. The ITS analysis demonstrates a small significant decrease (by 1% per month) in inappropriate MRI use to evaluate wrist pain after guideline publication. Inappropriate imaging accounted for $44,493,234 in total payments and $8,307,540 in OOP expenses over the study period. Collectively, these findings suggest high prevalence of low-value MRI use for wrist pain.

Previous research shows that MRI is a low-value screening tool for many upper extremity conditions.^4,5,31,32 Policy experts have considered several system-level strategies to reduce low-value care.^33,34 For instance, targeting health insurance plan features, such as patient cost-sharing or prior authorization requirements, could deter overuse.^35,36 Rosenthal et al studied insurance plan type and receipt of 6 health care services deemed unnecessary by the Choosing Wisely campaign, which was an initiative to reduce ordering of diagnostic tests, imaging, and medical treatments unlikely to change patient health or clinical management.³⁶ The authors found that enrollment in an HMO insurance plan was associated with decreased likelihood of receiving early imaging in low back pain.³⁶ Enrollment in a high-deductible health plan was also a negative predictor of early imaging, but to a lesser extent.³⁶ Consistent with this, we found that patients with HMO plans were nearly 20% less likely to obtain inappropriate imaging for wrist pain compared with patients with PPO plans. Altogether, these findings suggest that managed care plans may contain mechanisms—such as prior authorization or utilization review—to support guideline-adherent use of diagnostic imaging.

Another system-level strategy to reduce low-value care is through physician and hospital participation in accountable care organizations (ACOs), in which reimbursement is tied to care quality and cost reduction. Analysis of both the Medicare Pioneer ACO Model and Oregon’s Medicaid ACO model found modest reductions in low-value services, including advanced imaging use for nonspecific back pain and uncomplicated headaches.^37,38 Similarly, given that many patients with wrist pain in our study were referred for MRI without an initial x-ray, we propose that advanced imaging for nonspecific wrist pain could be a suitable service category that ACOs should consider discouraging in the future.

Although we observed that patients with more comorbidities had greater odds of receiving imaging, previous research has found that such patients receiving MRI for musculoskeletal pain were less likely to receive surgery.³⁹ The value of these studies in these patients is low if patients are ultimately ineligible for or not amenable to surgical intervention. As such, an upper extremity surgeon should guide imaging decisions for patients with wrist pain. To ensure high-value care, physicians should preemptively consider whether advanced imaging will change management, such as surgical or pharmaceutical intervention, which may not be appropriate for patients with high disease burden.^39,40 Furthermore, inappropriate imaging may have greater downstream consequences.⁴¹ Jacobs et al found that patients with injudicious MRI for musculoskeletal conditions incurred higher costs, rates of surgery, and opioid use.²⁶ Similarly, we found that patients with an inappropriate MRI had higher wrist pain–related OOP expenses and total payments compared with patients without any MRI studies.

Our ITS found an approximately 1% monthly decrease in inappropriate MRI use after ACR guideline implementation. Multiple possible explanations exist for this modest decrease. First, MRI is well reimbursed.⁴² This provides little economic motivation for physicians to seek higher-value diagnostic protocols first. Second, the perceived medicolegal environment of the US may contribute to increased use, as physicians who are concerned with legal risks may overrely on MRI.^42-44 Third, insufficient guideline visibility or enforcement may hinder adherence. Physician-level interventions such as education, appropriateness checklists, and clinical decision support have been effective at reducing inappropriate imaging use.^25,28,45-47 Evidence suggests that a layering approach, combining multiple strategies, may lead to more substantial changes in physician practices.^48-50 For example, Dudzinski et al randomly assigned attending cardiologists to an appropriate use educational intervention to promote judicious transthoracic echocardiogram ordering practices.⁴⁹ This consisted of a lecture followed by monthly emails detailing the attending’s proportion of inappropriate studies with a rationale for why each echocardiogram was inappropriate based on published criteria.⁴⁹ Consistent with results of other studies,^50,51 the authors found that scheduled feedback significantly reduced the prevalence of inappropriate studies, suggesting that active engagement is more effective for changing physician behavior than relatively passive interventions. Finally, the referral process for hand specialists can contribute to injudicious diagnostic imaging.^4,5,52 Hartzell et al found that of patients who presented at a hand clinic, 74% had received prior testing or imaging; of this, 90% was unnecessary.⁵ Prior research suggests that hand specialists prescribe higher-value diagnostic testing.³⁹ Michelotti et al found that hand surgeons more often ordered MRI to question a specific anatomic injury compared with referring physicians, who more commonly ordered MRI for generic wrist pain; MRI for a specific injury was more likely to impact management.³⁹ Our findings support continued thoughtful MRI use by specialists, such as hand surgeons, who have better awareness of whether testing will impact patient decision-making. Taken together, our findings suggest a need for further system- and physician-level interventions to increase guideline awareness and adherence.

Limitations

This study has several limitations inherent to administrative data. First, claims data may not capture the full clinical complexity of each wrist pain etiology. Additionally, it is possible that pertinent information obtained in a physician visit was not assigned a corresponding ICD-9/ICD-10 code. Incorrect coding or failure to code all tests performed within claims data, such as a preceding x-ray, may present potential sampling bias. To mitigate this, we used a 1-year pre–index visit look-back period to verify that patients had no prior diagnosis codes or testing that would impact the appropriateness of MRI. Although it is possible that a patient did have a prior x-ray, it would be unlikely for it to occur without an associated claim. Second, our cohort is limited to patients with employer-sponsored insurance; however, in the US, private employer-sponsored insurance represents most insured patients.⁷ Third, we incorporated relevant sociodemographic and clinical factors into our regression models; however, there may be unmeasured confounders that impact our findings. Additionally, we excluded patients with missing insurance information from our regression models, which represented approximately 1.9% of our total cohort. To mitigate confounding, we ensured a similar distribution of other covariates in our study cohort before and after excluding patients with missing data.

CONCLUSIONS

This study shows the high, although gradually declining, prevalence of inappropriate MRI to evaluate wrist pain. Although establishing guidelines is necessary to address overuse, our findings suggest the need for additional interventions to systematically reduce inappropriate MRI studies. Clinical guidelines should be revised to consider patient risk factors and whether testing will impact treatment recommendations. As policy makers refine high-value care initiatives, it is essential to consider disincentivizing inappropriate advanced imaging use without compromising access to and quality of care.

Acknowledgments

Trista M. Benítez, MPH, and Meghan N. Cichocki, BS, contributed equally to this work and are listed as co–first authors.

Author Affiliations: Section of Plastic Surgery, Department of Surgery, University of Michigan Medical School (TMB, MNC, AVS, KCC, EDS), Ann Arbor, MI; Department of Biostatistics, University of Michigan School of Public Health (WJ, LW), Ann Arbor, MI; US Department of Veterans Affairs (VA) Center for Clinical Management Research, VA Ann Arbor Healthcare System (EDS), Ann Arbor, MI.

Source of Funding: None.

Author Disclosures: Ms Benítez receives institutional support through a Diversity Supplement from the National Institutes of Health awarded to the section of Plastic Surgery at the University of Michigan. Dr Chung receives funding from the National Institutes of Health, a research grant from Sonex Health to study carpal tunnel outcomes, and book royalties from Wolters Kluwer and Elsevier. Dr Sears is supported by Career Development Award No. IK2 HX002592 from the VA Health Services Research and Development Service. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (TMB, MNC, AVS, KCC, EDS); acquisition of data (WJ, LW); analysis and interpretation of data (TMB, MNC, WJ, AVS, LW, EDS); drafting of the manuscript (TMB, MNC, WJ, AVS, LW, KCC, EDS); critical revision of the manuscript for important intellectual content (TMB, MNC, AVS, LW, KCC, EDS); statistical analysis (TMB, WJ, LW); provision of patients or study materials (EDS); and administrative, technical, or logistic support (KCC).

Address Correspondence to: Erika D. Sears, MD, MS, Section of Plastic Surgery, Department of Surgery, University of Michigan Medical School, 2130 Taubman Center, SPC 5340, 1500 E Medical Center Dr, Ann Arbor, MI 48109-5340. Email: endavis@med.umich.edu.

REFERENCES

1. Ferguson R, Riley ND, Wijendra A, Thurley N, Carr AJ, Bjf D. Wrist pain: a systematic review of prevalence and risk factors—what is the role of occupation and activity? BMC Musculoskelet Disord. 2019;20(1):542. doi:10.1186/s12891-019-2902-8

2. LaPointe J. Despite financial incentives, medical imaging rates reaccelerating. RevCycleIntelligence. September 5, 2019. Accessed November 21, 2021. https://revcycleintelligence.com/news/despite-financial-incentives-medical-imaging-rates-reaccelerating

3. Lehnert BE, Bree RL. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support? J Am Coll Radiol. 2010;7(3):192-197. doi:10.1016/j.jacr.2009.11.010

4. Sears ED, Lu YT, Wood SM, et al. Diagnostic testing requested before surgical evaluation for carpal tunnel syndrome. J Hand Surg Am. 2017;42(8):623-629.e1. doi:10.1016/j.jhsa.2017.05.006

5. Hartzell TL, Shahbazian JH, Pandey A, et al. Does the gatekeeper model work in hand surgery? Plast Reconstr Surg. 2013;132(3):381e-386e. doi:10.1097/PRS.0b013e31829acc74

6. Rubin DA, Roberts CC, Bencardino JT, et al; Expert Panel on Musculoskeletal Imaging. ACR Appropriateness Criteria chronic wrist pain. J Am Coll Radiol. 2018;15(5S):S39-S55. doi10.1016/j.jacr.2018.03.021

7. Hansen L. IBM MarketScan Research Databases for life sciences researchers. IBM Watson Health white paper. April 2018. Accessed April 26, 2022. https://web.archive.org/web/20220521165844/https://www.ibm.com/downloads/cas/0NKLE57Y

8. Billig JI, Sterbenz JM, Zhong L, Chung KC. Gender disparities in preoperative resource use for wrist arthroscopy. Plast Reconstr Surg. 2018;142(5):1267-1274. doi:10.1097/PRS.0000000000004840

9. Nguyen LL, Barshes NR. Analysis of large databases in vascular surgery. J Vasc Surg. 2010;52(3):768-774. doi:10.1016/j.jvs.2010.03.027

10. Mazzali C, Duca P. Use of administrative data in healthcare research. Intern Emerg Med. 2015;10(4):517-524. doi:10.1007/s11739-015-1213-9

11. Moll K, Wong HL, Fingar K, et al. Validating claims-based algorithms determining pregnancy outcomes and gestational age using a linked claims-electronic medical record database. Drug Saf. 2021;44(11):1151-1164. doi:10.1007/s40264-021-01113-8

12. Columbo JA, Kang R, Trooboff SW, et al. Validating publicly available crosswalks for translating ICD-9 to ICD-10 diagnosis codes for cardiovascular outcomes research. Circ Cardiovasc Qual Outcomes. 2018;11(10):e004782. doi:10.1161/CIRCOUTCOMES.118.004782

13. Quan H, Li B, Saunders LD, et al; IMECCHI Investigators. Assessing validity of ICD-9-CM and ICD-10 administrative data in recording clinical conditions in a unique dually coded database. Health Serv Res. 2008;43(4):1424-1441. doi:10.1111/j.1475-6773.2007.00822.x

14. Sears JM, Rundell SD. Development and testing of compatible diagnosis code lists for the Functional Comorbidity Index: International Classification of Diseases, Ninth Revision, Clinical Modification and International Classification of Diseases, 10th Revision, Clinical Modification. Med Care. 2020;58(12):1044-1050. doi:10.1097/MLR.0000000000001420

15. Dreckmann SC, von Schroeder HP, Novak CB, Baltzer HL. Utility of specialized imaging for diagnosis of chronic wrist pain. J Wrist Surg. 2019;8(6):497-502. doi:10.1055/s-0039-1697022

16. Billig JI, Chen JS, Lu YT, Chung KC, Sears ED. The economic burden of out-of-pocket expenses for plastic surgery procedures. Plast Reconstr Surg. 2020;145(6):1541-1551. doi:10.1097/prs.0000000000006847

17. George E, Tsipas S, Wozniak G, et al. MRI of the knee and shoulder performed before radiography. J Am Coll Radiol. 2014;11(11):1053-1058. doi:10.1016/j.jacr.2014.05.014

18. US Bureau of Labor Statistics. Consumer Price Index for all urban consumers: medical care in U.S. city average (CPIMEDSL). FRED, Federal Reserve Bank of St Louis. Accessed August 17, 2022. https://fred.stlouisfed.org/series/CPIMEDSL

19. Graves JM, Mackelprang JL, Moore M, et al. Rural-urban disparities in health care costs and health service utilization following pediatric mild traumatic brain injury. Health Serv Res. 2019;54(2):337-345. doi:10.1111/1475-6773.13096

20. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27. doi:10.1097/00005650-199801000-00004

21. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. doi:10.1097/01.mlr.0000182534.19832.83

22. Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299-309. doi:10.1046/j.1365-2710.2002.00430.x

23. Mol PG, Wieringa JE, Nannanpanday PV, et al. Improving compliance with hospital antibiotic guidelines: a time-series intervention analysis. J Antimicrob Chemother. 2005;55(4):550-557. doi:10.1093/jac/dki037

24. Lee HH, Dalesio NM, Lo Sasso AT, Van Cleve WC. Impact of clinical guidelines on revisits after ambulatory pediatric adenotonsillectomy. Anesth Analg. 2018;127(2):478-484. doi:10.1213/ane.0000000000003540

25. Wang KY, Yen CJ, Chen M, et al. Reducing inappropriate lumbar spine MRI for low back pain: radiology support, communication and alignment network. J Am Coll Radiol. 2018;15(1, pt A):116-122. doi:10.1016/j.jacr.2017.08.005

26. Jacobs JC, Jarvik JG, Chou R, et al. Observational study of the downstream consequences of inappropriate MRI of the lumbar spine. J Gen Intern Med. 2020;35(12):3605-3612. doi:10.1007/s11606-020-06181-7

27. Flaherty S, Zepeda ED, Mortele K, Young GJ. Magnitude and financial implications of inappropriate diagnostic imaging for three common clinical conditions. Int J Qual Health Care. 2019;31(9):691-697. doi:10.1093/intqhc/mzy248

28. Blackmore CC, Mecklenburg RS, Kaplan GS. Effectiveness of clinical decision support in controlling inappropriate imaging. J Am Coll Radiol. 2011;8(1):19-25. doi:10.1016/j.jacr.2010.07.009

29. Flynn TW, Smith B, Chou R. Appropriate use of diagnostic imaging in low back pain: a reminder that unnecessary imaging may do as much harm as good. J Orthop Sports Phys Ther. 2011;41(11):838-846. doi:10.2519/jospt.2011.3618

30. Cascade PN. The American College of Radiology: ACR Appropriateness Criteria project. Radiology. 2000;214(suppl):3-46. doi:10.1148/radiology.214.1.r00ja493

31. Nystrom LM, Reimer NB, Dean CW, Bush CH, Scarborough MT, Gibbs CP Jr. Evaluation of imaging utilization prior to referral of musculoskeletal tumors: a prospective study. J Bone Joint Surg Am. 2015;97(1):10-15. doi:10.2106/jbjs.N.00186

32. Wilson RJ, Zumsteg JW, Hartley KA, et al. Overutilization and cost of advanced imaging for long-bone cartilaginous lesions. Ann Surg Oncol. 2015;22(11):3466-3473. doi:10.1245/s10434-014-4325-y

33. Herzlinger RE. Let’s put consumers in charge of health care. Harv Bus Rev. 2002;80(7):44-50, 52-55, 123.

34. Choosing Wisely: A Special Report on the First Five Years. American Board of Internal Medicine Foundation; 2017. Accessed January 17, 2023.https://www.choosingwisely.org/files/Choosing-Wisely-at-Five.pdf

35. Zheng S, Ren ZJ, Heineke J, Geissler KH. Reductions in diagnostic imaging with high deductible health plans. Med Care. 2016;54(2):110-117. doi:10.1097/MLR.0000000000000472

36. Rosenthal MB, Colla CH, Morden NE, et al. Overuse and insurance plan type in a privately insured population. Am J Manag Care. 2018;24(3):140-146.

37. Schwartz AL, Chernew ME, Landon BE, McWilliams JM. Changes in low-value services in year 1 of the Medicare Pioneer accountable care organization program. JAMA Intern Med. 2015;175(11):1815-1825. doi:10.1001/jamainternmed.2015.4525

38. McConnell KJ, Renfro S, Chan BK, et al. Early performance in Medicaid accountable care organizations: a comparison of Oregon and Colorado. JAMA Intern Med. 2017;177(4):538-545. doi:10.1001/jamainternmed.2016.9098

39. Michelotti BF, Mathews A, Chung KC. Appropriateness of the use of magnetic resonance imaging in the diagnosis and treatment of wrist soft tissue injury. Plast Reconstr Surg. 2018;141(2):410-419. doi:10.1097/prs.0000000000004023

40. American Geriatrics Society Expert Panel on the Care of Older Adults With Multimorbidity. Guiding principles for the care of older adults with multimorbidity: an approach for clinicians. J Am Geriatr Soc. 2012;60(10):E1-E25. doi:10.1111/j.1532-5415.2012.04188.x

41. Mafi JN, Russell K, Bortz BA, Dachary M, Hazel WA Jr, Fendrick AM. Low-cost, high-volume health services contribute the most to unnecessary health spending. Health Aff (Millwood). 2017;36(10):1701-1704. doi:10.1377/hlthaff.2017.0385

42. van Beek EJR, Kuhl C, Anzai Y, et al. Value of MRI in medicine: more than just another test? J Magn Reson Imaging. 2019;49(7):e14-e25. doi:10.1002/jmri.26211

43. Emil S, Laberge JM. Canada-trained pediatric surgeons: a cross-border survey of satisfaction and preferences. J Pediatr Surg. 2007;42(5):878-884. doi:10.1016/j.jpedsurg.2006.12.054

44. Cohen E, Rodean J, Diong C, et al. Low-value diagnostic imaging use in the pediatric emergency department in the United States and Canada. JAMA Pediatr. 2019;173(8):e191439. doi:10.1001/jamapediatrics.2019.1439

45. Xu SS, Berkovitz N, Li O, Garvin G. Reduction in inappropriate MRI knee studies after implementation of an appropriateness checklist: experience at a tertiary care centre. Eur J Radiol. 2020;123:108781. doi:10.1016/j.ejrad.2019.108781

46. Goldzweig CL, Orshansky G, Paige NM, et al. Electronic Health Record-Based Interventions for Reducing Inappropriate Imaging in the Clinical Setting: A Systematic Review of the Evidence. US Department of Veterans Affairs; January 2015. Accessed May 22, 2023.https://www.hsrd.research.va.gov/publications/esp/EHR.pdf

47. Brunner MC, Sheehan SE, Yanke EM, et al. Joint design with providers of clinical decision support for value-based advanced shoulder imaging. Appl Clin Inform. 2020;11(1):142-152. doi:10.1055/s-0040-1701256

48. Kini V, Breathett K, Groeneveld PW, et al; American Heart Association Council on Quality of Care and Outcomes Research. Strategies to reduce low-value cardiovascular care: a scientific statement from the American Heart Association. Circ Cardiovasc Qual Outcomes. 2022;15(3):e000105. doi:10.1161/HCQ.0000000000000105

49. Dudzinski DM, Bhatia RS, Mi MY, Isselbacher EM, Picard MH, Weiner RB. Effect of educational intervention on the rate of rarely appropriate outpatient echocardiograms ordered by attending academic cardiologists: a randomized clinical trial. JAMA Cardiol. 2016;1(7):805-812. doi:10.1001/jamacardio.2016.2232

50. Chaudhuri D, Montgomery A, Gulenchyn K, Mitchell M, Joseph P. Effectiveness of quality improvement interventions at reducing inappropriate cardiac imaging: a systematic review and meta-analysis. Circ Cardiovasc Qual Outcomes. 2016;9(1):7-13. doi:10.1161/CIRCOUTCOMES.115.001836

51. Willens HJ, Nelson K, Hendel RC. Appropriate use criteria for stress echocardiography: impact
of updated criteria on appropriateness ratings, correlation with pre-authorization guidelines, and effect of temporal trends and an educational initiative on utilization. JACC Cardiovasc Imaging. 2013;6(3):297-309. doi:10.1016/j.jcmg.2012.11.009

52. Lu YT, Haase SC, Chung TT, Chung KC, Sears ED. The impact of pre-referral advanced diagnostic testing on wait time to see a hand surgeon for common upper-extremity conditions. J Hand Surg Am. 2019;44(12):1013-1020.e2. doi:10.1016/j.jhsa.2019.09.009