Opioid Safety Initiative Associated With Decreased Emergency Department Opioid Prescribing

ABSTRACT

Objectives: The opioid abuse epidemic has focused attention on efforts to decrease opioid prescribing. Although education and feedback interventions are potential levers to affect opioid prescribing, their incremental contribution against a background of declining opioid prescriptions is unclear.

Study Design: We retrospectively evaluated opioid prescribing frequencies after an emergency physician–specific education and feedback initiative in an integrated health care system.

Methods: We evaluated opioid prescriptions for adult patients discharged from 21 emergency departments (EDs) between January 1, 2014, and December 31, 2018. Applying interrupted time series methodology to account for time trends, we analyzed pre- to postintervention changes in prescribing of any opioid and in opioid prescriptions for greater than 20 tablets. We studied all ED visits, visits for back pain and acute extremity fracture, and visits stratified by physicians with high vs low frequency of opioid prescribing. We identified patient, physician, and visit characteristics associated with postintervention ED opioid prescriptions (2018).

Results: Of 1.01 million preintervention and 1.59 million postintervention ED visits, after adjusting for the background trend over time, the intervention was associated with a 3.4% decrease in frequency of opioid prescriptions post intervention (95% CI, –4% to –2.8%), with similar decreases in high-quantity prescriptions (> 20 tablets) and back pain– and acute extremity fracture–related ED visits. Postintervention adjusted analyses indicated no significant association between opioid prescription and race/ethnicity or prior history of opioid abuse.

Conclusions: The ED Opioid Safety Initiative was associated with a near-term decrease in multiple categories of opioid prescribing, including for selected subgroups of common painful conditions.

Am J Manag Care. 2022;28(6):e203-e211. https://doi.org/10.37765/ajmc.2022.89158

Takeaway Points

A health system–wide education and feedback intervention for emergency physicians was associated with reductions in emergency department opioid prescribing.

• Intervention-associated changes were sustained over 19 months.
• Prescribing changes were nuanced across subgroups of higher-prescribing physicians and specific painful conditions.

Opioid prescribing in US emergency departments (EDs) increased in the 1990s and 2000s, driven partly by regulatory and quality guidelines on pain management, with more than 20% of patients discharged from EDs being prescribed opioid pain relievers (OPRs) by 2010.^1-5 Subsequent concern about OPR overprescribing causing an epidemic of opioid dependence and overdoses in part drove a decline in OPR prescriptions.⁵ Emergency physicians (EPs) have a modest role in OPR prescribing, long-term opioid use, and misuse.^6-13 Geographic, clinician-level, and condition-related OPR prescribing variations are prevalent.^7,8,13-17

Interventions to reduce OPR prescribing and drive consistent practice (eg, prescription data monitoring programs, prescribing defaults, education and feedback) may be associated with OPR prescribing reductions, but many studies do not distinguish intervention-associated changes from temporal trends, or they demonstrate mixed effects.^18-26 Health systems may invest in measures to combat the opioid epidemic, but evaluating interventions’ impact is important, especially for future resource allocation.²² Thus, evaluating an ED-targeted education and feedback intervention in the context of declining OPR prescribing would be informative.

Kaiser Permanente Northern California (KPNC) launched an Opioid Safety Initiative for all EPs to promote OPR prescribing best practices in 2016. We evaluated the association of the Opioid Safety Initiative with EP OPR prescribing changes using interrupted time series (ITS) analyses overall, for specific painful conditions, and by EP OPR prescribing frequency. We hypothesized that after the Opioid Safety Initiative, frequency of ED OPR prescribing would decline overall and more steeply, especially among EPs with higher baseline OPR prescribing.

METHODS

Study Setting

We conducted a retrospective cohort study of adult ED visits from January 1, 2014, to December 31, 2018, in the 21 medical centers of KPNC, a private, not-for-profit integrated health care system covering 4 million patients (approximately 33% of the region’s insured population); its members are socioeconomically, racially, and ethnically diverse and similar to the surrounding population in health status.²⁷ Board-certified or -eligible EPs staff the EDs. The KPNC Institutional Review Board granted a waiver of informed consent for this Health Insurance Portability and Accountability Act–compliant study of deidentified aggregate data.

Opioid Safety Initiative

The ED Opioid Safety Initiative consisted of education and OPR prescribing dashboards. Voluntary education occurred over 5 months from January 1, 2016, to May 31, 2016 (intervention period), with 96% of staff EPs participating in a combination of self-study and live lecture modules describing best practices in ED opioid prescribing (optimizing duration, strength, and dose of opioid medications), limiting opioids for non–opioid-sensitive chronic conditions (eg, back pain, migraine), and opioid alternatives. This education was specific to the ED, and it aligned with 2012 emergency medicine professional society recommendations and non–ED-targeted CDC chronic pain management guidelines (2016) to prescribe no more than 20 tablets (3 days’ supply) for acute pain.^28,29 Prior to the study, electronic health record (EHR) prescription defaults and a statewide prescription drug database were in place. In October 2018 (near the end of the observation period), a requirement to consult the statewide prescription drug database was instituted, but ED prescriptions for less than 7 days’ supply were exempted. Payer-imposed quantity limits were not in use at any point. Following the Opioid Safety Initiative education, department champions distributed monthly anonymized ED OPR prescribing dashboards and best practice reminders. Non-ED physicians in the health care system completed separate opioid safety training.³⁰ Because the education was delivered to all health system EPs, there is no contemporaneous control group.

Study Design

We evaluated population-level changes in OPR prescribing associated with the Opioid Safety Initiative. To discern the strength of the intervention against the backdrop of nationally and regionally decreasing OPR prescribing, we used ITS, an analytic method that ascertains intervention-associated changes while accounting for temporal trends when a control group is unavailable.³¹ ITS is among the strongest methods to analyze real-world natural experiments, when randomized controlled trials are not available, in making use of time series data while accounting for baseline time trends.³² The study period was divided into the preintervention baseline (January 1, 2014, to December 31, 2015); intervention (January 1, 2016, to May 31, 2016); and postintervention (June 1, 2016, to December 31, 2018) periods. All ED visits by patients older than 17 years discharged home from the ED, with health plan membership in at least 9 of the 12 months prior to the eligible ED visit (for data completeness), were included. Admitted patients were excluded.

Measures

All arenas of care (inpatient, outpatient, emergency) utilize a single integrated EHR that includes clinical documentation and comprehensive pharmacy, laboratory, and imaging data (Epic). Study data were electronically extracted from the EHR and other administrative databases.

Outcomes

The primary study outcome was discharge OPR prescription frequency; the secondary outcome was frequency of discharge OPR prescription for more than 20 tablets (3 days’ supply). The following medications were included: hydrocodone, oxycodone, tramadol, methadone, oxymorphone, codeine, fentanyl, morphine, and hydromorphone, including combinations with ibuprofen and/or acetaminophen. Cough and cold preparations containing codeine were excluded.

Variables

We electronically extracted patient characteristics and clinical covariates including demographics (age, sex, self-reported ethnicity or race, primary non-English language preference, low socioeconomic status based on census tract median income and education), visit acuity, Charlson Comorbidity Index (CCI) score, recent opioid prescriptions (within 180 days prior to eligible ED visit), history of opioid abuse or misuse (International Classification of Diseases, Ninth Revision [ICD-9] or Tenth Revision [ICD-10] codes), parenteral opioid administration during ED visit, facility, and timing of visit outside of business hours (Monday-Friday 8:30 am-5:30 pm).

Physician assignment to patients is random, and all EPs see a variety of patients and conditions. To avoid handoff-related crossover, we included only ED encounters with the same attending and prescribing EP, excluding less than 3% of ED visits with discordant attending and prescribing EPs. All OPR prescriptions, whether filled by the patient or not, were included.

Subgroup Analyses

We evaluated 2 sets of subgroups: condition specific, hypothesizing that OPR prescriptions might change differently in chronically vs acutely painful conditions (back pain and acute extremity fracture, respectively, identified by ICD-9 and ICD-10 codes), and among higher- vs lower-prescribing EPs. Condition-specific subgroups allowed analysis of prescribing changes given the possibility that decreased opioid prescribing may have affected the proportion of acute pain or pain-medication–seeking ED visits.

Among EPs who practiced regularly across the full study period (at least 50 patient encounters per year each year of the study period), we assigned each a baseline prescribing quartile (frequency of any OPR prescription per 1000 ED visits attended during the preintervention period). We evaluated pre- to postintervention OPR prescribing (primary outcome) for the highest vs lowest quartile prescribers.

Data Analysis

We described the outcome and variables of interest with univariate analysis for patient, physician, and facility characteristics, comparing frequencies of these covariates pre- vs post intervention. Continuous variables were reported as means (with SDs). Chi-square tests were applied to dichotomous variables, reported as frequencies.

Using ITS methodology, we evaluated changes in OPR prescribing associated with the Opioid Safety Intervention (eAppendix A [eAppendices available at ajmc.com]). ITS uses measurements derived from regression models to evaluate whether the impact of an intervention against the background of an already decreasing or increasing frequency of the examined outcome is statistically different from that which have would occurred if the background trend had progressed without the overlay of an intervention.³¹ A statistically significant jump in mean value of the outcome post intervention compared with preintervention would support the conclusion that the intervention had an effect separate from the preexisting trend in the outcome frequency. Similarly, a statistically significant difference in the slope post intervention compared with preintervention would provide evidence for the intervention’s impact in changing the frequency of the outcome (OPR prescription) at a more rapid rate than would occur without the intervention.

When utilized without a separate nonintervention comparison group, ITS has limitations, specifically if there was a shift in the study population characteristics associated with the preintervention outcome vs the postintervention outcome, or a change in background trend simultaneous with but unrelated to the intervention. In this study, subpopulations (back pain and acute fractures) occurring at a stable rate over the study period were selected for evaluation. Finally, KPNC membership gradually increased over time; however, no major changes in membership composition were observed between the preintervention and postintervention periods.

We separately modeled monthly frequencies of the primary and secondary outcomes (any OPR prescription and OPR prescription >20 tablets) over the study period and compared rates of change from preintervention to post intervention. We calculated the level change and slope change in OPR prescribing from pre- to post intervention, excluding the 5-month implementation period. We constructed separate ITS for the entire study cohort and for subgroups (back pain, extremity fracture, and highest- vs lowest-quartile OPR-prescribing EPs). ITS models for the secondary outcome were constructed for the entire study cohort and the back pain and extremity fracture subgroups.

Separately, for 2018, the final year of observation, we constructed a regression model to evaluate patient and visit characteristics associated with primary outcome, adjusting for facility, season of ED visit, and mean CCI score. By the last year of the study, OPR prescriptions for more than 20 tablets were so infrequent (< 0.5% of ED visits) that rarity of the secondary outcome limited analysis. Missingness of data was treated as follows: Patient characteristics and diagnoses were considered present if they were coded in the EHR. Absence of a characteristic or variable was considered evidence of absence of that condition. In the regression model, 63 of 643,310 observations were dropped because of missing demographic variables (eg, age, sex).

Analyses were performed using SAS 9.4 (SAS Institute).

RESULTS

Study Population

We studied 2,600,856 total study-eligible ED visits, including 1,013,130 ED visits in the preintervention study period and 1,587,726 ED visits in the postintervention study period. Patient and visit characteristics are generally comparable in the pre- and postintervention periods (Table 1). Over the study period, 1830 EPs attended eligible ED visits. Preintervention, 12,052 back pain and 24,803 acute extremity fracture ED visits (1.19% and 2.45% of total ED visits, respectively) were identified. Post intervention, 19,996 ED visits (1.25%) were for back pain and 37,555 (2.37%) were for acute extremity fracture.

Overall Trends in OPR Prescribing

Prior to the intervention, OPR prescribing was declining by 0.069% per month. The intervention was associated with a statistically significant 3.4% sudden decrease in OPR prescriptions (95% CI, –4% to –2.8%) and almost a doubling of the preintervention slope of decrease in OPR prescribing over time (preintervention decreases of –0.069% per month vs postintervention decreases of –0.135% per month; 95% CI, –0.105% to –0.027%) (Figure 1 and eAppendix B). For OPR prescriptions of more than 20 tablets (secondary outcome), we also observed a statistically significant postintervention level change (–0.530%; 95% CI, –0.619% to –0.441%) associated with the intervention (Figure 2 and eAppendix B) but no significant change in slope of decline.

Subgroup Analyses

In separate ITS analyses of the primary outcome (overall OPR prescriptions) for back pain and acute extremity fracture ED visits, the intervention was associated with a statistically significant sudden decrease in OPR prescriptions of 9.42% (95% CI, –12.20% to –6.64%) for back pain and 6.06% (95% CI, –8.54% to –3.59%) for acute extremity fracture (Figure 3 and eAppendix B). We also observed a statistically significantly steeper downward slope in back pain OPR prescriptions than in the preintervention period (–0.270% per month post intervention vs –0.065% per month preintervention; 95% CI, –0.453% to –0.086% per month) (eAppendix B).

Baseline OPR prescribing quartile was assigned to 569 eligible EPs, with lowest vs highest EP OPR prescribing quartile frequencies at 19.4% vs 21.8%, respectively. Among both highest- and lowest-quartile EPs, the intervention was associated with a near-term decrease in OPR prescribing (highest quartile: –3.43%; 95% CI, –4.20% to –2.66%; lowest quartile: –3.19%; 95% CI, –3.95% to –2.43%) (eAppendix B).

Characteristics of Patients Receiving OPR Prescriptions Post Intervention

In 2018, the final study year, multivariable analysis indicated that male sex, younger age, ED visit during business hours, and history of OPR prescription within 180 days prior to index visit were associated with lower likelihood of receiving any OPR prescription at ED discharge (Table 2). Race or ethnicity, prior diagnosis of opioid abuse or misuse, low socioeconomic status, and primary non-English language were not significantly associated with higher or lower likelihood of ED OPR prescription.

DISCUSSION

In evaluating a large health system’s EP Opioid Safety Initiative, we found that the intervention was associated with a statistically significant 3.4% near-term decrease in opioid prescriptions at ED discharge. Across all subgroups, the intervention was associated with statistically significant near-term decreases in opioid prescribing, with a statistically significant near doubling of the preexistent prescribing decline for overall ED visits and a larger decline for back pain visits. In the final year of observation, disparities in likelihood of opioid prescription by race or ethnicity, low socioeconomic status, or primary non-English language preference were not observed.

Previous evaluations have concluded that tailored, multifaceted interventions decrease opioid prescribing, but distinguishing intervention-associated changes from secular trends external to the study setting is challenging, especially given that secular trends in prescribing likely also capture unstudied interventions.^{19,20,22,24,33,34} Population-level ITS studies describe similar postintervention decreases in opioid prescribing after rollout of an opioid safety dashboard35 and publication of CDC pain guidelines²³ in ambulatory settings. These interventions may collectively be a lever driving national trends of declining opioid prescribing while their local implementation may yield observable changes at the point of uptake. The current study did not compare internal rates of OPR prescribing with national or local rates, but instead measured intervention-related changes from the preintervention baseline trend within the study setting. Although directly comparable measures are not available, our study’s preintervention 21% rate of OPR prescribing was very generally comparable with the national 22% rate in the same period, and we noted a 13% rate of OPR prescriptions in the last month of observation compared with the nationally reported rate at 15% in 2016-2017.⁴ Elsewhere, statewide opioid use guideline implementation alone was associated with a small and nonselective near-term decrease in ED opioid prescribing without a trend change.²⁶ Our finding of a near-term decrease and accelerated decline in OPR prescribing associated with implementation of a specific health system’s specific opioid prescribing initiative supports the hypothesis that measures to facilitate uptake of guidelines may contribute to sustained and specific changes in clinical practice that underlie national and local trends.

We evaluated specific painful conditions with stable ED visits over time and physician prescribing quartiles. The intervention-associated accelerated trend of decline in OPR prescriptions for back pain ED visits is encouraging given sparse evidence for opioid efficacy in back pain and the relatively high frequency of OPR prescriptions for back pain ED visits—up to 50% in some settings.^29,36-38 High vs low OPR–prescribing EPs were closer at baseline (19.4% vs 21.8%) relative to similar studies describing 3-fold differences between high- and low-frequency EP prescribers, and we observed a drop in prescribing associated with the intervention for both groups.^7,16 The varied subgroup findings suggest that the era of a one-size-fits-all approach to opioid prescribing interventions is past.

Finally, we evaluated patient characteristics that may influence OPR prescribing. Previous studies describe association of older age, lower socioeconomic status, uninsured status, and White race/ethnicity with higher likelihood of OPR prescription.^2,34,39-41 We similarly observed that younger age was associated with decreased likelihood of OPR prescription, possibly related to public health attention to opioid dependence and overdose. However, low socioeconomic status, non-English language, non-White race/ethnicity, and opioid use disorder were not significant predictors of ED OPR prescription post intervention in the study setting. The absence of observed racial/ethnic and socioeconomic disparities in OPR prescribing odds may reflect the study population’s insured status and membership in an integrated health care system with complete health records, strong outpatient follow-up access, and dissemination of consistent OPR prescribing guidelines in the Opioid Safety Initiative.

Limitations

Although concurrent events during the study period, such as the publication of CDC guidelines for chronic pain management, may also have influenced opioid prescribing, they were not ED specific.²⁸ The American College of Emergency Physicians clinical policy on opioid prescribing was not updated during the study period.²⁹ Separately, although a statewide prescription database monitoring requirement was added near the end of the observation period, it does not overlap substantially with our study intervention and setting, as ED prescriptions of fewer than 7 days’ supply were exempt from the requirement.⁴² Beyond these, lawsuits against opioid manufacturers were publicized after 2019; however, ongoing litigation through the study period may have factored in the overall downward trend in prescribing noted both before and after the intervention. Timely research of opioid prescribing patterns in the ED should be interpreted in the context of this rapidly changing clinical and policy area.

Although the general scale of identified declines in opioid prescribing is loosely comparable with various published national observations, whether the study-identified adjusted intervention-related declines would be noted in the absence of the intervention is impossible to determine without a randomized controlled trial, which would be infeasible. As is common in observational real-world natural experiments, the studied Opioid Safety Initiative was implemented across the entire study health system simultaneously and there was no internal nonintervention control group available. Accordingly, we applied ITS analysis to distinguish intervention-associated prescribing changes from background trends within the study setting. Importantly, the study pre- and postintervention slopes are interpretable only relative to each other and within the study setting—these findings may not be directly generalizable to external health care settings with differing baseline trends. Still, although national trends of decline in opioid prescribing are generally consistent with the downward slopes identified in the preintervention period (< 2% over 24 months), we observed a significant near-term decrease (3.4%) in association with the intervention itself, absent concurrent interventions in the study time frame in our integrated system or ED-specific national policy changes.

Because the interventions were integrated, we could not separately assess the education and feedback interventions to understand their individual contributions to OPR prescribing. Additionally, as this is an observational study, unmeasured confounders could have contributed to the observed effects, but we applied statistical methods that evaluate any changes associated with the intervention in the context of background trends. Restricting inclusion to health plan membership improved data completeness but excluded uninsured patients. We examined ordered prescriptions to evaluate physician behavior but not appropriateness or consumption of OPR prescriptions. We could not evaluate if visits for acute pain or drug-seeking behavior varied over the study; however, our condition-specific subgroups comprised stable proportions of total ED visits in the pre- and postintervention periods while total ED volume was stable or increased. We did not exclude patients who may have been prescribed opioids at the end of life, but we do not suspect that these visits changed substantially during the study, nor is ED management of end-of-life pain common in our setting. Despite these limitations, we were able to evaluate population-level OPR prescribing trends associated with an opioid prescribing intervention.

CONCLUSIONS

In an integrated health care system, we observed a significant association between initiation of an opioid education and feedback initiative and near-term decreases in opioid prescribing. These changes were followed by a significantly steeper decline in opioid prescribing over the subsequent 18 months compared with the preintervention period. Although the nonrandomized design cannot prove causality, the integrated nature of the health care system analyzed, lack of other specific concordant interventions, and use of statistical techniques demonstrating that these observed changes were unlikely to be due to random chance and were beyond expected temporal changes are consistent with a beneficial effect of the intervention on prescribing patterns. Although we cannot exclude unmeasured confounders as a contributor to all or part of the effects seen, these findings are relevant to other settings seeking to evaluate methods for translating guidelines into clinical practice.

Acknowledgments

The authors wish to thank Cynthia Campbell, PhD, for her consultation on study design and variables as well as critical appraisal of the manuscript and results. They also wish to thank Adina Rauchwerger, MPH, for editing of the manuscript.

Author Affiliations: The Permanente Medical Group (MK, SB, SVA), Oakland, CA; Department of Emergency Medicine, Kaiser Foundation Hospital (MK), Fremont, CA; Department of Emergency Medicine, Kaiser Foundation Hospital (SB), Redwood City, CA; Kaiser Permanente Division of Research (AE, MER), Oakland, CA.

Source of Funding: The Permanente Medical Group’s Division of Applied Research and Evaluation supported this project.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (MK, SB, SVA, MER); acquisition of data (MK, AE); analysis and interpretation of data (MK, AE, MER); drafting of the manuscript (MK, MER); critical revision of the manuscript for important intellectual content (MK, AE, SVA); statistical analysis (MK, AE); provision of patients or study materials (MK, SB); obtaining funding (MK, SVA, MER); administrative, technical, or logistic support (SB, MER); and supervision (SVA).

Address Correspondence to: Mamata Kene, MD, MPH, The Permanente Medical Group, 39400 Paseo Padre Pkwy, Fremont, CA 94538. Email: Mamata.V.Kene@kp.org.

REFERENCES

1. Gleber R, Vilke GM, Castillo EM, Brennan J, Oyama L, Coyne CJ. Trends in emergency physician opioid prescribing practices during the United States opioid crisis. Am J Emerg Med. 2020;38(4):735-740. doi:10.1016/j.ajem.2019.06.011

2. Kea B, Fu R, Lowe RA, Sun BC. Interpreting the National Hospital Ambulatory Medical Care Survey: United States emergency department opioid prescribing, 2006-2010. Acad Emerg Med. 2016;23(2):159-165. doi:10.1111/acem.12862

3. Pezalla EJ, Rosen D, Erensen JG, Haddox JD, Mayne TJ. Secular trends in opioid prescribing in the USA. J Pain Res. 2017;10:383-387. doi:10.2147/JPR.S129553

4. Rui P, Santo L, Ashman JJ. Trends in opioids prescribed at discharge from emergency departments among adults: United States, 2006-2017. Natl Health Stat Report. 2020;(135):1-12.

5. Phillips DM. JCAHO pain management standards are unveiled. JAMA. 2000;284(4):428-429. doi:10.1001/jama.284.4.423b

6. Axeen S, Seabury SA, Menchine M. Emergency department contribution to the prescription opioid epidemic. Ann Emerg Med. 2018;71(6):659-667.e3. doi:10.1016/j.annemergmed.2017.12.007

7. Barnett ML, Olenski AR, Jena AB. Opioid-prescribing patterns of emergency physicians and risk of long-term use. N Engl J Med. 2017;376(7):663-673. doi:10.1056/NEJMsa1610524

8. Friedman BW, Ochoa LA, Naeem F, et al. Opioid use during the six months after an emergency department visit for acute pain: a prospective cohort study. Ann Emerg Med. 2020;75(5):578-586. doi:10.1016/j.annemergmed.2019.08.446

9. Lyapustina T, Castillo R, Omaki E, et al. The contribution of the emergency department to opioid pain reliever misuse and diversion: a critical review. Pain Pract. 2017;17(8):1097-1104. doi:10.1111/papr.12568

10. Meisel ZF, Lupulescu-Mann N, Charlesworth CJ, Kim H, Sun BC. Conversion to persistent or high-risk opioid use after a new prescription from the emergency department: evidence from Washington Medicaid beneficiaries. Ann Emerg Med. 2019;74(5):611-621. doi:10.1016/j.annemergmed.2019.04.007

11. Menchine MD, Axeen S, Plantmason L, Seabury S. Strength and dose of opioids prescribed from US emergency departments compared to office practices: implications for emergency department safe-prescribing guidelines. Ann Emerg Med. 2014;64(4 suppl):S1. doi:10.1016/j.annemergmed.2014.07.026

12. Butler MM, Ancona RM, Beauchamp GA, et al. Emergency department prescription opioids as an initial exposure preceding addiction. Ann Emerg Med. 2016;68(2):202-208. doi:10.1016/j.annemergmed.2015.11.033

13. Hoppe JA, Kim H, Heard K. Association of emergency department opioid initiation with recurrent opioid use. Ann Emerg Med. 2015;65(5):493-499.e4. doi:10.1016/j.annemergmed.2014.11.015

14. Hoppe JA, McStay C, Sun BC, Capp R. Emergency department attending physician variation in opioid prescribing in low acuity back pain. West J Emerg Med. 2017;18(6):1135-1142. doi:10.5811/westjem.2017.7.33306

15. Zhu W, Chernew ME, Sherry TB, Maestas N. Initial opioid prescriptions among U.S. commercially insured patients, 2012-2017. N Engl J Med. 2019;380(11):1043-1052. doi:10.1056/NEJMsa1807069

16. Ward MJ, Kc D, Jenkins CA, Liu D, Padaki A, Pines JM. Emergency department provider and facility variation in opioid prescriptions for discharged patients. Am J Emerg Med. 2019;37(5):851-858. doi:10.1016/j.ajem.2018.07.054

17. Guy GP Jr, Zhang K, Bohm MK, et al. Vital signs: changes in opioid prescribing in the United States, 2006-2015. MMWR Morb Mortal Wkly Rep. 2017;66(26):697-704. doi:10.15585/mmwr.mm6626a4

18. Chiu AS, Jean RA, Hoag JR, Freedman-Weiss M, Healy JM, Pei KY. Association of lowering default pill counts in electronic medical record systems with postoperative opioid prescribing. JAMA Surg. 2018;153(11):1012-1019. doi:10.1001/jamasurg.2018.2083

19. Losby JL, Hyatt JD, Kanter MH, Baldwin G, Matsuoka D. Safer and more appropriate opioid prescribing: a large healthcare system’s comprehensive approach. J Eval Clin Pract. 2017;23(6):1173-1179. doi:10.1111/jep.12756

20. Meisenberg BR, Grover J, Campbell C, Korpon D. Assessment of opioid prescribing practices before and after implementation of a health system intervention to reduce opioid overprescribing. JAMA Netw Open. 2018;1(5):e182908. doi:10.1001/jamanetworkopen.2018.2908

21. Rutkow L, Chang HY, Daubresse M, Webster DW, Stuart EA, Alexander GC. Effect of Florida’s prescription drug monitoring program and pill mill laws on opioid prescribing and use. JAMA Intern Med. 2015;175(10):1642-1649. doi:10.1001/jamainternmed.2015.3931

22. Weiner SG, Price CN, Atalay AJ, et al. A health system–wide initiative to decrease opioid-related morbidity and mortality. Jt Comm J Qual Patient Saf. 2019;45(1):3-13. doi:10.1016/j.jcjq.2018.07.003

23. Bohnert ASB, Guy GP Jr, Losby JL. Opioid prescribing in the United States before and after the Centers for Disease Control and Prevention’s 2016 opioid guideline. Ann Intern Med. 2018;169(6):367-375. doi:10.7326/M18-1243

24. Smalley CM, Willner MA, Muir MR, et al. Electronic medical record-based interventions to encourage opioid prescribing best practices in the emergency department. Am J Emerg Med. 2020;38(8):1647-1651. doi:10.1016/j.ajem.2019.158500

25. Andereck JW, Reuter QR, Allen KC, et al. A quality improvement initiative featuring peer-comparison prescribing feedback reduces emergency department opioid prescribing. Jt Comm J Qual Patient Saf. 2019;45(10):669-679. doi:10.1016/j.jcjq.2019.07.008

26. Sun BC, Lupulescu-Mann N, Charlesworth CJ, et al. Impact of hospital “best practice” mandates on prescription opioid dispensing after an emergency department visit. Acad Emerg Med. 2017;24(8):905-913. doi:10.1111/acem.13230

27. Gordon N, Lin T. The Kaiser Permanente Northern California adult member health survey. Perm J. 2016;20(4):15-225. doi:10.7812/TPP/15-225

28. Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain — United States, 2016. MMWR Recomm Rep. 2016;65(1):1-49. doi:10.15585/mmwr.rr6501e1

29. Cantrill SV, Brown MD, Carlisle RJ, et al; American College of Emergency Physicians Opioid Guideline Writing Panel. Clinical policy: critical issues in the prescribing of opioids for adult patients in the emergency department. Ann Emerg Med. 2012;60(4):499-525. doi:10.1016/j.annemergmed.2012.06.013

30. Awsare S, Havens C, Lippi J. Facing the opioid crisis: practical, effective actions we can take. Gastroenterology. 2019;156(6):1538-1541. doi:10.1053/j.gastro.2018.12.035

31. Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017;46(1):348-355. doi:10.1093/ije/dyw098

32. Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299-309. doi:10.1046/j.1365-2710.2002.00430.x

33. Kline TV, Savage RL, Greenslade JH, Lock CL, Pattullo C, Bell AJ. Affecting emergency department oxycodone discharge prescribing: an educational intervention. Emerg Med Australas. 2019;31(4):580-586. doi:10.1111/1742-6723.13261

34. Smith BC, Vigotsky AD, Apkarian AV, Schnitzer TJ. Temporal factors associated with opioid prescriptions for patients with pain conditions in an urban emergency department. JAMA Netw Open. 2020;3(3):e200802. doi:10.1001/jamanetworkopen.2020.0802

35. Lin LA, Bohnert ASB, Kerns RD, Clay MA, Ganoczy D, Ilgen MA. Impact of the Opioid Safety Initiative on opioid-related prescribing in veterans. Pain. 2017;158(5):833-839. doi:10.1097/j.pain.0000000000000837

36. Abdel Shaheed C, Maher CG, Williams KA, Day R, McLachlan AJ. Efficacy, tolerability, and dose-dependent effects of opioid analgesics for low back pain: a systematic review and meta-analysis. JAMA Intern Med. 2016;176(7):958-968. doi:10.1001/jamainternmed.2016.1251

37. Jauregui JE, Nutt RJ, Margolis AM. Frequency of opioid prescribing for acute low back pain in a rural emergency department. Adv Emerg Nurs J. 2020;42(3):210-214. doi:10.1097/TME.0000000000000310

38. Jeffery MM, Hooten WM, Hess EP, et al. Opioid prescribing for opioid-naive patients in emergency departments and other settings: characteristics of prescriptions and association with long-term use. Ann Emerg Med. 2018;71(3):326-336.e19. doi:10.1016/j.annemergmed.2017.08.042

39. Sinnenberg LE, Wanner KJ, Perrone J, Barg FK, Rhodes KV, Meisel ZF. What factors affect physicians’ decisions to prescribe opioids in emergency departments? MDM Policy Pract. 2017;2(1):2381468316681006. doi:10.1177/2381468316681006

40. Bedene A, Lijfering WM, Niesters M, et al. Opioid prescription patterns and risk factors associated with opioid use in the Netherlands. JAMA Netw Open. 2019;2(8):e1910223. doi:10.1001/jamanetworkopen.2019.10223

41. Friedman J, Kim D, Schneberk T, et al. Assessment of racial/ethnic and income disparities in the prescription of opioids and other controlled medications in California. JAMA Intern Med. 2019;179(4):469-476. doi:10.1001/jamainternmed.2018.6721

42. Becerra X. Cures 2.0 certification. California Office of the Attorney General. April 9, 2018. Accessed February 9, 2021. https://oag.ca.gov/sites/all/files/agweb/pdfs/pdmp/cures-advisory-memo.pdf