A Randomized, Pragmatic, Pharmacist-Led Intervention Reduced Opioids Following Orthopedic Surgery

ABSTRACT

Objectives: To determine whether a pharmacist-led, patient-directed intervention can reduce opioid use following total hip arthroplasty (THA) or total knee arthroplasty (TKA).

Study Design: Randomized trial.

Methods: Patients scheduled to undergo THA or TKA (during 2015 and 2016) were randomized to usual care or intervention. We ranked patients according to predicted risk of persistent opioid use and selected the top 60% for inclusion (n = 561); all contributed to the analysis. Intervention patients were mailed materials 2 weeks before and after surgery, plus they received telephone intervention from specially trained pharmacists if they filled opioid prescriptions in the 28 to 90 days following surgery. Our primary outcome was the dispensed morphine equivalents (DME) in the 90 days following surgery, modeled using a natural log transformation.

Results: A total of 561 patients were randomized (286 usual care, 118 THA and 168 TKA; 275 intervention, 107 THA and 168 TKA); the mean age was 66 years, and 60% were female. Overall, we found no meaningful reduction in DME for intervention versus usual care (geometric mean ratio, 0.92 [95% CI, 0.69-1.21]). However, there was effect modification by whether the patient had TKA or THA (interaction P <.01). Those undergoing THA in the intervention group used significantly less DME than did those undergoing THA in the usual care group (geometric mean ratio, 0.52 [95% CI, 0.33-0.82]).

Conclusions: Our pharmacist-led, patient-directed intervention to reduce opioid use demonstrated a reduction in opioid dispensings in the 90 days following THA but not TKA.

Am J Manag Care. 2018;24(11):515-521Takeaway Points

• This randomized pragmatic trial showed that a pharmacist-led, patient-directed intervention demonstrated a reduction in opioid dispensings in the 90 days following hip replacement but not knee replacement.
• We found little evidence that reductions in opioid dispensings from our health plan resulted in patients seeking opioid prescriptions outside of the health plan.
• Our intervention was designed to be incorporated into existing clinical workflow, particularly in integrated care settings.

Opioid use has escalated greatly in the United States in recent years, from about 100 million prescriptions filled in 1992 to nearly 250 million in 2015.¹ This escalation has been coincident with a 14% increase in opioid overdose fatalities.² After primary care, orthopedic surgery generates the highest rates of opioid prescriptions among adults.³ Many patients use opioids both before and after surgery.⁴

Opioids clearly have a role in managing pain for orthopedic surgery patients, but evidence suggests that exposure should be kept as short-term as possible.⁵ Experts have noted that a key part of minimizing opioid exposure is for the orthopedic surgical team to set patient expectations for pain and its treatment.⁶ The American Academy of Orthopaedic Surgeons also recognized that making opioids the focus of pain management has led to unintended consequences and that research and education are needed for physicians, caregivers, and patients.⁷

In this article, we present the main outcomes for the FLOOD (FDA: Lowering Orthopaedic Opioid Dosing) study, a patient-level, pragmatic, randomized trial focused on patients undergoing total knee arthroplasty (TKA) and total hip arthroplasty (THA). We were interested in whether the promotion of targeted, specially designed, patient-focused education on opioid use and patient expectations for pain control, delivered around the time of surgery, would lead to reduced opioid use.

METHODS

Intervention Design and Funding

FLOOD was a 2-arm, randomized, pragmatic clinical trial funded by the FDA (FDA contract number: HHSF223201400146C) and registered with ClinicalTrials.gov (NCT02576392).

Setting

The setting for the trial was Kaiser Permanente Northwest (KPNW), a health maintenance organization serving about 580,000 members in northwest Oregon and southwest Washington. The local institutional review board reviewed and approved the study, including a waiver of informed consent. Study data were collected and managed using REDCap (Research Electronic Data Capture) tools hosted at KPNW’s Center for Health Research.⁸

Participant Selection and Randomization

Using the electronic health record (EHR), we identified, on a weekly basis, patients 20 years and older who were scheduled to undergo TKA or THA in the subsequent 14 to 21 days. Consistent with pragmatic trial principles, exclusions were minimized and limited to patients (1) already enrolled in a pain trial, (2) with less than 1 year of baseline KPNW membership (necessary to characterize the cohort), or (3) with a history of KPNW Pain Clinic visits. This resulted in 131 patients being excluded. Once patients were identified, we used a multivariable prediction model⁹ that we developed to rank them according to their risk of becoming persistent opioid users in the 90 to 180 days post surgery; we enrolled patients with a predicted risk in the top 60%.

Computer-generated randomization assignments were used within each weekly cohort to allocate patients (simple randomization; no blocking or stratification was used) in a 1:1 ratio to the intervention or usual care arm. Allocation was concealed from the investigators and clinicians providing care, although the pharmacists delivering the intervention were not blinded. We enrolled 561 patients in the study. A total of 495 patients were required to achieve 85% power to detect a 20% relative reduction in the primary outcome: a difference in means for the natural log-transformed dispensed morphine equivalents (DME) over the first 90 days (ie, difference between the trial arms). We assumed a 2-sided significance level of 5% and that 10% of patients would be nonadherent to the intervention, and we therefore increased the size to accommodate the dilution of effect using Lachin’s formula.¹⁰

The study enrollment period was July 15, 2015, to April 25, 2016. Intervention and outcome assessment continued through July 2016.

Study Intervention

The study design diagram is shown in Figure 1. Usual care participants had access to the full range of services for TKA and THA candidate patients at KPNW. These services included handouts and a class in preparation for surgery that advised patients on the risks and benefits of surgery, pain control measures, exercise recommendations, and the need for postsurgical assistance.

Intervention patients had access to all usual healthcare services. They also were eligible for an intervention that consisted of 3 parts: (1) About 10 days prior to surgery, a mailed brochure described what patients should expect regarding opioid use and pain control after surgery; (2) about 15 days following THA/TKA, another brochure was mailed that explained opioid use topics, including rationale for opioid use following surgery, opioid tapering expectations following surgery, and potential adverse effects of opioids; and (3) if patients filled a prescription for an opioid in the 28 to 90 days following surgery, they received a follow-up telephone call from a pharmacist who used motivational enhancement principles¹¹ to reinforce the information in the brochure and allow patients the opportunity to discuss clinical issues related to medications and pain control. If intervention patients were hospitalized for any reason during follow-up, we ceased all intervention activities, because the clinical courses of those patients were altered. However, their outcomes were still included in our intent-to-treat analysis.

The mailed materials were developed by our research team using qualitative methods, with input from patients and orthopedic clinicians.¹² The pharmacists who delivered the intervention were part of a KPNW team focused on working with patients receiving high-dose opioids to gradually taper their opioid consumption. These pharmacists had specific training in the use of opioids and in patient communication, including motivational interviewing. The brochures and other materials that we developed, including a series of vignettes to illustrate the conversations that intervention pharmacists might have with patients, are freely available.

Follow-up

Patients were followed for outcome assessment for 90 days, starting with the day of hospital discharge following surgery. All randomized patients were included in our primary (intent-to-treat) analysis. We also undertook a secondary per-protocol analysis. The per-protocol analysis excluded both intervention and usual care patients whose surgeries were cancelled and not rescheduled within a time frame that would have allowed for complete follow-up, and it also excluded patients who were hospitalized for any reason during the 90-day follow-up.

Study Measurements

The primary outcome was the total dispensing of opioid medications in the 90 days following surgery (including prescription claims from non-KP pharmacies), expressed as DME. We preferentially used the CDC recommendations¹³ for converting opioids to DME, supplemented with other sources for missing products.^14,15

Secondary outcomes over the 90 days following surgery were the (1) count of opioid dispensings; (2) count of dispensings for nonopioid pain-related medications, including nonsteroidal anti-inflammatory drugs (NSAIDs), sedative hypnotics, anticonvulsants, antianxiety agents, and antidepressants (including “opioid-sparing” agents); (3) count of face-to-face office visits; (4) count of telephone encounters; (5) count of email encounters; (6) count of physical therapy and occupational therapy visits (separate and combined); (7) count of KPNW emergency department (ED)/urgent care visits; and (8) count of ED visits at non-KPNW facilities.

Covariates

We compared the randomized groups on age, sex, race, surgical type, predicted probability of long-term opioid use, and the variables in the opioid prediction model using standardized differences. All data on comorbid conditions and use of opioids and other services were gathered from KPNW’s EHR (lists of International Classification of Diseases, Ninth Revision and International Classification of Diseases, Tenth Revision codes used to identify comorbid conditions available upon request).

Use of Non-KPNW Pharmacies

We linked the KPNW opioid dispensing data (from Oregon pharmacies) with the state of Oregon’s prescription drug monitoring program (PDMP) to examine whether patients in the study arms differentially filled opioid prescriptions at non-KPNW pharmacies. We calculated the number of excess fills found in the PDMP data.

Statistical Analysis

Because of the anticipated right skewness, our a priori analytic plan specified transformation of morphine equivalents to the natural log scale; visual inspection confirmed normality of the transformed data. We also plotted the cumulative DME by group and TKA/THA. For continuous outcomes, we used ordinary least squares regression to compare the 2 randomized groups. For count data, we used negative binomial regression. Any variables with a standardized difference greater than 0.1 between the randomized groups were included as control variables; this was reassessed for each outcome and model. Our a priori analytic plan specified testing for effect modification on intervention group by (1) THA or TKA, (2) baseline quartile of opioid use, and (3) quartile of predicted opioid use. Outcome assessment was carried out by an analyst blinded to patient’s treatment status. All analyses were conducted using SAS version 9.4 (SAS Institute; Cary, North Carolina) or Project R version 3.1 (R Foundation for Statistical Computing).

RESULTS

Our study flow CONSORT diagram is shown in Figure 1. Of the 1065 patients identified as TKA or THA surgical candidates, 934 were eligible for the study. We randomized the 561 patients who were in the top 60% of predicted risk for persistent opioid use. There were 275 patients (168 TKA, 107 THA) in the intervention group and 286 patients (168 TKA, 118 THA) in the usual care group. Of the 275 intervention patients, 274 were sent the first mailing, 250 were sent the second mailing, and 120 received a call from a pharmacist. A total of 33 (12%) intervention and 42 (15%) usual care patients had a cancellation or rehospitalization during follow-up. These 75 patients were excluded from our secondary (per-protocol) analysis.

Table 1 shows imbalance (standardized difference >0.1, bolded) for some variables (anticonvulsants and medications for anxiety and depression, comorbid substance abuse and depression, primary care utilization, and predicted probability of opioid persistence) in the year prior to study enrollment. We controlled for these variables, consistent with our a priori analysis plan. In addition, all subgroup analyses controlled for their specific set of variables that were imbalanced.

Our primary outcome analysis is shown in Figure 2 and Table 2. Overall, we found no meaningful reduction in DME in the 90 days following surgical discharge by intervention versus usual care (geometric mean ratio, 0.92 [95% CI, 0.69-1.21]). However, we did find significant effect modification by whether the patient had TKA or THA (interaction P <.001; data not shown). Figure 2 illustrates this interaction; the separation between intervention and usual care DME occurs around day 20 for both TKA and THA (panel A), but those who underwent THA (panel B) show a greater separation by treatment status than do those who underwent TKA (panel C). Those who underwent THA in the intervention group used significantly less DME than did those in the usual care subgroup (geometric mean ratio, 0.52 [95% CI, 0.33-0.82]). Patients in the TKA intervention subgroup did not have lower DME than usual care patients who underwent TKA (geometric mean ratio, 1.28 [95% CI, 0.90-1.82]). We did not find evidence of effect modification by baseline quartile of opioid use (P = .78) or quartile of predicted opioid use (P = .91). Our secondary per-protocol findings (overall adjusted geometric mean ratio, 0.82 [95% CI, 0.65-1.02]; THA subgroup, 0.55 [95% CI, 0.37-0.85]; TKA subgroup, 1.00 [95% CI, 0.79-1.26]) were similar to findings from the primary intent-to-treat analysis (all subgroup analyses available upon request).

Our analysis of secondary outcomes is shown in Table 3. We found no evidence of differences in secondary outcomes between the intervention and usual care groups, but we did observe potential increases in target nonopioid dispensings (adjusted rate ratio [RR], 1.09 [95% CI, 0.93-1.28]), face-to-face office visits (adjusted RR, 1.04 [95% CI, 0.94-1.16]), telephone encounters (adjusted RR, 1.11 [95% CI, 0.98-1.27]), and occupational/physical therapy visits (adjusted RR, 1.11 [95% CI, 0.88-1.40]) for intervention patients compared with usual care patients.

Among opioid prescriptions dispensed from KPNW pharmacies, most (76%) were from KPNW pharmacies in Oregon. There were 725 prescription fills from Oregon KPNW pharmacies in the intervention group and 811 in usual care. We found from the PDMP that there were 49 additional prescriptions filled at non-KPNW pharmacies in Oregon (14 among intervention patients and 35 among usual care patients), suggesting that 2% more prescriptions were filled at non-KPNW pharmacies in the intervention group and 5% more in usual care.

DISCUSSION

We found that a pharmacist-led opioid reduction initiative for patients undergoing orthopedic surgery had little effect on opioid use among the overall population of patients who underwent TKA or THA, but it had clinically and statistically significant effects on opioid use among the THA subgroup.

Why was there superior effectiveness of the intervention for the THA subgroup? The reasons for this are not clear, but they may be related to the greater magnitude and longer duration of pain associated with TKA surgery. Specifically, our follow-up time was limited to 90 days after surgical discharge, a period during which patients undergoing TKA may be advised that continued use of opioids is clinically reasonable, whereas those undergoing THA are less likely to receive that message. Extended follow-up time may reveal further insights into the intervention’s effects among patients undergoing TKA; however, original design decisions and the duration of funding limited us to 90 days of follow-up. We do not know whether patients actually took the medications that were dispensed, but it is possible that patients undergoing THA were being dispensed opioids following surgery at a level higher than their clinical need compared with patients undergoing TKA, making it more likely that those undergoing THA were able to decrease their opioid use while retaining adequate pain control. We also do not know whether patients read the materials; patients undergoing TKA may have been less likely to do so. Finally, as this was a very low-intensity intervention, a more intensive intervention may be merited to address pain management needs and encourage nonopioid pain management alternatives for TKA patients.

One potential clinical response to opioid reduction activities is a substitution effect on both medications and healthcare visits. To evaluate whether that type of substitution took place, we examined the use, during the 90-day follow-up period, of specific drugs (eg, NSAIDs, sedative hypnotic medications) and specific types of healthcare visits (eg, face-to-face office visits, telephone and email encounters, physical therapy and occupational therapy visits). We did not identify significant increases in medication or visits, although we did observe potential increases in the mean number of face-to-face office visits, telephone encounters, and occupational/physical therapy visits for intervention patients compared with usual care patients. Increases in those services are not necessarily clinically undesirable and, in fact, were encouraged by our mailed materials, but because they are resource-additive, we suggest that further evaluation of patient-directed opioid-sparing interventions should assess whether substitution effects occur.

A potential unintended consequence of a patient-directed opioid-sparing intervention is circumvention by “doctor shopping” outside of the health plan. Using PDMP data from Oregon, we found little evidence that the intervention caused patients to seek opioid prescriptions outside of KPNW. The percentage of patients with opioid prescriptions filled at non-KPNW pharmacies in the usual care group (5%) was actually slightly higher than in the intervention group (2%). Although we found that most (76%) of KPNW opioid prescriptions were dispensed from KPNW pharmacies in Oregon, this analysis was limited to the Oregon PDMP only, so it is possible that we missed fills if patients went to neighboring states.

We identified evidence reviews that consistently reported a lack of improvement in pain, function, health-related quality of life, or postoperative anxiety related to preoperative education among patients undergoing TKA/THA.^16-18 However, these reviews did not examine the impact of preoperative education on postoperative opioid use, and postoperative education was not a focus. In fact, we did not find studies that used patient-directed educational interventions to reduce opioid exposure in patients undergoing TKA/THA. Previous research shows that once patients use opioid therapy for 90 days, they are likely to remain on opioid therapy for years.¹⁹ Thus, the need to identify effective methods for decreasing opioid exposure is critical. However, a recent systematic review revealed that there is insufficient evidence available on tapering methods to draw reliable conclusions.²⁰ Research has also not focused on pharmacist-led approaches to opioid reduction, although our findings are consistent with those of previous research showing that pharmacy-led interventions can improve medication use and attainment of therapeutic goals,²¹ particularly when interventions involve specially trained pharmacists.^22-24 Our study was designed to use the pharmacist resources efficiently by referring patients for outreach only when they refilled opioids, thus creating a natural touchpoint for the intervention. Further work could be usefully undertaken to understand the effectiveness of different combinations of the intervention (eg, mailed education only, pharmacist only) and effects of the intervention over longer periods of time.

Strengths and Limitations

Our study has many strengths. First, our mailed educational materials were created using qualitative methods with involvement from patients who had undergone TKA or THA and from orthopedic clinicians.¹² When creating the text, we strove to use language that could be understood by participants who read at an eighth-grade level. This is critical in light of a recent report showing that more than 20% of orthopedic patients using opioids had inadequate health literacy.²⁵ Additionally, our study was conducted in an integrated care delivery system with intervention pharmacists who had specific training in opioid use and communication techniques such as motivational interviewing. We believe that the conceptual framework and findings are applicable to other settings and pharmacists, particularly because we have made the materials, including vignettes describing the intervention, publicly available for replication.²⁶ Finally, we used an opioid persistence risk score to aid in patient selection. Selecting patients based on their predicted risk of opioid persistence increased efficiency by reserving the intervention for patients who had the highest capacity to benefit from opioid reduction.^27,28

One limitation of the trial was a modest imbalance in several characteristics that could confound the benefit of the intervention versus usual care. However, we adjusted for those imbalances using a regression model. The benefit of the intervention as determined by the intent-to-treat analysis may have been biased toward a null effect by patients who had to reschedule their surgery date or who were rehospitalized during follow-up (12% of intervention patients; 15% of usual care patients). The 90-day duration of follow-up may have been insufficient to observe the complete benefit of the intervention, especially for the TKA subgroup. The study size for the trial was based on the entire population—patients undergoing THA and TKA&mdash;so the subgroup-specific intervention benefits are less precise.

CONCLUSIONS

Our pharmacist-led, patient-directed intervention to reduce opioid use demonstrated a reduction in opioid dispensings in the 90 days following THA, but not TKA. Reducing opioid exposure following TKA may require additional efforts. We observed no significant unintended consequences in opioid-seeking behavior. Further work is needed to understand the longer-term outcomes of the intervention.Author Affiliations: Kaiser Permanente Center for Health Research (DHS, JLK, LLD, JM, XY, JS, AP, ESJ), Portland, OR; Kaiser Permanente Northwest (KR, LAT, DB), Portland, OR.

Source of Funding: US Food and Drug Administration, FDA contract number: HHSF223201400146C.

Author Disclosures: Dr DeBar reports grants pending and received from a Patient-Centered Outcomes Research Institute contract focused on pain and opioids. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (DHS, JLK, LLD, JS, AP, KR, LAT, DB, ESJ); acquisition of data (DHS, JLK, JM, XY, JS, KR); analysis and interpretation of data (DHS, JLK, LLD, XY, JS, LAT, ESJ); drafting of the manuscript (DHS, JLK, LLD, XY, JS); critical revision of the manuscript for important intellectual content (DHS, JLK, LLD, JS, AP, KR, LAT, DB, ESJ); statistical analysis (DHS, XY, ESJ); provision of patients or study materials (JM, JS, DB); obtaining funding (DHS, ESJ); administrative, technical, or logistic support (JM, AP); and supervision (DHS).

Address Correspondence to: David H. Smith, PhD, RPh, Kaiser Permanente Center for Health Research, 3800 N Interstate Ave, Portland, OR 97227. Email: david.h.smith@kpchr.org.REFERENCES

1. Pezalla EJ, Rosen D, Erensen JG, Haddox JD, Mayne TJ. Secular trends in opioid prescribing in the USA. J Pain Res. 2017;10:383-387. doi: 10.2147/JPR.S129553.

2. Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in drug and opioid overdose deaths—United States, 2000-2014. MMWR Morb Mortal Wkly Rep. 2016;64(50-51):1378-1382. doi: 10.15585/mmwr.mm6450a3.

3. Volkow ND, McLellan TA, Cotto JH, Karithanom M, Weiss SR. Characteristics of opioid prescriptions in 2009. JAMA. 2011;305(13):1299-1301. doi: 10.1001/jama.2011.401.

4. Zywiel MG, Stroh DA, Lee SY, Bonutti PM, Mont MA. Chronic opioid use prior to total knee arthroplasty. J Bone Joint Surg Am. 2011;93(21):1988-1993. doi: 10.2106/JBJS.J.01473.

5. Reuben DB, Alvanzo AA, Ashikaga T, et al. National Institutes of Health Pathways to Prevention Workshop: the role of opioids in the treatment of chronic pain. Ann Intern Med. 2015;162(4):295-300. doi: 10.7326/M14-2775.

6. Morris BJ, Mir HR. The opioid epidemic: impact on orthopaedic surgery. J Am Acad Orthop Surg. 2015;23(5):267-271. doi: 10.5435/JAAOS-D-14-00163.

7. Information statement: opioid use, misuse, and abuse in orthopaedic practice. American Academy of Orthopaedic Surgeons website. aaos.org/uploadedFiles/PreProduction/About/Opinion_Statements/advistmt/1045%20Opioid%20Use,%20Misuse,%20and%20Abuse%20in%20Practice.pdf. Published October 2015. Accessed October 10, 2018.

8. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. doi: 10.1016/j.jbi.2008.08.010.

9. Kuntz J, Johnson EJ, Petrik A, Yang X, Smith DH. Identifying patients for clinical trials by risk stratification: the case of an intervention to reduce post-operative opioid use in total joint replacement. Pharmacoepidemiol Drug Saf. 2015;24(suppl 1):472. doi: 10.1002/pds.3838.

10. Lachin JM. Introduction to sample size determination and power analysis for clinical trials. Control Clin Trials. 1981;2(2):93-113. doi: 10.1016/0197-2456(81)90001-5.

11. Rollnick S, Miller WR, Butler CC. Motivational Interviewing in Health Care: Helping Patients Change Behavior. New York, NY: Guilford Press; 2008.

12. Smith DH, Kuntz J, DeBar L, et al. A qualitative study to develop materials educating patients about opioid use before and after total hip or total knee arthroplasty. J Opioid Manag. 2018;14(3):183-190. doi: 10.5055/jom.2018.0448.

13. Technical assistance guide no. 01-13: calculating daily morphine milligram equivalents. Prescription Drug Monitoring Program Training and Technical Assistance Center website. pdmpassist.org/pdf/BJA_Performance_Measure_Aid_MME_conversion_FINAL_20170920_revised.pdf. Published February 28, 2013. Updated September 2017. Accessed March 10, 2017.

14. Deyo RA, Smith DH, Johnson ES, et al. Opioids for back pain patients: primary care prescribing patterns and use of services. J Am Board Fam Med. 2011;24(6):717-727. doi: 10.3122/jabfm.2011.06.100232.

15. Von Korff M, Saunders K, Thomas Ray G, et al. De facto long-term opioid therapy for noncancer pain [erratum in Clin J Pain. 2014;30(9):830]. Clin J Pain. 2008;24(6):521-527. doi: 10.1097/AJP.0b013e318169d03b.

16. McDonald S, Page MJ, Beringer K, Wasiak J, Sprowson A. Preoperative education for hip or knee replacement. Cochrane Database Syst Rev. 2014;(5):CD003526. doi: 10.1002/14651858.CD003526.pub3.

17. Louw A, Diener I, Butler DS, Puentedura EJ. Preoperative education addressing postoperative pain in total joint arthroplasty: review of content and educational delivery methods. Physiother Theory Pract. 2013;29(3):175-194. doi: 10.3109/09593985.2012.727527.

18. Aydin D, Klit J, Jacobsen S, Troelsen A, Husted H. No major effects of preoperative education in patients undergoing hip or knee replacement—a systematic review. Dan Med J. 2015;62(7):A5106.

19. Vanderlip ER, Sullivan MD, Edlund MJ, et al. National study of discontinuation of long-term opioid therapy among veterans. Pain. 2014;155(12):2673-2679. doi: 10.1016/j.pain.2014.09.034.

20. Chou R, Deyo R, Devine B, et al. The Effectiveness and Risks of Long-Term Opioid Treatment of Chronic Pain. Rockville, MD: Agency for Healthcare Research and Quality; 2014.

21. Rotta I, Salgado TM, Silva ML, Correr CJ, Fernandez-Llimos F. Effectiveness of clinical pharmacy services: an overview of systematic reviews (2000-2010). Int J Clin Pharm. 2015;37(5):687-697. doi: 10.1007/s11096-015-0137-9.

22. Lowrie R, Lloyd SM, McConnachie A, Morrison J. A cluster randomised controlled trial of a pharmacist-led collaborative intervention to improve statin prescribing and attainment of cholesterol targets in primary care. PLoS One. 2014;9(11):e113370. doi: 10.1371/journal.pone.0113370.

23. Entezari-Maleki T, Dousti S, Hamishehkar H, Gholami K. A systematic review on comparing 2 common models for management of warfarin therapy; pharmacist-led service versus usual medical care. J Clin Pharmacol. 2016;56(1):24-38. doi: 10.1002/jcph.576.

24. Feldstein AC, Smith DH, Perrin N, et al. Improved therapeutic monitoring with several interventions: a randomized trial. Arch Intern Med. 2006;166(17):1848-1854. doi: 10.1001/archinte.166.17.1848.

25. Hadden KB, Prince LY, Barnes CL. Health literacy and opioid use in orthopaedic patients. J Surg Orthop Adv. 2016;25(4):234-237.

26. Smith DH. FLOOD (FDA: Lowering Orthopaedic Opioid Dosing) study: downloadable program materials. Kaiser Permanente Center for Health Research website. research.kpchr.org/Research/Our-Researchers/David-Smith/FLOOD-Study. Published 2018. Accessed October 10, 2018.

27. Vickers AJ, Bennette C, Kibel AS, et al. Who should be included in a clinical trial of screening for bladder cancer?: a decision analysis of data from the Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial. Cancer. 2013;119(1):143-149. doi: 10.1002/cncr.27692.

28. Vickers AJ, Kramer BS, Baker SG. Selecting patients for randomized trials: a systematic approach based on risk group. Trials. 2006;7:30. doi: 10.1186/1745-6215-7-30.

Download PDF: A Randomized, Pragmatic, Pharmacist-Led Intervention Reduced Opioids Following Orthopedic Surgery