Publication
Article
The American Journal of Managed Care
A Group Randomized Trial to Improve Safe Use of Nonsteroidal Anti-inflammatory Drugs
Objective: To determine whether audit/feedback and educationalmaterials improve adherence to recommendations forlaboratory monitoring and cytoprotective agents to detect andprevent adverse events caused by nonsteroidal anti-inflammatoryagents (NSAIDs).
Study Design: Controlled, cluster-randomized trial.
Methods: Physicians commonly prescribing NSAIDs were identifiedwithin a large managed care organization and randomizedto a control or an intervention group (audit/feedback with peer-derivedbenchmarks and continuing medical education). Medicalrecords were examined 10 months before and after the interventionfor clinical data and receipt of complete blood count (CBC), creatininetesting, and cytoprotective agents (process measures). Primaryanalysis compared intervention versus control physicians amongthose who initially performed below a peer-derived benchmark.General estimating equations accounted for patient clustering.
Results: Of 101 physicians initially randomized, 85 remainedeligible (38 internists, 36 family physicians, 11 rheumatologists)postintervention. Mean percent change in performance betweenintervention and control physicians for CBC monitoring was 16%versus 10%; for creatinine monitoring, 0% versus 17%; and use ofcytoprotective agents, -3% versus -1%. None of these changeswere significant. Rheumatology specialty, number of NSAID prescriptionsand physician visits, and patient risk factors for NSAID-relatedtoxicity were more strongly associated with improved safetypractices than the intervention.
Conclusions: Audit/feedback and educational materials had noobserved effect on improving NSAID-related safety practices.Potentially contributing factors include high baseline performance(ceiling effect), dilution of the intervention effect by case mix andprovider factors, nonreceipt of intervention materials, and diverseindications for lab tests.
(Am J Manag Care. 2005;11:537-543)
Nonsteroidal anti-inflammatory drugs (NSAIDs)are among the most commonly used medicationsworldwide.1,2 Adverse events associated withchronic NSAID use are common and often result in substantialmorbidity and mortality.3 Among arthritispatients who use NSAIDs regularly, approximately 0.5%are hospitalized each year for a serious gastrointestinal(GI) event.4,5 Other adverse events associated withNSAID use include hypertension, worsened congestiveheart failure, and several forms of renal injury. NSAIDsare among the most common medications prescribedinappropriately to older Americans, based on the presenceof comorbid diseases known to increase the riskfor NSAID toxicity.6
In light of widespread NSAID use and the potentialfor related toxicity, several physician groups have promulgatedguidelines recommending NSAID safety-monitoringpractices.7-10 These include identifying users athigh risk for common NSAID-related complications,monitoring with complete blood count (CBC) and creatininetesting to screen for early GI and renal toxicity,and using cyclooxygenase-2 (COX-2) selective NSAIDsor coprescription of cytoprotective agents in high-riskpatients. However, physician adherence to these recommendationsis low.11-13 For example, 1 study of 1355rheumatoid arthritis patients found that physicianadherence to recommended NSAID toxicity monitoringwas only approximately 50%.11 Historically, variousstrategies designed to alter physician behavior, includingguideline formulation and written educational materials,have met with poor results. One strategy that hasshown promise in effecting change in provider practicepatterns is audit and feedback.14,15 With the goal ofimproving the quality of care provided to chronicNSAID users, we conducted a group randomized trial ofa multimodal intervention designed to improve safetypractices related to NSAID prescribing. We hypothesizedthat intervention physicians would show greaterimprovement in 3 safe NSAID-prescribing processmeasures between the baseline and follow-up periodscompared with improvement shown by the controlphysicians.
METHODS
Baseline Cohort
After approval by the University of Alabama atBirmingham institutional review board, we identified alladult patients with pharmacy benefits enrolled in alarge regional managed care organization (MCO) receiving1 or more NSAID prescriptions during a 7-monthperiod from August 1999 to February 2000. NationalDrug Codes were used to identify NSAID prescriptionsfrom pharmacy claims. No information regarding use ofover-the-counter NSAIDs was available. We focused onpatients of providers who were more likely to prescribeand continue to follow long-term NSAID users (family orgeneral practitioners, internists, and rheumatologists)and deliberately oversampled rheumatologists. Thesepatients and their associated prescribing physicianscomprised the baseline cohort. Patient and providerinformation was obtained using the MCO administrativeclaims and provider databases.
From a total of 2334 eligible patients using NSAIDs,680 (29%) patients and their corresponding 136providers (5 patients per provider) were randomly selected,with preference given to patients with 3 or moreNSAID prescriptions. We requested and received a totalof 452 records (66% record response rate) from 103physicians (43 internists, 44 general and family practitioners,16 rheumatologists) (76% physician responserate).
Baseline Medical Record Abstraction
Medical record review included all chart documentationand laboratory data between August 1999 andJune 2000. This 10-month interval ensured a minimumof 90 days of follow-up past the last NSAID prescriptionidentified via pharmacy claims. Trained abstractorsused a customized version of the MedQuest softwarefor chart abstraction (Fu Associates, Ltd, under contractfrom the Health Care Financing Administration;http://www.fu.com/products.htm). Process measurescharacterizing safe NSAID prescribing were developedusing the published literature.8,16 Medical record abstractorsachieved 97% interrater reliability for all mainvariables.
Multimodal Intervention to ImproveNSAID Safety
Participating physicians were stratified by specialty(ie, rheumatologist vs other physician type) and randomizedto either the intervention or the control arm.Two University of Alabama at Birmingham physiciansinvolved in the study design were excluded from randomization.After dissemination of interventionmaterials, we were notified that 1 physician in theintervention group had died.
Physicians randomized to the intervention arm wereexposed to a multimodal intervention. Under the generaltheme of "Safer Use of NSAIDs," the mailed interventionpacket contained (1) a brochure consisting ofprovider-specific audit and feedback on indicators ofsafe NSAID practices for the physician's NSAID-treatedpatients, along with comparison with the performanceof the top 10% of cohort physicians (theAchievable Benchmark of Care17); (2) key NSAID-relatedliterature citations, with the opportunity forcontinuing medical education (CME) credits; and (3) ahyperlink to a case-based, educational Web sitefocused on safe NSAID-prescribing.
The intervention packet was disseminated twice.The first was mailed via the US Postal Service inFebruary 2002, and a second delivery of the materialoccurred via overnight courier 1 month later. Severalweeks after the initial mailing, a faxed questionnairewas distributed to assess receipt of materials and toobtain physician feedback on the intervention. The faxwas broadcast weekly for 5 consecutive deliveries inApril and May of 2002.
NSAID Safety Quality Indicators
Using data from the 10-month observation period,performance feedback was given to each physician forthe following 3 process-of-care measures: (1) the proportionof the physician's patients with an increasedrisk for NSAID GI toxicity who had at least 1 CBC testperformed (increased risk was defined as age ≥65 years,concomitant glucocorticoid or warfarin use, and/or historyof GI bleeding including peptic ulcer disease); (2)the proportion of the physician's patients with anincreased risk for NSAID renal toxicity who had at least1 creatinine test performed (increased renal risk wasdefined as age ≥65 years; concomitant use of anangiotensin-converting enzyme inhibitor, diuretic, orother antihypertensive agent; and/or documentedhypertension or diabetes); and (3) the proportion ofusers of traditional NSAIDs with NSAID-related GI risk(defined above) receiving a cytoprotective agent (misoprostolor a proton pump inhibitor).
Follow-up Cohort
Approximately 6 months after the intervention wasdistributed, we again identified all NSAID prescriptionswritten by both intervention and control physicians.Patients were eligible to be included in the follow-upcohort if they received 1 or more NSAID prescriptionsin the 7 months from February 2002 to August 2002. Ofthe 101 physicians in the baseline cohort, 5 did nothave any eligible patients for the follow-up cohort (2intervention and 3 control physicians). From a total of2426 eligible patients using NSAIDs, 563 (23%) patientsand their corresponding 96 providers (5.9 ± 0.70patients per provider) were randomly selected with theintent to obtain 5 patients per provider. Preference inthe random selection of patients was again given to individualsreceiving ≥3 NSAID prescriptions written by thesame physician. Medical records were reviewed by 5trained abstractors over a similar 10-month period forthe same 3 primary processes of care of interest usingthe methods and tools described above. The abstractorswere blind to physicians' study arm assignment.
Statistical Analyses
The primary analysis was to compare the absoluteperformance for each of the 3 processes of care (CBCtesting, creatinine testing, and coprescription of cytoprotectiveagents) for the intervention andcontrol physicians who performed belowthe Achievable Benchmark of Care in thebaseline period. In secondary analyses, weexamined change in performance amongonly high-risk patients (CBC testing andcytoprotective use among high-GI-riskpatients, and creatinine testing among high-renal-risk patients). We also allowed satisfactionof the cytoprotective-use processmeasure to include substitution of COX-2NSAIDs (eg, celecoxib, rofecoxib) for traditional,nonselective NSAIDs. Finally, weexamined factors specific to patients,providers, and receipt of the interventionthat were associated with improvementbetween the 2 time periods. Improvementwas defined as postintervention physicianperformance that exceeded preinterventionphysician performance.
Main analyses were performed usingpostintervention patient-level variablesaggregated at the provider level. For theprimary analysis, the performance ofthe physicians was analyzed by theirassignment to either the intervention orthe control group. In secondary/exploratoryanalyses that were likely underpowered,intervention physicians also were categorizedas either having (1) never respondedor (2) returned the mailed postcard or faxindicating receipt of the interventionpacket, or logged onto the NSAID CMEWeb site. In multivariable models thatexamined improvement in performance,adjustment was made for each physician's baselineperformance.
t
P
Student tests or analysis of variance and chi-squaretests were used to compare continuous and categoricalvariables, respectively. Generalized estimating equationswere used to control for differences in case mix,account for cluster randomization, and adjust for thephysician's baseline performance rates.18 Univariateassociations were required to be significant at <.25 forinclusion in further model building. The study was initiallydesigned to have 80% power to detect a 10% differencebetween intervention and control groups for eachof the 3 primary end points, which led to the determinationof the number of physicians to recruit and patientrecords to review. Data collection and management wereperformed using Microsoft Access (Microsoft Corporation,Redmond, Wash), and statistical tests wereperformed using SAS software (SAS Institute, Cary, NC).
RESULTS
Patient Selection and Chart Requests
After our selection process and subsequent requestfor medical records, we received 66% of the requestedmedical charts (n = 312). There were no significant differencesin sex and age between patients whose recordswere reviewed and those whose records were not sent tous for review. A total of 101 physicians were randomizedinto the intervention or the control arms of thestudy. An overview of the trial design is presented inFigure 1.
Six months after our intervention, we received 83%of the requested medical records from 85 of the 96physicians (n = 421). Characteristics of the interventionand control physicians and their associated NSAID-treatedpatients are described in Table 1. Only minimaldifferences were observed between the demographicsand patient case mix of intervention physicians comparedwith control physicians. Fewer than 5% of thepatients in the baseline cohort were represented in thefollow-up cohort.
Response to Intervention Targeted atImproving NSAID Safety
After the mailed intervention, 20 of the 50 interventionphysicians responded (40%) via the Web site (n =1), postcard (n = 14), or fax (n = 11) (numbers includemultiple responses from individual physicians). CMEwas awarded to 15 physicians.
Primary and Secondary Analyses
Performance rates for the 3 main processmeasures including CBC testing, creatininetesting, and cytoprotective use for the preinterventionand postintervention time periodswere similar at baseline between the interventionand control group physicians(Figure 2). For CBC testing, the performanceof intervention physicians increased from31% in the baseline time period to 47% in thefollow-up period. The performance of controlphysicians similarly increased from 40% atbaseline to 50% in follow-up. For creatininetesting, the performance of interventionphysicians was 47% and did not change, incontrast to the performance of control physicians,which increased from 41% at baselineto 58% in follow-up. Only small, nonsignificantchanges were observed in the use ofcytoprotective agents between the baselineand follow-up time periods for both arms ofthe intervention. There were no significantdifferences in performance between interventionand control physicians at either timepoint for any of the 3 process-of-care measures.In secondary analyses, no differencesin performance between the interventionand control groups were observed when all cohort physicianswere included, when the analysis was restricted tohigh-risk patients only, or when the cytoprotectiveprocess measure included COX-2-selective NSAID use(data not shown).
Multivariable analysis examining factors associatedwith improvement in NSAID safety practices is presentedin Table 2. Because performance in the baselineperiod was expected to impact improvement measuredin the follow-up period, we included baseline performanceas a covariate in our model examining improvement.A greater number of patient office visits, moreNSAID prescriptions, and higher risk patients wereassociated with an increased likelihood of adherence toNSAID safety-monitoring guidelines. Rheumatologistsalso were more likely than family practice physicians toincrease their rate of toxicity monitoring. The performanceof intervention physicians was analyzed both inaggregate (data not shown) and separately in subgroupsbased on their confirmation of receipt of interventionmaterials and/or participation in CME activities.Neither analysis showed significant differences in thelikelihood for improvement in performance comparedwith control physicians. However, when absolute performancewas examined, the performance of physicianswho confirmed receipt of the intervention materialsand/or engaged in the optional CME activities showed ahigher but nonsignificant trend compared with both theremaining intervention physicians and also the controlphysicians (data not shown).
DISCUSSION
With the goal of improving NSAID safety practices,we conducted a group randomized trial of a multimodalintervention utilizing audit and feedback, and comparisonwith the performance of the top 10% of cohortphysicians. We recruited physicians that prescribeNSAIDs in both primary care and specialty settingswithin a diverse managed care environment. Despiteour use of a multimodal methodology previously shownto be effective for changing practice in other chronicdiseases,17 we failed to detect significant differences inNSAID safety-monitoring practices. Provider andpatient factors and health services utilization insteadwere more strongly associated with both absolute andrelative improvement in adherence to NSAID safety-monitoringguidelines.
Several methodologies, including some of those usedin this study, have achieved varying degrees of successin changing physician practice and improving adherenceto evidence-based clinical practice guidelines.Those demonstrating only limited benefit include traditionalCME activities and passively consumed writteneducational materials.19,20 Others showing more promiseinclude electronic reminder systems,21 local opinionleaders,22 and academic detailing.23 We used auditand feedback, which previously have been demonstratedby us and others to be efficacious in a variety ofchronic diseases.17,24,25 At best, small to moderate effectsizes have been observed with most of these interventions,and no single intervention or set of interventionshas been proven effective to alter physician practicepatterns in all settings.21
Although the results of our trial were negative, weincorporated previously recognized important elementsof successful uses of audit and feedback. We offeredaudit and feedback that was specific to each physician'spractice. We included comparison with boththe physician's peer group and with the top performersvia the Achievable Benchmark of Care.17 Weattempted to enhance the validity of our interventionby referencing evidence-based guidelines advocatingidentification of patients at high risk for NSAID-relatedcomplications and appropriate monitoring. Wecoupled our audit and feedback with CME activitiesoffered in both written and electronic format.Processes of care were assessed in a blinded fashionby multiple abstractors who achieved high rates ofinterobserver agreement.
Despite these strengths, several limitations mayhave contributed to our negative results. Because wefocused on laboratory tests as a process measure oftoxicity monitoring and because these tests areordered for a variety of indications, we likely had alower than optimal "signal-to-noise" ratio in our laboratory-monitoring outcomes. Similar issues are relevantfor prescription of cytoprotective medications.Additionally, we had greater-than-expected attritionamong study physicians and more variability in ourprocess-of-care measures than we expected. Post-hocpower calculations based on our actual data suggestedthat we were powered at 80% to detect a 17% differencebetween intervention and control groups, a largerdifference than observed for changes in CBCtesting, for example.
Physicians who had 100% adherence to the NSAIDsafety practice guidelines in the preintervention periodreduced the number of individuals for whom measurableimprovement could be demonstrated (a ceilingeffect). High ceiling effects are known to reduce themeasurable effectiveness of audit and feedback.25Difficulties in capturing the attention of busy physiciansalso likely contributed to our negative results.Indeed, fewer than half of the doctors randomized tothe intervention responded that they received theintervention packet despite repeated mailings.However, our results suggested a trend towards higherperformance among physicians who reported receipt ofthe intervention materials and participated in the CMEactivities.
Our negative results stand in contrast to our previoussuccess with using audit and feedback with the peer-derivedAchievable Benchmark of Care to improve carefor diabetic patients.17 In contrast to nationally recognizeddiabetes quality measures,26 quality measures forNSAID monitoring may be less widely known or lessaccepted by physicians. Although ≥83% of patientsreceived at least 3 NSAID prescriptions in 7 months, theremainder received only 1 or 2, and their physiciansmay not have believed that these patients were appropriatecandidates for the measured processes of carebecause they were not chronic users. Moreover, some ofthe NSAID users we identified were lower risk based onyounger age or lack of medical comorbidities.27 Thepeer-derived Achievable Benchmark of Care benchmarkfor all 3 process measures was at near or equal to100%, perhaps reducing its face validity. Althoughrecent safety data for NSAIDs suggest the need for revisionof NSAID prescription and monitoring guidelines,our study predated most of these safety concerns andshould not have been temporally affected by thesenewer data.
Our results highlight the challenges of interventionsdesigned to improve adherence of NSAID safety-monitoringguidelines. Difficulties in engaging physiciansdespite repeated mailings, a ceiling effect in baselineperformance rates, and nonspecific indications for theprocesses of care of interest were significant factorsthat likely contributed to the negative effect of our multimodalintervention for increasing NSAID toxicitymonitoring and use of cytoprotective therapy. In contrastto our intervention, disease severity and frequencyof care were more important in identifying thosemost likely to get toxicity monitoring. The clinical andmethodologic insights gained from this work will leadfuture studies to greater success in improving physicianadherence to evidence-based practice guidelines.
From the Division of Clinical Immunology and Rheumatology (JRC, KGS); the Divisionof Continuing Medical Education (JO); the Division of General Internal Medicine (JJA, AG);and the Division of Preventive Medicine (JJA, LJ, SP); University of Alabama at Birmingham,Birmingham, Ala; and the Center for Health Services Research in Primary Care, Durham VAMedical Center, and the Division of General Internal Medicine, Duke University, Durham,NC (SHK).
This work was supported by grant HS10389 from the Agency for Healthcare Researchand Quality and grant P60 AR48095 from the Multipurpose Arthritis and MusculoskeletalDiseases Center.
Address correspondence to: Kenneth G. Saag, MD, Msc, University of Alabama atBirmingham, FOT 820, 510 20th Street South, Birmingham, AL 35294. E-mail:ksaag@uab.edu.
Arthritis Rheum.
1. Baum C, Kennedy D, Forbes M. Utilization of nonsteroidal antiinflammatorydrugs. 1985;28:686-692.
Ann Intern Med.
2. Roth S. Non-steroidal anti-inflammatory drugs, gastropathy, deaths and medicalpractice. 1988;109:353-354.
N Engl J Med.
3. Wolfe MM, Lichtenstein DR, Singh G. Gastrointestinal toxicity of nonsteroidalantiinflammatory drugs. 1999;340:1888-1899.
J Rheumatol.
4. Fries JF, Bruce B. Rates of serious gastrointestinal events from low dose use ofacetylsalicylic acid, acetaminophen, and ibuprofen in patients with osteoarthritisand rheumatoid arthritis. 2003;30:2226-2233.
Arthritis Rheum.
5. Fries JF, Murtagh K, Bennett M, Zatarain E, Lingala B, Bruce B. The rise anddecline of nonsteroidal antiinflammatory drug associated gastropathy in rheumatoidarthritis. 2004;50:2433-2440.
J Am Geriatr Soc.
6. Hanlon J, Schmader K, Boult C, et al. Use of inappropriate prescription drugsby older people. 2002;50:26-34.
Can Med Assoc J.
7. Tannenbaum H, Davis P, Russell AS, et al. An evidence-based approach toprescribing NSAIDs in musculoskeletal disease: a Canadian consensus. CanadianNSAID Consensus Participants. 1996;155:77-88.
Am J Kidney Dis.
8. Henrich W, Agodoa L, Barrett B. Analgesics and the kidney: summary and recommendationsto the Scientific Advisory Board of the National Kidney Foundationfrom an Ad Hoc Committee of the National Kidney Foundation. 1996;27:162-165.
Am J Gastroenterol.
9. Lanza FL. A guideline for the treatment and prevention of NSAID-inducedulcers. Members of the Ad Hoc Committee on Practice Parameters of theAmerican College of Gastroenterology. 1998;93:2037-2046.
Arthritis
Rheum.
10. Guidelines for monitoring drug therapy in rheumatoid arthritis. AmericanCollege of Rheumatology Ad Hoc Committee on Clinical Guidelines. 1996;39:723-731.
JAMA.
11. MacLean CH, Louie R, Leake B, et al. Quality of care for patients withrheumatoid arthritis. 2000;284:984-992.
J Clin Rheum.
12. Rothenberg R, Holcomb J. Guidelines for monitoring of NSAIDs. Who listened?2000;6:258-265.
J Rheumatol.
13. Patino F, Olivieri J, Allison J, et al. Nonsteroidal antiinflammatory drug toxicitymonitoring and safety practices. 2003;30:2680-2688.
JAMA.
14. Davis D, Thomson M, Oxman A, Haynes R. Changing physician performance:a systematic review of the effect of continuing medical education strategies.1995;274:700-705.
BMJ (Clinical Research
ed).
15. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA.Closing the gap between research and practice: an overview of systematic reviewsof interventions to promote the implementation of research findings. The CochraneEffective Practice and Organization of Care Review Group. 1998;317(7156):465-468.
Am J Gastroenterol.
16. Lanza F. A guideline for the treatment and prevention of NSAID-inducedulcers. Members of the Ad Hoc Committee on Practice Parameters of theAmerican College of Gastroenterology. 1998;39:2037-2046.
JAMA.
17. Kiefe C, Allison J, Williams O, Person S, Weaver M, Weissman N. Improvingquality improvement: a randomized trial of Achievable Benchmark of Care feedback.2001;285:2871-2879.
Design and Analysis of Group-randomized Trials.
18. Murray DM. New York,NY: Oxford University Press; 1998.
Cochrane Database Syst Rev.
19. Freemantle N, Harvey E, Wolf F, Grimshaw J, Grilli R, Bero L. Printed educationalmaterials: effects on professional practice and health care outcomes.2000.
JAMA.
20. David D, O'Brian M, Freemantle N, Wolf F, Mazmanian P, Taylor-Vaisey A.Impact of formal continuing medical education: do conferences, workshops,rounds, and other traditional continuing education activities change physicianbehavior or health care outcomes? 1999;282:867-874.
Cochrane Database Syst Rev.
21. Hulscher M, Wensing M, Van der Weijden T, Grol R. Interventions to implementinterventions in primary care. 2004;2.
The Cochrane Library.
22. Thomson O'Brien M, Oxman A, Haynes R, Davis D, Freemantle N, HarveyE. Local opinion leaders: effects on professional practice and health care outcomes(Cochrane Review). Chichester, UK: John Wiley &Sons, Ltd; 2004.
The Cochrane Library.
23. Thomson O'Brien M, Oxman A, Davis D, Haynes R, Freemantle N, HarveyE. Educational outreach visits: effects on professional practice and health care outcomes(Cochrane Review). Chichester, UK: John Wiley &Sons, Ltd; 2004.
Cochrane Database Syst Rev.
24. Thomson O'Brien M, Oxman A, Davis D, Haynes R, Freemantle N, HarveyE. Audit and feedback versus alternative strategies. 2004;2.
The Cochrane Library.
25. Jamtvedt G, Young J, Kristoffersen D, Thomson O'Brien M, Oxman A. Auditand feedback: effects on professional practice and health care outcomes(Cochrane Review). Chichester, UK: John Wiley & Sons,Ltd; 2004.
The State of Health Care Quality:
2004.
26. National Committee for Quality Assurance. Available at: http://www.ncqa.org/about/about.htm. Accessed July 12, 2005.
Arthritis Rheum.
27. Singh G, Ramey DR, Triadafilopoulos G, Brown B, Balise R. GI score: a simpleself-assessment instrument to quantify the risk of serious NSAID related GIcomplications in RA and OA. 1998;41:S75.
