Publication
Article
Increasing Adherence to a Community-Based Guideline for Acute Sinusitis Through Education, Physician Profiling, and Financial Incentives
Author(s):
Objectives: To implement a large-scale multifaceted interventionconsisting of physician education, profiling, and a financialincentive, to improve treatment quality for acute sinusitis.
Study Design: Cohort trial using a historical control of treatmentpatterns among approximately 500 internists, 200 family practitioners,and 200 pediatricians in a northeastern community-wideindividual practice association.
Participants and Methods: Episode treatment group methodswere adapted to identify cases (episodes) and to assess care patternsfor acute sinusitis among 420 000 health maintenance organizationpatients seen between January 1, 1999, and December 31,2001. The intervention consisted of care pathway development,physician and patient education, physician profiling, and a financialincentive.
P
Results: A statistical process control chart showed a shift towardrecommended treatment patterns after our intervention. The rate ofexceptions per episode of acute sinusitis decreased 20%, from 326exceptions per 1000 episodes between January 1, 1999, andOctober 31, 2000, to 261 between November 1, 2000, andDecember 31, 2001. Decreased use of less effective or inappropriateantibiotics accounted for most of the change (199 to 136exceptions per 1000 episodes [32% change]). Azithromycin usedecreased 30%, from 97 to 68 prescriptions per 1000 episodes.Firstline antibiotic (amoxicillin and doxycycline) use increased14%, from 451 to 514 prescriptions per 1000 episodes. Inappropriateradiology use decreased 20%, from 15 to 12 per1000 episodes. These changes were significant at < .005.
Conclusion: A multifaceted program, including education, physicianprofiling with actionable recommendations, and a financialincentive, significantly increased physicians' adherence to a community-developed care pathway and was successful at improving adherenceto recommended patterns of antibiotic use in acute sinusitis.
(Am J Manag Care. 2004;10:670-678)
For the past decade, the healthcare industry hasstruggled to identify methods to significantlymodify clinical practice. A recent review of typicaleducational programs confirms the inability of traditionalcontinuing medical education to change clinicalbehavior.1 Introducing evidence-based clinical guidelinesalso has failed to improve clinical care.2,3
The need to identify reliable ways to improve clinicalcare has led several teams to prepare and test new typesof interventions. Avorn and Solomon3 identified specificinterventions that improved appropriate antibioticuse. These approaches included reminders at the pointof care, academic detailing, and order entry programs.In 2001, Grol4 argued that the complexity of changingclinical practice behaviors requires more than a singleintervention such as an educational program, financialincentive, or practice profile. To promote successfulpractice outcomes and adherence to guidelines, Grolproposed creating an integrated combination of self-reinforcinginterventions such as evidence-based guidelines,professional education, assessment andaccountability, patient empowerment, and total qualitymanagement. Bodenheimer, Wagner, and Grumbach5,6also recently endorsed a multiple intervention, multilevelmodel for improving chronic disease care.
The primary aim of this study was to demonstratethe ability of such a multifaceted intervention programto improve the evaluation and management (E&M) ofacute sinusitis, especially in regards to appropriateantibiotic prescribing. Interventions included physicianeducation, a locally developed acute sinusitis care pathway,feedback through a physician profiling system, afinancial incentive for adherence to our care pathway,and patient education. The intervention was applied toa community-wide panel of more than 900 primary carephysicians covering 420 000 health maintenance organization(HMO) members. The high penetrance of theHMO in the local market allowed us to examine and profileindividual physicians on large numbers of caseswithout needing to pool data among multiple payers.
METHODS
Physicians
This project evolved from a collaboration betweenthe Rochester Individual Practice Association, Inc(RIPA) and BlueCross BlueShield of the RochesterArea's HMO, Blue Choice. (BlueCross BlueShield of theRochester Area has since been renamed ExcellusBlueCross BlueShield of the Rochester Region.) Thebaseline measurement period was January 1, 1999,through October 31, 2000. The intervention periodstarted November 1, 2000, and was measured throughDecember 31, 2001. In 2000, RIPA was a communitywidepanel of approximately 3000 practitioners and 900primary care physicians (500 internists, 200 familypractitioners, and 200 pediatricians) serving 420 000HMO subscribers. Rochester Individual PracticeAssociation, Inc, physicians were located in the 9-countyregion surrounding Rochester. The region includesurban, suburban, and rural communities.
Credentialing information was used to identify allinternists, family physicians, and pediatricians. All credentialedphysicians actively seeing patients wereincluded in the program. Cases of acute sinusitis seenby nurse practitioners or physician assistants wereassigned to their supervising physician.
Patients
Analysis included all HMO patients treated by theidentified physicians or their nurse practitioners andphysician assistants. The HMO provided an administrativedata set with scrambled patient identification numbers.Because neither patient-specific information norreviewed medical records were used, informed consentwas not obtained.
Case Identification
Episode treatment group (ETG) methods wereapplied to claims data to identify cases, or "episodes," ofacute sinusitis without sinus surgery and their relatedservices (Episode Treatment Grouper, version 4.0;Symmetry Health Data Systems, Inc, Phoenix, Ariz; incorporatedin Episode Profiler, version 4.6, CareEnhanceResource Management Software; McKesson HealthSolutions, LLC, Cambridge, Mass). In the ETG model, anepisode consists of a series of healthcare services relatedto a group of specific diagnoses for 1 patient. An episodeof care may contain single or multiple services, includingoffice visits, diagnostic tests, therapeutic interventions,emergency department visits, and prescribed medications.Episode treatment groups are structured so thateach contains 1 disease entity (as is the case for acutesinusitis) or clinically similar diseases.
International Classification of Diseases, NinthRevision (ICD-9-CM)
Current Procedural Terminology
ICD-9
Analysis of each episode began by finding an anchoringE&M service. The anchoring E&M service could havean code for acute sinusitis or for amore general illness such as a "viral upper respiratorytract infection." The software examined each subsequentclaim by procedure per codes, by diagnosis per codes, or by medicationgeneric code number and national drug code. It alsosearched backward 1 month for antibiotics prescribedbefore the initial patient encounter. The software thendecided whether to add the claim to the current episodeof care and, if so, whether the episode should remain inthe current ETG or be assigned to another ETG. Forexample, if the episode began with a viral upper respiratorytract infection and then later had an E&M service foracute sinusitis, it was removed from the former ETG intothe sinusitis ETG. If sinus surgery occurred subsequentto the anchoring visit, the episode was transferred to theETG for sinusitis with surgery. An episode was consideredcompleted when no additional services were billedfor the ETG for a fixed period (the "clean" period).
ICD-9
Acute sinusitis was ETG 0333 in the Symmetrygrouper. The following codes classified E&M servicesas acute sinusitis: 461, 461.0, 461.2, 461.3, 461.8,and 461.9. The clean period for ETG 0333 was 60 days.
The episode grouping software examined the completedatabase of all inpatient and outpatient claimspaid by the HMO for its members between January 1,1999, and December 31, 2001. After December 31,2001, we used a 2-month claims run-out period, correspondingto the episode's clean period, to capture lateclaims for services provided within the study period.Episodes were analyzed by the month and year of theirfirst service. An independent practice association(IPA)—HMO profiling team reviewed all identified problemsregarding data accuracy, collection, and analysisso that the system could be improved continuously.
Developing an Acute Sinusitis Care Pathway
In early 2000, an IPA multidisciplinary task forcewas convened to create a local acute sinusitis care pathway.The acute sinusitis task force included pediatricians,family practitioners, internists, otolaryngologists,an allergist, and an infectious disease specialist. Thetask force was charged with identifying the most importantevidence-based elements of quality care that couldbe measured using an administrative database. Table 1shows pathway elements generated by the task force.Lists of suggested firstline and secondline antibiotics, aswell as nonrecommended less effective or inappropriateantibiotics, were published (Table 2). The task forcebased many of its recommendations on a 2000 report bythe Sinus and Allergy Health Partnership,7 modified bylocal experience and antibiotic resistance patterns.8 Forexample, doxycycline was listed as an alternative firstlineantibiotic for patients older than 8 years who wereallergic to amoxicillin.
Creating a Pathway Scoring Measure
A second software program, the Referral ProfilerCustomization Utility (version 4.6, McKesson), wasadapted to analyze the acute sinusitis care pathway.The presence or absence of services, and their correctsequence, defined the rate of pathway adherence.Complete and incomplete episodes of care were analyzedbased on the type and sequence of initial services,rather than the length or total cost of the episode.
exceptions
The referral profiler generated the number of deviationsfrom the care pathway for each episode.Variations from the pathway were termed torecommended care. The total number of exceptionsdivided by the number of episodes gives the exceptionsper episode, the metric used as the core physician profilingmeasure. Table 3 summarizes the acute care servicesidentified by the referral profiler customization.
Multifaceted Intervention
Education.
One of us (JC) developed 2-hour interactivesmall group programs to educate panel practitionersabout the sinusitis recommendations of the taskforce. The IPA conducted programs on the judicious useof antibiotics for adults and children with upper respiratorytract infections in the last 2 months of 1999. Therecommendations were updated and the programsrepeated in April 2001 based on the IPA guidelines.Program recommendations were supported by tool kitsand patient materials conforming to the guideline recommendations.One of us (JK) produced a 1-pagepatient pathway that was distributed in a physiciannewsletter and could be downloaded from the RIPA Website.9
Practitioner Profiling Reports.
Physician profilingbegan in 1999. Beginning in April 2000, profiles for primarycare physicians included an acute sinusitis carepathway report. Beginning in September 2000, theexceptions per episode measure was added. The pathwayreport also detailed the frequency of specific exceptions.In that way, a practitioner could determine, for example,that 75% of his or her pathway exceptions resulted fromusing ineffective antibiotics, while 25% were because ofnot seeing patients before prescribing antibiotics.
Starting in December 2000, physician profile coverletters reported the exceptions per episode score andthe physician's performance in RIPA's newly initiatedvariable withhold program. By April 2001, the cover lettersuggested how to improve the exception to pathwaymeasure. Examples of physician-specific suggestionsincluded "do more E&M visits before prescribing antibiotics"or "use fewer less effective antibiotics."Physicians received 5 profiles between April 2000and December 2001.
Physician Financial Incentives.
For primarycare physicians, a scoring system was developedbased 20% on patient satisfaction, 40% on efficiency,and 40% on quality measures. The exceptionsper episode score for sinusitis counted for up to halfof the quality component. The results for patientsatisfaction, efficiency, and quality were combinedto obtain a total score.
As a capitated IPA, RIPA kept a percentage ofthe capitation in reserve (the "withhold") toaccommodate for increases in use. From 1999through 2001, the percentage withheld by RIPAwas 15% on each physician service. In 2000, thewithhold was decreased to 10% for the top 5% ofperformers in the scoring system and increased to20% for the bottom 5% of performers. Each primarycare physician's sinusitis exceptions per episodescore, total score, and corresponding specialtymean scores appeared on the physician's profilecover letters.
Control Group
The panel cohort served as its own historical control.The educational components began in fall 1999and continued throughout the study. The majorintegrated interventions(the report of the exceptionsper episode score, theannouncement of the variablewithhold and its connectionto the pathwayscore, and publication ofthe patient handout)occurred in September andOctober of 2000. Therefore,physician performancebetween January 1,1999, and October 31,2000 (the preinterventionperiod), was used as a controlfor results for casesbeginning between November1, 2000, and December31, 2001 (the postinterventionperiod).
Statistical Analysis
t
Standard control chartmethods were used to constructand analyze a statisticalprocess control chartfor exceptions per 1000episodes by episode startmonth.10 The test wasused to assess the statisticalsignificance of differencesbetween thepreintervention and postinterventionperiods. Analysiswas performed with theSPSS software package(release 11.01.1; SPSS Inc,Chicago, Ill).
RESULTS
Physician Intervention Group
There were approximately900 credentialedprimary care physicians asof December 1999,October 2000, andDecember 2001. Thesedates correspond to thebeginning of the educationalinterventions, themajor integrated interventions, and the end of thepostintervention analysis period, respectively. Therewere approximately 500 internists, 200 family practitioners,and 200 pediatricians at each of those pointsin time.
Episodes of Acute Sinusitis
P
There were 96 766 episodes of acute sinusitis duringthe 3-year study. Although there was an anticipatedseasonal variation in the number of episodes permonth, there was no change in the number of sinusitisepisodes per month before and after October 2000(2687 before and 2689 after, > .99).
Results of Reporting and IncentivizingPathway Adherence
t
P
Physician behavior changed significantly in thepostintervention period (Figure, Table 4, and Table 5).The Figure shows a statistical process control chart ofexceptions per 1000 episodes by episode start month.The centerline (mean) for all 36 months (36 datapoints) was 300 exceptions per episode. The exceptionrate was below that mean value from the third monthafter our intervention onward, a total of 12 consecutivedata points. Because more than 8 successive points fellon the same side of the centerline, this represents specialcause variation, ie, a change assignable to our intervention.10 The mean overall exceptions per episode ratedecreased 20%, from 326 per 1000 episodes before ourmajor intervention to 261 after. By test, the changewas significant at < .005 (Table 4).
P
Approximately 95% of the change resulted from alower rate of exceptions for use of less effective or inappropriateantibiotics. These decreased from 199 to 136exceptions per 1000 episodes, a change of 32%. Theremainder of the change was related to radiology exceptions.Radiology exceptions decreased by 20%, from 15to 12 per 1000 episodes. These changes were significantat < .005. There was no significant decrease in exceptionsassociated with prescribing of antibiotics beforethe patient had an office visit.
P
P
P
P
P
The mean number of antibiotic prescriptionsdecreased slightly, from 883 to 861 per 1000 episodesfrom before to after the study interventions, a 2.5%decrease ( < .005). Firstline antibiotic use increased14%, from 451 to 514 prescriptions per 1000 episodes.Use of less effective or inappropriate antibioticsdecreased 29%, from 311 to 222 prescriptions per 1000episodes. These changes were significant at < .005.The 2 most commonly used broad-spectrum antibioticsdeemed less effective or inappropriate wereazithromycin and clarithromycin. Both had a significantdecrease after the study intervention (Table 5).Azithromycin use decreased 30%, from 97 to 68 prescriptionsper 1000 episodes ( < .005). Clarithromycinprescriptions decreased 33%, from 30 to 20 per 1000episodes ( < .005). There was no significant change inthe use of appropriate secondline antibiotics (121 prescriptionsper 1000 episodes before intervention to 125after, = .15).
P
P
Use of radiological services also changed significantlyafter the study interventions. The rate of plain x-ray filmsinus series decreased from16.9 to 12.2 per 1000episodes, or 28% ( < .005)(Table 5). The rate of sinuscomputed tomographic scansfell by 39%, from 13.5 to 8.2per 1000 cases in the postinterventionperiod ( < .005).
P
Evaluation and managementservices rates changedlittle. There was no significantchange in primary carephysician E&M services.The otolaryngology consultationrate was unchangedafter the study interventions,at 4.7 consultationsper 1000 episodes of sinusitis(Table 5). Allergy consultationsdecreased, from 5.9to 4.1 per 1000 episodes(31%, < .005).
COMMENT
We have shown that a combination of a locally developedcare guideline, physician profiles containing specificfeedback, and a financial incentive influencedphysicians' treatment patterns for targeted acute sinusitismeasures. Most important, there was a significantincrease in use of firstline antibiotics, associated with alarge decrease in use of less effective or inappropriateantibiotics, including the 2 most common broad-spectrumantibiotics in that category. Secondary outcomesincluded a decrease in plain x-ray film sinus studies,sinus computed tomographic scans, and allergist consultationrates. The rate of otolaryngology consultationsdid not change. Taken together, these results suggestthat our multiple intervention model was successful inchanging physician behavior to improve adherence to acommunity-developed care guideline.
The decrease we observed in the use of broad-spectrumantibiotics judged less effective or inappropriatecontrasts favorably with the results of a recent study bySteinman et al,11 who reported that broad-spectrumantibiotic use in minor upper respiratory tract infectionsdoubled nationwide in the 1990s, from 24% to 48%of all antibiotic prescriptions. In that study,azithromycin and clarithromycin were found to haveincreased from 1% or 2% of prescriptions to 13%, whileamong our primary care physicians their use significantlydecreased.
This study reinforces the adage that "you get whatyou pay for." We tracked and scored sinus x-ray timingand sinus computed tomographic scans and observedreduced use. Although we recommended using antibioticsless often, we did not score or reward it. As aresult, the total number of antibiotics prescribed did notdecrease to a clinically meaningful degree, although thechange was statistically significant. Measures that werereported and rewarded changed more dramatically.Future programs should target reductions in totalantibiotic use.
Our findings support previous studies demonstratingthat changing physician behavior is best accomplishedby using multifaceted interventions. Reviews of thistopic have shown that didactic interventions alone wereineffective,1,4 while multiple interventions were the mosteffective.4,12 A 2001 comprehensive review of guidelineimplementation reinforced this conclusion.13 Theauthors, drawing on a Cochrane Effective Practice andOrganization of Care systematic analysis, recommendeda multifaceted intervention, including local modificationof guidelines to achieve local buy-in, local consensusconferences, and feedback of performance comparedwith that of peers. Working independently, we fashioneda similar program, with the important addition of afinancial incentive. In our community, local buy-in,interactive education, specific feedback, and a connectionto the physician compensation mechanism resultedin significant improvements in our end points.
We believe that our physician profiles were effectivebecause they contained specific action items, such as"use fewer less effective antibiotics," individualized foreach primary care physician. The feedback was timely,in that profiles were distributed approximately 3 timesa year. In these ways, we followed the recommendationsof Phillips et al14 for overcoming "clinical inertia."
Another unique element of our program was our metric,exceptions per episode. Any episode of acute illnessinvolves a series of clinical decisions. The exceptionsper episode score summarizes clinical decision makinginto a single measure. Allowing more than 1 exceptionper episode made our program more rigorous by makingit more difficult to reach our target of 75% care pathwayadherence. Using a continuous variable encouragedincremental improvement, because physicians receivedcredit for each additional decision that adhered to ourrecommendations. Furthermore, the measure could beapplied to administrative data, obviating the need formedical chart review, which would have been impossiblewith 32 000 cases of sinusitis per year.
Smithson and Koster15 wrote that financial incentivesare the most powerful devices for changing physicianbehavior. Our variable withhold allowed 5% ofpractitioners to receive 5% lower, and 5% to receive 5%higher, withholds based on their profile scores.Historically, RIPA has returned between 50% and 100%of primary care specialty withholds. Therefore, a primarycare physician's take-home income could havechanged only about 1.25% to 2.5% either way (5% ×50%-100% withhold return × 50% overhead). Our physiciansappeared to be sensitive to small financial incentives.Perhaps the need for a greater incentive wasminimized by the quality of the clinical pathway andthe feedback offered, supporting the value of a multipleintervention model. Our incentive plan is consistentwith a broader movement to connect physician reimbursementto quality improvement.16
This study had several limitations. We created a variablewithhold program but could not apply it selectivelybecause RIPA was a community-wide physician panel.This made a concurrent nonintervention physiciancontrol group impossible. Rochester is unusual in thatthe penetrance of a single HMO was more than 70% ofthe commercial (non-Medicare and non-Medicaid) populationduring the study. The high community penetranceand the large number of sinusitis cases madeprofiling individual practitioners practical. Reproducingthis model for an individual insurer or IPA with lowermarket penetration might necessitate pooling data fromother payers. Recently, the National Committee forQuality Assurance and the Institute of Medicine haveadvocated such a plan to achieve patient populationslarge enough to profile individual physicians.17
Because we used a historical control, we could notaccount for the effect of national trends in antibioticprescribing. However, the timing and the pattern of thechanges in sinusitis care that we observed are not consistentwith duplication of external trends. Two recentstudies11,18 have shown that overall antibiotic usedecreased nationally through the 1990s. In contrast,among Blue Choice HMO patients in the Rochesterarea, immediately before our first educational interventionin 1999, antibiotic prescribing for sinusitispeaked, at almost 970 prescriptions per 1000 episodesin October of that year. After our intervention, antibioticuse for acute sinusitis declined in peak seasons ata rate similar to that of other communities. It seemsunlikely that outside trends would happen to catch upwith the Rochester physician community at the sametime that we introduced our major intervention.
Statistical control chart analysis confirmed thatthere was a shift in the process of sinusitis treatmentafter our intervention. Nonetheless, the process wasnot completely "in control" before or after the intervention:several months in both periods had fewer ormore exceptions than would be expected. This analysissuggests that there may be other secondary causes forvariation in the process. For example, sinusitis may bemilder in May and June than in other months andtherefore attributed to allergic or viral causes. If thoseepisodes were treated less often with antibiotics, therewould be fewer opportunities for exceptions. Furtherstudies beyond the scope of this article would be neededto elucidate those secondary effects.
However, excluding the months not in control as notbeing statistically comparable yields an exception rateof 327 per 1000 episodes before the intervention and263 after the intervention, identical to the resultsobtained when including those episodes. Therefore,those secondary causes did not bias our results, andthey do not detract from the dramatic process shift thatoccurred after our intervention.
Our pattern of antibiotic use changed differentlyfrom national trends as well. Neither Steinman et al11nor McCaig et al18 showed a decrease in antibiotic prescribingspecifically for sinusitis. We saw a decrease inthe use of targeted broad-spectrum antibiotics, in contrastto national trends.11 Moreover, our interventionaffected behaviors outside of antibiotic use.
Our analysis included pediatric and adult patients.There could be concern that a difference in sinusitistreatment between pediatricians and other primarycare physicians would affect our results. We believe thatthat was not the case.
less effective or inappropriate
The acute sinusitis task force was careful to makesure that our recommendations were relevant to boththe adult and pediatric age groups. These basic pointsapply to children and to adults. All the appropriate firstlineor secondline antibiotics (except doxycycline inyoung children) can be used in pediatric and adult cases.The antibiotic designations for sinusitis apply to children and adults.
Our main results, a decrease in use of less appropriateantibiotics, an increase in use of firstline antibiotics,and a decrease in inappropriate sinus imaging, thereforeshould be independent of the age group examined.Our profiling data showed that exceptions per episodedecreased for all 3 primary care specialties. Nonetheless,differential guideline adherence among specialtiesremains an important issue beyond the scope ofthis article.
Administrative data are susceptible to inaccuracies.For example, we were not able to track antibiotics givenout as samples or purchased outside of HMO insurancecoverage. However, throughout the study, approximately90% of patients had prescription drug coverage (J.Owerbach, PhD, director of Excellus BlueCrossBlueShield pharmacy benefits, written communication,March 13, 2003).
Physician coding practices could be a source of inaccuracy.For example, a physician might code thosecases for which he or she decided to prescribe antibioticsas acute sinusitis and code the rest as viral illness.Some physicians might have used codes for chronic orallergic sinusitis, which were not included in our studyepisodes. To guard against this, we used multiple interventionsthroughout the study to promote accurate coding.Arguing against a change in coding practice is thebefore and after consistency of the number of sinusitisepisodes according to billing data. Furthermore, ouryearly incidence of sinusitis was approximately 7.6%(32 000 episodes among 420 000 patients). McCaig etal18 found an approximate 9% annual visit rate forsinusitis, suggesting that our physician sinusitis codingrate was similar to national patterns.
This model likely can be generalized to other acuteconditions. We created a similar pathway for otitismedia. We are in the process of analyzing the results ofthat pathway.
We believe that a similar system, modeling the recommendationsof Bodenheimer, Wagner, and Grumbach5,6and Casalino et al,19 can improve the care of chronic diseasesas well. Toward that end, Excellus BlueCrossBlueShield of the Rochester Region, RIPA, and theRochester Health Commission have received a grantfrom the Rewarding Results program to apply this systemto improving the care of patients with diabetes mellitus,asthma, coronary artery disease, and depression.
In 2000, Avorn and Solomon3 wrote:
Given the availability of information systems thattrack every detail of a physician's prescribing behaviorand can increasingly be linked to clinical data, more canbe done to improve prescribing. In the coming decade,it will become possible to assess physician prescribingin terms not merely of dollars spent but of clinicalintelligence.
We have demonstrated that currently available informationsystems, coupled with physician education,feedback, and a modest financial incentive, can assessand affect physician prescribing.
Acknowledgments
We thank Richard Besser, MD, and Robert Betts, MD, for theirhelpful comments during the preparation of the manuscript. Weacknowledge the assistance of David Francis, MD, regarding statisticalmethods. We would like to express our gratitude to Jean Vittfor her help in manuscript preparation, and to Excellus BlueCrossBlueShield analysis and information management and informationtechnology personnel, who were essential in implementing thecare pathways and physician profiles.
From the Rochester Individual Practice Association, Inc (RAG, HB, JC, GP), andExcellus BlueCross BlueShield of the Rochester Region (MM, DB, JK), Rochester, NY.
This work was partially funded by Rewarding Results, a joint initiative of the RobertWood Johnson Foundation, Princeton, NJ; The Commonwealth Fund, New York, NY; andthe California Healthcare Foundation, Oakland.
Address correspondence to: Robert A. Greene, MD, Rochester Individual PracticeAssociation, Inc, 3540 Winton Place, Rochester, NY 14623. E-mail: rgreene@ripa.org.
JAMA.
1. Davis D, O'Brien MAT, Freemantle N, Wolf F, Mazmanian P, Taylor-Vaisey A.Impact of formal continuing medical education: do conferences, workshops,rounds, and other traditional continuing education activities change physicianbehavior or health care outcomes? 1999;282:867-874.
J Gen Intern Med.
2. Rollman B, Hanusa B, Lowe H, Gilbert T, Kapoor W, Schulberg H. A randomizedtrial using computerized decision support to improve treatment of majordepression in primary care. 2002;17:493-503.
Ann Intern Med.
3. Avorn J, Solomon DH. Cultural and economic factors that (mis)shape antibioticuse: the nonpharmacologic basis of therapeutics. 2000;133:128-135.
JAMA.
4. Grol R. Improving the quality of medical care. 2001;284:2578-2584.
JAMA.
5. Bodenheimer T, Wagner EH, Grumbach K. Improving primary care for patientswith chronic illness. 2002;288:1775-1779.
JAMA.
6. Bodenheimer T, Wagner EH, Grumbach K. Improving primary care for patientswith chronic illness: the chronic care model, part 2. 2002;288:1909-1914.
Otolaryngol Head Neck Surg.
7. Sinus and Allergy Health Partnership. Antimicrobial treatment guidelines foracute bacterial rhinosinusitis. 2000;123(suppl 1,part 2):S1-S32.
8. Rochester Individual Practice Association, Inc. Report of the RIPA AcuteSinusitis Task Force. Available at: http://www.ripa.org/pathways.asp?cat=3.Accessed May 2, 2003.
9. Rochester Individual Practice Association, Inc, and Blue Choice (ExcellusBlueCross BlueShield of the Rochester Region). Understanding sinusitis. Availableat: http://www.ripa.org/documents/sinusitis.pdf. Accessed March 10, 2003.
Improving Healthcare With Control Charts: Basic and AdvancedSPC Methods and Case Studies.
10. Carey RG. Milwaukee, Wis: ASQ Quality Press; 2003:13-25.
Ann Intern Med.
11. Steinman MA, Gonzales R, Linder J, Landefeld CS. Changing use of antibioticsin community-based outpatient practice, 1991-1999. 2003;138:525-533.
JAMA.
12. Solomon DH, Hashimoto H, Daltroy L, Liang MH. Techniques to improvephysicians' use of diagnostic tests: a new conceptual framework. 1998;280:2020-2027.
Med Care.
13. Gross PA, Greenfield S, Cretin S, et al. Optimal methods for guideline implementation:conclusions from Leeds Castle meeting. 2001;39(suppl 2):II85-II92.
Ann Intern Med.
14. Phillips L, Branch W, Cook C, et al. Clinical inertia. 2001;135:825-834.
J Ambulatory Care Manag.
15. Smithson KW, Koster J. Incentives and the management of physician behaviorin health service organizations. 1997;20:8-16.
N Engl J Med.
16. Bodenheimer T. The American health care system. 1999;340:584-588.
Leadership by Example: CoordinatingGovernment Roles in Improving Health Care Quality.
17. Corrigan JM, Eden J, Smith BM, eds. Washington, DC: Institute ofMedicine; 2002. Available at:http://books.nap.edu/books/0309086183/html/index.html. Accessed March 10, 2003.
JAMA.
18. McCaig L, Besser R, Hughes J. Trends in antimicrobial prescribing rates forchildren and adolescents. 2002;287:3096-3102.
JAMA.
19. Casalino L, Gillies R, Shortell S, et al. External incentives, information technology,and organized processes to improve health care quality for patients withchronic diseases. 2003;289:434-441.
