Publication
Article
The American Journal of Managed Care
Changing Medication Use in Managed Care: A Critical Review of the Available Evidence
Objective: To review the effectiveness of strategies to improve the quality and efficiency of medication use in managed care organizations (MCOs).
Study Design: Systematic review of published intervention studies.
Methods: Studies were identified by using computerized and manual literature searches and personal contacts, and were categorized by intervention type and adequacy of research design according to commonly accepted criteria. Reported significance and magnitude of the changes in key outcomes were used to summarize the effects of studies with adequate research designs.
Results: The searches identified 105 studies, 70 of which were reported since 1996. Overall, 46% of the studies met the minimum criteria for methodologic adequacy (n = 48). Consistently effective interventions included dissemination of educational materials with drug samples, participatory clinical guideline development, group or one-to-one educational outreach, and enhanced patient-specific feedback. Disease management (primarily for depression and diabetes) showed promise in improving short-term outcomes. Dissemination of educational materials and aggregated feedback alone were ineffective. Interventions in staff-model health maintenance organizations were more effective than those conducted in group-model health maintenance organizations.
(Am J Manag Care 2003;9:715-731)
Conclusion: High-quality studies of interventions to improve drug use in MCOs are increasing in frequency. There is evidence for the effectiveness of several strategies to change drug use, but little is known about longer-term clinical outcomes. Few welldesigned, published studies have assessed the efficacy or safety of financial incentives for physicians, tiered copayments for patients, or formularies - despite their widespread use.
The structure of the US healthcare system has changed rapidly over the last few decades. Managed care organizations (MCOs), including health maintenance organizations (HMOs) and preferred provider organizations (PPOs), now dominate the marketplace, with more than 200 million enrollees1 and more than 90% of physicians having some form of managed care contract.2
One significant challenge facing MCOs today is controlling rising medical costs without denying or limiting access to care that improves health outcomes. Medication expenses, driven by increased ingredient costs per prescription and higher utilization, account for as much as 15% of the total medical costs of some health plans.1 A variety of approaches have been used by MCOs to improve the quality of medication use and contain escalating medication costs.
Managed care organizations have established financial arrangements that shift drug costs and risk from health plans to members and providers. In recent years, health plans have moved from fixed copayment structures to 3-tier pharmacy benefit designs, where members incur the lowest out-of-pocket costs for generic drugs, higher costs for preferred brand names, and the highest costs for non-preferred medications. The proportion of health plans offering a 3-tiered design rose from 36% in 1996 to 83% in 2001, the most pronounced change in pharmaceutical benefits management in that period.3 In contrast, fewer than 25% of HMOs used incentive-based financial contracting to influence physician prescribing in 2001, a decline of approximately 20% since 1999.1
Most MCOs use formularies to control medication use.1,4 Under certain circumstances, MCOs will pay for non-formulary medications. However, 84% of HMOs require physicians to obtain prior authorization to prescribe these medications.1 Some form of generic substitution is used in 86% of HMOs; and stepped-therapy protocols, which require patients to try older, lower-cost drugs in a therapeutic class before resorting to newer, higher-cost alternatives, are used by 76%.1 Therapeutic interchange, the substitution of a therapeutically equivalent drug for the one prescribed, is increasing in popularity and is used by approximately half of HMOs.1,4
Managed care organizations also are strengthening efforts to monitor and educate physicians to improve the quality of prescribing. Approximately two thirds use physician profiling (peer comparison feedback) in conjunction with recommendations for changes in practice. 5 Most education is targeted at high-cost physicians and frequently involves in-person pharmacist consultations. A number of plans also have launched member education programs to influence patients' demand for pharmaceuticals.6
Finally, many plans are introducing or expanding disease management interventions designed to improve care delivery and health outcomes for high-cost, high-risk populations.7 At least one quarter of Americans are enrolled in plans offering disease management programs, which most commonly focus on diabetes, asthma, and congestive heart failure.8 Although not viewed as a primary strategy for containing pharmaceutical costs, these programs frequently shift patterns of use of key medications.6
More than 80% of MCOs evaluate quality improvement initiatives on a regular basis,5,8 but anecdotes about the impacts of these efforts far outweigh solid evidence. The aims of this review are:
- To describe interventions to improve the quality and efficiency of medication use in the US managed care setting.
- To detail the key features of the interventions.
- To summarize the effects of the studies with methodologically acceptable designs.
- To identify intervention strategies that appear to be most successful in changing outcomes.
METHODS
Search Strategy
We performed a systematic review of published and unpublished studies of interventions to improve the quality or efficiency of medication use in MCOs. Studies were included in the review if they were conducted in the US managed care setting (all forms of HMOs, independent practice associations [IPAs], and PPOs); described an intervention or policy targeting drug use (including over-the-counter medications or herbals); included a clear description of methods; and measured drug-related outcomes. Clinical effectiveness trials of medications, descriptive studies, and those examining vaccinations were excluded.
We searched computerized retrieval systems (including Medline, Healthstar, Current Contents, Cochrane Collaboration, EMBASE, ASI, IPA, and the International Network for Rational Use of Drugs [INRUD] database) from 1966 through June 2001. We combined search terms that characterized the study setting (e.g., managed care programs, HMOs, PPOs), drug use (e.g., prescription and nonprescription drugs, drug therapy, drug utilization, drug monitoring, herbals, vitamins), health professions and practice patterns (e.g., physician, physician practice patterns, pharmacist, clinical pharmacy, pharmacy services, nurses), intervention types (e.g., formulary, disease and disease management, education, practice guidelines, cost containment, quality assurance/ improvement, risk sharing, reimbursement mechanisms) and methodology (e.g., intervention studies, randomized controlled trials [RCTs], program evaluation, health services research, comparative studies). We also hand-searched tables of contents of journals not included in the computerized databases, as well as reference lists from reviews and included studies.
Rating Study Quality
We abstracted important features of each study, including setting, objective(s), problem(s) addressed, intervention components, predominant intervention strategy, intervention target(s), and up to 3 primary outcome measures identified by the authors (an additional drug-related outcome was included if the primary outcomes were not drug related). We expressed study outcomes as either absolute or relative changes, depending on the original measurement scale. Outcomes measured as percentages were expressed as absolute differences in percentage; for other outcomes, measured as rates or scores, differences were expressed as relative changes (also stated as percentages).
We classified the research designs of the reviewed studies according to their ability to control for common threats to validity.9 Designs that met our definition for methodological adequacy included RCTs, pre/post studies with nonrandomized comparison group(s), and interrupted time series analysis with or without a comparison group. Designs deemed methodologically inadequate included pre/post studies without a comparison group, nonrandomized post-only designs, and cross-sectional studies. In addition, pre-post designs with comparison groups were required to statistically test differences in change between groups. Finally, we required sample sizes of at least 20 subjects in each comparison group to increase the reliability of reported findings.
RESULTS
Studies Identified
We identified 105 studies that met our inclusion criteria, of which 48 (46%) met the minimally acceptable methodological criteria. Intervention studies are being published with increasing frequency; 70% of all studies and 58% of acceptable studies were published since 1996. The majority of studies were conducted in staff, group-, and mixed-model settings; only 7 studies were conducted in IPA and network-model organizations.
Features of Studies
Studies were classified according to predominant strategy and key components of the intervention (Table 1). Disease management and educational studies each accounted for 31% of studies, and monitoring and feedback interventions accounted for another 19% of studies. Despite the almost universal use of formularies or financial strategies,1 studies evaluating such policies were relatively few (12% and 6% of all studies, respectively). Interventions almost always were multimodal.
Twenty-five studies of disease management, 10 of educational interventions, and 9 of monitoring and feedback met minimum criteria for methodological adequacy. Acceptable studies of formulary changes and financial strategies were too few (4 formulary and 1 financial) to draw valid conclusions, and were excluded from further analysis. Table 2 summarizes the key aspects of the 48 interventions evaluated in the 44 published studies.
Educational Interventions
Dissemination of Educational Materials Alone.
The dissemination of educational materials alone (eg, newsletters, drug bulletins, audiovisual materials) is a common intervention and assumes that rational information is sufficient to change behavior. Two interventions evaluated the effects of such programs. The first aimed to improve patient compliance with a range of cardiac risk reduction therapies (benazepril, metoprolol, simvastatin, and transdermal estrogen) by mailing videotapes about prescribed medications and associated disease states.10 The second attempted to decrease total antibiotic use for acute bronchitis by placing posters and information sheets in primary care examination rooms. Both interventions were unsuccessful in changing target outcomes.11
Dissemination of Educational Materials with Drug Samples.
For many years pharmaceutical companies have given healthcare providers free drug samples to distribute to patients to increase use of new medications or reinforce the use of existing medications. This strategy was combined with the dissemination of educational materials in 2 interventions. Providing 2 quarts of oral rehydration solution (ORS) to the parents of children with an office visit for acute diarrhea increased ORS use and decreased unscheduled follow-up visits.12 A second study, providing patients with a 30-day supply of atenolol to promote prescription refill compliance, resulted in more new and existing patients acquiring a 180-day supply of the drug.13
Group Education.
Group education relies on didactic or problem-based approaches to effect behavior change.14 Despite the widespread use of conferences, seminars, and tutorials to educate physicians, only 4 studies in this review used group education as the primary strategy to improve medication use. One intervention successfully decreased the number of antibiotics dispensed to children less than 6 years of age by using group detailing conducted by a "peer leader," brochures on antibiotic prescribing endorsed by the Centers for Disease Control and Prevention (CDC), and feedback on previous prescribing, in conjunction with CDC brochures mailed to patients and supporting materials in waiting rooms.15 Another intervention reduced antibiotic prescribing for acute uncomplicated bronchitis in adults without increasing the number of non-antibiotic prescriptions and return office visits by combining group detailing, feedback on previous prescribing, mailed written materials and magnets sent to patients' homes, and posters in physicians' offices.11 A small-group patient education program conducted by a nurse educator, coupled with printed and audiovisual materials and asthma management devices, increased daily use of steroid inhalers and peak flow meters. However, the proportion of patients requiring specialty care did not change.16
The final study, using an 11-member continuous quality improvement (CQI) team, attempted to increase the percentage of patients taking antidepressant medication and reduce self-reported depression symptoms.17 Although the team produced a number of recommendations to improve adherence to published guidelines (e.g., a hotline for psychiatric consultations, a primary care-based screening and diagnosis process, a method for pharmacist-assisted management of depression care), only the hotline was implemented. Not surprisingly, significant changes in process or outcomes were not found.
Participatory Guideline Development.
Participatory guideline development involves a structured group process that combines learning, consideration of evidence about appropriate treatment, and discussion about how actual practice conforms with group standards. An intervention utilizing this approach for allergic rhinitis was associated with modest improvements in patients' self-reported allergy prevention scores and reductions in indirect costs.18,19
One-to-One Educational Outreach.
Helicobacter pylori
One-to-one educational outreach (often referred to as academic detailing) is among the most effective techniques used to change prescribing.20 Two interventions in this review were effective in changing medication use. A patient-focused intervention used one-to-one clinical pharmacist counseling coupled with medication coupons for subsequent prescriptions, follow-up telephone calls, medication pillboxes, and medication calendars to improve patient compliance with bismuth subsalicylate, metronidazole, and tetracycline hydrochloride therapy for infection.21 Another study used 4 one-to-one pharmacist visits to physicians to increase the percentage of depressed patients taking antidepressant medications and decrease self-reported depressive symptoms.17
Monitoring and Feedback Interventions
Feedback interventions report a physician's past or current prescribing and are frequently compared with peer behavior or accepted standards. The assumption underlying these strategies is that physicians are likely to change their behavior due to discomfort about outlier status or as a result of specific recommendations from credible individuals or programs.22
Lists of Prescribed Medications.
Monthly profiles of medications taken by patients, inserted into their medical records, had no impact on the number of prescriptions written or on expenditures.23
Aggregated Feedback.
One study used a mailed memo to physicians promoting cimetidine over other histamine H2-receptor antagonists (H2 blockers), along with aggregated reports about the number, type, and cost of their H2-blocker prescriptions.24 Compared with a mailed memo alone, the medication listings had no impact on cimetidine prescribing. However, a significant response to the intervention was observed in the group-model but not in the network-model HMO.
Patient-Specific Feedback.
The effects of patient-specific feedback were mixed. An intervention designed to reduce costs by shifting prescribing from one antihistamine (loratidine) to another (fexofenadine) included letters to physicians about the preferred medication plus lists of patients taking loratidine.25 However, this feedback was unsuccessful in changing the market share of the 2 drugs. In another study, an automated record system with feedback to physicians about individual patients who did not comply with standards of hypertension care was unable to improve initial laboratory testing, blood pressure control, or patient follow-up compared with a peer comparison intervention.26 In addition, written feedback to physicians on patient-specific compliance with depression treatment algorithms was unable to increase the proportion of patients receiving adequate pharmacotherapy or to reduce patient visits and self-reported depressive symptoms.27
Two feedback interventions were successful in achieving improved outcomes. One involved concurrent audit to detect deficiencies in patient care and automated rapid feedback to the responsible provider in time for the provider to correct the deficiency.28 The system resulted in an improvement in the follow-up of positive throat cultures for group A beta-hemolytic streptococcus (strep throat). The other study used weekly surveillance, with messages recommending actions for individual patients according to predefined hypertension protocols. In patients with newly discovered elevated diastolic blood pressure, the system improved patient follow-up, increased the number of repeat blood pressure measurements, and increased the number of patients with a diastolic blood pressure of less than 100 mm Hg.29
Enhanced Patient-Specific Feedback.
All of the interventions using patient-specific feedback combined with additional strategies (most often educational) improved outcomes. In the previously described study aimed at decreasing the number of prescriptions written for loratidine and increasing the number of prescriptions written for fexofenadine, patient-specific feedback alone was unsuccessful. However when educational fact sheets for physicians, letters to patients, and a coupon for the first fexofenadine prescription were added, the market share of fexofenadine was increased while use of loratidine was reduced.25 A multifaceted intervention involving letters and materials mailed to physicians, patient-specific therapeutic reevaluation forms, and performance feedback shifted prescribing to cimetidine rather than other H2-receptor antagonists; the magnitude of the effect was greater in a staff-model versus a group-model HMO.30 Another program successfully replaced short-acting nifedipine with alternative medications by using preprinted letters for physicians to mail to patients (either indicating an automatic change in medication or requesting that the patient schedule an appointment for medication review) plus physician education (treatment guidelines for ischemic heart disease and a summary of the evidence questioning the safety of short-acting nifedipine). 31 In the same depression study mentioned previously, an intervention using the feedback strategy plus care management (telephone monitoring of antidepressant use, side effects, and severity of depression symptoms) increased the number of patients receiving adequate pharmacotherapy and decreased self-reported depressive symptoms, but did not change the number of patient visits.27
Disease Management Interventions
Disease management programs are population-based approaches for identifying, treating, and monitoring persons at risk of chronic illness.8 These programs improve care by providing teams of trained healthcare personnel and organized systems of care.7
Depression.
Patients with depression tend to have high medical costs, multiple unexplained symptoms, and greater social and vocational disability compared with other patients.32 Stepped collaborative care interventions33- 40 educate patients, bolster the function of primary care physicians (PCPs), provide on-site psychiatric consultants, and provide pharmacologic treatment when necessary. For patients with major depression at baseline, this approach improved adherence to antidepressant medication at 90 days,33 6 months,34 and 12 months,37 and decreased self-reported depressive symptoms at 90 days33 and 7 months.36 However, the improved outcomes seen in the early stages of some interventions33,36 did not persist at 24 months.39
A program using patient education, physician training, telephone treatment coordination, and PCP-managed antidepressant therapy increased antidepressant medication use and decreased self-reported depressive symptoms. However, self-reported physical functioning did not change.41 A psychiatric consultation service coupled with physician education also improved antidepressant medication compliance and the number of patients receiving adequate antidepressant treatment.42 In addition, an approach using only nurse-managed telephone care decreased self-reported depressive symptoms and increased antidepressant use and patient satisfaction.43
In a quality improvement program, primary care leaders trained nursing and medical staff to administer psychotherapy and pharmacotherapy.44-47 The program decreased depressive symptomatology, increased antidepressant medication use, and increased health-related quality of life.44 However, these effects were not maintained at 24 months.45 The average costs of the intervention were significantly higher than those of usual care. However, intervention patients had fewer days with depression burden and were employed for more days during the study period.46 The intervention increased PCP knowledge of psychotherapy but did not change knowledge of depression treatment and antidepressant medication.47
Anxiety.
Anxiety disorders are associated with significant morbidity, disability, and cost.48 Psychologist and PCP teams used psychotherapy and medication to reduce self-reported symptoms and increase satisfaction with care for patients with generalized anxiety disorder or anxiety secondary to depression.49
Diabetes.
Fewer than half of diabetic patients have good glycemic control, which is associated with improved clinical outcomes.50 The following interventions had mixed success at reducing blood glucose levels. A CQI program, involving modifications to the diabetes care process, and educational outreach by trained diabetes resource nurses reduced patient glycosylated hemoglobin (HbA1c) levels at 12 months but did not reduce the number of outpatient medical visits.51
Clinics involving physicians, nurse and pharmacist visits, and peer support meetings improved self-reported general health and increased primary care visits, but had no impact on reducing HbA1c levels.52 A program coordinated by a diabetes nurse educator–also involving psychologists, nutritionists, and pharmacists– improved HbA1c levels and the number of visits to a nutritionist, but did not increase home blood glucose monitoring.53 A pharmacist-managed program successfully reduced HbA1c levels but not blood pressure,54 whereas a nurse-coordinated program under the supervision of a PCP and an endocrinologist reduced HbA1c and fasting blood glucose levels, but had no impact in reducing blood pressure.55
Cardiac Risk Factor Modification.
Systematic modification of coronary risk factors is rarely integrated into the medical care of patients treated for acute myocardial infarction.56 A physician-directed and nurse-managed program increased smoking cessation and patient functional capacity and reduced plasma low density lipoprotein.56
Multiple Conditions.
Group outpatient visits with a PCP and nurse plus one-to-one consultations with the PCP reduced primary care, subspecialist physician, and same-day visits. However, the number of prescriptions filled remained unchanged.57 Chronic care clinics involving nurse, pharmacist, and physician visits and peer support failed to improve self-reported physical functioning or reduce self-reported depression, incontinence, or the number of high-risk medications filled at 24 months.58 Remote home healthcare, using telephone and video visits to replace some in-person visits, was as effective as in-person care, as measured by medication compliance, patient knowledge, and achievement of self-care.59
DISCUSSION
Principal Findings
Several findings from this review are consistent with systematic reviews of interventions in other settings. These data confirm that dissemination of educational materials alone is ineffective and unlikely to change behavior.60,61 Implementation of clinical practice guidelines is effective when combined with a local consensus process,62 and one-to-one educational outreach visits are effective in reducing inappropriate prescribing.14,61 Further, multifaceted interventions that help to predispose, enable, and reinforce desired behaviors are more likely to be successful.63
Of the 3 provider-focused group interventions in our review, 2 reported improvements in prescribing practices. The third study, a CQI program, proved ineffective. However, the latter finding is not surprising given previous work indicating that well-controlled CQI programs are frequently ineffective in changing processes of care and health outcomes.64
Previous evaluations of the effectiveness of monitoring and feedback interventions reported small to moderate changes in the quality and cost of care.65-67 The studies in this review support the effectiveness of surveillance systems with patient-specific feedback to physicians, especially when coupled with physician or patient education programs.
Our review also contributes findings that have not been addressed previously. First, the dissemination of educational materials with free medication samples was effective in improving patient compliance. This is a potentially important finding that should be investigated further.
Two studies in this review suggest that interventions are more effective in more heavily managed settings. 24,30 This finding is not surprising as healthcare providers in staff-model HMOs are employed solely by the MCO and have strong relationships with other providers such as pharmacists. In contrast, physicians in group-model HMOs are only affiliated and are not as well integrated with MCOs. For example, staff pharmacists employed in staff-model plans are more likely to be viewed as clinical partners and educators, whereas less integrated pharmacists in IPA settings often are viewed as regulators and cost cutters.68
To our knowledge, this review is the first to examine systematically the effects of disease management programs incorporating pharmacotherapy as a part of the interventions. Our results show that programs focusing on a specific chronic illness are generally effective in changing processes of care and to a lesser extent in improving short-term health outcomes. The results are not as encouraging in the few studies that examined outcomes beyond 12 months.
Limitations
This review has a number of methodological limitations. Despite our intensive efforts, the collection of intervention studies is incomplete. Many studies on improving pharmaceutical use are not available in the public domain, especially those conducted within MCOs or by the pharmaceutical industry. Publication bias may be present and lead to an overestimate of intervention impacts (for example, well-controlled, negative studies may not have been submitted for publication). Furthermore, study designs were frequently inadequate; more than half of the studies identified in this review were not considered because of design flaws. There also are fundamental difficulties in comparing interventions with diverse objectives, measurement methods, and outcomes. Many of the reviewed interventions are multifactorial and difficult to characterize in a simple classification.
Unanswered Questions and Future Directions
Despite the substantial number of interventions to improve drug use in managed care, our understanding of the impacts of these interventions still is limited. Although PPOs and lightly managed HMOs play prominent roles in the US managed care industry, we found no studies conducted in these settings. The majority of studies in our review were conducted in staff-, group-, and mixed-model HMOs, more heavily managed settings that now play a diminished role in the market. Future research should establish which interventions are effective in less heavily managed settings.
There is a glaring lack of evidence concerning the effects of financial and formulary-related interventions. It is alarming to consider how little publicly available empirical evidence underlies the most common approaches used in managed care today. Because of the far-reaching impact that these strategies may have on outcomes, both intended and unintended, it is imperative that managed care policy decisions be informed by rigorous evidence that addresses both cost and quality of care.
Little attention has been paid to assessing the cost-effectiveness of interventions. Cost-effectiveness data are needed to select among many different approaches for improving the costs and quality of care.
CONCLUSIONS
This review provides encouraging evidence about the growing number and quality of interventions to improve medication use in the last 5 years. As evidenced by the studies in this review, policymakers in managed care environments now have an improved knowledge base on which to build educational, monitoring and feedback, and disease management programs to improve medication use. Despite the widespread use of financial and formulary strategies, there are few publicly available, well-designed studies assessing their efficacy and safety. We encourage MCOs and other organizations to publish existing studies or to develop research initiatives in these areas to further expand our evidence base.
1. Aventis Pharmaceuticals Inc. HMO-PPO/Medicare-Medicaid Digest: Managed Care Digest Series. Bridgewater, NJ: Aventis Pharmaceuticals; 2002.
Physician Socioeconomic
2. Wassenaar JD, Thran SL, eds. Statistics: 2000-2002 Edition. Chicago, Ill: American Medical Association; 2001.
3. 2001 Managed Care Formulary Drug Audit. Newtown, Pa: Scott-Levin; 2002.
4. Litton LM, Sisk FA, Akins ME. Managing drug costs: the perception of managed care pharmacy directors. Am J Manag Care. 2000;6:805-814.
5. Gold MR, Hurley R, Lake T, Ensor T, Berenson RA. A national survey of the arrangements managed-care plans make with physicians. N Engl J Med. 1995;333:1678-1683.
6. Mays GP, Hurley RE, Grossman JM. Consumers face higher costs as health plans seek to control drug spending. Center for Studying Health System Change. 2001;45:1-4.
7. Bodenheimer T. Disease management—promises and pitfalls. N Engl J Med. 1999;340:1202-1205.
8. Welch WP, Bergsten C, Cutler C, Bocchino C, Smith RI. Disease management practices of health plans. Am J Manag Care. 2002;8:353-361.
9. Cook TD, Campbell DT. Quasi-Experimentation: Design & Analysis Issues for Field Settings. Boston, Mass: Houghton Mifflin Company; 1979.
10. Powell KM, Edgren B. Failure of educational videotapes to improve medication compliance in a health maintenance organization. Am J Health Syst Pharm. 1995;52:2196-2199.
11. Gonzales R, Steiner JF, Lum A, Barrett PH, Jr. Decreasing antibiotic use in ambulatory practice: impact of a multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults. JAMA. 1999;281:1512-1519.
12. Duggan C, Lasche J, McCarty M, et al. Oral rehydration solution for acute diarrhea prevents subsequent unscheduled follow-up visits. Milbank Q. 1999;104:e29.
13. Sclar DA, Chin A, Skaer TL, et al. Effect of health education in promoting prescription refill compliance among patients with hypertension. Clin Ther. 1991:13:489-495.
14. Soumerai SB, McLaughlin TS, Avorn J. Improving drug prescribing in primary care: a critical analysis of the experimental literature. Milbank Q. 1989;67:268-317.
15. Finkelstein JA, Davis RL, Dowell SF, et al. Reducing antibiotic use in children: a randomized trial in 12 practices. Milbank Q. 2001;108:1-7.
16. Legorreta AP, Leung KM, Berkbigler D, Evans, Liu R. Outcomes of a population-based asthma management program: quality of life, absenteeism, and utilization. Ann Allergy Asthma Immunol. 2000;85:28-34.
17. Brown JB, Shye D, McFarland BH, et al. Controlled trials of CQI and academic detailing to implement a clinical practice guideline for depression. Jt Comm J Qual Improv. 2000;26: 39-54.
18. Gregory C, Cifaldi M, Tanner LA. Targeted intervention programs: creating a customized practice model to improve the treatment of allergic rhinitis in a managed care population. Am J Manag Care. 1999;5:485-496.
19. Santos R, Cifaldi M, Gregory C, Seitz P. Economic outcomes of a targeted intervention program: the costs of treating allergic rhinitis patients. Am J Manag Care. 1999;5:S225-S234.
20. Soumerai SB, Avorn J. Principles of educational outreach ("academic detailing") to improve clinical decision making. JAMA.1990:263:549-556.
21. Lee M, Kemp JA, Canning A, et al. A randomized controlled trial of an enhanced patient compliance program for Helicobacter pylori therapy. Arch Intern Med. 1999;159:2312-2316.
22. Eisenberg JM. Doctors' Decisions and the Cost of Medical Care. Ann Arbor, Mich: Health Administration Press Perspectives; 1986.
23. Johnson RE, Campbell WH, Azevedo DJ, Christensen DB. Studying the impact of patient drug profiles in an HMO. Med Care. 1976;14:799-807.
24. Schectman JM, Kanwal NK, Schroth WS, Elinsky EG. The effect of an education and feedback intervention on group-model and network-model health maintenance organization physician prescribing behavior. Med Care. 1995;33:139-144.
25. Benedetto SR, Sloan AS, Duncan BS. Impact of interventions designed to increase market share and prescribing of fexofenadine at HMOs. Am J Health Syst Pharm. 2000;57:1778-1785.
26. Winickoff RN, Wilner S, Neisuler R, Barnett GO. Limitations of provider interventions in hypertension quality assurance. Am J Public Health. 1985;75:43-46.
27. Simon GE, Von Korff M, Rutter C, Wagner E. Randomised trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care. BMJ. 2000;320:550-554.
28. Barnett GO, Winickoff R, Dorsey JL, Morgan MM, Lurie RS. Quality assurance through automated monitoring and concurrent feedback using a computer-based medical information system. Med Care. 1978;16:962-970.
29. Barnett GO, Winickoff RN, Morgan MM, Zielstorff RD. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care. 1983;21:400-409.
30. Brufsky JW, Ross-Degnan D, Calabrese D, Gao X, Soumerai SB. Shifting physician prescribing to a preferred histamine-2-receptor antagonist. Effects of a multifactorial intervention in a mixedmodel health maintenance organization. Med Care. 1998;36: 321-332.
31. Kaplan RC, Psaty BM, Kriesel D, et al. Replacing short-acting nifedipine with alternative medications at a large health maintenance organization. Am J Hypertens. 1998;11:471-477.
32. Wells KB, Stewart A, Hays RD, et al. The functioning and well-being of depressed patients. Results from the Medical Outcomes Study. JAMA. 1989;262:914-919.
33. Katon W, Von Korff M, Lin E, et al. Collaborative management to achieve treatment guidelines. Impact on depression in primary care. JAMA. 1995;273:1026-1031.
34. Katon W, Von Korff M, Lin E, et al. Stepped collaborative care for primary care patients with persistent symptoms of depression: a randomized trial. Arch Gen Psychiatry. 1999;56:1109-1115.
35. Lin EH, Von Korff M, Russo J, et al. Can depression treatment in primary care reduce disability? A stepped care approach. Arch Fam Med. 2000;9:1052-1058.
36. Katon W, Robinson P, Von Korff M, et al. A multifaceted intervention to improve treatment of depression in primary care. Arch Gen Psychiatry. 1996;53:924-932.
37. Katon W, Rutter C, Ludman EJ, et al. A randomized trial of relapse prevention of depression in primary care. Arch Gen Psychiatry. 2001;58:241-247.
38. Simon GE, Katon W, Rutter C, et al. Impact of improved depression treatment in primary care on daily functioning and disability. Psychol Med. 1998;28:693-701.
39. Lin EH, Simon GE, Katon WJ, et al. Can enhanced acutephase treatment of depression improve long-term outcomes? A report of randomized trials in primary care. Am J Psychiatry. 1999;156:643-645.
40. Von Korff M, Katon W, Bush T, et al. Treatment costs, cost offset, and cost-effectiveness of collaborative management of depression. Psychosom Med. March/April 1998;60:143-149.
41. Katzelnick DJ, Simon GE, Pearson SD, et al. Randomized trial of a depression management program in high utilizers of medical care. Arch Fam Med. 2000;9:345-351.
42. Katon W, Von Korff M, Lin E, Bush T, Ormel J. Adequacy and duration of antidepressant treatment in primary care. Med Care. 1992;30:67-76.
43. Hunkeler EM, Meresman JF, Hargreaves WA, et al. Efficacy of nurse telehealth care and peer support in augmenting treatment of depression in primary care. Arch Fam Med. 2000;9:700-708.
44. Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care: a randomized controlled trial. JAMA. 2000; 283:212-220.
45. Sherbourne CD, Wells KB, Duan NH, et al. Long-term effectiveness of disseminating quality improvement for depression in primary care. Arch Gen Psychiatry. 2001;58:696-703.
46. Schoenbaum M, Unutzer J, Sherbourne C, et al. Cost-effectiveness of practice-initiated quality improvement for depression: results of a randomized controlled trial. JAMA. 2001;286:1325-1330.
47. Meredith LS, Jackson-Triche M, Duan NH, et al. Quality improvement for depression enhances long-term treatment knowledge for primary care clinicians. J Gen Intern Med. 2000;15:868-877.
48. Franco K, Tamburino M, Campbell N, et al. The added costs of depression to medical care. PharmacoEconomics. 1995;7(4): 284-291.
49. Price D, Beck A, Nimmer C, Bensen S. The treatment of anxiety disorders in a primary care HMO setting. Psychiatr Q. 2000; 71:31-45.
50. The Diabetes Control and Complications Research Group. The effects of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993;329:977-986.
51. O'Connor PJ, Rush WA, Peterson J, et al. Continuous quality improvement can improve glycemic control for HMO patients with diabetes. Arch Fam Med. 1996;5:502-506.
52. Wagner EH, Grothaus LC, Sandhu N, et al. Chronic care clinics for diabetes in primary care: a system-wide randomized trial. Diabetes Care. 2001;24:695-700.
53. Sadur CN, Moline N, Costa M, et al. Diabetes management in a health maintenance organization. Efficacy of care management using cluster visits. Diabetes Care. 1999;22:2011-2017.
54. Kelly C, Rodgers PT. Implementation and evaluation of a pharmacist-managed diabetes service. J Manag Care Pharm. 2000;6:488-493.
55. Aubert RE, Herman WH, Waters J, et al. Nurse case management to improve glycemic control in diabetic patients in a health maintenance organization. A randomized, controlled trial. Ann Intern Med. 1998;129:605-612.
56. DeBusk RF, Miller NH, Superko HR, et al. A case-management system for coronary risk factor modification after acute myocardial infarction. Ann Intern Med. 1994;120:721-729.
57. Beck A, Scott J, Williams P, et al. A randomized trial of group outpatient visits for chronically ill older HMO members: the Cooperative Health Care Clinic. J Am Geriatr Soc. 1997;45:543-549.
58. Coleman EA, Grothaus LC, Sandhu N, Wagner EH. Chronic care clinics: a randomized controlled trial of a new model of primary care for frail older adults. J Am Geriatr Soc. 1999;47:775-783.
59. Johnston B, Wheeler L, Deuser J, Sousa KH. Outcomes of the Kaiser Permanente Tele-Home Health Research Project. Arch Fam Med. 2000;9:40-45.
60. Freemantle N, Harvey EL, Wolf F, et al. Printed educational materials to improve the behavior of health care professionals and patient outcome [update software]. Cochrane Database Syst Rev. 2002;3.
61. Anderson GM, Lexchin J. Strategies for improving prescribing practice. Can Med Assoc J. 1996;154:1013-1017. 62. Implementing clinical guidelines. Can guidelines be used to improve clinical practice? Effective Health Care. 1994:1(8).
63. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider behavior: an overview of systematic reviews of interventions. Med Care. 2001;39:II2-45.
64. Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality improvement on clinical practice: what will it take to accelerate progress. Milbank Q. 1998;76:1-37.
65. Mugford M, Banfield P, O'Hanlon M. Effects of feedback of information on clinical practice. BMJ. 1991;303:398-402.
