Evaluation of Laboratory Monitoring Alerts Within a Computerized Physician Order Entry System for Medication Orders

Background: Errors involving medication use are common. Computerized physician order entry (CPOE) can improve prescribing practices. Few studies have examined the effect of CPOE in combination with decision support tools on prescribing practices in the outpatient setting. Less is known about prescribers' adherence to laboratory monitoring recommendations.

Objective: To evaluate if reminders presented during CPOE for medications would increase physicians' compliance with guidelines for laboratory monitoring at initiation of therapy.

Study Design: Randomized prospective intervention study.

Methods: Two hundred seven primary care physicians in a group-model managed care organization were randomized to receive or not receive drug laboratory monitoring alerts within the CPOE system. Adherence to laboratory monitoring recommendations for patients prescribed selected medications was compared between physician groups.

P

P

P

Results: There was no significant difference between the control and intervention group physicians in the overall rate of compliance with ordering the recommended laboratory monitoring for patients prescribed study medications. Laboratory monitoring was performed as recommended 56.6% of the time in the intervention group compared with 57.1% of the time in the control group (= .31). In cases in which a statistically significant difference was demonstrated, improved compliance favored the intervention group (eg, 71.2% vs 62.3% [= .003] for gemfibrozil and 75.7% vs 73.9% [= .05] for statins).

Conclusions: As CPOE becomes more prevalent, additional research is needed to determine effective decision support tools. These findings then should be communicated to the developers and users of computerized medical record systems.

(Am J Manag Care. 2006;12:389-395)

To Err Is Human:

Building a Safer Health System

The Institute of Medicine report identified the prevention of medication errors as one of the priority areas for transforming healthcare.¹ The Institute of Medicine also stated that increasing the use of information technology is a priority to reduce the error rate during the delivery of healthcare.² Errors in medication use are one of the most common types of medical errors.^3-9 In the Harvard Medical Practice Study II,⁷ adverse drug events accounted for 19% of injuries in hospitalized patients. Medication use in the ambulatory care setting carries similar risks.^10-15 For example, Gandhi et al¹¹ found that 25% of ambulatory patients experienced adverse drug events. Furthermore, published research suggests that 25% to 75% of all adverse drug events are preventable.^3,11,16

Errors can occur at several points in the process of medication use, including drug ordering, dispensing, administration, and monitoring for efficacy or toxicity. One of the strategies advocated to prevent medication dispensing errors is the adoption of alerts and reminders at the point of ordering medications within a computerized physician order entry (CPOE) system. Studies have shown that computerized order entry has improved prescribing practices,¹⁷ as well as the appropriateness of ordering diagnostic tests.¹⁸ In controlled settings, computer-generated reminders increase physicians' adherence to practice guidelines.^19,20 Using CPOE within the inpatient setting decreased the inappropriate use of medications and improved patient outcomes.^17,21

Electronic medical record (EMR) systems and decision support systems use nonintrusive or intrusive alerts and reminders. Nonintrusive alerts generate warnings or present information on the computer screens but do not require specific actions. Intrusive alerts do not allow EMR users to complete an order or task until they respond to the alerts by canceling the alert message, changing the order, or giving a reason why they are continuing with the current task without changes.²² A disadvantage of intrusive alerts and reminders is their potential to intrude into and to disrupt the clinical work flow. In addition, nonintrusive and intrusive alerts, if frequent, tend to be ignored and foster "work-arounds" to avoid the alerts. Users of CPOE who were surveyed indicated that nonintrusive alerts and guidelines were preferred as the method for presenting drug prescribing information, although they also acknowledged that these would likely be less effective than a more intrusive alert.²² Nonintrusive interventions are less intrusive to work flow.

Many medication safety initiatives have focused on detecting medication interactions or risky medication use in high-risk patients. For certain medications with well-defined undesirable organ-system effects, clinicians should monitor patients' key laboratory values before starting therapy and at regular intervals after starting therapy. There should also be a focus on medication safety initiatives to ensure laboratory monitoring. Few studies have examined the effect of CPOE in combination with decision support tools on prescribing practices or on laboratory monitoring in the outpatient setting. A new arena for studying practical applications of computerized alerts and reminders is the use of computerized medication ordering combined with nonintrusive decision support information in the outpatient setting to guide clinicians in the appropriate use of laboratory monitoring. The objective of this study was to evaluate whether nonintrusive reminders presented on the computerized order entry screen for medication orders in the outpatient setting would increase clinicians' compliance with guidelines for laboratory monitoring on initiation of selected medications.

METHODS

Study Design

lexicon

The setting for this study was a group-model managed care organization with more than 350 000 members at the time the study was conducted. During the study period, the organization used a fully integrated EMR with a CPOE system. This proprietary system (Clinical Information System) was developed in a joint venture with IBM (Boulder, Colo). The system was entirely "paperless," it was used at every outpatient encounter, and the various sections (eg, outpatient documentation, pharmacy, laboratory, and radiology) interacted with each other. The total population of healthcare professionals who used the Clinical Information System included approximately 550 physicians, 3900 health plan staff, and 100 medical students or resident physicians. The Clinical Information System was used to document all patient care contacts in the outpatient setting. The system used a controlled medical terminology (also called the ) that allowed clinicians to use expressive clinically accurate terms to document interventions (including synonyms, acronyms, and abbreviations). The controlled medical terminology lexicon was used to document patient complaints and assessments and to order tests and medications during the delivery of patient care. All patient progress notes, medication orders, and laboratory results were archived for the purpose of retrieval, research, and analysis. This system was continuously available to providers at 16 ambulatory practice sites. Between 1998 and 2003, more than 10 million medications were prescribed using this system. Within the EMR, each clinician had a "custom formulary" of medications downloaded into his or her computerized order entry index.

Physicians' Desk Reference

After approval by the Kaiser Permanente Institutional Review Board, internal medicine and family practice physicians within the 16 clinical facilities of the managed care organization were randomized into 2 groups. The randomization was performed using SAS (SAS Institute Inc, Cary, NC). Physicians randomized to the control group received the standard list of medications loaded into their custom formularies. Physicians randomized to the intervention group also received the standard list of medications, but their medication list contained additional information in the pharmacy information field of the order entry screen that was specific to recommended laboratory monitoring for selected medications (Table 1). This field was positioned directly under the "instructions for use" field of the order entry screen and was clearly visible at the time of ordering each time an intervention physician ordered one of the medications listed in Table 1. The intent was to make the decision support function (guideline information) closely linked to each specific medication and integrated within the order entry system. The added information was specific for the individual medication and presented guidelines for appropriate baseline (eg, monitoring at therapy initiation) and ongoing (eg, monitoring during continuing therapy) laboratory monitoring for that particular medication, although baseline laboratory monitoring is the focus of the research we present herein. Development of the alert messages has been previously described.²³ Briefly, drugs and laboratory tests were selected based on the presence of Food and Drug Administration black box warnings, published clinical guidelines, and potentials for adverse clinical consequences related to lack of monitoring. Black box warnings are typically used for drugs that carry the potential for life-threatening adverse events. In a sequential process, first the (http://www.pdr.net/) was reviewed to identify drugs prescribed in ambulatory care that had black box warnings for baseline laboratory monitoring. The information gleaned from this review was supplemented with information from the Food and Drug Administration Web site (http://www.fda.gov/). Next, nationally available published guidelines and internal clinical guidelines were searched for other medication-related laboratory monitoring recommendations. A draft list of recommended drug-laboratory monitoring pairs was compiled from these sources and was circulated to practicing physicians, clinical pharmacists, and clinician leaders in the health plan. Their feedback was incorporated into the final list of drugs requiring laboratory monitoring. Table 1 lists the medications and the laboratory monitoring alerts associated with each medication that were displayed on the order entry screen and that we analyzed for baseline laboratory monitoring. Several clinical physicians reviewed the language of the alerts to ensure clarity and usability. The nonintrusive alerts were modeled after a previously published study²³ that used intrusive alerts at the point of filling the prescription in the pharmacy, which was shown to work well.

Physicians in the intervention arm received academic detailing before the start of the study. During academic detailing, each intervention clinician received 1-on-1 training sessions designed to teach him or her where to look for the information on the order entry screen and how to use this information to change medication dosing or to order laboratory tests at the time of order entry. Only intervention group physicians were made aware of the nonintrusive alerts. Control group physicians were not informed of the study and did not receive the nonintrusive alerts on the order entry screen.

The study period included November 1, 2002, through October 31, 2003. After the study period ended, prescribing data for study medications were analyzed for the patients receiving the medications. For patients prescribed medications that had laboratory monitoring recommendations associated with them, we queried the laboratory database to determine if the appropriate laboratory tests had been completed within 2 weeks after the medication order had been dispensed to the patient. This time frame was chosen because it indicated that the laboratory test had been obtained within an appropriate time frame relative to when the medication was dispensed to the patient. In addition, we included "recently performed" laboratory tests in the definition of appropriate laboratory testing. We defined recently performed laboratory tests as those tests completed within 180 days before dispensing of the medication and 14 days after dispensing of the medication. The clinician was defined as having followed the laboratory monitoring guideline if results of completed laboratory tests were available for review in the EMR within these same time frames. Therefore, if results of the recommended laboratory tests were available for review between 180 days before and 14 days after the time of the initial medication dispensing, the prescribing clinician was considered to have complied with the monitoring guideline. Results from the database interrogations were compared between physicians in the control group and those in the intervention group to determine if there was a difference between groups in rate of compliance with laboratory monitoring recommendations for the study medications.

Statistical Analysis

To be included in the analysis of drug dispensings, patients who received a study medication must have (1) been health plan members for at least 180 days before and 14 days after the drug dispensing; (2) received the drug dispensing between November 1, 2002, and October 31, 2003; (3) not received a dispensing of that drug within the previous 180 days (ie, the drug dispensing was the index dispensing); and (4) had the drug ordered by a physician in the control or intervention group of the study. The unit of analysis was the patient-drug therapy combination. To accommodate the defined time frame of drug dispensing and laboratory monitoring for the study, the administrative database was queried for laboratory tests completed May 1, 2002, through November 14, 2003. For each drug or drug class, it was determined whether the appropriate laboratory tests were completed in the defined time frame. An error in laboratory monitoring was defined as a dispensing that did not have all of the appropriate laboratory tests completed within 180 days before and 14 days after the drug was dispensed. Univariate and bivariate statistics were computed to describe the characteristics of members who received medications prescribed by clinicians in the control and intervention groups. The first observation per member in the database was used for the patient demographic characteristics, whereas all unique drug dispensings per member were used to describe the laboratory testing between the groups. We used the first observation per member to avoid the issue of making multiple observations within the same person. The intent was to avoid potentially biasing the results because of multiple observations (ie, within-person correlation). Therefore, the earliest unique member prescription was determined (within the study period). This was used as an index dispensing, and all laboratory value dates were compared with this occurrence. Nonparametric tests were used to statistically compare laboratory error rates by drug or drug class and by gender between the intervention and control groups.

RESULTS

P

One hundred three primary care physicians were randomized to the control arm of the study, while 104 were randomized to the intervention arm. The median age and age ranges of the patients were similar, as was the distribution between male and female patients between the 2 groups of physicians in the study. The physicians ordered 34 242 prescriptions for study medications for 26 586 different patients during the study period. The median (5th and 95th percentile) ages for the patients were 64 years (40 and 85 years) in the intervention group and 64 years (40 and 84 years) in the control group (= .90, χ² test). The mean + SD age at dispensing was 63.20 + 13.98 years (range, 18-89 years). Table 2 lists the characteristics of the patients prescribed study medications during the study period. Of these patients, 20 433 (76.9%) were prescribed only 1 study medication, 4903 (18.4%) were prescribed 2 study medications, and the remaining 1250 patients (4.7%) were prescribed 3 to 6 study medications (resulting in 34 242 patient-drug therapy combinations). For the 6153 patients in the study population who were prescribed more than 1 study medication, 83% of the medications were prescribed by the same provider. Among those prescribed the remaining 17% of medications, 813 patients had 2 study medications prescribed by alternate study arm providers. There was little prescribing by alternate study arm providers for patients prescribed more than 2 study medications, however.

P

There was no significant difference between the control and intervention group physicians in the overall rate of compliance with ordering the recommended laboratory monitoring for patients prescribed study medications, as determined by the appropriate laboratory result available in the EMR for review 180 days before to 14 days after the time of the medication order. Laboratory monitoring was performed within the recommended guidelines only 56.8% (19 451 of 34 242 index dispensings) of the time (= .31, χ² test). The respective values for the intervention and control groups were 56.6% (10 494/18 556) and 57.1% (8957/15 686) of the time.

P

P

The overall rates of compliance with laboratory monitoring guidelines for male and female patients who had medications ordered by control and intervention physicians were not statistically different. Male patients who had medication orders entered into the EMR by control physicians had a laboratory monitoring rate of 58.5%, and those with medication orders entered by intervention physicians had a rate of 57.5% (= .18). Female patients had monitoring rates of 55.7% and 55.9% (= .82) for the intervention and control arms of the study, respectively.

P

P

P

Laboratory monitoring rates for individual medications varied from 0.0% to 93.7% (Table 3). In cases in which a statistically significant difference was demonstrated, the improvement favored the patients of the physicians in the intervention group. The laboratory monitoring rates among patients prescribed medications by the intervention group compared with control group physicians were 71.2% vs 62.3% (= .003) for gemfibrozil, 42.9% vs 0.0% (= .03) for methotrexate, and 75.7% vs 73.9% (= .05) for statins.

DISCUSSION

Electronic medical record systems are heralded as a means to reduce medication errors. Hospitals and clinics are under mounting pressure to replace paper-based drug ordering systems with computers, eliminating the confusion caused by physicians' handwriting and greatly improving the ability to track patients' medication histories. Many EMRs have software that automatically warns physicians if what they are about to prescribe might cause an allergic reaction or interact with other medications. The Leapfrog Group, a powerful coalition of 170 companies and organizations that buy healthcare, has made the criteria of computerized drug ordering one of its main measures of hospital quality.²⁴ However, only about 10% of US hospitals and fewer than 12% of physician practices have fully installed computerized drug ordering.²⁵ In addition, a recent report indicates that e-prescribing and computerized physician order entry introduce new types of errors, including the following: fragmented computerized order entry displays that prevent a coherent view of patients' medications, pharmacy inventory displays that are mistaken for dosage guidelines, ignored antibiotic renewal notices that are placed on paper medical charts rather than in the CPOE system, separation of functions that facilitate double dosing and incompatible orders, and inflexible ordering formats that result in generating wrong orders.²⁶ Too many alerts and reminders, reminders presented at inappropriate times, and poor context or formatting of alerts may deter their use by providers.²²

In this article, we evaluate the use of nonintrusive reminders within a CPOE system. Guidelines for performing baseline laboratory monitoring for selected medications were presented to ordering providers as a text-based message, as part of the pharmacy instructions on the computerized medication order entry screen. Physicians in the intervention arm of the study received training showing them where to look for this message and how to order the required laboratory tests as part of the ordering process. Physicians in the control arm of the study did not receive additional training and did not have laboratory monitoring instructions displayed in the pharmacy instruction field of the computerized order entry screen.

The control and intervention physicians prescribed similar numbers of medications for their patients. Although the intervention physicians received the nonintrusive alerts and the academic detailing to reinforce the presence of the alerts and the method of using them, the control and intervention physicians performed laboratory monitoring at the same rates for patients who had the specific medications ordered for them during the study period. When the monitoring rates for individual drugs were compared, only the monitoring rates for gemfibrozil, methotrexate, and statins were statistically better for the patients who had medications ordered by physicians in the intervention group. The nonintrusive alerts seemed to only selectively work for gemfibrozil and statin medications. The number of orders for methotrexate was small; only 16 prescriptions for methotrexate were ordered during the study period. Intervention physicians ordered 7 prescriptions, and control physicians ordered 9 prescriptions. To see a laboratory monitoring rate difference of 40% (+40%) with a significance (α level) of .05 and a power (β error) of 80%, we would have needed to observe at least 38 prescriptions for methotrexate. Therefore, there was insufficient power to make a conclusion about the effectiveness of the alert for methotrexate orders.

P

For most of the medications, no statistical differences existed between the intervention and control groups of physicians for the monitoring rates of patients receiving any of the other individual medications analyzed. In light of the overall lack of effect seen for nonintrusive alerts on the monitoring rates for most medications, the positive effects seen for gemfibrozil and statin medications may be explained by chance. By analyzing the monitoring rates for 25 medications, one would expect to see an effect in 1.25 to 2.5 medications by chance alone at = .05. Another explanation is that during the study period several parallel efforts targeted the lowering of cholesterol among health plan members that included medication prescribing and increased cholesterol monitoring. The additional focus on cholesterol initiatives may have helped reinforce the nonintrusive alert messages seen by the intervention clinicians.

A possible reason why this intervention failed to alter physician practice is that the nonintrusive messages providing guidelines for performing baseline laboratory monitoring may not be sufficiently intense to change physicians' habits at the point of ordering medications. The academic detailing may have failed to train the intervention physicians well enough to change their behavior. However, the intervention physicians received 1-on-1 training, the use of alert messages was clarified, and each intervention physician demonstrated his or her ability to find the messages and to understand their use on the order entry screen. Another explanation for the lack of behavioral differences between the 2 clinician groups is that the physicians in the control group may have received laboratory monitoring "reminders" in other ways (eg, from ongoing monthly educational seminars, from the clinical pharmacists assigned to each clinic, or from other resources available in the office or in the EMR).

This study reinforces results from other studies^27,28 indicating that nonintrusive reminders may not improve adherence to guideline recommendations. To improve clinical practice and to reduce errors with the expectation of improving patient outcomes, a balance between generating too many alerts and reminders must be paired with delivering the message by the most productive method. As computerized order entry becomes increasingly prevalent, more research is needed to find this correct balance. These findings then should be communicated to the developers and users of computerized medical record systems.

Acknowledgments

We acknowledge, with our thanks, the programming expertise of Michael A. Bodily, MS, and Thomas J. Koehler, RPh. We also thank Julia A. Kelleher, PharmD, for her expertise related to laboratory monitoring of medications.

From the Clinical Research Unit, Colorado Permanente Medical Group (TEP, DMM) and Kaiser Foundation Health Plan (MR, EL); and the School of Pharmacy (MR) and School of Medicine (DMM), University of Colorado at Denver and Health Sciences Center, Denver.

This study was supported by cooperative agreement U18 HS11843 from the Agency for Healthcare Research and Quality (AHRQ). The AHRQ did not participate in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; or the preparation, review, or approval of the manuscript.

Address correspondence to: Ted E. Palen, PhD, MD, MSPH, Clinical Research Unit, Colorado Permanente Medical Group, PO Box 378066, Denver, CO 80237-8066. E-mail: ted.e.palen@kp.org.

Download PDF: Evaluation of Laboratory Monitoring Alerts Within a Computerized Physician Order Entry System for Medication Orders