Publication
Article
The American Journal of Managed Care
Author(s):
A targeted computerized alert at the time of physician order entry reduced the use of D-dimer testing among patients 65 years and older.
Objective:
To assess the effect of a targeted age-specific computerized alert to reduce D-dimer testing in elderly patients.
Study Design:
A single-crossover cluster randomized trial of computerized alerts during physician order entry involving 8 ambulatory care clinics in a group-model integrated care delivery system.
Methods:
The rate of completed D-dimer tests per 1000 patient visits, ratio of completed venous ultrasonography to completed D-dimer tests, and rate of completed venous ultrasonography per 1000 patient visits.
Results:
The rate of completed D-dimer tests per 1000 visits among patients 65 years and older in intervention clinics decreased from 5.02 to 1.52 (95% confidence interval [CI], -4.20 to -2.80;P <.001), which persisted throughout the study period. The rate of completed D-dimer tests per 1000 visits among patients 65 years and older in control clinics decreased from 3.14 to 2.11 (95% CI, -1.66 to -0.04; P <.001 for interaction). After activation of the alert in control clinics, the rate of completed D-dimer tests per 1000 visits among patients 65 years and older decreased from 2.11 to 0.81 (95% Cl, -1.79 to -0.80; P <.001). After activation of the alert in each clinic group, the ratios of completed venous ultrasonography to completed D-dimer tests increased from 1.17 to 4.05 (95% CI, 2.52-3.22) and from 2.25 to 7.29 (95% CI, 3.74-6.35) in intervention clinics and control clinics, respectively (P <.001 for both).
Conclusion:
An electronic age-specific alert targeted to a specific condition reduced D-dimer testing in this elderly population of outpatients and demonstrated a persistent effect.
(Am J Manag Care. 2010;16(11):e267-e275)
Physicians often find it difficult to remember to follow evidence-based clinical practice guidelines. Many researchers have suggested that computer-generated alerts within electronic medical records may serve as reminders to improve adherence to best practices. However, too many alerts produce alert fatigue and may lead physicians to ignore them. Our study shows that an alert targeted to a specific order and for specific patients can alter a physician's ordering behavior and promote improved adherence to a clinical practice guideline.
The risk of developing a blood clot in the venous circulation increases exponentially with age from approximately 30 cases per 100,000 persons aged 25 to 35 years to 300 to 500 cases per 100,000 persons aged 70 to 79 years.1 Rates of pulmonary embolism (PE) increase from 120 per 100,000 persons aged 65 to 69 years to approximately 700 per 100,000 persons 85 years and older; about half of all deaths after venous thromboembolism (VTE) are attributed to PE.2,3 Venous thromboembolism most commonly includes deep vein thrombosis (DVT) or PE. Symptoms common to PE and DVT appear in various clinical conditions. Clinical diagnosis of suspected acute VTE is difficult. Therefore, objective testing combined with a clinical risk algorithm is recommended.4-7
Since the late 1980s, high-resolution venous ultrasonography has become widely used to diagnose or exclude DVT in most community practices throughout the United States.7-9 In the early 1980s, a quantitative D-dimer test based on enzyme-linked immunosorbent assay (ELISA) was introduced as a means to exclude acute VTE. Investigators have studied the role of D-dimer in the diagnosis of DVT and PE.10-12 Controlled studies13-17 using a sensitive automated ELISA D-dimer assay have shown that D-dimer levels below an established cutoff of 500 ng/mL combined with a clinical risk algorithm enable exclusion of acute venous thrombosis in up to 30% of outpatients (to convert D-dimer level to nanomoles per liter, multiply by 5.476). However, the negative predictive value of the test worsens as patient age increases.18-21 Although the sensitivity and the negative predictive value of the test are about 98%, the specificity is less than 30%, and the overall accuracy of the test is only 35% for patients 65 years and older.22-24 Previously, our group demonstrated that more of these tests are ordered in clinical practices where D-dimer testing is readily available.24 In real-world experience, it was found that more than 73% of all D-dimer tests among patients 65 years and older had positive results, and almost 65% of these were false positives that required additional radiologic evaluation. Only about 27% of all D-dimer tests performed among the elderly had negative results; therefore, only these patients avoided additional radiologic procedures. In comparison, among patients younger than 65 years, the test had a negative predictive value exceeding 99%, a specificity exceeding 70%, and an overall accuracy exceeding 70%.
In 2007, the American Academy of Family Physicians and the American College of Physicians25 jointly published a guideline onthe diagnosis of lower extremity DVT and PE, which recommended the use of clinical prediction rules to establish pretest probability of disease and advised using D-dimer testing to exclude VTE in younger patients without associated comorbidity or history of VTE. However, the guideline notes that D-dimer testing may be insufficient to rule out VTE in older patients.26 Other researchers also present evidence supporting that older patients should not receive D-dimer testing and recommend that an alternative strategy based on noninvasive examinations such as venous ultrasonography should be the first choice for evaluation of possible DVT.9,26-28
In 2001, Kaiser Permanente of Colorado adopted the quantitative D-dimer test for use in clinical practice. Since then, the number of D-dimer tests ordered for patients 65 years and older has increased 5-fold, accounting for about one-third of all D-dimer tests ordered. The convenience of ordering a laboratory test and of potentially avoiding a trip by the patient to a diagnostic radiology center may be partially fueling the increased use of the D-dimer test. However, because of the quantity of tests being ordered in patients 65 years and older and owing to poor test performance in this age group, we have also seen a significant increase in the number of positive D-dimer test results, which required additional testing using radiologic evaluation. Most positive D-dimer test results in the elderly (65%) were determined to be false positives on further evaluation of patients’ venous ultrasonography results.24 Therefore, we wondered whether a computerized alert within the physician order entry module of the electronic medical record (EMR) that we use (Kaiser Permanente HealthConnect [KPHC]; Epic Systems, Verona, Wisconsin) could reduce the number of D-dimer tests ordered for elderly patients. We hypothesized that computergenerated reminders given to primary care providers at thetime of ordering a D-dimer test for patients 65 years and older for suspected VTE would decrease the rate of D-dimer test ordering among providers receiving the alert compared with providers not receiving the alert. This article reports our results of a randomized controlled study using this alert within a group-model health maintenance organization using an EMR for documentation and order entry of all outpatient clinical care.
METHODS
Between September 2005 and October 2007, we performed an internally funded cluster randomized trial involving 8 ambulatory care clinics in a group-model integrated care delivery system in Denver, Colorado. The institutional review board of Kaiser Permanente of Colorado approved the study before the start. This research did not require contact with study subjects (physicians or patients). Data used were retrospective and were extracted in a deidentified format before analysis began. This trial began before July 1, 2008, and did not require trial registration to meet guidelines of the International Committee of Medical Journal Editors.
Participant Selection and Randomization
We identified 8 primary care clinics, each with at least 3000 patients 65 years and older. We performed a simple cluster randomization to assign half of the clinics to a control group and the other half to an intervention group. A cluster randomized design was selected to decrease the risk of interprovider contamination among providers within the same facility. Primary care providers within the designated clinics were unaware that a research study was occurring in regard to D-dimer test ordering related to the alerts.
Intervention
All primary care providers received training and information about the use of clinical risk algorithms for patients with suspected venous thrombosis and about the performance characteristics of the D-dimer test among various age groups.23,24 This training took place during one of the monthly continuing medical education meetings for primary care providers 4 months before the intervention. Primary care providers in intervention clinics also saw the following pop-up alert within the physician order entry system at the time of placing a computerized D-dimer test order for patients 65 years and older, advising against ordering the D-dimer test: “D-dimer testing is not recommended for patients 65 years
Figure 1
of age and over because the accuracy is only 35% in this age group. Instead, order diagnostic imaging as appropriate.” Providers in control clinics did not receive this alert during the initial intervention. After 20 months, the alert was activated in control clinics; it remained active in intervention clinics.The specific metrics for each step of the project are shown in .
Data on D-dimer tests and venous ultrasonography among patients having office visits in intervention clinics and control clinics were collected before and after activation of the alert. Data included clinic site, date of D-dimer order and result, and date of radiology test (Current Procedural Terminology code 93971 [ultrasonography of extremities]) and result after a D-dimer test. Except for patient age, other patient characteristics were unavailable in the data sets used for analyses.
Outcome Measurements
The primary outcome measure was the number of D-dimer tests ordered and completed per 1000 patient visits (including all office visits for all patients in each age group for each clinic). This rate excluded D-dimer tests that occurred within 14 days of each other for the same patient. The secondary outcome measure, the effect on ultrasonography ordering for evaluation of DVT, was assessed by determining the ratio of completed venous ultrasonography to completed D-dimer tests and the rate of completed venous ultrasonography per 1000 patient visits for each age group.
Statistical Analysis
Outcome variables of interest were counts (eg, D-dimer tests estimated relative to a number of visits to account for differing ordering opportunities). Descriptive rates and Fisher exact test confidence intervals (CIs) were calculated for intervention groups and control groups for each period. Count data are often modeled using Poisson distribution regression analysis, but negative binomial models were a better fit because of the overdispersion evident among these data.29 Models were run using commercially available software (GLIMMIX program in SAS; SAS Institute Inc, Cary, North Carolina), with the numerator of the rate as the outcome and a logged value for the denominator used as an offset term. Analysts were unaware of the clinic assignments during the data collection steps, but were aware of the clinic status during the statistical analysis phase. Each clinic had 2 periods of interest (preintervention and postintervention), and the test of intervention effectiveness was evaluated using the interaction term of the intervention variable with time. This test evaluates if the change in the outcome variable over time was significantly different for intervention clinic versus control clinic groups. Estimates of clinic-level intraclass correlations were low (<0.01), and the addition of a random-effects term for facility clusters in the negative binomial models produced comparable results. The minimal influence on model results suggests limited clustering effects within the specific clinics.
RESULTS
Table 1
Figure 2
The mean age of patients 65 years and older receiving D-dimer testing was 75.6 years in intervention clinics and 75.7 years in control clinics. The mean age of patients younger than 65 years receiving D-dimer testing was 48.7 years in intervention clinics and 49.1 years in control clinics. Among patients 65 years and older, the baseline rates during 8 months before the study for completed D-dimer tests per 1000 clinic visits was 5.02 in intervention clinics and 3.14 in control clinics (rate difference, 1.88; 95% CI, 1.03-2.74; P <.001). After the alert was activated in intervention clinics, the rate of completed D-dimer tests per 1000 clinic visits for patients 65 years and older (September 2005 through April 2007) dropped significantly in all clinics, but the rate dropped significantly more in intervention clinics than in control clinics (from 5.02 to 1.52; 95% CI, −4.20 to −2.80; P <.001 in intervention clinics and from 3.14 to 2.11; 95% CI, −1.66 to −0.40; P <.001 in control clinics) ( and ). The rate differences between intervention clinics and control clinics were 1.52 and 2.11, respectively (rate difference, −0.59; 95% CI, −0.98 to −0.19; P = .003) (Table 1).
Activation of the alert in control clinics was followed by a significant decrease in the rate of completed D-dimer tests per 1000 clinic visits for patients 65 years and older in these clinics. The rate decreased from 2.11 to 0.81 (95% CI, −1.79 to −0.80; P < .001). During the period when the alert was active in all 8 clinics (May through September 2007), the rate of completed D-dimer tests per 1000 clinic visits for patients 65 years and older was 1.89 in intervention clinics and 0.81 in control clinics (rate difference, 1.08; 95% CI, 0.42-1.74;
P = .002). After the initial rate change in intervention clinics, the new lower rate was maintained, indicating persistent effectiveness of the alert over time (Table 1 and Figure 2).
The baseline rates of completed D-dimer tests per 1000 clinic visits for patients younger than 65 years were not significantly different between intervention clinics and control clinics. The rate of completed D-dimer tests per 1000 clinic visits in control clinics for patients younger than 65 years did not significantly change over the course of the study; the rate for these patients in intervention clinics did not initially change after the alert was established but showed a slight increase by the end of the study period (Table 1). As the alert was not active for patients in this age group, these rates indicate that physician decision making for these patients was unaffected.
Figure 3
Because the alert advised against ordering a D-dimer test and recommended using a radiologic test (ie, venous ultrasonography) to evaluate patients 65 years and older for suspected DVT, we expected to see an increase in the rate of venous ultrasonography. We evaluated the ratio of completed venous ultrasonography to completed D-dimer tests during the study period (). The baseline ratios of completed venous ultrasonography to completed D-dimer tests for patients 65 years and older were 1.17 in intervention clinics and 1.55 in control clinics (ratio difference, −0.38; 95% CI, −0.63 to −0.11; P = .004). After alert activation, significant increases occurred in the ratios of completed venous ultrasonography to completed D-dimer tests for patients 65 years and older. Although the ratios increased in both clinic groups, the increase in intervention clinics was much greater. During the period when the alert was activated in intervention clinics and control clinics, the ratios of completed venous ultrasonography to completed D-dimer tests for patients 65 years and older were 3.85 in intervention clinics and 7.29 in control clinics (ratio difference, −3.44; 95% CI, −4.84 to −2.04; P <.001). We saw a persistent ratio in intervention clinics when the alert was activated in both clinic groups (from 4.05 to 3.85; 95% CI, −0.83 to 0.45; P = 0.57). The ratio in control clinics when the alert was active showed a significant increase (from 2.25 to 7.29; 95% CI, 3.74-6.35; P <.001), similar to our observation when the alert was initially activated in intervention clinics. The increase in venous ultrasonography per D-dimer tests may have been driven by the large decrease in D-dimer tests in the intervention group (the denominator in this comparison). However, the increase in the ratio of completed venous ultrasonography to completed D-dimer tests was more than can be accounted for by just the decrease in D-dimer tests.
Figure 4
The ratio of completed venous ultrasonography to completed D-dimer tests for patients younger than 65 years showed a slight increase after the alert was activated in intervention clinics, but decreased over time. A slight increase in the ratio was also observed after initiation of the alert in control clinics. No persistent trends were observed in intervention clinics or control clinics among either age group of patients for the rate of completed ultrasonography per 1000 patient visits (). The overall few patients being evaluated for VTE among all patients having clinic visits may account for why we did not see a consistent change in the rate of venous ultrasonography per 1000 clinic visits.
Table 2
The rate comparisons in Table 1 provide useful information but do not directly test whether changes at intervention clinics were significantly different from those in control clinics, considering the difference in baseline rates between the clinics. To more rigorously answer this question, we used negative binomial models to test interactions of the intervention variable with time (). These models focused on changes from baseline until alerts were operating in intervention clinics for patients 65 years and older. The use of D-dimer tests for patients 65 years and older decreased in intervention clinics and control clinics during periods when the alert was active in intervention clinics; however, the decrease was significantly lower for intervention clinics than for control clinics (P <.001). The results did not change after taking into account potential clustering of effects within facilities. The ratio of completed venous ultrasonography to completed D-dimer tests increased in intervention clinics and control clinics during the intervention period, but the increase was significantly higher for intervention clinics (P = .006). The rate of completed D-dimer tests per 1000 visits among patients younger than 65 years did not change in intervention clinics or control clinics during the intervention period. However, for patients younger than 65 years, there was a slight increase in completed venous ultrasonography (Table 1, Figure 3, and Figure 4).
DISCUSSION
In theory, alerts within an EMR may help decrease prescribing errors, increase rates of appropriate preventive screening and prevention interventions, and improve the delivery of evidence-based medical practice.30-34 Computerized alerts have also been effective in improving physician prescribing practices.35,36 However, the use of alerts in practice has not always proven to work as promised. Clinicians can experience alert fatigue, in which receipt of too many alerts becomes frustrating, leading to ignoring
or overriding the messages.30,37,38
Our results show that a targeted alert designed to foster evidence-based guidance (as recommended by the American Academy of Family Physicians and the American College of Physicians) can reduce the use of D-dimer testing among patients 65 years and older.25,26 One reason that our alert mayhave been effective is that it provided alternative diagnostic
suggestions, rather than just discouraging the use of the D-dimer test. In addition, the effectiveness of the alert persisted over time, suggesting that patient-specific alerts presented at an appropriate time at the point of care may foster improved adherence to evidence-based guidelines.
Alert messages generated in an EMR guiding the use of D-dimer testing may not be generalizable to other tests for which alternative testing is not readily available. In addition, the utilization of alerts within an EMR used for documenting care within an integrated healthcare system may not be generalizable to nonintegrated health systems in which additional testing (ie, radiologic services) may not be readily available within the system. We were also limited in the data available from the EMR system database. We did not have access to the reasons why clinicians ordered imaging procedures. This precluded inclusion of chest computed tomography as an outcome measure, as this is used for many reasons other than evaluation of PE. However, lower extremity ultrasonography is almost exclusively ordered for the evaluation of DVT.
Despite randomization, the baseline utilization of D-dimer testing differed between intervention clinics and control clinics, so there were likely unmeasured differences between clinics in terms of practice patterns, patient data, or clinician characteristics. We attempted to adjust for these differences with intervention variable-×-time analyses, but recognize that this might not fully adjust for unmeasured differences between study groups. We chose a cluster randomization model to limit the potential of interprovider contamination due to communication between providers in the same medical facility regarding the use of D-dimer testing. We wanted to limit the possibility of a scenario in which one provider gets the alert, another in close proximity does not get the alert, and they would talk about the alert message between themselves. Future studies may be designed to randomize at the provider level to confirm these results. In addition, Kaiser Permanente of Colorado has had a consistently stable Medicare population. We would not expect differences in the characteristics of this patient population to manifest during the course of this short study. We observed a trend toward decreased D-dimer testing for patients 65 years and older in all clinics. This observation may be because of continuing education materials and programs and other communications to providers informing them about poor performance of the D-dimer test among patients in this age group. We also noted an increase in the rate of D-dimer testing for patients younger than 65 years. This observation may have been an unintended consequence of the alert itself. That is, did the message that D-dimer testing may not be appropriate in the elderly lead clinicians to conclude that D-dimer is appropriate in the nonelderly (although the alert did not appear when ordering a D-dimer test for patients in the younger age group). This increase in D-dimer testing among the younger group may have been appropriate or inappropriate, but seems to have occurred in both clinic groups after the alert was in widespread use. This question remains for future studies and analyses.
Further studies are needed to test the best use and structure of computerized alerts. We suggest that alerts should be focused on high-cost, high-variation, and high-frequency conditions to minimize the number of clinician workflow interruptions. Our study was not designed to compare alerts with and without information about alternative diagnostic and treatment strategies, although we believe that alerts should contain this information to direct clinicians toward evidence-based alternatives rather than simply provide “negative guidance.” As EMR technology improves, it might be possible in the future to tailor alert messages to clinicians based on their practice patterns, so that each clinician receives alerts only in areas where their individual practice could benefit. Systems might also be able to remove alerts for individual clinicians who have demonstrated persistent improvement in ordering practices. These approaches should mitigate the occurrence of alert overload and minimize the possibility that clinicians disregard alert messages.
Acknowledgments
We thank Dr Donna Lynn, President of the Kaiser Foundation Health Plan of Colorado, and Dr Bill Marsh, Associate Medical Director, Colorado Permanente Medical Group, for stimulating interest in this project and for their support.
We thank Megan H. Hawthorn for her administrative assistance and support with implementing the alert message within the electronic medical record. We also thank Marian Bailey for her help with data acquisition.
Author Affiliations: From the Institute for Health Research (TEP, DWP, AJS, SMS), Kaiser Permanente of Colorado, Denver, CO.
Funding Source: The Kaiser Foundation Health Plan of Colorado provided funding to conduct this study. The funds covered the staff time to conduct the research. Study investigators acted independently, and the Kaiser Foundation Health Plan of Colorado had no role in the study beyond approving the staff time it took to conduct it.
Author Disclosures: The authors (TEP, DWP, AJS, SMS) are employed by the Colorado Permanente Medical Group, which is a for-profit entity. None of the authors report a financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Previous Publication: An abstract of this study was presented at the 14th Annual HMO Research Network Conference; April 15, 2008; Minneapolis, MN.
Authorship Information: Concept and design (TEP, DWP, AJS); acquisition of data (TEP, AJS); analysis and interpretation of data (TEP, DWP, AJS, SMS); drafting of the manuscript (TEP, DWP); critical revision of the manuscript for important intellectual content (TEP, DWP, SMS); statistical analysis (TEP, SMS); obtaining funding (TEP); administrative, technical, or logistic support (TEP); and supervision (TEP).
ted.e.palen@kp.org.
Address correspondence to: Ted E. Palen, PhD, MD, MPSH, Institute for Health Research, Kaiser Permanente of Colorado, PO Box 378066, Denver, CO 80237-8066. E-mail:
1. Silverstein MD, Heit JA, Mohr DN, Petterson TM, O'Fallon WM, Melton LJ III. Trends in the incidence of deep vein thrombosis and pulmonary embolism: a 25-year population-based study. Arch Intern Med. 1998;158(6):585-593.
2. Cushman M. Epidemiology and risk factors for venous thrombosis. Semin Hematol. 2007;44(2):62-69.
3. Hirsh J, Hull RD, Raskob GE. Clinical features and diagnosis of venous thrombosis. J Am Coll Cardiol. 1986;8(6)(suppl B):114B-127B.
4. Goodacre S, Sutton AJ, Sampson FC. Meta-analysis: the value of clinical assessment in the diagnosis of deep venous thrombosis. Ann Intern Med. 2005;143(2):129-139.
5. Roy PM, Meyer G, Vielle B, et al; EMDEPU Study Group. Appropriateness of diagnostic management and outcomes of suspected pulmonary embolism. Ann Intern Med. 2006;144(3):157-164.
6. Bosson JL, Pernod G, Joubin E, et al. Non-conform diagnostic management of pulmonary embolism suspected patients is responsible for a higher risk of thrombotic event occurrence. J Mal Vasc. 2007; 32(1):15-22.
7. Qaseem A, Snow V, Barry P, et al; Joint American Academy of Family Physicians/American College of Physicians Panel on Deep Venous Thrombosis/Pulmonary Embolism. Current diagnosis of venous thromboembolism in primary care: a clinical practice guideline from the American Academy of Family Physicians and the American College of Physicians [republished from Ann Fam Med. 2007;5(1):57-62]. Ann Intern Med. 2007;146(6):454-458.
8. Lensing AW, Prandoni P, Brandjes D, et al. Detection of deep-vein thrombosis by real-time B-mode ultrasonography. N Engl J Med. 1989;320(6):342-345.
9. Righini M, Le Gal G, Perrier A, Bounameaux H. The challenge of diagnosing pulmonary embolism in elderly patients: influence of age on commonly used diagnostic tests and strategies. J Am Geriatr Soc. 2005;53(6):1039-1045.
10. Janssen MC, Wollersheim H, Verbruggen B, Novakova IR. Rapid D-dimer assays to exclude deep venous thrombosis and pulmonary embolism: current status and new developments. Semin Thromb Hemost. 1998;24(4):393-400.
11. Legnani C, Pancani C, Palareti G, et al. Comparison of new rapid methods for D-dimer measurement to exclude deep vein thrombosis in symptomatic outpatients. Blood Coagul Fibrinolysis. 1997;8(5):296-302.
12. Freyburger G, Trillaud H, Labrouche S, et al. D-dimer strategy in thrombosis exclusion: a gold standard study in 100 patients suspected of deep venous thrombosis or pulmonary embolism: 8 DD methods compared. Thromb Haemost. 1998;79(1):32-37.
13. Perrier A, Desmarais S, Miron MJ, et al. Non-invasive diagnosis of venous thromboembolism in outpatients. Lancet. 1999;353(9148): 190-195.
14. Prisco D, Grifoni E. The role of D-dimer testing in patients with suspected venous thromboembolism. Semin Thromb Hemost. 2009;35(1):50-59.
15. Janssen MC, Wollersheim H, Verbruggen B, Novakova IR. Rapid D-dimer assays to exclude deep venous thrombosis and pulmonary embolism: current status and new developments. Semin Thromb Hemost. 1998;24(4):393-400.
16. Legnani C, Pancani C, Palareti G, Guazzaloca G, Fortunate G, Grauso F, et al. Comparison of new rapid methods for D-dimer measurement to exclude deep vein thrombosis in symptomatic outpatients. Blood Coagul Fibrinolysis. 1997;8(5):296-302.
17. Freyburger G, Trilluad H, Labrouche S, Gauthier P, Javorschi S, Bernard P, et al. D-dimer strategy in thrombosis exclusion - a gold standard study in 100 patients suspected of deep venous thrombosis or pulmonary embolism: 8 DD methods compared. Thromb Haemost. 1998;79(1):32-37.
18. Masotti L, Ceccarelli E, Cappelli R, Forconi S. Plasma D-dimer levels in elderly patients with suspected pulmonary embolism. Thromb Res. 2000;98(6):577-579.
19. Righini M, de Moerloose P, Reber G, Perrier A, Bounameaux H. Should the D-dimer cut-off value be increased in elderly patients suspected of pulmonary embolism [letter]. Thromb Haemost. 2001;85(4):744.
20. Righini M, Goehring C, Bounameaux H, Perrier A. Effects of age on the performance of common diagnostic tests for pulmonary embolism. Am J Med. 2000;109(5):357-361.
21. Tardy B, Tardy-Poncet B, Viallon A, et al. Evaluation of D-dimer ELISA test in elderly patients with suspected pulmonary embolism. Thromb Haemost. 1998;79(1):38-41.
22. Pieper CF, Rao KM, Currie MS, Harris TB, Chen HJ. Age, functional status, and racial differences in plasma D-dimer levels in community-dwelling elderly persons. J Gerontol A Biol Med Sci. 2000;55(11): M649-M657.
23. Wells PS, Anderson DR, Bormanis J, et al. Value of assessment of pretest probability of deep-vein thrombosis in clinical management. Lancet. 1997;350(9094):1795-1798.
24. Palen TE, Adcock DM. Performance characteristics of three quantitative D-dimer assays for outpatient evaluation of venous thromboembolism and its use in a clinical guideline for a group model HMO. J Clin Ligand Assay. 2005;28(3):123-129.
25. Qaseem A, Snow V, Barry P, et al; Joint American Academy of Family Physicians/American College of Physicians Panel on Deep Venous Thrombosis/Pulmonary Embolism. Current diagnosis of venous thromboembolism in primary care: a clinical practice guideline from the American Academy of Family Physicians and the American College of Physicians. Ann Fam Med. 2007;5(1):57-62.
26. Stein PD, Hull RD, Patel KC, et al. D-dimer for the exclusion of acute venous thrombosis and pulmonary embolism: a systematic review. Ann Intern Med. 2004;140(8):589-602.
27. Antonelli F, Villani L, Masotti L, Landini G. Ruling out the diagnosis of venous thromboembolism in the elderly: is it time to revise the role of D-dimer? Am J Emerg Med. 2007;25(6):727-728.
28. Schutgens RE, Haas FJ, Biesma DH. Reduced efficacy of clinical probability score and D-dimer assay in elderly subjects suspected of having deep vein thrombosis. Br J Haematol. 2005;129(5):653-657.
29. Gardner W, Mulvey EP, Shaw EC. Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychol Bull. 1995;118(3):392-404.
30. Feldbaum J. Doing alerts right: implemented correctly, alerts can greatly enhance a CPOE system, but done wrong, they can send physicians back to pens and paper. Healthc Inform. 2008;25(2):95-96.
31. Lapane KL, Waring ME, Schneider KL, Dube C, Quilliam BJ. A mixed method study of the merits of e-prescribing drug alerts in primary care. J Gen Intern Med. 2008;23(4):442-446.
32. de Lusignan S, Teasdale S. Achieving benefit for patients in primary care informatics: the report of a international consensus workshop at Medinfo 2007. Inform Prim Care. 2007;15(4):255-261.
33. Lerma Gaude V, Poveda Andres JL, Font Noguera I, Planells Herrero C. Alerts system associated with computerized physician order entry: analysis and identification of improvement points [in Spanish]. Farm Hosp. 2007;31(5):276-282.
34. Solberg LI, Wei F, Butler JC, Palattao KJ, Vinz CA, Marshall MA. Effects of electronic decision support on high-tech diagnostic imaging orders and patients. Am J Manag Care. 2010;16(2):102-106.
35. Feldstein A, Simon SR, Schneider J, et al. How to design computerized alerts to safe prescribing practices. Jt Comm J Qual Saf. 2004;30(11):602-613.
36. Galanter WL, Polikaitis A, DiDomenico RJ. A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry. J Am Med Inform Assoc. 2004;11(4):270-277.
37. van Wyk JT, van Wijk MA, Sturkenboom MC, Mosseveld M, Moorman PW, van der Lie J. Electronic alerts versus on-demand decision support to improve dyslipidemia treatment: a cluster randomized controlled trial. Circulation. 2008;117(3):371-378.
38. Khajouei R, Jaspers MW. CPOE system design aspects and their qualitative effect on usability. Stud Health Technol Inform. 2008;136:309-314.