Effects of Documentation-Based Decision Support on Chronic Disease Management

Objective:

To evaluate whether a new documentation-based clinical decision support system (CDSS) is effective in addressing deficiencies in the care of patients with coronary artery disease (CAD) and diabetes mellitus (DM).

Study Design:

Controlled trial randomized by physician.

Methods:

We assigned primary care physicians (PCPs) in 10 ambulatory practices to usual care or the CAD/DM Smart Form for 9 months. The primary outcome was the proportion of deficiencies in care that were addressed within 30 days after a patient visit.

Results:

The Smart Form was used for 5.6% of eligible patients. In the intention-to-treat analysis, patients of intervention PCPs had a greater proportion of deficiencies addressed within 30 days of a visit compared with controls (11.4% vs 10.1%, adjusted and clustered odds ratio =1.14; 95% confidence interval, 1.02-1.28; P = .02). Differences were more pronounced in the “on-treatment” analysis: 17.0% of deficiencies were addressed after visits in which the Smart Form was used compared with 10.6% of deficiencies after visits in which it was not used (P <.001). Measures that improved included documentation of smoking status and prescription of antiplatelet agents when appropriate.

Conclusions:

Overall use of the CAD/DM Smart Form was low, and improvements in management were modest. When used, documentation-based decision support shows promise, and future studies should focus on refining such tools, integrating them into current electronic health record platforms, and promoting their use, perhaps through organizational changes to primary care practices.

(Am J Manag Care. 2010;16(12 Spec No.):SP72-SP81)

• A novel note-based decision support system built into an electronic health record was associated with an increase in the proportion of deficiencies in the management of heart disease and diabetes addressed within 30 days of a visit in which it was used.

• However, because it was used in only 5.6% of eligible patients, its overall effect on care was small (absolute increase of 1.3% of deficiencies addressed).

• Improvements in chronic disease management likely require financial incentives to improve care, multifaceted quality improvement efforts, distribution of work to a patient care team, and health information tools that support these activities.

Clinical decision support systems (CDSSs) integrated within electronic health records (EHRs) hold the promise of improving healthcare quality, but to date the effectiveness of CDSSs and EHRs has been less than expected, especially with respect to the ambulatory management of chronic diseases.^1,2 In part this is because clinicians do not use CDSSs fully, if at all. In fact, several studies of CDSSs show low rates of use among clinicians.^3-7 Barriers to clinicians’ use of CDSSs have included lack of integration into work flow, software usability issues, and relevance of the content to the patient at hand.⁸ At Partners HealthCare, we developed Smart Forms to facilitate documentation-based clinical decision support. Rather than being interruptive in nature, the Smart Form enables writing a multiproblem visit note while capturing coded information and providing sophisticated decision support in the form of tailored recommendations for care. The most recent version of the Smart Form was designed around 2 chronic diseases: coronary artery disease (CAD) and diabetes mellitus (DM).⁹

In a previous study, we conducted pilot testing with 30 clinicians during a 6- to 8-week period.10 When deficiencies in CAD/DM management were present, they were more often addressed in the month following visits in which the Smart Form was used compared with preintervention visits. Specific deficiencies that were more often addressed included documentation of blood pressure, smoking status, height and weight, and prescription of beta-blockers.

To more rigorously assess the effect of the intervention in a broader patient population, we conducted a randomized controlled trial (RCT) of the CAD/DM Smart Form and evaluated its effects on chronic disease management.

METHODS

Overview

We conducted a controlled trial, randomized by provider, in primary care clinics associated with Partners HealthCare System. The Partners Human Research Committee approved

the study.

Setting and EHR

Partners HealthCare System is an integrated regional healthcare delivery network in eastern Massachusetts. Partners includes more than 20 primary care clinics affiliated with Brigham and Women’s Hospital or Massachusetts General Hospital, 5 acute care hospitals, several specialty and rehabilitation hospitals, and other affiliated ambulatory practices. The main EHR used in Partners ambulatory clinics is the Longitudinal Medical Record (LMR), a proprietary, Certification Commission for Healthcare Information Technology—certified EHR.¹¹

CAD/DM Smart Form

Figure

The CAD/DM Smart Form has been described previously⁹ (see ). The goal of Smart Forms was to integrate clinical data display, clinical decision support, ordering, and documentation. The Smart Form is a documentation tool, and as such, has many features in common with other latest- generation EHRs, including the ability to add, edit, and delete coded and/or structured clinical information such as medical problems, medications, and vital signs, and to easily import that information into a visit note. Like some systems that use disease-specific templates,¹² the Smart Form organizes clinical data around certain diseases to facilitate decision making and also highlights and “requests” missing coded information such as blood pressure, height, weight, and smoking status.

The Smart Form also is a CDSS and as such generates output that integrates patient demographic and clinical data with rule-based logic derived from guidelines for the management of CAD and DM.⁹ The output includes assessments of the current state of clinical care (eg, low-density lipoprotein cholesterol above the goal of 100 mg/dL) and suggested orders for medication additions or changes, laboratory studies, appointments and referrals, and printing of patient educational materials. If a suggested order is selected by the user, the action is carried out (ie, it is linked to provider order entry, such as prescription writing), and the EHR is automatically updated (ie, the medication list reflects the change). In addition, the selected action can be easily added to the note with a few keystrokes or mouse clicks (see Figure).

Ideally, the Smart Form would replace the users’ usual note-writing tools, including the standard free text or template- based note-writing function within the LMR for all patients with the conditions supported by it. In its current form, the Smart Form has to be actively chosen by the user when beginning a note-writing session.

Clinicians and Patients

Table 1

We recruited 10 adult primary care practices at Brigham and Women’s Hospital and Massachusetts General Hospital that use the LMR out of a total of 15 practices that were invited. Practices that agreed to participate were informed about the Smart Form and told that primary care physicians (PCPs) would be randomized to receive it or usual care. Eligible patients were defined as those with CAD or DM who had a visit with a PCP who belonged to 1 of the study practices from the date the practice was given the Smart Form until the end of the study period 9 months later. Practices received the Smart Form on a rolling basis from March 3, 2007, through August 10, 2007. To qualify, patients had to have CAD or DM on their EHR problem list as of the day prior to the start date of the RCT for that practice (see for a list of qualifying conditions). We previously found these definitions of CAD and DM to have a positive predictive value of 94% and 96%, respectively.¹³

Primary care physicians were assigned to receive the Smart Form or usual care on the basis of random number generation in Microsoft Excel (Redmond, WA). Those PCPs assigned to the intervention arm were notified by e-mail and received brief instruction on the use of the Smart Form at an on-site practice meeting. A computerized video tutorial about the Smart Form could be accessed at any time from within the application’s help menu. In addition, we took several steps to better engage clinicians in Smart Form use:

• We returned to each clinic to meet again with clinicians, encouraged use of the Smart Form, and performed on-site training, emphasizing integration into clinicians’ existing work flow.

• We tracked use by clinician and sent customized emails every 1-2 months to PCPs depending on whether they used the Smart Form frequently, infrequently, or never, reminding them to use it, encouraging use, and soliciting feedback on usability, as appropriate.

• We identified and contacted frequent users to find out why they liked the Smart Form to discover ways those lessons could be communicated to other intervention PCPs.

• Halfway through the study, we began sending monthly Tips for Users by e-mail, highlighting appealing but less obvious features of the Smart Form or ways to address potential usability issues mentioned by other users.

Outcomes and Data Sources

The primary outcome was the mean percentage of deficiencies in CAD/DM management addressed within 1 month of an index visit (ie, any visit by an eligible patient to their PCP during the study period). There were 9 possible performance measures for which patients with CAD could have a deficiency, including documentation measures such as smoking status, medication measures such as antiplatelet use, and management measures such as blood pressure control to recommended goals. Measures for patients with DM included the same measures for CAD except for antiplatelet and betablocker use, plus 6 others (see Table 1 for definitions).

Care management deficiencies were identified by querying the EHR and Partners Clinical Data Repository as of the day prior to the index visit. If the deficiency was present, we then queried the same data sources 30 days after the index visit to evaluate whether the deficiency had been addressed. Deficiencies were considered addressed if missing or out-of-date documentation was subsequently supplied, an indicated medication was prescribed or a contraindication documented, or action was taken in response to suboptimal management (see Table 1).

For each index visit, we analyzed whether each applicable deficiency was addressed and the percentage of applicable deficiencies that were addressed. As a secondary analysis, we evaluated the proportion of management goals met as of the day prior to the RCT study period and the last day of the study period for each practice (ie, patient-level as opposed to visit-level analysis).

Data Analysis

Baseline characteristics of clinicians and patients were analyzed using standard descriptive statistics. The primary outcome was the mean percentage of deficiencies in care management addressed per visit in an intention-to-treat analysis. In other words, outcomes of all patients assigned to intervention PCPs were compared with outcomes of all patients assigned to usual care PCPs, regardless of whether the PCP used the Smart Form at a given visit or with a given patient. The primary outcome was analyzed using binomial logistic regression (ie, with the dependent variable in the form X/N, where N equals the number of deficiencies and X equals the number of deficiencies addressed). To adjust for potential confounders, we created multivariable models that included all patient- and provider-level covariates that were significant predictors of the outcome from bivariable testing at a P <.10 level. Nonsignificant collinear terms then were removed from the final model for parsimony. We used generalized estimating equations (with PROC GENMOD in SAS 9.1 [SAS Institute, Inc, Cary, NC]) to adjust for clustering of patients by provider (ie, taking into account the fact that outcomes were analyzed at the visit or patient level while randomization occurred at the provider level and that patients of one provider might be more similar to one another than patients of other providers). These analyses were repeated for each individual quality measure using logistic regression models.

For patient-level analyses, binomial logistic regression was used in which X/N represented the total number of goals met over the number of applicable goals for that patient (ie, depending on the patient’s conditions). Repeated-measures analysis was conducted to evaluate whether the intervention was associated with greater improvement in the proportion of goals met at the end of the study compared with the beginning (ie, reported P values are for the interaction term of [intervention arm]*[time]).

Last, we repeated the above using “on-treatment” analyses, comparing outcomes for patients of PCPs who used the Smart Form during a given visit (in visit-level analyses) or with a given patient at all (for patient-level analyses) with outcomes for both control patients and patients of intervention PCPs for whom the Smart Form was not used. These 2 “nonuse” groups were combined in some analyses to evaluate the overall association between Smart Form use and outcomes. Unless otherwise stated, 2-sided P values less than .05 were considered significant. All analyses were conducted using SAS version 9.1.

RESULTS

eAppendix

The 10 primary care practices enrolled in the study included 239 PCPs who had 7009 patients who met inclusion criteria, who in turn had more than 18,000 primary care visits during the study period. The PCPs included roughly equal proportions of faculty and resident physicians. (See the at www.ajmc.com.) The mean number of patient visits per week per physician was 15.6, reflecting a large number of PCPs (including residents) who perform clinical work part-time, which is typical of an academic medical center. There were no statistically significant differences between PCPs in the 2 arms of the study.

Table 2

Patients had an average of more than 4 visits in the year prior to the study and more than 8 problems on their problem list (). More patients had DM than CAD, and more than 10% had both. Given the large sample size and randomization by provider, there were several statistically significant differences between the intervention and control groups, although the magnitude of the differences was small.

Overall use of the Smart Form was low. The PCPs assigned to the intervention arm used the Smart Form for 5.6% of eligible patients and at 3.6% of eligible visits. Use was higher for patients with DM (7.4%) than for patients with CAD (3.5%). Patients for whom the Smart Form was used tended to have more visits per year, to have more comorbidities, less likely to be white, to be more likely to have Medicaid insurance, and to have younger physicians caring for them (data not shown).

Table 3

Table 4

Despite low use of the Smart Form, patients of PCPs assigned to the intervention arm were more likely to have deficiencies in care addressed in the month following the index visit (intention-to-treat analysis, ). For example, if a patient did not have an up-to-date blood pressure measurement recorded in the vital signs section of the EHR prior to the visit, in the intervention arm this deficiency was addressed in 31.7% of patient visits, whereas it was only addressed in 23.8% of visits of patients in the control arm. This was driven by a 48% absolute difference in this measure when the Smart Form was used (75.0% vs 27.1%; ). Differences in individual measures were not statistically significant in adjusted and clustered analyses. However, the overall proportion of deficiencies addressed was significantly higher in the intervention arm in adjusted, clustered analysis (11.4% vs 10.1%; odds ratio [OR] = 1.14; 95% confidence interval [CI], 1.02-1.28; P = .02). A higher proportion of addressed deficiencies also was found in patients who were male, were Hispanic (as opposed to non-Hispanic white), had private insurance (as opposed to Medicare), and had fewer visits per year (data not shown).

In an on-treatment analysis, use of the Smart Form was associated with marked increases in the proportion of deficiencies addressed when compared with patients in the control arm or intervention patients in whom the Smart Form was not used (Table 4). Significant differences were noted in correction of deficiencies of documentation (eg, blood pressure, smoking status, foot and eye exams for diabetic patients) and also deficiencies in management (eg, prescription of antiplatelet medications in patients with CAD, change in antihypertensive therapy if blood pressure was above goal). The overall proportion of deficiencies addressed was 17% when the Smart Form was used, 11% in intervention patients when the Smart Form was not used, and 10% in the control group. Use of the Smart Form was associated with increased odds of having the deficiencies addressed in adjusted and clustered analysis (OR = 1.58; 95% CI, 1.31-1.90) compared with all patients in whom the Smart Form was not used.

In a secondary analysis, we examined whether a greater proportion of patients met their management goals at the end of the study compared with the beginning of the study. In an intention-to-treat analysis, there was no difference between intervention and control groups in the extent of improvement over time (adjusted and clustered OR = 1.02; 95%CI, 0.99-1.04). In an on-treatment analysis, significant differences were seen when comparing patients for whom the Smart Form was used with all patients for whom the Smart

Form was not used (OR = 1.16; 95% CI, 1.08-1.25; P <.001), including significant differences in the extent of improvement in up-to-date documentation of blood pressure results, smoking status, foot exams, and eye exams, as well as the proportion of patients who had become nonsmokers (data not shown).

DISCUSSION

We found that the introduction of documentation-based clinical decision support led to a statistically significant, but clinically small, improvement in the care of patients with CAD/DM in primary care. Overall use of documentation-based decision support was low—fewer than 6% of eligible patients. However, when used, the Smart Form was associated with substantial improvements in the documentation and management of patients with CAD and DM: an approximately 6.4% absolute increase in the proportion of deficiencies addressed at patient visits and a 16% increase in the relative odds of care goals being met by the end of the study period. The biggest improvements were seen in addressing deficiencies of documentation, although some deficiencies related to other aspects of management also improved.

There are several possible reasons for low use of Smart Forms in this study. Some are likely related to usability. Smart Form use required PCPs to actively choose a different way to document their visits. Although Smart Forms may have improved the documentation experience for some PCPs, some users did not find it intuitive to use.¹⁴ Despite anecdotal complaints about LMR usability in the usual note-writing environment, users often were reluctant to change well-worn habits of use, in particular for the minority of patients with CAD or DM. It is likely that use could have been increased through further refinement of the interface and/or by integrating Smart Forms into the usual EHR note-writing environment. Other studies have shown low use of CDSS, mainly driven by issues of usability and integration into work flow.⁸

Other possible reasons for low use are related to external factors. When visiting primary care practices, we were told on several occasions by PCPs that while seeing a patient every 12 minutes in the office they had no time to learn how to use a new system, especially one that might slow them down at first (ie, by documenting blood pressures). In fact, PCPs may not be the most appropriate targets for at least certain components of documentation-based decision support and chronic disease management. For example, one practice that excluded itself from the study decided to engage a medical assistant in use of the Smart Form. This assistant used the Smart Form to address all deficiencies of documentation. The practice then created a registry of all patients with DM, using data captured by the Smart Form to manage patients not meeting goals. In this context, the Smart Form was very well received.

In other words, improved management of patients with chronic diseases likely requires 4 components working in concert: (1) financial incentives to improve care via activities outside of patient visits such as those provided by a patient-centered medical home^15-17 or capitation,¹⁸ (2) a concerted multifaceted quality improvement effort, (3) distribution of work to a patient care team, and (4) well-designed health information technology tools that support these activities. Health information technology tools are unlikely to substantially improve care without the other 3 components. (In this regard it is notable that the one measure with the fewest baseline deficiencies and the one most often addressed during the study—having an up-to-date glycosylated hemoglobin A1C result&mdash;is part of a pay-for-performance contract at Partners HealthCare System.) Primary care physicians may need to be involved in certain parts of the disease management process (eg, documenting reasons why recommended actions are not taken, choosing among possible alternatives in conjunction with the patient or caregiver, managing the most complex of patients), but not others (eg, improving documentation of missing information, managing routine components of chronic care along specified protocols). With some minor modifications, Smart Forms could be used in this way, possibly in association with Quality Dashboards and other case management tools.¹⁹

To the extent that Smart Forms did improve care, we believe that they represent a novel form of documentation-based decision support. Components that may be worthy of adoption by other EHRs include requests for entry of coded data necessary to drive decision support, recommendations for care that are actionable (eg, linked to order entry), automatic documentation of actions taken, and ability to provide patient-specific educational tools.

There are several possible reasons why the largest effects of the intervention were seen regarding deficiencies in documentation compared with other management issues. The Smart Form was documentation-based, highlighting documentation deficiencies prominently and making it easy for users to correct them. Also, it may be easier to address a documentation deficiency than it is to overcome clinical inertia (eg, to increase the dose of an antihypertensive medication in response to poor blood pressure control).

These results should be viewed in light of the study’s limitations. We cannot exclude the possibility that the association of Smart Form use with improved management was one of confounding by indication: physicians who chose to use the Smart Form had already decided that they were ready to address that patient’s deficiencies in care. However, the significant (albeit small) improvement even in an intention-to-treat analysis makes it unlikely that this is the only explanation. The study was conducted in one medical system using

a proprietary EHR. However, it was conducted in 10 different primary care practices with varying characteristics (small and large, mostly full-time vs part-time clinicians, hospital-based, community-based, and community health center practices), and the concept of documentation-based decision support could be applied to any EHR, including vendor products.

Future research should focus on integrating components of documentation-based decision support into mainstream EHR documentation tools, using these tools to support multidisciplinary teams involved in chronic disease management, and conducting studies to assess care in conjunction with different payment models such as the patient-centered medical home.²⁰

In conclusion, the CAD/DM Smart Form had a modest effect on chronic disease management, mostly because of low use. Documentation-based decision support may be more effective in conjunction with different models of primary care and multidisciplinary quality improvement efforts.

Acknowledgments

We would like to thank Julie Fiskio, BA, for data analysis; Yelena Kleyner, BSc, and Megan Janes, BA, for research assistance; Anatoly Postilnik, MSc, for his efforts in developing Smart Forms; and Qi Li, MD, MBA, for his leadership as the Longitudinal Medical Record project manager.

Author Affiliations: From the Division of General Medicine and Primary Care (JLS, JAL, DTY, JSE, BM), Brigham and Women's Hospital, Boston, MA; Department of Medicine (JLS, JAL, MBP, DTY, JSE, BM), Harvard Medical School, Boston, MA; Clinical and Quality Analysis (DTY, KEM, LAV, AJM) and Clinical Informatics Research and Development (RT, JSE, BM), Partners HealthCare System, Boston, MA.

Funding Source: This study was supported by a grant from the Agency for Healthcare Research and Quality (1 R01 HS015169-01). Dr Schnipper was supported by a mentored career development award from the National Heart, Lung, and Blood Institute (1 K08 HL072806-01). Dr Linder was supported by a career development award from the Agency for Healthcare Research and Quality (K08 HS014563).

Author Disclosures: The authors (JLS, JAL, MBP, DTY, KEM, LAV, RT, AJM, JSE, BM) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JLS, JAL, MBP, LAV, RT, AJM, JSE); acquisition of data (JAL, DTY, KEM, LAV, JSE); analysis and interpretation of data (JLS, JAL, MBP, DTY, LAV, RT, AJM, JSE); drafting of the manuscript (JLS, JAL, KEM, RT, AJM, BM); critical revision of the manuscript for important intellectual content (JAL, MBP, DTY, KEM, LAV, RT, BM); statistical analysis (JLS, JAL, DTY); obtaining funding (JLS, JAL, LAV, BM); administrative, technical, or logistic support (JAL, KEM, LAV, RT, BM); and supervision (JLS, JAL, LAV, BM).

Address correspondence to: Jeffrey L. Schnipper, MD, MPH, Division of General Medicine and Primary Care, Brigham and Women's Hospital, 1620 Tremont St, Boston, MA 02120. E-mail: jschnipper@partners.org.

1. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223-1238.

2. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167(13):1400-1405.

3. Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008;16(2):79-91.

4. Filippi A, Sabatini A, Badioli L, et al. Effects of an automated electronic reminder in changing the antiplatelet drug-prescribing behavior among Italian general practitioners in diabetic patients: an intervention trial. Diabetes Care. 2003;26(5):1497-1500.

5. Meigs JB, Cagliero E, Dubey A, et al. A controlled trial of web-based diabetes disease management: the MGH diabetes primary care improvement project. Diabetes Care. 2003;26(3):750-757.

6. Tierney WM, Overhage JM, Murray MD, et al. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003;18(12):967-976.

7. Tierney WM, Overhage JM, Murray MD, et al. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005;40(2):477-497.

8. Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M. Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003;326(7384):314.

9. Schnipper JL, Linder JA, Palchuk MB, et al. "Smart Forms" in an Electronic Medical Record: documentation-based clinical decision support to improve disease management. J Am Med Inform Assoc. 2008;15(4):513-523.

10. Schnipper JL, McColgan KE, Linder JA, et al. Improving management of chronic diseases with documentation-based clinical decision support: results of a pilot study. AMIA Annu Symp Proc. 2008:1050.

11. Certification Commission for Healthcare Information Technology. Partners HealthCare System Longitudinal Medical Record (LMR) 5.1.1. 2006. http://www.cchit.org/products/2006/ambulatory/1441. Accessed December 8, 2009.

12. Henry SB, Douglas K, Galzagorry G, Lahey A, Holzemer WL. A template-based approach to support utilization of clinical practice guidelines within an electronic health record. J Am Med Inform Assoc. 1998;5(3):237-244.

13. Maviglia SM, Teich JM, Fiskio J, Bates DW. Using an electronic medical record to identify opportunities to improve compliance with cholesterol guidelines. J Gen Intern Med. 2001;16(8):531-537.

14. Horsky J, McColgan K, Pang JE, et al. Complementary methods of system usability evaluation: surveys and observations during software design and development cycles. J Biomed Inform. 2010;43(5):782-790.

15. Reid RJ, Coleman K, Johnson EA, et al. The group health medical home at year two: cost savings, higher patient satisfaction, and less burnout for providers. Health Aff (Millwood). 2010;29(5):835-843.

16. Gilfillan RJ, Tomcavage J, Rosenthal MB, et al. Value and the medical home: effects of transformed primary care. Am J Manag Care. 2010;16(8):607-614.

17. Grumbach K, Bodenheimer T, Grundy P. The Outcomes of Implementing Patient-Centered Medical Home Interventions: A Review of the Evidence on Quality, Access and Costs from Recent Prospective Evaluation Studies. Washington, DC: Patient-Centered Primary Care Collaborative; August 2009.

18. Tu K, Cauch-Dudek K, Chen Z. Comparison of primary care physician payment models in the management of hypertension. Can Fam Physician. 2009;55(7):719-727.

19. Linder JA, Schnipper JL, Einbinder JS, Li Q, Middleton B. Improving care for acute and chronic problems with Smart Forms and Quality Dashboards. AMIA Annu Symp Proc. 2006:1193.

20. Bates DW, Bitton A. The future of health information technology in the patient-centered medical home. Health Aff (Millwood). 2010;29(4):614-621.

Download PDF: Effects of Documentation-Based Decision Support on Chronic Disease Management