Publication
Article
The American Journal of Managed Care
Author(s):
Health information technology that is implemented as part of a multifaceted quality improvement initiative can lead to improvements in hypertension care and outcomes.
Objectives: To assess the impact of an electronic medical record (EMR) with clinical decision support (CDS) and performance feedback on provider adherence to guideline-recommended care and blood pressure (BP) control compared with a standard EMR alone.
Study Design: Quasi-experimental with repeated measures.
Methods: The study was conducted in a 4-site, federally qualified health center, Open Door Family Medical Centers, located in New York. The research team, Open Door leadership, providers, and staff developed and implemented a tailored multicomponent CDS system, which included a BP alert, a hypertension (HTN) order set, an HTN template, and clinical reminders. We extracted patient-level data for each encounter 17 months prior to implementation of the intervention (June 2007-October 2008) and 15 months post-intervention (April 2009-June 2010), from the EMR’s data tables for all adult nonobstetric patients with a diagnosis of HTN (N = 3636).
Results: Rates of HTN control were significantly greater in the post-intervention period compared with the baseline period (50.9% vs 60.8%; P <.001). Process measures, derived from the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure Guidelines, also improved significantly. Logistic regression with generalized estimating equations showed that patients were 1.5 times more likely to have controlled BP post-intervention than pre-intervention. Correlates of poor BP control were black race, higher body mass index, diabetes, female gender, income, and a greater number of prescribed antihypertensive medications.
Conclusions: Our findings suggest that health information technology that is implemented as part of a multicomponent quality improvement initiative can lead to improvements in HTN care and outcomes.
(Am J Manag Care. 2011;17(12 Spec No.):SP103-SP110)
An analysis of a multifaceted, technology-driven intervention to improve hypertension control in community health centers resulted in the following key findings:
Hypertension (HTN) is the most prevalent modifiable risk factor for cardiovascular disease among US adults, and it is among the most common reasons for an outpatient medical visit.1 Despite the availability of effective medications and published guidelines for the treatment of HTN, half of US adults who have been diagnosed with HTN have poorly controlled blood pressure (BP).2 Several recent literature reviews support a variety of implementation strategies aimed at increasing provider adherence to guideline-recommended care processes, including clinical reminder systems and performance feedback.3-9
Numerous studies have also examined the efficacy of clinical decision support systems (CDSS) in improving the quality of preventive care.7,10-12 Yet, despite the theoretical and intuitive benefits of such technologies, the existing literature has demonstrated mixed empirical results.11-13 Moreover, few studies have examined the deployment of CDSS to improve the quality of HTN management.7,14-16 Finally, most studies of CDSS have been conducted in academic health centers or the Veterans Administration system.7,14,16,17
We are aware of only 1 study of CDSS conducted in community health centers (CHCs).18 Yet, problems with quality of care and disparities are a particular concern for CHCs, which provide care for more than 15 million Americans, many of whom are most at risk for cardiovascular disease—related morbidity and mortality.15,19 Hicks et al found that among CHCs participating in Health Resources Service Administration Collaboratives, those with CDSS provided better quality of care than those without this technology.15 With primary care practices, including CHCs, responding to federal “Meaningful Use” requirements to adopt health information technology (HIT) by 2014, there is a need for greater understanding of technology-driven quality improvement strategies.20 The primary aim of this study was to compare the impact of a multicomponent intervention, which included an electronic medical record (EMR) with CDSS and registry-linked performance feedback, with a standard EMR alone on provider adherence to care recommendations and BP control as defined by the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7 Guidelines).1
METHODS
Study Design
We used a quasi-experimental design with repeated measures. For data analysis, BP measures were collected for each patient encounter for 17 months pre-intervention (June 2007-October 2008) and 15 months (April 2009-June 2010) post-intervention. The 5-month period from November 2008 to March 2009 was the intervention adoption period.
Study Setting and Subjects
The study was conducted at a 4-site federally qualified CHC, Open Door Family Medical Centers (Open Door), located in New York. The health centers provide primary care to approximately 40,000 patients annually, its population being primarily Hispanic (73.5%), with 60% foreign born. Thirty-five percent of patients had Medicaid and 58% were uninsured. In May 2007, Open Door installed eClinical-Works (eCW), an EMR and practice management system, at all 4 sites. Prior to the intervention, HTN control was part of the quarterly provider report card, which included 15 quality improvement measures, that was e-mailed to providers. However, the measures were not benchmarked against other providers or a target outcome. The EMR did not include CDSS components related to HTN.
We created a longitudinal database from EMRs for all adult nonobstetric patients with a diagnosis of HTN who had at least 1 visit during the study period. Hypertensive patients were defined as those with a documented International Classification of Diseases, Ninth Revision, code for HTN (401.0, 401.1, 401.9).
Intervention
The study team, which included Open Door leadership, providers and staff (ie, Chief Medical Officer [CMO], Director of Performance Improvement [DPI], Executive Director, and a physician representative), and staff from Primary Care Development Corporation, a non-profit organization with expertise in practice change and improvement, developed a set of measurable clinical goals related to HTN management that were based on JNC 7 Guidelines (eAppendix A).1 The goals were agreed to by the clinic’s Quality Improvement Committee, and became the basis for the development of the intervention and the set of data to be extracted from the EMR for analysis.
As part of the intervention development, we conducted quantitative and qualitative interviews with all clinical staff and administrative leadership. The results established a baseline for evaluating any changes in attitudes as well as informed the study team about what tools the providers wanted from the EMR, their facility in using the EMR, training requirements, and other workflow and support issues. The main intervention components included CDSS, provider performance feedback, and provider training. Based on the pre-intervention interviews, consultation with the study team, and review of the literature, several features for the CDSS were implemented.10,12 These included:
1. Alerts, highlighting in red an elevated BP;
2. A template, to present the provider with information to be obtained from the patient related to his/her HTN, and facilitate documentation;
3. Medication adherence forms, to prompt clinical support staff to ask patients about taking their medications and document the responses;
4. An Order set focused on HTN, allowing the provider to access a single screen when ordering tests or treatment (eAppendix B); and
5. Clinical reminders, to prompt providers to screen for tobacco use and/or update indicated tests (eg, lipid profile).
We provide more detail about the CDSS features in eAppendix C. In brief, the system was integrated with charting and order entry processes to support overall HTN-care workflow. For example, CDSS features appeared at the time and location of decision making (eg, when ordering medication), rather than just providing assessments, and occurred in multiple locations in the EMR.
The individual performance feedback was designed to enhance the existing report card system. The DPI ran quarterly reports on performance measures consistent with the main study outcomes, such as the percentage of HTN patients with visits in the previous quarter, whose BP was controlled. The CMO presented the overall performance of each site compared with Open Door’s target BP control at a regular quarterly staff meeting. Provider-level performance reports, which were benchmarked to predetermined targets were also e-mailed quarterly to providers, who could access their personal report through the EMR.
For clinical staff (physician assistants, nurse practitioners, medical doctors), the CMO conducted two 2-hour training sessions. The first included a review of JNC 7 HTN Guidelines and an overview of the HTN quality initiative, including clinical goals and objectives, as well as available baseline data. At the second training session, the CMO and research team demonstrated the new CDSS features, distributed a customized user manual dedicated to HTN CDSS features, and reviewed new policies and procedure guidelines for HTN management that were developed by the study team. Clinical support staff, including medical assistants, registered nurses, and licensed practical nurses, were trained together in a single 45-minute session using a curriculum customized to their roles (eg, importance of recording height and weight, asking about medication adherence, and using CDSS to record assessments).
Measures
The primary outcome was BP control before and after the intervention, as defined by JNC 7 criteria (BP <130/80 mm Hg for patients with diabetes or kidney disease and <140/90 mm Hg for all other patients). This was analyzed at the patient level and the encounter level. At the patient level, we report pre- and post-intervention mean systolic and diastolic blood pressures (SBP, DBP) and the proportion of patients with BP controlled at the last visit at both time periods (Table 1). The process measures included the proportion of patients who had a documented body mass index (BMI), electrocardiogram (ECG), lipid profile, and nutrition visit in each study period, timeliness of followup visits when BP was out of normal range, and medication intensification. Medication intensification was defined as the percentage of visits with a new medication added, titration of the original antihypertensive medication, or both, among visits with an elevated BP.
We describe the process for extracting encounter-level data from eCW’s back-end database in a previous article.21 Briefly, data from eCW’s underlying tables were extracted using BridgeIT, a data management utility that mapped Open Door’s EMR data into easily accessed and queried Microsoft Access software. Study variables were extracted from the data warehouse for each patient encounter, saved in a Microsoft Excel file, and transmitted securely to the Data Coordinating Center at Columbia University.
Statistical Analysis
After excluding patients of ethnicities other than black, white, or Hispanic and those with more than 30 visits, 3636 patients (28,263 encounters) were included in the final analysis. Data collected during the intervention adoption period were excluded from all analyses except for the time series in the Figure.
The distribution of sample characteristics (Table 2) among black, white, and Hispanic ethnicities was analyzed using analysis of variance (ANOVA) tests and x2 analysis. A preand post-intervention comparison on BP outcomes, as well as clinical process measures, was conducted using ANOVA tests and Fisher exact tests (Table 1). We also graphed changes in BP control over time using aggregated monthly encounter level data (Figure 1) and conducted an ANOVA test for significance of the BP trends over the 36-month study period.
A prediction model (Table 3) for BP control at any visit was developed using logistic regression considering the following covariates: age, BMI, gender, income level, race/ethnicity, diabetes status, and the number of HTN-related medications. Treatment effect was captured by an indicator that takes on a value of 1 if the particular visit occurred during the post-intervention period and 0 if the visit occurred during the baseline period. Generalized estimating equations22 were used to adjust for patients contributing multiple visits to the analysis. Coding for other categorical and continuous variables were based on which yielded the best fit in the univariate model where fit was evaluated using the Quasilikelihood under the Independence Model Criterion23 which is analogous to Akaike’s Information Criterion. The multivariate logistic model was built using a stepwise procedure with an entry/exit value of P = 0.05.
RESULTS
Table 2 shows the characteristics of patients with a diagnosis of HTN that had at least 1 visit during the study period. Patients had an average of 8.84 (standard deviation 6.62) clinic visits during the 36-month study period (mean of 5 visits both pre- and post-intervention with no significant difference).
As shown in Table 1, average SBP and DBP were significantly lower in the post-intervention period compared with pre-intervention. Rates of BP control at the last visit during each study period also improved from 50.9% pre-intervention to 60.8% post-intervention. Significant improvements in BP control were also observed among patients with diabetes (33.3% at pre-intervention and 46.9% at post-intervention). However, rates of BP control remained lower than among patients without diabetes.
Table 1 also shows significant improvements in all process measures except medication intensification. Figure 1 shows the trend of monthly cross-sectional measures toward improved HTN control over the course of the study. This trend was statistically significant (P <.001).
Controlling for other factors that were significantly associated with BP control, in Table 3 the logistic regression model shows that patients were 1.5 times more likely to have BP-controlled post-intervention than pre-intervention. Correlates of poor BP control were black race, male gender, income, greater BMI, diabetes, and larger number of prescribed antihypertensive medications.
DISCUSSION
We found that a multicomponent intervention that included provider performance feedback and CDSS promoted adherence to HTN clinical guidelines was associated with improvements in BP control. Provider feedback is a wellestablished strategy for improving adherence to guidelinerecommended care, and evidence is growing that CDSS and provider feedback are effective individually and in combination in facilitating quality improvement (QI).4,5,24-28 Moreover, as demonstrated in the study sites, clinical information systems embedded in EMR have the potential to provide an easy way to generate performance feedback. Therefore, we chose to combine these 2 guideline implementation strategies.
Results from studies of CDSS in the management of HTN have varied.7,14,16,18 We found improvements in most of our process measures, suggesting that clinicians were activated by the intervention to change practice patterns. However, similar to Roumie et al, we did not find a relationship between BP control and changes in medication intensification.7 In contrast, the Hicks study, which used CDSS to offer prescribing advice, did observe changes in medication intensification, but this was not associated with improvements in BP.18 Our study differed from Hicks in that the CDSS did not focus on medication management. Based on input from Open Door providers, the clinicians were given autonomy to use order sets or other system functions to obtain additional information as needed, but we did not specifically prompt providers to use a particular medication. Decisions about whether to change or add medications are complex, and unlike decisions about whether to order an ECG or laboratory test, depend on several patient-level factors (eg, adherence), which may be difficult to address and analyze. This may limit the usefulness of medication intensification as a measure of quality improvement. In the current study, without additional data about patient-level factors that may influence provider prescribing patterns, and lacking provider-reported reasons for not changing medications when BP was out of normal range, it is difficult to interpret our finding that medication intensification did not improve.
The lack of improvement in patient outcomes (ie, BP control) in the Hicks study may have also been related to the narrow focus of the CDSS functionality, which only provided prescribing advice, but did not address other processes related to HTN care. The current study took a more pragmatic “real-world” approach than previous studies in not limiting the CDSS to a single function and adopting a participatory approach to develop the CDSS, which was characterized by local user involvement in specifying the CDSS functions for each clinical objective.5,29,30 More specifically, the study team went through a systematic process adapted from Osheroff et al and informed by 2 HIT implementation models (Technology Acceptance Model and the Øvretveit et al model) to select features that were feasible based on how easy it would be to build within the current eCW platform; on skills, resources and preferences of the clinic; and the usability, acceptability, and ease of use from the provider’s perspective.31-33
A lack of attention to practice context and unique conditions that distinguish healthcare practices can impede effective adoption of QI interventions.30 Our process to tailor the QI intervention, which included preand post-intervention interviews with providers to assess agreement with guideline recommendations and other factors associated with adoption of a CDSS, allowed the sites to translate the guideline into locally accepted standards of practice.31-34 Based on findings from postintervention qualitative interviews with providers, this process enabled a closer fit between the goals of the intervention and practice conditions.35
Additionally, consistent with the chronic care model for systems-level QI, the CDSS was viewed as being only 1 component of practice change needed to reach the targets for each quality indicator. 36 For example, the CDSS provided opportunities for changes in the clinical team’s responsibilities. Staff at Open Door were trained to use the new CDSS to screen for adherence to medications, removing this task from the provider and engaging clinical support staff in this dimension of patient care. In a recent review of QI strategies for improving BP control, team changes (eg, creating new roles), was 1 of 3 strategies associated with the largest effects.9
Another factor that may have contributed to our positive findings was the features of the CDSS. There is evidence that specific characteristics of CDSS are necessary to affect quality of care. In a recent review of the CDSS literature, Kawamoto et al found that the following 4 features were strongly associated with positive findings: 1) automatic provision of decision support as part of the clinical workflow; 2) provision of recommendations rather than just assessments; 3) provision of decision support at the time and location of decision making; and 4) computer-based decision support.12 As eAppendix C indicates, Open Door’s CDSS had many of these features. In addition, the system required minimal training; offered several methods for obtaining similar information, allowing providers to choose from a menu of options to obtain information; and did not mandate practice patterns.
Despite increasing evidence for and specification of effective HIT strategies, including CDSS, there are gaps in our understanding of how these interventions work, and what the organizational and provider factors are that facilitate adoption of HIT.37 This limits our ability to generalize research findings to other settings and to disseminate promising interventions. In this study we used qualitative and quantitative methods to understand how the multicomponent intervention impacted intermediate outcomes or care processes that might mediate improved patient outcomes.31,32,37 Consistent with previous research, our qualitative findings indicated several factors that influenced the positive outcomes, including provider engagement in the intervention design, a rigorous facilitated implementation process that occurred within a broader organizational QI framework, and health center capacity to process data.35 Additionally, we found improvements in most of our quantitative process measures, suggesting that clinicians were activated by the intervention to change practice patterns. The inconsistencies in the HIT literature, however, indicate a need to further elucidate contextual factors that may facilitate adoption and better system design. Moreover, research is needed to assess the sustainability of practice changes in response to CDSS and other QI interventions.17
There are several limitations in this study. First, the study did not include a control condition; however, our analyses, both the trend analysis and predictive multivariate model, strongly suggest that the intervention affected patient outcomes. We conducted several additional analyses (not shown) that increase confidence in the findings. For example, we examined the trends over the 36-month study period in BP control of only new patient visits and found that they remained below 50%, indicating that increases in BP control were not related to new patients who might have entered the CHC with well-controlled BP. We also examined BP control pre- and post-intervention among only those patients who had at least 1 visit in both study periods, and also among patients who were only in the baseline compared with those only in the post-intervention period (cross-sectional analysis). Each of these analyses demonstrated significant improvements in BP control. Second, the longitudinal design is unbalanced, meaning that BP values were not observed at distinct time points and not all patients contributed the same number of BP measurements. However, the model adjusts for this discrepancy. Third, we are not able to separate the effect of the multiple components of the intervention. However, CDSS and the use of clinical information systems to provide feedback are key components of recommended care processes that have been associated with improvements in the quality of chronic care.38 How best to design each component (eg, unblinded vs blinded feedback, personalized algorithms vs generic CDSS) to optimize the synergy of multiple systemsand improve outcomes requires further research. Finally, we were not able to consistently and reliably collect data on the use of CDSS. Our post-intervention interviews with providers did offer insight into system use; however, this area requires further study.
In summary, this study found that a theory-driven approach to tailoring HIT to local context through user input and an iterative testing process can facilitate adoption of HIT. Moreover, when implemented as part of a multifaceted QI initiative, tailored to the local context, and developed with local user input, HIT can play a central role in assessing performance, improving adherence to care standards, and improving HTN-related patient outcomes.
Author Affiliations: From Division of General Internal Medicine (DS, T-YT, OO), New York University School of Medicine, New York, NY; Open Door Family Medical Center (DW, PF, AC, LF), Ossining, NY; Department of Sociomedical Sciences (MM), Mailman School of Public Health, Columbia University, New York, NY; EMMES Corporation (AGM), Rockville, MD; Primary Care Development Corporation (HK), New York, NY.
Funding Source: Agency for Healthcare Research and Quality (1R18HS017167-01).
Author Disclosures: The authors (DS, T-YT, AGM, DW, PF, AC, MM, OO, LF, HK) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (DS, DW, PF, AC, MM, OO, LF, HK); acquisition of data (DS, PF, AC, HK); analysis and interpretation of data (DS, T-YT, AGM, DW, PF, MM, OO); drafting of the manuscript (DS, T-YT, AGM, OO, HK); critical revision of the manuscript for important intellectual content (T-YT, AGM, MM, OO, HK); statistical analysis (T-YT, AGM); provision of study materials or patients (DS, DW, LF); obtaining funding (DS, LF, HK); administrative, technical, or logistic support (DS, DW, AC, HK); and supervision (DS, DW, LF, HK).
Address correspondence to: Donna Shelley, MD, MPH, 227 East 30th St, Rm 608, New York, NY 10016. E-mail: Donna.shelley@nyumc.org.
1. Chobanian AV, Bakris GL, Black HR, et al. Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. Hypertension. 2003;42(6):1206-1252.
2. Egan BM, Zhao Y, Axon RN. US trends in prevalence, awareness, treatment, and control of hypertension, 1988-2008. JAMA. 2010;303(20): 2043-2050.
3. Frijling BD, Lobo CM, Hulscher ME, et al. Intensive support to improve clinical decision making in cardiovascular care: a randomised controlled trial in general practice. Qual Saf Health Care. 2003;12(3): 181-187.
4. Grimshaw J, Eccles M, Tetroe J. Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004;24(suppl 1):S31-37.
5. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391): 1225-1230.
6. Oxman AD, Clarke MJ, Stewart LA. From science to practice. Metaanalyses using individual patient data are needed. JAMA. 1995;274(10):845-846.
7. Roumie CL, Elasy TA, Greevy R, et al. Improving blood pressure control through provider education, provider alerts, and patient education: a cluster randomized trial. Ann Intern Med. 2006;145(3):165-175.
8. Shojania KG, Ranji SR, McDonald KM, et al. Effects of quality improvement strategies for type 2 diabetes on glycemic control: a metaregression analysis. JAMA. 2006;296(4):427-440.
9. Walsh JM, McDonald KM, Shojania KG, et al. Quality improvement strategies for hypertension management: a systematic review. Med Care. 2006;44(7):646-657.
10. Damiani G, Pinnarelli L, Colosimo SC, et al. The effectiveness of computerized clinical guidelines in the process of care: a systematic review. BMC Health Serv Res. 2010;10:2.
11. Garg AX, Adhikari NK, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223-1238.
12. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494): 765.
13. Bryan C, Boren SA. The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008;16(2):79-91.
14. Bosworth HB, Olsen MK, Dudley T, et al. Patient education and provider decision support to control blood pressure in primary care: a cluster randomized trial. Am Heart J. 2009;157(3):450-456.
15. Hicks LS, O’Malley AJ, Lieu TA, et al. The quality of chronic disease care in U.S. community health centers. Health Aff (Millwood). 2006; 25(6):1712-1723.
16. Murray MD, Harris LE, Overhage JM, et al. Failure of computerized treatment suggestions to improve health outcomes of outpatients with uncomplicated hypertension: results of a randomized controlled trial. Pharmacotherapy. 2004;24(3):324-337.
17. Demakis JG, Beauchamp C, Cull WL, et al. Improving residents’ compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000;284(11): 1411-1416.
18. Hicks LS, Sequist TD, Ayanian JZ, et al. Impact of computerized decision support on blood pressure management and control: a randomized controlled trial. J Gen Intern Med. 2008;23(4):429-441.
19. Landon BE, Hicks LS, O’Malley AJ, et al. Improving the management of chronic disease at community health centers. N Engl J Med. 2007;356(9):921-934.
20. US Department of Health and Human Services. Center for Medicaid and Medicare Programs. Federal Register vol 75, No. 144, July 2010. http://edocket.access.gpo.gov/2010/pdf/2010-17207.pdf. Accessed August 15, 2011.
21. Shelley D, Tseng T, Andrews H, et al. Predictors of blood pressure control among hypertensives in community health centers. Am J Hypertens. 2011;24(12):1318-1323.
22. Liang K-Y, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73(1):13-22.
23. Pan W. Akaike’s information criterion in generalized estimating equations. Biometrics. 2001;57(1):120-125.
24. Casalino L, Gillies RR, Shortell SM, et al. External incentives, information technology, and organized processes to improve health care quality for patients with chronic diseases. JAMA. 2003;289(4):434-441.
25. Grimshaw JM, Eccles MP. Is evidence-based implementation of evidence-based care possible? Med J Aust. 2004;180(6 suppl):S50-S51.
26. Rhydderch M, Elwyn G, Marshall M, Grol R. Organisational change theory and the use of indicators in general practice. Qual Saf Health Care. 2004;13(3):213-217.
27. Shojania KG, Grimshaw JM. Evidence-based quality improvement: the state of the science. Health Aff (Millwood). 2005;24(1):138-150.
28. Weingarten SR, Henning JM, Badamgarav E, et al. Interventions used in disease management programmes for patients with chronic illness-which ones work? meta-analysis of published reports. BMJ. 2002;325(7370):925.
29. Minkler M. Community-based research partnerships: challenges and opportunities. J Urban Health. 2005;82(2) (suppl 2):ii3-ii12.
30. Ruhe MC, Carter C, Litaker D, Stange KC. A systematic approach to practice assessment and quality improvement intervention tailoring. Qual Manag Health Care. 2009;18(4):268-277.
31. Øvretveit J, Scott T, Rundall TG, Shortell SM, Brommels M. Improving quality through effective implementation of information technology in healthcare. Int J Qual Health Care. 2007;19(5):259-266.
32. Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology: toward a unified view. MIS Quarterly. 2003; 27(3):425-478.
33. Osheroff JA, Pifer EA, Teich JM, et al. Improving Outcomes with Clinical Decision Support: An Implementer’s Guide. Chicago, IL: Health Information and Management Systems Society; 2005.
34. Robinson JC, Casalino LP, Gillies RR, Rittenhouse DR, Shortell SS, Fernandes-Taylor S. Financial incentives, quality improvement programs, and the adoption of clinical information technology. Med Care. 2009;47(4):411-417.
35. Millery M, Shelley D, Wu D, et al. Qualitative evaluation to explain success of multifaceted technology-driven hypertension intervention. Am J Managed Care. 2011;17(12 Spec No.):SP95-SP102.
36. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: translating evidence into action. Health Aff (Millwood). 2001;20(6):64-78.
37. Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4:67.
38. Hung DY, Shelley DR. Multilevel analysis of the chronic care model and 5A services for treating tobacco use in urban primary care clinics. Health Serv Res. 2009;44(1):103-127.