Leveraging EHRs to Improve Hospital Performance: The Role of Management

Objectives

Recent studies fail to find a consistent relationship between adoption of electronic health records (EHRs) and improved hospital performance. We sought to examine whether the quality of hospital management modifies the association between EHR adoption and outcomes related to cost and quality.

Study Design

Retrospective study of a random sample of US acute care hospitals.

Methods

Management quality was assessed via phone interviews with clinical managers predominantly from cardiac units in a random sample of 325 hospitals using a validated scale of management practices in 4 areas: operations, performance monitoring, target setting, and talent management. American Hospital Association InformationTechnology Supplement data captured whether or not these hospitals had at least a basic EHR. Acute myocardial infarction (AMI) outcomes included risk-adjusted 30-day mortality, average length-of-stay, and average payment per discharge measured using MedPAR data. Ordinary least squares regressions assessed whether management quality modifies the relationship between EHR adoption and AMI outcomes.

Results

P

P

P

While we found no association between EHR adoption and our outcomes, management quality modified the relationship in the predicted direction. For length of stay, the coefficient on the in- teraction between EHR and management was —1.48 (= .05) and for payment, it was — 7786.74 (= .014). We did not find strong evidence of effect modification for mortality (coefficient = —0.05; = .37).

Conclusions

Coupled with ongoing policy efforts to achieve nationwide EHR adoption is a growing unease that our national investment may not result in better, more efficient care. Our study is among the first to offer empirical evidence that management quality may help explain why some hospitals see substantial gains from EHR adoption while others do not.

Am J Manag Care. 2014;20(11 Spec No. 17):SP511-SP519

The US healthcare system is in the midst of an ambitious nationwide effort to adopt and use electronic health records (EHRs). Under the 2009 Health Information Technology for Economic and Clinical Health(HITECH) Act, hospitals that implement EHRs and demonstrate that they meet federally defined meaningful use criteria are eligible for financial incentive payments.^1,2 HI-TECH was motivated by widespread agreement that the use of EHRs would result in substantial performance improvement. Key EHR functions—easier access to patient medical records, electronic medication ordering, and clinical decision support—should all lead to better care, less duplication, and ultimately, more efficient care. Unfortunately, the evidence to date on the impact of widespread EHR uptake has been somewhat disappointing,3-6 and suggests that the average effect of EHR adoption may be close to zero. These results have been puzzling, giving rise to the idea that factors beyond the adoption of EHRs may be influential.

In industries such as retail and banking, the degree to which organizations are well managed is an important determinant of whether performance improves after adopting new forms of information technology (IT).^7,8 It is possible that a similar phenomenon is at play in healthcare. The intuition that management should matter is simple: IT is a tool that influences performance based largely on how it is used, and without a concerted and thoughtful approach to managing how IT is leveraged to improve performance, its use will not achieve the potential quality and efficiency gains. In fact, implementing IT systems in poorly managed organizations may be disruptive in ways that cause productivity losses. However, the empirical evidence for this hypothesis in healthcare settings is largely lacking.

In this paper, we leverage a unique data set on hospital management practices collected through interviews with managers in a national random sample of acute care hospitals in order to explore whether management quality modifies the relationship between EHR adoption and hospital outcomes. We focus on outcomes related to acute myocardial infarction (AMI) because management survey interviewees were predominantly from cardiology units. We hypothesize that in well-run institutions, EHRs would be associated with better performance, while in poorly run institutions, EHR adoption may be associated with worse performance.

METHODS

Data, Measures, and Sample Our study uses 4 sources of data. To determine whether a hospital had an EHR, we used the American Hospital Association (AHA) IT Supplement that was administered to all AHA member hospitals in 2009 (with a response rate of 69%) and captures IT functions in place at the end of 2008.⁹ We determined which hospitals had at least a basic EHR using the Jha et al (2009) definition of the 10 specific functions that comprise a basic EHR, such as computerized provider order entry (CPOE) for medications, test result viewing, problem lists, and medication lists.10 We used the broader AHA Annual Survey for data on key hospital characteristics: size (<100 beds, 100-399 beds, 400+ beds), teaching status (major teaching, minor teaching, nonteaching), ownership (private, for-profit; private, not-for-profit; public), census region, urban location, percent of discharges from patients with Medicare, and whether or not the hospital was part of a system.

To capture management quality, we used data from the World Management Survey that was collected in 2009 from a national random sample of 325 AHA-member acute care hospitals. Management practices were measured using a team of trained analysts who conducted phone interviews with unit or departmental managers in cardiac or orthopedics units in sample hospitals. Interviews were conducted between March and October 2009 to capture management practices in 4 broad areas, each of which could potentially influence how an EHR is used in a given hospital (eAppendix Table A1, available at www.ajmc.com). The first area, operations management, captures the extent to which primary clinical procedures are standardized and how well the clinical staff is trained in these protocols. The second area, performance monitoring, focuses on how well the organization’s performance monitoring system informs day-to-day operations of the hospital. The third area, target setting, captures how tightly targets are linked to the hospital’s wider objectives and how well targets cascade down and are clear to employees. The last area is talent management, which captures various dimensions of human capital management, including the types of systems that are in place to recruit hospital staff, whether the hospital evaluates employees and rewards them based on performance, and whether there is an active promotion system that strives to retain and incentivize employees.

The tool applies definitions and scores from 1 (“worst practice”) to 5 (“best practice”) along 20 dimensions across the 4 areas listed above (operations, performance, targets, and talent), with an average of the 20 scores serving as the overall management score. In other words, a score of 5 represents excellent management practices across all areas. In eAppendix Table A2, we provide a list of sample questions for each area as well as information for accessing the complete survey instrument. The reliability of these measures and their robustness to many different forms of psychological bias are described by Bloom and Van Reenen (2007).¹¹ Interviewers were extensively trained in advance on hospital management, and a double-blind technique was employed in which interviewers were not told in advance about the hospital’s performance and interviewees were not told in advance how their answers would be scored. The data set also included interview and interviewee characteristics, such as the duration of the interview and the tenure of the interviewee in their position.

eAppendix Table A3

To capture both cost and quality performance of hospitals in our sample, we examined 3 risk-adjusted outcomes: 30-day mortality, average length of stay (LOS), and average payment per discharge (what the hospital received in payment for the stay). Risk adjustment accounted for patients’ age, sex, and comorbidities (using the Elixhauser Index). All measures were generated from Medicare data for fee-for-service beneficiaries 65 years or older who were hospitalized for AMI. We used 2009 as our focal year, but we also examined 2010 to assess whether associations persisted 1 year later. We chose to focus on AMI outcomes because we were concerned that there might be substantial unit-to-unit variation in both management and outcomes within hospitals. Since the majority of interviewees in the management survey represented cardiology units, AMI-focused outcomes were thought to be the most reliable indicators to examine the association with management quality and EHR adoption. Our analytic sample is the 191 US acute care hospitals that participated in the management survey and that responded to AHA IT Supplement in 2009. In , we compare our analytic sample with the broader AHA sample on key hospital characteristics.

Analytic Approach

To assess whether management quality modifies the association between EHR adoption and hospital performance, we estimated ordinary least squares regressions in which we predicted (a) 30-day AMI mortality rate, (b) average LOS for AMI discharges, and (c) average payment for AMI discharges. Our focal predictors are the hospital’s management score (range of 1-5) and whether or not the hospital had adopted at least a basic EHR. We include a set of control variables to account for factors that may confound the relationship among management quality, EHR adoption, and outcomes: size, teaching status, ownership, urban versus rural location, percent Medicare discharges, and whether the hospital is affiliated with a larger system. All our models include hospital-level weights that adjust for potential nonresponse bias due to known differences between respondents and nonrespondents to the AHA IT Supplement. Weights are derived by estimating the propensity to respond to the survey using a logistic regression model with key hospital characteristics as predictors, and the inverse of this propensity value serves as the weight.¹⁰

To assess whether management quality modifies the association between EHR adoption and our outcomes, we included a term that interacts the management score with EHR adoption. We also calculated and graphed predictive margins across the range of management scores in increments of 0.5 (approximately equal to the standard deviation of 0.55 in our sample, eAppendix Table A4) in order to compare predicted outcomes for hospitals with and without EHRs.

We also conducted a set of robustness tests to assess whether our results change under different specifications. First, we included in our models a set of variables that capture potential confounders driven by differences in interview-level characteristics. Specific variables included the duration of the interview (in minutes), the reliability of the interviewee (assessed by scoring both their knowledge of management practices and their willingness to reveal infor- mation), whether the interviewee was a nurse or a physician, the seniority of the interviewee, and the number of years that the interviewee had been in their current position. Our second set of robustness tests relaxed the requirement that a hospital have all the functions that comprise a basic EHR and examined 3 specific EHR functions: test result viewing, CPOE, and decision support. Our third and final set of robustness tests examined each individual area of management independently: operations management, performance monitoring, target setting, and people management. This enables us to assess whether any observed relationships are primarily driven by 1 particular area of management.

RESULTS

11

Of the 191 hospitals in our sample 12% had adopted at least a basic EHR as of the end of 2008 (eAppendix Table A4).The average management score of 3.01 (SD = 0.55; range 1.4-4.3) was effectively at the midpoint of the 1-5 scale. For our primary outcomes of interest, the 30- day mortality rate was 19% (SD = 17%; range 0%-100%). The average risk-adjusted LOS for patients hospitalized for AMI was 4.89 days (SD = 1.99; range 0.86-10.38) and average payment per discharge was $12,544 (SD = $7414; range $995-$41,955).

Table 1

P

P

P

P

When we examined key characteristics stratified by EHR adoption status, we found few differences (). Hospitals with an EHR were slightly better managed, but the difference was not statistically significant (mean score of 3.18 compared with 2.99 in hospitals without an EHR; = .191). Compared with those without an EHR, hospitals with an EHR were more likely to be a major teaching hospital as well as private, not-for-profit (14% versus 5% for major teaching, value across categories of teaching .060; 77% vs 63% private, not-for-profit, value across categories of ownership .058). Finally, hospitals with EHRs had a somewhat lower proportion of Medicare admissions (45% compared with 51% among hospitals without EHRs, = .006).

P

Table 2

P

P

In our baseline models, we failed to find evidence that basic EHR adoption is independently associated with improved performance for any of our 3 outcomes. EHR adoption was associated with slightly higher 30-day mortality for AMI patients (coefficient = 0.03; = .93; Model 1, ) and slightly lower average LOS (coefficient = —0.011; = .98; Model 3, Table 2) compared with those without EHRs, but these were not statistically significant. Hospitals with EHRs also had slightly higher average payment per AMI discharge (coefficient = $184.59; = .88; Model 5, Table 2).

P

P

P

P

P

When we examined whether management modifies the association between EHR adoption and our outcomes of interest, all 3 outcomes were in the predicted direction, and 2 of the 3 were statistically significant. We did not find evidence of effect modification for mortality (coefficient on interaction between EHR and management = —0.05; = .37) (Model 2; Table 2). For LOS, the coefficient on the interaction between EHR and management was —1.48 (= .05), and for payment, the coefficient on the interaction term was —7786.74 (= .014) (Models 4 and 6; Table 2). These associations persisted when we used outcomes from 2010 instead of 2009; for LOS, the coefficient on interaction between EHR and management was —1.34 (= .06) and for payment the coefficient was —6228.49 (= .001).

Figures 1

2

3

We depict these associations in , , and , which graph the predictive margins across the range of management scores for hospitals with, and without, EHRs. In both hospitals with and without EHRs, an increase in management score is associated with a decrease in 30-day mortality. However, as we hypothesized, at the lower end of the management quality scale, hospitals without EHRs perform better than those with EHRs while the opposite relationship holds at the higher end of the scale (Figure 1). Similarly, in hospitals with and without EHRs, an increase in management score is associated with a shorter LOS, but the decrease is much smaller for hospitals without EHRs compared with those with EHRs (Figure 2). Finally, an increase in management score is associated with decreasing average payment per discharge among hospitals with EHRs, while the reverse is true for hospitals without EHRs (Figure 3).

To help interpret the magnitude of these modified effects, we compared the predictive margins for hospitals with and without EHRs at 2 levels of management: relatively poor management (a score of 2) and relatively strong management (score of 4). For example, for LOS, in a hospital that is not well managed, hospitals with EHRs have a LOS of 6.9 days compared with 5.0 days in hospitals without an EHR (28% higher). In contrast, in well-managed hospitals, those with EHRs have a LOS of 3.7 days compared with 4.7 days in hospitals without EHRs (27% lower).

P

eAppendix Tables A7-A10

Our various robustness tests generated results consistent with our primary analyses. First, the modified associations persisted after we adjusted for differences in interview-level characteristics (Appendix Table A5). Next, when we replaced our dichotomous measure of EHR adoption with individual EHR component functions—results viewing, CPOE for medications, and decision support&mdash;our results remained largely unchanged for decision support and results viewing. However, for results viewing, the coefficient on the interaction term with management on 30-day mortality became significant in the predicted direction (coefficient on the interaction between results viewing and management = —0.15; = .03; Appendix Table A6). For CPOE, none of our effect modifications were statistically significant. Finally, we also found directionally consistent results when we replaced the composite management score with scores for each of the 4 individual areas separately, although the interaction effect for LOS was no longer statistically significant ().

DISCUSSION

Coupled with ongoing policy efforts to achieve nationwide EHR adoption is a growing unease that our national investment in EHRs may not pay off in terms of better, more efficient care. We too find no association, on average, between EHR adoption and our 3 measures of hospital performance for AMI patients. However, our study also points to a path forward. These findings are among the first to provide empirical evidence that the quality of management may play a role in explaining when EHR adoption translates into improved performance. Specifically, when hospitals are poorly managed, EHRs are associated with worse efficiency—higher LOS and payments&mdash;and when hospitals are well managed, having an EHR is associated with greater efficiency. We find some evidence that this relationship holds for mortality as well. These findings suggest that improving hospital management may be an important factor for ensuring that EHRs drive increases in hospital productivity.

12

13,14

What might well-managed hospitals be doing with their EHRs that differs from what takes place in poorly managed hospitals? We did not find evidence that 1 particular dimension of management accounted for our observed associations. We therefore suspect that several factors stemming from overall high-quality management are at work. First, well-managed hospitals may have established a clear vision and set of performance improvement goals motivating EHR adoption while less well-managed hospitals may have adopted EHRs in response to external pressures with no clear plan for EHR-enabled improvement.In addition, in well-managed hospitals, the goals for EHR adoption may have been developed in partnership with end users (eg, frontline physicians and nurses) and therefore had greater staff “buy-in” both during and after implementation.

15-17

Beyond setting goals and securing staff buy-in, well-managed hospitals may be more aggressively pursuing advanced EHR functionalities, such as clinical decision support, and other approaches to increasing care standardization (ie, care protocols embedded in the EHR), compared with their less well-managed counterparts. These approaches have been shown to result in substantial value,but require engaged managers and additional organizational investment to alter work flows and overcome barriers. Finally, well-managed hospitals may be using EHR data for learning through perfor- mance measurement and monitoring. EHRs offer timely, clinically rich data that can be analyzed in real time; for example, EHRs can provide clinicians with information about hypertension control among their diabetic patients or show clinicians how they compare with their peers on these outcomes. Well-managed organizations are likely harnessing these valuable insights to drive improvements in care. We suspect that many of these strategies are missing at poorly managed institutions.

18

While a better understanding of the particular man- agement strategies that lead to improved performance following EHR adoption is needed, a critical question for policymakers is how to expedite the process of not only identifying these management strategies but also encouraging their widespread uptake. To date, the federal criteria for meaningful use of EHRs have primarily focused on promoting EHR functions, rather than encouraging hospitals to leverage EHRs in broader ways. Thus, while meaningful use criteria may help motivate the uptake of specific functions—such as clinical decision support&mdash;the criteria are less focused on pushing hospitals to transform their management or care delivery processes. Proposed future stages of meaningful use do markedly expand performance measurement and broader uses of EHR data.However, more substantial changes in financial incentives tied to performance, such as value-based purchasing, will be required to motivate hospital leadership to focus their attention here. If that leads hospitals to improve their management practices, our findings suggest that this may enable hospitals to realize more value from their EHR investments.

19

There are important limitations to our study. First, for a subset of the hospitals in our sample, management data did not come from cardiac units; therefore, by relying on AMI outcome measures, we may not capture the effect modification of interest. However, we believe that this is primarily an issue of power and it would not introduce systematic bias into our results. Similarly, we may have been underpowered to detect the relationship of interest for 30-day risk-adjusted AMI mortality, which had much less variation across hospitals compared with our other outcome measures. Second, our data predate much of the HITECH activity and as a result, a relatively small number of hospitals in our sample had adopted at least a basic EHR. These early adopters could have been unique and the associations that we observe may have changed as more hospitals have adopted EHRs.However, only 1 characteristic of EHR adopters significantly differed from nonadopters—percent of Medicare admissions&mdash;and we included this variable as a control in our analyses. Further, we think it is likely that HITECH makes effective management even more pertinent in order to shepherd the changes brought about by EHR adoption under the rapid adoption timelines included in HITECH. Third, our payment variable is based on administered prices and we do not adjust for differences across hospitals in wage or price modifiers. While these differences could drive the relationship we observe for payment, our finding would also result under the following scenario (which is consistent with our hypothesis): better-managed hospitals with EHRs deliver higher-quality care to AMI patients, and this care results in fewer complications and the assignment of lower-cost diagnosis-related groups.

20

Finally, we are not able to assess causality and could only assess associations. Indeed, there are likely other factors associated with high-quality management that could explain our results. In particular, work outside the health sector has shown that better management is associated with higher worker skills. Thus, it may be that the combination of EHRs and more highly skilled clinicians is what drives our findings.Nonetheless, the role of management in enabling IT-driven performance gains is well-established in other industries, and it is therefore likely that it contributes, at least in part, to EHR-enabled performance improvement in healthcare organizations as well.

CONCLUSIONS

In summary, this paper is among the first to identify a key factor that may explain why some hospitals realize substantial gains from EHR adoption while others end up worse off. By focusing on the quality of management in a random sample of US hospitals, we are able to observe effect modification for 2 important outcomes: LOS and payment. We also see a directionally consistent, though not statistically significant, association with lower 30-day mortality. Our results suggest that hospitals as well as policy makers may want to bolster the extent to which managers focus their efforts on leveraging newly available EHR capabilities, with the goal of driving performance improvement. Identifying and promoting other factors that lead to improved efficiency from health IT will be critical to ensure that our large national investment in EHRs translates into real productivity gains for our healthcare system.

Author Affiliations:

University of Michigan School of Information and School of Public Health, Ann Arbor (JA-M); Harvard University, PhD Program in Health Policy, Cambridge, MA (KWS); Harvard University, School of Public Health, Cambridge, MA (AKJ).

Source of Funding:

None.

Author Disclosures:

The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information:

Concept and design (JA-M, KWS, AKJ); acquisition of data (JA-M, AKJ); analysis and interpretation of data (JA-M, KWS, AKJ); drafting of the manuscript (JA-M, KWS, AKJ); critical revision of the manuscript for important intellectual content (JA-M, KWS, AKJ); statistical analysis (JA-M); administrative, technical, or logistic support (AKJ); supervision (AKJ).

Address correspondence to:

Julia Adler-Milstein, PhD, University of Michigan, School of Information, 4376 North Quad, Ann Arbor, MI 48109. E-mail: juliaam@umich.edu.

1. Blumenthal D. Launching HITECH.

N Engl J Med

. 2010;362(5): 382-385.

N Engl J Med

2. Blumenthal D,Tavenner M.The “meaningful use” regulation for electronic health records. . 2010;363(6):501-504.

Arch Intern Med

3. Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, Powe NR. Clinical information technologies and inpatient outcomes: a multiple hospital study. . 2009;169(2):108-114.

Am J Med

4. Himmelstein DU, Wright A, Woolhandler S. Hospital computing and the costs and quality of care: a national study. . 2010;123(1): 40-46.

Health Aff (Millwood)

5. McCullough JS, Casey M, Moscovice I, Prasad S.The effect of health information technology on quality in U.S. hospitals. . 2010;29(4):647-654.

Health Care Manage Rev

6. Menachemi N, Chukmaitov A, Saunders C, Brooks RG. Hospital quality of care: does information technology matter? the relationship between information technology adoption and quality of care. . 2008;33(1):51-59.

Amer Econ Rev

7. Bloom N, Sadun R, Van Reenen J. Americans do IT better: US multi-nationals and the productivity miracle. . 2012;102(1): 167-201.

J Manag Infor Sys

8. Hitt LM, Wu DJ, Xiaoge Z. Investment in enterprise resource planning: business impact and productivity measures. . 2002;19(1):71-98.

AHA Healthcare IT Database: A Supplement to the AHA Annual Survey of Hospitals.

9. American Hospital Association. Chicago, IL: American Hospital Association, 2009.

N Engl J Med

10. Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in U.S. hospitals. . 2009;360(16):1628-1638.

Quar J Econ

11. Bloom N, Van Reenen J. Measuring and explaining management practices across firms and countries. . 2007;122(4): 1351-1408.

12.The National Learning Consortium and Health InformationTechnology Resource Center. Change management in EHR implementation: primer. HealthIT.gov website. http://www.healthit.gov/sites/default/ files/tools/nlc_changemanagementprimer.pdf. Published April 30, 2013. Accessed November 10, 2014.

Health Aff (Millwood)

13. Gold MR, McLaughlin CG, Devers KJ, Berenson RA, Bovbjerg RR. Obtaining providers’ ‘buy-in’ and establishing effective means of information exchange will be critical to HITECH’s success. . 2012;31(3):514-526.

Acad Med

14. MassaroTA. Introducing physician order entry at a major academic medical center: I: impact on organizational culture and behavior. . 1993;68(1):20-25.

Ann Intern Med

15. Bright TJ, Wong A, Dhurjati R, et al. Effect of clinical decision-support systems: a systematic review. . 2012;157(1):29-43.

JAMA

16. Garg AX, Adhikari NKJ, McDonald H, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. . 2005;293(10):1223-1238.

17. McCarthy D, Mueller K, Wrenn J, Fund C. Geisinger Health System: achieving the potential of system integration through innovation, leadership, measurement, and incentives. NewYork, NY:The Commonwealth Fund; 2009.

Federal Register.

18. Office of the National Coordinator for Health InformationTechnology. HIT Policy Committee: request for comment regarding the Stage 3 definition of meaningful use of electronic health records (EHRs). November 26, 2012. https://federalregister .gov/a/2012-28584.

Health Aff (Millwood)

19. DesRoches CM, Charles D, Furukawa MF, et al. Adoption of electronic health records grows rapidly, but fewer than half of US hospitals had at least a basic system in 2012. . 2013; 32(8):1478-1485.

Acad Manage Perspect

20. Bloom N, Genakos C, Sadun R, Van Reenen J. Management practices across firms and countries. . 2012;26(1): 12-33.