The Impact of Health Information Technologies on Patient Satisfaction

ABSTRACT

Objectives: To determine if health information technologies (IT) impact patient responses on the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) satisfaction survey.

Study Design: A retrospective, pre-post observational study to compare the percentage of top-box responses by HCAHPS composite domain before and after implementation of electronic medication administration record (eMAR), computerized provider order entry (CPOE), and electronic progress notes (PN).

Methods: We defined 3 pre-post comparison periods for introduction of eMAR, CPOE, and PN, and 2 control periods. The pre-implementation periods comprised the 4 months prior to tool addition. Postimplementation periods comprised the 4 months from the second to the fifth month, inclusive, following the unit going live with a tool. Changes in the percentage of top-box scores were tested using logistic regression. The combined changes during the health IT implementation periods and the combined changes during control periods were tested and compared using contrasts in the logistic regression model.

Results: Only PN had a significant negative impact on 2 questions in unadjusted analysis, both of which became nonsignificant in adjusted analyses. eMAR had a significant negative impact on 1 question in adjusted analyses only. The combined impact of health IT had mixed results, none of which were significant. During control periods, there was improvement in all domains, with statistically significant improvements in Discharge Information and Communication About Medicines.

Conclusions: Health IT investments appear to have no impact on HCAHPS scores. Stakeholders should investigate synergistic opportunities between these resource-intensive initiatives to optimize costs, quality, and patient experience.

The American Journal of Accountable Care. 2016;4(4):9-15

The introduction of CMS’ Meaningful Use incentive payments in 2011 has encouraged hospitals to increasingly utilize health information technologies (IT).¹ When implementing new tools, an institution should consider how health IT can influence the quality, safety, and outcomes measures that determine reimbursements from CMS’ value-based purchasing (VBP) program.² In 2013, CMS added the patient experience of care domain to VBP through the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS), which linked patient-reported satisfaction scores to value-based payments.³ Consequently, within a 2-year span, healthcare systems have had to simultaneously invest millions of dollars and large amounts of clinician and administrator time into new initiatives to improve patient experience and meet Meaningful Use policies.^4-8 Mechanisms by which health IT investments can impact HCAHPS scores should be an important consideration for administrators, policy makers, and clinicians responsible for optimizing delivery of affordable and high-quality care.⁹

Currently, there are limited studies with mixed results investigating direct relationships between health IT utilization and HCAHPS scores. A study of all hospitals eligible to participate in the VBP program for 2013 found no significant differences in total patient experience of care scores by level of advanced electronic health record usage.⁹ Another national study found positive associations between the average responses to the HCAHPS global domain questions and medium and high health IT usage.¹⁰ Only 1 study evaluated effects of health IT on composite domains of the HCAHPS survey.¹¹ Using 2013 VBP data, this study showed that health IT utilization does improve patient satisfaction with discharge information, but does not influence satisfaction with provider communication or the patient’s willingness to recommend the hospital.¹¹

The purpose of our study was to evaluate if investments in health IT accelerated performance in patient-reported satisfaction scores, which have steadily risen since our institution began using the HCAHPS survey in 2008. Our administration uses a combination of HCAHPS and other valuable patient satisfaction indicators, but our study focused on HCAHPS scores as they are a generalizable measure upon which significant reimbursement is contingent. Since 2008, our institution has implemented 3 point of care tools—the electronic medication administration record (eMAR), computerized provider order entry (CPOE), and electronic progress notes (PN)—that have been shown to have positive associations with process efficiency, staff satisfaction, and quality measures.^12-15 To investigate whether these tools improve care from the patient perspective, we compared HCAHPS scores by composite domain before and after tool implementation. By determining how health IT utilization influences patient perceptions of their care, we aimed to inform stakeholders about opportunities to capitalize on existing synergistic effects to provide high-quality care while maximizing value-based reimbursements.

METHODS

Study Design

A retrospective, pre-post observational study to compare HCAHPS scores before and after implementation of eMAR, CPOE and PN.

Setting and Sample

We analyzed surveys from 31 inpatient units—13 medical, 7 surgical, 5 stepdown/intensive care unit (ICU), and 6 women’s and children’s services—collected from 2008 to 2015 at 2 hospitals in our independent academic health system based in Wilmington, Delaware. Christiana Hospital is a 913 bed suburban teaching hospital and Wilmington Hospital is a 241-bed urban teaching hospital.

We defined 5 comparison periods: 3 pre-post periods (A, B, and C) for the introduction of eMAR, CPOE, and PN, respectively; and 2 control periods (D and E) in which health IT was unchanged. The Figure shows the periods of interest for surgical units at Christiana. The control periods, D and E, were included to determine how HCAHPS scores changed during the years when health IT was constant. By comparing changes during control periods to changes during implementation periods, the general improvement in HCAHPS scores at our institution over the 7-year timeframe of our study could be specifically linked to periods of control or implementation.

The pre-implementation periods comprised the 4 months prior to the addition of a tool onto the unit. The postimplementation periods comprised the 4 months from the second to the fifth month, inclusive, following the unit going live with a tool. Based on methods of previous studies, we chose 4-month periods, excluding the first month after implementation, to allow sufficient time for provider adjustment while capturing a small enough period to link survey scores to the addition of a new tool.^16-20

HCAHPS surveys were combined into periods based on patient discharge date. Each tool was implemented on different units at different times.

Outcomes

We compared percentages of top-box responses for 14 HCAHPS questions focused on patient interactions. Top-box responses are the measures used to calculate the patient experience of care score that determines CMS reimbursements.²¹ “Top box” is defined as the highest positive category for all questions, except for hospital rating, for which top box includes the highest 2 categories. The patient interaction questions we selected fall into 6 domains: Global, Nurse Communication, Doctor Communication, Discharge Information, Communication About Medicine, and Pain Management.

Data Analysis

Each question was transformed to binary variables of top box versus not top box. Changes for each of the 5 periods (A, B, C, D, E) were tested using logistic regression. The combined changes during the 3 health IT implementation periods (A, B, C) and the combined changes during control periods (D, E) were tested using contrasts in the logistic regression model. Statistical significance is noted when P <.005.

Adjustment for variables that have been suggested to impact HCAHPS responses were done using logistic regression, including age, gender, race, overall self-reported health (excellent, very good, good, fair, poor) and unit category (medical, stepdown/ICU, surgical, women’s and children’s services).²²

RESULTS

Our health system’s Data Information and Analytics Office of Quality and Safety produced 47,057 HCAHPS surveys from 2008 to 2015. After excluding incomplete surveys, and those completed by minors and patients discharged from units that did not meet inclusion criteria, we had 11,728 surveys for analysis. Table 1 shows demographics information for patients whose surveys were included and unit category distributions.

Period A comprised 2947 surveys. The pre-eMAR period had 1638 surveys from May 2008 to October 2008 for medical, surgical, and stepdown/ICU and from October 2013 to February 2014 for women’s and children’s services. The post-eMAR period had 1309 surveys from September 2008 to February 2009 for medical and surgical units, December 2008 to March 2009 for stepdown/ICU, and March 2014 to April 2014 for women’s and children’s services. All surveys in Period A are from Christiana Hospital because eMAR was implemented at Wilmington Hospital prior to our institution using the HCAHPS survey.

Period B comprised 4325 surveys. The pre-eMAR + CPOE period had 2060 surveys from November 2009 to February 2010 for medical, surgical, and stepdown/ICU; from March 2014 to April 2014 for women’s and children’s services; and from September 2009 to December 2009 for all units at Wilmington. The post-eMAR + CPOE period had 2265 surveys from April 2010 to July 2010 for medical, surgical, and stepdown/ICU at Christiana, and from February 2010 to May 2010 for all units at Wilmington. Women’s and children’s services could not be included in this period because the time between eMAR and CPOE addition was too short.

Period C comprised 4446 surveys. The pre-eMAR + CPOE + PN period had 2102 surveys from January 2014 to May 2014 for all units at both Christiana and Wilmington, and from April 2014 to May 2014 for women’s and children’s services. The post-eMAR + CPOE + PN period had 2344 surveys from June 2014 to October 2014 for all units at both hospitals.

Period D (control) comprised 3369 surveys from February 2009 to October 2009 for medical, surgical, and step-down/ICU at Christiana; and from August 2008 to August 2009 at Wilmington. Period E (control) comprised 4367 surveys from August 2010 to December 2013 at Christiana, and from June 2010 to December 2013 at Wilmington.

The percentage of top-box scores increased, but not significantly, for 13 of the 14 questions as units evolved from paper only, pre-eMAR documentation systems in May 2008 to full health IT implementation in October 2014 (Table 1).

Neither eMAR nor CPOE implementation significantly impacted the percentage of top-box responses as seen in Period A and Period B pre-post unadjusted comparisons (Table 2). However, decreases in top-box responses for the Discharge Information question, “Did hospital staff talk with you about whether or not you would have the help you needed when you left the hospital?” and for the Global question of hospital rating, were statistically significant following PN implementation, as seen in Period C pre-post comparisons.

The unadjusted combined impact of health IT, determined by the sum of changes in proportions of top-box responses in periods A, B and C, was mixed across composite domains with no significance (Table 3). During control periods D and E, there were positive changes in top-box responses in all composite domains. Improvements in Discharge Information and Communication About Medicine were statistically significant.

Adjusted analyses for age, gender, race, overall self-reported health, and unit category changed the statistical significance for only 5 comparisons. In Period C the decrease in hospital rating became nonsignificant when adjusted for age, and the decrease in the Discharge Information question became nonsignificant when adjusting for either age or unit category. In Period A, following eMAR introduction the decrease in top-box responses for “How often did the hospital staff do everything they could to help with your pain?” became statistically significant when adjusted for age or unit category.

DISCUSSION

Our results indicated that improvements in HCAHPS scores at our institution were not associated with health IT implementation. By comparing the sum of the proportional changes in percentages of top-box scores following the addition of eMAR, CPOE, and PN to the sum of proportional changes during the control periods, we determined that improvements in HCAHPS scores were associated with the control periods. Periods of health IT implementation showed a mix of positive and negative impacts—none of which were significant. Individually, only PN had significant negative impact on 2 questions, both of which became nonsignificant in adjusted analyses; only 1 question became significant in adjusted analyses. Because the tools, individually and combined, had very little to no impact in both unadjusted and adjusted analyses, our study suggests an absence of a relationship between these electronic point of care tools and patient satisfaction.

There has been speculation that health IT implementation contributing to patient satisfaction is related to streamlined clinician workflow allowing extra time for interactions with patients.^16,23-25 However, our study’s evaluation of tools that have been demonstrated to improve process efficiencies provides no evidence to support this relationship.^14-15 Studies showing positive associations between health IT and patient satisfaction can be limited in their scope and generalizability by evaluating only the Global HCAHPS domain or using measures unique to their institutions.^10,20,24 Additional studies have found more specific positive associations between health IT and the Discharge Information domain suggesting that process improvements in this area may influence patient perceptions of their care.^11,25,26 Our study similarly found statistically significant improvement in Discharge Information, but our methods linked the positive changes to control periods. Utilizing health IT for more efficient workflow—particularly for nurses who are consistently identified as primary drivers of patient satisfaction&mdash;may be a synergistic opportunity to optimize costs, quality, and patient experience, but the relationship requires further exploration.^8,25,27

Much of the existing literature examining health IT and patient satisfaction focuses on unique technologies and satisfaction measures for specific patient populations in predominantly ambulatory settings.^28,29 We believe that this study is the first to examine the impact of multiple types of health IT on all questions of the patient satisfaction domain of VBP at the inpatient unit level, adjusted for patient characteristics. Because we evaluated the impact of off-the-shelf technologies that are commonly used in hospitals nationwide, our results are highly generalizable. Assessing the impact of health IT implementation on HCAHPS scores rather than our own survey questions allows us to evaluate metrics most important to administers, clinicians, and policy makers responsible for ensuring that patients receive high-quality care that is cost-effective for hospitals to deliver in terms of VBP. Including control periods and comparing short pre-post periods allowed us to examine the impact of health IT implementation on patient satisfaction more accurately than in previous studies, where comparisons have been made using patient-reported data collected years apart or at a single point in time.^9-11,25

Limitations

Our study does have limitations. Although HCAHPS scores are a critical performance indicator for VBP that evaluate consistencies in institutional improvement compared with national improvements in patient experience, they do not provide complete information about patient satisfaction. To truly understand the mechanisms of any interactions between health IT and patient satisfaction, future studies may consider additional measures, such as Press Ganey surveys, which also provide valuable information to administrators at our institution. Additionally, although a pre-post design limits conclusions of causality, we provide evidence of a lacking synergistic relationship between 2 costly initiatives that needs to be investigated further.

CONCLUSIONS

Our results suggest that investment in health IT appears to have no impact on patient satisfaction, as measured by HCAHPS scores. Stakeholders should explore further opportunities for synergy as resources continue to be directed toward these 2 expensive and important initiatives.

Author Affiliations: Research Assistant, Value Institute (CJM, RC, JG), Christiana Care Health System (SS, ER), Newark, DE

Source of Funding: None.

Author Disclosures: The authors are employees of Christiana Care Health System, which implemented the technologies described in this manuscript. There is no other relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (CJM, JG, ER); acquisition of data (CJM, JG, SS); analysis and interpretation of data (CJM, RC, JG, SS); drafting of the manuscript (CJM, RC); critical revision of the manuscript for important intellectual content (ER, JG, RC); statistical analysis (RC); administrative, technical, or logistic support (JG, SS); and supervision (JG, ER).

Send Correspondence to: Christine J. Manta, BA, Value Institute, Christiana Care Health System, 4755 Ogletown-Stanton Rd, Newark, DE 19718. E-mail: christinemanta@me.com.

REFERENCES

1. Unofficial recitations of portions of 42 CFR Part 495 and 45 CFR Part 170. HealthIT.gov website. https://www.healthit.gov/sites/default/files/meaningfulusetablesseries1_110112.pdf. Accessed November 15, 2016.

2. CMS. Proposed changes to the hospital inpatient prospective payment systems and fiscal year 2009. Fed Regist. 2008;73(84):23528-23938.

3. Tourison C. National provider call: Hospital Value-Based Purchasing (VBP) Program—fiscal year 2016 overview for beneficiaries, providers and stakeholders. http://docplayer.net/4565882-National-provider-call-hospital-value-based-purchasing-vbp-program.html. Published April 29, 2014. Accessed November 15, 2016.

4. Menachem N, Collum TH. Benefits and drawbacks of electronic health records systems. Risk Manag Healthc Policy. 2011:4;47-55. doi: 10.2147/RMHP.S12985.

5. Silow-Carrroll S, Edwards JN, Rodin D. Using electronic health records to improve quality and efficient: the experiences of leading hospitals. The Commonwealth Fund website. http://www.commonwealthfund.org/~/media/Files/Publications/Issue%20Brief/2012/Jul/1608_SilowCarroll_using_EHRs_improve_quality.pdf. Published July 2012. Accessed November 15, 2016.

6. Bluementhal D. Stimulating the adoption of health information technology. N Engl J Med. 2009;360(15):1477-1479. doi: 10.1056/NEJMp0901592.

7. Long L. Impressing patients while improving HCAHPS. Nurs Manage. 2012; 43(12):32-37. doi: 10.1097/01.NUMA.0000422891.99334.68.

8. Wolosin R, Ayala L, Fulton BR. Nursing care, inpatient satisfaction, and value based purchasing: vital connections. J Nurs Adm. 2012;42(6);321-325. doi: 10.1097/NNA.0b013e318257392b.

9. Jarvis B, Johnson T, Butler P, O’Shaughnessy K, Fullam F, Tran L, Gupta R. Assessing the impact of electronic health records as an enabler of hospital quality and patient satisfaction. Acad Med. 2013;88(10);1471-1477. doi: 10.1097/ACM.0b013e3182a36cab.

10. Restuccia JD, Cohen AB, Horwitt JN, Schwartz M. Hospital implementation of health information technology and quality of care: are they related? BMC Med Inform Decis Mak. 2012;12:109. doi: 10.1186/1472-6947-12-109.

11. Mitchell JP. Electronic healthcare’s relationship with patient satisfaction and communication [published online March 26, 2015]. J Healthc Qual.

12. Hurley AC, Bane A, Fotakis S, et al. Nurses’ satisfaction with medication administration point of care technology. J Nurs Adm. 2007;37(7-8):343-349.

13. Butler J, Speroff T, Arbogast PG, et al. Improved compliance with quality measures at hospital discharge with a computerized physician order entry system. Am Heart J. 2006;151(3);643-653.

14. Swanson Kazley A, Diana ML. Hospital computerized provider order entry adoption and quality: an examination of the United States. Health Care Manage Rev. 2011;36(1):86-94. doi: 10.1097/HMR.0b013e3181c8b1e5.

15. Appari A, Carian EK, Johnson ME, Anthony DL. Medication administration quality and health information technology: a national study of US Hospitals. J Am Med Inform Assoc. 2012;19(3):360-367. doi: 10.1136/amiajnl-2011-000289.

16. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12(5);505-516.

17. Ammenwerth E, Eichstädter R, Haux R, Pohl U, Rebel S, Ziegler S. A randomized evaluation of a computer-based nursing documentation system. Methods Inf Med. 2001;40(2):61-68.

18. Pierpont GL, Thilgen D. Effect of computerized charting on nursing activity in intensive care. Crit Care Med. 1995;23(6):1067-1073.

19. VanDenKerkhof EG, Goldstein DH, Lane J, Rimmer MJ, Van Dijk JP. Using a personal digital assistant enhances gathering of patient data on an acute pain management service: a pilot study. Can J Anaesth. 2003;50(4):368-375.

20. Migdal CW, Namavar AA, Mosley VN, Afsar-Manesh N. Impact of electronic health records on the patient experience in a hospital setting. J Hosp Med. 2014;9(10):627-633. doi: 10.1002/jhm.2240.

21. A step-by-step guide to calculating the patient experience of care domain score in the hospital value-based purchasing FY 2013 actual percentage payment summary report. HCAHPS website. http://hcahpsonline.org/Files/Hospital%20VBP%20Domain%20Score%20Calculation%20Step-by-Step%20Guide_V2.pdf. Published October 26, 2012. Accessed November 15, 2006.

22. Elliot MN, Lehrman WG, Goldstein E, Hambarsoomian K, Beckett MK, Giordano LA. Do hospitals rank differently on HCAHPS for different patient subgroups? Med Care Res Rev. 2010;67(1);56-73. doi: 10.1177/1077558709339066.

23. Hendrich A, Chow MP, Skierczynski BA, Lu Z. A 36-hospital time and motion study: how do medical-surgical nurses spend their time? Perm J. 2008;12(3);25-34.

24. Queenan CC, Angst CM, Devaraj S. Doctors’ orders—if they’re electronic, do they improve patient satisfaction? a complements/substitutes perspective. J Oper Manag. 2011;29:639-649.

25. Kazley AS, Diana ML, Ford EW, Menachemi N. Is electronic health record use associated with patient satisfaction in hospitals? Health Care Manage Rev. 2012;37(1):23-30. doi: 10.1097/HMR.0b013e3182307bd3.

26. Boulding W, Glickman SW, Manary MP. Schulmn KA, Staelin, R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):41-48.

27. Klinkerberg WD, Boslaugh S, Waterman BM, et al. Inpatients’ willingness to recommend: a multilevel analysis. Health Care Manage Rev. 2011;36(4):349-358 . doi: 10.1097/HMR.0b013e3182104e4a.

28. Rozenblum R, Donzé J, Hockey PM, et al. The impact of medical informatics on patient satisfaction: a USA-based literature review. Int J Med Inform. 2013;82(3):141-158. doi: 10.1016/j.ijmedinf.2012.12.008.

29. Irani JS, Middleton JL, Marfatia R, Omana ET, D’Amico F. The use of electronic health records in the exam room and patient satisfaction: a systematic review. J Am Board Fam Med. 2009;22(5):553-562. doi: 10.3122/jabfm.2009.05.080259.

Download PDF: The Impact of Health Information Technologies on Patient Satisfaction