Assessing Electronic Health Record Implementation Challenges Using Item Response Theory

ABSTRACT

Objectives: To assess the importance of commonly identified issues in electronic health record (EHR) implementation using item response theory (IRT).

Study Design: Secondary data from the 2012 American Hospital Association’s Annual Survey Information Technology Supplement were used in the analyses. Results were compared and contrasted with the standard descriptive statistic frequencies that have been used to guide most recommendations made using the same data.

Methods: IRT was used to measure the magnitude of difficulty that particular challenges pose in implementing EHRs that meet federal guidelines for Meaningful Use.

Results: The IRT analyses yielded significantly different results from descriptive statistics in estimating the magnitude of specific EHR implementation challenges. In particular, IRT revealed that “obtaining physician cooperation” and “ongoing costs of maintaining and upgrading systems” were the most challenging implementation features. However, the frequency counts identified “upfront capital costs” and “complexity of meeting Meaningful Use criteria within implementation timeline” as the most challenging implementation features.

Conclusions: For managers and policy makers, having an accurate assessment of EHR implementation challenges is essential to designing effective programs. IRT provides a statistical approach that allows prior studies to be assessed more accurately and future studies to retain the easier-to-use, check-all-that-apply survey structure while gaining valuable information.

Am J Manag Care. 2016;22(12):e409-e415Take-Away Points

Many surveys of hospital administrators have sought to identify barriers to the Meaningful Use of electronic health record (EHR) technology. However, the surveys used often lack the precision to provide a list of barriers ordered from most difficult to least difficult. Instead, surveys tend to report the most common or well-known barriers to adoption.

• The analysis identifies obtaining physician cooperation as the most challenging barrier to EHR adoption.
• Some frequently cited barriers, such as dealing with the complexity of the Meaningful Use requirements are shown to be less challenging.

Implementing an electronic health record (EHR) in a hospital brings many challenges—some bigger than others.¹ Managers often recognize common but readily addressed EHR implementation issues, while more daunting challenges are identified less frequently or later in the process. Prior research has shown that hospitals are slow to implement key EHR features that enhance patient safety and care quality.^2,3 In particular, professional buy-in and implementation costs have been identified as major impediments to EHR use.⁴ The latter issue—implementation cost barriers—is further exacerbated because demonstrating an EHR’s return on investment is difficult.⁵ Determining which EHR implementation issues hinder the widespread “Meaningful Use” of the technology is important.⁶

The problem in identifying and prioritizing EHR implementation issues stems from managerial knowledge gaps arising from a lack of experience with the process. To identify knowledge gaps, professional organizations survey health system leaders about their EHR implementation challenges.^7,8 The American Hospital Association (AHA)’s Annual Survey Information Technology (IT) Supplement (referred to hereafter as “AHA Survey”) asks managers to pick among a list of options for all EHR implementation challenges that their hospital currently faces or anticipates. Although this approach is likely to capture how common an issue is among hospitals, it does not explicitly address how difficult a specific challenge is. Moreover, using response frequencies as a proxy measure for EHR implementation difficulty may lead to inaccurate inferences about the areas where managers should focus their efforts.

The purpose of this paper is 2-fold: first, the challenges hospital administrators face in implementing an EHR were examined using a holistic approach to item difficulty; second, an established psychometric methodology was employed to quantify the relative difficulty levels of various EHR implementation challenges, based on managers’ responses to a “pick all that apply” questionnaire. Item response theory (IRT)⁹ was used to analyze the AHA Survey’s questions related to EHR implementation challenges in meeting federal requirements for Meaningful Use.¹⁰ Results were compared and contrasted with the standard, descriptive statistics that have been widely reported.

Background

A 2014 literature review of hospital EHR implementation studies identified 6 main categories of barriers¹¹: 1) financial capabilities, related to the total cost of EHR system ownership (acquisition and operation); 2) vendor selection, related to integrating ITs into the workplace; 3) IT staffing, related to having employees with prior implementation experience; 4) health system culture, related to organizational collaboration challenges; 5) organizational complexity, related to the bureaucracy and change management capacity of the hospital; and 6) clinical professional barriers, related to the ITs’ impact on providers’ abilities to deliver care. Many dimensions play roles in the EHR implementation process, but 2 themes emerge consistently across the 6 domains identified in the prior research: experience with EHR implementation and the complexity of managing the particular barrier.

The common factors that affect a barrier’s potential importance can be put into a 2-dimensional framework of barrier complexity and experience. The experience dimension can be translated into where a health system is in its tacit knowledge about EHR implementation barriers. IRT is specifically designed to order respondents’ relative knowledge about specific content. The model is presented in the Figure.

The most complex managerial challenge when implementing an EHR is gaining clinician buy-in.¹² By both social and legal conventions, physicians have the professional autonomy to make clinical decisions.¹³ The introduction of an EHR with predefined order sets, automated reminders, and decision support is perceived by many physicians as a direct attempt to diminish their professional autonomy. In addition, many physicians are not directly employed by health systems, further reducing their organizational commitment. Taken together, the resistance to managerial oversight and lack of organizational identification make gaining physician buy-in a major issue. Nurses have a lesser degree of professional autonomy and are typically employed by the hospital; hence, they tend to be less of a management challenge than physicians when it comes to EHR implementation and use. Nevertheless, the physician—nurse relationship is critical to EHR implementation success.

In the Figure, the clinical staff response (items 3 and 4) begins during the EHR adoption phase. Normatively, doctors and nurses should be brought in at the earliest stage. In practice, they tend to be included after the decision to adopt the EHR has been made during vendor selection.¹⁴ The importance of clinical staff engagement is an ongoing management challenge, as the degree of clinical functionality in the EHR and required care process documentation is ever-increasing. Therefore, management experience with implementing more sophisticated functionalities is a learned capability that continues ad infinitum.

The financial capacity (items 1 and 2) of the health system to support the EHR is also an ongoing challenge. The initial cost of purchasing an EHR is considerable, but knowable; however, it is the ongoing cost of operating the systems that many managers underestimate.¹⁵ Losses in productivity have a concomitant impact on revenues,¹⁶ and the double impact of unexpectedly high ownership costs and reduced revenue cause managers significant challenges. Managers typically become fully cognizant of these total cost of ownership (TCO) issues only after they begin the implementation process; the start date in the Figure reflects this fact.

Similar to the experience-informed learning that occurs with the recognition of TCO issues, the additional levels of organizational complexity (item 10) that are part-and-parcel of an EHR implementation become clearer over time.¹⁷ Post-EHR implementation, health systems have to roll out additional functionalities to meet organizational needs and comply with ever-increasing regulations. Hence, this construct also encompasses the entire EHR lifecycle.

Organizations’ cultures (item 9) are also impacted by the introduction of new technologies and the workflow changes that come with them.¹⁴ Depending on when organizational members are engaged in the EHR adoption and implementation processes, the impact on organizational culture can start very early on and continue indefinitely. One means of making the cultural shift a positive one is to engage staff members during the vendor selection phase to promote buy-in.

Vendor selection and operations are closely related to issues of EHR product certification (item 6), ensuring system security against privacy breaches (item 5), and ongoing system support (item 7). A major criterion in vendor selection is to reduce the management burden related to these items.^18,19 Therefore, they appear lower on the management complexity axis. Despite being lower on the complexity dimension, ongoing vendor relations mean that it is an enduring management activity.

The lowest EHR implementation challenge construct, in terms of management complexity, is information technology staffing (item 8).¹⁴ The employees dedicated to supporting the technology are easier to manage because, unlike that of clinical staff, their job description explicitly includes following the direction of leadership. Nevertheless, managers often underestimate the number of IT staff needed to support an EHR.²⁰ Therefore, the management challenge continues well into the implementation phase until the organization reaches some level of homeostasis with respect to the EHR.

The EHR adoption/implementation challenges used in the AHA Survey are not a perfect fit to the constructs listed in the literature review of Boonstra and colleagues.¹¹ Nevertheless, the AHA Survey does touch on each of the major domains in some fashion and serves as a good starting point from which to explore EHR implementation challenges from a health system perspective.

METHODS

The AHA Survey (2013)²⁰ was funded by the Office of the National Coordinator for Health Information Technology (ONC) and sent to 5756 US hospitals (including those not members of the AHA). The survey had a 59% response rate for a sample of 3396. The survey question, “What is (are)/would be the primary challenge(s) in implementing an EHR system that meets the federal requirements for meaningful use? (Please check all that apply)” was analyzed. The item offered 10 structured responses, including: 1) upfront capital costs/lack of access to capital to install systems, 2) ongoing cost of maintaining and upgrading systems, 3) obtaining physician cooperation, 4) obtaining other staff cooperation, 5) concerns about security or liability for privacy breaches, 6) uncertainty about certification requirements, 7) limited vendor capacity, 8) lack of adequate IT personnel in hospital to support implementation/maintenance, 9) challenge/complexity of meeting all Meaningful Use criteria within implementation timeframe, and 10) complexity associated with coordinating decisions with system-level leadership. These responses were used in the IRT phase.

IRT is a collection of modeling techniques for analyzing item-level data obtained to measure variation between respondents. Unlike regression methods, IRT does not require a dependent variable against which to parse variance; instead, it compares the items against one another to establish the pattern of responses from a sample of survey or test takers. In addition, by evaluating the confidence intervals (CIs) of the item difficulty measurements, it is possible to assess if there are potentially missing items in the scale. When sorted by item difficulty, each CI should overlap with the next item in the list. CIs that do not overlap indicate that there may be another item that would improve the classification of survey respondents. The CI comparison can be used to detect potential gaps in an assessment scale’s items.

One important assumption of IRT is that the scale being assessed is unidimensional. To test this assumption, a factor analysis is performed in the items to ensure they are measuring a common construct (see eAppendix A [eAppendices available at www.ajmc.com] for a description of the factor analysis). Following that procedure, the IRT itself can be undertaken to assess both item and respondent characteristics (see eAppendix B for a description of the IRT).

RESULTS

Tables 1 and 2 display the characteristics of the hospitals in the sample and the correlations among the responses to the 10-item questionnaire, asking: “What is (are, or would be) the primary challenge(s) in implementing an EHR/EHR system that meets the federal requirements for Meaningful Use? (Please check all that apply).” These items were factor-analyzed using the principal factors and unrotated modeling specifications. The amount of variance explained by the first factor was 73.9%, and the factor loading for each item was ≥0.460. The ratio of the Eigen value for the first factor (3.184) to the second (0.839) was 3.8. Overall, these results indicated that the responses form a single construct, with varying levels of organizational challenge being the distinguishing feature.

The results of the IRT analyses are presented in Table 3, ranked by item difficulty. The “obtaining physician cooperation is a primary challenge” response had the highest item difficulty rating (b = 0.331) and good discrimination (a = 1.039), but was the fourth most frequently identified challenge (43.12% of responses). The 2 challenges related to EHR financing— ongoing costs of maintaining and upgrading systems” (b = 0.286) and “upfront capital costs/lack of access to capital to install systems” (b = 0.267)—were the second and third most difficult items, respectively, based on the IRT analysis. However, these 2 items were the first (upfront capital cost) and fifth (ongoing cost) most common responses in the descriptive statistics using frequency counts. In addition to being a major challenge, the “ongoing cost” response also had the highest discrimination of any item (a = 1.696). IRT analysis re-ordered the challenges relative to the response frequency approach for assessing item importance.

The fourth and fifth items based on the difficulty measure related to technical aspects of implementing an EHR. “Challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” was the fourth most difficult challenge (b = —0.198). This item also had lower discrimination (a = 0.993). The “lack of adequate IT personnel in the hospital to support implementation/maintenance” item was the next most difficult (b = —0.303) and had a slightly higher discrimination index (a = 1.041) than the “…meeting all Meaningful Use criteria…” item. “Obtaining other staff cooperation” item (b = —1.150) did not group with the other personnel challenge related to gaining physician buy-in. However, it did have relatively high discrimination (a = 1.331) compared with the physician item (a = 1.039). The last item with good discrimination was “concerns about security or liability for privacy breaches” (a = 1.303). The remaining items had both low difficulty and discrimination measures, indicating that these issues are not impacting EHR deployment.

Examining the item difficulty CIs, there are 4 potential gaps in the scale used to measure EHR implementation challenges. The first gap appears between the items “upfront capital costs/lack of access to capital to install systems” and “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline.” There are 4 more potential gaps noted in Table 3. Potential missing items are discussed in the next section.

DISCUSSION

The findings have both policy and methodological implications. “Obtaining physician cooperation” is the biggest challenge to implementing an EHR based on the IRT analysis. However, the Meaningful Use program provides no explicit rewards to doctors in the hospital environment. In the ambulatory setting, individual physicians are targeted for rewards if they are early adopters, and penalties if they procrastinate about EHR implementation. The rationale for the rewards in the ambulatory environment was to compensate doctors for their productivity levels likely suffering in the near term after implementation. The effect of an EHR implementation in the hospital is similar for doctors, but no offsetting compensation is made available. Hence, increased resistance from physicians is likely to impact hospital managers charged with EHR implementation.

The IRT results also offer methodological insights. “Obtaining physician cooperation” was the most challenging EHR implementation barrier in the IRT results. These findings are similar to those of other studies determining that many physicians do not routinely use EHRs to improve care quality.²¹ The “upfront capital cost” response was the most frequently identified EHR implementation challenge, but it was only the third most difficult measure based on an IRT analysis. The switching of positions from the counts listing to the IRT is an indication that descriptive statistics alone do not provide a complete or accurate assessment. The “ongoing…” and “upfront…” cost measures were also very challenging for hospitals.^22,23 The Item Discrimination Index is highest for the challenge related to “ongoing costs.” Therefore, this concern is the item most likely to differentiate respondents.

Although inferential statistics in general and logistic regression, in particular, have the potential to highlight differences in hospitals’ EHR implementation challenges, they also have limitations. Logistic regression models require a dependent variable to assess an item’s relative importance. For example, the study by Adler-Milstein and colleagues,²⁴ which analyzed the same battery of EHR implementation challenges using Meaningful Use attestation as the dependent variable, found that cost measures, rather than physician cooperation, were the biggest challenge. The difference arises because EHR implementation challenge items are not being compared with one another; instead, variance is being allocated across them in a zero-sum fashion.

In contrast, IRT identifies that the “challenge/complexity of meeting Meaningful Use within specified timelines” was a major challenge regardless of other organizational characteristics. In the regression analysis, both groups believed the issue to be important, but no statistical difference was detected, so the item was not considered to be significant in the discussion. Moreover, the relatively low Item Discrimination Index for the “complexity of meeting Meaningful Use within specified timelines” measure indicates that the variable will not differentiate hospitals on other characteristics. Therefore, this is a relatively difficult challenge common to most facilities regardless of other characteristics.

IRT identifies the relative placement of EHR implementation barriers by utilizing response patterns within individual hospitals to inform item hierarchical ordering. Although the Meaningful Use attainment measure can be used as proxy for EHR implementation difficulty, it did not assess the underlying organizational capabilities and resources that make some barriers more difficult than others for a particular hospital. In that respect, the regression analysis approach is underspecified, given that omitted variables are likely. Therefore, having a means of assessing organizational barriers/challenges that does not require other constructs to be fully identified allows for alternative explanations as to why some EHR implementation challenges are greater than others.

There are 3 interpretations for why the IRT produced different results than either the frequency count or logistic regression approaches used in the other studies. From a learning perspective, those facilities with more experience will have a better understanding of different challenges’ operational impact. Therefore, those that have achieved Meaningful Use Stage 2 are more likely to prioritize challenges relying on the benefit of hindsight. A second interpretation relates to the innovation adoption-implementation lifecycle. Hospitals further into the process will be facing different challenges than those just beginning. Specifically, “obtain physician buy-in” is likely to be easier when the EHR is an abstract idea and more difficult when the reality of using the systems is encountered. Organizational capacity provides a third view for considering EHR adoption-implementation challenges. Under this interpretation, hospitals with better management will identify the more intractable EHR adoption-implementation challenges with greater frequency.^25,26 Through any of these perspectives, the “obtaining physician cooperation” challenge would be the furthest along on the learning curve.

Another important feature of the IRT analysis was the identification of potential gaps in the EHR implementation challenge scale as it was fielded. The potential gap between “upfront capital costs/lack of access to capital to install systems” and “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” is at the highest item difficulty level. The “upfront capital…” item is clearly written, but the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” response has 2 potential issues.

First, the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” item is poorly phrased. The part of the question using the phrase “challenge/complexity” lacks specificity. What aspect of the implementation is “complex”? EHR implementations require health systems to “build” many of the user interfaces, create order sets, and link to other information systems. Respondents may be responding to this aspect of EHR implementation complexity. Alternatively, EHR implementations require significant staff training evolutions that are different than the concepts captured in the “obtaining staff/physician cooperation” items. There is no mention of staff training in the scale as it was put to the respondents.

A second issue is that the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” item has 2 questions embedded within it. The “challenge/complexity” phrase is one element and the EHR “…implementation timeline” is a second distinct concept. Creating 1 or 2 time-related items would be an improvement. One item would address meeting the timelines set by the organization for implementing the EHR. The second would assess the health system’s ability to meet the Meaningful Use target timelines. Given health systems could join the Meaningful Use program at different points without penalty; the 2 timelines are not necessarily the same.

Disaggregating and more accurately specifying the concepts in the “challenge/complexity of meeting all Meaningful Use criteria within implementation timeline” question could also fill in the gaps that appear in the scale. Per the earlier discussion, breaking out the EHR “build” features could help fill the gap above the “lack of adequate IT personnel in the hospital to support implementation/maintenance” and “obtaining other staff cooperation” items. One possible solution is to look to the survey itself. The survey asks respondents to list “other barriers to EHR implementation.” Exploring those might yield potential items to fill in the gaps identified in the scale as it is currently employed. The potential “staff training” and “EHR development” items discussed above were drawn from that list.

Having EHR implementation challenge scales that are lacking potential items is an important issue for policy makers. The ONC funds research into areas that have been identified as impediments to effective EHR use. Lacking accurate and complete feedback from health system administrators diminishes the ONC’s ability to target EHR implementation challenges.

Similar to having incomplete information, having the EHR implementation challenges incorrectly sorted may lead to misguided investment strategies. As the IRT analysis demonstrates, “obtaining physician cooperation” is the most difficult item. Changing the culture of the medical community is a far more difficult undertaking than funding EHR purchases and implementations. To alter the physician community’s beliefs, it may be necessary to start while physicians are in medical school. Such a transformation would take a generation to implement.²⁷ Ongoing organizational training may help to improve EHR acceptance.

CONCLUSIONS

Both EHR implementation and methodological lessons can be drawn. With respect to EHR implementation, among the challenges evaluated, “obtaining physician cooperation” is the most difficult part of the process. For those hospitals in the initial stages of EHR implementation, the importance of gaining physician buy-in may not be apparent. Similarly, the importance of managing ongoing ownership and operating costs for EHR systems may not be fully understood by early adopters compared with late adopters. Public policies that provide rewards for implementing more sophisticated EHR functionalities can be tailored to ameliorate these challenges. The IRT analysis also helps quantify the relative difficulty of each challenge that health systems face.

Many surveys of healthcare providers and managers employ multiple selection items (eg, “pick all responses that apply”) to reduce respondent burden. To interpret which responses are most significant, researchers typically employ either simple statistics, such as frequency counts to order items, or use inferential approaches, like linear regression, to measure importance in relationship to some other factor. Neither approach is ideal. The frequency counts do not provide an accurate picture of how difficult a particular response is vis-à-vis other options. Frequency is about how often something is encountered, not necessarily how important a particular challenge is to the implementation success. The regression approach is limited to addressing the relationship of survey responses to a particular variable of interest rather than allowing them to be context-free and self-referent.

The IRT approach addresses the 2 aforementioned shortcomings and provides an effective complement to those analytic strategies. For those designing surveys, including the IRT approach as part of the analytic strategy should be established a priori. For those analyzing secondary data with "pick all that apply" options, using IRT in conjunction with other techniques could yield a more complete picture of the phenomena.Author Affiliations: Health Policy and Management, Johns Hopkins Bloomberg School of Public Health (KSC, HK, MP, EWF), Baltimore, MD.

Source of Funding: None.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (KSC, EWF); acquisition of data (EWF); analysis and interpretation of data (KSC, MP); drafting of the manuscript (KSC, HK, MP, EWF); critical revision of the manuscript for important intellectual content (HK, EWF); statistical analysis (MP).

Address Correspondence to: Eric W. Ford, MPH, PhD, 624 N. Broadway, Ste 533, Baltimore, MD 21205. E-mail: ewford@jhu.edu.

REFERENCES

1. Ancker JS, Brenner S, Richardson JE, Silver M, Kaushal R. Trends in public perceptions of electronic health records during early years of Meaningful Use. Am J Manag Care. 2015;21(8):e487-e493.

2. Ford EW, McAlearney AS, Phillips MT, Menachemi N, Rudolph B. Predicting computerized physician order entry system adoption in US hospitals: can the federal mandate be met? Int J Med Inform. 2008;77(8):539-545.

3. Rahimi B, Timpka T, Vimarlund V, Uppugunduri S, Svensson M. Organization-wide adoption of computerized provider order entry systems: a study based on diffusion of innovations theory. BMC Med Inform Decis Mak. 2009;9:52. doi: 10.1186/1472-6947-9-52.

4. Coustasse A, Shaffer J, Conley D, Coliflower J, Deslich S, Sikula A. Computer Physician Order Entry (CPOE): benefits and concerns—a status report. JITR. 2013;6(3):16-31. doi: 10.4018/jitr.2013070102.

5. Huerta TR, Thompson MA, Ford EW, Ford WF. Electronic health record implementation and hospitals’ total factor productivity. Decision Support Systems. 2013;55(2):450-458. doi: 10.1016/j.dss.2012.10.004.

6. Ford EW, Menachemi N, Huerta TR, Yu F. Hospital IT adoption strategies associated with implementation success: implications for achieving Meaningful Use. J Healthc Manag. 2010;55(3):175-188; discussion 188-189.

7. Himmelstein DU, Wright A, Woolhandler S. Hospital computing and the costs and quality of care: a national study. Am J Med. 2010;123(1):40-46. doi: 10.1016/j.amjmed.2009.09.004.

8. Teufel RJ 2nd, Kazley AS, Andrews AL, Ebeling MD, Basco WT Jr. Electronic medical record adoption in hospitals that care for children. Acad Pediatr. 2013;13(3):259-263. doi: 10.1016/j.acap.2013.01.010.

9. Embretson SE, Reise SP. Item Response Theory. Hove, UK: Psychology Press; 2013.

10. Diana ML, Harle CA, Huerta TR, Ford EW, Menachemi N. Hospital characteristics associated with achievement of Meaningful Use. J Healthc Manag. 2014;59(4):272-284; discussion 285-286.

11. Boonstra A, Versluis A, Vos JF. Implementing electronic health records in hospitals: a systematic literature review. BMC Health Serv Res. 2014;14:370. doi: 10.1186/1472-6963-14-370.

12. McAlearney AS, Chisolm DJ, Schweikhart S, Medow MA, Kelleher K. The story behind the story: physician skepticism about relying on clinical information technologies to reduce medical errors. Int J Med Inform. 2007;76(11-12):836-842.

13. McKay PS. Interdependent decision making: redefining professional autonomy. Nurs Admin Q. 1983;7(4):21-30.

14. Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. doi: 10.1186/1472-6963-10-231.

15. Slight SP, Quinn C, Avery AJ, Bates DW, Sheikh A. A qualitative study identifying the cost categories associated with electronic health record implementation in the UK. J Am Med Inform Assoc. 2014;21(e2):e226-e231. doi: 10.1136/amiajnl-2013-002404.

16. Brooks R, Grotz C. Implementation of electronic medical records: how healthcare providers are managing the challenges of going digital. Journal of Business & Economics Research. 2010;8(6):73-84.

17. Cresswell K, Sheikh A. Organizational issues in the implementation and adoption of health information technology innovations: an interpretative review. Int J Med Inform. 2013;82(5):e73-e86. doi: 10.1016/j.ijmedinf.2012.10.007.

18. Cherry BJ, Ford EW, Peterson LT. Experiences with electronic health records: early adopters in long-term care facilities. Health Care Manage Rev. 2011;36(3):265-274. doi: 10.1097/HMR.0b013e31820e110f.

19. Vest JR, Yoon J, Bossak BH. Changes to the electronic health records market in light of health information technology certification and Meaningful Use. J Am Med Inform Assoc. 2013;20(2):227-232. doi: 10.1136/amiajnl-2011-000769.

20. AHA Annual Survey database. AHA Data Viewer website. http://www.ahadataviewer.com/book-cd-products/AHA-Survey/. Accessed September 30, 2013.

21. Gellert GA, Ramirez R, Webster SL. The rise of the medical scribe industry: implications for the advancement of electronic health records. JAMA. 2015;313(13):1315-1316. doi: 10.1001/jama.2014.17128.

22. King J, Patel V, Jamoom E, DesRoches C. The role of health IT and delivery system reform in facilitating advanced care delivery. Am J Manag Care. 2016;22(4):258-265.

23. Hay JW. Hospital cost drivers: an evaluation of 1998-2001 state-level data. Am J Manag Care. 2003;9(spec no 1):SP13-SP24.

24. Adler-Milstein J, DesRoches CM, Kralovec P, et al. Electronic health record adoption in US hospitals: progress continues, but challenges persist. Health Aff (Millwood). 2015;34(12):2174-2180. doi: 10.1377/hlthaff. 2015.0992.

25. Leung RC. Health information technology and dynamic capabilities. Health Care Manage Rev. 2012;37(1):43-53. doi: 10.1097/HMR.0b013e31823c9b55.

26. Smith AL, Bradley RV, Bichescu BC, Tremblay MC. IT governance characteristics, electronic medical records sophistication, and financial performance in US hospitals: an empirical investigation. Decision Sciences. 2013;44(3):483-516. doi: 10.1111/deci.12019.

27. Ford EW, Menachemi N, Phillips MT. Predicting the adoption of electronic health records by physicians: when will health care be paperless? J Am Med Inform Assoc. 2006;13(1):106-112. 

Download PDF: Assessing Electronic Health Record Implementation Challenges Using Item Response Theory