What Determines Successful Implementation of Inpatient Information Technology Systems?

Objectives:

To identify the factors and strategies that were associated with successful implementation of hospital-based information technology (IT) systems in US Department of Veterans Affairs (VA) hospitals, and how these might apply to other hospitals.

Study Design:

Qualitative analysis of 118 interviews conducted at 7 VA hospitals. The study focused on the inpatient setting, where nurses are the main patient-care providers; thus, the research emphasized the impact of Computerized Patient Record System and Bar Code Medication Administration on nurses. Hospitals were selected to represent a range of IT implementation dates, facility sizes, and geography. The subjects included nurses, pharmacists, physicians, IT staff, and managers. Interviews were guided by a semi-structured interview protocol, and a thematic analysis was conducted, with initial codes drawn from the content of the interview guides. Additional themes were proposed as the coding was conducted.

Results:

Five broad themes arose as factors which affected the process and success of implementation: (1) organizational stability and implementation team leadership, (2) implementation timelines, (3) equipment availability and reliability, (4) staff training, and (5) changes in work fl ow.

Conclusions:

Overall IT implementation success in the VA depended on: (1) whether there was support for change from both leaders and staff, (2) development of a gradual and flexible implementation approach, (3) allocation of adequate resources for equipment and infrastructure, hands-on support, and deployment of additional staff, and (4) how the implementation team planned for setbacks, and continued the process to achieve success. Problems that developed in the early stages of implementation tended to become persistent, and poor implementation can lead to patient harm.

(Am J Manag Care. 2012;18(3):157-162)The information technology (IT) implementation experience of a large diverse system, like the US Department of Veterans Affairs (VA), which has a mix of rural, urban, teaching, and community hospitals, provides valuable information about the issues surrounding implementation for hospitals in the private sector. IT implementation success in the VA depended on:

• Whether there was support for change from both leaders and staff.

•  Development of a gradual and fl exible implementation approach.

•  Allocation of adequate resources for equipment and infrastructure, hands-on support, and deployment of additional staff.

•  How the implementation team planned for setbacks, and continued the process to achieve success.

Computerized patient records and bar-code medication systems continue to gain favor in healthcare.^1-5 In the hospital setting, 2 of the most important technologies are computerized patient records and medication administration systems.^1,5,6 These systems are expected to bring about improvements in patient safety, work processes, and staff morale,6-11 and reduce rates of medication errors. ^12-14 Research to date has reached mixed conclusions as to whether such improvements have occurred.^10,11,15-24 Several studies demonstrate that the implementation process for hospital health information technology (HIT) is important to determining overall success.^24-28

The US Department of Veterans Affairs (VA) has made one of the largest investments in HIT in the United States, implementing a fully integrated system across its 162 hospitals nationwide.^28,29 The VA is the nation’s largest integrated healthcare system, with more than 7.2 million veterans enrolled for health services.³⁰ Their development of an HIT system can be traced to the 1970s. The VA’s Computerized Patient Record System (CPRS) and Bar Code Medication Administration (BCMA) are central to inpatient care.^28,29 CPRS was phased in over a decade starting in the early 1990s, and consists of a comprehensive electronic patient medical record with computerized physician ordering, covering both outpatient and inpatient services. By 2002, all VA hospitals had implemented CPRS; the vast majority had implemented this system by 2000. BCMA, on the other hand, was implemented over a much shorter time period, with VA headquarters requiring implementation 1 year after the software became available.²⁹ This system created a computerized pharmacy ordering, distribution, and administration system for use in the inpatient setting. Bedside scanning of patient identifi cation wristbands and medications was the key component of the system, providing validation that each medication matched the orders for each patient.

This paper reports the findings from the qualitative component of a national, retrospective, mixed-methods study of the implementation of CPRS and BCMA. There were no prospective systemwide or multisite evaluations of the implementation of CPRS or BCMA. The study focused on the inpatient setting, where nurses are the main patient care providers; thus, the research emphasized the impact of CPRS and BCMA on nurses. The qualitative portion of the study focused on 4 issues: (1) understanding the approaches used to implement the VA’s HIT systems, (2) identifying factors that affected the process of implementation, (3) ascertaining the issues that determined success of implementation, and (4) understanding what nursing staff and leaders believe are the current strengths and weaknesses of CPRS and BCMA. The experience of a large diverse system like the VA, which has a mix of rural, urban, teaching, and community hospitals, provides valuable information about the issues surrounding information system implementation for hospitals in the private sector.

METHODS

We conducted site visits at 7 VA hospitals selected to represent a range of implementation timelines, geography, and staff characteristics. All site visits were conducted after receiving approval from each facility’s Institutional Review Board (IRB), as well as the IRBs of the University of California, San Francisco, Stanford University, and the Boston VA Healthcare System.

eAppendix A

In advance of site visits, an Advisory Committee was assembled, consisting of VA medical, pharmacy, and nursing leaders, as well as representatives of the VA headquarters. The committee was asked to identify issues and themes that they anticipated to be important to understanding the impact of the implementation and use of the VA’s HIT systems on nurses. A semi-structured interview guide was developed after this meeting, based on a review of the literature on technology implementation and the effects of IT systems, and the committee’s suggestions. The committee members provided feedback regarding the content of the guide. This guide was used both to conduct the interviews and provide themes for the initial coding of the data. The guide is available in at www.ajmc.com.

Site selection was based on a unique Web-based survey of VA facilities documenting when each major component of CPRS and BCMA was implemented, VA staff satisfaction survey data, facility-level staff turnover data, geography, and the level of care provided by each VA hospital. We initially identifi ed 27 facilities that represented a range of characteristics, and then consulted with the Advisory Committee and the VA Headquarters Nursing Offi ce to recruit a final set of 7 facilities.

eAppendix B

A total of 118 interviews were conducted over a 15-month period (June 2006 through September 2007) with nurses, pharmacists, nurse managers, information technology staff, and senior management. At most sites, physicians were not included because the study’s focus was inpatient IT systems, which are primarily used by nurses. Interview subjects were selected by site coordinators, who were recommended by members of the Advisory Committee and/or the VA Headquarters Nursing Offi ce, and were employees of the VA facilities. The principal investigator (PI) provided the site coordinators with a detailed list of the job classifications of the people to be interviewed (see at www.ajmc.com for categorizations and counts of the interview subjects). Interviews lasted 30 to 60 minutes and were held in private meeting and conference rooms, using the semi-structured interview protocol. Notes were taken by the investigators and were entered into ATLAS.ti to facilitate analysis after the site visit was complete.

A thematic analysis was conducted with initial codes drawn from the content of the interview guides. Additional themes were proposed as the coding was conducted and were added if there was concurrence among members of the research team. All coding was completed by the PI, while other members of the research team reviewed codes of 1 to 2 interviews per site. Investigators who visited each site reviewed the themes identifi ed from that site’s interviews.

RESULTS

Staff and managers faced numerous challenges while CPRS and BCMA were being implemented. As one VA employee stated, these IT systems changed “how we organize, document, and communicate regarding patient care,” changes that touched all aspects of healthcare delivery. The all-encompassing scope of IT implementation led to what another staff member described as “a big culture change.” This process of change can be tumultuous. A number of staff and managers used terms such as “frightened,” “nervous,” and “scary” to describe how nurses felt about CPRS and BCMA at first; one manager said there was “reluctance by many staff members to truly embrace the system, see the opportunities.” Another manager said, “a lot of clinical staff thought if they didn’t use CPRS it would go away.” The most resistant nurses and physicians reportedly left the VA through retirement or turnover.

Managers and staff agreed that employees who were not “tech savvy” had more diffi culty adapting to HIT; one manager noted that “less computer-skilled nurses struggle more.” Another manager observed that “some staff didn’t know how to use computers or mice,” which hampered their training. Many managers also noted that clinicians “tend to have weak keyboarding skills” because their jobs did not normally require much typing. At some sites, managers reported that older nurses were less likely to be comfortable with computers.

Five themes arose as important infl uencers of the process and success of implementation: (1) organizational stability and implementation team leadership, (2) implementation timelines, (3) equipment availability and reliability, (4) staff training, and (5) changes in work fl ow.

Organizational Stability and Implementation Team Leadership

The broader culture of the facility fundamentally affected the success at each site. One VA employee said, in a “large organizational deployment, [the organization] needs [to be] very stable and fault-tolerant… In planning, [you] have to have good leadership to articulate the nursing position.” Sites with unsupportive management teams, or where staff did not respect the ability of management, faced more challenges both during and after implementation of BCMA and CPRS. Managers often set the tone for how their departments or staff accepted HIT. As one leader noted, “if nurse managers were in support, you could get a lot farther.”

The teams that led the CPRS and BCMA implementations were crucial to success. For CPRS, the medical staff was usually viewed as most important, at least in part because CPRS has a central role in the outpatient setting. At every site we visited, staff and leadership recognized that nursing involvement in the BCMA implementation was crucial, and that pharmacy and IT staff had to be partners in the process. As one manager said, “success is all about teamwork.”

At some sites, the responsibility of implementation was not accepted by all parts of the team. For example, at 2 sites, pharmacists thought BCMA was primarily a nursing program, and thus “meetings didn’t go anywhere.” These sites had more diffi culty, because problems were not addressed in a cohesive way. Several staff members and managers noted that implementation of both systems required involvement of end users; sites that had physicians and staff nurses in visible roles during implementation achieved buy-in from other care providers more easily.

Implementation Timelines

Flexibility in implementation helped staff adapt to CPRS and BCMA, regardless of how the implementation was specifically planned. CPRS evolved over time, with its antecedents developed in the early 1980s. There was no implementation deadline for CPRS, and most VA facilities used a gradual unitby- unit rollout to implement CPRS. One nurse noted that, at her site, “not much prep work was done before implementation, but that was somewhat okay because it was phased in.”

In contrast, the implementation of BCMA was mandated by VA headquarters with a June 30, 2000, deadline, which was only 1 year after the software was made available. The short timeline for BCMA implementation made staff feel pressured to use a system that they perceived had fl aws. One staff member argued “the software wasn’t ready, and the hardware had not been researched,” making it a “big mistake” to require implementation at that time. Some sites introduced BCMA in departments with relatively stable patient populations, such as in the long-term care ward. Other sites selected psychiatric units for the initial implementation, because patients go to a specifi c desk to receive their medications. In these wards, mobile carts and wireless scanners were not needed to operate BCMA, thus reducing the complexity of the implementation. These approaches seemed to work well because they gave the implementation team the opportunity to gain experience with the system in a controlled environment.

Equipment Availability and Reliability

For both CPRS and BCMA, VA facilities were allowed to select their own hardware. With CPRS, a variety of confi gurations could be used, with mobile computer carts, a centralized station, or bedside computers being acceptable options. Adequate access to and reliability of hardware and computer networks were essential during the implementation of CPRS. Many of the people we interviewed reported problems in this area, and staff spent much time and energy dealing with computer issues. As one interviewee said, “there were also system issues at fi rst; not enough laptops, problems with wireless access, inadequate server. CPRS itself wasn’t causing an increase in needed RN hours. But, the IT problems were causing problems that increased RN hours.” The question of how many computers would be adequate for use of CPRS does not have a clear answer. During the implementation period, when staff members were unfamiliar with the system and needed more time to become familiar with it, perceived shortages of computers were frustrating to them.

Network problems plagued some facilities. One person observed that, at their site, “when the system was fi rst rolled out, there wasn’t enough wireless bandwidth, and the network coverage was spotty in some patient-care areas.” This led to the system being perceived as “very slow.” Some interviewees attributed speed problems to the computer servers not having enough capacity or being defi cient. Network and server problems were reported that would disconnect users or “would lock up or kick people out, so notes were lost.” Over time, these problems were resolved because demand on the system declined as the staff became more effi cient with its use, as well as because hardware and networks were upgraded.

Hardware and network problems were even more signifi - cant for BCMA. Sites needed mobile carts that could carry medications, scanners, and computers. The computers had to connect to the hospital’s server, thus hospitals had to use wireless networks. They also needed to purchase bar-coded wristbands for patients and develop a method to ensure that all medications in the pharmacy were bar-coded. There was no guidance from headquarters about equipment, so every site scrambled to do its own research and decision making.

During the first months of BCMA implementation, every site had problems with medications being miscoded, items not scanning, and empty unit-dose packages being delivered to the wards. Sites also had problems with batteries not holding charges or being recharged regularly; mobile carts were large and diffi cult to move at some sites; and network problems plagued some facilities. Nurses complained that they had “a computer that is buggy,” the “computer would just kick you out,” the “machine will crash in the middle of a medication pass,” or that “one machine’s keys were too sensitive.” Most sites reported diffi culties with patient wristbands; at many sites, nurses would cut the bands off patients’ wrists to scan them, or scan from extra wristbands in the patients’ charts. At some VA hospitals, staff were implicitly or explicitly permitted to use various workarounds with BCMA, such as typing codes directly into the computer or doing all scanning after medication administration. One manager noted, “staff had to be given leeway to deal with things that did not work.” However, such circumvention has continued at some sites. During a few site visits, we observed nurses typing codes, rather than scanning them, and we noted bar codes taped to computers.

Staff Training

National training was provided for leaders of the facility implementation teams; these leaders were responsible for developing a local training program. Most VA facilities began the training process in classrooms. At several sites, managers reported diffi culty getting staff into training; it was not clear if staff were resistant to attending training, or too busy to leave their units when their classes were scheduled. After classroom training, some sites provided support during CPRS implementation through a “help desk.” Many sites had support staff available in patient care units to offer one-on-one hands-on assistance to staff. Most of the leaders and staff we interviewed observed that “learning by doing” was important.

For BCMA, most sites used a combination of classroom training and one-on-one training during medication delivery. One manager said, “we didn’t let staff do it themselves for at least 5 days.” At more than 1 hospital, the training team administered medication with staff for a fi xed number of medication administration cycles, or until each nurse was comfortable with the system. Most sites had 24-hour support available for some period of time after BCMA was implemented, and this was universally noted as “essential.” At some sites, support staff was not initially on call 24 hours per day, but this changed when it was recognized how important such support was. One interviewee noted that “everyone is less stressed when 24-hour help is provided.”

Changes in Work Flow

The use of CPRS and BCMA interrupted the fl ow of care for many physicians and nurses. As one manager noted, “Most RNs had a hard time charting in the computer when you do things, how you do things, and a lot of people couldn’t type.” Some issues were related to work fl ow; for example, a staff member noted, a “doctor might order meds but won’t have finished progress notes,” thus forgetting to complete the record. Nearly all staff and managers found that CPRS placed substantial demands on their time during implementation, but most sites could not allocate additional nursing staff to patient care during implementation.

Staff and management reported that the time demands of BCMA were extremely high at fi rst. Previous research has documented problems with degraded communication between nurses and physicians, nurses failing to complete other care activities due to the workload created by BCMA, and an increased focus on managing BCMA rather than focusing on other patient needs.²¹ Many scanning problems were difficult to troubleshoot; as one nurse stated, “there were too many exceptions in the scanning process.” At most VA sites, no additional staffi ng was available to nursing or pharmacy to deal with the time needed to adapt to BCMA. Responsive leaders listened to managers and tried to meet staffi ng needs, but there was no budget available to increase the number of nursing staff per shift.

Limitations to the Study

A retrospective analysis is limited in that memories of the implementation may be inaccurate or biased by the passage of time, and some personnel whose perspectives might be valuable may no longer be available to interview. In addition, the site coordinators were VA employees, and could have biased the study findings by their selection of interview subjects. To reduce this risk, the informed consent materials were designed to recognize that employees are vulnerable subjects, and interviewees could decline to be interviewed without informing the site coordinator.

The thematic analysis was based on notes taken by the investigators, rather than on full transcripts. This was an intentional decision, trading the total number of interviews, as well as sites, for the detail that might have been obtained in full transcripts. The analysis was conducted by 1 investigator, with review by collaborators, which does not assure reliability to the same degree as would dual-coding of all data. The volume of interviews conducted precluded dual-coding in the timeline and budget afforded this study.

Finally, the VA is unique in that it is a fully integrated healthcare system with sites across the United States. Clinicians and managers are employees of the VA and have clear incentives to adhere to organizational policies. The VA’s experience with HIT may be different from that of freestanding hospitals and of hospitals in which physicians are not employees.

DISCUSSION

Overall, IT implementation success in the VA depended on: (1) whether there was support for change from both leaders and staff, (2) development of a gradual and fl exible implementation approach, (3) allocation of adequate resources for equipment and infrastructure, hands-on support, and deployment of additional staff, and (4) how the implementation team planned for setbacks and then continued the process to achieve success.

When contemplating a new HIT system, hospital leaders must ensure that they have strong support for the change. Previous research has found that the lack of one or more “project champions” is perceived to be the most important cause of problems when implementing HIT systems, with a lack of dedication from top-level management being the secondmost important factor.²⁸

The team in charge of the new system must systematically plan for all aspects of the implementation, including training, support, work flow changes, and communication.²⁷ At the same time, the implementation plan must be fl exible and gradual enough to adjust as new challenges arise. In the VA, CPRS exemplifi ed some of these “best practices”; the system was developed over many years specifi cally for the VA, the system was gradually implemented within and across VA sites, and at most sites there were strong teams supporting the system. BCMA, in contrast, encountered problems that could have been avoided. The fact that staff resisted the system at some sites and BCMA’s benefi ts were not realized as quickly as possible is not surprising.

A successful HIT implementation requires substantial dedication of resources, particularly personnel resources. Adequate training and support staff are required; failure to provide 24-hour support was identifi ed by the people we interviewed as a near-fatal fl aw during implementation. Additional patient care staff should be deployed during a transitional period because patient care providers need a substantial amount of time to learn new methods of keeping patient records and administering medications. In general, this is a temporary requirement, and less time is spent on record keeping activities once the HIT system is established.³¹

Healthcare leaders must expect that there will be setbacks during an HIT implementation, and must push through these setbacks thoughtfully and deliberately. The concerns of the care providers who use the systems must be respected and considered throughout the transitional period. For example, hospitals need to make a conscientious effort to learn and solve the underlying causes of workarounds, which are often infrastructure and software functionality problems.^32,33

A poorly planned implementation can have deleterious effects on the quality of patient care, and can even increase inpatient mortality.²³ In the long term, nearly all VA facilities have rectifi ed diffi culties they had during implementation, and today, CPRS and BCMA are often credited as being central to the VA’s efforts to achieve high-quality ratings while controlling medical care costs.^13,34-40 Because the VA is an integrated system, computerized patient records can transmit information between physician practices, hospitals, long-term care facilities, and other care settings, furthering the benefi ts of HIT systems. Although the VA may have been the best possible environment for innovations such as CPRS and BCMA, during and after implementation the VA had to overcome many challenges—challenges every facility must address to achieve a successful implementation.

Acknowledgments

We appreciate the guidance of the Advisory Committee of this study: Cathy Rick (VA Headquarters), Oyweda Moorer (VA Headquarters), Ginny Creasman (Cincinnati VA), Geri Coyle (Martinsburg VA), and Bryan Volpp (Martinez VA). We also thank the VA leaders at the 7 sites visited for this research. Susan Schmitt provided excellent assistance to the data analysis in this project. Dennis Keane and Melanie Chan were extremely helpful in planning the site visits and Advisory Committee meetings.

Author Affiliations: From Philip R. Lee Institute for Health Policy Studies (JS), University of California, San Francisco, CA; Boston VA Health Care System (JFB), Department of Health Policy and Management (JFB), Boston University, Boston, MA; Palo Alto VA Health Care System (CSP), Menlo Park, CA; Stanford University (CSP), Palo Alto, CA.

Funding Source: This research was supported by the Robert Wood Johnson Foundation and the Gordon and Betty Moore Foundation.

Author Disclosures: The authors (JS, JFB, CSP) report no relationship orfinancial interest with any entity that would pose a confl ict of interest with the subject matter of this article.

Authorship Information: Concept and design (JS, JFB, CSP); acquisition of data (JS, JFB, CSP); analysis and interpretation of data (JS, JFB, CSP); drafting of the manuscript (JS); critical revision of the manuscript for important intellectual content (JS, JFB, CSP); provision of study materials or patients (JS); obtaining funding (JS); administrative, technical, or logistic support (JS); and supervision (JS).

Address correspondence to: Joanne Spetz, PhD, Professor, Philip R. Lee Institute for Health Policy Studies, University of California, San Francisco, 3333 California St, Ste 265, San Francisco, CA 94118. E-mail: Joanne.Spetz@ucsf.edu.1. Kadas RM. The computer-based patient record is on its way: HMOs, the economy and HIPAA will drive adoption. Healthc Inform. 2002;19(2): 57-58.

2. Anderson GF, Frogner BK, Johns RA, Reinhardt UE. Health care spending and use of information technology in OECD countries. Health Aff (Millwood). 2006;25(3):819-831.

3. Burt CW, Hing E. Use of computerized clinical support systems in medical settings: United States, 2001-03. Adv Data. 2005;353:1-8.

4. Poon EG, Jha AK, Christino M, et al. Assessing the level of healthcare information technology adoption in the United States: a snapshot. BMC Med Inform Decis Mak. 2006;6(1):1.

5. Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: dispensing and administration— 2002. Am J Health Syst Pharm. 2003;60(1):52-68.

6. Coye MJ, Bernstein WS. Improving America’s health care system by investing in information technology. Health Aff (Millwood). 2003;22(4): 56-58.

7. Chaiken BP, Holmquest DL. Patient safety: modifying processes to eliminate medical errors. Nurs Outlook. 2003;51(3):S21-S24.

8. Miller RH, Sim I. Physicians’ use of electronic medical records: barriers and solutions. Health Aff (Millwood). 2004;23(2):116-126.

9. Thompson TJ, Brailer DG. The decade of health information technology: delivering consumer-centric and information-rich health care framework for strategic action. Washington, DC: US Department of Health and Human Services; 2004.

10. May S. Bar-code technology: a point-of-care medication management system prevents errors and reduces paperwork. Healthc Inform. 2002;19(5):69-70.

11. Shekelle PG, Morton SC, Keeler EB. Costs and benefi ts of health information technology. Rockville, MD: Agency for Healthcare Research and Quality; 2006.

12. Bates DW. Using information technology to reduce rates of medication errors in hospitals. Br Med J. 2000;320(7237):788-791.

13. Coyle GA, Heinen M. Scan your way to a comprehensive electronic medical record: augment medication administration accuracy and increase documentation effi ciency with bar coding technology. Nurs Manage. 2002;33(12):56, 58-59.

14. Johnson CL, Carlson RA, Tucker CL, Willette C. Using BCMA software to improve patient safety in Veterans Administration Medical Centers. J Healthc Inform Manage. 2002;16(1):46-51.

15. Kazley AS, Ozcan YA. Do hospitals with electronic medical records (EMRs) provide higher quality care? an examination of three clinical conditions. Med Care Res Rev. 2008;65(4):496-513.

16. Menachemi N, Chukmaitov A, Saunders C, Brooks RG. Hospital quality of care: does information technology matter? the relationship between information technology adoption and quality of care. Health Care Manage Rev. 2008;33(1):51-59.

17. Asch SM, McGlynn EA, Hogan MM, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004;141(12):938-945.

18. Jones SS, Adams JL, Schneider EC, Ringel JS, McGlynn EA. Electronic health record adoption and quality improvement in US hospitals. Am J Manag Care. 2010;16(12 suppl HIT):SP64-SP71.

19. Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff (Millwood). 2010;29(4):655-663.

20. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, effi ciency, and costs of medical care. Ann Intern Med. 2006;144(10):742-752.

21. Patterson ES, Cook RI, Render ML. Improving patient safety by identifying side effects from introducing bar coding in medication administration. J Am Med Inform Assoc. 2002;9(5):540-553.

22. Nebeker JR, Hoffman JM, Weir CR, Bennett CL, Hurdle JF. High rates of adverse drug events in a highly computerized hospital. Arch Intern Med. 2005;165(10):1111-1116.

23. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics. 2005;116(6):1506-1512.

24. Spetz J, Keane D. Information technology implementation in a rural hospital: a cautionary tale. J Healthc Manage. 2009;54(5):337-348.

25. Wideman MV, Whittler ME, Anderson TM. Barcode medication administration: lessons learned from an intensive care unit implementation. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation. Vol 3. Rockville, MD: Agency for Healthcare Research and Quality; 2005:437-451.

26. Ovretveit J, Scott T, Rundall TG, Shortell SM, Brommels M. Implementation of electronic medical records in hospitals: two case studies. Health Policy. 2007;84(2-3):181-190.

27. Kraus S, Barber TR, Briggs B, et al. Implementing computerized physician order management at a community hospital. Jt Comm J Qual Patient Saf. 2008;34(2):74-84.

28. Paré G, Sicotte C, Jaana M, Girouard D. Prioritizing the risk factors influencing the success of clinical information system projects: a Delphi study in Canada. Methods Inf Med. 2008;47(3):251-259.

29. Brown SH, Lincoln MJ, Groen PJ, Kolodner RM. VistA--U.S. Department of Veterans Affairs national-scale HIS. Int J Med Inform. 2003;69(2-3):135-156.

30. National Commission on VA Nursing. Caring for America’s veterans: attracting and retaining a quality VHA nursing workforce. Washington, DC: Department of Veterans Affairs; May 2004.

31. Wong DH, Gallegos Y, Weinger MB, Clack S, Slagle J, Anderson CT. Changes in intensive care unit nurse task activity after installation of a third-generation intensive care unit information system. Crit Care Med. 2003;31(10):2488-2494.

32. Koppel R, Wetterneck T, Telles JL, Karsh BT. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc. 2008;15(4):408-423.

33. Carayon P, Wetterneck TB, Schoofs Hundt A, et al. Evaluation of nurse interaction with bar code medication administration (BCMA) technology in the work environment. J Patient Saf. 2007;3:34-42.

34. United States Congressional Budget Offi ce. The health care system for veterans: an interim report. Washington, DC: Congressional Budget Office, Pub. No. 3016; December 2007.

35. Achtmeyer CE, Payne TH, Anawalt BD. Computer order entry system decreased use of sliding scale insulin regimens. Methods Inf Med. 2002;41(4):277-281.

36. Calabrisi RR, Czarnecki T, Blank C. The impact of clinical reminders and alerts on health screenings: the VA Pittsburgh Healthcare System achieves notable results by enhancing an automated clinical reminder system within its CPR—and has the data to prove it. Health Manage Technol. 2002;23(12):32-34.

37. Cornia PB, Amory JK, Fraser S, Saint S, Lipsky BA. Computer-based order entry decreases duration of indwelling urinary catheterization in hospitalized patients. Am J Med. 2003;114(5):404-407.

38. Stair TO. Reduction of redundant laboratory orders by access to computerized patient records. J Emerg Med. 1998;16(6):895-897.

39. Fletcher RD, Dayhoff RE, Wu CM, Graves A, Jones RE. Computerized medical records in the Department of Veterans Affairs. Cancer. 2001; 91(8)(suppl):1603-1606.

40. Young D. Veterans Affairs bar-code-scanning system reduces medication errors. Am J Health Syst Pharm. 2002;59(7):591-592.

Download PDF: What Determines Successful Implementation of Inpatient Information Technology Systems?