Publication
Article
The American Journal of Managed Care
Automated Patient Assessments After Outpatient Surgery Using an Interactive Voice Response System
Author(s):
Using an interactive voice response system to contact patients after outpatient surgery will likely result in improved efficiency without a decrease in assessment quality.
Objective
: To test the feasibility and utility of an interactive voice response system (IVRS) for monitoring patients after outpatient surgery.
Methods
: We studied consecutive patients undergoing gynecologic day surgery. The IVRS called patients on the first postoperative day and asked them if they were experiencing new problems. Feasibility was assessed in terms of call responses and acceptance by patients. Utility was measured in terms of the ability of the IVRS to identify adverse events (AEs), defined as procedure-related symptoms requiring a physician or hospital visit. We contacted patients 30 days later to elicit their perceptions of the IVRS and determine AE status.
Results
: Follow-up was complete for 249 of 270 enrolled patients (92%). The IVRS successfully contacted 130 patients (52%). Of the 22 patients (17%) who required a follow-up phone call, 9 had a new problem related to surgery, 7 had new or worsening symptoms, 6 wanted to speak with a nurse, and 1 had a medication-related problem. Patients remembering the automated call (n = 96) reported the system easy to use (82%) and comprehend (86%). Most patients (68%) preferred the IVRS to a personal follow-up call (probability greater than 50%, P <.001). AEs occurred in 40 patients (16%; 95% confidence interval = 12%, 21%). The IVRS did not identify any AEs because 90% of these occurred after the automated call.
Conclusion
: An IVRS-based method of monitoring outpatient surgery patients is feasible. To improve utility, calls must occur later than first postoperative day.
(Am J Manag Care. 2008;14(7):429-436)
To monitor for adverse events in patients undergoing outpatient surgery, many institutions have a policy for nurses to call patients on the first postoperative day, a process that is expensive and difficult to audit.
An interactive voice response system (IVRS) can be used for postoperative calls.
Use of the IVRS is more efficient and does not compromise the quality of the assessments.
However, to improve utility, calls must occur later than the first postoperative day.
Patients who undergo outpatient surgery do not stay in a hospital facility the evening after their procedure. With advances in minimally invasive surgery and anesthetic techniques, the volume of outpatient surgery is increasing dramatically and accounts for the majority of surgeries.1,2 Although outpatient surgeries are generally safe, risk is increased in certain circumstances.3-5 Outpatient surgery complications can be especially damaging because they occur at the patients’ home outside of a monitored setting.
Fortunately, most complications occurring after outpatient surgery are self-limited. The most common complications include symptoms such as pain, nausea and vomiting, bleeding, and headache.6 Although these symptoms often are of minor clinical importance,4,6,7 it is important to monitor them for 3 reasons. First, symptoms can herald severe underlying problems such as wound infections or postoperative myocardial infarctions. Second, these symptoms can progress to severe outcomes such as shock, respiratory failure, or death, especially if there is a delay in treatment.8,9 Finally, symptoms can cause anxiety and impede a rapid return to normal function.
One common method of assessing complications after outpatient surgery is a nurse callback program. In such programs, a nurse calls patients on the day after surgery and will subsequently coordinate treatment if a patient is experiencing problems.10-12 A limitation of such programs is their expense. To reduce costs, one could automate aspects of it using an interactive voice response system (IVRS),13 which is a technology allowing patients to interact with databases using a telephone.14-21 The IVRS could make an initial screening call to all patients to determine whether they were having a problem. Then, the IVRS could notify a nurse to follow up with the few patients who were screened “positive.” This practice would obviate the need for a nurse to call every patient, significantly reducing the monitoring program’s expense.
Two issues need to be considered by decision makers when contemplating whether to implement an IVRS-based patient monitoring system. First, will patients accept the technology? If patients cannot use the IVRS, then the technology is not capable of improving monitoring. Second, will the IVRS-based monitoring result in improved identification and management of complications? If the system does not alert providers to patient problems in a timely manner, then the system is not useful. We designed this study to assess these factors. We were primarily interested in understanding issues related to feasibility to determine whether further study of IVRS to manage patients was merited. We also were interested in understanding the potential utility of IVRS in terms of managing complications, particularly its ability to capture the timing and types of outcomes patients experienced. Because these data were not previously described, this study was necessary for the design of future IVRS-based interventions.
Design, Setting, and Patients
METHODS
Figure 1
The study intervention is illustrated in . After patient consent, we collected demographic and medical data. When patients were discharged from the outpatient surgery unit, the study nurse entered the patient’s ID, phone number, and discharge date into the IVRS; this information was physically stored on a server in the hospital’s data center. The nurse performed data entry using a Web-accessible personal computer in the outpatient surgery unit. The IVRS called patients on the day after their discharge from the day surgery unit. If there was no response to the call, the IVRS reattempted the call every 2 hours until a response occurred. If no response occurred before 9:00 pm that day, the system stopped calling the patient.
When a call was answered, the IVRS asked a series of questions that required a YES/NO answer. The IVRS used speech recognition software to update the database and prompt further actions. The system first asked: “Are you the patient discharged from the Royal Alexandra Hospital yesterday?” If the person responded NO, then the IVRS thanked the individual and disconnected. If the person responded YES, IVRS asked 4 questions and stored the responses:
• “Since your surgery, have you had any new or worsening symptoms?”• “Since your surgery, have you had any problems related to your surgery?”
• “Would you like to speak to a nurse?”
The IVRS transferred the patient to the provincial teletriage program (called HealthLink) after their first YES response to 1 of these 4 questions. HealthLink is a call center with nurses available to respond to patient queries at all times. Calls are answered by nurses, who are trained to triage patients according to the patient’s problem and prespecified call algorithms. Call center nurses have specific algorithms to deal with postoperative problems.
The IVRS we used was CallAssure. CallAssure can be run on any modern personal computer equipped with a telephony card and at least 2 analog lines. In addition, CallAssure requires access to an e-mail server (for notification messages) and, optionally, a printer (for reports). CallAssure uses Nuance 8.5 as its speech recognition software. Nuance reports an overall accuracy of 97%, but this rate is likely much higher for responses to questions requiring YES/NO answers.22 We did not use routines for dealing with grammar mismatch errors to keep the system as simple as possible. Also, in previous testing, we discovered that patients found it difficult to revert to touch-tone inputs as many people use handheld devices. In our experience, these devices made it very difficult to use touch-tone input.
Follow-up and Study Outcomes
We used SAS version 9.1 (SAS Institute, Inc, Cary, NC) for all data analysis. We report descriptive statistics using median and interquartile range (IQR) for continuous variables and frequency distributions for categorical variables. For outcomes, we present the proportion of patients with the outcome of interest with 95% confidence intervals (CIs), which we calculated using the Wilson score method.23
We used a multivariable logistic regression model to measure the association between IVRS call response and the likelihood of experiencing an adverse event while controlling for patient and surgery factors. We first measured the association of experiencing an adverse event with patient and surgery factors at the univariate level by using the χ2 test or the Wilcoxon rank sum test, respectively. We entered variables into our multivariable model if they were statistically associated with adverse event status at the P <.20 level.
Role of the Sponsors
Figure 2
summarizes the study flow and the main results for all patients. We enrolled 270 of 312 eligible patients during the study period. Thirty-four patients were not enrolled because they were missed by the research nurse, as she could not approach all patients. Eight patients refused participation in the study because they did not wish to take part in a research project. Of the 270 enrolled patients, 4 were not entered into the IVRS because of administrative errors and another 17 were lost to follow-up (6%). Our results are based on the 249 (92%) patients for whom we have 30-day follow-up data.
Most women had no comorbidities apart from the indication for surgery, virtually none had functional disabilities affecting their ability to perform activities of daily living, and few had surgeries in the preceding 12 months. The vast majority of patients were discharged on the day of surgery. The most common surgical procedures involved an abdominal approach (eg, laparoscopic surgery for endometriosis) or deep perineal approach (eg, dilation and curettage, cone biopsy).
We present call flow data in Figure 2. The IVRS made 318 calls to the 249 patients in the study. Most patients received 1 call; however, the system called back until the call was answered by the patient. Thus, many patients received more than 1 call, including 16 patients who were called 5 times. Of the telephone calls to the 249 patients, 163 were answered. Of these 163 calls, 130 were picked up by an individual who responded YES to the question regarding whether they were the correct patient. If patients responded YES to this question, then they were very likely to answer all of the remaining questions with YES or NO responses. Of the 130 patients, 129 (99%) answered at least 1 other automated survey question and 104 (80%) answered all automated survey questions. Note that the IVRS connected patients to the HealthLink after a YES response. Therefore, if a patient responded YES to any question, then she did not have the opportunity to answer remaining questions. In total, 119 of 130 (92%) patients responded to all the questions they were intended to answer.
A minority of patients indicated via the IVRS that they were having a problem or that they wanted to speak to a nurse (Figure 2). Of the 130 patients who were contacted successfully, 9 patients stated they were having a problem related to their surgery, 7 reported having new or worsening symptoms, 6 reported a desire to speak with a nurse, and 1 reported having problems related to medications. None of these patients spoke to a HealthLink nurse. This is because after patients were connected to the HealthLink service, they hung up before speaking with a nurse. We surmise that this may have occurred because our call dialog did not explicitly mention they would be placed on hold. Therefore, no actions were taken as a result of the automated call.
Table 2
At the time of our 30-day follow-up call, 40 of 249 women (16%; 95% CI = 12%, 21%) told us they had experienced an adverse event (Figure 2, ). These adverse events included wound infections (n = 16 [6%]; 95% CI = 4%, 10%), excessive pain (n = 9 [4%]; 95% CI = 2%, 11%), bleeding (n = 6 [2%]; 95% CI = 1%, 5%), and miscellaneous other types of problems (n = 9 [4%]; 95% CI = 2%, 11%). These other adverse events included problems related to anesthesia (superficial phlebitis, pharyngotracheitis) and nausea and vomiting. A third of all adverse events required a visit to a hospital (including emergency department visits), whereas the remainder were managed in private doctors’ offices. Three quarters of adverse events required some form of corrective action, including prescription of antibiotics (for wound infections) and increased analgesia medications (for pain). The median onset of these adverse events was 10 days after the surgery (IQR = 3-14 days). Only 4 adverse events (10%) occurred within 1 day of the surgery.
DISCUSSIONWe designed this study to determine the feasibility and utility of an IVRS to monitor patients following outpatient surgery. Overall, our data show that the system was feasible, with 52% of patients being reached by the IVRS. Almost all of these patients understood the call dialog and a majority preferred an automated call to a personal one.
Despite our optimism regarding feasibility, our results raised concerns related to the IVRS’s utility. The system failed to connect patients to the teletriage program. This was because patients hung up while they waited to speak with a Health- Link nurse. This problem may be obviated by changing the IVRS call dialog to explicitly instruct patients to hold while a nurse is made available. We also found that patients experienced most of their adverse events after the IVRS call. This occurred because we designed our IVRS to call on the first postoperative day (ie, our design reflected existing work flows in most day surgery units). This problem is easily remedied by delaying the call or repeating it several days later. The most important finding of our study was that 1 in 6 women experienced an adverse event. This finding emphasizes the need for improved monitoring of patients undergoing outpatient surgery. Given the expense of having nurses perform monitoring, we believe our study supports a need for further efforts to adapt IVRS technologies for this task.
A follow-up telephone call to assess patients following outpatient surgery appears to be a good idea. However, there are limited data describing its rationale and benefit.11,24 Despite this, many health facilities and some health systems have a policy to attempt to call patients for this purpose. It is unknown how many facilities routinely contact patients after outpatient surgery, how effectively surgery units contact patients, and what follow-up occurs once a problem is identified. It is likely that facilities do not strictly adhere to the policy and do not track statistics for 2 reasons: a shortage of nursing staff to place the phone calls, and a lack of data systems to help support the work flow. The IVRS we tested could solve these 2 problems, while still being acceptable to patients.
Our study does not support the conclusion that IVRS calls improve patient outcomes after day surgery. The association of successful call completion with a lower adverse event risk is intriguing, but is almost certainly due to confounding. Although it is possible that patients may not have answered the telephone because they were having symptoms, we think this is unlikely given the adverse event timing we observed. It is remotely possible that patients who answered the IVRS were more compliant with follow-up instructions, which led to a reduced risk. This seems highly unlikely.
When we designed the IVRS, we debated whether methods to verify the true identity of a patient should be included. These methods could have required the patient to provide their birth date, the citation of a unique personal identification number at the time of enrollment, or the use of biometrics such as voice recognition. In our final design, we favored omitting these methods for 2 reasons. Most importantly, the current business practice of manually calling patients does not require the patient to verify who they are. Second, we felt any technical solution would be associated with increasing complexity, which might decrease usability and patient acceptance.
We performed a well-designed study. Its strengths included very broad inclusion criteria, a small proportion of patients lost to follow-up, and clinically important outcomes captured in all patients. Our conclusions were weakened because we are unaware why some patients did not respond to the IVRS. Some potential reasons include technical factors (eg, the IVRS did not call patients as programmed) or patient factors (eg, patients may not have understood the call prompts). We think it is more likely that patients did not answer the phone when called, as this factor has explained previous unsuccessful attempts to contact patients using manual calls.24,25 Similarly, we do not know whether the people who answered the IVRS responded to questions truthfully. An indication of this possibility is that 34/130 (26%) patients could not recall the IVRS call when we personally interviewed them. However, we think it is unlikely that people gave untruthful answers because prior IVRS research evaluating correspondence between personal and IVRS interviews shows that patients generally give consistent answers using both approaches. It is very possible that given patients’ busy lives, they might not recall a brief phone call (median duration = 60 s; IQR = 55-68 s) 4 or 5 weeks earlier. Regardless, even if all the patients who did respond to the IVRS but could not remember the call had negative opinions regarding the automated calls, the majority of patients in our study would still have favorable opinions of the technology.
Based on this study, we recommend several changes to the design of the telephone-based interventions to monitor patient safety following outpatient surgery (whether automated or not). First, patients should be called later than 1 day after the surgery. Ninety percent of adverse events started after the first postoperative day. Either a second call should be made several days later, or the first call needs to be made at that time. Second, more attempts to call patients are required. Many patients in our study did not respond despite several attempts by the system. Third, an enhanced response to patients requiring follow-up is essential. If one is going to automatically link patients to a teletriage program, then the technology needs to work. In our study, we found that patients hung up after they were connected to the teletriage program, probably because they did not understand they had to wait to speak to a nurse. This problem could be fixed by explaining with a call script the need to hold for a nurse. Fourth, the outpatient surgery unit needs to be notified of the potential problem. It would not be too onerous for the unit to personally call these patients, as only 17% of patients said YES to the IVRS-based question (Figure 2). Alternatively, the unit could focus on the patients not responding to the IVRS, as these were the ones who appeared to be at higher risk of problems. Finally, patients should be able to make in-bound calls to the IVRS.
In conclusion, our data show that it is feasible to monitor patients following outpatient surgery using an IVRS. If a particular outpatient surgery unit requires proof that follow-up calls improve patient outcomes, then we recommend they wait for randomized trial evidence before implementation. However, if an outpatient unit feels that follow-up calls must be done to meet regulatory requirements or because they simply want to enhance contact with patients following day surgery, then, given our findings, using an IVRS-based approach appears justified, assuming minor modifications in the technology’s design are made.
Author Affiliations: From the Department of Medicine, University of Ottawa (AJF, KGS, CVW), Ottawa, Ontario; the Clinical Epidemiology Program, Ottawa Health Research Institute (AJF, KGS, CVW), Ottawa, Ontario; Capital Health (RL, RM, WF, TEF, SJ-K), Edmonton, Alberta; the Research, Evaluation and Quality Improvement Center, University of Alberta (RM, SJ-K), Edmonton, Alberta; the Department of Obstetrics and Gynaecology, The Ottawa Hospital (WF), Ottawa, Ontario; and the Faculty of Medicine, University of Calgary (TEF), Calgary, Alberta.
Funding Source: This research was funded by the Canadian Patient Safety Institute, the Ottawa Hospital Center for Patient Safety, and Capital Health, Edmonton area. Vocantas Inc provided technical support for the project.
Author Disclosures: Dr Forster is supported by a Ontario Ministry of Health Career Scientist Award and is a coinventor on a patent for an interactive voice response system. Dr Shojania is supported by a Government of Canada Research Chair in Patient Safety and Quality Improvement. The other authors (RL, RM, JWF, TEF, SJ-K, CVW) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Authorship Information: Concept and design (AJF, RL, RM, JWF); acquisition of data (AJF, RL, RM, JWF, SJ-K); analysis and interpretation of data (AJF, TEF, KGS); drafting of the manuscript (AJF, KGS); critical revision of the manuscript for important intellectual content (AJF, RL, RM, JWF, TEF, KGS, CVW); statistical analysis (AJF, CVW); provision of study materials or patients (AJF, RL, RM, JWF, SJ-K); obtaining funding (AJF, RL, RM); administrative, technical, or logistic support (RL, TEF, SJ-K, CVW); and supervision (AJF, JWF).
Address correspondence to: Alan J. Forster, MD, MSc, Clinical Epidemiology Program, Ottawa Health Research Institute, 725 Parkdale Ave, Ottawa, Ontario K1Y 4E9, Canada. E-mail: aforster@ohri.ca.
1. Eggertson L. Ten year trend: surgeries up, hospitalizations down. CMAJ. 2007;176(6):756.
3. Tham C, Koh KF. Unanticipated admission after day surgery. Singapore Med J. 2002;43(10):522-526.
5. Twersky R, Fishman D, Homel P. What happens after discharge? Return hospital visits after ambulatory surgery. Anesth Anal. 1997;84(2):319-324.
7. Rawal N. Postoperative pain treatment for ambulatory surgery. Best Pract Res Clin Anaesthesiol. 2007;21(1):129-148.
9. Grogaard B, Kimsas E, Raeder J. Wound infection in day-surgery. Ambul Surg. 2001;9(2):109-112.
11. Fallis WM, Scurrah D. Outpatient laparoscopic cholecystectomy: home visit versus telephone follow-up. Can J Surg. 2001;44(1):39-44.
13. Forster AJ, van Walraven C. Using an interactive voice response system to improve patient safety following hospital discharge. J Eval Clin Pract. 2007;13(3):346-351.
15. DeMolles DA, Sparrow D, Gottlieb DJ, Friedman R. A pilot trial of a telecommunications system in sleep apnea management. Med Care. 2004;42(8):764-769.
17. Mahoney D, Tennstedt S, Friedman R, Heeren T. An automated telephone system for monitoring the functional status of communityresiding elders. Gerontologist. 1999;39(2):229-234.
19. Pinto BM, Friedman R, Marcus BH, Kelley H, Tennstedt S, Gillman MW. Effects of a computer-based, telephone-counseling system on physical activity. Am J Prev Med. 2002;23(2):113-120.
21. Leirer VO, Morrow DG, Tanke ED, Pariante GM. Elders’ nonadherence: its assessment and medication reminding by voice mail. Gerontologist. 1991;31(4):514-520.
23. Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998;17(8):857-872.
25. Dudas V, Bookwalter T, Kerr KM, Pantilat SZ. The impact of follow-up telephone calls to patients after hospitalization. Dis Mon. 2002;48(4):239-248.
