Cost Implications of Human and Automated Follow-up in Ambulatory Care

Objectives

To compare the costs of human and automated follow-up processes in ambulatory care.

Study Design

Analysis of costs of nurse-initiated and interactive voice response (IVR) system follow-up interventions.

Methods

Using national cost data and data on follow-up processes and outcomes from a previous study, we examined the costs to the healthcare system and providers of developing a follow-up process using nurse-initiated telephone calls compared with calls made by an IVR.

Results

Whether using nurse-initiated telephone calls or IVR calls, costs over the first 2 years of follow-up for a practice assumed to have 4800 acute care patient visits per year are approximately the same. After 2 years, IVR follow-up is approximately $9000 per year less expensive than nurse follow-up. In addition, overall cost savings are greater with IVR.

Conclusions

Follow-up of ambulatory care patients is a way to assess risks of future problems and associated costs and to improve quality of care. An automated follow-up process using IVR is more efficient than one based on nurse-initiated follow-up calls.

Am J Manag Care. 2014;20(11 Spec No. 17):SP531-SP540

Take-Away Points

Follow-up interventions with patients can potentially produce cost savings for the healthcare industry, and for providers specifically, by appropriately managing patients with varying degrees of risk.

• Follow-up processes can identify patients who are unimproved after an initial physician visit
• An automated follow-up process using an interactive voice response (IVR) system is less costly, with greater cost savings, after the first 2 years than processes using nurses to follow up with patients.
• Because most patients will not need further treatment after their initial visit, the IVRS is more efficient than nurses at identifying the high-risk patients who do need additional medical attention.

led ACOs.

1

Driven by the value-based and financial implications of healthcare reform, providers have begun to transform their delivery structure, emphasizing the shift from inpatient to ambulatory settings and manag- ing patients in an appropriate setting along the care continuum. Reflecting such trends, CMS has reported more than half of Medicare Accountable Care Organizations (ACOs) are led by office-based physicians, outnumbering hospital-

2,3

4-6

One critical comn health management, which involves patient monitoring and proper follow-up care by primary care providers.While the true performance of an ACO depends on its ability to simultaneously manage patients with varying degrees of risks, the most critical group needing urgent attention is high-risk patients. The high-risk patients are the smallest group within the overall patient population, but they represent the highest healthcare spending.Therefore, providers are likely to produce cost savings if they can reduce or prevent multiple visits by, or hospital readmissions of, high-risk patients. Thus, in addition to performing risk profiling to proactively segment the patient population, providers should also identify the risks associated with a given acute episode via active monitoring and proper follow-up.

7

While important, such practices are in fact rare in ambulatory settings.Little data exist on either the cost implications of doing such follow-up or on the cost savings that might be anticipated by closer or more complete monitoring of ambulatory patients.

8

9-11

We developed a mechanism for automated follow-up of ambulatory patients seen for acute illnesses.The automation mechanism studied was an interactive voice response (IVR) system that provides reports to providers through their electronic medical record. IVR has been studied for econom- ic and healthcare utilization impact, reminders to patients for screening tests, and for pharmacologic monitoring.

12

12

Berner et al evaluated patient-reported outcomes of improvement, as well as patient interaction with the health system.The present study uses data from the Berner et al studyto examine the cost implications of automated, compared with human, follow-up in ambulatory care.

METHODS

Data Sources

8,12

the

The development of the IVR system and the study of outcomes have been described in detail elsewhere.The study, approved by the Institutional Review Board at University of Alabama at Birmingham (UAB), was conducted over 3 phases with 3 cohorts of patients in each of 3 ambulatory care sites that led up to full automation of the follow-up and feedback process. One site was the “sick call” clinic for acute illnesses in human immunodeficiency (HIV) patients who were being followed in the UAB HIV clinic setting. The second site was a family medicine clinic, affiliated with UAB, but located in a different city. The third site was a cerebral palsy clinic that treated a variety of disabled patients.

In Phase 1, patients were contacted by a study staff member 3 weeks after a primary care visit for an acute problem and asked about symptom improvement (improved, or not improved [same or not improved/worse]), medication compliance, and interaction with the health system prior to the follow-up call (called clinic, or made an appointment with their own physician, or saw some- one else [another physician, emergency department, hospital]). This was considered to be baseline data with no intervention.

In Phase 2, with a new cohort of patients, the same questions were asked by a staff member in a phone call 1 week after the visit. In Phase 3, with a third cohort of patients, the questions were asked 1 week after the visit via IVR. Only the first 2 sites participated in Phase 3. In all cases, patients who were unimproved were transferred via telephone to their clinic, where usual care processes were employed.

12

13

14

Table 1

For the present study, we used the data on patient-reported symptom improvement and interaction with the health system from the Berner et al study.We also used data from that research on the average personnel time necessary to pre- pare for and make the phone calls, which was not reported previously. We obtained the average personnel costs for physician office—based nurses from the Bureau of Labor Statistics.The 2012 healthcare costs for established and new patient visits were obtained from CMS.We chose the Level 3 Evaluation & Management (E&M) code for the visits, to represent an average cost. These data are shown in .

Analyses

The analyses for this study were based on the following assumptions:

1. An attempt will be made to contact all acute care patients for follow-up.
2. A nurse would be the most likely healthcare staff member to make the follow-up call in the human follow-up situation.
3. All IVR system setup costs will be paid in Year 1.
4. Patients who are unimproved/no change at 1 week (in the original study these were patients who were routed to the clinic) would discuss the situation further with a nurse. Since 1 week is still early in the healing process, the discussion with the nurse would be the most likely end point for these patients.
6. Patients who report they are unimproved/worse at 1 week would discuss the problem with the nurse and would also schedule another appointment with their primary care physician.
7. The populations reached (ie, patients who answer the phone) or not reached (patients who do not answer the phone) are not different in significant ways. The assumptions for this analysis are based on a primary care practice that sees an average of 100 acute care patients per week for 48 weeks.

This analysis is performed from the perspective of the healthcare system, showing potential costs to the system rather than to individuals. Analyses included comparisons of costs of IVR with human calls, costs of interaction with the health system if early (eg, 1 week) follow-up was initiated compared with no follow-up, and potential cost savings with early follow-up (human and IVR). Because the IVR system setup costs are much higher than maintenance costs, the time horizon includes both the first year (when setup costs are incurred) and the second year (when only maintenance costs are incurred). We also include a cumulative 5-year cost estimate.

Calculation of Cost Savings From Follow-up

The cost savings from follow-up is represented by the difference between the costs that are potentially preven- ted if early follow-up is done (potentially preventable costs) and the costs of conducting early follow-up (1 week post visit), by either a nurse or an IVR system.

Calculation of potentially preventable costs.

Potentially preventable costs are those costs that could potentially be affected by the follow-up process. These costs include the costs of interaction with the health system (interaction costs) for patients who have not improved or who have gotten worse and who eventually sought care on their own, when their conditions had progressed to a more seri- ous state. It is assumed that addressing the problems early by means of the follow-up process, even if this leads to interaction costs, could prevent higher costs later. The costs are referred to as potentially preventable because there is no guarantee that the follow-up and early intervention would prevent them.

There are some interaction costs that the follow-up process cannot prevent. These unavoidable costs include: 1) interaction costs of patients who are not able to be reached for follow-up, and 2) interaction costs of patients who complete the follow-up phone call, but who have al- ready initiated the interaction with the health system prior to receiving the follow-up call. If a patient incurs costs prior to receiving the follow-up call, the costs are consid- ered unavoidable since they cannot be prevented by the follow-up process, although other costs subsequent to the follow-up phone call may be prevented. Those who are reached but who do not complete the call are assumed to be improved and to not have additional interaction costs.

Potentially preventable costs are calculated by subtracting the unavoidable costs from the total interaction costs without follow-up.

There are additional interaction costs resulting from the follow-up activities that are necessary to address the needs of those patients found to be unimproved. These costs are included in the costs of follow-up (see below).

Calculation of total costs of interaction with the health system without follow-up

.

The costs of total interaction are based on the costs of the types of interaction after the initial visit and the percentage of patients who incur these costs.

The categories of patient interaction costs include the costs of 1) phone calls with the health providers (estimated at 5 minutes of nurse’s time); 2) appointments with their primary care provider (Medicare reimbursement for Level 3 E&M code 99213, for established patient); and 3) seeing another provider (Medicare reimbursement for Level 3 E&M code 99203, for new patient). Table 1 lists these costs.

12

Based on the data from Berner et al,which examined patient interaction with the health system during the first 3 weeks after the visit without any follow-up, we assume that approximately 5% of patients will call the office and talk with the nurse only, 10% will have a return visit with their physician, and 11% will see a different physician. Table 1 summarizes these data.

Patients completing the call

.

12

The magnitude of potentially preventable costs depends on the number of patients completing the call. We estimate that 1) 65% of the 4800 patients (n = 3120) will be able to be reached, 2) 97% of those who are reached by telephone (n = 3026) will complete the nurse-initiated call, and 3) 48% (n = 1498) will complete the IVR calls.

Interaction costs of patients not reached

.

Total costs of interaction with the health system after the initial visit are calculated on the total patient population base (n = 4800) using the information described above. Since 65% of the patients are able to be reached, we assume that 35% of total costs of interaction with the health system are the unavoidable interaction costs for those who are not reached.

Interaction with health system prior to the 1-week follow-up call.

12

eAppendix A

www.ajmc.com

Based on the data of Berner et alfor patients who complete the nurse calls prior to the 1-week call, 7% will have already called the clinic, 4% will have had an appointment with their physician, and 3% will have seen another physician. For those who complete the IVR call, 8% will have seen their own physician and 5% will have seen another physician. Nobody receiving the IVR call reported only calling the clinic (see Table 1). The costs for these phone calls and physician visits are considered unavoidable costs, meaning that even with the 1-week follow-up these calls and visits and associated costs will still have occurred. Detailed calculations on total cost of interaction and unavoidable costs are in , available at .

Calculation of Cost of Early Follow-up by a Nurse or IVR System for 2 Years

Table 1 includes a description of the data based on which the follow-up costs are calculated. The cost of conducting a follow-up call is composed of 2 parts: the preparation cost and the actual call cost. The preparation cost is similar for the 2 follow-up modes, because in each, a nurse is deployed to prepare and review all patients’ records to obtain relevant information required for the call. The preparation cost is the product of the estimated preparation time in minutes for each patient and the per minute nurse salary. Based on the time data collected at all study sites, the average preparation time is 2.8 minutes. Per the Bureau of Labor Statistics, an average nurse salary is $62,120 per year. Using a typical 168 hours worked per month, the per minute nurse salary is $0.51. Thus, the preparation cost per patient is $1.44.

Per our assumption for this analysis, a primary care practice sees an average of 100 acute patients per week. Using a typical 48 weeks worked per year, the total number of patients seen in a year is 4800. As a result, the annual preparation cost is $6902.22. This cost is the same for both the nurse calls and the IVR calls.

Nurse follow-up

. In addition to the costs to prepare for calling all patients, the nurse follow-up costs include the time to 1) complete the initial follow-up call to patients who are reached, 2) place calls to patients who are never reached or who hang up, and 3) assess/discuss the need for an appointment with patients who are not improved. In addition to the nurse’s time, there is the cost of an additional physician visit (Medicare reimbursement for Level 3 E&M code for established patient) for those patients who report being worse.

12

The initial call cost is the product of the estimated time a nurse spends on the call with the patients and the per minute nurse salary. For the initial call, a nurse spends an estimated 5.7 minutes on the phone with each patient who can be reached and completes the call (3026 patients). Of these, 85% say they are better, so no further calls or conversations are necessary.The cost for the follow-up call for these patients is $2.93 per patient, for a total cost of $7530.26 per year. For the patients who report improvement, the only costs are for the initial follow-up calls.

For the remaining patients who do not tell the nurse that they are better, the nurse spends an additional 5 minutes to further determine the condition of the patient. During this conversation, if the patient indicates no change, the nurse may determine that no additional follow-up is necessary and ask the patient to continue the prescribed treatment or provide other counseling. This applies to about 12% of the patients who complete the call. The cost of this call is $2.93 for the initial call, and $2.57 for the additional 5 minutes of the nurse’s time to determine if the patient needs to see a physician for follow-up. Thus, the total cost per call for each of these patients is $5.50, resulting in a total cost of $1995.63 per year. For the 3% of patients who say that their condition is worse, the nurse will schedule a follow-up appointment. The total cost for these patients includes $2.93 for the initial call, $2.57 for the follow-up conversation, and the additional cost of the established doctor’s appointment, estimated at $70.46, equaling $75.96 per patient, or $6896.11 per year. Finally, those patients who are not reached or do not complete the call also incur costs as the nurse spends time conducting the call. We estimated that these calls take 2.5 minutes, at a cost of $1.28 per patient, for a total of $2277.12 per year.

eAppendix B Table 1

Assuming unchanged salaries/reimbursement over the 2-year period, these costs would be the same for the second year. provides details on the cost calculations.

IVR follow-up

. The costs of IVR follow-up for the first year include the costs of 1) implementing the IVR system (hardware, software, vendor service agreement); 2) the nurse’s time to prepare the patient information for all patients for the IVR system; 3) the additional nurse’s time to assess/discuss need for an appointment with patients who are not improved; and 4) a physician visit (estimated as the Medicare reimbursement for Level 3 E&M code for an established patient) for those patients who report being worse.

12

12

For the IVR call, the initial IVR system implementation cost is $20,000, which is included in the total cost of Year 1. The annual maintenance cost of $2000 is incurred in Year 2 and later. There are no additional charges for the individual calls. Of the patients who complete the IVR call, 64% will report being improved.Although in the Berner et al studyno patients in the IVR follow-up group reported being worse, it seems reasonable to assume conservatively that the proportions of unimproved patients would be similar to those for the nurse calls at 1 week. Thus, we estimate that 30% of the total number of patients who complete the IVR call would report no change and 6% would report being worse.

A patient who responds that there is no change in his/her symptoms is directed to a nurse for further assessment, and the cost for the 5-minute assessment, as described for the nurse calls, is $2.57. This will result in a total cost of $1153.66 per year for the practice. On the other hand, a patient who indicates his/her symptoms have gotten worse is directed to a nurse for further assessment and receives a follow-up doctor appointment. Thus, the total cost for this patient is $73.03, and the total cost per year to the practice is $6561.99.

eAppendix B Table 2

Assuming unchanged salaries/reimbursement for Year 2, the costs for the second year would substitute the IVR maintenance-alone costs for the IVR system setup/ maintenance costs from Year 1, with all other costs the same as for Year 1. provides the detailed calculations for the IVR system follow-up costs.

Sensitivity Analysis

We performed a sensitivity analysis on several of the variables by changing the numbers up or down by 10% and 20% and examined the impact on the difference in cost savings between IVR and the nurse calls after 5 years.

RESULTS

Table 2

shows the anticipated follow-up costs for both the nurse-initiated and the IVR follow-up calls for 4800 patients per year. The setup costs for the IVR system make it more expensive in the first year, but less expensive in the second and subsequent years. The 2-year costs for follow-up are similar for the 2 systems ($51,202 for nurse calls and $51,336 for IVR), but after that, the cost of the IVR calls is about $9000 less per year than of the nurse-initiated calls.

Table 3

shows the interaction costs for 3 groups, those 1) without any follow-up, 2) unable to be reached, 3) who had interaction prior to the nurse-initiated call or the IVR call; it also shows the potential cost savings of each type of follow-up. eAppendix A provides more detail. The potential cost savings over the first 2 years for nurse-initiated calls and IVR calls are $30,476 and $34,822, respectively. After 5 years, the total potential savings for the IVR call is approximately $38,000 more than for the nurse-initiated calls ($114,055 vs $76,190).

Table 4

shows the results of the sensitivity analysis. Decreasing certain variables (number of patients, time the nurse spends, salary of the nurse or whoever makes the call, and number of patients who complete the call) all reduce the difference between the 2 modes of follow-up. On the other hand, increasing these variables makes the advantage of the IVR system even greater. For the 2 modes to come out even in cost savings after 5 years, the decrease would have to be more than 58% for the number of patients and even higher for the other variables. The variables that are most sensitive to changes in the rates that are assumed in the analysis are percentage of patients not improved in 1 week for both IVR and nurse calls, and the number of patients.

Changes in the reimbursement for visits for new and established patients have very little impact on the difference between IVR and nurse cost savings. A 20% increase or decrease in reimbursement results in approximately a $2000 change in either direction. If the IVR system implementation costs are more than estimated, the savings with IVR will decrease. However, the implementation costs would have to increase by 235% for the cost savings to be equal to those of the nurse calls, whereas any decrease in the cost of the IVR system would increase its cost savings advantage. An increase or decrease in the percentage of unimproved patients would obviously have a direct effect on both modes of follow-up. If the percentage of unimproved patients in the IVR system condition were actually 20% higher than we estimated or if the unimproved patients in the nurse calls were 20% less, there would still be a greater cost savings for IVR over the nurse calls, but the difference would be less. For there to be no difference in the 2 groups, the percentage of unimproved patients in the IVR group would have to be 47% higher or the percentage in the nurse group would have to be 41% lower. The sensitivity analysis shows that while both means of follow-up can potentially save costs, the savings will be greater with IVR under most reasonable conditions.

DISCUSSION

The nurse-initiated calls not only cost more in every year except the first year, but over the 5-year time horizon the savings from IVR calls are almost 50% greater than for nurse-initiated calls. One reason for the differences is that there are differences in the sources of costs for the IVR and nurse-initiated calls. The vast majority of the costs for IVR calls are in the initial setup. From Year 2 forward, the maintenance cost is assumed to be only $2000 per year. In addition, there are no additional costs for attempts to call patients who are not reached or who hang up. For the nurse-initiated calls, there are annual recurring costs, not only for the time spent on patients who complete the call, most of whom are doing well, but also for time spent calling patients who do not complete the call. Over a 5-year period these costs are higher than those for the initial setup and subsequent maintenance of the IVR system.

12

12

A potential concern with IVR calling is that because so many patients hang up on the call, fewer who need help receive it. The prior study found that the results of the IVR and the human calls were not statistically significantly different in the proportion of patients found to be unimproved if the number of individuals reached, rather than the number completing calls, was used as the denominator.Because of the nature of people’s responses to IVR calls, and given the data that a higher proportion of patients called by the IVR system, compared with a human call, reported being the same or worse, the assumption that those who hang up do not need assistance seems reasonable.In addition, unless a dedicated nurse is assigned to do the follow-up, leaving the calls to a nurse who may be busy with other tasks could lead to more delays in reaching the patients than would occur with the automatic calling of the IVR system.

The impact is that the costs of taking action on the calls (additional nurse time and appointments) are similar for the 2 types of calls, with the cost of actions initiated by IVR calls slightly higher. While a substantial amount of nurse follow-up time is spent with patients who will not need further assistance, a higher proportion of those who complete the IVR calls have new or unresolved problems. In terms of quality assurance, ensuring that patients who need the physician’s attention actually get it, IVR may be a more efficient approach. However, even if the costs incurred after the initial call were identical, IVR would still be less expensive over the 5-year period because of the dif- ferences in the costs of the initial follow-up calls.

Several factors might alter some of the cost estimates that we used, differentially affecting one or the other type of follow-up process. We did not include costs of training the nurses in the process or of training new personnel if there are changes over time. Nurses were used to make all of the telephone calls in this analysis. While a health- care practitioner may need to perform the follow-up call, a non-medically trained, lower-paid employee could perform the work to prepare for the calls and might even make the initial calls. Using lower-paid personnel to per- form these calls would decrease the costs associated with the nurse-initiated calls and increase the potential cost savings, making the nurse-initiated calls more favorable.

Using the IVR system for the calls provides a defined time for the phone calls, while a more extended conversation is likely with the nurse-initiated call. Any increase in the length of the nurse’s call would increase the potential cost savings of the IVR calls compared with the nurse-initiated calls.

14

13

We used national figures for physician feesand nurse’s salaries,but the costs will differ slightly depending on local variations.

If physicians order more diagnostic tests for patients who are worse, there will be additional costs that we did not include, although these costs would be similar for human or IVR follow-up. Similarly, to arrive at our figure of total costs without follow-up, we assumed that all patients will go to a physician, either their personal physician or a new provider. However, if patients were to go to the emer- gency department, the cost would be greater.

Limitations

Differences in our initial assumptions could alter some of the cost estimates. For instance, we assumed for this study that the practice would see 4800 acute care patients per year, resulting in a greater cost savings for the IVR system. As the sensitivity analysis shows, if the practice saw more than that number of patients, the difference in cost savings for IVR would be enhanced, while the practice would have to see fewer than 2000 acute care patients per year to make the cost savings for nurse-initiated calls more favorable over 5 years.

In addition, most of the cost estimates used in this analysis came from data on interaction with the health system from a single study, which could limit generalizability of the precise estimates, especially of the interaction costs. However, the major sources of differences between the IVR and nurse calls were for the initial calls, rather than the subsequent interaction costs, and the sensitivity analysis also showed that the general advantage of IVR is likely to be maintained even with differences as high as 20% in the estimates.

CONCLUSIONS

This study demonstrates cost savings as a difference between the potentially preventable costs for care services and the expenses spent on follow-up calls by either human/nurse or IVR. The results showed that IVR was a more efficient follow-up method than nurse-initiated calls. Although the initial setup costs for IVR raise the costs in the first year, subsequent-year costs are considerably lower, while the costs of nurse-initiated calls remain constant each year. Second, costs are involved in a nurse attempting to make multiple calls to patients who are never reached, and most of the patients who the nurse does reach will not be experiencing problems. Conversely, a significant number of patients who answer and complete the IVR calls are in need of further attention.

The results from this study align well with the incentive structure under the new payment models introduced in an accountable care environment, because the cost savings generated by follow-up/monitoring interventions benefit both the payers and the providers. While the providers incur the operational expenses of conducting the follow-up, follow-up and monitoring practices enable providers to better manage their high-risk patients, prevent rising-risk patients from becoming high-risk, and keep low-risk patients healthy. Early detection of new or unresolved issues and prevention of more acute conditions could specifically result in higher performance on many of the required ACO quality measures, resulting in more possibility for shared savings. In addition, providers can leverage such capability when negotiating commercial ACO-like contracts with private payers. While follow-up in any form is important for these reasons, the analysis provided here indicates that automating the process may be a more efficient mechanism than relying on human follow-up. Thus, providers should consider the use of this type of follow-up to increase potential cost savings in their care delivery process.

Author Affiliations:

Department of Health Services Administration, University of Alabama at Birmingham, (ESB, AP, MNR); Rawls College of Business, Health Organizational Management Program, Texas Tech University, Lubbock (JHB); The Advisory Board Company, Washington, DC (AP).

Source of Funding:

This study was supported by grant number HS017060 from the Agency for Healthcare Research and Quality (Eta S. Berner, EdD, principal investigator).

Author Disclosures:

Dr Panjamapirom is an employee of the Advisory Board Company. Drs Berner and Burkhardt and Ms Ray report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information:

Concept and design (ESB, AP, MNR); acquisition of data (ESB, AP, MNR); analysis and interpretation of data (ESB, JHB, AP); drafting of the manuscript (ESB, JHB); critical revision of the manuscript for important intellectual content (ESB, JHB, AP, MNR); statistical analysis (JHB); obtaining funding (ESB); administrative, technical, or logistic support (ESB, AP, MNR); and supervision (ESB, MNR).

Address correspondence to:

Eta S. Berner, EdD, Professor and Director of the Center for Health Informatics for Patient Safety/Quality, Dept of Health Services Administration, University of Alabama at Birmingham, 1705 University Blvd, SHPB 590J, Birmingham, AL 35294. E-mail: eberner@uab.edu.

1. More partnerships between doctors and hospitals strengthen coordinated care for Medicare beneficiaries [news release]. Baltimore, MD: CMS; December 23, 2013. http://www.cms.gov/Newsroom/ MediaReleaseDatabase/Press-Releases/2013-Press-Releases-Items/2013-12-23.html. Accessed January 26, 2014.

Health Aff (Millwood).

2. Shortell SM, Casalino LP, Fisher ES. How the center for Medicare and Medicaid innovation should test accountable care organizations. 2010;29(7):1293-1298.

3. Sharon Silow-Carroll, Jennifer N. Edwards;The Commonwealth Fund. Early adopters of the accountable care model: a field report on improvements in health care delivery.

http://www.commonwealthfund.org/~/media/Files/Publications/Fund%20Report/2013/Mar/1673_Silow- Carroll_early_adopters_ACO_model.pdf. Published March 2013. Accessed July 21, 2014.

.

4. CMS. Chronic Conditions Among Medicare Beneficiaries, Chartbook: 2012 EditionBaltimore, MD: HHS; 2012.

Health Aff (Millwood).

5.Thorpe KE, Howard DH.The rise in spending among Medicare beneficiaries: the role of chronic disease prevalence and changes in treatment intensity. 2006;25(5):w378-w388.

N Engl J Med.

6. BodenheimerT, Berry-Millett R. Follow the money--controlling expenditures by improving care for patients needing costly services. 2009;361(16):1521-1523.

Preventing Medication Errors: Quality Chasm Series.

7. Committee on Identifying and Preventing Medication Errors. Washington, DC: The National Academies Press; 2007.

J Med Syst.

8. Willig JH, Krawitz M, Panjamapiron A, et al. Closing the feedback loop: an interactive voice response system to provide follow-up and feedback in primary care settings. 2013;37(2):9905.

Am J Manag Care.

9. Smith DH, Feldstein AC, Perrin NA, et al. Improving laboratory monitoring of medications: an economic analysis alongside a clinical trial. 2009;15(5):281-289.

Am J Manag Care.

10. Smith DH, Feldstein AC, Perrin N, et al. Automated telephone calls to enhance colorectal cancer screening: economic analysis. 2012;18(11):691-699.

Am J Manag Care.

11. Haas JS, Klinger E, Marinacci LX, et al. Active pharmacovigilance and healthcare utilization. 2012;18(11):e423-e428.

J Gen Internal Med

12. Berner ES, Ray MN, Panjamapirom A, et al. Exploration of an automated approach for receiving feedback after outpatient acute care visits. . 2014;29(8):1105-1112.

13. Occupational Employment and Wages, May 2012, 29-1141. Registered Nurses, Offices of Physicians. US Department of Labor, Bureau of Labor Statistics website. http://www.bls.gov/oes/2012/may/ oes291141.htm. Updated 2012. Accessed May 19, 2014.

14. Physician Fee Schedule 2012. CMS website. http://www.cms.gov/ apps/physician-fee-schedule/overview.aspx. Updated 2012. Accessed May 19, 2014.