Impact of Emergency Physician–Provided Patient Education About Alternative Care Venues

ABSTRACT

Objectives: Interventions that focus on educating patients appear to be the most effective in directing healthcare utilization to more appropriate venues. We sought to evaluate the effects of mailed information and a brief scripted educational phone call from an emergency physician (EP) on subsequent emergency department (ED) utilization by low-risk adults with a recent treat-and-release ED visit.

Study Design: Patients were randomized into 3 groups for post-ED follow-up: EP phone call with mailed information, mailed information only, and no educational intervention. Each intervention group was compared with a set of matched controls.

Methods: We undertook this study in 6 EDs within an integrated healthcare delivery system. Overall, 9093 patients were identified; the final groups were the phone group

(n = 609), mail group (n = 771), and matched control groups for each (n = 1827 and n = 1542, respectively). Analysis was stratified by age (<65 and ≥65 years). Patients were educated about available venues of care delivery for their future medical needs. The primary outcome was the rate of 6-month ED utilization after the intervention compared with the 6-month utilization rate preceding the intervention.

Results: Compared with matched controls, subsequent ED utilization decreased by 22% for patients 65 years or older in the phone group (P = .04) and by 27% for patients younger than 65 years in the mail group (P = .03).

Conclusions: ED utilization subsequent to a low-acuity ED visit decreased after a brief post-ED education intervention by an EP explaining alternative venues of care for future medical needs. Response to the method of communication (phone vs mail) varied significantly by patient age.

Am J Manag Care. 2018;24(5):225-231Takeaway Points

Emergency department (ED) crowding, especially for low-acuity visits, is a significant public health issue. We studied the impact of providing patients with simple nonmedical education about alternative venues of care following a recent ED visit.

• Patients 65 years or older had a 22% reduction in future ED utilization after phone follow-up by an emergency physician (EP).
• Patients younger than 65 years had a 27% reduction in future ED utilization after receiving mailed educational information.
• Phone follow-up by EPs may be a valuable tool to affect future ED utilization.
• Targeting interventions based upon age-specific responses warrants further study.

Emergency department (ED) crowding is a public health problem that compromises patient care and adversely affects clinical outcomes.^1-3 Low-acuity ED visits place a strain on already crowded EDs and are an expensive source of healthcare utilization and patient cost sharing, especially for conditions that can be managed appropriately in an ambulatory setting.^4,5 Attempts to reduce ED utilization have had variable success,^6-15 although most successful programs emphasize patient education delivered through care coordination and management.^{10,12,13,15-17} Simply providing generic nonmedical information about alternative venues of care other than the ED appears to be effective.^{4,6,10,14,18-20} Most studies have utilized nonphysicians, such as nurses, case managers, discharge planners, and pharmacists, to provide this information to patients.^{6,8,10,12,13,15-17,19,21,22}

We hypothesized that providing patients with a simple educational intervention on available resources and venues of care within our organization could lead to reductions in future ED utilization for low-acuity problems. Our primary goal was to evaluate the impact of a brief educational phone call by an emergency physician (EP) and/or mailed information following a treat-and-release ED visit on subsequent 6-month ED utilization in a randomized population of low-risk, low-acuity adult patients.²³ We also sought to assess the impact of these interventions on utilization of the organization’s Advice and Appointment Call Center (AACC) and outpatient visits. We made an a priori hypothesis that the effects of these interventions might differ by age group (<65 vs ≥65 years).

METHODS

Setting

This study was approved by the Kaiser Permanente Northern California (KPNC) Institutional Review Board for the Protection of Human Subjects, which has jurisdiction over all facilities included in this report.

We conducted this multicenter, randomized, controlled trial from October 2014 through July 2015 within KPNC, an integrated healthcare delivery system. Under KPNC’s mutual exclusivity agreement, approximately 9500 physicians in The Permanente Medical Group, Inc, care for 4.1 million Kaiser Foundation Health Plan (KFHP), Inc, members at 21 hospitals and more than 200 outpatient clinics.^24-26 The Epic (Epic Systems; Verona, Wisconsin) electronic health record, known internally as Kaiser Permanente HealthConnect, was fully deployed in all KPNC facilities in 2010. Members account for approximately 33% of the insured population in Northern California and are representative of the geographic areas served.²⁴ KPNC’s secure online services allow patients to access their medical information, refill prescriptions, make appointments, and communicate with their providers via email.^25-27 The KPNC AACC, which includes staffing by EPs, handles approximately 12 million calls per year.²⁸ The AACC provides healthcare advice, appointment scheduling, and messaging with primary care providers. Patients who need additional medical evaluation are directed to the most appropriate venue of care, including the ED. KPNC has 21 EDs that receive more than 1 million visits per year. We included patients with treat-and-release visits to 6 EDs with a range of annual censuses: 2 higher-census EDs (63,000 and 81,000 visits/year), 2 medium-census EDs (46,000 and 49,000 visits/year), and 2 lower-census EDs (26,000 and 27,000 visits/year).

Physician Selection

Two EPs from each of 6 KPNC EDs volunteered to participate in the study. Study EPs received standardized training from the principal investigator (PBP) about the study, its goals, and their roles in contacting patients by phone. Training included reviews of a standardized phone script (eAppendix A [eAppendices available at ajmc.com]), a phone log (eAppendix B), and postcontact letters (eAppendices C and D) and an information pamphlet that were mailed to each intervention group. The principal investigator fielded queries from the EPs and oversaw the entire enrollment process with the project manager (MNG).

Patient Identification and Selection

We identified patients who met the following inclusion criteria: (1) 18 years or older, (2) KFHP membership, and (3) a low-acuity treat-and-release ED visit during October-November 2014 but without AACC contact in the 24 hours prior to their ED visit in 1 of the 6 KPNC EDs (eAppendix E). We defined low-acuity as 1) having a Laboratory-based Acute Physiology Score, version 2 (LAPS2) score—an acute physiology score based on 16 laboratory tests, vital signs, pulse oximetry, and neurological status in the preceding 72 hours&mdash;less than 50 at the time of the ED index visit, a score associated with a 30-day mortality risk less than 1.5%²⁹; and 2) being discharged directly home from the ED. Patients who left against medical advice, were discharged to a skilled nursing facility or long-term acute care facility, or were transferred to a non—health plan facility were excluded. We also excluded non–English-speaking patients, those who could not respond on their own or through a family member or guardian, and those who died during the 6-month postintervention period (eAppendix E). We conducted separate analyses for patients younger than 65 years and those 65 years or older. Lastly, in our capitated prepaid healthcare system, patients are not obligated to follow recommended care paths and there are no sanctions (economic or otherwise) for patients who choose different venues of care.

Interventions

We identified 3 arms for this study: phone and mail intervention (phone group), mail intervention only (mail group), and no intervention (control group). Prior to the start of this study, our study statistician (PK) identified a target of 600 patients for each of the 2 intervention arms. From a previous pilot study,²³ we had learned that we were successful in reaching 50% of patients by phone in a short time frame (within 2 weeks). Additionally, we needed to sample enough patients so that study physicians would reach 50% of patients who were younger than 65 years and 50% of patients who were 65 years or older. To ensure we would be successful in reaching 600 patients, we needed to sample twice that number. For the first arm, each study EP received a weekly list of 25 eligible patients seen at their medical center ED who were randomly assigned to the phone group. EPs contacted patients by phone within 2 weeks of their index ED visit. EPs worked their way through the list of patients until they had enrolled half of these patients each week to a total of 50 patients contacted for the study period. If a patient did not consent to be interviewed, was unavailable to talk, or could not be reached by phone, the EP moved on to the next patient on the list, a method of patient selection previously described.³⁰ The target was for 50% of contacted patients to be younger than 65 years and 50% of contacted patients to be 65 years or older. This weighting of patients was part of the study design because we postulated that older patients (≥65 years) utilized the ED at a different rate than younger ones.

For arm 1 (phone group), we developed a detailed phone script that described services available through the organization’s AACC and online services.^27,28 The conversation opened with patient consent and closed with an opportunity for feedback and questions. Each EP maintained a phone log of their calls to document basic information about their encounter. We mailed information about the organization’s AACC and online services to this phone group. We mailed information to arm 2 patients (mail group) who had no phone contact with a study EP. The third arm (no-intervention group) was not called or sent mailed information.

Data Collection

During the 6 months following the index ED visit, we captured data on 3 dependent variables: ED visits, outpatient visits, and AACC contacts (multiple calls on the same day were grouped together and counted as 1 contact). We extracted the following independent variables from KPNC clinical and research databases using previously described methods³¹: age, sex, LAPS2, Charlson Comorbidity Index score³², and COmorbidity Point Score, version 2 (COPS2), a longitudinal score based on all patient diagnoses incurred in the preceding 12 months²⁹ (Table 1). We ascertained mortality in the 6 months following the index ED visit (primary source of mortality data was the California Department of Health Services, which maintains a registry of deaths and their causes).²⁹

Before study interventions began, unknown to the EPs making calls or to the research assistants sending out mailings, 2 clerical errors occurred: (1) 300 patients not slated to receive a phone call were placed into the phone intervention group, and (2) the selection process from a spreadsheet led to an excess of patients 65 years or older in the mail group. To mitigate this difference, patients in each of the 2 intervention groups were separately matched to patients who did not receive any intervention. These 2 groups of matched control patients were obtained using the designmatch package in R, which utilizes the Gurobi optimization solver to construct optimally matched samples.³³ For our matching algorithm, we specified that matched patients should not differ by more than 0.1 SD in any of the following variables: age, LAPS2, COPS2, and the numbers of preintervention ED visits, AACC contacts, and outpatient visits. We also required that the matched groups had the same proportions of male patients and patients 65 years or older. Once we specified these criteria, we employed the designmatch algorithm to provide as many matched controls as possible for each case. The algorithm identified 3 matched controls for every phone intervention patient and 2 matched controls for every mail intervention patient.

Analysis

Patients who died before reaching the end of the study period were removed from analysis. The overall mortality rate for the cohort during the 6-month interval after the index ED visit was 5% or lower (Table 1), and sensitivity analyses showed no connection between intervention group placement and likelihood of early death.

For each of the 3 dependent variables, we modeled the outcome frequency in the 6 months after the index ED visit as a function of LAPS2, COPS2, age at admission, age category (<65 and ≥65 years), sex, outcome frequency in the 6 months prior to the intervention, and intervention group. We analyzed the data using generalized linear models and evaluated the fit of Poisson, negative binomial, and normal (log linked) distribution assumptions for all 3 variables. We selected the model predictors using backward selection with P values >.05 as removal criteria. Because of the skewness of the data, we first determined model fit and only then applied the model to the age group—separated cohorts. We employed a combination of mean average error, root mean squared error, deviance, and the Akaike information criterion³⁴ to compare and identify the model that best fit the data.

RESULTS

We identified 9093 low-acuity treat-and-release adult ED patients who were eligible for the study (Figure 1). After randomization and matching, our final study cohort consisted of the following groups: phone intervention (n = 609) with their matched controls (n = 1827) and mail intervention (n = 771) with their matched controls (n = 1542) (Figure 2). The control groups were well matched with each of the 2 comparison intervention groups (Table 1).

The Poisson distribution assumption was the best fit for ED visit frequency and the normal distribution best fit the AACC contact and outpatient visit count. We found that age group interaction was significant for all outcomes, so we fit separate models by age group for each outcome using the same predictors. As a result, we generated an estimate of the effect of each intervention for each metric by age group.

We directly compared the phone intervention with mailed information group with their matched controls. Likewise, we directly compared the mail-only intervention group with their matched controls. We did not compare the 2 intervention groups with each other, as those 2 groups were not matched for demographics.

We found no significant differences in the combined age groups for phone or mailed information interventions compared with their matched controls. Also, no postintervention changes were found in the following: phone group ED utilization compared with matched controls for all patients and those younger than 65 years, mail group ED utilization compared with matched controls for all patients and those 65 years or older, AACC use or outpatient visit frequency for either intervention in both age groups, and overall ED visits, AACC contacts, and outpatient visits during the 6 months prior to and the 6 months following the index ED visits for the 9093 patients (Table 2). The vast majority of study patients had 2 or fewer ED visits before and after the intervention month, a highly skewed utilization distribution (eAppendix F).

We saw significant decreases in ED revisits that varied by age group and intervention type. For patients 65 years or older, phone intervention was associated with 0.78 times the number of ED revisits for control patients (95% CI, 0.62-0.99; P = .04), a 22% relative decrease (Table 2). For patients younger than 65 years, mail intervention was associated with 0.73 times the number of ED revisits for control patients (95% CI, 0.55-0.98; P = .03), a 27% relative reduction.

DISCUSSION

We found that future ED utilization was reduced by 22% for low-risk patients 65 years or older after a brief phone call from EPs followed by mailed information about AACC and online KPNC services for future medical care needs. ED utilization was reduced by 27% for low-risk patients younger than 65 years who were only sent mailed information.

Educating patients about how best to access their healthcare system can affect demand and the choices that patients make to access future care.^10,18 Focusing on ED-based care interventions that interface with outpatient care appears to be most effective in reducing ED utilization.¹⁷ Our postvisit phone and mail education interventions provided simple but specific patient education about access points for patients’ future healthcare needs. Education, case coordination and management, and linkage with primary care have been shown to be effective strategies to reduce ED utilization.^4,9,10,12-17 Similarly, discharge interventions are most effective when combined across the hospital—home interface,^13,18 with phone follow-up specifically being highly correlated with success.^6,10,16 Further, the recent concept of patient-centered medical homes has been associated with reductions in ED utilization.^11,15,35 Our simple educational interventions linked patients to their medical homes after ED discharge, favorably impacting future ED utilization.

EPs served as educational advocates via phone follow-up, a practice that has received little research attention. Similar research to reduce healthcare utilization has been undertaken with outreach by nonphysician personnel (eg, nurses, case managers, healthcare advocates, discharge planners, pharmacists, social workers).^6,10,13,22 In a preliminary pilot study within our organization, we found that future ED utilization was significantly reduced when EPs made postvisit phone calls, but no change occurred when emergency nurses made similar contact, suggesting that patients were perhaps more receptive to receiving information from EPs. Physician costs to provide phone education would be higher when compared with nonphysician staff costs, although specific cost and complexity details of many nonphysician staff intervention programs have not been well described. Following brief training, physicians functioned independently, alleviating the need to create complex, expensive mechanisms for nonphysician staffing to provide postvisit education. Based on the knowledge that the use of telemedicine can get patients to the “right place” before they even step foot into the ED,^23,36 perhaps higher physician costs can be offset by utilizing such telemedicine capabilities as we did in our study. Future studies to identify the differences in follow-up that involve physicians compared with nonphysician staff would be of great interest.

Although phone response rates have been shown to be high for all ages,^37,38 a generational divide in technology use has been described.^39-41 Internet usage is increasing among the elderly,^42-44 even though they may prefer more traditional paper-based services over Web-based services compared with the younger population.⁴² In our study, patients of different ages responded differently to our 2 methods of communication in terms of future ED utilization: Older patients more often answered phone calls, whereas younger patients were more difficult to reach by phone. Younger patients appeared to respond more to mailed information. These were interesting findings that are not well described in the literature. Studies have shown that older patients are readily contacted by phone (46%-69% with a single call and 79%-86% after 3 or 4 attempts),^21,45 perhaps indicating that they may be more receptive to phone education; in our study, older patients were more easily contacted than younger patients. The age-specific responses we found are intriguing with regard to the effects of different interventions. Future research to investigate this disparity in follow-up communication modalities would be helpful to delineate more specific strategies that can better target patient education for ED utilization.

Identifying the optimal population of ED patients for study on the effects of interventions on future ED utilization is fraught with challenges. Although many studies on reducing ED utilization have focused on high-utilizing ED patients,^7,8,15,20 frequent ED usage may be short-lived, often due to an intense but temporary need for recurrent ED services, with almost 75% of high utilizers returning to a general baseline of extremely low ED utilization within 1 year.⁴⁶ Additionally, although it may have limited our effect size, the decision not to limit our study to high utilizers makes our findings more generalizable to the overall population in which the vast majority of patients have few, if any, ED visits each year, something our study identified. Finally, we uniquely employed a validated acute physiology score to select treat-and-release ED patients with a low predicted 30-day mortality risk.²⁹ Using such tools to more objectively identify low-acuity patients most likely to benefit from future outpatient management options may be a worthwhile strategy for future studies.

Limitations

Our study has several limitations. The study was conducted in an integrated healthcare organization with an AACC and online healthcare services, which may not be generalizable to organizations that lack these resources. However, given that integrated systems are becoming more common, the strategies we describe may be useful.^26,27

Only patients who completed the full intervention were included in the phone group. Phone group patients with partial interventions were not included in that group (ie, if a call was not made [n = 466] or information was not mailed [n = 4]). From the phone group, patients who were not contacted by an EP but who were mailed information were placed into the mail intervention group (n = 240).

Although it would have been interesting to compare the 2 intervention groups directly with each other, this was not possible because the demographics of these 2 groups were not matched. Consequently, each intervention group was compared only with its matched control group. Future studies that directly compare the effects of phone follow-up and mailed information follow-up on subsequent ED utilization would be useful.

The clerical errors described earlier resulted in imbalanced groups. Therefore, we employed a process in which patients in each of the 2 intervention groups were separately matched to patients who did not receive any intervention. We believe that we alleviated bias from these early sampling errors by our matching and risk-adjusted modeling practices, which resulted in similar populations of patients in the subsequent matched groups (Table 1).

Our study has identified very interesting results. However, given the limitations of the study design, its power, and the differences between the 2 age group cohorts, our conclusions will need further replication.

CONCLUSIONS

We found that ED utilization for low-acuity treat-and-release adult patients with average preintervention ED use patterns was reduced by 22% to 27% when EPs provided simple educational information by phone and/or when information was mailed to patients about non-ED options for managing their future medical care needs. We found that responses to phone and mail interventions varied by patient age, suggesting a role for targeting specific interventions based on age. Identifying the optimal ED patient population for interventions targeting ED utilization for low-risk situations is an area that deserves further study.&ensp;

Acknowledgments

The authors thank Debbie Amaral, Jason Yang, and Dr Troy Falck at the Kaiser Permanente Northern California Advice and Appointment Call Center for their assistance identifying patients for this study.

They appreciate the contributions from each of the following 12 emergency physicians who participated actively to make phone contact with their assigned patients and maintained phone logs of their work: Drs Christine Roland, Joanna Osuga, John Dani, Gus Garmel, James Kirchberg, Jonathan Rosenson, Donald Miller, Stephen Fuette, Robert Fan, Aaron Goldfarb, Orin Eddy, and Kara Takeuchi.Author Affiliations: Department of Emergency Medicine, The Permanente Medical Group, Kaiser Permanente Medical Centers (PBP, DRV), Sacramento and Roseville, CA; Kaiser Permanente Division of Research (DRV, MNG, DAW, PK, VL, GJE), Oakland, CA.

Source of Funding: Financial support for this study came from Garfield Memorial National Research Fund (Oakland, CA); The Permanente Medical Group, Inc; and Kaiser Foundation Hospitals, Inc. Dr Liu was supported by NIH K23GM112018.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (PBP, DRV, PK, VL, GJE); acquisition of data (PBP, MNG, PK, GJE); analysis and interpretation of data (PBP, MNG, DAW, PK, VL, GJE); drafting of the manuscript (PBP, DRV, MNG, DAW, PK, GJE); critical revision of the manuscript for important intellectual content (PBP, DRV, DAW, PK, VL, GJE); statistical analysis (PBP, DAW, PK); provision of patients or study materials (PBP); obtaining funding (PBP, DRV, GJE); administrative, technical, or logistic support (PBP, MNG, GJE); and supervision (PBP, GJE).

Address Correspondence to: Pankaj B. Patel, MD, Department of Emergency Medicine, Kaiser Permanente Roseville Medical Center, 1600 Eureka Rd, Roseville, CA 95661. Email: pankaj.patel@kp.org.REFERENCES

1. Institute of Medicine. Hospital-Based Emergency Care: At the Breaking Point. Washington, DC: The National Academies Press; 2006.

2. Bernstein SL, Aronsky D, Duseja R, et al; Society for Academic Emergency Medicine, Emergency Department Crowding Task Force. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):1-10. doi: 10.1111/j.1553-2712.2008.00295.x.

3. Sun BC, Hsia RY, Weiss RE, et al. Effect of emergency department crowding on outcomes of admitted patients. Ann Emerg Med. 2013;61(6):605-611.e6. doi: 10.1016/j.annemergmed.2012.10.026.

4. Morgan SR, Chang AM, Alqatari M, Pines JM. Non-emergency department interventions to reduce ED utilization: a systematic review. Acad Emerg Med. 2013;20(10):969-985. doi: 10.1111/acem.12219.

5. Weinick RM, Burns RM, Mehrotra A. Many emergency department visits could be managed at urgent care centers and retail clinics. Health Aff (Millwood). 2010;29(9):1630-1636. doi: 10.1377/hlthaff.2009.0748.

6. Crocker JB, Crocker JT, Greenwald JL. Telephone follow-up as a primary care intervention for postdischarge outcomes improvement: a systematic review. Am J Med. 2012;125(9):915-921. doi: 10.1016/j.amjmed.2012.01.035.

7. Althaus F, Paroz S, Hugli O, et al. Effectiveness of interventions targeting frequent users of emergency departments: a systematic review. Ann Emerg Med. 2011;58(1):41-52.e42. doi: 10.1016/j.annemergmed.2011.03.007.

8. Kumar GS, Klein R. Effectiveness of case management strategies in reducing emergency department visits in frequent user patient populations: a systematic review. J Emerg Med. 2013;44(3):717-729. doi: 10.1016/j.jemermed.2012.08.035.

9. Flores-Mateo G, Violan-Fors C, Carrillo-Santisteve P, Peiró S, Argimon JM. Effectiveness of organizational interventions to reduce emergency department utilization: a systematic review. PLoS One. 2012;7(5):e35903. doi: 10.1371/journal.pone.0035903.

10. Racine AD, Alderman EM, Avner JR. Effect of telephone calls from primary care practices on follow-up visits after pediatric emergency department visits: evidence from the Pediatric Emergency Department Links to Primary Care (PEDLPC) randomized controlled trial. Arch Pediatr Adolesc Med. 2009;163(6):505-511. doi: 10.1001/archpediatrics.2009.45.

11. Reid RJ, Johnson EA, Hsu C, et al. Spreading a medical home redesign: effects on emergency department use and hospital admissions. Ann Fam Med. 2013;11(suppl 1):S19-S26. doi: 10.1370/afm.1476.

12. Bristow DP, Herrick CA. Emergency department case management: the dyad team of nurse care manager and social worker improve discharge planning and patient and staff satisfaction while decreasing inappropriate admissions and costs: a literature review. Lippincotts Case Manag. 2002;7(3):121-128.

13. Mistiaen P, Francke AL, Poot E. Interventions aimed at reducing problems in adult patients discharged from hospital to home: a systematic meta-review. BMC Health Serv Res. 2007;7:47. doi: 10.1186/1472-6963-7-47.

14. DeVries A, Li CH, Oza M. Strategies to reduce nonurgent emergency department use: experience of a northern Virginia employer group. Med Care. 2013;51(3):224-230. doi: 10.1097/MLR.0b013e3182726b83.

15. Raven MC, Kushel M, Ko MJ, Penko J, Bindman AB. The effectiveness of emergency department visit reduction programs: a systematic review. Ann Emerg Med. 2016;68(4):467-483.e15. doi: 10.1016/j.annemergmed.2016.04.015.

16. Reinius P, Johansson M, Fjellner A, Werr J, Ohlén G, Edgren G. A telephone-based case-management intervention reduces healthcare utilization for frequent emergency department visitors. Eur J Emerg Med. 2013;20(5):327-334. doi: 10.1097/MEJ.0b013e328358bf5a.

17. Katz EB, Carrier ER, Umscheid CA, Pines JM. Comparative effectiveness of care coordination interventions in the emergency department: a systematic review. Ann Emerg Med. 2012;60(1):12-23.e1. doi: 10.1016/j.annemergmed.2012.02.025.

18. Pines JM, Lotrecchiano GR, Zocchi MS, et al. A conceptual model for episodes of acute, unscheduled care. Ann Emerg Med. 2016;68(4):484-491.e3. doi: 10.1016/j.annemergmed.2016.05.029.

19. Goodman RM. ED use associated with primary care office management. Am J Manag Care. 2013;19(5):e185-e196.

20. Emergency Severity Index (ESI): a triage tool for emergency department. Agency for Healthcare Research and Quality website. ahrq.gov/professionals/systems/hospital/esi/esi1.html. Published November 2012. Updated November 2014. Accessed April 5, 2018.

21. Jones JS, Young MS, LaFleur RA, Brown MD. Effectiveness of an organized follow-up system for elder patients released from the emergency department. Acad Emerg Med. 1997;4(12):1147-1152. doi: 10.1111/j.1553-2712.1997.tb03698.x.

22. Dudas V, Bookwalter T, Kerr KM, Pantilat SZ. The impact of follow-up telephone calls to patients after hospitalization. Am J Med. 2001;111(9B):26S-30S. doi: 10.1016/S0002-9343(01)00966-4.

23. Patel PB, Vinson DR. Emergency physicians seek…and are finding their place. Ann Emerg Med. 2016;68(3):397-398. doi: 10.1016/j.annemergmed.2016.05.005.

24. Gordon NP. Similarity of the adult Kaiser Permanente membership in Northern California to the insured and general population in Northern California: statistics from the 2011 California health interview survey. Kaiser Permanente website. divisionofresearch.kaiserpermanente.org/projects/memberhealthsurvey/SiteCollectionDocuments/chis_non_kp_2011.pdf. Published June 19, 2015. Accessed April 5, 2018.

25. Selevan J, Kindermann D, Pines JM, Fields WW. What accountable care organizations can learn from Kaiser Permanente California’s acute care strategy. Popul Health Manag. 2015;18(4):233-236. doi: 10.1089/pop.2014.0157.

26. Pines J, Selevan J, McStay F, George M, McClellan M. Kaiser Permanente — California: a model for integrated care for the ill and injured. The Brookings Institution website. brookings.edu/wp-content/uploads/2016/07/KaiserFormatted_150504RH-with-image.pdf. Published May 4, 2015. Accessed April 5, 2018.

27. Silvestre AL, Sue VM, Allen JY. If you build it, will they come? the Kaiser Permanente model of online health care. Health Aff (Millwood). 2009;28(2):334-344. doi: 10.1377/hlthaff.28.2.334.

28. Conolly P, Levine L, Amaral DJ, Fireman BH, Driscoll T. TPMG Northern California appointments and advice call center. J Med Syst. 2005;29(4):325-333.

29. Escobar GJ, Gardner MN, Greene JG, Draper D, Kipnis P. Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446-453. doi: 10.1097/MLR.0b013e3182881c8e.

30. Auerbach AD, Kripalani S, Vasilevskis EE, et al. Preventability and causes of readmissions in a national cohort of general medicine patients. JAMA Intern Med. 2016;176(4):484-493. doi: 10.1001/jamainternmed.2015.7863.

31. Escobar GJ, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and postdischarge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916-923. doi: 10.1097/MLR.0000000000000435.

32. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613-619. doi: 10.1016/0895-4356(92)90133-8.

33. Package ‘designmatch.’ R Project website.&thinsp;cran.r-project.org/web/packages/designmatch/designmatch.pdf. Updated May 1, 2017. Accessed April 5, 2018.

34. Hardin JW, Hilbe JM. Generalized Linear Models and Extensions. 3rd ed. College Station, TX: Stata Press; 2012.

35. Pines JM, Keyes V, van Hasselt M, McCall N. Emergency department and inpatient hospital use by Medicare beneficiaries in patient-centered medical homes. Ann Emerg Med. 2015;65(6):652-660. doi: 10.1016/j.annemergmed.2015.01.002.

36. Telehealth: helping hospitals deliver cost-effective care. American Hospital Association website. aha.org/content/16/16telehealthissuebrief.pdf. Published April 22, 2016. Accessed April 5, 2018.

37. Nota SP, Strooker JA, Ring D. Differences in response rates between mail, e-mail, and telephone follow-up in hand surgery research. Hand (N Y). 2014;9(4):504-510. doi: 10.1007/s11552-014-9618-x.

38. Garcia I, Portugal C, Chu L-H, Kawatkar AA. Response rates of three modes of survey administration and survey preferences of rheumatoid arthritis patients. Arthritis Care Res (Hoboken). 2014;66(3):364-370. doi: 10.1002/acr.22125.

39. Newitz E. Email is the new generation gap. Gizmodo website. gizmodo.com/email-is-the-new-generation-gap-1743697716. Published November 27, 2015. Accessed April 5, 2018.

40. Weiss S. Generation gap: how technology has changed how we talk about work. Forbes. May 16, 2012. forbes.com/sites/ciocentral/2012/05/16/generation-gap-how-technology-has-changed-how-we-talk-about-work. Accessed April 5, 2018.

41. Anderson M. For vast majority of seniors who own one, a smartphone equals “freedom.” Pew Research Center website. www.pewresearch.org/fact-tank/2015/04/29/seniors-smartphones. Published April 29, 2015. Accessed April 5, 2018.

42. Horevoorts NJE, Vissers PAJ, Mols F, Thong MSY, van de Poll-Franse LV. Response rates for patient-reported outcomes using web-based versus paper questionnaires: comparison of two invitational methods in older colorectal cancer patients. J Med Internet Res. 2015;17(5):e111. doi: 10.2196/jmir.3741.

43. Rainie L. Senior citizens and digital technology. Pew Research Center website. pewinternet.org/2012/09/15/senior-citizens-and-digital-technology. Published September 15, 2012. Accessed April 5, 2018.

44. Narasimha S, Madathil KC, Agnisarman S, et al. Designing telemedicine systems for geriatric patients: a review of the usability studies. Telemed J E Health. 2017;23(6):459-472. doi: 10.1089/tmj.2016.0178.

45. Poncia HD, Ryan J, Carver M. Next day telephone follow up of the elderly: a needs assessment and critical incident monitoring tool for the accident and emergency department. J Accid Emerg Med. 2000;17(5):337-340. doi: 10.1136/emj.17.5.337.

46. Johnson TL, Rinehart DJ, Durfee J, et al. For many patients who use large amounts of health care services, the need is intense yet temporary. Health Aff (Millwood). 2015;34(8):1312-1319. doi: 10.1377/hlthaff.2014.1186.

Download PDF: Impact of Emergency Physician&ndash;Provided Patient Education About Alternative Care Venues