Proactive Care Management of AI-Identified At-Risk Patients Decreases Preventable Admissions

ABSTRACT

Objectives: We assessed whether proactive care management for artificial intelligence (AI)–identified at-risk patients reduced preventable emergency department (ED) visits and hospital admissions (HAs).

Study Design: Stepped-wedge cluster randomized design.

Methods: Adults receiving primary care at 48 UCLA Health clinics and determined to be at risk based on a homegrown AI model were included. We employed a stepped-wedge cluster randomized design, assigning groups of clinics (pods) to 1 of 4 single-cohort waves during which the proactive care intervention was implemented. The primary end points were potentially preventable HAs and ED visits; secondary end points were all HAs and ED visits. Within each wave, we used an interrupted time series and segmented regression analysis to compare utilization trends.

Results: In the pooled analysis of high-risk and highest-risk patients (n = 3007), potentially preventable HAs showed a statistically significant level drop (–27% [95% CI, –44% to –6%]), without any corresponding change in trends. Potentially preventable ED visits did not show a substantial level drop in response to the intervention, although a nonsignificant differential change in trend was observed, with visit rates decelerating 7% faster in the intervention cohorts (95% CI, –13% to 0%). Nonsignificant drops were observed for all HAs (–19% [95% CI, –35% to 1%]; P = .06) and ED visits (–15% [95% CI, –28% to 1%]; P = .06).

Conclusions: A care management intervention targeting AI-identified at-risk patients was followed by a onetime, significant, sizable reduction in preventable HA rates. Further exploration is needed to assess the potential of integrating AI and care management in preventing acute hospital encounters.

Am J Manag Care. 2024;30(11):In Press

Takeaway Points

Proactive care management, guided by artificial intelligence (AI), significantly reduces potentially preventable hospital admissions.

• Following a care management intervention directed at patients identified as at risk through AI, there was a significant onetime reduction in rates of preventable hospital admissions.
• Findings suggest the potential effectiveness of a population health approach that integrates AI and care management in preventing acute hospital encounters.
• Further investigation is needed to determine its potential success in enhancing utilization patterns, reducing costs, and improving both patient experience and health-related quality of life.

US health care expenditures greatly exceed those of other developed countries and are only projected to increase. Acute hospital encounters (inpatient hospital admissions [HAs] and emergency department [ED] visits) contribute substantially to these costs. Almost 26 million HAs occurred in 2017, totaling $434 billion in costs.¹ Of these, approximately 3.5 million (13%) adult HAs were considered potentially preventable, costing almost $34 billion.² In the same year, ED visits in the US cost a total of $76.3 billion.³ In addition to contributing to enormous health care costs, ED visits and HAs pose a significant burden to patients as a large component of out-of-pocket spending.⁴

By implementing a population health approach that applies preventive measures that help address problems before they necessitate an acute hospital encounter, health systems can help to decrease costs and improve patient experience. However, not all patients are at equal risk for acute hospital encounters. Identification of at-risk patients should enable application of additional resources to these patients.

The goal of the UCLA Health Population Risk Model, an artificial intelligence (AI)–based predictive model, was to provide a quantitative risk score of future acute hospital encounters for the approximately 400,000 patients in UCLA Health primary care. We hypothesized that (1) we could apply this quantitative risk score generated from the electronic health record and administrative claims data to identify patients at highest risk for preventable acute hospital encounters and (2) using this tool to reallocate care management resources to prioritize those at greater risk would result in reduced use of preventable acute hospital encounters. The aims of this paper are to describe the use of the UCLA Health Population Risk Model by the ambulatory care management team and present the results of a preplanned analysis of the association between implementation of the model and subsequent hospital utilization.

METHODS

Population Risk Model Design

The UCLA Health Population Risk Model was designed with demographic and clinical data from the electronic health record (Epic), administrative claims data from the health plans, and indirect socioeconomic data using the University of Wisconsin’s Area Deprivation Index scores mapped to each patient. The Area Deprivation Index ranks neighborhoods at the census block group level for socioeconomic disadvantage. The final model included 144 independent variables and was designed to predict predictable HAs and ED visits. The criteria used for predictable HAs were a nontrauma, non–newborn delivery admission in the future 12 months or 2 or more admissions of any kind in the future 12 months. A predictable ED visit was defined as a visit with a principal diagnosis in a CMS Hierarchical Condition Category in the future 12 months or 1 or more ED visits of any kind in the future 12 months. A high proportion of trauma admissions are attributable to chance factors, thus making them difficult to predict. Newborn delivery admissions were excluded because they are not the type of HAs we hoped to prevent. Please see eAppendix A, “Gradient Tree Boosting Model Brief: Hospital Admissions and ED Visits,” for further details regarding model design (eAppendices available at ajmc.com).

Study Population

Patients 18 years or older receiving primary care and attributed to a UCLA Health primary care provider (PCP) at 48 UCLA Health clinics were included. UCLA Health’s primary care attribution model assigns a patient to a practice if a patient meets at least 1 of the following criteria: 2 non–urgent care visits with a PCP in the previous 3 years; 1 non–urgent care visit with a PCP in the previous 3 years and 1 urgent care visit with a PCP in the previous 3 years; more than 1 preventive visit in the past year; or assigned to UCLA Health through a capitation contract such as a commercial health maintenance organization. Care management support was organized through 4 UCLA Health Primary Care regionally based care management pods in Southern California (Westside 1, Westwood and South Bay, Northwest Valley and Motion Picture & Television Fund [MPTF], Westside 2 and Westside 3) (eAppendix Figure).

The at-risk population was defined by the UCLA Population Risk Model as all patients with a predicted future risk of greater than 35% of having an acute hospital encounter over the next 12 months (defined as either a predictable HA or ED visit as above). The at-risk patients were further risk stratified to highest-, high-, and rising-risk categories. The highest-risk category was defined as the top 5% of the total at-risk population. The high-risk category included those patients with a predicted future risk greater than 35%, with 2 or more ED visits and/or 1 or more HAs in the prior year, excluding those related to obstetrics and trauma. The rising-risk category included patients with a predicted future risk greater than 35%, with 0 or 1 ED visits in the prior year and no HAs in the prior year, excluding those related to obstetrics and trauma. The lists of highest-risk, high-risk, and rising-risk patients were updated every 3 months.

Care Management Model to Support At-Risk Population

Our population health management team applied a platform (known as the UCLA Proactive Care Model, described in detail in eAppendix B) to support the UCLA Population Risk Model patients through identification, communication, and outreach as described below.

Highest-risk management. The highest-risk patients had their care managed by a UCLA Health clinical adviser (CA) who was either a registered nurse (RN) or a licensed clinical social worker (LCSW). The CAs were trained on this Proactive Care Model during in-person and virtual sessions. A standard workflow that outlined responsibilities and steps in the care model was created and shared with the team as outlined below.

CA RNs provided support through performing comprehensive chart review, ascertaining the PCP plan of care, and speaking with patients to identify barriers to adherence. CA RNs directed patients to appropriate sites of care and monitored for concerning symptoms and medical nonadherence. CA RNs also noted and addressed polypharmacy and evaluated safety issues such as falls.

CA LCSWs addressed behavioral health issues including depression and elder abuse. CA LCSWs engaged patients in their own care and addressed barriers to care by assisting with needs such as transportation.

High-risk management. High-risk patients were supported by administrative specialists with deep knowledge of the UCLA Health system referred to as comprehensive care coordinators (CCCs), whose focus was on ensuring the patient was following the PCP’s plan of care as well as helping the patient close prioritized care gaps (eg, cancer screening). CCCs also addressed access issues such as transportation needs and prescription refill coordination. CCCs were encouraged to escalate complex patient needs to a CA RN or CA LCSW.

Rising-risk management. Rising-risk patients were supported by clinic-based administrative staff referred to as patient service representatives (PSRs), who were instructed to reach out to the rising-risk patients to address missed appointments, open referrals, open laboratory orders, specialty access scheduling, health maintenance scheduling, home health referrals, and pending authorizations.

Outreach frequency. The CA RNs, CA LCSWs, CCCs, and PSRs aimed to reach out to patients on their respective lists at least once every quarter or as needed for follow-up. In addition to the Proactive Care component, the team managed patients during transitions of care.^5,6

Study Design

We employed a stepped-wedge cluster randomized design, assigning groups of pods to 1 of 4 single-cohort waves during which the Population Risk Model intervention was implemented. Four cohorts of clinics were included in the analysis, with 3 implementing the intervention during the study period and the fourth used as a comparison control in all waves. Within each wave, we used an interrupted time series design to compare utilization trends before and after rollout and between the cohort rolling out the intervention and the cohorts that had not already implemented the intervention in prior waves.

Please see the eAppendix Figure for description of the UCLA Primary Care pods. The UCLA Population Risk Model rollout was as follows:

• Cohort 1, consisting of the Westside 1 pod, started the intervention on February 26, 2019.
• Cohort 2, consisting of the Westwood and South Bay pods, started the intervention on October 7, 2019.
• Cohort 3, consisting of the Northwest Valley and MPTF pods, started the intervention on January 13, 2020 (pushed back 1 week due to the holidays).
• Cohort 4, consisting of the Westside 2 and Westside 3 pods, started the intervention on July 13, 2020 (pushed back due to COVID-19–associated policies).

Outcomes and Data Analysis

The primary end points of the study were potentially preventable HAs and potentially preventable ED visits per 1000 patient-months. Potentially preventable HAs were defined as those not related to trauma or obstetrics. A potentially preventable ED visit was defined as a visit with a principal diagnosis in a CMS Hierarchical Condition Category. We included all HAs when there were 2 or more admissions and all ED visits when there were 2 or more ED visits within the prior 12 months. Secondary end points were all HAs and all ED visits. Segmented regression analysis was performed to estimate changes in the level and trend in each utilization end point before vs after rollout and compare these changes between intervention and control cohorts. Mixed-effects Poisson models were used to model the number of encounters at the patient-month level. Clustering of patient-months by patient and by clinic was modeled using patient and clinic random effects. Models included fixed effects for study arm (intervention vs control) and study period (pre- vs post rollout), a linear time effect (in months), and interactions of these 3 terms. Estimation of level and trend effects within study arms and comparisons across study arms were performed using linear contrasts. Effects were evaluated in terms of incidence rate ratios (IRRs) and 95% CIs. Separate analyses were performed for each wave, including only cohorts that had not rolled out the intervention prior to that wave, to avoid double-counting patient months. A pooled analysis was performed by averaging the results from waves 1 to 3 (on the log IRR scale) and using bootstrap resampling to obtain statistical inferences. Subgroup analyses for ED visits and hospitalizations were performed in each of the eligible risk categories (high risk vs highest risk).

An additional subgroup analysis was performed using the group classified as rising risk to evaluate the less intensive care coordination intervention applied to these patients. The rising-risk analysis was performed using only the postintervention data because event rates were nearly zero by definition in this subset prior to intervention rollout.

Significance was defined as a 95% CI not including 0% change, and all analyses were performed using R 4.1.0 (R Foundation for Statistical Computing).

RESULTS

A summary of results, showing differences in level changes by risk group and acute encounter type, can be found in Table 1. A total of 3007 unique patients were included in the analytic sample (wave 1, cohort 1: 416, cohort 2: 442, cohort 3: 590, cohort 4: 514; wave 2, cohort 2: 378, cohort 3: 316, cohort 4: 499; wave 3, cohort 3: 347, cohort 4: 573 unique patients for the primary analysis of high- and highest-risk patients) (Table 2). The Figure shows ED visits, HAs, potentially preventable ED visits, and potentially preventable HA rates by cohort and wave.

Pooled Analyses of High- and Highest-Risk Patients

In the pooled analysis of high- and highest-risk patients, potentially preventable HAs showed a significant level drop (–27% [95% CI, –44% to –6%]), without any corresponding change in trends. Potentially preventable ED visits did not show a substantial level drop in response to the intervention, although a nonsignificant differential change in trend was observed, with visit rates decelerating 7% faster in the intervention cohorts (95% CI, –13% to 0%) (Table 3).

In the pooled analysis of all (preventable and nonpreventable) acute hospital encounters, a nonsignificant level drop in both HAs and ED visits was observed (HAs: –19% [95% CI, –35% to 1%]; P = .06; ED visits: –15% [95% CI, –28% to 1%]; P = 0.06) (Table 3). There were nonsignificant level reductions observed during each wave of the intervention (HAs: wave 1, –17% [95% CI, –39% to 14%]; P = .24; wave 2, –16% [95% CI, –39% to 16%]; P = .29; wave 3, –24% [95% CI, –48% to 9%]; P = .15; ED visits: wave 1, –13% [95% CI, –30% to 7%]; P = .20; wave 2, –12% [95% CI, –30% to 12%]; P = .29; wave 3, –19% [95% CI, –39% to 5%]; P = .13). Differential changes in trend were not observed in either HAs or ED visits.

In subgroup analyses of all acute hospital encounters, we found that the level drop in HAs was more pronounced in the high-risk patients than in the highest-risk patients (–23% [95% CI, –40% to 1%] vs –5% [95% CI, –41% to 53%]), whereas the level drop in ED visits was qualitatively more pronounced in the highest-risk patients than in the high-risk patients (–24% [95% CI, –48% to 11%] vs –12% [95% CI, –28% to 8%]) (Table 4 [A]). Patients in the rising-risk subgroup saw significant level drops in both HAs and ED visits (HAs: –30% [95% CI, –50% to –2%]; ED visits: –42% [95% CI, –60% to –15%]) but also saw a nonsignificant acceleration in the trend in ED visit rates (5% [95% CI, 0%-11%]).

DISCUSSION

Implementing a population health approach, we utilized an AI-informed care management intervention to tailor patient outreach based on patient risk levels. Employing a stepped-wedge cluster randomized design, our study focused on the impact of the intervention on hospital and ED utilization. The analysis of high- and highest-risk patients revealed a significant reduction in potentially preventable HAs, with no corresponding change in trends. Although potentially preventable ED visits did not significantly decrease, a nearly significant change in the rate of deceleration was observed, indicating a 7% more rapid decline in visit rates in the intervention cohorts. In the pooled analysis of all acute hospital encounters for high- and highest-risk patients, the care management intervention led to a significant onetime reduction in HAs and ED visits, with consistent but nonsignificant reductions observed in subsequent intervention waves. These findings, showing consistent effect sizes and internal replication, strongly support the genuine impact of the program across various time periods and patients. Furthermore, the results of our analyses suggest that the consistent reduction in overall utilization reflect a varying mix of reductions in potentially preventable and nonpreventable ED visits, as well as primarily potentially preventable HAs. In other words, the overall drop in HAs was largely driven by drops in potentially preventable HAs, whereas the drop in ED visits was not principally driven by drops in potentially preventable ED visits (although these did decline more gradually).

When examining the subgroup analyses of the potentially preventable acute hospital encounters, it is interesting that the intervention was followed by a numerically higher drop in preventable HAs in highest-risk compared with high-risk patients but did not substantially lower ED visits for either subgroup. It is possible that the highest-risk patients received more frequent attention from the care team and had enhanced access to the health care system, possibly preventing direct HAs and/or readmissions. Patients in the rising-risk subgroup experienced nonsignificant declines in both preventable ED visits and HAs. This suggests that the outpatient intervention by the care team may be effective in reducing both types of acute hospital encounters in this subgroup, who, by definition, do not fall on the severe end of the illness spectrum.

Furthermore, it is important to note that physicians typically make decisions regarding HAs based on the necessity for hospitalization, which may involve proactive management of underlying disease processes or social factors to improve the management of the condition and reduce hospitalizations. These decisions are often grounded in objective evaluation of medical data, particularly by physicians in settings such as EDs or primary care clinics. This approach may contribute to a decrease in the percentage of patients admitted to hospitals.

In contrast, the decision to visit the ED is more commonly made by the patient (often without consulting their PCP), which may introduce a greater subjective element. Patients consider a variety of factors, including their access to health care (especially outside regular hours), the perceived urgency of their condition, and social factors. This blend of subjective and objective considerations may explain some of the discrepancies observed between the impact on HAs and ED visits. Proactive care coordination interventions for the high-risk patients might have less of an effect on preventing ED visits than hospitalizations due to the differing nature of decision-making processes involved in each setting.

Although we believe this study to be among the first to describe an AI-informed care management intervention in a broad primary care population, it is important to interpret these findings in the context of previously reported interventions targeting more defined patient populations. For example, a trial in which patients with metastatic cancer receiving outpatient chemotherapy were randomly assigned to symptom monitoring (with alerts sent to clinicians for severe or worsening symptoms) vs usual care found that patients receiving intervention were less frequently admitted to the ED (34% vs 41%; P = .02) or hospitalized (45% vs 49%; P = .08).⁷ In addition, patients receiving the intervention remained on chemotherapy longer, had better health-related quality of life, and lived longer.^7,8

The merging of an AI/machine learning model with a population health care management team represents an important step forward in health care. Although the concepts of machine learning and care coordination are not novel, what distinguishes this program is the practical implementation of a machine learning model within the domain of care management. By integrating the model into the care management process, health care teams can leverage the power of data-driven insights and predictive analytics to improve patient outcomes. This novel approach enhances the team’s ability to identify and address gaps in care, optimize resource allocation, and revolutionize health care delivery by harnessing the strengths of both machine learning and care coordination to provide more effective and efficient patient care.

Limitations

Our study has several limitations. These observational data represent a snapshot of UCLA Health primary care clinics during the study period, and examining these same trends over a longer time period would be useful to add more context to our findings. In addition, it was conducted in a single urban academic health system, limiting generalizability.

CONCLUSIONS

The care management intervention targeting patients flagged as at risk by the UCLA Health Population Risk Model was followed by a significant level drop in potentially preventable HAs. In the context of a changing health care system where population management and patient-centeredness take precedence, this proactive, AI-informed care management intervention warrants further investigation into whether and how it might successfully improve utilization patterns, reduce costs, and improve patient experience and health-related quality of life. Although we studied the impact of care management on utilization, the benefits of care management extend to many unmeasurable benefits, including coordinated care, patient education, and patient engagement, as well as other measurable benefits such as access, quality measures, and patient experience measures. Further work to confirm our findings and refine optimal strategies for engaging both providers and patients in preventive measures that improve care should be prioritized. For instance, expanding research to diverse health care settings beyond a single urban academic health system can enhance the generalizability of findings and assess the intervention’s effectiveness across varied populations and contexts. Exploring additional outcome measures beyond hospital and ED utilization, such as patient-centered outcomes including health-related quality of life, patient satisfaction, and cost-effectiveness, can provide a more comprehensive understanding of the intervention’s impact on patient outcomes and health care delivery. Finally, conducting qualitative research to understand patient and provider perspectives on the care management intervention can shed light on factors influencing its effectiveness, acceptance, and implementation barriers. By pursuing these research directions, we can further advance our understanding of AI-informed care management interventions and their potential to optimize health care delivery, improve patient outcomes, and enhance the patient experience. 

Author Affiliations: Department of Radiation Oncology (ACR) and Division of General Internal Medicine and Health Services Research, Department of Medicine (CWV, SSV, CAS), David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA; Office of Population Health & Accountable Care, UCLA Health Faculty Practice Group (NR, HLR, SDM), Los Angeles, CA; now with UCR Health, UC Riverside School of Medicine (NR), Riverside, CA; UCLA Faculty Practice Group and Medical Group, University of California, Los Angeles (SAS), Los Angeles, CA; UCLA Health, University of California, Los Angeles (EAJ), Los Angeles, CA; VA Greater Los Angeles Healthcare System Geriatric Research Education and Clinical Center (CAS), Los Angeles, CA.

Source of Funding: Dr Sarkisian was supported by the National Institutes of Health/National Institute on Aging Midcareer Award in Patient-Oriented Aging Research (1K24AG047899) and National Institutes of Health/National Center for Advancing Translational Sciences via UCLA Clinical and Translational Science Institute (UL1TR001881).

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (NR, SAS, EAJ, HLR, SDM, SSV, CAS); acquisition of data (NR, SAS, EAJ, HLR, SDM); analysis and interpretation of data (ACR, NR, CWV, SAS, SDM, SSV, CAS); drafting of the manuscript (ACR, NR, EAJ, HLR, CAS); critical revision of the manuscript for important intellectual content (ACR, NR, SAS, EAJ, HLR, SDM, SSV, CAS); statistical analysis (CWV, SSV); obtaining funding (EAJ, CAS); administrative, technical, or logistic support (NR, CWV, SAS, HLR, SDM); and supervision (NR, SAS, CAS).

Address Correspondence to: Ann C. Raldow, MD, MPH, Department of Radiation Oncology, David Geffen School of Medicine, University of California Los Angeles, 200 Medical Plaza, Ste B-265, Los Angeles, CA 90095. Email: araldow@mednet.ucla.edu.

REFERENCES

1. Liang L, Moore B, Soni A. National inpatient hospital costs: the most expensive conditions by payer, 2017. Agency for Healthcare Research and Quality statistical brief 261. July 2020. Accessed September 7, 2022. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb261-Most-Expensive-Hospital-Conditions-2017.jsp

2. McDermott KW, Jiang HJ. Characteristics and costs of potentially preventable inpatient stays, 2017. Agency for Healthcare Research and Quality statistical brief 259. June 2020. Accessed September 7, 2022. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb259-Potentially-Preventable-Hospitalizations-2017.jsp

3. Moore BJ, Liang L. Costs of emergency department visits in the United States, 2017. Agency for Healthcare Research and Quality statistical brief 268. December 2020. Accessed September 7, 2022. https://hcup-us.ahrq.gov/reports/statbriefs/sb268-ED-Costs-2017.jsp

4. Cooper Z, Scott Morton F, Shekita N. Surprise! out-of-network billing for emergency care in the United States. J Polit Econ. 2020;128(9):3626-3677. doi:10.1086/708819

5. Coffey C, Greenwald J, Budnitz T, Williams MV, et al. Project BOOST Implementation Guide. Society of Hospital Medicine; 2013. Accessed September 7, 2022. https://www.hospitalmedicine.org/globalassets/professional-development/professional-dev-pdf/boost-guide-second-edition.pdf

6. van Walraven C, Dhalla IA, Bell C, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551-557. doi:10.1503/cmaj.091117

7. Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol. 2016;34(6):557-565. doi:10.1200/JCO.2015.63.0830

8. Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. 2017;318(2):197-198. doi:10.1001/jama.2017.7156