Which Patients Are Persistently High-Risk for Hospitalization?

ABSTRACT

Objectives: Many healthcare systems use prediction models to estimate and manage patient-level probability of hospitalization. Patients identified as high-risk at one point in time may not, however, remain high-risk. We aimed to describe subgroups of patients with distinct longitudinal risk score patterns to inform interventions tailored to patients’ needs.

Study Design: Retrospective national cohort study.

Methods: Using a previously validated prediction algorithm, we identified a cohort of 258,759 patients enrolled in the Veterans Health Administration (VHA) who were in the top 5% of risk for hospitalization within 90 days. During each of the following 24 months, patients were placed in 1 of 6 categories: death, hospitalized, no VHA care, persistently high-risk for hospitalization (≥10% probability), initially high-risk then persistently low-risk (<10% probability), and intermittently high-risk. We used multivariable logistic regression to identify characteristics predictive of being persistently high-risk through the last study month.

Results: After 2 years, 17.7% had died, 13.8% had remained persistently high-risk for hospitalization, 41.5% had become persistently low-risk, and 19.9% were intermittently high-risk. Predictors of being persistently high-risk included urban residence, chronic medical comorbidities, auditory and visual impairment, chronic pain, any cancer diagnosis, and social instability.

Conclusions: Few patients who were high-risk for hospitalization at baseline remained so. Nonrandomized evaluations of interventions that identify patients based on a single high-risk score may spuriously appear to have positive effects. Clinical interventions may need to focus on individuals who are persistently high-risk.

Am J Manag Care. 2019;25(9):e274-e281Takeaway Points

Many patients identified as high-risk for near-term hospitalizations using a risk prediction model at baseline did not remain high-risk over a 2-year period.

• After 2 years, just 14% remained persistently high-risk for hospitalization. Of the remainder, 20% were intermittently high-risk, 42% became persistently low-risk, and 18% had died.
• Predictors of being persistently high-risk included age, gender, urban residence, chronic pain, any cancer diagnosis, chronic medical comorbidities, and social instability.
• Nonrandomized evaluations of interventions that target patients based on a single high-risk score may spuriously appear to improve patient outcomes due to natural decreasing risk over time.

The development of advanced healthcare analytics allows large healthcare organizations to use risk stratification as a population management strategy to proactively identify patients at high risk for poor health outcomes. These high-risk patients may benefit from customized interventions with the goals of improving outcomes and providing healthcare more efficiently.¹ Risk stratification or risk prediction tools are often based on historical administrative claims and utilization data.² However, the stability of risk estimates over time is uncertain. Programs that target patients identified as being high-risk at one point in time do not take into account the dynamic nature of changing risk over time; therefore, such programs may vary in effectiveness across patients.¹ Different clinical interventions may be indicated for patients who are persistently versus temporarily high-risk.^3-5

Studies have supported the notion that many high-cost, high-needs patients are just temporarily high-risk.^3-5 Documented predictors of hospital and emergency department (ED) utilization include schizophrenia, homelessness, opioid prescriptions, depression, substance use, social isolation, and heart failure.^6-8 However, little information exists to characterize patients who remain persistently high-risk for hospitalization. Prediction of persistent high risk over time may enhance risk stratification as a population management strategy and inform efforts to target and tailor resource-intensive clinical interventions.

The Veterans Health Administration (VHA) incorporated risk stratification measures into practice nationally in 2012. It reports weekly estimates of the probability of patient hospitalization in the next 90 days for the entire primary care patient population for use by primary care teams to inform clinical decision making.^9-11 We aimed to inform population management programs for high-risk patients¹² by following a cohort of high-risk patients over a 2-year period. Our objectives were to identify and describe groups of high-risk patients with distinct longitudinal risk score patterns and identify baseline characteristics that predicted which patients were likely to remain high-risk over time.

METHODS

Data Sources

Data for this study were obtained from the VHA Corporate Data Warehouse, which included patient demographics, clinical diagnoses, admission and discharge status, patient risk scores, and utilization records.¹³ We also obtained data from the VHA Vital Status file to ascertain patients’ dates of death.

Study Sample

We identified a national cohort of 258,759 patients who had a 10% or greater absolute probability of 90-day hospitalization on our index date (November 23, 2012), were assigned to a VHA primary care provider, and were not hospitalized in the VHA on the index date, because risk scores are not generated for hospitalized patients. We followed patients for 2 years after the index date.

Measurement of Risk

We obtained patients’ monthly risk using the predicted probability of hospitalization in the next 90 days in the Care Assessment Need (CAN) model, a validated risk prediction algorithm.⁹ The CAN model (eAppendix Table 1 [eAppendix available at ajmc.com]) uses demographics, utilization of VHA health services, comorbidities, prescribed medications, vital signs, and veteran-specific variables.⁹

Risk scores were generated on a weekly basis for all patients in VHA primary care. Patient-month measures of risk were calculated as the mean of weekly probabilities for that calendar month, and missing values occurred when veterans were hospitalized (including acute care, residential treatment facility admission, nursing home admission, or hospice admission) or dropped out of VHA care.

Dependent Variables

We defined patients as “high-risk” if the probability of 90-day hospitalization was 10% or greater, which corresponds to approximately the top 5% of VHA patients (n = 258,759). Our main dependent variable was a patient-month measure of risk persistence, defined hierarchically (Figure 1).

Patients with at least 1 missing CAN probability in a calendar month were categorized into 1 of the following mutually exclusive outcomes: death, hospitalization in VHA, or dropped out of VHA care (no VHA encounters in previous 24 months). By definition, patients can move into and out of the “hospitalization” or “dropped out of care” categories. All patients except those who died were recategorized the following month.

We categorized patients with nonmissing CAN scores as persistently high-risk, intermittently high-risk, or initially high-risk then persistently low-risk for every month based on all patient-month observations in the study period. Patient-month observations were coded as “persistently high-risk” if hospitalization probability was 10% or greater during the current month and all prior months. “Initially high-risk” was defined as probability less than 10% during all subsequent months. “Intermittently high-risk” was defined as having a combination of high (≥10%) and low (<10%) hospitalization risk across the months and not meeting the criteria for persistently high-risk or initially high-risk. These categorizations ignore the patient-months designated as “hospitalization” or “dropped out of care.” By definition, patients can only move out of the persistently high-risk group, patients can only move into the initially high-risk group, and patients may move into or out of the intermittently high-risk group.

Patient Characteristics and Utilization

We examined patient characteristics measured at baseline: demographics (age, gender, marital status, ethnicity, urban vs rural residence), medical and psychiatric comorbidities, any cancer diagnosis, and markers of social instability.¹⁴ Markers of social instability included discharges against medical advice⁴; number of zip code changes in the year prior⁴; International Classification of Diseases, Ninth Revision (ICD-9) diagnosis of nonadherence to medical treatment; and homelessness. Homelessness was measured by use of housing services¹⁵ or ICD-9 diagnosis codes indicating lack of housing. ICD-9 diagnoses for comorbidities were adapted from previously published research¹⁶ and from the VHA Health Economics Resource Center¹⁷ (eAppendix Table 2). Numbers of hospitalizations and outpatient visits and no-show rates were calculated from the index date over the subsequent 2 years. No-shows were recorded for each instance in which a patient did not arrive to a scheduled appointment.¹⁸

For outpatient visits during the follow-up period, we counted VHA encounters that occurred face-to-face or by telephone with any healthcare provider (eg, physician, nurse, psychologist). We categorized outpatient visits based on the specific type of service (ie, primary care, mental health, palliative care, ED).¹⁹ Mental health visits included primary care—mental health integration encounters.²⁰ Inpatient admissions were categorized into medical, surgical, and psychiatric admissions.¹⁹

Analyses

Our dependent variable in bivariable and multivariable analyses was the patient’s risk category (ie, persistently high, intermittently high, or initially high) in the final study month (n = 194,597). We used analysis of variance to examine whether there were differences in means among the continuous variables (inpatient/outpatient utilization, ED visits, and zip code changes) across the high-risk categories. In bivariable analyses, we performed ordered logit regression to estimate a single equation for each independent variable over the 3 ordered levels of the dependent variable: persistently, intermittently, and initially high-risk. Proportional odds ratios (ORs) were generated from the ordered logit models to estimate the change in odds in each predictor for patients at (1) persistently high-risk versus intermittently high-risk and (2) intermittently high-risk versus initially high-risk.

We performed multivariable logistic regression to predict being persistently high-risk versus the other 2 groups after adjusting for independent variables found to be statistically significant (P <.05) from the ordered logit regression analyses described previously. Model covariates included age, gender, and race/ethnicity, and we excluded variables used to calculate the CAN probabilities (eAppendix Table 1).

For continuous independent variables that are not normally distributed (ie, number of discharges against medical advice, palliative care encounters, mental health visits, and zip code changes), we applied a log (base 10) transformation to create a less skewed distribution. As a result, one would need to have a 10-fold increase in the independent variables to achieve the ORs indicated for the transformed variables.

Because the ordered logit model assumes proportional odds, we also estimated a generalized ordered logistic regression in sensitivity analysis. This model estimates ORs for variables that violate the proportional odds assumption separately for each comparison.²¹ OR estimates from the generalized ordered logit model were of similar size and significance as those from the ordered logit model (eAppendix Table 3). In sensitivity analyses, we also used a multinomial logit model to compare patients who were persistently high-risk with those who were intermittently high-risk and to compare those who were persistently high-risk with those who were initially high-risk. Directionality and significance were similar to the ordered logit model (eAppendix Table 4).

We conducted stratified analyses by age group (≥65 years and <65 years) to examine whether there were differences in risk patterns between VHA enrollees eligible for Medicare and younger veterans; results were similar for both age groups (eAppendix Table 5), so we report only the combined results.

Analyses were conducted with SAS version 9.4 (SAS Institute Inc; Cary, North Carolina) and SAS Enterprise Miner.

Institutional Review Board

This evaluation was designed to support VHA operations and quality improvement for the VHA Offices of Primary Care and Clinical Systems Development and Evaluation and was exempt from institutional review board approval or waiver.^22,23

RESULTS

Figure 2 shows the monthly risk status of the high-risk patient cohort over a 24-month period (quantitative results in eAppendix Table 6). After 1 year, 28.6% (n = 74,060) were persistently high-risk for hospitalization, 10.3% (n = 26,541) had died, 23.8% (n = 61,556) were initially high-risk for hospitalization, and 30.6% (n = 79,078) were intermittently high-risk for hospitalization. By the end of the 2-year period, just 13.8% (n = 35,770) remained persistently high-risk for hospitalization; 17.7% (n = 45,805) had died, 41.5% (n = 107,473) were initially high-risk, and 19.9% (n = 51,354) were intermittently high-risk, and the remaining 7.1% (n = 18,357) were hospitalized or left VHA care at study end.

Patients who were older than 45 years, male, unmarried, black, and living in an urban area had greater odds of being persistently high-risk for hospitalization (vs intermittently high-risk) and of being intermittently high-risk for hospitalization (vs initially high-risk) (Table 1 [part A and part B]). Patients with nearly all medical and mental comorbidities had greater odds of being persistently or intermittently high-risk for hospitalization. The largest differences between the risk categories were found with congestive heart failure (OR, 2.60; 95% CI, 2.55-2.65), chronic kidney disease (OR, 2.48; 95% CI, 2.39-2.58), and dementia (OR, 1.96; 95% CI, 1.90-2.00). Patients with social instability, such as discharges against medical advice, number of zip code changes, and diagnoses of nonadherence, had greater odds of being both persistently and intermittently high-risk for hospitalization. Median no-show rates in a given year were similar for all risk categories.

Patients who were persistently high-risk for hospitalization were more likely than those who were intermittently high-risk or initially high-risk to have a VHA hospitalization (89%, 75%, and 41%, respectively) or ED visit (93%, 86%, and 57%, respectively) during the follow-up period (Table 2). Also, persistently high-risk patients had more frequent VHA hospitalizations per year than the initially high-risk group, with a mean (SD) of 1.9 (3.5) hospitalizations per year; most (1.5 [3.1]) of these admissions were medical.

Patients who were persistently high-risk for hospitalization also had higher rates of outpatient utilization: They had a mean (SD) of 83.2 (81.2) total VHA outpatient encounters (telephone and in-person visits) per year, almost double the number of outpatient encounters per year among patients who were only initially high-risk for hospitalization (Table 2). Persistently high-risk patients had a mean (SD) of 10.6 (15.9) encounters in primary care, 18.1 (64.9) encounters in mental health, and 0.1 (1.9) encounters in palliative care per year. In contrast, patients who were only initially high-risk for hospitalization had, on average, 44.4 (58.4) outpatient encounters per year, 6.7 (11.0) in primary care, 9.5 (41.1) in mental health, and 0.02 (0.8) in palliative care. Intermittently high-risk patients had mean utilization in between the persistently high-risk and initially high-risk groups.

In multivariable analyses, we identified 20 statistically significant (P <.05) predictors of being persistently high-risk for hospitalization, including age, gender, urban residence, visual and hearing impairment, chronic pain (back and neck pain, arthritis, and headache), chronic medical comorbidities (chronic kidney disease, coronary artery disease, congestive heart failure, diabetes, hypertension, and nicotine use), number of visits to palliative care, cancer diagnosis, number of mental health visits, and all markers of social instability (Table 3).

DISCUSSION

In this study, we examined the patterns of risk status over time among a population of high-risk patients receiving continuous VHA care. Consistent with findings of prior research on costs and readmissions,^3,24 the majority of these patients did not remain persistently high-risk during 2 years of follow-up; just 29% were persistently high-risk after 1 year, and 14% remained persistently high-risk after 2 years. Almost half (42%) were classified as being persistently low-risk by study end.

These findings may partly explain seemingly positive results from interventions that enroll patients who are high-risk at baseline but that report pre—post findings without a comparison group. The improvement in patient outcomes could, in fact, be due to naturally decreasing risk over time rather than due to the intervention itself. Several randomized studies of care management interventions for high-need, high-cost patients have found no differences in cost and utilization between intervention and usual-care groups.^25-27

We found significant and meaningful differences in socio&shy;demographics, clinical comorbidities, and utilization among the 3 trajectory groupings of high-risk patients. The presence of chronic medical or behavioral conditions (particularly heart failure and chronic kidney disease), a nonadherence diagnosis, and higher VHA utilization rates were associated with greater duration of remaining persistently high-risk for hospitalization, which indicates that programs for high-risk patients can focus efforts on patients with these characteristics. Persistently and intermittently high-risk patients used a remarkable amount of VHA services—as many as an average of 63 to 83 outpatient visits and 0.9 to 1.9 admissions per year&mdash;suggesting that they were actively engaged with ambulatory care providers. Some acute care may be unavoidable despite high utilization of outpatient services, especially for patients who rapidly decompensate or cannot be managed effectively with ambulatory care alone.²⁸ Persistently high-risk patients were more likely to live in urban areas, but many (30%) lived in rural areas, indicating that smaller, rural community clinics or virtual care modalities may have an important role to play in mitigating risk for these patients.

Our results confirm published findings described in other predictive models for high utilization, readmission, and mortality: Age, gender, number of zip code changes, number of discharges against medical advice,^6,7,29 frequent use of outpatient medical and mental health services,⁶ chronic pain,⁶ nonadherence, and tobacco use²⁹ significantly predicted long-term persistence of high risk. Other known predictors of high utilization (eg, socioeconomic status, depression, alcohol use disorder, marital status) were not included in our model due to endogeneity. Unlike previous studies, we report that hearing and visual impairment, cancer diagnosis, and palliative care utilization were also predictors of patients being persistently high-risk. Interestingly, unlike prior studies^6,29 that preceded the VHA comprehensive primary care teams for homeless patients,³⁰ our findings show that homeless patients who used housing support services were less likely to be persistently high-risk. It is possible that VHA programs that address housing instability were able to effectively change the duration for which a patient is high-risk.

Developing effective programs for high-risk patients has proved challenging because most randomized trials in this area have not shown improvement in outcomes.^26,31,32 One possibility is that high-risk patients are so heterogeneous that a one-size-fits-all approach may not be effective. Different clinical interventions may be more suitable for patients who are persistently high-risk versus those who are only initially or intermittently high-risk or for patients with various combinations of factors that predict risk.^33,34 For instance, persistently high-risk patients may benefit from longitudinal primary care programs tailored for specific vulnerable populations (eg, primary care for homeless, homebound, or elderly patients; those receiving dialysis; those with chronic pain or serious mental illness); other high-risk patients may benefit from time-limited care management models that augment primary care, such as collaborative care models for high-needs patients.^26,35 Large healthcare systems may find it helpful to use empiric clustering models to further distinguish subgroups of high-risk patients based on clinical characteristics.^4,5,36,37

Our findings suggest ways to tailor care management interventions for patients with sustained needs. Although the extent to which a patient’s underlying risk for hospitalization is modifiable remains unclear, evidence-based practices can be implemented to possibly reduce a patient’s risk for hospitalization. For instance, interventions to target housing instability, such as the Housing First model, may reduce a patient’s risk for future hospitalization.³⁰ Healthcare teams who target persistently high-risk patients could include behavioral health specialists familiar with managing treatment nonadherence, holistic approaches to chronic pain, and guideline-concordant care for ambulatory care—sensitive conditions (eg, congestive heart failure, diabetes, hypertension). Comprehensive assessments could identify depression and patient factors underlying nonadherence and social instability in high-risk patients and then prioritize which patients should receive the most intensive interventions to reduce future risk. Future studies could examine whether intervening on predictors of persistent high risk is effective in changing a patient’s risk trajectory.

Limitations

Measures of socioeconomic status, including income, employment status, and education status, were likely predictors of risk persistence, but they were not included due to lack of availability in VHA administrative data.²⁹ To avoid a problem in tautology, we excluded VHA risk prediction model variables (eg, mental health conditions, cognitive impairment) from our model to predict whether or not patients remain at high risk. However, we do report some clinical characteristics not included in the risk prediction model that contribute to hospitalization risk, such as chronic pain conditions, markers of social instability, and sensory impairments. Lastly, our sample is limited to the VHA population, which includes veterans who are less likely to have childhood-onset medical conditions due to military medical eligibility, are more likely to have psychiatric conditions, and may use services in multiple healthcare settings, not just the VHA.³⁸ We were unable to draw from non-VHA data sources for this analysis to account for healthcare services that were not paid by the VHA. Although the actual percentage of persistently high-risk patients might differ by payer,³⁹ the overall patterns are likely to be similar.

CONCLUSIONS

Because most high-risk patients may be only briefly high-risk for hospitalization, nonrandomized evaluations of interventions that identify patients based on a single risk score may appear to have positive effects on patient health due to naturally decreasing risk over time in the majority of patients. Identified predictors of patients remaining persistently high-risk could inform efforts to tailor the intensity and type of interventions to improve health for veterans with the most sustained needs.

Acknowledgments

This study was funded by the Veterans Health Administration (VHA) Patient Aligned Care Team Demonstration Laboratory Coordination Center (XVA-61-041). Dr Wong is supported by VHA Career Development Award CDA 13-024. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the University of California, the University of Washington, the University of Pittsburgh, or the University of Michigan.Author Affiliations: Center for the Study of Healthcare Innovation, Implementation and Policy, Veterans Affairs (VA) Greater Los Angeles Health Care System (ETC), Los Angeles, CA; David Geffen School of Medicine, University of California Los Angeles (ETC), Los Angeles, CA; Office of Clinical Systems Development and Evaluation, Veterans Health Administration (RP), Washington, DC; Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System (ESW), Seattle, WA; Department of Health Services (ESW) and Department of Medicine (SDF), University of Washington, Seattle, WA; Center for Health Equity Research and Promotion, VA Pittsburgh (AMR), Pittsburgh, PA; Department of Medicine, University of Pittsburgh (AMR), Pittsburgh, PA; Center for Clinical Management Research, VA Ann Arbor (SV), Ann Arbor, MI; Department of Internal Medicine, University of Michigan (SV), Ann Arbor, MI; VA Health Economics Resource Center (JY), Menlo Park, CA; VA Center for Innovation to Implementation (JY), Menlo Park, CA; Department of General Internal Medicine, University of California San Francisco School of Medicine (JY), San Francisco, CA.

Source of Funding: This study was funded by the Veterans Health Administration Patient Aligned Care Team Demonstration Laboratory Coordination Center (XVA-61-041).

Author Disclosures: Dr Wong reports prior ownership of common stock in UnitedHealth Group Inc. Dr Vijan has a pending VA grant on a similar subject. The remaining authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (ETC, ESW, AMR, SDF, JY); acquisition of data (ETC, RP); analysis and interpretation of data (ETC, RP, ESW, AMR, SV, JY); drafting of the manuscript (ETC, RP, SDF); critical revision of the manuscript for important intellectual content (ETC, RP, ESW, AMR, SDF, SV, JY); statistical analysis (ETC, RP, ESW, SV); provision of patients or study materials (ETC); obtaining funding (ETC); administrative, technical, or logistic support (ETC); and supervision (ETC).

Address Correspondence to: Evelyn T. Chang, MD, MSHS, VA Greater Los Angeles Healthcare System, 11301 Wilshire Blvd, Los Angeles, CA 90073. Email: Evelyn.Chang@va.gov.REFERENCES

1. Hong CS, Hwang AS, Ferris TG. Finding a match: how successful complex care programs identify patients. California Health Care Foundation website. chcf.org/wp-content/uploads/2017/12/PDF-FindingMatchComplexCare.pdf. Published March 2015. Accessed April 1, 2015.

2. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):1688-1698. doi: 10.1001/jama.2011.1515.

3. Johnson TL, Rinehart DJ, Durfee J, et al. For many patients who use large amounts of health care services, the need is intense yet temporary. Health Aff (Millwood). 2015;34(8):1312-1319. doi: 10.1377/hlthaff.2014.1186.

4. Wong ES, Yoon J, Piegari RI, Rosland AM, Fihn SD, Chang ET. Identifying latent subgroups of high-risk patients using risk score trajectories. J Gen Intern Med. 2018;33(12):2120-2126. doi: 10.1007/s11606-018-4653-x.

5. Yoon J, Chee CP, Su P, Almenoff P, Zulman DM, Wagner TH. Persistence of high health care costs among VA patients. Health Serv Res. 2018;53(5):3898-3916. doi: 10.1111/1475-6773.12989.

6. Doran KM, Raven MC, Rosenheck RA. What drives frequent emergency department use in an integrated health system? national data from the Veterans Health Administration. Ann Emerg Med. 2013;62(2):151-159. doi: 10.1016/j.annemergmed.2013.02.016.

7. Raven MC, Billings JC, Goldfrank LR, Manheimer ED, Gourevitch MN. Medicaid patients at high risk for frequent hospital admission: real-time identification and remediable risks. J Urban Health. 2009;86(2):230-241. doi: 10.1007/s11524-008-9336-1.

8. Yoon J, Yano EM, Altman L, et al. Reducing costs of acute care for ambulatory care—sensitive medical conditions: the central roles of comorbid mental illness. Med Care. 2012;50(8):705-713. doi: 10.1097/MLR.0b013e31824e3379.

9. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care. 2013;51(4):368-373. doi: 10.1097/MLR.0b013e31827da95a.

10. Fihn SD, Francis J, Clancy C, et al. Insights from advanced analytics at the Veterans Health Administration. Health Aff (Millwood). 2014;33(7):1203-1211. doi: 10.1377/hlthaff.2014.0054.

11. Nelson KM, Chang ET, Zulman DM, Rubenstein LV, Kirkland FD, Fihn SD. Using predictive analytics to guide patient care and research in a national health system [published online April 22, 2019]. J Gen Intern Med. doi: 10.1007/s11606-019-04961-4.

12. Chang ET, Raja PV, Stockdale SE, et al. What are the key elements for implementing intensive primary care? a multisite Veterans Health Administration case study. Healthc (Amst). 2018;6(4):231-237. doi: 10.1016/j.hjdsi.2017.10.001.

13. Corporate Data Warehouse (CDW). US Department of Veterans Affairs website. hsrd.research.va.gov/for_researchers/vinci/cdw.cfm. Accessed December 17, 2018.

14. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care. 2010;48(11):981-988. doi: 10.1097/MLR.0b013e3181ef60d9.

15. Gabrielian S, Yuan AH, Andersen RM, Rubenstein LV, Gelberg L. VA health service utilization for homeless and low-income veterans: a spotlight on the VA Supportive Housing (VASH) program in greater Los Angeles. Med Care. 2014;52(5):454-461. doi: 10.1097/MLR.0000000000000112.

16. Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol. 2011;64(7):749-759. doi: 10.1016/j.jclinepi.2010.10.004.

17. Yoon J, Scott JY, Phibbs CS, Wagner TH. Recent trends in Veterans Affairs chronic condition spending. Popul Health Manag. 2011;14(6):293-298. doi: 10.1089/pop.2010.0079.

18. VHA Support Service Center (VSSC). Care Assessment Need (CAN) score report. US Department of Veterans Affairs website. https://vssc.med.va.gov/VSSCMainApp/. Accessed March 1, 2017.

19. Hebert PL, Liu CF, Wong ES, et al. Patient-centered medical home initiative produced modest economic results for Veterans Health Administration, 2010-12. Health Aff (Millwood). 2014;33(6):980-987. doi: 10.1377/hlthaff.2013.0893.

20. Post EP, Metzger M, Dumas P, Lehmann L. Integrating mental health into primary care within the Veterans Health Administration. Fam Syst Health. 2010;28(2):83-90. doi: 10.1037/a0020130.

21. Williams R. Generalized ordered logit/partial proportional odds models for ordinal dependent variables. Stata J. 2006;6(1):58-82. doi: 10.1177/1536867X0600600104.

22. VHA Handbook 1058.05, VHA operations activities that may constitute research. US Department of Veterans Affairs website. research.va.gov/resources/policies/oro-120811.cfm. Published 2011. Accessed June 8, 2012.

23. Puglisi T. Reform within the Common Rule? Hastings Cent Rep. 2013;43(S1):S40-S42. doi: 10.1002/hast.140.

24. Fuda KK, Immekus R. Frequent users of Massachusetts emergency departments: a statewide analysis. Ann Emerg Med. 2006;48(1):9-16. doi: 10.1016/j.annemergmed.2006.03.001.

25. Katz MH. Trust but verify (ideally with a randomized clinical trial). JAMA Intern Med. 2017;177(2):162-163. doi: 10.1001/jamainternmed.2016.8433.

26. Zulman DM, Pal Chee C, Ezeji-Okoye SC, et al. Effect of an intensive outpatient program to augment primary care for high-need Veterans Affairs patients: a randomized clinical trial. JAMA Intern Med. 2017;177(2):166-175. doi: 10.1001/jamainternmed.2016.8021.

27. Yoon J, Chang E, Rubenstein LV, et al. Impact of primary care intensive management on high-risk veterans’ costs and utilization: a randomized quality improvement trial. Ann Intern Med. 2018;168(12):846-854. doi: 10.7326/M17-3039.

28. Reuben DB, Borok GM, Wolde-Tsadik G, et al. A randomized trial of comprehensive geriatric assessment in the care of hospitalized patients. N Engl J Med. 1995;332(20):1345-1350. doi: 10.1056/NEJM199505183322007.

29. Calvillo-King L, Arnold D, Eubank KJ, et al. Impact of social factors on risk of readmission or mortality in pneumonia and heart failure: systematic review. J Gen Intern Med. 2013;28(2):269-282. doi: 10.1007/s11606-012-2235-x.

30. O’Toole TP, Buckel L, Bourgault C, et al. Applying the chronic care model to homeless veterans: effect of a population approach to primary care on utilization and clinical outcomes. Am J Public Health. 2010;100(12):2493-2499. doi: 10.2105/AJPH.2009.179416.

31. Peterson GG, Geonnotti KL, Hula L, et al. Association between extending CareFirst’s medical home program to Medicare patients and quality of care, utilization, and spending. JAMA Intern Med. 2017;177(9):1334-1342. doi: 10.1001/jamainternmed.2017.2775.

32. Peterson K, Helfand M, Humphrey L, Christensen V, Carson S. Evidence brief: effectiveness of intensive primary care programs. US Department of Veterans Affairs website. hsrd.research.va.gov/publications/esp/intensive-primary-care.pdf. Published February 2013. Accessed December 4, 2015.

33. Edwards ST, Peterson K, Chan B, Anderson J, Helfand M. Effectiveness of intensive primary care interventions: a systematic review. J Gen Intern Med. 2017;32(12):1377-1386. doi: 10.1007/s11606-017-4174-z.

34. Hochman M, Asch SM. Disruptive models in primary care: caring for high-needs, high-cost populations. J Gen Intern Med. 2017;32(4):392-397. doi: 10.1007/s11606-016-3945-2.

35. Counsell SR, Callahan CM, Buttar AB, Clark DO, Frank KI. Geriatric Resources for Assessment and Care of Elders (GRACE): a new model of primary care for low-income seniors. J Am Geriatr Soc. 2006;54(7):1136-1141. doi: 10.1111/j.1532-5415.2006.00791.x.

36. Prenovost KM, Fihn SD, Maciejewski ML, Nelson K, Vijan S, Rosland AM. Using item response theory with health system data to identify latent groups of patients with multiple health conditions. PLoS One. 2018;13(11):e0206915. doi: 10.1371/journal.pone.0206915.

37. Lee NS, Whitman N, Vakharia N, Taksler DG, Rothberg MB. High-cost patients: hot-spotters don’t explain the half of it. J Gen Intern Med. 2017;32(1):28-34. doi: 10.1007/s11606-016-3790-3.

38. West AN, Charlton ME, Vaughan-Sarrazin M. Dual use of VA and non-VA hospitals by veterans with multiple hospitalizations. BMC Health Serv Res. 2015;15:431. doi: 10.1186/s12913-015-1069-8.

39. Powers BW, Chaguturu SK. ACOs and high-cost patients. N Engl J Med. 2016;374(3):203-205. doi: 10.1056/NEJMp1511131.

Download PDF: Which Patients Are Persistently High-Risk for Hospitalization?