Publication
Article
The Effectiveness of a Care Coordination Home Telehealth Program for Veterans With Diabetes Mellitus: A 2-Year Follow-up
Author(s):
Objectives: To assess healthcare use among veterans with diabetes mellitus (DM) enrolled in a Department of Veterans Affairs (VA) Care Coordination Home Telehealth (CCHT) program during 24 months and to contrast this utilization with the service use of a comparison group of veterans with DM not enrolled in the program.
Study Design: Two-year, retrospective, concurrent matched cohort study design.
Methods: The VA CCHT program included older veterans with type 2 DM at high risk for multiple VA inpatient and outpatient visits. Healthcare utilization (hospitalizations, length of stay, and outpatient visits by type) was assessed at baseline and at 24 months after intervention for the treatment (n = 400) and comparison (n = 400) groups. Propensity scores were used to improve the balance between the treatment and comparison groups. A difference-in-differences approach was used to control for selection bias and for intervening time factors.
Results: Two years after enrollment, the treatment group exhibited a statistically significant reduction in the likelihood of allcause and DM-related hospitalizations. In a subgroup analysis in which we controlled for patients' baseline glycosylated hemoglobin levels, the treatment group had a lower likelihood of having any care coordinator-initiated primary care clinic visits (in which the care coordinator initiated referral to primary care based on health information received from patients' CCHT technology).
Conclusion: After controlling for selection bias and for intervening time factors, the VA CCHT program reduced avoidable healthcare services for DM (such as hospitalizations) and reduced care coordinator-initiated primary care clinic visits.
(Am J Manag Care. 2006;12:467-474)
Diabetes mellitus (DM) is associated with high rates of morbidity and costs for the Department of Veterans Affairs (VA).1 In a 2004 study, Weinstock et al2 reported that veterans with DM received 30% of all VA pharmacy prescriptions. Diabetes mellitus that is not managed appropriately can lead to adverse outcomes such as limb amputations, and veterans with DM who have had limb amputations experience 1.6 times as many hospitalizations as veterans without DM.1,3 Many veterans with DM face healthcare access barriers such as having to travel long distances to VA medical centers to use outpatient services.4
During the past decade, the VA has undergone a tremendous structural transformation from a hospital-based system to a system geared to greater ambulatory care. A more recent development has been the patient-centered care coordination perspective, which aims to reduce healthcare access barriers for veterans and to provide care in a more efficient manner for community-dwelling veterans with chronic diseases who have complex needs.5 This extends the notion of disease management by more efficiently integrating the various healthcare needs of veterans with disabling chronic diseases (such as DM) with system resources. Care coordination aims to unify care so that the amount and timing of care for each veteran are appropriate for that veteran and not simply based on customary population- based rules that may not fit the individual veteran's needs.5 Home telehealth technologies, which integrate information and communication technologies through communication services such as messaging devices (interactive caller ID-type devices) and videophones, allow the veteran and his or her care coordinator to maintain direct communication.6-8 Care coordination and supportive home telehealth technologies may be especially helpful for people who travel long distances for care or who experience lengthy clinic wait times.9
A principal goal in the VA Care Coordination Home Telehealth (CCHT) program is to reduce avoidable and costly VA healthcare services such as hospitalizations. Two observational studies10,11 of veterans with DM examined healthcare use 1 year before and 1 year after enrollment into a VA CCHT program. Multivariable results indicated a significant reduction in the proportion of patients who were hospitalized and a reduction in the mean length of stay for those who were hospitalized. Although the findings from these 2 studies were informative, they used a simple pre-post design that may be substantially affected by selection bias or by regression to the mean, statistical phenomena occurring whenever a nonrandom sample is derived from a population and 2 flawed correlations are assessed.12 Indeed, many disease management investigations have faced considerable limitations such as regression to the mean, selection bias, poor follow-up, limited settings, and publication bias.13-18 More rigorous research is needed to assess the quality of programs such as the VA CCHT.19
The present study was designed to assess healthcare service use in an ethnically diverse group of veterans with DM who were enrolled in a VA CCHT program during 24 months and to contrast this utilization with the service use of a comparison group of veterans with DM who did not receive any intervention. The objective of this study was to rigorously test the effectiveness of the VA CCHT intervention to determine if there was a reduction in avoidable, costly services during 24 months by (1) using propensity scores to improve the balance between the treatment and comparison groups and (2) using a difference-in-differences (DiD) approach to control for regression to the mean and for selection bias.
METHODS
Description of the Intervention
The VA CCHT program was implemented at 4 VA medical centers in a Veterans Integrated Service Network that covered most of Florida, Puerto Rico, and southern Georgia. The predominant type of home telehealth technology used was a messaging device. This device operates using basic telephone service and an electrical outlet. Patients used this messaging device daily to answer scripted questions about their DM symptoms and health status. The program consisted of nurse care coordinators (registered nurses or advanced registered nurse practitioners) who used disease management protocols to manage treatment and to educate the veterans about their disease to prevent more costly interventions (hospitalizations or emergency department [ED] visits). The care coordinators monitored responses daily and made clinical judgments regarding whether a telephone call should be placed to the patient or an appointment should be made with the patient's physician. In the instances in which a telephone call was made, it lasted on average 15 to 30 minutes. The duration of the telephone call depended on the complexity of the issue. The care coordinators performed multiple tasks that might include, but were not limited to, performing patient assessments, placing new orders for patient medications, helping patients manage their medications, scheduling new appointments in particular VA clinics, reminding patients of their clinic appointments, and assisting with technology difficulties. In rare circumstances, a telemonitor and a videophone with 2-way audio/video connectivity were used for weekly contact.
Study Design
We used a retrospective, concurrent matched cohort study design that was institutional review board approved. The VA CCHT program included veterans with DM who were at high risk for expensive, multiple VA inpatient and outpatient visits, including ED visits. Veterans with DM were eligible for the program if they had 2 or more VA hospitalizations or 2 or more VA ED visits in the 12 months before enrollment. Veterans needed access to a working telephone line and had to be noninstitutionalized to be enrolled. The treatment and comparison groups were matched on the basis of the treatment group members' study enrollment dates, so that both groups had identical distributions of enrollment and service periods.20 Both groups had to remain enrolled in the VA for the entire 24-month observation window. Three comparison group participants were randomly selected for each member of the treatment group to ensure an adequately sized comparison group. To improve the match between the treatment and comparison groups, we used propensity scores by (1) estimating a model of the probability that a patient "selects" into the treatment group as opposed to the comparison group, (2) dividing our sample into the quintiles of the predicted propensity scores distribution (<20%, 20% to <40%, 40% to <60%, 60% to <80%, and ≥80%), and (3) randomly sampling equal numbers of individuals in the treatment and comparison groups from each quintile. Although the application of propensity scores improved the balance between the treatment and comparison groups, there remained a few statistically significant differences between the groups, as summarized in Table 1. To control for these remaining differences, we used a DiD approach to measure the treatment effect, as explained herein.
Any simple pre-post comparison of use for a treatment group may be biased by regression to the mean in healthcare use. Regression to the mean in healthcare use refers to the tendency of high users in any single period to exhibit lower use in subsequent periods (ie, for their healthcare use to regress to the mean).18 Regression to the mean can lead to faulty conclusions and to a misunderstanding of test results.12 Because the treatment group was chosen based on high levels of prior use, we might expect the treatment group to exhibit lower use in subsequent periods strictly from regression to the mean, independent of any intervention effect. Consequently, failure to correct for regression to the mean in use may cause researchers to incorrectly attribute reduction in use to the intervention.12,18 In addition, because veterans with DM were not randomly assigned to the treatment and comparison groups, selection bias could be present. Selection bias occurs when some poorly observed factor influences the assignment of the patient to the treatment group vs the comparison group, as well as the healthcare outcomes of the patient.
To control for any regression to the mean and selection bias, we used a DiD evaluation design. The DiD approach has long been used in studies of labor economics21 and has recently been applied to health services research.22,23 The DiD approach seeks to measure a treatment effect while accounting for any pretreatment differences (ie, selection bias) between the treatment and comparison groups, as well as any intervening time factors (ie, regression to the mean). The concept of the DiD approach is illustrated in the Figure, which shows a hypothetical example of pretreatment and posttreatment outcomes in the treatment and comparison groups before and after an intervention. Before the intervention, the difference between the treatment and comparison groups measures any innate or intrinsic difference between the 2 groups. In the Figure, this is indicated by "Comparison Group/Pre"minus "Treatment Group/Pre." As shown in the Figure, the comparison group has higher outcome values than the treatment group before the intervention.
difference-
in-differences
Following the intervention, the difference between the treatment and comparison groups measures the treatment effect plus intrinsic difference. In the Figure, this is indicated by "Comparison Group/Post" minus "Treatment Group/Post." To calculate the treatment effect alone, we must subtract the intrinsic difference between the 2 groups from the combined treatment effect plus intrinsic difference. The intrinsic difference is indicated by the difference between the treatment and comparison groups before the intervention, while the treatment effect plus intrinsic difference is indicated by the difference between the treatment and comparison groups following the intervention. So, to calculate the treatment effect alone, we subtract the difference between the treatment and comparison groups before the intervention from the difference between the treatment and comparison groups following the intervention. Therefore, the treatment effect is measured as the difference between 2 differences, hence the term . By obtaining the treatment effect alone and by eliminating any intrinsic difference between the treatment and comparison groups, we controlled for observed differences between the treatment and comparison groups (Table 1) and for any unobserved differences that might bias the treatment effect.
Service Use Outcome Measures
International Classification of Diseases, Ninth
Revision
Service use outcomes were measured at baseline and at 24 months after enrollment in the VA CCHT program. We derived service use from the National Patient Care Database, a Veterans Health Administration database that consists of inpatient and outpatient data. Inpatient service use included hospitalizations (all cause vs DM related) and length of stay (all cause vs DM related). Outpatient service use was measured as ambulatory care clinic visits and was categorized as (1) ED (all cause vs DM related), (2) care coordinator-initiated primary care clinic visits (in which the care coordinator initiates the referral to primary care based on health information received from the patient's home telehealth technology), (3) podiatry, (4) ophthalmology, or (5) diabetes specialty clinic. Because we used the Veterans Health Administration database for our data, we used codes to calculate the Dartmouth-Manitoba comorbidity index developed by Romano et al.24 We added an additional code for depression but used it as a dummy variable. We collected data on the participants' age, marital status (married or not married), ethnicity (white, Hispanic, black, or other), facility site (A, B, C, or D), and service-connected disability (none, 10%-49%, or ≥50% based on disabilities that were sustained or aggravated during military service). These sociodemographic and comorbidity variables were used as covariates in the models. We obtained baseline glycosylated hemoglobin (A1C) levels from patient medical records for 1 site (facility site B). We conducted a subanalysis among this group and used these values as an important clinical control variable.
Statistical Analysis
P
We used logistic regression analysis for binary service use outcomes (eg, hospitalizations, ≥1 ED visits, and ≥1 care coordinator-initiated primary care clinic visits) and ordinary least squares regression analysis for continuous outcomes (eg, length of hospital stay). All statistical tests were 2-tailed, and the analyses were performed using SAS version 8.0 (SAS Institute, Cary, NC). Statistical significance was set at <.05. We used the estimator for the interaction effect by Ai and Norton25 for the logistic regression models because the interaction effect in nonlinear models must be calculated differently from that in linear models.
RESULTS
Demographic Characteristics at Baseline
Table 1 gives descriptive assessments of the treatment and comparison groups before the intervention. Before the intervention, there were no statistically significant differences in age, marital status, ethnicity, facility site, service-connected disability, hospitalizations (DM-related), length of stay (DM-related), or outpatient visits between the 2 groups. There were comorbidity differences between the treatment and comparison groups, with the treatment group reporting more comorbidities (1.4 vs 0.7).
Changes in Service Use
P
P
P
P
Table 2 gives the service use multivariable preintervention and postintervention outcomes in the treatment and comparison groups. The treatment effect, which was measured using the DiD approach, is given in the column on the far right. These outcomes were adjusted for sociodemographic and health status characteristics. Two years after enrollment, there was a significant difference between the treatment and comparison groups in the likelihood of all-cause hospitalizations, decreasing in the treatment group from 38.8% to 30.0% (= .01) and increasing in the comparison group from 31.2% to 33.1% (= .61). The treatment group had a significant reduction in DM-related hospitalizations during 24 months from 35.3% to 26.9% (= .02). The comparison group demonstrated little change in DM-related hospitalizations. The DiD estimate of -9.1 percentage points was significant at = .02, suggesting that the intervention reduced hospitalizations by almost 25%.
P
P
P
A -6.1-percentage point change in DM-related hospitalizations was attributable to the intervention (= .08) (Table 2). Care coordinator-initiated primary care clinic visits increased by 8.7 percentage points under the intervention (= .04), while podiatry visits decreased by 6.2 percentage points (= .07).
Although the results in Table 2 suggest that the intervention dramatically increased ED visits, this result is an artifact of the inclusion criteria for the treatment and comparison groups. Because the treatment group enrolled a disproportionate share of patients with DM with multiple hospitalizations, the comparison group enrolled a disproportionate share of patients with DM with multiple ED visits to meet the inclusion criteria. This resulted in 98% of the comparison group having an ED visit before the intervention, thereby rendering any meaningful comparison impossible.
P
P
P
P
Table 3 gives results only from site B, at which baseline A1C levels for the treatment and comparison groups were available for inclusion in our models. During 24 months, the treatment group experienced a significant decrease in all-cause hospitalizations from 60.3% to 40.7% (= .03). The comparison group showed a slight increase from 36.8% to 38.8% (= .84). As measured by our statistical models, the difference between these 2 groups was -18.0 percentage points (= .055), indicating that the intervention reduced hospitalizations. The treatment group showed a decrease in DM-related hospitalizations from 54.6% to 37.0% (= .04), but a statistically significant treatment effect was not detected.
P
P
P
In terms of the care coordinator-initiated primary care clinic visits, the treatment group experienced a significant reduction during 24 months, decreasing from 59.0% to 21.0% (<.001), while the comparison group experienced a reduction that approached significance, from 38.0% to 22.6% (= .06) (Table 3). The estimated intervention effect was -21.0 percentage points (= .03), indicating that the intervention reduced the number of patients requiring care coordinator-initiated primary care clinic visits. Patients' baseline A1C levels had a statistically significant main effect on hospitalizations, such that patients with higher baseline A1C levels had a greater odds of having at least 1 hospitalization at 24 months after the intervention.
DISCUSSION
We found a significant difference between the treatment and comparison groups in the proportions with at least 1 hospitalization (all cause) during 24 months, with the treatment group exhibiting a significant reduction. There were significant differences between the treatment and comparison groups for care coordinator- initiated primary care clinic visits, with the treatment group experiencing a slight increase in visits and the comparison group exhibiting a slight decrease in visits.
We conducted a subanalysis for 1 site in which we controlled for baseline A1C levels. We found a significant difference between the treatment and comparison groups for all-cause hospitalizations. The treatment group experienced a significant reduction in the proportion with at least 1 hospitalization during 24 months. There was a significant difference between the treatment and comparison groups for care coordinator-initiated primary care clinic visits. The treatment and comparison groups experienced reductions in care coordinator-initiated primary care clinic visits, but the reduction was larger in the treatment group.
This retrospective study examined the effectiveness of a VA CCHT program for a large group of ethnically diverse patients with DM by examining healthcare service use outcomes at 24 months after the intervention. In the absence of a randomized controlled trial, our quasiexperimental design attempted to overcome methodological shortcomings by using the following 2 important state-of-the-art procedures: (1) propensity scores to improve the balance between the treatment and comparison groups and (2) a DiD approach to evaluate treatment effectiveness.
During 24 months, the treatment group experienced a significant reduction in the proportion with at least 1 hospitalization. These findings using propensity scores and a DiD approach support the pre-post observational findings of previous studies10,11 that showed a significant reduction in hospitalizations during 12 months. Our results suggest a reduction in hospitalizations by the VA CCHT program during 24 months associated with better home management of patients with DM.
In our subgroup analysis, we examined 1 of 4 sites in greater detail. These investigations yielded an intriguing finding with reference to the care coordinator-initiated primary care clinic visits. The treatment group showed a stronger significant decline in care coordinator-initiated primary care clinic visits than the comparison group. In a matched cohort study of the VA CCHT program for patients with DM, Chumbler et al20 reported that at 12 months the treatment group experienced a significant increase in initial care coordinator-initiated primary care clinic visits, suggesting that veterans' health status was monitored and clinical needs were met just in time before health deterioration. During 24 months, however, our results show a decrease in the need for these appointments. With daily monitoring via the home telehealth technology, care coordinators may have been able to identify subtle health changes, assist patients in managing their health problems, and resolve these problems before they became serious enough for a care coordinator-initiated primary care clinic visit. The explanation of these findings is congruent with the tenets of the chronic care model.26-28 That is, with the paradigm shift to coordinated care and the increase in patient self-management, DM symptoms are brought under better management, requiring fewer care coordinator- initiated primary care clinic visits.
Contrary to our findings in the subanalysis sample, there was a significant increase in the likelihood of care coordinator-initiated primary care clinic visits in the overall sample. Unfortunately, we do not have available data to definitively understand why these findings occurred. It may be that the site at which we conducted the subanalysis contained more motivated staff, who more carefully followed the protocols and performed better triage, referring only those patients who required the care of a VA primary care physician. It is possible that the program was established differently at site B and comprised care coordinators who were more experienced. Future research should further examine these important managerial and clinical issues.
Some limitations in our study deserve mention. Although patients were comparable with respect to age, ethnicity, service-connected disability, DM-related hospitalizations, and DM-related length of stay, they were not comparable with respect to comorbidities. Although we took careful steps to use the same inclusion criteria to retrospectively obtain a comparison group that was comparable to the treatment group, the treatment group tended to be selected based on hospitalization criteria, while the comparison group was chosen largely based on ED visit criteria. Conceptually, our DiD approach should correct for these differences. Although our propensity scores and DiD approach were chosen to help compensate for differences, they cannot automatically account for clinical variables that are potentially important to a patient population with DM such as body mass index, insulin use, low-density lipoprotein cholesterol levels, systolic and diastolic blood pressures, and time since diagnosis.13,29
Our study included limited sociodemographic information, such as the presence and level of social support and whether patients had insurance other than VA coverage, which may have affected our outcome findings. Although the ED visits reported in the comparison group appeared to be an artifact of selection bias, the treatment group had high rates of hospitalizations and ED visits. This is not due to the fact that the treatment group was selected for hospitalizations, whereas the comparison group was selected for ED visits. Rather, it may be attributable to the fact that the comparison group was selected primarily on the basis that the treatment group contained most of the available patients who had multiple hospitalizations. Unfortunately, we do not have access to data on the number of contacts or the duration of the contacts between the care coordinators and the patients. Further research might incorporate such information to examine the associations between CCHT correspondence and change in service use.
Costs are of significant concern when implementing and evaluating new programs. Our data precluded us from assessing the cost-benefit ratio or the cost-effectiveness of the CCHT program for veterans with DM. We did not have access to program development or implementation costs at the 4 sites. To obtain a more comprehensive evaluation of the program effectiveness, future research should collect program and service use costs for the treatment and comparison groups.
Our study was implemented in 1 Veterans Health Administration geographical region; thus, generalizability may be an issue. The patients included in this VA CCHT program and the sites at which it was implemented differ notably from those in previous disease management-oriented investigations, which were primarily executed in managed care and other private insurance-type settings. 13 However, the region chosen for our study includes a large, ethnically diverse population with DM with existing management issues and who needed more intensive follow-up than the traditional medical care system provided.10 Our results can be informative as similar programs and interventions are planned to be implemented throughout the VA and outside the VA.
Our findings suggest that the VA CCHT program can minimize avoidable healthcare service use at 24 months after implementation, as evidenced by the reduction in hospitalizations in the overall group and in the subanalysis group and by the reduction in care coordinator-initiated primary care clinic visits in the subanalysis group. These results are based on the use of a well-matched comparison group and a DiD approach that controlled for selection bias, avoiding the pitfalls that are common in disease management program investigations.
From the Veterans Affairs Health Services Research and Development/Rehabilitation Research and Development Rehabilitation Outcomes Research Center, North Florida/South Georgia Veterans Health System (TEB, NRC, WBV, RJB, HQ); Veterans Affairs Health Services Research and Development Stroke Quality Enhancement Research Initiative (TEB, NRC, WBV, RJB); Departments of Health Services Research, Management and Policy (NRC), Epidemiology and Health Policy Research (WBV), and Aging and Geriatrics (RJB), University of Florida, Gainesville; and Veterans Health Administration Office of Care Coordination, Washington, DC, and Lake City, Fla (RK).
This research was funded by the Veterans Affairs (Veterans Integrated Health System-8) Community Care Coordination Service and was supported by the Veterans Affairs Health Services Research and Development/Rehabilitation Research and Development Rehabilitation Outcomes Research Center, North Florida/South Georgia Veterans Health System.
This study was presented at the 24th Veterans Affairs Health Services Research and Development National Meeting; February 16, 2006; Arlington, Va.
Address correspondence to: Tracey E. Barnett, PhD, Veterans Affairs Health Services Research and Development/Rehabilitation Research and Development Rehabilitation Outcomes Research Center, North Florida/South Georgia Veterans Health System, 1601 SW Archer Rd, 151B, Gainesville, FL 32608. E-mail: tracey.barnett@va.gov.
