Asthma Disease Management: Regression to the Mean or Better?

Objectives: To assess the effectiveness of disease managementas an adjunct to treatment for chronic illnesses, such as asthma,and to evaluate whether the statistical phenomenon of regressionto the mean is responsible for many of the benefits commonlyattributed to disease management.

Study Design: This study evaluated an asthma disease managementintervention in a Colorado population covered byMedicaid. The outcomes are presented with the interventiongroup serving as its own control (baseline and postinterventionmeasurements) and are compared with a matched control groupduring the same periods.

Methods: In the intervention group, 388 asthmatics entered and258 completed the 6-month program; 446 subjects participated inthe control group. Facilities charges were compared for bothgroups during the baseline and program periods. Both groups werewell matched demographically and for costs at baseline.

P

Results: Using the intervention group as its own control revealeda 49.1% savings. The control group savings were 30.7%. Therefore,the net savings were 18.4% (< .001) for the intervention group vscontrols. Although the demonstrated savings were less using a controlgroup to correct for regression to the mean, they were statisticallysignificant and clinically relevant.

Conclusion: When using a control group to control for the statisticaleffects of regression to the mean, a disease managementintervention for asthma in a population covered by Medicaid iseffective in reducing healthcare costs.

(Am J Manag Care. 2004;10:948-954)

During the last decade, disease management hasbecome a widely accepted adjunct to conventionaltherapy.¹ Despite this activity, considerabledebate remains as to the components of thisintervention, its relative effectiveness, and, most of all,the methods of assessing its value.² Because diseasemanagement has arisen at a time when medical interventionsare often measured in terms of their effects onthe cost of delivering care, a significant topic of interestfor disease management has been its effect on return oninvestment (ROI).³

regression to the mean,

Questions arise about what constitutes the bestmeasures to be used to evaluate ROI outcomes. Equallyas important as the actual measurements are the factors(intended and extraneous) that affect the observedchanges when ROI is measured. A factor related toeffectiveness measurements of disease managementprograms is a statistical termfirst coined in 1886 by Galton⁴ when observing heightsin families. The critical element in the application of theobservations of Galton is that, if a series of events istracked, the events tend on their own to return to a predictablemean. The more extreme that the initial observationis relative to the true mean, the more likely it isthat regression to the mean will be observed. Mostimportant, this regression would occur even withoutactive intervention.⁵

A method of measuring the effect of an interventionis to have a control group that does not receive treatment,but still has an equal chance of being affected byother (random) events. This option is not always availablewhen evaluating the effectiveness of certain diseasemanagement programs because of population size, ethicalissues, and lack of a control population that facesthe same or similar issues as the intervention group. Infact, a meticulously chosen control group shouldinclude individuals who could otherwise be substitutedfor those within the intervention group.

This article focuses on the results of a disease managementintervention for asthma in a population coveredby Medicaid. The outcomes are presented in 2ways: (1) with the intervention group serving as its owncontrol (ie, baseline and postintervention measurements)and (2) by comparing the intervention group with a similarmatched control group during the same periods. Bothof these methods are common within the literature.⁶ Byusing 2 methods of assessing the same program, weare able to evaluate the principle of regression to themean in measuring the effectiveness of disease managementinterventions in an asthmatic population. We alsodemonstrate the importance of having a matched controlgroup rather than using only preintervention andpostintervention data as a measure of the effectiveness.

METHODS

Study Populations

International Classification of Diseases, Ninth

Revision [ICD-9-CM]

The population studied was covered by a ColoradoMedicaid health plan. Individuals were referred to theNational Jewish Medical and Research Center DiseaseManagement Program for asthma (NJDMP) based ontheir having had a prior diagnosis of asthma(code 493) and a minimum level ofhealthcare use (hospitalization and emergency departmentvisits) in the 12 months before their referral. Thepopulation of asthmatics was divided into 2 subgroups:(1) those who participated in the NJDMP and (2) a controlgroup that did not receive any specific intervention.Both subgroups were followed up for 6 months. Theobservation was limited to the short period duringwhich an asthmatic would be most likely to experiencean exacerbation of his or her illness, primarily duringthe fall and winter months. To ensure complete evaluationof baseline and postintervention costs, all individualsfrom both subgroups had to have been enrolled withthe health plan for at least 1 year.

Intervention Subgroup Enrollee Identification

To maximize the statistical power to observe changesdue to the NJDMP, it was the stated intention of thestudy to enroll a sufficient number of patients to have250 individuals complete 6 months in the program.Given an expected dropout rate of about 35% (becauseof loss of eligibility, relocation, etc), approximately 385individuals were sought for initial enrollment.

Control Subgroup Identification

Individuals who were eligible for inclusion in thecontrol group were potential candidates from the basepopulation of referrals who did not enter the program.In addition to the basic requirements, eligibility forinclusion in the control group required that the individualshad been actively enrolled in Medicaid during thebaseline and program periods (so that a complete costcomparison could be made for both groups during thesame periods). Because eligibility lists can be overinclusive(ie, individuals who are no longer using Medicaidinsurance may still be included on Medicaid eligibilitylists because they did not disenroll), active participationduring the study period was defined as having at least$1 in claims of any type (facilities, professional, orpharmacy claims, whether or not they were paid) forany reason during the 1-year period that included the6-month program period.

Individuals in the control group did not enter the diseasemanagement program for several reasons. Eightyfivepercent of the nonparticipation was because of aninability to be reached at the initial screening telephonecall. The timing of these telephone calls included day,night, and weekend times, as they were assigned tonurses in a random order. The number of available slotsfor participants was about a third of the number (n =1249) of individuals who were referred to the NJDMP aseligible for participation. Because it was the intention ofthe program to take participants and controls through asimilar environmental exposure period, all efforts weremade to quickly enroll the subjects into the program. Of388 individuals, 98% enrolled between October 16 andNovember 21, 2002. This meant that most individualswho were unavailable at the time of initial contact werenot telephoned again because the program had beenfilled. Other reasons for nonparticipation in the programincluded individuals with incorrect telephone numbers,individuals who were not on the initial list sent forscreening but who appeared in the subsequent month'slist, and individuals who declined to participate. This lastgroup accounted for about 8% of individuals. To ensurestatistical rigor, these individuals were excluded fromthe control group.

The assignment of individuals to the treatment orcontrol groups was dominated by a random variable:being at home at the time of the initial screening telephonecall. Therefore, these 2 groups were statisticallyand clinically well matched as cases and controls.

Data Analysis

The data analyzed in the study came from UB-92claims (ie, hospital and facilities charges) reported byColorado Medicaid. Individuals with asthma were identifiedby analyzing the following claims: (1) any claimwith a primary diagnosis of asthma (code 493.xx) and(2) any claim with a secondary diagnosis of asthma, providedthat the primary diagnosis was in the respiratorysystem (codes 460.xx to 519.xx).

The program duration was approximately 6 months(ie, the period in which the participants would receiveproactive telephone calls from the nurses). The baselineperiod was defined for program participants as the yearbefore enrollment. For the control group, the periodsinvestigated spanned November 15, 2001, to May 15,2002, for the baseline period, and November 15, 2002,to May 15, 2003, for the program period. These datescorresponded approximately to the median entry andexit dates for program participants. A claim was countedin the analysis if the start date for a given healthcareuse occurred in the time frame in question.

Data Screening and Transformation

Similar to symptom data and many other data sets inhealthcare, use cost data do not conform to a normaldistribution. Instead, these data represent a distributionthat is markedly skewed in a positive direction. That is,most individuals will have claims costs in the lowerrange, with a small number of individuals having muchhigher claims costs. The effect of this type of distributionis that the mean value deviates from the medianvalue and the standard deviation is minimally informativein terms of denoting extreme scores. In this case,many more people will fall into the range of extremescores than would be expected if the data were normallydistributed.

As a result of this nonnormality, mean differencesmay be misleading; therefore, data should be transformedso that the distribution approaches normalitybefore any inferential statistical analyses are performed.In this case, the most appropriate transformation is alog transformation of the data; therefore, for the purposeof analysis, the data are redefined as the naturallog of the original cost plus $1. (The adding of $1ensures that any individuals with $0 in claims for agiven period are still counted in the analysis.)

Because a significant number of individuals had $0claims data in both groups, even the transformed datadid not conform to a normal distribution; rather, thedistribution was bimodal with a small spike at $0.Nevertheless, the resulting distribution is much closerto the normal bell-shaped distribution, and the normalparametric statistics should be robust to the small deviationsfrom normality. However, for completeness, thenonparametric Wilcoxon rank sum test was also used totest for group differences. Because the resulting metricis not meaningful in presenting costs in their absoluteterms, the data are presented in the normal metric ofdollars spent, but analyses of group differences wereperformed on the log-transformed data.

Data screening was performed to ensure that programparticipants and control group subjects were statisticallycomparable to each other. Although the datawere positively skewed for both groups, as alreadydescribed, there was a small number of potential controlsubjects who had high use for nonasthma claims duringthe baseline period. Although asthma claims were similarfor participants and potential controls, the inclusionof these outliers distorted the total claims by causing alarge increase in nonasthma (thus, total) use. Therefore,potential control group subjects were eliminated if theyhad total claims that exceeded 5 standard deviationsabove their observed (untransformed) mean. Althoughthis value eliminated the top 2.8% of potential individuals,it was chosen because the resultant control groupwas statistically appropriate in all respects. The controlgroup was similar to the participant group in asthma,nonasthma, and total claims. In addition, elimination ofthese individuals with extreme nonasthma use (>5 SDsabove their untransformed mean) did not change themean value of asthma claims for the control group; itonly reduced nonasthma claims at baseline to a valuecomparable to that of participants.

Calculation of Return on Investment

Return on investment can be thought of as the savingsa payer (such as Medicaid) realizes over and abovethe costs expended. Theoretically, ROI can be negativeor positive. Values less than 1 indicate that the cost ofthe program was greater than any savings provided, valuesequal to 1 indicate that the cost of the program wasequal to the savings provided, and values greater than 1indicate that there were savings over and above the costof the program. The formula used to calculate ROI is asfollows: (year-1 costs minus year-2 costs) divided bydisease management program costs.

Interventions in the National Jewish Medicaland Research Center Disease Management Programfor Asthma

The main activities that took place within the interventiongroup were physician education, patient education,and case management. Based on the needs andprevious patterns of healthcare access of this population,we focused on 3 areas: (1) increasing the use ofanti-inflammatory medications, (2) having the participantstelephone our reactive care line early in an attackinstead of going to an emergency department, and (3)decreasing nighttime symptoms, the most frequent timefor emergency services. Physician and patient educationwas provided in different ways and included manytopics.

Case management was implemented through a teamof specialized respiratory nurses. Patients receivedbetween 5 and 7 proactive telephone calls following aninitial questionnaire. Patients were encouraged to contactnurses when their asthma was symptomatic. If thisoccurred, they received follow-up telephone calls toensure that appropriate care was being pursued.

In a timely manner, physicians received reports fromthe case managers summarizing their patients'statusand providing healthcare use and productivity data.Physicians were also encouraged to follow the mostrecent national guidelines⁷ in the care of their patients.Each patient's physician was asked to provide a writtenasthma action plan. If this was not provided, the patientprovided a description of his or her medications andinformation on how to use them. This was recorded inan asthma action plan and sent to the patient's physicianfor verification. If plans were not in accord withnational guidelines, the physician's office was contactedand appropriate recommendations made. Physicianswere encouraged to contact the NJDMP with anychanges in their recommendations.

RESULTS

From the population of referred individuals, 388entered the program and 258 had continuous participationfor 6 consecutive months in the intervention group.Rapid enrollment was used in an attempt to control forvariations in climatic, viral infection, and pollutionexposures among patients. The peak of the flu season inColorado occurred approximately at the same time asall patients completed entry into the program.

There were 17 864 asthmatics (48% male) agedbetween 1 and 89 years in the total population ofColorado Medicaid. From this asthmatic population, asubgroup was identified for referral to the NJDMP consistingof individuals who (1) had a code-493 asthmaclaim in the year before program commencement and(2) had a minimum of 2 emergency department or hospitalclaims submitted to Medicaid (whether or not theywere paid) for any reason during the previous year.Colorado Medicaid members have a mean of 0.6 hospitalor emergency department visits per year. There were1249 individuals referred to the NJDMP for possibleenrollment, and the total claims paid for these individualswas $2 210 813.

The demographic data for the intervention group canbe found in Table 1. The 130 subjects who did not completethe program were disenrolled for various reasons.The most frequent reason was that these individuals losteligibility for insurance through Medicaid; the secondmost common reason was that nurses were unable tocontact the patients because they had moved or lost telephoneservice. Only 2 individuals dropped out of the programbecause they no longer wanted to participate in it.

Of those who were referred for enrollment to theNJDMP, 840 did not enter the program (potential controlgroup). The reasons fortheir not entering the diseasemanagement program are listedin the "Control SubgroupIdentification"subsection ofthe "Methods"section. Thedemographic data for the controlgroup can be found inTable 1.

From this group of 840potential control group subjects,446 met the selectioncriteria for inclusion. The remaining 394 were eliminatedfor the following reasons: 38 were missing all paymentdata during the baseline period, 237 had lost eligibilityby Colorado Medicaid, 62 had no charges (facilities, professional,or pharmacy) in the 12 months following thebaseline period and were presumed to be no longer eligiblefor insurance through Medicaid, 30 had baselineexpenditures exceeding 5 SDs above their group mean,and 27 refused to participate in the program.

For the intervention and control groups, claims wereavailable for analysis for the year before enrollment andfor the enrollment period. Six months of claims datawere used as the baseline costs for each group (Table2). The final analysis of claims during the program periodwas conducted on a data set generated 90 days afterthe end of the intervention period to allow sufficienttime for billing and payment of most claims (estimatedby the state to be &#8805;95% of the claims).

The clinical results related to the 3 focus areas for theintervention group were positive. The level of use of antiinflammatorymedications rose from 72.6% of patientsusing these medications to 85.2% at 6 months (12.6%improvement). We tracked 21 individuals who telephonedearly during the course of an asthma exacerbation,and an emergency department visit was averted inall of these cases. The last index we followed was thenumber of nighttime symptoms. At the conclusion of thestudy, the intervention group recorded 75% fewer nighttimesymptoms than at baseline. All of these focus areasplayed a part in the reduction in the use of asthma-emergencydepartment visits from 253 during the baselineperiod to 36 during the intervention period.

The total costs for all emergency department visitsand hospitalizations for the intervention group in thebaseline period were $544 847 ($351.97 per member permonth [PMPM]). The total costs for the control groupwere $991 710 ($361.79 PMPM). The costs during the 6-month intervention period (including the cost of the program)were $341 856 ($220.84 PMPM) for theintervention group and $755 173 ($250.75 PMPM) for thecontrol group.

Table 2 presents total PMPM costs and the percentagechange from the baseline period to the programperiod for both groups. All differences were statisticallysignificant in parametric and nonparametric statisticalanalysis, and probability values are reported for logtransformeddata, as already described.

t

P

t

P

As can be seen from Table 2, costs for the 2 groupswere similar at baseline. During the program period,costs for both groups declined: total costs for the controlgroup decreased by 30.7%, while costs for the interventiongroup (without the costs of the program included)declined by 49.1%. A statistical comparison of the logtransformedvalue of these difference scores for the 2groups yielded a value of 117.1 (< .001). For completeness,the difference scores were also comparedusing the nonparametric Wilcoxon rank sum statistic(which is treated as for samples > 20), and the resultswere unchanged (< .001).

Calculated Return on Investment forthe Intervention

As shown in Table 2, the total cost for the interventiongroup during the program period was $179.17.The cost of the NJDMP was $41.67 PMPM. Therefore,the total costs for asthma and nonasthma care plusthe cost of the program were $220.84 PMPM. Returnon investment was calculated as follows: ($351.97minus $179.17) divided by $41.67 equals $4.15.

Cost Differences Between the Interventionand Control Groups

P

As already described, the total savings in facilitiescharges between the baseline and program periods realizedfor the intervention group were 49.1%. After thecosts of the program were added, the savings were37.3% ($131.13 PMPM), while the reduction in totalcosts for the control group was 30.7% ($111.04 PMPM).Therefore, the net savings over and above the cost ofthe program were 9.1% greater for the interventiongroup than the control group. The differences in percentagereductions (including program costs) were analyzedand found to be statistically significantly different(< .001).

One way to look at ROI would be to consider this differenceof 9.1% as the actual effect of the NJDMP on theColorado population covered by Medicaid. This wouldmean a savings of 9.1% on the total initial costs of$544 847, or $49 582, for this small population of 258individuals. This recharacterization would yield anactual ROI of $1.77 during the 6 months of the program.Using the equation presented herein, this would mean asavings of $0.77 for every dollar spent over and abovethe savings accounted for by regression to the mean.

DISCUSSION

Disease management is rapidly growing in practiceand in the issues surrounding its definition and measurement.A key area of dispute is defining measurementsto evaluate the outcome variables used to assessthe efficacy of disease management.²

This study addresses some of the most common disputesabout measurement of disease management interventioneffectiveness in a single population ofasthmatics. These are discussed in detail herein andinclude (1) the effects of seasonality when individualsenter the program during an extended period, (2) theresults of measuring the effect on different populationsin the baseline and intervention periods, (3) the problemof having a population serve as its own control, and(4) the effects of regression to the mean.

Many asthmatics are affected by seasonal exposuresdue to allergy, weather, or infections. In this study, weprovided disease management for 6 consecutive monthsduring the fall and winter months for the entire population,thus avoiding the potentialconfounding effects ofseasonality.

A second common problemin analyzing data fromasthma disease managementprograms is that the individualsevaluated in the baselineperiod may not be the sameas those evaluated in theintervention period. In thisstudy, we only enrolled thoseindividuals who had previouslybeen enrolled in theColorado Medicaid programfor a minimum of 1 year. This allowed us to have accessto data for a full year before the study. In this way, wewere able to track the members within the interventionand control groups during the same 6 months for 2 consecutiveyears.

The third issue in evaluating data from disease managementprograms, using a single population as its owncontrol, is a common one. In this study, there were sufficientnumbers of asthmatic members to have definedintervention and control groups. As already shown,these groups were similar in their demographics andprior total healthcare use.

If we had used only the preintervention and postinterventiondata for the intervention group in this analysis,we would have demonstrated a 49.1% reduction incosts, with an ROI of $4.15. The use of a control group,although not perfect in evaluating the effect of programs,allows controlling for the effect of exposures andother unknown variables (including regression to themean) on the evaluation process. In this study, the controlgroup also experienced a significant reduction incosts during the same period of observation, but thisreduction was statistically smaller than the reductionexperienced by the intervention group. By having a controlgroup, we were able to better evaluate the effect ofthe disease management program on this population.

The last area that is often mentioned as being of concernin evaluating data for disease management outcomesis the statistical effect of regression to the mean.We assume that the 30.7% reduction in costs for thecontrol group from the baseline period to the programperiod is predominantly due to this. The interventiongroup, in contrast, showed a reduction of 49.1%.Therefore, subtracting the 30.7% that is presumed to bedue to regression to the mean, the remaining (statisticallysignificant) 18.4% reduction is attributable to programintervention. Furthermore, these results arepractical. On a PMPM basis, the savings for the interventiongroup were more than $45 per month beyondregression to the mean and beyond program costs; thattotals nearly $550 per year in savings for each member.

In healthcare, the effect of regression to the meancan lead to errant conclusions that an outcome is due totreatment when it is actually the result of chance. Onthe other hand, overstating the importance of regressionto the mean can lead to the statistical phenomenonknown as a type II error (ie, failing to find an effectwhen one is in fact present). This type of error can becostly if it causes decision makers to overlook a potentiallyvaluable tool for reducing healthcare costs.

It is important to control for regression to the meanbecause its presence calls into question the tenability ofthe claims being made. In our study, without a well-matchedcontrol group, the reduction in total healthcarecosts from the baseline period to the program periodcould not unambiguously be attributed to theintervention. However, data were collected on a participantgroup and a statistically similar control group.This design allowed for assessment of the effect ofregression to the mean, so that its presence could beruled out or its magnitude estimated in explaining thedecrease seen in the treatment group.

In looking at total costs, Table 2 shows that the controlgroup costs decreased significantly from baseline tothe program period. This was not an unanticipated finding,as a criterion for referral to the program was significanthealthcare use during baseline. The resultssupport the concept of regression to the mean as significantlyaffecting the measurement of interventions indisease management. This finding emphasizes the needfor having a well-matched control group in the studyprocess.

Regression to the mean applies most strongly incases in which the observation is far from the truemean; the farther the observed score is from the truemean, the more likely it is that regression to the meanwill occur. In the present study, the control group andthe intervention group were selected for their high baselinehealthcare use costs and were similar. Therefore,although regression to the mean would be expected tobe (and was) present in this sample, both groups wouldbe equally subject to its effect. The control groupshowed a 30.7% decline in costs from baseline to theprogram period; this decline was presumably due toregression to the mean. In contrast, the interventiongroup declined by 49.1% in the program period comparedwith baseline, and this change was significantlygreater than the control group change. Therefore, therewas a significant additional decline (18.4%) in costs forthe treatment group that was not due to regression tothe mean.

There is a related, albeit distinct, statistical phenomenonthat could be called into question in the presentstudy; that is, the presence of selection bias in determiningcases and controls. If the assignment of individualsto treatment vs control groups was not random butsomehow systematically differentiated the 2 groups, thegeneralizability of any findings could be limited.However, there is no statistical evidence that this wasthe case in the present sample. From a group assignmentpoint of view, selection of cases and controls wasclose to random. In most cases in which an individualwas telephoned but did not enroll, the reason was thatno contact was made at the time of the telephone call.Absence from home is not a potential source of selectionbias, because generally those who were not at homewere healthy and working, while those who were athome were sicker and not working. Telephone callswere made at all times of the day and evening andwould not systematically exclude any particular typeof participant. The 8% of potential participants whorefused to participate were not included in the analysis.Asthma severity ranged from mild to severe in bothgroups, so there was no disparate selection by severity.Age and sex distributions were also almost identical.Finally, from an outcomes point of view, costs fortreatment at baseline were similar for cases and controls.It is possible that the group of individuals whowere at home for the initial telephone call was differentfrom those who were not at home for the call.However, all individuals included in the data analysiswere enrolled in the Medicaid program. This probablymakes the economic differences small. In addition, itis possible that those at home had a different level ofasthma severity than those who were not. However,the entry criteria were the same for both groups, andthe level of baseline costs was similar. Therefore, thereis no evidence that selection bias may have affectedthe findings.

Another potential weakness of the study is the lackof available data for pharmacy and physician visits. Itis possible that there was increased pharmacy use inthe intervention group, which would further narrowthe difference in total costs between the 2 groups. Inthe self-reported data, there was an increase in the useof asthma controller medications. Costs for these medicationswould add to the total costs in the interventiongroup. With respect to physician office visits, it isnot clear if there would be any differences betweengroups. Additional studies should attempt to capturethis information.

CONCLUSIONS

This study has looked at methods variables in theassessment of disease management intervention in anasthmatic population. A clear need was demonstratedfor a control group in analyzing the effectiveness of theintervention to rule out improvement as simply being afunction of regression to the mean. Using 2 methods ofanalysis, it is evident that disease management for asthmain a Medicaid-enrolled population is effective inreducing healthcare costs.

From the National Jewish Medical and Research Center, Denver, Colo.

This research was funded by an unrestricted grant from Novartis, Basel, Switzerland,and AstraZeneca, Waltham, Mass.

Address correspondence to: David Tinkelman, MD, National Jewish Medical andResearch Center, 1400 Jackson Street, Denver, CO 80206. E-mail: tinkelmand@njc.org.

US News World Rep.

1. Clark K. The doctor gets a checkup. February 2, 2004:44-46.

Manag

Care.

2. Johnson A. Measuring DM's net effect is harder than you might think. 2003;6:28-32.

Manag Care.

3. Carroll J. Health plans demand proof that DM saves them money. 2000;9:25-30.

J Anthropol Inst.

4. Galton F. Regression towards mediocrity in hereditary stature. 1886;15:246-263.

BMJ.

5. Morton V, Torgerson DJ. Effect of regression to the mean on decision makingin health care. 2003;326:1083-1084.

Dis Manag Health Outcomes.

6. Walker DR, McKinney BK, Cannon-Wagner M, Vance R. Evaluating diseasemanagement programs. 2002;10:613-619.

Expert Panel Report 2: Guidelines for the Diagnosis and Management of Asthma:

Update on Selected Topics.

7. National Asthma Education and Prevention Program Expert Panel Report.Bethesda, Md: National Institutes of Health; 2002.NIH publication 97-4051.

Download PDF: Asthma Disease Management: Regression to the Mean or Better?