Managed Care Organizations' Performance in Delivery of Childhood Immunizations (HEDIS, 1999-2002)

Objectives: To examine recent trends in childhood immunizations recommended by the Advisory Committee for Immunization Practices measured by the Health Plan Employer Data and Information Set (HEDIS) and to describe the factors associated with higher rates over time.

Design: The HEDIS performance measures from 1999 to 2002 and plan characteristics include approximately 400 enrollees per plan each year.

Methods: Longitudinal regression analysis of commercial managed care organizations' HEDIS measures. The outcome measure was the proportion of children aged 24 to 35 months in the plan who received 4 doses of diphtheriatetanus-pertussis vaccine, 3 doses of polio vaccine, 1 dose of measles-mumps-rubella vaccine, 3 doses of Haemophilus influenzae type b vaccine, and 3 doses of hepatitis B vaccine.

P

P

P

Results: The mean immunization rate for health insurance plans increased from 65.7% in 1999 to 67.9% to 2002. Plans that reported publicly had higher childhood immunization rates than plans that did not report publicly (< .001). Plans with higher proportions of Hispanics or African Americans had lower childhood immunization rates (< .001). Immunization rates varied significantly by type of visit; plans with higher proportions of children making visits to their primary care physician had higher rates of immunization (< .001).

Conclusions: Managed care organizations' performance measured by childhood immunization rates varies by organizational and demographic factors. Our findings suggest that plans should ensure efficient and accurate data collection systems and should encourage their providers to assess for immunizations at sick-child and well-child care visits.

(Am J Manag Care. 2007;13:193-200)

Health plan decision makers should be aware of the following regarding childhood immunization rates:

• Plans that reported publicly had higher childhood immunization rates than plans that did not report publicly (P < .001).

P

• Immunization rates varied significantly by the type of visit. Plans with higher proportions of children making visits to their primary care physician had higher rates of immunization, whereas plans with higher proportions of infants making sick-child care visits (outpatient or emergency department) had lower rates of immunization (P < .001).

Measures of quality give purchasers of healthcare and consumers of this care information about how insurance plans perform. The Health Plan Employer Data and Information Set (HEDIS) from the National Committee for Quality Assurance (NCQA) provides comparative information across health plans to measure the quality of care and preventive services for health plan beneficiaries. More than 66% of commercial health plans report to NCQA, representing 85% of the commercially enrolled managed care organization (MCO) population, or approximately 70 million to 73 million enrollees.¹

Results have shown that differences in childhood immunization rates vary depending on the collection methods (eg, National Immunization Survey, National Health Interview Survey, or HEDIS).² Investigations using HEDIS data have described factors associated with immunization rates (eg, physician credentials)³; however, little is known about factors associated with immunization rates accounting for time. For HEDIS, reporting these associations can help administrators of health insurance plans in identifying characteristics that are more predictive of delivery of immunization services. This knowledge can assist in the development of strategies to maintain high immunization rates, as well as inform potential purchasers of insurance plans about what plan characteristics are associated with sustained high immunization rates over time. The method of longitudinal regression analysis has been rarely used when examining HEDIS data, although its necessity has been acknowledged.³

Because 6 or more encounters with a healthcare provider are needed in the first 2 years of life to fully immunize a child, timely receipt of childhood immunizations, captured in the immunization rates, are an indication of how well the healthcare delivery system is working. Influential employers such as General Motors, a company that has given employees significant financial incentives to choose health plans with better HEDIS scores, emphasize childhood immunization performance among important measures to assess.⁴ In the present study, longitudinal regression analysis was used to examine the factors associated with higher childhood immunizations rates reported by public reporting and nonpublic reporting commercial health plans to the NCQA.

MATERIALS AND METHODS

Design

The analyses included the HEDIS data sets from 1999 to 2002 for public reporting and nonpublic reporting commercial health insurance plans (for care received from 1998 to 2001). Plans included in the analyses were operating in all US states (except Montana), in Guam, and in Washington, DC. The sample sizes were 423 plans in 1999, 383 plans in 2000, 371 plans in 2001, and 332 plans in 2002. Variables examined from the Consumer Assessment of Health Plans survey included the proportions of respondents by race/ethnicity.⁵ Variables were examined from 8 HEDIS measurement categories.⁵ The NCQA accreditation status changed during the years, so plans were considered accredited if their status was excellent, commendable, full accreditation, or 1-year accreditation; otherwise, the plan was not considered accredited by the NCQA. Because this study constituted analysis of secondary data without identifiers, it did not require a review by the Centers for Disease Control and Prevention Institutional Review Board, and consent was not required.

Study Population

For HEDIS clinical measures (eg, immunizations) that require medical record review, a sample of 411 enrollees is required to obtain a valid rate. If the plan has fewer than 411 eligible persons for a particular measure, all eligible enrollees are sampled. For measures that use administrative data only (eg, child access to a primary care provider), the health plan samples the entire eligible population. For the Consumer Assessment of Health Plans survey, a minimum of 100 responses is required for reporting particular rates.

For immunization measures, children were systematically selected from the enrollment cohort who reached their second birthday in the reporting year and who met the continuous enrollment criterion of no more than 1 gap in enrollment of up to 45 days during the 12 months before their second birthday. Children identified as having contraindications (eg, anaphylactic reaction to a vaccine or its components) were excluded from all immunization statistics.⁵ Childhood immunization statistics from plans were calculated and reported (1) by using only administrative records specific to the plan (administrative method) or (2) by using administrative data supplemented with medical record data (hybrid method), which may contain information about immunizations received outside of the plan. To avoid ascertainment bias and to maintain data comparability, only plans using the hybrid method were included for immunization measures. Few plans that used the administrative method were excluded (16 plans in 1999, 5 plans in 2000, 11 plans in 2001, and 8 plans in 2002).

Explanatory Variables

The plan characteristics examined were those groups of measures, defined by NCQA, that have been shown previously in the literature to be associated with receipt of immunizations⁶ or those that could theoretically be associated with preventive services, including receipt of immunizations. These domains include access and availability of care and health plan stability. Health plan descriptive characteristics included NCQA accreditation status, region of operation, public reporting status, vaccine policy of states of operation, enrollment size (below or above the median), race/ethnicity proportions (based on the sample of Consumer Assessment of Health Plans survey respondents), and MCO type (eg, health maintenance organization, point of service, or combined health maintenance organization and point of service). The Vaccines for Children program is a federal entitlement program that supplies public and private healthcare providers with federally purchased vaccines at no cost for administration to eligible children. Vaccine policies include (1) Vaccines for Children program only, (2) Vaccines for Children program and underinsured, (3) universal,⁷ and (4) multiple states, in which the insurance plan operates in several states with various vaccine policies. Because of policy changes, the vaccine policy variable was coded for the immunization policy in effect in the state of operation for each year that a plan reported to the NCQA. Variables assessing access and availability of care were calculated and reported using the administrative method and included the proportion of board-certified providers per number of enrollees, proportion of total providers per total commercial enrollees, proportion of children aged 25 months to 6 years who made visits to their primary care physician during the reporting year (visits are for well-child or sick-child care), and proportion of outpatient sick-child care visits and emergency department visits made by children younger than 1 year annually. Variables used to assess health plan stability included the number of years in business by MCO type and the practitioner turnover rate for the reporting year. Plans reporting to be in business for more than 25 years were considered as being in business for 25 years.

Main Outcome Measures

Haemophilus influenzae

Because the NCQA releases data aggregated by managed care plans rather than by enrolled individuals, the unit of analysis was the plan. Childhood immunization statistics were calculated as the proportion of children sampled in the plan who received 4 doses of diphtheria-tetanus-pertussis vaccine, 3 doses of polio vaccine (inactivated from 1999 to 2002 and oral vaccine in 1999), 1 dose of measles-mumps-rubella vaccine, 3 doses of type b vaccine, and 3 doses of hepatitis B vaccine as recommended by the Advisory Committee for Immunization Practices. Immunizations were only included if they were determined to be valid according to HEDIS criteria (Table 1). Because of the vaccine shortages in 2001 and 2002, plans had the option to report the previous year's immunization statistics if the plan or entity did not change (ie, did not merge or split into different plans) and they had reported in the previous year.⁵ To include all available data, all variables of interest reported by these plans were included except for the redundant immunization statistics reported during 2 consecutive years.

Statistical Analysis

P

P

Data were examined using SAS software version 9.1.3 (SAS Institute, Cary, NC). Data were modeled longitudinally using PROC MIXED to fit linear mixed models that included a random intercept, an unstructured covariance, and a robust variance estimator, the sandwich estimator, which was used to calculate valid standard errors while accounting for misspecification of the covariance and a normal distribution. Plan characteristics were fixed effects. An advantage of fitting linear mixed models is that PROC MIXED uses all available data in the analysis.⁸ Therefore, the plans that opted to report the previous year's immunization data during the years of shortage were included in the model using their independent variables, although the dependent variable data were missing. In the bivariate analyses, variables of interest were modeled with the year of report and an interaction term (eg, MCO type) to determine if they were significantly associated with the change in proportion immunized over time at < .05. Testing for interactions among variables was performed (NCQA accreditation status with region, public reporting status, and proportion of visits with primary care physicians), and region was tested with race/ethnicity proportions. A significant interaction term consisting of an independent variable with time (year or report) indicates that one group's rate of change is faster than another group's rate of change. Backward elimination was used to determine the multivariate model; fit statistics were used to determine the best fit of the model, and variables significant at < .05 remained in the model.

RESULTS

Immunization Rates

The rates for full immunization remained below 70% during the study period. The mean + standard deviation (SD) rates were as follows: 65.7% + 13.5% (range, 0.0%-93.6%) for 1999, 63.3% + 14.9% (range, 5.5%-92.9%) for 2000, 65.6% + 13.7% (range, 11.9%-90.5%) for 2001, and 67.9% + 12.2% (range, 17.6%-88.6%) for 2002 (Table 2).

Plan Characteristics

Two hundred forty-seven plans reported for 1 year, 132 plans reported for 2 years, 110 plans reported for 3 years, and 147 plans reported for 4 years. Health plan characteristics are given in Table 2 and Table 3 for continuous and categorical variables, respectively. The proportion of publicly reporting plans increased from 67.7% in 1999 to 83.7% in 2002 (Table 3). Similarly, the proportion of NCQA-accredited plans increased from 48.3% in 1999 to 65.2% in 2002. The proportions of enrollees by race/ethnicity remained steady during the 4 years (Table 2).

Factors Associated With Higher Immunization Rates Over Time

Bivariate Longitudinal Regression Analysis.

Many analyzed plan characteristics were significantly associated with immunization rates over time (Appendix available at www.ajmc.com). Time effects were significant for all the independent variables; interactions between independent variables and time were not significant except for region and proportion of Pacific Islanders.

Multivariate Longitudinal Regression Analysis.

P

P

P

To account for the variance within plans across years, we allowed the intercept to vary randomly. Plan characteristic variables were fixed effects. Factors that remained in the final multivariate model at < .05 included region, year of report, NCQA accreditation status, number of years in business, proportion of Hispanics, proportion of Pacific Islanders, proportion of African Americans, enrollment size (≥ 120 000 vs < 120 000), reporting status (ie, public vs nonpublic), and proportion of children aged 25 months to 6 years who made visits to their primary care physician during the measurement year. The only significant interaction detected was between year of report and region (Table 4). Although immunization rates dropped from 1999 to 2000, they increased significantly from 1999 to 2001 (= .04) and increased nonsignificantly from 1999 to 2002 (= .65). Publicly reported plans had 3.2 percentage point higher immunization rates than nonpublicly reported plans. The NCQA-accredited plans had 2.6 percentage point higher immunization rates than the nonaccredited plans. For each 10 percentage point increase in the proportion of members in health plans that were of Hispanic or African American race/ethnicity, plans had a 2.0 percentage point decrease in immunization rates, and for Pacific Islanders, plans had a 6.0 percentage point increase in immunization rates. For each 10 percentage point increase in the proportion of children aged 25 months to 6 years visiting their primary care physician in the year of report, plans had a 2.0 percentage point increase in immunization rates. Insurance plans with at least 120 000 enrollees had a 2.2 percentage point increase in immunization rates compared with plans that had fewer than 120 000 enrollees. Insurance plans that reported a region of operation other than the northeast had significantly lower immunization rates. The only significant interaction found was that the central region's fully immunized proportion dropped significantly and the northeast's fully immunized proportion increased faster during 2001 as opposed to 1999.

P

P

P

After controlling for covariates, the mean predicted immunization rates were 65.36% for 1999, 63.45% for 2000, 66.58% for 2001, and 67.10% for 2002. Although the mean + SD rate decreased by -1.96 + 0.86 in 2000 from 1999, it was not statistically significant (= .16). In 2001 and 2002, the mean + SD rates increased by 3.13% + 0.76% (< .001) and 0.52% + 0.68% (= .99) compared with 1999.

DISCUSSION

Overall, childhood immunization rates in these plans are improving, and NCQA accreditation and public reporting are associated with higher immunization rates. Plans with higher proportions of African Americans or Hispanics had lower immunization rates. Plans with higher proportions of children aged 25 months to 6 years visiting their primary care physician were also associated with higher rates of childhood immunizations.

During our study (1999-2002), there were delays in the receipt of vaccine^9,10 (vaccine shortages) and suspension of the thimerosal-containing hepatitis B vaccine for the birth dose of this vaccine.¹¹ Our finding that the proportion fully immunized dropped from 1999 to 2000 could be due to disruptions in vaccine supply; however, other national investigations conducted during times of vaccine shortage found no effect on receipt of the first dose of measles-mumps-rubella vaccine and a differential delay in receipt of the fourth dose of diphtheria-tetanus-pertussis vaccine only among children vaccinated at public clinics, with no similar pattern found among children vaccinated at private practices.¹² The significant interaction between the central and northeast regions during 2001 could possibly be explained by the supply disruptions, although the national study¹² examining this found that only children living in the southern United States were differentially affected by the shortages.

Consistent with previous findings,¹³ childhood immunization rates improved from 1999 to 2002. This trend will likely continue because this measure is an important factor in employers' purchasing decisions¹⁴ and in determination of the NCQA's requirements for accreditation. Consideration for receiving a positive NCQA accreditation status includes attaining at least the mean immunization rate for childhood vaccines among all health plans reporting in a given year. In a survey of firms that offered insurance for health maintenance organizations, 36% reported that NCQA or other accreditation is considered important in selecting or evaluating health plans.¹⁵ The finding that nonpublicly reported plans have significantly lower immunization rates might not be an important issue in the future, as 22 states now require that insurance plans report to HEDIS,¹⁶ and it is expected that more states will follow this direction. Perhaps legislation will provide more incentive to increase immunization rates among administrators of plans with low rates who typically decide not to publicly report.

Plans with higher proportions of African Americans or Hispanics had lower immunization rates, whereas plans with higher proportions of Pacific Islanders had higher immunization rates. This suggests that plans serving larger proportions of some racial/ethnic groups might offer inferior services. However, it is uncertain if African Americans and Hispanics are enrolled in the plans with lower immunization rates; this relationship merits further research.

Plans with higher proportions of children visiting their primary care physician within the past year were associated with higher rates of childhood immunizations. The children included in this measure had no more than 1 gap in enrollment of up to 45 days during the 12 months before their second birthday, so this finding may not be applicable to members with more health insurance gaps. This finding is consistent with a growing body of literature that has documented the association of low vaccination rates among children who have dropped out of their medical home.^17-19 There may be opportunities to increase vaccination rates by being more intent on ensuring at least an annual visit to a primary care physician for every child to be screened for appropriate immunizations; visits to primary care physicians include appointments for prevention or for illness. Other investigators found that missed opportunities at eligible sick-child care visits may reflect practitioners' failure to screen for immunization status, unwillingness to provide preventive services during sick-child care visits, or conservative interpretation of contraindication guidelines.²⁰ Missed opportunities for immunizations could be avoided by encouraging providers to screen for appropriate immunizations to ensure that children are up to date at sick-child and well-child care visits.²¹

Healthy People 2000

Healthy People 2010

Despite the modest increases in immunization rates, many plans' immunization rates were lower than other nationally reported figures by approximately 10 percentage points (eg, the National Immunization Survey) and were lower than the 80% target established in ²² and reiterated in .²³ The HEDIS rates are likely lower because the purpose of HEDIS is to measure immunization rates for health plans to determine how well the healthcare system is working (eg, including only valid doses and taking into account timeliness of vaccination), whereas the National Immunization Survey measures immunization rates for geographic regions to give an idea of how well the population is protected against vaccine-preventable diseases (eg, counting all vaccine doses delivered).² Another factor that could at least partially explain why the HEDIS rates are lower than other national figures may be that managed care plans may not have adequate tracking systems to collect and document vaccination status.¹³ Suboptimal immunization rates reported to HEDIS indicate that managed care plans need to ensure that their child members are being fully immunized against vaccine-preventable diseases.

A limitation of this study was the aggregated nature of the data. For example, it was not possible to examine individual observations; it was only possible to analyze the plan-level proportions. Also, not all plans reported data to HEDIS each year. The method used assumes that missing data are missing at random, yet this might not have always been the case. A strength of this study is that selection bias was limited by including nonpublicly reported plans.²⁴ In addition, serial correlation was accounted for by using a longitudinal model. Because the data were observational, we can say that factors were associated with immunization rates but were not necessarily causal.

Healthy

People 2010

Our results strengthen the case for states to require managed care plans to report to HEDIS to ensure efficient and accurate data collection systems, as well as NCQA possibly mandating a more stringent requirement for accreditation for the childhood immunization measure closer to the goal of 80% instead of the mean of reporting plans' immunization rates. Finally, the findings of this study indicate that further research is needed to improve adoption among MCOs of interventions known to increase vaccination rates and to identify new or modified strategies that may be more effective and more acceptable in different managed care environments.

Acknowledgments

We would like to thank Felicita David, MS, and Holly Groom, MPH, for help with data management. We would also like to thank Noelle Molinari, PhD, for helping with data interpretation.

Author Affiliations:

From the Immunization Services Division, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention (BB, AS, FZ), and Science Applications International Corporation (YK), Atlanta, Ga; and National Committee for Quality Assurance, Washington, DC (SS).

The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention, US Department of Health and Human Services.

Correspondence Author:

Barbara Bardenheier, MPH, Immunization Services Division, National Center for Immunization and Respiratory Diseases, Centers for Disease Control and Prevention, 1600 Clifton Rd, MS E-52, Atlanta, GA 30333. E-mail: bfb7@cdc.gov.

Download PDF: Managed Care Organizations' Performance in Delivery of Childhood Immunizations (HEDIS, 1999-2002)