Effectiveness and Cost of Influenza Vaccine Reminders for Adults With Asthma or Chronic Obstructive Pulmonary Disease

ABSTRACT

Objectives: To assess the effectiveness and cost of interactive voice response (IVR) reminders for influenza vaccination compared with postcards, among adults with asthma or chronic obstructive pulmonary disease (COPD).

Study Design: Pragmatic, 3-arm, randomized control trial.

Methods: The trial was conducted in an integrated healthcare organization during 2012 and 2013, using an existing IVR system. All adults aged 19 through 64 years with asthma or COPD (n = 12,285) were randomized to receive 1 of the following vaccination reminders: 1) postcard reminder only, 2) IVR reminder only, or 3) postcard plus IVR reminder. The primary outcome was influenza vaccination by October 31, 2012; the secondary outcomes were influenza vaccination by December 31, 2012, and by March 31, 2013.

Results: For subjects receiving an IVR call, 57% received a message on their answering machine; 27% answered the call; and 16% were not reached. Influenza vaccination rates were 29.5%, 31.1%, and 30.6% in the postcard-only, IVR-only, and postcard-plus-IVR study arms, respectively. After controlling for relevant covariates, IVR reminders were not significantly more or less effective than postcard reminders. Program costs were $0.78, $1.23, and $1.93 per subject for postcard-only, IVR-only, and postcard-plus-IVR reminders, respectively. Extrapolating costs to the entire population at the study site that typically receives influenza vaccination reminders (approximately 100,000 individuals), reminder costs would have been $0.55, $0.05, and $0.60 per subject for postcard-only, IVR-only, and postcard-plus-IVR reminders, respectively.

Conclusions: IVR reminders are not more effective at promoting influenza vaccination than postcard reminders, but IVR reminders may be less expensive for large patient populations.

Am J Manag Care. 2015;21(7):e405-e413

Take-Away Points

This study was conducted in an integrated healthcare delivery system. Adults with asthma or chronic obstructive pulmonary disease were randomized to receive interactive voice response (IVR) calls, postcards, or both as a reminder for influenza vaccination.

• Influenza vaccination rates were not significantly higher or lower among those who received IVR calls versus postcards.
• When costs were extrapolated to the entire population at the study site that typically receives influenza vaccination reminders (approximately 100,000 individuals), IVR was the least costly reminder method.
• Based on study findings, IVR was adopted as the primary strategy for annual influenza vaccination reminders at the study site.

Influenza infections cause substantial morbidity and mortality every year in the United States.¹ Individuals with chronic medical conditions such as asthma or chronic obstructive pulmonary disease (COPD) are at increased risk of influenza-associated morbidity compared with the general population.^2-5 Influenza vaccination is 1 of the primary means of reducing this disease burden, and while annual vaccination is recommended for everyone 6 months and older,^6,7 influenza vaccination is particularly important for individuals with chronic medical conditions. Despite these recommendations, influenza vaccination rates in individuals with chronic medical conditions⁸ have been consistently lower than national goals.⁹

Patient reminder systems have proven effective at increasing influenza vaccination rates in adults; recipients of reminders typically have vaccination rates 15 percentage points higher than those who do not receive reminders, though effectiveness has varied widely across published studies.^10,11 Most prior studies of influenza vaccination reminders have relied on letter, postcard, or auto-dialer reminders^12-14—methods that are not interactive and not easily tailored to individual patients. However, newer technologies such as interactive voice response (IVR) systems create opportunities to provide more individualized reminders, which may prove more effective. IVR systems can gather information from call recipients and use this information within complex branching logic to provide a more tailored message.¹⁵ While IVR systems have been used for a variety of health promotion and disease management purposes,^16-18 such systems have not been used extensively for influenza vaccination reminders. In addition, few studies have examined the cost of IVR systems compared with standard mail or auto-dialer—based reminders.

To address these gaps in knowledge, a study was conducted to examine the effectiveness and cost of different types of influenza vaccination reminders among adults with asthma or COPD. The specific objectives of this study were: 1) to assess the effectiveness of IVR reminders (either alone or in conjunction with postcard reminders) compared with postcard reminders only; 2) to determine the cost associated with each reminder method; and 3) to estimate the projected cost of these reminder methods if they were used in the future for all high-risk adults and children at the study site, a large managed care organization.

METHODS

Study Setting

The study was conducted between July 2012 and March 2013 in Kaiser Permanente Colorado (KPCO), a managed care organization with approximately 480,000 members in the metropolitan Denver area. KPCO uses an electronic health record (EHR), which captures demographic data, health plan enrollment information, encounter data including diagnosis codes, and immunization administration information. The local human subjects review board approved the study and written consent was not required.

Study Population

All adults aged 19 to 64 years at KPCO with a diagnosis of asthma or COPD were identified. Children and the elderly were not included in the trial, because they were already receiving vaccination reminders based upon their age. Subjects with asthma were included if they had an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis code of asthma (493.x) in the prior 3 years. From this population, subjects were excluded if they had no dispensing of an asthma-related medication in the prior 2 years. Subjects with COPD were included if they had an ICD-9-CM diagnosis code of COPD (491.x, 492.x, 493.2, and 496.x) at any time in the past. From the population with asthma or COPD, subjects were excluded if they lived in a household with other individuals in high-risk categories for influenza morbidity,^2-4 because these households were already scheduled to receive postcard reminders for influenza vaccination as usual care.

Study Design and Randomization

A 3-arm, randomized control trial was conducted of different reminder strategies for annual influenza vaccination. Subjects were aware of what type of reminder they received; however, the study aims were not described in the reminders. Randomization was performed by simple random allocation with no restrictions.

Reminder Intervention

At KPCO, it is considered usual care for every individual with asthma or COPD to receive an annual postcard reminder for influenza vaccination. In the current study, subjects received 1 of the following interventions: 1) postcard reminder only (usual care); 2) IVR reminder only; or 3) postcard-plus-IVR reminder. The content of the postcard and IVR reminders was similar: subjects were encouraged to receive influenza vaccination; groups at increased risk from influenza were highlighted; subjects were informed that no appointment was needed for vaccination; and subjects were told that vaccination was provided at no cost. However, subjects receiving IVR reminders could access additional information during the IVR call, as described below.

Postcards were mailed to subjects via standard mail during the last 2 weeks of September 2012. As is typical for standard postcards, no “return request” was made on the study postcards; therefore, it was not possible to know how many postcards were undeliverable.

An existing IVR system was used to contact subjects by telephone. The caller identification displayed “Kaiser Permanente” on the subject’s phone. The IVR reminders were designed to be interactive; using the numbers on a touch-tone telephone, subjects could listen to general information about influenza vaccination and hear a message from an asthma/COPD specialist at KPCO, with the option to listen to additional information about influenza infections if desired. A maximum of 2 telephone calls were made per subject. If the IVR system reached an answering machine, a message was left encouraging influenza vaccination. Calls were made to the primary listed telephone number in the EHR, and the IVR system requested the responder to verify their identity. The IVR system tracks when the call ends, providing specific information on how much call content each subject received. If after 2 attempts the IVR system was not able to detect a person or answering machine, the call was classified as not delivered. IVR calls were made during the last 2 weeks of September 2012; subjects in the postcard-plus-IVR reminder group may have received their IVR call before or after their postcard.

Outcome Measures

The primary study outcome was receipt of influenza vaccine by October 31, 2012, as documented in the EHR. This date was chosen because we speculated that any impact of reminders on behavior was most likely to occur within the month following the intervention. As secondary outcomes, receipt of influenza vaccine by December 31, 2012, and by March 31, 2013, were examined. Additionally, the costs of each of the 3 interventions were examined; costs were calculated for the study population and were also extrapolated to the entire population at KPCO that typically receives influenza vaccination reminders (approximately 100,000 individuals).

Statistical Analyses

Vaccination rates were compared among the 3 study arms using pairwise comparison statistics. Wald asymptotic confidence limits were used to test for differences in rates. Analyses were based on intention-to-treat.

Multivariable analyses were used to examine the effect of reminder type on receipt of influenza vaccine, while controlling for relevant covariates. For these analyses, the dependent variable was vaccination (“yes” or “no”), and the primary predictor variable was the study arm. Other covariates included age, sex, race, Hispanic ethnicity, Chronic Disease Score (CDS),¹⁹ Charlson comorbidity index (CCI) score,²⁰ insurance type, KPCO group coverage (employees of the healthcare system and their family members), outpatient visit rate, and prior receipt of influenza vaccine. CDS and CCI are comorbidity measures; CDS is based on current medication use, while CCI is based on diagnosis codes. A Poisson regression model with robust error variance²¹ was used to examine the relative risk of receipt of vaccine. Each demographic and clinical covariate of interest was tested individually for its association with risk of vaccination. A priori, study arm, age, gender, race, and Hispanic ethnicity were included in a multivariable model. Other covariates with P <.20 in bivariable models were included in preliminary models; those with an adjusted P ≥.05 were removed in a step-wise fashion to arrive at a final model. CCI was removed from the final model due to an adjusted P ≥.05. All analyses were performed using SAS version 9.2 (SAS Institute, Cary, North Carolina).

Intervention Cost Analyses

The methods used to estimate intervention costs associated with the 3-arm trial were consistent with those used in other intervention trials conducted at KPCO and elsewhere.^22-24 All intervention-related costs associated with the postcard and IVR reminders were measured. All staff tracked their time spent on reminder activities. Measured personnel hours were converted into costs, based on salary and benefits per labor category, using the highest wage range of the 2011 United States Bureau of Labor Statistics and a fringe rate of 35%.

In addition to labor costs, other intervention-related expenses were accounted for. Costs for supplies, printed materials, mailing of postcards, and the costs associated with the IVR system (telephone lines, server, licensing) were recorded. The IVR system is used for other interventions within KPCO; therefore, only the portion of IVR system costs that were associated with developing and delivering the influenza vaccination reminders was used in cost calculations. The cost per subject of each study arm was calculated by dividing the number of participants in the study arm by the total costs for the intervention in that arm.

Extrapolation of Costs

In prior years, KPCO sent approximately 100,000 postcard reminders for influenza vaccination to members at increased risk of influenza-related morbidity and mortality. While this study was conducted among patients with asthma or COPD, KPCO planned to use these results to make decisions about reminder methods for the entire high-risk population. Therefore, additional analyses were conducted to extrapolate cost estimates to a population size of 100,000. For extrapolation, it was assumed that the costs associated with cohort identification, content development, and voice talent would be fixed (ie, would not vary with the number of patients contacted), but mailing cost would vary directly with the number of mailed reminders. IVR costs associated with licensing fees and phone lines would increase only when additional capacity was needed.

RESULTS

Figure

A total of 12,428 adults aged 19 to 64 years were identified with asthma or COPD. As shown in the , 141 subjects were excluded due to having no available telephone number, and 2 were excluded due to having no mailing address. The remaining 12,285 subjects were randomly allocated to 1 of the 3 study arms, resulting in 4095 subjects per arm.

Table 1

Subjects’ baseline demographic and clinical characteristics are summarized in . Asthma was the qualifying condition for 94% of the population, COPD for 5%, and asthma with COPD for 1%.

Information on completion of the reminder calls, for the IVR-only and the postcard-plus-IVR arms, was as follows: for 56.6% and 57.9%, respectively, messages were left on answering machines; 5.6% and 5.2%, respectively, listened to a vaccination reminder but hung up prior to receiving additional information; 20.4% and 19.3%, respectively, listened through to the message from the asthma/COPD specialist; 1.3% and 1.5%, respectively, listened to all content and requested more information from the influenza hotline; and 16% and 16% of calls, respectively, were incomplete with no answer and no message left.

Table 2

By October 31, 2012, 29.5% of subjects in the postcard-only arm, 31.1% in the IVR-only arm, and 30.6% in the postcard-plus-IVR arm had received influenza vaccination. As shown in , no significant difference was found in vaccination rates among the 3 study arms for the primary outcome. Table 2 also shows influenza vaccination receipt by December 31, 2012, and March 31, 2013.

Table 3

Multivariable analyses of factors associated with receipt of influenza vaccination are shown in . After adjusting for other covariates, vaccination was not significantly higher for the IVR-only and postcard-plus-IVR arms compared with the postcard-only arm. Of interest, several covariates were significantly associated with vaccination, which was significantly more likely in older age groups, for example. Compared with those with a traditional managed care insurance plan, those with a deductible plan were significantly less likely to receive influenza vaccine (adjusted odds ratio, 0.89; 95% CI, 0.84-0.94).

Table 4

The cost associated with the different types of reminders is shown in . In the postcard-only arm, the cost per subject to receive the intervention was $0.78; for IVR-only, the cost was $1.23; and for postcard-plus-IVR, the cost was $1.93. When these costs were extrapolated to a theoretical population of 100,000 subjects, the anticipated costs per subject would be $0.55, $0.05, and $0.60 in the postcard-only, IVR-only, and postcard-plus-IVR groups, respectively. We performed an additional analysis to find the “break-even” point; 7916 postcards could be sent for the same cost as 100,000 IVR calls.

DISCUSSION

In this investigation of different reminder methods for influenza vaccination among adults with asthma or COPD, using IVR-based reminders was not significantly more or less effective for promoting influenza vaccination compared with postcard reminders or postcard-plus-IVR reminders. In addition, the IVR calls were not as interactive or tailored as was intended, as only 20% listened to a message from an asthma/COPD specialist, and only 2% requested information from the influenza hotline. The intervention achieved only modest influenza vaccination rates of 45% to 47% by the end of the influenza season, and the strongest predictor of vaccination was receipt of influenza vaccine in prior years. In cost analyses in the study population of 12,285 individuals, IVR reminders were more costly than postcard reminders due to the higher fixed costs of the IVR system. However, when costs were extrapolated to the entire high-risk population at KPCO, IVR-based reminders would have been the least expensive among the methods tested. These findings led KPCO to use IVR-based reminders as the primary method for influenza vaccination reminders for the entire high-risk population at KPCO for the 2013-2014 influenza season.

Although it was anticipated that IVR reminders would be more effective than postcard reminders for encouraging influenza vaccination, vaccination rates by October 31, 2012, were similar across the 3 study arms. IVR is considered a promising tool for health promotion and disease management because messages can be personalized and respondents can interact with the system through voice or touch-tone responses via telephone.^25-27 However, in this study, use of the IVR system did not result in substantial interaction with respondents, because most either had a message left on an answering machine or did not choose to receive additional influenza information through the hotline. It is possible that higher immunization rates could have been achieved had more individuals listened to the additional information provided.

Nationally, during the 2012-2013 influenza season, an estimated 47% of adults aged 19 to 64 years with chronic medical conditions were immunized against influenza,⁸ and 34.6% of adults with asthma aged 18 to 49 years were immunized.²⁸ These influenza immunization rates are considerably lower than the national goal of 90% coverage.⁹ The current study was conducted among insured adults in a managed care organization with walk-in immunization available—a setting presumably with fewer barriers to vaccination than faced by the general population. However, across the 3 study arms, only 45% to 47% were immunized by the end of the influenza season; several factors may have contributed to the lower-than-expected rates. Because KPCO patients are sent reminders every year and are potentially exposed to other types of reminders within KPCO (eg, in Kaiser newsletters), awareness may be high enough that patients do not need the additional cue to action that reminders provide. Additionally, it is possible some subjects were vaccinated outside of KPCO, and that this information was not entered into the KPCO EHR, resulting in falsely low rates.²⁹ Finally, misperceptions about the need for influenza vaccination are prevalent, even among individuals with chronic medical conditions^30,31; it is possible that the brief reminders used in the current study did little to change attitudes and behaviors.

While annual influenza vaccination reminders have proven effective in a variety of settings when compared with no reminders, much less is known about the cost of various reminder methods. Interestingly, IVR-based reminders were more costly than postcards for the study population of 12,285, but substantially less costly than postcards when applied to a much larger population, a finding that led KPCO to adopt IVR-based reminders for the entire high-risk population for the subsequent influenza season.

Limitations

This study is subject to several important limitations. The study did not include a control group that received no reminders; however, because vaccination reminders are a recommended standard of care nationally, and a long-standing part of usual care at KPCO, having a “no reminder” study arm was not appropriate on ethical grounds. Reminders were left on answering machines, but it is not known whether these messages were ultimately heard by the intended recipients. Some subjects may have received influenza vaccination outside of KPCO, such as at a pharmacy or workplace, and this information would not routinely be captured within the EHR.²⁹ While standard cost-capture methods were used, it is possible that not all reminder costs were measured. Finally, IVR systems can be expensive, costing $50,000 or more to initially implement or purchase. The IVR purchase or start-up costs were not included in our IVR reminder cost estimates because the KPCO IVR system has been in place for more than a decade, it is currently used for multiple purposes, and it is unlikely that a healthcare entity would purchase an IVR system solely for influenza vaccination reminders. This may limit the generalizability of our cost findings to organizations with existing IVR systems or those willing to purchase one, and we are unaware of published estimates of how prevalent IVR systems are among healthcare organizations.

CONCLUSIONS

In summary, in a randomized control trial, IVR-based reminders were not more or less effective for promoting annual influenza vaccination than postcard reminders. While IVR is a promising new technology for health promotion, when IVR-based reminders were used for influenza vaccination, many patients did not receive any tailored messaging as intended. Regardless of the type of reminder used in this study, only modest influenza vaccination rates were achieved, indicating that additional strategies may be needed to overcome misperceptions about the need for annual influenza vaccination.

Acknowledgments

We gratefully acknowledge Jeff Holzman for data management and Jonah Langer and Jenn Boggs for their technical assistance with the IVR delivery system.

Author Affiliations: Institute for Health Research (JAS, JE, DPR, MFD) and Department of Population and Prevention Services (CM, CK, CL), Kaiser Permanente Colorado, Denver, CO; Department of Pediatrics, University of Colorado School of Medicine (MFD), Aurora, CO.

Source of Funding: This study was funded by an internal pilot grant from Kaiser Permanente Colorado’s Institute for Health Research.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. Trial registration: Clinicaltrials.gov: NCT01852656, Effectiveness of Influenza Vaccine Reminder Systems.

Authorship Information: Concept and design (JAS, DPR, MFD, CK, CM, CL); acquisition of data (JAS, JE, MFD, CK, CM, CL); analysis and interpretation of data (JAS, DPR, JE, MFD, CK, CM); drafting of the manuscript (JAS, DPR, JE, MFD, CL); critical revision of the manuscript for important intellectual content (JAS, DPR, JE, MFD, CM); statistical analysis (JAS, JE); obtaining funding (MFD); administrative, technical, or logistic support (JAS); supervision (MFD).

Address correspondence to: Jo Ann Shoup, MS, Institute for Health Research, Kaiser Permanente Colorado, 10065 E Harvard Ave, #300, Denver, CO 80231. E-mail: jo.ann.shoup@kp.org

REFERENCES

influenza. Crit Care Med. 2010;38(4 suppl):e91-e97. Review.

1. Rothberg MB, Haessler SD. Complications of seasonal and pandemic

respiratory virus infections on persons with chronic underlying conditions.

JAMA. 2000;283(4):499-505.

2. Glezen WP, Greenberg SB, Atmar RL, Piedra PA, Couch RB. Impact of

Edwards KM. Winter viruses: influenza- and respiratory syncytial

virus-related morbidity in chronic lung disease. Arch Intern Med.

3. Griffin MR, Coffey CS, Neuzil KM, Mitchel EF Jr, Wright PF,

2002;162(11):1229-1236.

of respiratory viruses in patients hospitalized with near-fatal asthma,

acute exacerbations of asthma, or chronic obstructive pulmonary

disease. Am J Med. 2003;115(4):272-277.

4. Tan WC, Xiang X, Qiu D, Ng TP, Lam SF, Hegele RG. Epidemiology

5. Skrepnek GH, Skrepnek SV. Epidemiology, clinical and economic

burden, and natural history of chronic obstructive pulmonary disease

and asthma. Am J Manag Care. 2004;10(5 suppl):S129-S138. Review.

of the Advisory Committee on Immunization Practices

(ACIP)—United States, 2012-13 influenza season. MMWR Morb Mortal

Wkly Rep. 2012;61(32):613-618.

6. CDC. Prevention and control of influenza with vaccines: recommendations

influenza with vaccines: recommendations of the Advisory Committee

on Immunization Practices (ACIP), 2010. MMWR Recomm Rep.

2010;59(RR-8):1-62.

7. Fiore AE, Uyeki TM, Broder K, et al; CDC. Prevention and control of

System [data sources]. Flu vaccination coverage, United States,

2012-2013 influenza season. CDC website. http://www.cdc.gov/flu/

fluvaxview/coverage-1213estimates htm#key-findings 2013. Updated

September 25, 2013. Accessed October 22, 2013.

8. National Immunization Survey and Behavioral Risk Factor Surveillance

Washington, DC: HHS; 2000.

9. Healthy People 2010: Understanding and Improving Health. 2nd ed.

10. Jacobson Vann JC, Szilagyi P. Patient reminder and patient recall

systems to improve immunization rates. Cochrane Database Syst Rev.

2005;(3):CD003941.

vaccination rates of those 60 years and older in the community.

Cochrane Database Syst Rev. 2010; (9):CD005188. Review.

11. Thomas RE, Russell M, Lorenzetti D. Interventions to increase influenza

and cost-effectiveness of letters, automated telephone messages, or

both for underimmunized children in a health maintenance organization.

Pediatrics. 1998;101(4):E3.

12. Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness

13. Suh CA, Saville A, Daley MF, et al. Effectiveness and net cost

of reminder/recall for adolescent immunizations. Pediatrics.

2012;129(6):e1437-e1445.

practice-based recall for childhood immunizations: a randomized

controlled comparative effectiveness trial. Am J Public Health.

14. Kempe A, Saville A, Dickinson LM, et al. Population-based versus

2013;103(6):1116-1123.

of studies 1989-2000. Behav Res Methods Instrum Comput.

2002;34(3):342-353.

15. Corkrey R, Parkinson L. Interactive voice response: review

delivered through interactive voice response telephone messages to

promote weight loss in a pre-diabetic population. Patient Educ Couns.

2008;72(1):34-41.

16. Estabrooks PA, Smith-Ray RL. Piloting a behavioral intervention

17. Estabrooks PA, Shoup JA, Gattshall M, Dandamudi P, Shetterly S,

Xu S. Automated telephone counseling for parents of overweight children:

a randomized controlled trial. Am J Prev Med. 2009;36(1):35-42.

18. Skolarus TA, Holmes-Rovner M, Hawley ST, et al. Monitoring quality

of life among prostate cancer survivors: the feasibility of automated

telephone assessment. Urology. 2012;80(5):1021-1026.

automated pharmacy data. J Clin Epidemiol. 1992;45(2):197-203.

19. Von Korff M, Wagner EH, Saunders K. A chronic disease score from

20. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of

classifying prognostic comorbidity in longitudinal studies: development

and validation. J Chronic Dis. 1987;40(5):373-383.

Statistical Models. 4th ed. Boston, MA: WCB/McGraw-Hill; 1996.

21. Neter J, Kutner MH, Nachtsheim CJ, Wasserman W. Applied Linear

22. Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing behavioral

interventions: a practical guide to enhance translation. Ann

Behav Med. 2009;37(2):218-227.

23. Ritzwoller DP, Sukhanova AY, Glasgow RE, et al. Intervention costs

and cost-effectiveness for a multiple-risk-factor diabetes self-management

trial for Latinas: economic analysis of ¡Viva Bien! Transl Behav

Med. 2011;1(3):427-435.

24. Ritzwoller DP, Glasgow RE, Sukhanova AY, et al; Be Fit Be Well

Study Investigators. Economic analyses of the Be Fit Be Well program:

a weight loss program for community health centers. J Gen Intern

Med. 2013;28(12):1581-1588.

management of chronic disease. Am J Manag Care. 2000;6(7):817-827.

25. Piette JD. Interactive voice response systems in the diagnosis and

26. Stacy JN, Schwartz SM, Ershoff D, Shreve MS. Incorporating

tailored interactive patient solutions using interactive voice response

technology to improve statin adherence: results of a randomized

2009;12(5):241-254.

clinical trial in a managed care setting. Popul Health Manag.

Strategies for Behavior Change. New York, NY: Routledge; 2012.

27. Noar SM, Harrington NG, eds. eHealth Applications: Promising

28. CDC. Vaccination coverage among persons with asthma—United

States, 2010-2011 influenza season. MMWR Morb Mortal Wkly Rep.

2013;62(48):973-978.

vaccination status at four Vaccine Safety Datalink sites. Am J Prev

Med. 2009;37(6):552-555.

29. Greene SK, Shi P, Dutta-Linn MM, et al. Accuracy of data on influenza

attitudes, and behavior among high-risk outpatients. Arch Intern

Med. 1992;152(1):106-110.

30. Nichol KL, Lofgren RP, Gapinski J. Influenza vaccination: knowledge,

influenza vaccine safety for individuals with chronic medical illness.

Prev Med. 2007;45(1):80-82.

31. Kempe A, Daley MF, Crane LA, et al. Misperceptions regarding

Download PDF: Effectiveness and Cost of Influenza Vaccine Reminders for Adults With Asthma or Chronic Obstructive Pulmonary Disease