Randomized Trial of an Electronic Asthma Monitoring System Among New York City Children

Objectives: To test the efficacy of an electronic asthma monitoring system (AMS) to reduce pediatric emergency department (ED) visits and hospitalizations for asthma.

Study Design: Randomized clinical trial. Methods: Families of pediatric patients with asthma aged 8 to 17 years were recruited at 6 medical centers. Children were randomly assigned to the American Medical Alert Corporation pediatric AMS or a paper diary. The numbers of and costs associated with ED visits and hospitalizations for the 2 groups in the year following randomization were compared using t tests of statistical significance.

Results: Of 59 children recruited to the trial, 29 were randomized to the AMS and 30 to the diary. The 2 groups were similar in demographic and clinical characteristics. During their study year, 24 AMS group members logged on a mean (SD) of 211.0 (117.3) days; 13 diary group members provided data on a mean (SD) of 136.6 (128.0) days. During the 32 months that the study was in progress, the case managers logged on a mean (SD) of 171.0 (97.2) days. Overall, 35 children had at least 1 ED visit, but only 7 children were hospitalized. The 2 groups had no statistically significant differences in the numbers of or charges associated with ED visits or hospitalizations.

Conclusion: Electronic devices are being developed to make chronic disease management easier for patients and their families, but they should not be adopted without careful study, including randomized trials, to ascertain their use, costs, and benefits.

(Am J Manag Care. 2009;15(11):809-814)

In a randomized trial, children with asthma who used an electronic asthma monitoring system had as many emergency department visits and hospitalizations as children who used a paper diary.

• Clinical trial methods can be a valuable aid to decision making about reimbursement for costly new technologies.
• Case managers are typically busy dealing with urgent conditions and needy patients. Expecting them to log on to a Web site on a daily basis without a prompt may be unrealistic.
• Whether adding a prompt would yield better results for the electronic system can be determined but would require further research.

Asthma is the most common chronic disease in children. Despite intensive asthma control efforts during the past decade, the disease is the number one cause of emergency department (ED) visits and hospitalizations. Asthma impairs the quality of life of patients and families in the short term, and frequent absences from school may limit the aspirations of children with poorly controlled asthma, as well as their potential contributions to society in the long term. For patients with more than mild asthma with infrequent exacerbations, treatment involves daily use of a controller medication, usually an inhalable corticosteroid, plus rescue medication for use during exacerbations.¹ Good outcomes for patients with asthma rely on effective patient education, regular and correct use of medications, and early recognition of symptoms requiring medication or medical attention. Poor outcomes in the treatment of asthma may be attributed in part to suboptimal patient adherence to the asthma action plan (treatment plan), ineffective communication between patient and physician,² and inadequate or nonexistent asthma action plans.^3,4 To assess adherence to the treatment plan, physicians often ask their patients to keep an asthma diary, but compliance with diaries is generally poor. Sharek et al⁵ studied children who averaged 0.41 hospitalizations and 3.5 ED asthma “sick” visits in the 2 months before enrollment in the study. They found that, among parents of 119 disadvantaged inner-city children with moderate-to-severe asthma, compliance with an asthma diary was 64%. However, new technologies offer the prospect of improving adherence,⁶ information about adherence,⁷ or both.

In New York City, approximately 300,000 children (17%) have been diagnosed as having asthma at some time in their lives.⁸ The prevalence of asthma is highest among the low-income neighborhoods of the South Bronx, East Harlem, and Central Harlem. Children who live in those neighborhoods are almost 3 times as likely to be hospitalized for asthma as those in higher-income neighborhoods in part because of inadequate control of their condition.⁸

The hospitals of the New York City Health and Hospitals Corporation (HHC) are primary providers of healthcare for low-income children in New York City. In 2002, more than 20,000 pediatric visits to HHC EDs and urgent clinics carried a primary diagnosis of asthma (6% of all pediatric ED visits and 35% of urgent outpatient visits). Since 1997, pediatric ED visits and hospitalizations have declined somewhat,⁹ but better asthma management seems achievable and may reduce the economic and human costs of this chronic condition.

In 2002, in an attempt to reduce the ED and hospitalization rates among his most difficult pediatric patients with asthma,¹⁰ the director of pediatrics at HHC’s Coney Island Hospital provided the Health Buddy, a computerized device developed by the American Medical Alert Corporation (AMAC [Oceanside, NY]) and programmed with their pediatric asthma monitoring system (AMS), to 69 children aged 8 to 16 years who had been diagnosed as having moderate-to-severe asthma. The patients had had a mean of 2.4 ED visits per month and 1 hospitalization every 7 weeks during the previous winter. Based on the prior winter’s data from Coney Island Hospital, the children were expected to have 2 to 3 ED visits per month and 3 to 4 hospitalizations during the 6-month study period from October 2002 to March 2003. In fact, only 1 child had an ED visit and was hospitalized (W. Seigel, MD, and M. Rutstein, RN, oral communication, July 2003). These results stimulated considerable interest in the new technology. We initiated a randomized controlled trial to test the hypothesis that among children with moderate-to-severe asthma those assigned to use the AMS would have fewer ED visits and hospitalizations than those assigned to use a paper diary.

Methods

MetroPlus Health Plan, a wholly owned subsidiary of HHC and its primary managed care plan, covers more than 105,000 children, mainly through Medicaid. We conducted the study among MetroPlus Health Plan member children receiving care for asthma at the following 6 HHC medical centers: Bellevue Medical Center, Elmhurst Hospital Center, Harlem Hospital Center, Jacobi Medical Center, North Central Bronx Hospital, and Woodhull Medical and Mental Health Center. Together these hospitals receive approximately 3000 emergent visits per year from children aged 8 to 17 years with a primary diagnosis of asthma.

The AMAC System

The AMAC electronic monitoring systems are used to assist patients with self-management of various chronic conditions, including congestive heart failure and diabetes mellitus. The pediatric AMS is installed in patients’ homes to help them track their status with respect to their condition, to remind them to take their medication as prescribed, to instruct them to consult their provider when necessary, and to keep their case manager apprised of their status. The AMS uses patient dialogues written for a fourthgrade reading level but is considered interpretable by most children at least 8 years old. The hardware is a hand-sized electronic device with 4 keys and cords that plug into an electrical outlet and a telephone jack. It operates toll free.

Daily at a predetermined time, the device beeps and invites the child to answer a short list of questions (varied from encounter to encounter) about his or her asthma symptoms and use of medications since the last check. If the responses to these questions indicate that the child is having an asthma exacerbation, the system responds with appropriate instructions. The data are uploaded to a central site, where a clinician or case manager is responsible for reviewing them; he or she may then telephone the home to assess the child’s status, adjust the child’s medications if necessary, or have the child come in to see the physician.

Questions and answers are designed to identify symptoms early. Additional trivia questions are designed to keep children engaged while they learn about managing their condition.

The Paper Diary

The diaries (see eAppendix available at www.ajmc.com) were designed to replicate, to the extent that paper could do so, the questions and information that the AMS used. However, the primary purpose of the AMS was not data collection but facilitation of asthma self-management. The diaries provided the control group with a tool that, although commonly used for the same purpose, lacked the electronic reminder features of the AMS. We viewed it as a kind of placebo. For each month, a cover sheet contained instructions for self-monitoring of peak expiratory flow, definitions of asthma zones, and the names of the child’s controller and rescue medications. On subsequent pages were questions about whether the child had gone to school, how the day had gone, the morning and evening peak expiratory flows, medication use, any symptoms, and what the child had done about the symptoms. Columns for each day of the week contained “Ys” for yes, “Ns” for no, or emoticons to be circled as appropriate. Each day’s column was a different color. Each child received a large white 3-ring binder with 12 tabs (labeled for each month of the study year) and a cover sheet and five 1-week diary pages per tab. In a pocket of the binder were 12 addressed and postage-paid envelopes in which each month’s diaries could be mailed to the case managers. The binders were attractively personalized with the child’s name.

Subjects

Patients aged 8 to 17 years who had 2 or more ED visits or 1 hospitalization with a primary diagnosis of asthma at 1 of 6 participating HHC medical centers in the year before recruitment were invited to participate in the trial. Continuous eligibility in MetroPlus Health Plan was not a requirement for participation in the study. The study was approved by the institutional review boards of the participating hospitals and Columbia University.

After giving their informed consent (via parent or guardian) and assent, participating children were randomly assigned to receive the AMS or to be control subjects. All study materials were made available in English and Spanish. In a few cases, interpreters translated the consent and assent forms into other languages for the parents.

Each hospital assigned a clinical case manager to be responsible for daily review of and, if appropriate, response to the information that the AMS automatically sent to a passwordprotected Web site every night. Upon enrolling a child, the case manager sent the child’s contact information to MetroPlus Health Plan, which held the randomization codes and forwarded the information to AMAC staff or the diary health educators, depending on the randomization assignment.

The AMAC staff or the diary health educators then arranged to visit the child, to bring the AMS or diary, and to train the child in its use. All children were taught the definitions of the green, yellow, and red zones commonly used in asthma action plans.

All training included instruction in the correct use of the peak expiratory flow monitor, determination of the child’s personal best and yellow and red zone ranges, and directions to monitor peak expiratory flow and to use the AMS or diary daily. All participating facilities used uniform clinical protocols for managing pediatric patients with asthma. All study participants received appropriate standard care during their trial participation. AMAC trained the case managers and set up access to the AMS Web site for the patients on their computers. Case managers were instructed to contact patients and to arrange for them to see their provider when they entered the red zone.

Data Collection

Upon enrollment, all study participants completed a demographic and behavioral questionnaire. The primary outcomes of interest were ED visits for asthma, hospitalizations for asthma, and their costs. These data were obtained from the MetroPlus Health Plan member utilization database.

Data Analysis

Children in the AMS and diary groups were compared with respect to demographic characteristics and asthma status at baseline. _X² tests were used to assess the statistical significance of differences between the 2 groups. The mean numbers of and charges for ED visits and hospitalizations in the year following enrollment were compared between groups using t tests and _X² tests.

Results

From August 2004 through April 2006, a total of 59 subjects were recruited (Table 1), of whom 29 were randomized to the AMS and 30 to the diary. The groups did not differ with respect to sex (30 girls and 29 boys), age at baseline (age range, 8-15 years for both), race/ethnicity (23 non-Hispanic black, 26 Hispanic, and 10 other or unknown), or recent history of asthma attacks. Overall, 50 subjects had had at least 1 ED visit in the prior year, and 24 subjects had had at least 1 hospitalization.

Adherence to the treatment assignments differed by group. Of 29 children assigned to the AMS, 5 never used the device; the remaining 24 logged on a mean of 211.0 days (Table 2). Their responses indicated that they were in the yellow zone on a mean of 66.1 days and in the red zone on a mean of 69.3 days (children could be in both the yellow zone and the red zone in a single day). Of 30 children assigned to the diary, only 13 submitted any diaries; the 13 submitted diary entries for a mean of 136.6 days. During the 32 months from the enrollment of the first child to the completion of the last child’s study year, the case managers logged on a mean (SD) of 171.0 (97.2) days.

During their study year, 24 children (12 per group) had no ED visits; 26 (13 per group) had 1 to 3 visits, and 9 (4 AMS and 5 diary) had more than 3 visits (Table 3). Only 7 children were hospitalized overnight (5 AMS and 2 diary), and only 1 of those was hospitalized more than 3 times. Overall, the number who had at least 1 ED visit declined from 50 in the year preceding the study year to 24 in the study year, and the number who were hospitalized declined from 24 to 7 (P <.001 for both). Differences in the distribution of ED visits and hospitalizations between groups were not statistically significant. The mean ED and hospital charges were higher in the AMS group than in the diary group, but the differences were not statistically significant.

Discussion

Fifty-nine study children with moderate-to-severe and persistent asthma received care at 1 of 6 HHC medical centers. The children assigned to the AMS did not differ from those assigned to the diary with respect to ED visit and hospitalization numbers and costs.

A possible explanation is that expensive electronic devices are not necessary; a paper diary, attractively packaged, personalized with the child’s name, and presented with appropriate fanfare, is all that is needed to train children in asthma self-management and improve asthma outcomes. The reduction in ED visits and hospitalizations in both groups supports this explanation, which may be a variant of the Hawthorne effect. Because the outcomes of interest were fewer than expected, the study sample size may have been too small to show an effect, but to the extent that the outcomes differed at all, the difference favored the diary group. The reasons for those differences are unknown; the most plausible explanation is unstable data because of the small sample size.

The log data indicate that 24 of 29 children (83%) in the AMS group made use of their devices and that they did so on more than half of the days in their study year. Only 13 of 30 children (43%) in the diary group submitted diary entries, and those who submitted any entries did so for many fewer days than the AMS group logged on, but the diary group fared at least as well as the AMS group. Hence, although (as expected) the AMS group was more adherent to their treatment than the diary group, adherence did not result in better outcomes.

The decline in ED visits and hospitalizations seems not to be due to children having outgrown their symptoms. Based on the AMS log data, children were in the red or yellow zone or both on close to one-third of the days on which they logged on to the AMS. The children who logged on may have been more symptomatic than those who did not. In the AMS group, 5 children had a total of 21 hospitalizations, of which 1 child (who never used the AMS) accounted for 12.

Recruitment to the trial was slower and more difficult than had been expected. It took the 6 participating medical centers 20 months to recruit 59 children. Some families were not reachable by the available contact information. Others were unwilling to participate when asked. It seems reasonable to assume that the participating families were a selected group that would be no less adherent than nonparticipants to an asthma intervention. Many of the parents who chose not to participate seemed so burdened by their child’s asthma that they could not appreciate the potential of the AMS to relieve them of the stress involved in having to check the child’s symptoms and remind the child to take medication every day. It is possible that even those families that participated failed to perceive that potential benefit. Although such low-income urban families bear more than their share of the burden of asthma, their child’s asthma may be only one of their burdens and not always the most urgent. Participating in a trial may have seemed unlikely to be useful to them.

The case managers provided less monitoring than expected. In another study,¹¹ a similar device (without case manager involvement) had no effect on ED visits or hospitalizations. The Coney Island Hospital project¹⁰ may have owed its success to the commitment of the pediatric pulmonologist and the clinical coordinator. They were insistent that the children use the devices and were proactive in contacting children who, based on peak expiratory flow and symptoms, seemed to be at risk for an asthma exacerbation. The case managers in our study varied considerably in the frequency with which they logged on to review the data entered by their AMS patients, but overall they logged on much less often than their AMS patients. Given the number of days when the AMS patients were in their red zones, the case managers often may have failed to respond until several days later.

All participating pediatric pulmonology facilities were in large medical centers serving populations among which asthma is highly prevalent. In such facilities, case managers are likely to be dealing with constant pressure from patients with serious health problems who are physically present or in contact by telephone. Resources are often scarce and staffing minimal. Staff turnover and the replacement of individuals who had been trained in the use of the AMS and had participated in the initiation of the study by other staff during the study period may account for much of the failure to log on to the AMS and for the lack of observed benefit. Given that staff tend to focus on the squeakiest wheels, the AMS might be more effective if it had the ability to get the case manager’s attention when necessary. Perhaps a modified system could make the case manager’s computer beep or send an e-mail or page when a patient is in the yellow or red zone. However, determining whether such a modification would have better results would require another trial.

The AMS used in the trial involved a cost for installation and a monthly cost for rental of the device. The asthma diary in its personalized binder, with color-printed pages, envelopes, and postage, involved a 1-time cost that was less than 2 months’ rental. If the device had shown a benefit (ie, fewer hospitalizations or ED visits), we would have conducted a cost-benefit analysis, but it did not.

Electronic monitoring systems have shown a benefit in connection with other chronic diseases such as diabetes12 and cardiac conditions.13 Other devices are, no doubt, in development. Many may seem promising. The Coney Island Hospital¹⁰ experience with the AMS, for example, was convincing to many people, but it did not involve a simultaneous comparison group, and as we learned from our trial, the historical comparison was misleading. Technology, especially new technology, is expensive. It has enormous potential benefits, but they cannot be taken for granted. The difference in adherence between the AMS group and the diary group supported our expectation that children would be drawn to cool-looking electronic toys, but the lack of difference in results shows that patient adherence alone may not yield the desired outcome. Given the important role that the AMS gave to case managers, greater efforts to maintain their continued involvement might have altered the outcome. That is an important lesson of the trial. Our experience confirms that, whether managed care providers or others are covering the costs of innovations, randomized trials have merit as a way to evaluate the risks, benefits, and potential needs for modification of technologic innovations before large sums of money are committed.

Author Affiliations: From the Department of Epidemiology, Mailman School of Public Health (JSJ, MC), and the Department of Pediatrics, College of Physicians and Surgeons (VEH), Columbia University, New York, NY; and MetroPlus Health Plan (AL, AS) and the following hospitals of the New York City Health and Hospitals Corporation: Bellevue Medical Center (AHF), Elmhurst Hospital Center (LR), Harlem Hospital Center/Columbia University (VEH), Jacobi Medical Center/North Central Bronx Hospital (DS), and Woodhull Medical and Mental Health Center (ERF).

Funding Source: This study was supported by a grant from The Commonwealth Fund.

Author Disclosure: The authors (JSJ, AL, AHF, ERF, VEH, LR, DS, MC, AS) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (JSJ, AL, VEH, AS); acquisition of data (AL, AHF, ERF, VEH, LR, DS, AS); analysis and interpretation of data (JSJ, ERF, MC, AS); drafting of the manuscript (JSJ, AL, MC, AS); critical revision of the manuscript for important intellectual content (AHF, ERF, LR, DS, AS); statistical analysis (JSJ, MC); provision of study materials or patients (AHF, VEH, LR, DS); obtaining funding (AL, AS); administrative, technical, or logistic support (JSJ, AL, AHF, LR); and supervision (AHF, ERF, AS).

Address correspondence to: Judith S. Jacobson, DrPH, MBA, Department of Epidemiology, Mailman School of Public Health, Columbia University, 722 W 168th St, Rm R736, New York, NY 10032. E-mail: jsj4@columbia.edu.

1. Wise M, Gustafson DH, Sorkness CA, et al. Internet telehealth for pediatric asthma case management: integrating computerized and case manager features for tailoring a Web-based asthma education program. Health Promot Pract. 2007;8(3):282-291.

2. Braganza S, Sharif I, Ozuah PO. Documenting asthma severity: do we get it right? J Asthma. 2003;40(6):661-665.

3. Sheares BJ, Du Y, Vazquez TL, Mellins RB, Evans D. Use of written treatment plans for asthma by specialist physicians. Pediatr Pulmonol. 2007;42(4):348-356.

4. Bauman LJ, Wright E, Leickly FE, et al. Relationship of adherence to pediatric asthma morbidity among inner-city children. Pediatrics. 2002;110(1, pt 1):e6.

5. Sharek PJ, Mayer ML, Loewy L, et al. Agreement among measures of asthma status: a prospective study of low-income children with moderate to severe asthma. Pediatrics. 2002;110(4):797-804.

6. Charles T, Quinn D, Weatherall M, Aldington S, Beasley R, Holt S. An audiovisual reminder function improves adherence with inhaled corticosteroid therapy in asthma. J Allergy Clin Immunol. 2007;119(4):811-816.

7. Butz AM, Donithan M, Bollinger ME, Rand C, Thompson RE. Monitoring nebulizer use in children: comparison of electronic and asthma diary data. Ann Allergy Asthma Immunol. 2005;94(3):360-365.

8. Karpati AM, Matte T, Kass D, et al. Asthma can be controlled. NYC Vital Signs. 2003;2(4):1-4.

9. Garg R, Karpati A, Leighton J, Perrin M, Shah M. Asthma Facts. 2nd ed. New York, NY: New York City Dept of Health and Mental Hygiene; May 2003.

10. The Commonwealth Fund Quarterly. “Asthma Buddy” May Help Kids Breathe Easier, Stay in School. New York, NY. 2003;9(3)7,9.

11. Guendelman S, Meade K, Benson M, Chen YQ, Samuels S. Improving asthma outcomes and self-management behaviors of inner-city children: a randomized trial of the Health Buddy interactive device and an asthma diary. Arch Pediatr Adolesc Med. 2002;156(2):114-120.

12. Bellazzi R, Arcelloni M, Ferrari P, et al. Management of patients with diabetes through information technology: tools for monitoring and control of the patients’ metabolic behavior. Diabetes Technol Ther. 2004;6(5):567-578.

13. Anliker U, Ward JA, Lukowicz P, et al. AMON: a wearable multiparameter medical monitoring and alert system. IEEE Trans Inf TechnolBiomed. 2004;8(4):415-427.

Download PDF: Randomized Trial of an Electronic Asthma Monitoring System Among New York City Children