Publication
Article
The American Journal of Managed Care
Development and Validation of a Medication Intensity Scale Derived From Computerized Pharmacy Data That Predicts Emergency Hospital Utilization for Persistent Asthma
Author(s):
Objective: To validate a risk stratification scheme using computerized pharmacy data to predict emergency hospital utilization for persistent asthma.
Study Design: Retrospective cohort.
Methods: The development sample consisted of 1079 HMO members aged 18 to 56 years with persistent asthma. The scale used medication cut-points as predictors for next-year emergency hospital utilization in a stepwise logistic regression model. Prediction properties were evaluated in a validation sample of 24 370 patients aged 18 to 56 years in a separate persistent-asthma database.
Results: Increasing use of β-agonists (odds ratio [OR] of 2.2 for 5-13 vs 0-4 canisters; OR of 2.4 for >13 vs 5-13 canisters) and oral corticosteroids (OR of 2.6 for >2 vs 0-2 dispensing events) in the first year independently predicted emergency hospital utilization in the second year. Assigning 1 point for exceeding each of the above 3 medication thresholds led to a 4-level medication intensity scale that was significantly (P < .0001) related to validated measures of asthma symptom severity, asthma control, and asthma quality of life in the development sample. In the validation sample, this scheme identified a high-risk group that was 6 times more likely than the low-risk group to require subsequent emergency hospital care, with overall sensitivity of 65% and specificity of 54%. This scale did not perform as well as a scale based on both baseline emergency hospital care and pharmacy data.
Conclusion: This simple risk stratification scheme can be used for populations with persistent asthma for whom computerized pharmacy data, but not computerized prior utilization data, are available.
(Am J Manag Care. 2006;12:478-484)
Asthma is a common chronic medical condition that exacts a high human and economic cost in our society.1,2 The prevalence and impact of asthma, along with the availability of effective controller therapy, make it an appropriate disease for population management, which involves disease identification, risk stratification, and therapeutic intervention in specified populations. Risk stratification is used in population management to identify those asthmatic patients who are most likely to experience morbidity and resource utilization, and for whom targeted intervention should reduce these risks. In addition, risk adjustment is necessary when using databases to evaluate the effects of interventions on asthma outcomes.
Several prior studies developed and validated risk stratification schemes based on electronic data to predict subsequent emergency hospital utilization for asthma.3-6 However, all of these schemes included prior emergency hospital utilization as an important component of the risk prediction algorithm. Computerized pharmacy data are becoming increasingly available, but frequently they are not linked to hospital utilization data. Even if utilization data that could be linked to pharmacy data were available, it would be simpler to be able to use only 1 computer system for risk stratification.
We are aware of 3 prior studies that attempted to predict subsequent emergency hospital care based on stratification of computerized pharmacy data. Two were published only in abstract form,7,8 and the other one was not validated in a separate population.9 Thus, we thought it would be important to develop and validate a risk stratification scheme based on computerized pharmacy data alone that would predict subsequent emergency hospital utilization and identify patients for targeted intervention. In addition, we compared the performance of this medication-derived scale with that of a previously validated scale that included baseline emergency hospital utilization as well as pharmacy data.6
METHODS
This study is a retrospective cohort database study using 1 sample (the development sample) to develop the risk stratification scheme and a separate sample (the validation sample) to validate it. The study was approved by the Northern California, Northwest, and Southern California Regional Kaiser-Permanente institutional review boards.
Development Sample
Patients. Surveys were sent in August 2000 to a random sample of Kaiser-Permanente Medical Care Program adult members aged 18 to 56 years from the Northern California (n = 3072), Northwest (Oregon and Washington) (n = 543), and Southern California (n = 3251) regions who were diagnosed as having persistent asthma in 1999. The diagnosis of persistent asthma was based on the National Committee for Quality Assurance Health Plan Employer Data and Information Set (HEDIS) criteria10: (1) 4 or more asthma medications dispensing events, or (2) 1 or more emergency department visits or hospitalizations with a principal diagnosis of asthma, or (3) 4 or more asthma outpatient visits with 2 or more asthma medication dispensing events. The upper age of 56 years was chosen to reduce the inclusion of patients whose primary disease was chronic obstructive pulmonary disease, as was done in the HEDIS cohort.10 Completed surveys were returned between August and October 2000 by 4175 members (61%), of whom 3765 (90%) answered yes to the question, "Have you ever been told by a doctor that you have asthma?" Computerized pharmacy information and year 2001 asthma hospitalization and emergency department information were available for 1079 patients who met the HEDIS persistent-asthma criteria in 2000 and who were continuously enrolled in 2000 and 2001. These patients represent the development sample for this study.
Survey Information. The survey included 3 validated asthma tools: (1) the Mini Asthma Quality of Life Questionnaire,11 a disease-specific quality-of-life questionnaire with a 2-week window and in which higher scores indicate better quality of life; (2) the Asthma Therapy Assessment Questionnaire,12 a validated asthma- control tool that has 5 levels, corresponding to 0 to 4 asthma-control problems as assessed over the prior 4 weeks; and (3) the Asthma Outcomes Monitoring System,13 a symptom intensity scale that assesses symptoms over the previous 4 weeks.
Utilization Information. Survey records were matched using a unique medical record number to year 2000 and 2001 computerized pharmacy records. These data included canisters per year of short-acting β-agonists, canisters per year of inhaled corticosteroids (not weighted for potency or puffs per container), and dispensings per year of oral corticosteroids. The 1079 patients in the development sample also had information on year 2001 asthma emergency department (ED) visits and hospitalizations available from linked hospital and outpatient visit databases.
Development of the Medication Intensity Scale
Candidate predictors for the medication intensity scale were the number of year 2000 (1) β-agonist canisters, cut at more than 0, more than 1, more than 2...and more than 17 per year; (2) inhaled-corticosteroid canisters, cut at more than 0, more than 1, more than 2...and more than 17 per year; and (3) oral-corticosteroid dispensings, cut at 1 or more, 2 or more, 3 or more, and 4 or more. The outcome was emergency hospital utilization for asthma during the year 2001, defined as at least 1 hospitalization or ED visit for asthma (International Classification of Diseases, Ninth Revision [ICD-9] code 493.xx), expressed as a yes/no variable. Stepwise logistic regression was used to identify independent predictors from among the above candidate predictors. In this analysis, the predictor with the strongest relationship to the outcome entered the model first. Subsequently, only predictors independently related at P <.05 entered the model. Risk points then were assigned based on the relative odds ratios (ORs) of the predictors in the final logistic regression model.
Validation Sample
Patients. The validation sample was derived from patients in the Southern California Kaiser Permanente asthma database. This database makes use of administrative data to define patients as having persistent asthma if they meet 1 or more of the following criteria in a 12-month period:
- A hospital discharge principal diagnosis of asthma (ICD-9 code 493.xx).
- An ED visit with asthma as the principal diagnosis (ICD-9 code 493.xx).
- Four or more dispensings of any combination of inhaled β-agonists, inhaled corticosteroids, other inhaled anti-inflammatory drugs, and oral leukotriene modifiers, excluding oral corticosteroids (which may have been used for illnesses other than asthma) and oral theophylline (for which specific information was not available in this database).
These criteria are similar to the HEDIS criteria except for not capturing patients with 4 or more outpatient visits and 2 or more medications, because number of outpatient visits was not available in this dataset. Subjects in the validation sample were patients aged 18 to 56 years (to conform to the adult HEDIS population) who were included in the asthma database during both 2002 and 2003.
Utilization Information. The utilization information included canisters of β-agonists and dispensings of oral corticosteroids in 2002 and 2003 as well as asthma ED visits and hospitalizations during 2002 and 2003.
Data Analysis
Relationships between the medication intensity scale levels and the Mini Asthma Quality of Life Questionnaire, the Asthma Therapy Assessment Questionnaire, and the Asthma Outcomes Monitoring System were evaluated by means of 1-way analysis of variance. The outcome to be predicted in the development and validation samples was emergency hospital care during the second year of observation. Emergency hospital care was defined as 1 or more asthma hospitalizations or ED visits as a yes/no variable. Prediction properties of the scale were assessed by calculation of sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic (ROC) curve for various cut-offs of the scale. The latter was derived from the C statistic of a logistic regression model, with second-year emergency hospital care as the outcome and first-year scale cut-offs as the predictors.
The predictive properties of the medication scale in the current validation sample were compared with prediction properties in this sample of the previously validated risk stratification scheme that includes baseline- year emergency hospital care.6 In this scheme 2 points are assigned for baseline-year emergency hospital care; 1 point for any baseline-year dispensings of oral corticosteroids; and 1 point for dispensing of 15 or more β-agonist canisters in the baseline year. This leads to scale scores of 0-4.
All analyses were conducted using SAS software, version 8.2, for Windows (SAS Institute Inc, Cary, NC). All P values are 2 sided, and the term "significant" refers to a P value of <.05.
RESULTS
Characteristics of patients in the development and validation samples are compared in Table 1. Patients in the development sample were somewhat older, more likely to be female, and more likely to require emergency hospital care than patients in the validation sample (Table 1).
Increasing use of inhaled β-agonists and oral corticosteroids were significant independent predictors of future (2001) emergency hospital care for asthma in the development sample (Table 2). Two β-agonist thresholds (>4 vs 0-4 canisters and >13 vs 5-13 canisters) and 1 oral-corticosteroid threshold (>2) were independent predictors of subsequent emergency hospital care. No inhaled-corticosteroid predictor entered the model. Noting the similar ORs associated with exceeding each of the β-agonist thresholds and the oral-corticosteroid threshold, we defined a simple 4-level risk score that assigned 1 point for each risk threshold that was exceeded (Table 2). Thus, 1 point was conferred by more than 4 β-agonist canisters, another point for more than 13 β-agonist canisters, a third point for more than 2 dispensings of oral corticosteroids, and no points if none of the above thresholds were reached, leading to a total score of 0-3 for each patient.
The risk of year 2001 emergency hospital care approximately doubled with each increasing medication scale level from 0-2 and tripled at level 3 in the development sample (Table 3). Higher risk scores also were significantly associated with stepwise decreases in the Mini Asthma Quality of Life Questionnaire scores, indicating lower quality of life; increases in the Asthma Therapy Assessment Questionnaire scores, indicating less asthma control; and increases in the Asthma Outcomes Monitoring System scores, indicating greater symptom severity (Table 4).
Table 3 also shows the proportion of validation-sample patients in each level of the medication intensity scale. The validation sample had fewer patients in risk levels 1-3 (47.0%) than the development sample (62.2%). With each level increase in scale score, the risk of year 2003 emergency hospital care for patients in the validation sample increased by 50% (risk levels 0- 1) to 100% (risk levels 1-2 and 2-3). Patients in the highest risk group were more than 6 times as likely as patients in the lowest risk group to require emergency hospital care in 2003 (Table 3).
Prediction performance of the medication intensity scale for emergency hospital care during the second year of observation in the development and validation samples is shown in Table 5. For all cut-offs, sensitivities, positive predictive values, and areas under the ROC curve were lower in the validation sample than in the development sample, whereas specificities and negative predictive values were higher. However, a reasonable balance of sensitivity (64.6%) and specificity (54.1%) could be obtained with a medication intensity scale cutoff of more than 0 in the validation sample, and identification of more than 25% of the high-risk patients could be accomplished with nearly 90% specificity by using a cut-off of more than 1 in the validation sample. A relatively high positive predictive value (26.8%) compared with the baseline risk (6.1%) and a very high specificity (99.1%) could be obtained with a medication intensity scale cut-off of more than 2, although this stratum encompassed only slightly more than 1% of the total number of patients (Table 3).
The proportions of patients in the validation sample in each of the previously defined scale levels (using both pharmacy and baseline emergency hospital care data) were as follows: 57.9% (risk level 0), 28.6% (risk level 1); 5.5% (risk level 2); 7.0% (risk level 3); and 1.0% (risk level 4). Sensitivity, specificity, positive predictive value, negative predictive value, and area under the ROC curve of any risk points (ie, >0) all were somewhat higher with the previously defined scale, which included baseline emergency hospital utilization, compared with the current scale derived from pharmacy data only, but the absolute values of these differences were less than 7%. Somewhat larger sensitivity and positive predictive value differences as well as a larger difference in areas under the ROC curves between the 2 schemes were seen using scale values of more than 1 (Table 5).
DISCUSSION
An important component of population management is risk stratification, and risk adjustment is important in observational database research that evaluates asthma interventions. Previously validated risk stratification schemes have included prior ED visits or hospitalizations as important predictors of subsequent utilization.3-6 However, because such information is less available than computerized pharmacy data, we felt it was important to develop a risk stratification scheme that did not depend on knowing patients' prior emergency hospital utilization.
Among patients in the validation sample, the currently developed scheme identified 12% as having a risk score of more than 1 (see Table 3); this group accounted for more than 25% of the patients who subsequently required emergency hospital care (see Table 5, "Sensitivity"). The false-positive rate (1 minus the specificity) was only approximately 11%. A gradation of risk was identified with increasing risk points, with a 50% to 100% increase in the risk of subsequent emergency hospital care with each additional point (Table 3). More than 1 in 4 patients in the highest risk group subsequently had emergency hospital care in the validation sample (Table 3).
The medication intensity scheme developed in this study has 2 major strengths. First, it was validated in a larger separate sample seen later in time. It was expected that the prediction performance in the validation sample would be less strong than that in the development sample (Table 5), both because the scheme was developed from the data in the development sample and because the incidence of the outcome was higher in the development sample than in the validation sample. However, as discussed below, the prediction properties of the scale in the validation sample appear clinically useful. Second, because it requires only β-agonist canister and oral-corticosteroid dispensing information, and includes only 3 criteria, the current medication intensity scale should be relatively simple to obtain and apply.
There are 2 main weaknesses of the current scheme. The first is that it missed approximately 35% of the patients who subsequently required emergency hospital care (1 minus sensitivity of score > 0, Table 5). This could be because patients with generally controlled asthma still may experience acute severe episodes. Alternatively, lower perception of dyspnea could explain why some patients with low chronic medication utilization experience acute episodes.14 This study cannot differentiate between these alternatives or other explanations for the above findings. The second potential weakness is that 73% of highest risk patients did not require emergency hospital care in the next year (1 minus positive predictive value of score > 2, Table 5). (However, this is probably a less important disadvantage, because the survey data [Table 4] suggested that patients in the higher risk groups were experiencing more symptoms and decreased quality of life, which makes targeting such patients for intervention a fruitful effort.)
This study provides a method of risk adjustment for use in observational database studies when prior hospital utilization data are not available. Studies that assess pharmacologic interventions by using computerized pharmacy data could benefit from the use of the current scale for baseline risk adjustment. Such baseline risk adjustment also may be useful in assessments of quality of asthma care that use computerized pharmacy data.
The relationship of β-agonist use and oral-corticosteroid use to subsequent emergency hospital care was expected based on a number of prior studies.15-20 Inhaled corticosteroids did not appear to affect the risk of subsequent emergency hospital care when β-agonist and oral-corticosteroid use also were accounted for. These results seem contrary to prior data suggesting that inhaled-corticosteroid use reduces the risk of subsequent emergency hospital care.21-24 However, controller use can act as an independent severity marker.25 In addition, inhaled corticosteroids do not lead to controlled asthma in all patients.26 If inhaled corticosteroids are effective, this should result in less use of β-agonists. If inhaled corticosteroids are not effective enough, this should result in more use of β-agonists and possibly oral corticosteroids as well. Thus, β-agonist use and oral-corticosteroid use indirectly reflect inhaled-corticosteroid efficacy in treated patients, which is apparently more important in predicting subsequent utilization than inhaled corticosteroid use alone.
Asthma severity and asthma control are 2 related but separate constructs.27 Severity can be defined clinically in untreated patients,28 but to assess severity in treated patients, the Global Initiative for Asthma guidelines recommend consideration of the medication level.29 The medication intensity scale defined in this study appears to be a severity marker because it is associated with increased hospital utilization and it is inversely related to asthma control and quality of life.
One prior published study used computerized pharmacy data to develop a pharmaceutically based severity stratification scheme for an asthmatic population.9 The 5 levels tested were determined based on focus groups and defined based on the number of β-agonist canisters dispensed per year (cut-point of 4 canisters), other asthma medications besides systemic corticosteroids, and systemic corticosteroids (at least 1 prescription dispensed). The scheme was evaluated in relationship to acute bed days during the 12 months after the period when the severity level was assigned. Only the top 2 levels were associated with significantly increased hospital bed days, and validation in another population was not confirmed. Two other asthma severity stratification schemes using computerized pharmacy data have been proposed, but results have been published only in abstract form.7,8
Because prior emergency hospital utilization is the strongest risk factor for subsequent utilization,3-6 it is not surprising that a risk stratification scheme that included both baseline emergency hospital utilization and pharmacy data performed better than the current scheme based on pharmacy data alone, especially regarding the sensitivity and positive predictive value of a score more than 1 (Table 5). Thus, if baseline emergency hospital utilization data are available, the risk stratification system based on both types of data should be chosen. However, if baseline emergency hospital utilization data are not available or are difficult to obtain, the current medication-derived scheme should be useful because a reasonable balance of sensitivity (65%) and specificity (54%) could still be achieved with a score greater than 0, and a sensitivity of 27% with a specificity of 89% could be achieved with a score greater than 1. Which cut-off of the scale to use would depend on the purpose of the risk stratification exercise. If it was to identify patients for intervention, the intensity and expense of the intervention could determine the choice of cut-off. For a low-intensity intervention, a score greater than 0 may be chosen because of highest sensitivity (65%), with an acceptable false-positive rate (1 minus the specificity) of 46%. For a medium-intensity intervention, a cut-off of greater than 1 provides lower sensitivity (27%) but much higher specificity (89%). For a very intense intervention, a cut-off of greater than 2 identifies 1% of the population with a nearly 27% risk of subsequent emergency hospital care, with 99% specificity.
This study does have several potential limitations. It did not capture unscheduled outpatient visits other than ED visits, although ED visits would be the most expensive type of outpatient unscheduled asthma visits. Administrative data do not usually capture potentially important risk factors such as lower pulmonary function,18,30 race or ethnicity,1,31-33 or smoking.34,35 The study also did not capture clinical predictors such as symptom severity19,30,36,37 and quality of life,37-39 although the current study demonstrates a strong relationship between the medication intensity scale and these measures.
A final issue is the external validity of this study. Patients receiving care from Kaiser Permanente are demographically similar to the working family populations from which they are drawn (unpublished data). However, the asthma hospitalization and ED utilization rate in the validation sample was quite low compared with levels reported in other populations.40,41 This might be related to the access of this population to routine care, telephone interactions, urgent-appointment clinics, and specialty care.24 The results of this study should be generalizable to West Coast adult populations with persistent asthma (as defined by database criteria) who have similar access to healthcare. Further studies will be necessary to confirm its generalizability to other populations.
In summary, we developed and validated a simple and practical 4-level risk stratification scheme based on computerized pharmacy data for use in management of asthma populations and for risk adjustment in database studies. We hope that this scheme will prove useful for identifying patients in need of targeted intervention and that future outcome studies will show reduced morbidity in higher risk patients who receive such targeted intervention.
From the Departments of Allergy at San Diego (MS, RSZ), Sacramento (AL), and Vacaville (GM), Calif, the Center for Health Research, Portland, Ore (WMV), and the Care Management Institute, Oakland, Calif (DM, TBS, MSJ), Kaiser Permanente Medical Care Program; the Division of Allergy-Immunology, Department of Pulmonary and Critical Care Medicine, University of Pennsylvania School of Medicine, Philadelphia, Pa (AJA); and the Department of Epidemiology, Harvard School of Public Health, Boston, Mass (EFC).
This research was supported by the Kaiser Permanente Medical Care Program.
Address correspondence to: Michael Schatz, MD, Chief, Department of Allergy, Kaiser Permanente Medical Center, 7060 Clairemont Mesa Blvd, San Diego, CA 92111. E-mail: michael.x.schatz@kp.org.
REFERENCES
1. Mannino DM, Homa DM, Akinbami LJ, Moorman JE, Gwynn C, Redd SC. Surveillance for asthma—United States 1980-1999. MMWR. 2002;51:1-13.
2. Weiss KB, Sullivan SD. The health economics of asthma and rhinitis, I: assessing the economic impact. J Allergy Clin Immunol. 2001;107:3-8.
3. Grana J, Preston S, McDermott PD, Hanchak NA. The use of administrative data to risk-stratify asthmatic patients. Am J Med Qual. 1997;12:113-119.
4. Lieu TA, Quesenberry CP, Sorel ME, Mendoza GR, Leong AB. Computer-based models to identify high-risk children with asthma. Am J Respir Crit Care Med. 1998;157:1173-1180.
5. Lieu TA, Capra AM, Quesenberry CP, Mendoza GR, Mazar M. Computer-based models to identify high-risk adults with asthma: is the glass half empty or half full. J Asthma. 1999;36:359-370.
6. Schatz M, Nakahiro R, Jones CH, Roth RM, Joshua A, Petitti D. Asthma population management: development and validation of a practical 3-level risk stratification scheme. Am J Manag Care. 2004;10:25-32.
7. Leidy NK, Paramore LC, Watrous ML, et al. Development of an algorithm for estimating asthma severity from an administrative database [abstract]. Value Health. 1999;2:394.
8. Cai B, Blais L, Suissa S, et al. Distribution of asthma severity and change in severity over time in a general population [abstract]. Abstract presented at: American Lung Association/American Thoracic Society International Conference; April 23-28, 1999; San Diego, Calif.
9. Leone FT, Grana JR, McDermott P, MacPherson S, Hanchak NA, Fish JE. Pharmaceutically-based severity stratification of an asthmatic population. Resp Med. 1999;93:788-793.
10. National Committee for Quality Assurance. Technical Specifications. Washington, DC: National Committee for Quality Assurance; 2001. HEDIS 2002; vol 2.
11. Juniper EF, Guyatt GH, Cox FM, Ferrie DJ, King DR. Development and validation of the Mini Asthma Quality of Life Questionnaire. Eur Resp J. 1999;5:35-46.
12. Vollmer WM, Markson LE, O’Connor E, et al. Association of asthma control with health care utilization and quality of life. Am J Respir Crit Care Med. 1999; 160:1647-1652.
13. Bayliss MS, Espindle DM, Ware JE. Asthma Outcomes Monitoring System (AOMS) Administration, Scoring, and Interpretation Manual. Lincoln, RI: QualityMetric, Inc; 1999.
14. Magadle R, Berar-Yanay N, Weiner P. The risk of hospitalization and near-fatal and fatal asthma in relation to the perception of dyspnea. Chest. 2002;121:329-333.
15. Nestor A, Calhoun AC, Dickson M, Kalik CA. Cross-sectional analysis of the relationship between national guideline recommended asthma drug therapy and emergency/hospital use within a managed care population. Ann Allergy Asthma Immunol. 1998;81:327-330.
16. Anis AH, Lynd LD, Wang X-H, et al. Double trouble: impact of inappropriate use of asthma medication on the use of health care resources. CMAJ. 2001;164:625-631.
17. Shireman TI, Heaton PC, Gay WE, et al. Relationship between asthma drug therapy patterns and healthcare utilization. Ann Pharmacother. 2002;36:557-564.
18. Li D, German D, Lulla S, Thomas RG, Wilson SR. Prospective study of hospitalization for asthma: a preliminary risk factor model. Am J Respir Crit Care Med. 1995;151:647-655.
19. Wakefield M, Ruffin R, Campbell D, et al. A risk screening questionnaire for adult asthmatics to predict attendance at hospital emergency departments. Chest. 1997;112:1527-1533.
20. Adams RJ, Smith BJ, Ruffin RE. Factors associated with hospital admissions and repeat emergency department visits for adults with asthma. Thorax. 2000;55:566-573.
21. Donahue JG, Weiss ST, Livingston JM, et al. Inhaled steroids and the risk of hospitalization for asthma. JAMA. 1997;277:887-891.
22. Blais L, Ernst P, Boivin J-F, Suissa S. Inhaled corticosteroids and the prevention of readmission to hospital for asthma. Am J Respir Crit Care Med. 1998;158:126-132.
23. Adams RJ, Fuhlbrigge A, Finkelstein JA, et al. Impact of inhaled antiinflammatory therapy on hospitalization and emergency department visits for children with asthma. Pediatrics. 2001;107:706-711.
24. Schatz M, Cook EF, Nakahiro R, Petitti D. Inhaled corticosteroids and allergy specialty care reduce emergency hospital use for asthma. J Allergy Clin Immunol. 2003;111:503-508.
25. Schatz M, Nakahiro R, Crawford W, et al. Asthma quality of care markers using administrative data. Chest. 2005;128:1968-1973.
26. Bateman ED, Boushey HA, Bousquet J, et al. Can guideline-defined control be achieved? The Gaining Optimal Asthma Control Study. Am J Respir Crit Care Med. 2004;170:836-844.
27. Vollmer WM. Assessment of asthma control and severity. Ann Allergy Asthma Immunol. 2004;93:409-414.
28. National Asthma Education and Prevention Program. Expert Panel Report: Guidelines for the Diagnosis and Management of Asthma. Bethesda, Md: National Institutes of Health; 1991. Publication 91-3642.
29. Global Initiative for Asthma. Global Strategy for Asthma Management and Prevention. National Heart, Lung, and Blood Institute/World Health Organization workshop report (updated). Bethesda, Md: National Institutes of Health; 2002. Report 02-3659.
30. Cowie RL, Underwood MF, Revitt SG, Field SK. Predicting emergency department utilization in adults with asthma: a cohort study. J Asthma. 2001;38:179-184.
31. Gottlieb DJ, Beiser AS, O’Connor GT. Poverty, race, and medication use are correlates of asthma hospitalization rates. Chest. 1995;108:28-35.
32. Joseph CLM, Havstad SL, Ownby DR, Johnson CC, Tiley BC. Racial differences in emergency department use persist despite allergist visits and prescriptions filled for anti-inflammatory medications. J Allergy Clin Immunol. 1998;101:484-490.
33. Zoratti EM, Havstad S, Rodriguez J, et al. Health service use by African Americans and Caucasians with asthma in a managed care setting. Am J Respir Crit Care Med. 1998;158:371-377.
34. Cassino C, Ito K, Bader I, et al. Cigarette smoking and ozone-associated emergency department use for asthma by adults in New York City. Am J Respir Crit Care Med. 1999;159:1773-1779.
35. Dalcin PTR, Piovesan DM, Kang S, et al. Factors associated with emergency department visits due to asthma. Braz J Med Biol Res. 2004;37:1331-1338.
36. Tough SC, Hessel PA, Green FHY, et al. Factors that influence emergency department visits for asthma. Can Resp J. 1999;6:429-435.
37. Schatz M, Mosen D, Apter AJ, et al. The relationship of validated psychometric tools to subsequent medical utilization for asthma. J Allergy Clin Immunol. 2005; 115:564-570.
38. Eisner MD, Ackerson LM, Chi F, et al. Health-related quality of life and future health care utilization for asthma. Ann Allergy Asthma Immunol. 2002;89: 46-55.
39. Tierney WM, Roesner JF, Seshadri R, et al. Assessing symptoms and peak expiratory flow rate as predictors of asthma exacerbations. J Gen Intern Med. 2004;19:237-242.
40. Stempel DA, Carlson AM, Buchner DA. Asthma benchmarking for quality improvement. Ann Allergy Asthma Immunol. 1997;79:517-524.
41. Rabe KF, Adachi M, Lai CKW, et al. Worldwide severity and control of asthma in children and adults: The global Asthma Insights and Reality surveys. J Allergy Asthma Immunol. 2004;114:40-47.
