Publication
Article
The American Journal of Managed Care
Author(s):
There are concerns that retail clinics provide inferior quality of care. The authors found no difference in quality among retail clinics, physician offices, and emergency departments.
ABSTRACTObjectives: To compare antibiotic prescribing among retail clinics, primary care practices, and emergency departments (EDs) for acute respiratory infections (ARIs): antibiotics-may-be-appropriate ARIs (eg, sinusitis) and antibiotics-never-appropriate ARIs (eg, acute bronchitis).
Study Design: We analyzed retail clinic data from the electronic health records of the 3 largest retail clinic chains in the United States, and data on visits to primary care practices and EDs from the nationally representative National Ambulatory Medical Care Survey and National Hospital Ambulatory Medical Care Survey.
Methods: Using multivariate models, we estimated an adjusted antibiotic prescribing rate for each site of care, controlling for differences in patient characteristics and diagnosis.
Results: From 2007 to 2009 in the United States, there were 3 million, 167 million, and 29 million ARI visits at retail clinics, primary care practices, and EDs, respectively. For all ARI visits, the adjusted antibiotic prescribing rate at retail clinics (58%) was similar to the rate at primary care practices (62%; P = .09) and EDs (59%; P = .48). For antibiotics-may-be-appropriate ARI visits, the adjusted antibiotic prescribing rate (95%) at retail clinics was higher than at primary care practices (85%; P <.01) and EDs (83%; P <.01). For antibiotics-never-appropriate ARI visits, the adjusted antibiotic prescribing rate (34%) at retail clinics was lower than at primary care practices (51%; P <.01) and EDs (48%; P <.01).
Conclusions: Compared with primary care practices and EDs, there was no difference at retail clinics in overall ARI antibiotic prescribing. At retail clinics, antibiotic prescribing was more diagnosis-appropriate.
Am J Manag Care. 2015;21(4):294-302
Take-Away Points
There is concern that retail clinics could increase inappropriate antibiotic prescribing because they have financial incentives to overprescribe medications and they employ nurse practitioners. In the first evaluation of this question using medical record data, we found that the overall antibiotic prescribing rate was similar among retail clinics, physician offices, and emergency departments; furthermore, care at retail clinics was more guideline-concordant. For example, in the case of conditions for which antibiotics are never indicated, retail clinics had a lower antibiotic prescribing rate.
Acute respiratory infections (ARIs) such as pharyngitis, rhinosinusitis, and bronchitis are the most common symptomatic reasons why patients seek care in the United States.1 ARIs account for half of all adult antibiotic prescriptions and three-fourths of all pediatric antibiotic prescriptions.2 Physicians overprescribe antibiotics for ARIs and unnecessarily use broad-spectrum antibiotics, both of which lead to increased antibiotic resistance, adverse drug reactions, and increased cost.2-5
A growing fraction of ARI visits occur at retail clinics, which are located in drugstores and grocery stores and are typically staffed by nurse practitioners (NPs) as opposed to physicians. Retail clinics provide walk-in care for a limited set of conditions for which they have treatment protocols, including ARIs, other minor illnesses, immunizations, sports physicals, and other screening services—ARIs account for 60% of all retail clinic visits.6-8 Providers at retail clinics independently manage and prescribe medications. State regulations vary on whether NPs at retail clinics must maintain a collaborative or supervisory relationship with a physician.9 Currently, there are almost 6 million yearly visits to the 1600 retail clinics in the United States,10 and the number of retail clinics is expected to grow rapidly in the coming years.11
The impact of this shift to retail clinics in the care of ARIs, and subsequent antibiotic prescribing, is unknown. It is possible this shift may worsen existing problems with ARI antibiotic prescribing patterns, as most retail clinics are owned by for-profit pharmacy chains that have a financial incentive to provide prescription medicines.12-15 Alternatively, it is possible this shift will improve antibiotic prescribing, as NPs often provide care that is more consistent with guidelines16 and retail clinics strictly incorporate evidence-based guidelines into their electronic health records (EHRs).17-19
In prior studies comparing rates of antibiotic prescribing, little difference was observed between retail clinics and outpatient physician practices20,21; however, these were small studies that used a limited set of diagnoses and health plan claims data. In this study, we compared antibiotic prescribing at retail clinics, primary care practices, and emergency departments (EDs) for all ARIs using national data collected directly from the EHR or the medical chart.
METHODSRetail Clinic Visits
We obtained de-identified data from all visits from 2007 to 2009 (2007, 0.8 million visits; 2008, 2.4 million; 2009, 5.4 million) from 3 retail clinic operators: MinuteClinic, TakeCare, and LittleClinic. Together they operate 81% of all US retail clinics.8 These data came directly from the operators’ EHRs.
Primary Care Practice Visits
We obtained 2007 to 2009 data (to match the years of available retail clinic data) on visits to ambulatory practices from the National Ambulatory Medical Care Survey (NAMCS) and outpatient hospital practices from the National Hospital Ambulatory Medical Care Survey (NHAMCS). NAMCS and NHAMCS employ a complex sampling methodology to obtain a nationally representative estimate of outpatient visits in the United States, and the 2 surveys have been used in many studies to characterize physician care patterns in the United States.2,22-28 However, neither survey includes retail clinics. For each visit, standard 1-page encounter forms are completed by the clinician, office staff, or an outside coder using the medical chart.
We defined primary care visits in NAMCS as those associated with the physician specialties of family/general practice, internal medicine, or pediatrics, and in NHAMCS as visits to general medicine and pediatric clinics. We excluded the small number of visits with a nurse practitioner because we did not want to confound training and practice setting; we included these visits in a sensitivity analysis, and it had no impact on our findings. To exclude follow-up visits for ARIs, we limited our sample to visits where the treating provider indicated that the major reason for the visit was a new problem. After these exclusions, we sampled 6289 ARI visits. We also conducted a sensitivity analysis in which we examined all ARI visits, including those where it was not a new problem, and we found no substantive differences.
ED Visits
We obtained data on 7629 sampled ED visits for ARIs from 2007 to 2009 from the ED component of the NHAMCS survey. Similar to the physician outpatient visits in NAMCS, a standard encounter form is completed for each visit using the medical chart.
Data Elements
Across the 3 data sources, we looked at patient demographics, reasons for visit, diagnoses, and medications prescribed. We did not include patient temperature because for almost half of ARI visits at primary care practices, the temperature field was blank. Patient drug allergies were not available.
Defining ARI Visits
We identified ARI visits using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes.2 Based on prior work,3 we divided the 6 ARI diagnoses into “antibiotics-may-be-appropriate” diagnoses (streptococcal pharyngitis [034.x], otitis media [381.x, 382.x], sinusitis [461.x]) and “antibiotics-never-appropriate” diagnoses (upper respiratory infection [460.x, 465.x], nonstreptococcal pharyngitis and laryngitis [462.x, 464.x], bronchitis [466.x, 490.x, 491.21]). These definitions of antibiotic-appropriateness are forgiving to prescribing of antibiotics. For example, although we consider sinusitis an antibiotics-may-be-appropriate diagnosis, it is not clear that sinusitis ever requires antibiotic treatment.29
Excluding Visits With Competing Diagnoses
eAppendix Table 3
Patients may have multiple problems that require antibiotics, and the fraction of ARI patients with such competing diagnoses might differ across the 3 care sites. To address this potential concern, we excluded visits with a diagnosis code for a competing diagnosis that might require antibiotics (list of ICD-9-CM codes provided in , available at www.ajmc.com). This excluded 3%, 6%, and 10% of the ARI visits at retail clinics, physician offices, and EDs, respectively. In a sensitivity analysis, including such visits had no substantive impact on the results.
Classifying Prescriptions
Among all eligible ARI visits, we identified any oral antibiotic prescription associated with the visit. In the retail clinic, data prescriptions are identified using the National Drug Code system, while in the NAMCS and NHAMCS surveys, prescriptions are classified using the Multum Lexicon system. We defined broad-spectrum antibiotics as azithromycin and clarithromycin, quinolones, amoxicillin- clavulanate, ampicillin-sulbactam, and second- and third-generation cephalosporins.2
Analyses
We calculated the antibiotic prescribing rate for all ARI visits, for antibiotics-may-be-appropriate diagnoses, for antibiotics-never-appropriate diagnoses, and for each individual ARI diagnosis. We limited our analysis to patients 2 years and older because retail clinics rarely see children under that age.
We used multivariable models to assess the independent association between site of care and antibiotic prescribing, where the outcome variable is a binary (yes/no) indicator for whether an antibiotic was prescribed. Building on prior work,30 we combined the data from the 3 clinical sites into a single data set. In the physician office and ED data, each visit is given a sampling weight so that one can calculate nationally representative estimates. Since retail clinics represent a full national sample of visits, in the analyses, these visits were given a sampling weight of 1. The NAMCS and NHAMCS samples include identifiers allowing for control for clustering within individual clinicians and practices and EDs. For the retail clinic data, we could not identify individual clinicians and practice sites associated with a visit; we therefore accounted for a higher level of clustering within each of 3 retail clinic chains. Because it assumes that all retail clinic visits within a given retail clinic chain are similar, the intercluster correlation was large (0.02) and therefore standard errors were conservatively large.
We used separate models for all ARI visits, antibiotics-may-be-appropriate ARI diagnosis visits, and antibiotics-never-appropriate ARI diagnosis visits. Among all ARI visits where an antibiotic was prescribed, we created an analogous logistic regression model where the dependent variable was broad-spectrum antibiotic prescription. In each of the multivariate models we adjusted for gender, age, site of care, and whether the patient had a chronic illness. In the model where we focused on all ARI visits, we also adjusted for diagnosis type (antibiotics-may-be-appropriate diagnoses vs antibiotics-never-appropriate). We conducted sensitivity analyses where we: 1) did not adjust for diagnosis; 2) adjusted for the 6 specific ARI diagnoses instead of diagnosis type to provide a higher level of specificity; 3) examined only children (aged ≤17 years) as this is an important subpopulation; 4) examined only adults; and 5) examined only patients without a reported chronic illness, as this is the population most likely to be seen at retail clinics. The odds ratio results from all multivariate models are reported in the eAppendix.
To facilitate interpretation of the regression results, we used the method of predictive margins (also called recycled predictions) to report the predicted antibiotic prescribing rate and the predicted broad-spectrum antibiotic prescribing rate for each of the 3 care sites while adjusting for other covariates.31,32 Specifically, we used the estimated coefficients from the multivariate regression models to predict antibiotic prescribing for each patient visit, alternatively assigning all visits to a given value of the variable (eg, retail clinic, primary care practice, ED), but leaving all other explanatory variables at their original values. We then averaged the predictions across all patient visits. This allowed us to estimate the mean antibiotic prescribing rate if all patient visits occurred at a given site, but otherwise retained the original values of all their other characteristics. P values less than .05 were considered significant. All analyses were performed using SAS 9.2 (Cary, North Carolina).
RESULTS
Table 1
Between 2007 and 2009, there were 3 million, 162 million, and 29 million ARI visits, respectively, at retail clinics, primary care practices, and EDs. There were several notable differences in patient characteristics across the 3 care sites (). Children and adolescents (aged ≤17 years) made up a smaller fraction of ARI visits to retail clinics (27%) than to primary care practices (50%; P <.01) and EDs (43%; P <.01). A smaller fraction of patients reported a chronic illness at retail clinics (4%) compared with primary care practices (20%; P = .01) and EDs (22%; P <.01).
Mix of ARI Visits
Table 2
The mix of ARI visits varied by site of care. At retail clinics, a higher proportion of ARI visits (53%) led to an antibiotics-may-be-appropriate diagnosis (sinusitis, otitis media, and streptococcal pharyngitis) than at primary care physician offices (32%; P <.01) and EDs (29%; P <.01) ().
Unadjusted Differences in ARI Antibiotic Prescribing
Not adjusting for other factors, the fraction of all ARI visits at which an antibiotic was prescribed was similar at retail clinics (66%), primary care practices (62%; P = .28 comparison with retail clinics), and EDs (60%; P = .11) (Table 2). This pattern varied by antibiotic appropriateness. For antibiotics-may-be-appropriate diagnoses, retail clinic providers were more likely to prescribe antibiotics (95%) than primary care physicians (85%; P <.01) and ED physicians (83%; P <.01). In contrast, for antibiotics-never-appropriate diagnoses, retail clinics providers were less likely to prescribe antibiotics (33%) than primary care physicians (50%; P <.01) and ED physicians (50%; P <.01). One notable exception among antibiotics-never-appropriate diagnoses was for acute bronchitis/bronchiolitis where antibiotic prescribing at retail clinics, physicians, and EDs was 77%, 76%, and 70%, respectively. Among all ARI visits where an antibiotic was prescribed, retail clinic providers were less likely to prescribe a broad-spectrum antibiotic than primary care practices (40% vs 49%; P <.01), but there was no difference between retail clinics and EDs (40% vs 44%; P = .05).
Adjusted Differences in ARI Antibiotic Prescribing
Table 3
Figure
The adjusted antibiotic prescribing rate for all ARI visits at retail clinics (58%) was similar to the adjusted antibiotic prescribing rate at primary care practices (62%; P = .09) and EDs (60%; P = .48) (, ). For antibiotics-may-be-appropriate ARI visits, the adjusted antibiotic prescribing rate for retail clinics (95%) was higher than at primary care practices (86%; P <.01) and EDs (82%; P <.01). For antibiotics-never-appropriate ARI visits, the adjusted antibiotic prescribing rate at retail clinics (34%) was lower than at primary care practices (50%; P <.01) and EDs (48%; P <.01). For ARI visits at which an antibiotic was prescribed, the adjusted rate of broad-spectrum antibiotic prescribing at retail clinics was 42%, lower than the rate at primary care practices (49%; P = .04) but not different from the rate at EDs (44%; P = .34).
Sensitivity Analyses
The differences in antibiotic prescribing were consistent in both the children-only and adult-only analyses (eAppendix Table 2). If we do not account for ARI diagnosis in the model then there is no statistically significant difference in overall ARI antibiotic prescribing between the 3 care sites (eAppendix Table 2). When we only look at patients without a chronic illness, the differences in antibiotic prescribing were similar to those in the model with all visits (eAppendix Table 2).
DISCUSSION
In a large, nationally representative sample of ARI visits, retail clinics had an antibiotic prescribing rate similar to that of primary care practices and EDs. This is reassuring evidence that the shift in care to retail clinics will not negatively impact antibiotic prescribing in the United States. The lack of a difference is consistent with the larger literature, that demonstrates that nurse practitioners and physicians provide care of similar quality.33
While there is no difference across the sites in antibiotic prescribing rate for all ARI visits, there is a difference in both the mix of ARIs seen and in antibiotic prescribing by type of ARI. At retail clinics, half of ARI visits are for antibiotics-may-be-appropriate diagnoses (sinusitis, otitis media, and streptococcal pharyngitis). In contrast, onethird of ARI visits at physician offices and EDs are for antibiotics-may-be-appropriate diagnoses. For antibiotics-never-appropriate diagnoses, primary care practices have a much higher antibiotic prescribing rate. However, for antibiotics-may-be-appropriate diagnoses, we see the opposite pattern—retail clinics have a much higher antibiotic prescribing rate.
These results could be driven by how providers choose a diagnosis. Ideally, a provider makes a diagnosis that leads to the decision to prescribe antibiotics. However, it is also possible that providers first decide whether to give antibiotics and later code a diagnosis based on the decision to prescribe antibiotics. If the latter pattern is common at retail clinics, then it might imply there are no differences in true ARI mix across the sites but that retail clinic providers are more likely than primary care and ED physicians to list an ARI diagnosis for which antibiotics are appropriate (eg, listing sinusitis as the diagnosis instead of a viral upper respiratory condition). In such cases, retail clinic providers are not truly providing care that is more concordant with the diagnosis, as the diagnosis itself might be incorrect.
It is also possible that the difference in ARI mix across the care sites reflects true differences in patient populations. For example, it could be that patients with prior episodes of sinusitis self-select to a retail clinic, and this is why we see a higher fraction of antibiotics-may-be-appropriate ARI diagnoses at retail clinics. If this is the case, then the more diagnosis-concordant pattern we observe at retail clinics means that antibiotics are being reserved for those whom antibiotics might benefit. At many retail clinics, evidence-based guidelines are incorporated into the EHR; if providers choose to go outside the guidelines, the provider must provide a justification.17-19 This forcing mechanism may lead to decreased antibiotic prescribing and thus higher quality care.
There were differences in the type of antibiotics prescribed at retail clinics, primary care practices, and EDs. Across all ARI diagnoses, controlling for other factors, primary care practices were more likely than retail clinics to use broad-spectrum antibiotics such as azithromycin and quinolones. This pattern is concerning given that these broad-spectrum antibiotics often do not have higher efficacy, and their overuse could increase the prevalence of antibiotic-resistant bacteria. For example, the fraction of pneumococcal serotypes resistant to macrolides is 35.3%34 and to quinolones, 7.3%.35
Limitations
Our analyses could be biased by differences in the types of patients seen at the care sites. Although our multivariate models included numerous characteristics including age, gender, and chronic illnesses, there may be unaccounted differences. For example, patients who visit primary care practices with a given diagnosis may be sicker or require a broad-spectrum antibiotic because they have more allergies to narrow-spectrum antibiotics. We could not control for key signs and symptoms such as severity of fever. This may be particularly relevant to the ED, which is a very different practice setting, and patients may differ in acuity in ways not captured in our data. At retail clinics, the data came directly from the EHR, while at physician offices, the data were obtained from providers or chart review. It is possible that this difference in data collection might have driven the differences in the fraction of patients reported to have a chronic illness. For physician office visits, we only collected prescriptions on the day of the visit. Our antibiotic prescribing rate may be low for physician office visits because we did not capture antibiotic prescriptions provided over the phone in the days after the visit. For all care sites, we captured prescriptions, not whether the prescription was filled. Patients may have received prescriptions and been told to fill them if symptom improvement did not occur.
Patients with an ARI might seek care at a retail clinic instead of staying at home. If retail clinics “induce” these new ARI visits, then the growth of retail clinics could lead to more antibiotic prescribing, even though their antibiotic prescribing rate for a given ARI visit is similar. Based on prior work, we categorized ARI diagnoses into “antibiotics-may-be-appropriate” and “antibiotics-never-appropriate.” We used a definition of antibiotics-may-beappropriate diagnoses that may be overly inclusive, and we recognize that the clinical distinction between the categories of diagnoses may not be straightforward for all patients. We categorized visits by the diagnosis provided by the provider at the end of the visit. Ideally, we would categorize visits by the reason for visit, but these data were not consistently available across the care sites. Lastly, in this manuscript we compare rates of antibiotic prescribing at retail clinics, physician offices, and EDs. However, it is important to acknowledge that across all the care sites, the antibiotic prescribing rate for antibiotics-never-appropriate conditions is clearly too high, and broad-spectrum antibiotic use is likely excessive.
CONCLUSIONS
Compared with primary care practices and EDs, retail clinics have a similar antibiotic prescribing rate for ARIs, though retail clinic providers are less likely to prescribe broad-spectrum antibiotics. Retail clinic prescribing is more diagnosis-concordant in that retail clinics are more likely to prescribe antibiotics for antibiotics-may-be-appropriate ARI visits and less likely to prescribe antibiotics for antibiotics-never-appropriate diagnoses.Author Affiliations: Harvard Medical School (AM, CAG, JAL), Boston, MA; RAND Corporation, Pittsburgh, PA (CMS, RMB), and Boston, MA (AM, CAG); Division of Infectious Diseases, Boston Children’s Hospital (CAG), Boston, MA; Division of General Medicine and Primary Care, Brigham & Women’s Hospital (JAL), Boston, MA; Division of General Medicine, Beth Israel Deaconess Medical Center (AM), Boston, MA.
Source of Funding: This study was funded under a grant from the National Institutes of Allergy and Infectious Diseases (R21 AI097759-01, PI- Mehrotra). Dr Mehrotra also received support under a career development award from the National Institutes of Health (KL2 RR 24154-6). Dr Linder is also supported by grants from the National Institutes of Health (RC4 AG039115) and the Agency for Healthcare Research and Quality (R18 HS018419). The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article. Dr Mehrotra had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Authorship Information: Concept and design (AM, CAG, CMS, JAL); acquisition of data (AM); analysis and interpretation of data (AM, CMS, RMB); drafting of the manuscript (AM); critical revision of the manuscript for important intellectual content (CAG, CMS, RMB, JAL); statistical analysis (CMS, RMB); obtaining funding (AM, CAG, JAL); and supervision (AM, JAL).
REFERENCES
Address correspondence to: Ateev Mehrotra, MD, MPH, Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave, Boston, MA 02115. E-mail: Mehrotra@hcp.med.harvard.edu.
1. National Ambulatory Medical Care Survey: 2010 Summary Tables 2010. CDC website. http://www.cdc.gov/nchs/data/ahcd/namcs_summary/2010_namcs_web_tables.pdf. Accessed March 18, 2105.
2. Steinman MA, Gonzales R, Linder JA, Landefeld CS. Changing use of antibiotics in community-based outpatient practice, 1991-1999. Ann Intern Med. 2003;138(7):525-533.
3. Grijalva CG, Nuorti JP, Griffin MR. Antibiotic prescription rates for acute respiratory tract infections in US ambulatory settings. JAMA. 2009;302(7):758-766.
4. Linder JA, Bates DW, Lee GM, Finkelstein JA. Antibiotic treatment of children with sore throat. JAMA. 2005;294(18):2315-2322.
5. Linder JA, Stafford RS. Antibiotic treatment of adults with sore throat by community primary care physicians: a national survey, 1989-1999. JAMA. 2001;286(10):1181-1186.
6. Scott MK. Health Care in the Express Lane: The Emergence of Retail Clinics. Oakland, CA: California HealthCare Foundation; 2006.
7. Mehrotra A, Wang MC, Lave JR, Adams JL, McGlynn EA. Retail clinics, primary care physicians, and emergency departments: a comparison of patients’ visits. Health Aff (Millwood). 2008;27(5):1272-1282.
8. Rudavsky R, Pollack CE, Mehrotra A. The geographic distribution, ownership, prices, and scope of practice at retail clinics. Ann Intern Med. 2009;151(5):315-320.
9. State Practice Environment. American Association of Nurse Practitioners website. https://www.aanp.org/legislation-regulation/statelegislation-regulation/state-practice-environment. Updated March 5, 2015. Accessed March 18, 2015.
10. Mehrotra A, Lave JR. Visits to retail clinics grew fourfold from 2007 to 2009, although their share of overall outpatient visits remains low. Health Aff (Millwood). 2012;31(9):2123-2129.
11. Merlo L. CVS CEO’s Shareholder Letter. Woonsocket, RI, 2013. CVS Health website. http://investors.cvshealth.com/2013-in-review/ceoshareholder-letter.aspx. Published February 11, 2014. Accessed March 18, 2015.
12. AMA calls for investigation of store-based health clinics. American Medical Association website. http://www.prnewswire.com/news-releases/ama-calls-for-investigation-of-store-based-health-clinics-58310997.html. Published 2007. Accessed March 18, 2015.
13. Steenhuysen J. AMA to Seek probe of retail health clinics. Reuters Health Information. http://www.reuters.com/article/2007/06/26/idUSN2532441820070626. Published June 26, 2007. Accessed March 18, 2015.
14. O’Keefe L. Academy takes aim at retail-based clinics. AAP News. 2006;27;1-9.
15. Japsen B. AMA takes on retail clinics. Chicago Tribune. http://articles.chicagotribune.com/2007-06-25/business/0706240383_1_retailclinics-ama-doctors-groups. Published June 25, 2007. Accessed March 18, 2015.
16. Laurant M, Reeves D, Hermens R, Braspenning J, Grol R, Sibbald B. Substitution of doctors by nurses in primary care. Cochrane Database Syst Rev. 2005(2):CD001271.
17. Rosenbluth H. In rebuttal: health clinics provide vital service. Pittsburgh Post Gazette. http://www.post-gazette.com/opinion/Op-Ed/2006/11/07/In-Rebuttal-Health-clinics-provide-vital-service/stories/200611070204. Published November 7, 2006. Accessed March 18, 2015.
18. Woodburn JD, Smith KL, Nelson GD. Quality of care in the retail health care setting using national clinical guidelines for acute pharyngitis. Am J Med Qual. 2007;22(6):457-462.
19. Bohmer R. The rise of in-store clinics—threat or opportunity? N Engl J Med. 2007;356(8):765-768.
20. Mehrotra A, Liu H, Adams JL, et al. Comparing costs and quality of care at retail clinics with that of other medical settings for 3 common illnesses. Ann Intern Med. 2009;151(5):321-328.
21. Jacoby R, Crawford AG, Chaudhari P, Goldfarb NI. Quality of care for 2 common pediatric conditions treated by convenient care providers. Am J Med Qual. 2010;26(1):53-58.
22. Mechanic D, McAlpine DD, Rosenthal M. Are patients’ office visits with physicians getting shorter? N Engl J Med. 2001;344(3):198-204.
23. Gilchrist VJ, Stange KC, Flocke SA, McCord G, Bourguet CC. A comparison of the National Ambulatory Medical Care Survey (NAMCS) measurement approach with direct observation of outpatient visits. Med Care. 2004;42(3):276-280.
24. Merenstein D, Daumit GL, Powe NR. Use and costs of nonrecommended tests during routine preventive health exams. Am J Prev Med. 2006;30(6):521-527.
25. Roumie CL, Halasa NB, Edwards KM, Zhu Y, Dittus RS, Griffin MR. Differences in antibiotic prescribing among physicians, residents, and nonphysician clinicians. Am J Med. 2005;118(6):641-648.
26. Forrest CB, Whelan EM. Primary care safety-net delivery sites in the United States: a comparison of community health centers, hospital outpatient departments, and physicians’ offices. JAMA. 2000;284(16):2077-2083.
27. Scholle SH, Chang J, Harman J, McNeil M. Characteristics of patients seen and services provided in primary care visits in obstetrics/gynecology: data from NAMCS and NHAMCS. Am J Obstet Gynecol. 2004;190(4):1119-1127.
28. Steinman MA, Landefeld CS, Gonzales R. Predictors of broad-spectrum antibiotic prescribing for acute respiratory tract infections in adult primary care. JAMA. 2003;289(6):719-725.
29. Garbutt JM, Banister C, Spitznagel E, Piccirillo JF. Amoxicillin for acute rhinosinusitis: a randomized controlled trial. JAMA. 2012;307(7):685-692.
30. Mehrotra A, Zaslavsky AM, Ayanian JZ. Preventive health examinations and preventive gynecological examinations in the United States. Arch Intern Med. 2007;167(17):1876-1883.
31. Graubard BI, Korn EL. Predictive margins with survey data. Biometrics. 1999;55(2):652-659.
32. Setodji CM, Scheuner M, Pankow JS, Blumenthal RS, Chen H, Keeler E. A graphical method for assessing risk factor threshold values using the generalized additive model: the multi-ethnic study of atherosclerosis. Health Serv Outcomes Res Methodol. 2012;12(1):62-79.
33. Horrocks S, Anderson E, Salisbury C. Systematic review of whether nurse practitioners working in primary care can provide equivalent care to doctors. BMJ. 2002;324(7341):819-823.
34. Jenkins SG, Farrell DJ. Increase in pneumococcus macrolide resistance, United States. Emerg Infect Dis. 2009;15(8):1260-1264.
35. Adam HJ, Hoban DJ, Gin AS, Zhanel GG. Association between fluoroquinolone usage and a dramatic rise in ciprofloxacin-resistant streptococcus pneumoniae in Canada, 1997-2006. Int J Antimicrob Agents. 2009;34(1):82-85.