Publication
Article
The American Journal of Managed Care
TO THE EDITORS:
The article by Peikes et al in this issue of The American Journal of Managed Care (“Early Evaluations of the Medical Home: Building on a Promising Start”)1 critiques the findings of several recent studies of different models of “the medical home.” As lead investigators of one of these studies—the matched-pair, cluster-randomized evaluation of Guided Care—we were drawn to their discussion about the potential analytic effects of “clustering” (ie, the tendency for the outcomes of patients treated by the same physician or practice to be more similar than the outcomes of patients treated by different physicians or practices). For cluster-randomized studies, Peikes et al classify analyses that did not adjust for clustering as being “inconclusive” or of “uncertain statistical signifi cance” when the reported results were statistically significant (ie, P value <.05).
Their categorization of our findings is understandable. However, given that our evaluation enrolled only 7 matched pairs, it begs the question of how our analysis should account for clustering. More importantly, how should small, matched-pair, cluster-randomized studies of the future account for clustering, so that they can produce useful information about the effects of medical homes and other provider-based interventions?
Statisticians usually consider 2 strategies for analyzing matched-pair, cluster-randomized data: (1) site-level adjustment and (2) patient-level adjustment.2 For studies with small numbers of sites (in our case, matched pairs), the former approach is recommended. In this approach, the intervention-specific outcomes (or residuals, when covariate adjustment is required) within each matched pair are summarized into a scalar quantity (eg, mean). Statistical inference is then based solely on these pairs of numbers.
However, recent work by Imai et al3 argues that this approach can yield confi dence intervals and P values that are too conservative. In order to maximize the information content from small, matched-pair, cluster-randomized studies, it is essential that the inferential procedure be as precise as possible.
We intend to revisit our original analyses as well as the analytic strategy for the final results of the Guided Care trial that are to be reported later this year. We continue to
believe that small, matched-pair, cluster-randomized studies, which are more feasible within the context of our resource-constrained environment, can provide useful evidence to guide learning on patient-centered medical homes.
The authors would like to thank Constantine Frangakis, Thomas Louis, PhD, and Brian Caffo, PhD, from the Department of Biostatistics at the Johns Hopkins Bloomberg School of Public Health and Kosuke Imai, PhD, from the Department of Politics, Princeton University, for their assistance in the preparation of this editorial.
Daniel O. Scharfstein, ScD
Department of Biostatistics
Johns Hopkins Bloomberg School of Public Health
Baltimore, MD
Cynthia M. Boyd, MD, MPH
Division of Geriatric Medicine and Gerontology
Johns Hopkins University School of Medicine
Center on Aging and Health
Baltimore, MD
Jennifer L. Wolff, PhD
Department of Health Policy and Management
Johns Hopkins Bloomberg School of Public Health
Baltimore, MD
Chad Boult, MD, MPH, MBA
Department of Health Policy and Management
Johns Hopkins Bloomberg School of Public Health
Baltimore, MDAuthor Disclosures: The authors (DOS, CMB, JLW, CB) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.
Address correspondence to: Daniel O. Scharfstein, ScD, Professor, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, 615 N Wolfe St, Rm E3547, Baltimore, MD 21205. E-mail: dscharf@jhsph.edu.1. Peikes D, Zutshi A, Genevro JL, Parchman ML, Meyers DS. Early evaluations of the medical home: building on a promising start. Am J Manag Care. 2012;18(2):105-116.
2. Hayes RM, Moulton LH. Cluster-Randomised Trials. Boca Raton, FL: Chapman & Hall/CRC Press; 2009:233-254.
3. Imai K, King G, Nall C. The essential role of pair matching in clusterrandomized experiments, with application to the Mexican Universal Health Insurance Evaluation. Stat Sci. 2009;24(1):29-53.
