Identification of Patients Likely to Benefit From Care Management Programs

Objectives: To compare predictive modeling (PM), selection by primary care physician (PCP), and a combination of both as approaches to prospective patient identification for care management programs.

Study Design:

Observational study.

Methods:

A total of 6026 beneficiaries of a statutory health insurance program in Germany served as a sample for patient identification by PM and selection by PCP. The resulting mutually exclusive subpopulations were compared for care needs (eg, morbidity burden), healthcare utilization (previous all-cause hospitalizations and predicted costs), and prior participation in intensified care programs (as a proxy for amenability). Data sources were insurance claims data and a patient survey.

Results:

Patients were selected for eligibility in a care management program by PM (n = 301), selection by PCP (n = 203), or a combination of both (n = 32). Compared with 5490 nonselected patients, all eligible patients had significantly higher morbidity burden and more previous hospitalizations. Compared with selection by PCP, PM identified patients at significantly higher risk for future healthcare utilization, with predicted annual healthcare costs of €8760 (95% confidence interval [CI], €8314-€9205) vs €4541 (95% CI, €4094-€4989) (P <.01). Compared with patients selected by PM, patients selected by PCP had significantly higher rates of prior participation in intensified care programs (80.8% vs 56.4%, P <.01). Patients selected independently by both approaches seemed to be at high risk for future healthcare utilization, with predicted annual healthcare costs of €8279 (95% CI, €7465-€9092), and 84.6% had prior participation in intensified care programs.

Conclusions:

Identification of high-risk patients most likely to benefit from and participate in care management programs may be facilitated by a combination of PM and selection by PCP.

(Am J Manag Care. 2011;17(5):345-352)

Care management programs improve quality of care and reduce costs for patients at high risk for future healthcare utilization and actionable care needs. The best approach to identify patients most likely to benefit from care management programs remains unclear.

• Predictive modeling identifies patients who are at high risk for future healthcare utilization and who seem less approachable for care management.

• Selection by primary care physician identifies patients who are at lower risk for future healthcare utilization and who seem approachable for care management.

• When combined, the 2 approaches may complement each other to identify patients who are most likely to benefit from and participate in care management programs.

Care management programs are seen as an effective approach to meet the challenge of increasing numbers of patients with complex care needs.^1,2 Targeting these programs to patients with multiple chronic conditions and at high risk for cost-intensive care offers the greatest opportunities for improving quality of care and reducing healthcare costs.³ Because care management programs require allocation of restricted human and financial resources, the targeted population should be not only cost-intensive but also “care sensitive.” Care sensitivity implies 2 dimensions. First, patients have to be approachable (ie, willing and able to participate in intensified care programs). Second, their clinical needs have to be actionable (ie, care programs should be able to mitigate their needs). Therefore, practicable tools are needed to screen patient populations for “high-risk” and care-sensitive individuals.⁴

Since the 1980s, several statistical models commonly known as predictive models have been developed to predict future healthcare utilization and costs.^5,6 These models are based on morbidity, prior healthcare utilization, and cost data, which can easily be obtained from health insurance claims data. Predictive models can be used to identify high-risk individuals who may benefit from care management programs.⁷ Further characterization of these individuals has revealed actionable healthcare needs.⁸ When used to identify patients likely to benefit from care management programs, predictive modeling (PM)—based selection is hypothesized as superior to selection by primary care physician (PCP).⁹ However, to our knowledge, the approaches have not been directly compared. This article extends the existing literature on patient selection for care management programs with results from a direct comparison of case selection by PM, selection by PCP, and a combination of both. A long-term patient—provider relationship enables PCPs to assess a patient’s clinical risk and care sensitivity. The results of this comparison may be useful in developing an approach to identify care-sensitive patients who are at high risk for future healthcare utilization and who are likely to benefit from care management programs.

METHODS

In 2009, insurance claims data among a systematic sample of all beneficiaries of the General Regional Health Fund (Allgemeine Ortskrankenkasse [AOK]) from 10 primary care practices in southwestern Germany were obtained for PM and further analysis. The 10 primary care practices (6 single practices and 4 group practices with 2 PCPs) were recruited from rural areas (5 practices) and from urban areas (5 practices). In Germany, 85% of the population are insured by sickness funds. These statutory health insurance programs based on a social security—based healthcare system are mainly funded by earmarked premiums. In 2009, the AOK insured 34% of all patients insured by a social security–based healthcare system in Germany.¹⁰ The AOK covers all areas of healthcare delivery, and its contributions are jointly paid by beneficiaries and by employers. There are no age restrictions for AOK beneficiaries. Beneficiaries with their own income have copayments of up to 2% of their annual income.

The data set included medical and pharmacy claims from January 2007 to December 2008. Deidentified data were obtained from AOK beneficiaries of all ages. This explorative study was part of a series of studies to develop a care management program for high-risk patients in primary care. Details of the study have been described elsewhere.¹¹ The University Hospital Heidelberg Institutional Review Board, Heidelberg, Germany, approved the study. Informed consent was obtained from all participants in the survey.

Case Finding by PM

For PM, we used the software package Case Smart Suite Germany (Verisk Health, Munich, Germany). This PM program is an extension of diagnostic cost group PM, which has previously been applied in comparative case-finding studies.¹² Information from the past 2 years (2007-2008) served as inputs for PM, including all International Statistical Classification of Diseases, 10th Revision (German modification) (ICD-10-GM) diagnosis codes assigned in outpatient and inpatient settings, prior costs, and hospital admissions, as well as demographic data. Clinically similar ICD-10-GM codes are classified into diagnostic groups. These groups are then further collapsed into condition categories, which reflect similar levels of resource use and are organized by body system or disease (eg, congestive heart failure). Individuals may have multiple diagnostic groups or condition categories. In the next step, grouping into hierarchical clinical categories is applied to every individual. Therefore, each individual is labeled exclusively, with the highest hierarchical clinical category within 1 condition category (eg, acute congestive heart failure exacerbation). However, different coexisting hierarchical clinical categories within 1 individual will increase predicted future healthcare utilization. Based on logistic regression analysis, the software package computes a likelihood of hospitalization for each individual, which indicates the likelihood of at least 1 hospital admission within the next 12 months (2009-2010 for the present study).¹³ Despite the fact that hospitalizations are a source of major healthcare spending, predicted costs and predicted hospitalizations may not be used interchangeably to identify patients at high risk who may potentially benefit from care management interventions. Hospitalizations may reflect potentially actionable costs, as a significant proportion of these are owing to ambulatory care—sensitive conditions,¹⁴ whereas distinct cost-intensive procedures like fertility treatment, dialysis, biological agents, or chemotherapy seem to represent less actionable costs. For this study, patients with a likelihood of hospitalization score above the 90th percentile were defined as high-risk patients. We included patients with at least 1 of the following ICD-10-GM index conditions: type 2 diabetes mellitus (codes E11-E14), chronic obstructive pulmonary disease

(codes J43-J44), asthma (code J45), chronic heart failure (codes I11.0, I13.0, I13.2, I25.5, and I50), and late-life depression (codes F32-F33 [>60 years]). Excluded were patients with dementia (codes F00-F03), dialysis (codes Z49 and Z99.2), or active cancer disease (codes C00-C97).

Case Finding Using Selection by PCP

Fourteen PCPs from 10 participating primary care practices were asked to screen a list of all AOK beneficiaries in their practice to select up to 30 patients for future participation in a care management program aimed at reducing avoidable hospitalizations. Case selection was restricted to the same inclusion and exclusion criteria as aforedescribed. Primary care physicians were informed about the aims and intervention elements of the planned care management intervention. However, avoidable hospitalizations were not further specified before case selection. Primary care physicians were blinded to results of PM until they submitted their final list of selected patients.

Characterization of Selected Patients

We analyzed insurance claims data for all 6026 beneficiaries to determine morbidity burden and prior healthcare uti-lization. An adapted version of the Charlson Comorbidity Index was used to determine weighted comorbidity counts based on inpatient and outpatient ICD-10-GM diagnoses.¹⁵ This index has been shown to be predictive of 1-year mortality and healthcare costs.¹⁶ Single index condition prevalence, prior hospital admissions, and demographic variables were obtained from insurance claims data (2007-2008). We invited all selected patients (by PM, selection PCP, or a combination of both) who were enrolled in a PCP-centered care contract to fill out a paper-based questionnaire containing the following instruments: European Quality of Life 5D index¹⁷ to measure health-related quality of life, Medication Adherence Report Scale¹⁸ to measure self-reported medication adherence, and additional items to measure sociodemographic variables. Because of data security regulations, patient participation in the survey was restricted to enrollees in a PCP-centered care contract.¹⁹ Participation in the patient survey was incentivized by a lottery of 5 shopping vouchers (€100).

In Germany, population-based disease management programs (DMPs) are part of routine care for 4 of the index conditions (type 2 diabetes mellitus, chronic obstructive pulmonary disease, asthma, and congestive heart failure caused by coronary heart disease).²⁰ German DMPs consist of regular follow-up visits up to every 3 months. They include clinical examination, laboratory tests (eg, glycosylated hemoglobin tests), and patient education.²¹ However, essential elements of care management interventions like individualized assessment, care planning, and frequent symptom monitoring are not routinely part of DMPs.²² Participation in German DMPs is voluntary and free of charge. German sickness funds are free to set incentives for patients to be enrolled in intensified care programs. The AOK decided to partly exempt beneficiaries from copayments (up to €40 a year) if they are willing to participate in DMPs. We considered prior voluntary participation in at least 1 of these programs during 2007-2008 as a proxy for “approachability,” indicating patients’ willingness to actively manage their disease and to participate in a care management program.²³

Data Analysis

Quantitative data are presented as means, 95% confidence intervals (CIs) of means, absolute numbers, and proportions. Two-sided c2 test was used to compare distributions of categorical variables. Means of continuous variables were compared by univariate analysis of variance, with performance of Games-Howell test as a statistical post hoc test.²⁴ This test accounts for unequal variance and largely heterogeneous sample sizes. P <.05 (2-sided) was considered statistically significant. All statistical analyses were performed using SPSS version 18.0 (SPSS Inc, Chicago, IL).

RESULTS

The mean number of AOK beneficiaries per primary care practice was 603 (95% CI, 409-797). All beneficiaries were screened independently for case finding by PM and selection by PCP. Primary care physicians screened a mean of 464 (95% CI, 294-631) AOK beneficiaries per practice.

Predictive modeling identified 301 patients for eligibility in a care management program, while selection by PCP identified 203 patients who would seem to benefit from a care management intervention on the basis of clinical judgment. Another group of 32 patients was concordantly identified using both PM and selection by PCP. All groups were mutually exclusive.

Table 1

Eligible patients identified using PM or selection by PCP were significantly older (P <.01) than nonselected patients (). Compared with nonselected patients, eligible patients had a significantly higher likelihood of future hospitalization (P <.01) and a 2-fold (selection by PCP) to 4-fold (PM) higher morbidity burden as indicated by the Charlson Comorbidity Index (P <.01). Predicted annual healthcare costs for nonselected patients were €2882 (95% CI, €2798-€2966), whereas significantly higher costs (P <.01) were predicted for patients selected by PCP (€4541 [95% CI, €4094-€4989]), PM (€8760 [95% CI, €8314-€9205]), or a combination of both (€8279 [95% CI, €7465-€9092]). Compared with patients selected by PM, patients selected by PCP had significantly higher rates of prior participation in intensified care programs (80.8% vs 56.4%, P <.01).

Compared with nonselected patients, PM identified patients with significantly more prior hospitalizations (P <.01), whereas patients selected by PCPs showed no significant difference in prior hospitalizations. Compared with patients selected by PM, the mutually exclusive subgroup of patients identified concordantly by PM and selection by PCP demonstrated no significant differences in morbidity burden, previous hospitalizations, or predicted healthcare utilization.

The prevalences of 8 chronic conditions known to be clinically relevant and potentially sensitive to ambulatory care differed among patients identified using PM, selection by PCP, or a combination of both. In general, all 3 subpopulations showed higher prevalences of any of these conditions compared with nonselected patients. However, 1 of every 2 patients selected by PM had chronic heart failure, while selection by PCP resulted in a 3-fold lower prevalence of chronic heart failure. Patients identified concordantly by PM and selection by PCP had a prevalence of chronic heart failure similar to that among patients identified by PM.

Figure

Table 2

In total, 70.1% (376 of 536) of selected patients were enrolled in a PCP-centered care contract and were eligible for the patient survey (). Predictive modeling identified 58.5% (176 of 301) of enrollees, and selection by PCP identified 84.7% (172 of 203) of enrollees. In total, 40.7% (153 of 376) of patients responded to the survey. Patients selected by PCP responded more frequently to the patient survey than patients selected by PM (58.1% vs 22.7%, P <.01). A total of 46.1% of patients identified concordantly by PM and selection by PCP responded to the survey ().

Table 2

Most survey respondents had a low level of education (eighth grade or less) (). More than one-third of selected patients lived alone. Their health-related quality of life was poor overall. Patients selected by PM had significantly lower European Quality of Life 5D index scores (P <.05) than patients selected by PCP. One-third to one-half of selected patients reported problems with activities of daily living, and up to one-third of patients reported serious pain or bodily problems. Data from the Medication Adherence Report Scale showed that about one-half of selected patients might have had problems with medication adherence (summary score, <25). Prior participation in DMPs was significantly less frequent among patients selected by PM (P <.01) compared with patients selected by PCP.

Table 3

Analysis of insurance claims data revealed that nonrespondents to the survey were similar in age to respondents (). However, nonrespondents had significantly more hospital admissions, greater morbidity burden, and higher predicted future healthcare costs and hospitalizations (P <.01 for all) compared with survey respondents.

DISCUSSION

The objective of this study was to compare 3 different approaches for the identification of patients likely to benefit from care management programs. Compared with selection by PCP, diagnostic cost group—based PM for likelihood of hospitalization identified patients at greater risk for future healthcare utilization. These patients had a higher morbidity burden and clinically relevant and potentially ambulatory care–sensitive conditions. Further characterization of these high-risk patients revealed substantial restrictions in quality of life and activities of daily living, indicating that they might benefit from complex care management programs. However, a large proportion of these patients may not be approachable for such programs. Prior participation in population-based DMPs and response rates to the patient survey were significantly lower among the high-risk individuals selected by PM compared with those identified using selection by PCP or concordantly by PM and selection by PCP. This finding may be indicative of lower care sensitivity because of a lack of approachability. Remarkably, selection by PCP identified young patients with higher quality-of-life scores compared with those selected by PM. While chronic heart failure and depression are common target conditions for care management programs and were explicitly chosen as inclusion criteria for this study, only small proportions of patients with these chronic conditions were identified by PCPs compared with PM. In contrast to PM, PCPs did not seem to use prior hospitalizations as an indicator of eligibility for care management programs. Although the actual motives of PCPs for patient selection remain unclear, a significant proportion of patients identified by PM may be too ill to benefit from care management programs.

The small subgroup identified concordantly by PM and selection by PCP had a high risk of future healthcare utilization and seemed likely to be approachable for intensified care programs. Their risk profile was similar to that of patients selected by PM, but significantly more of the concordantly identified patients were willing to participate in intensified care programs.

Prior studies on clinical characteristics of high-risk individuals reported similar results. Forrest et al⁷ compared different versions of PM software and suggested that clinically based PM may help identify patients who would benefit from care management programs as indicated by prior healthcare utilization and morbidity burden. Sylvia et al⁸ analyzed the top 18% of high-risk patients who were predicted by adjusted clinical group PM about future healthcare costs. In our study, clinical characteristics of respondents were comparable to those of respondents. Similar to our findings, Sylvia et al reported that nonrespondents to their survey were at highest risk for future healthcare utilization. Guided Care, a large care management trial that targeted patients above the 75th percentile of risk for future healthcare costs using the Centers for Medicare & Medicaid Services hierarchical clinical categories model for PM, was able to recruit only about 38% of all eligible patients selected by PM.²⁵ Because of data security restrictions in the present study, nonrespondents herein could not be further characterized about risk profile and care sensitivity markers.

Implications for Care Management Programs

As care management programs are increasingly integrated into routine care, practicable concepts for case finding will be required. Data from our study support the hypothesis that PM may be superior to selection by PCP in screening for high-risk cases. However, because patients at highest risk may not be amenable to active participation in care management programs, PCPs may be able to refine PM selection to target patients as both high risk and care sensitive. A combined approach may be more favorable than PM or selection by PCP alone.26 Further research should determine underlying variables for care sensitivity that could be included in insurance claims data—based PM. This may result in more precise case-finding algorithms with shorter lists of potential participants. Because screening requires human and financial resources, this approach may enable more efficient care management programs that will reach more patients who are likely to benefit. Furthermore, standardized assessment tools should be developed and implemented to assist clinicians in refining PM-based selection. However, care management programs may be unable to reach a large proportion of patients who are at highest risk. This unsolved problem requires further investigation as to how these patients might be targeted by intensified care programs, if possible at all.

Limitations

The results of this study should be interpreted in light of several limitations. Although we performed an explorative study, our results may be of relevance as the first empirical comparison to date of different target populations for care management programs identified using PM and selection by PCPs. We included beneficiaries of a large German sickness fund who lived in a single geographic area and were patients at small-to-medium primary care practices. Interpretation of our results should consider potential differences among patient populations of other sickness funds, regions, and nations. The observational study design does not allow us to draw final conclusions about the most effective approach to identify patients who benefit from care management programs. This would require clinical trials comparing the effects of different case-finding approaches on patient outcomes and costs. However, our results may facilitate further research in this field. In addition, this study did not compare complex PM with simpler prediction tools like prior costs alone, which seem to be inferior in targeting patients to reduce hospitalization rates.⁷

Case selection by PCPs may be made on an individual basis. We did not assess motives and criteria that guide PCP-based patient selection for care management programs. Qualitative research may close this knowledge gap and contribute to a comprehensive understanding of care sensitivity, which has been only roughly operationalized in this study. We used prior participation in DMPs as a proxy for the care sensitivity domain of approachability. While this could be useful in the sense that patients who are unwilling to participate in less complex German DMPs would hardly be willing to participate in care management programs, it cannot automatically be seen as a valid predictor of actual participation in care management programs. Other indicators for care sensitivity may be revealed through further research in the field.

Insurance claims data analyses have specific limitations. Diagnostic coding may be strongly cost driven through remuneration procedures.²⁷ Therefore, insurance claims data—based analyses of morbidity burden may need careful interpretation. ²⁸ However, the combination of outpatient and inpatient diagnostic codes used in the present study may have reduced the coding biases.

Interpretation of our survey results should take into account the small sample sizes and high rates of nonresponse. Differing response rates of patients selected by PM and PCPs may have been influenced by selection bias. Primary care providers might have been more motivated to include patients whom they had proposed for participation in care management programs. However, because potential participants in these programs may be approached by their PCPs, this finding may be significant for program providers.

Finally, beneficiaries not enrolled in a PCP-centered care contract were excluded from the patient survey because of data security regulations. This may further limit interpretation of our results. However, enrollees in PCP-centered care contracts are known to be high-risk patients having a higher morbidity burden compared with beneficiaries who are not enrolled in these contracts.²⁹ This may have led to an underestimation of the care needs among the selected patient population in our study.

Author Affiliations: From the Department of General Practice and Health Services Research (TF, CM, DO, JS, FPK), University Hospital Heidelberg, Heidelberg, Germany; Institute of General Practice (AE), Frankfurt am Main, Germany; and Institute of General Practice (JG), Friedrich Schiller University Jena, Jena, Germany.

Funding Source: This study was supported by the General Regional Health Fund (Allgemeine Ortskrankenkasse), Baden-Württemberg, Germany.

Author Disclosures: The authors (TF, CM, AE, JG, DO, JS, FPK) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (TF, CM, AE, JG, JS, FPK); acquisition of data (TF); analysis and interpretation of data (TF, CM, JG, DO, FPK); drafting of the manuscript (TF, JG, FPK); critical revision of the manuscript for important intellectual content (CM, AE, JG, DO, JS, FPK); statistical analysis (TF, AE, DO); provision of study materials or patients (AE); obtaining funding (AE, JS); administrative, technical, or logistic support (AE); and supervision (AE, JS, FPK).

Address correspondence to: Tobias Freund, MD, Department of General Practice and Health Services Research, University Hospital Heidelberg, Vobstrasse 2, D-69115 Heidelberg, Germany. E-mail: tobias.freund@med.uni-heidelberg.de.

1. Bodenheimer T, Berry-Millet R. Care Management of Patients With Complex Health Care Needs. Princeton, NJ: Robert Wood Johnson Foundation; December 2009. Research synthesis report 19.

2. Starfield B, Lemke KW, Bernhardt T, Foldes SS, Forrest CB, Weiner JP. Comorbidity: implications for the importance of primary care in ‘case’ management. Ann Fam Med. 2003;1(1):8-14.

3. Goetzel RZ, Ozminkowski RJ, Villagra VG, Duffy J. Return on investment in disease management: a review. Health Care Financ Rev. 2005; 26(4):1-19.

4. Bernstein RH. New arrows in the quiver for targeting care management: high-risk versus high-opportunity case identification. J Ambul Care Manage. 2007;30(1):39-51.

5. Ash AS, Ellis RP, Pope GC, et al. Using diagnoses to describe populations and predict costs. Health Care Financ Rev. 2000;21(3):7-28.

6. Johns Hopkins ACG System. ACG predictive model. http://www.acg.jhsph.edu. Accessed September 24, 2010.

7. Forrest CB, Lemke KW, Bodycombe DP, Weiner JP. Medication, diagnostic, and cost information as predictors of high-risk patients in need of care management. Am J Manag Care. 2009;15(1):41-48.

8. Sylvia ML, Shadmi E, Hsiao CJ, Boyd CM, Schuster AB, Boult C. Clinical features of high-risk older persons identified by predictive modeling. Dis Manag. 2006;9(1):56-62.

9. Boyd CM, Boult C, Shadmi E, et al. Guided care for multimorbid older adults. Gerontologist. 2007;47(5):697-704.

10. German Ministry of Health. http://www.bundesgesundheits ministerium.de/cln_178/nn_1168278/DE/Ministerium/Statistik/Statistik.html. Accessed December 15, 2010.

11. Freund T, Wensing M, Mahler C, et al. Development of a primary care—based complex care management intervention for chronically ill patients at high risk for hospitalization: a study protocol. Implement Sci. 2010;5:e70.

12. Weir S, Aweh G, Clark RE. Case selection for a Medicaid chronic care management program. Health Care Financ Rev. 2008;30(1):61-74.

13. Verisk Health. Likelihood of Hospitalization (LOH): Kaiser study. http://www.veriskhealth.com/content/verisk-healthcare-white-papersand-research. Accessed September 28, 2010.

14. Purdy S, Griffin T, Salisbury C, Sharp D. Ambulatory care sensitive conditions: terminology and disease coding need to be more specific to aid policy makers and clinicians [published correction appears in Public Health. 2010;124(12):720]. Public Health. 2009;123(2):169-173.

15. Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA. New ICD-10 version of the Charlson Comorbidity Index predicted in-hospital mortality. J Clin Epidemiol. 2004;57(12):1288-1294.

16. Perkins AJ, Kroenke K, Unützer J, et al. Common comorbidity scales were similar in their ability to predict health care costs and mortality. J Clin Epidemiol. 2004;57(10):1040-1048.

17. Greiner W, Weijnen T, Nieuwenhuizen M, et al. A single European currency for EQ-5D health states: results from a six-country study. Eur J Health Econ. 2003;4(3):222-231.

18. Mahler C, Hermann K, Horne R, et al. Assessing reported adherence to pharmacological treatment recommendations: translation and evaluation of the Medication Adherence Report Scale (MARS) in Germany. J Eval Clin Pract. 2010;16(3):574-579.

19. Klingenberg A, Broge B, Herholz H, Szecsenyi J, Ose D. Evaluation of special GP-centered health care contracts in Germany by the participating GPs: present situation and future perspectives [in German]. Med Klin (Munich). 2010;105(2):89-95.

20. Busse R. Disease management programs in Germany’s statutory health insurance system. Health Aff (Millwood). 2004;23(3):56-67.

21. Szecsenyi J, Rosemann T, Joos S, Peters-Klimm F, Miksch A. German diabetes disease management programs are appropriate for restructuring care according to the chronic care model: an evaluation with the Patient Assessment of Chronic Illness Care instrument. Diabetes Care. 2008;31(6):1150-1154.

22. Freund T, Kayling F, Miksch A, Szecsenyi J, Wensing M. Effectiveness and efficiency of primary care based case management for chronic diseases: rationale and design of a systematic review and meta-analysis of randomized and non-randomized trials [CRD32009100316]. BMC Health Serv Res. 2010;10:e112.

23. Dunkelberg S, Zingel D, Noack A, van den Bussche H, Kaduszkiewicz H. Which patients are (not) included in the DMP diabetes programme [in German]? Gesundheitswesen. 2006;68(5):289-293.

24. Games PA, Howell JF. Pairwise multiple comparison procedures with unequal n’s and/or variances: a Monte Carlo study. J Educ Stat. 1976;1(2):113-125.

25. Boyd CM, Reider L, Frey K, et al. The effects of Guided Care on the perceived quality of health care for multi-morbid older persons: 18-month outcomes from a cluster randomized controlled trial. J Gen Intern Med. 2010;25(3):235-242.

26. Boult C, Pualwan TF, Fox PD, Pacala JT; HMO Workgroup on Care Management. Identification and assessment of high-risk seniors. Am J Manag Care. 1998;4(8):1137-1146.

27. Retchin SM, Ballard DJ. Commentary: establishing standards for the utility of administrative claims data. Health Serv Res. 1998;32(6): 861-866.

28. Erler A, Beyer M, Muth C, Gerlach FM, Brennecke R. Garbage in: garbage out? validity of coded diagnoses from GP claims records [in German]. Gesundheitswesen. 2009;71(12):823-831.

29. Freund T, Szecsenyi J, Ose D. Characteristics of beneficiaries of a GP-centred health care contract in Germany [in German]. Med Klin (Munich). 2010;105(11):808-811.

Download PDF: Identification of Patients Likely to Benefit From Care Management Programs