Adaptation and Psychometric Properties of the PACIC Short Form

Objectives:

The Patient Assessment of Chronic Illness Care (PACIC) is a widely used instrument to evaluate the quality and patient-centeredness of chronic illness care based on the Chronic Care Model (CCM). It is a validated and reliable instrument which consists of 20 items. Additionally, a short form with 11 items was developed. The aim of this study was to translate this short form into German and examine the psychometric properties among patients with a chronic illness in Germany.

Study Design:

Observational study design.

Methods:

We performed a translation and cultural adaptation of the PACIC short form into German. The German version was externally validated with the 20-item PACIC. Cronbach a, descriptive statistics, and principal component analysis were used to assess psychometric properties.

Results:

In total, 264 primary care patients completed the PACIC short form. The PACIC short form showed good convergent construct validity to the 20-item PACIC (Spearman rank correlation 0.82, P <.001) and high internal consistency (Cronbach a 0.87). Principal component analysis underlined the 1-dimensional structure of the instrument. No correlation between the mean overall score of the PACIC short form and the number of chronic conditions (r = 0.068; P = .273) was found.

Conclusions:

The PACIC short form showed good to very good psychometric properties and reliable measures regarding patient assessment of receiving care congruent with the CCM. It is a less burdensome instrument which can be used for further research of patients with more than 1 chronic condition.

(Am J Manag Care. 2012;18(2):e55-e60)This study looked at the psychometric properties of the German version of the Patient Assessment of Chronic Illness Care (PACIC) short form among patients with chronic illness.

• A major challenge for future healthcare is the care of patients with chronic conditions.

• Results showed that nearly 50% of respondents had more than 3 chronic conditions.

• The PACIC short form is a reliable instrument with good psychometric properties and can be used for patients with more than 1 chronic condition.

Care for patients with chronic conditions is one of the major challenges in future healthcare.¹ Patients with a chronic condition seek medical care more often than patients without a chronic condition. This may increase the cost of healthcare.^2,3 The Chronic Care Model (CCM), developed by Wagner et al, illustrates a multidimensional framework for improving the quality of chronic care based on 6 key dimensions: organization of healthcare, clinical information system, delivery system design, decision support, selfmanagement support, and community resources.⁴ The Patient Assessment of Chronic Illness Care (PACIC) is an instrument widely used to evaluate the quality and patient-centeredness of chronic illness care.⁵ It has been developed to measure implementation of the CCM at the level of patients assessing the behavior of professionals and practice teams within their care.⁵ It is a validated and reliable instrument for different chronic conditions^6-8 and has been translated into different languages.^9-11

However, it was found that the development of the structure of the PACIC instrument implicates some methodological limitations.^12-14 The 20-item PACIC based on a 5-factor solution (patient activation, delivery system design/decision support, goal setting/tailoring, problem solving/contextual, and follow-up/coordination) was evaluated with a confirmatory factor analysis. Some research groups showed that this determination of factorial validity, including the distribution of items as well as the option of handling the responses as interval or ratio levels, is the main point of criticism.^13-15 Therefore, a PACIC short form with an adapted response scale was developed to evaluate patient assessment of receiving care congruent with the CCM.

The aim of this study was to translate and examine the psychometric properties of the PACIC short form among patients in Germany with a chronic illness.

METHODS

We performed an observational study in 11 general practices in Germany (8 located in the federal state of Baden-Württemberg, and 3 in the federal\ state of Thuringia). All practices were teaching sites for medical students of either the University Hospital of Heidelberg or the University Hospital of Jena.

Translation and Cultural Adaption

To adapt the PACIC short form we followed the Principles of Good Practice for the Translation and Cultural Adaptation Process by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) task force¹⁶ as follows: We received permission from the authors who developed the PACIC short form, Gugiu et al from Western Michigan University, to translate and adapt a German version of the instrument.¹³ Two researchers (JS, TF) independently translated the English version of the PACIC short form 11-item scale into German. Divergent results were discussed during consensus meetings with a third researcher (KG). After a linguistic adaptation, no item was assumed to be completely inappropriate.

Recruitment and Data Collection

For recruiting practices we chose a convenience sample of teaching practices working either with the University Hospital of Heidelberg or the University Hospital of Jena. Within the practices of the participating general practitioners (GPs), patients 18 years or older, suffering from at least 1 major chronic condition (defined in accordance with the German Social Code Book V §62),¹⁷ were asked to participate in the survey. Patients with severe cognitive impairment or significant language barriers were excluded from the study by the individual GP of the practice. Patients were asked to fill out a depersonalized paper-based questionnaire and to send it back to the study center. We provided a postage-free envelope but no further financial incentives for patients. Written informed consent was obtained from each participant. At each practice site in Baden-Württemberg, the questionnaire was given to 50 participating patients (45 patients in each of the 3 practice sites in Thuringia). The depersonalized paper-based questionnaire consisted of sociodemographic data, a list of 20 chronic conditions, the 20-item PACIC, and the PACIC short form. All participating practices were supported for recruiting patients with 50 €.

Measures

Table 1

The PACIC short form consists of 11 items with an 11-point percentage scale ranging from 0% (“Never”) to 100% (“Almost”). In addition to the PACIC short form, we measured sociodemographic data with a set of questions from a German standard questionnaire and patients were also asked to select their chronic conditions from a list of 20 conditions, which were used in previous evaluation studies on chronic illness care.¹⁸ The sociodemographic data included questions regarding age, gender, marital status, and education (). We used the 20-item PACIC to assess convergent construct validity with the PACIC short form, which has been validated within multiple studies.^8-11,19 The items of the long version were scored on a 5-point Likert scale, ranging from “1” (“no/never”) to full accordance “5” (“yes/always”).

Statistical Analysis

All analyses were carried out using SPSS 18.0 software (SPSS Inc, Chicago, Illinois). The aim of the study was to assess psychometric properties of the PACIC short form and to examine the convergent construct validity with the 20-item PACIC. The reliability was assessed by using Cronbach’s alpha, which indicates whether an item of a scale is appropriate for assessing the underlying concept of its scale.²⁰ Values for Cronbach’s alpha range from 0 to 1; the closer they are to 0 the less the items are related to one another. Values above 0.60 are generally considered to indicate satisfying internal consistency and values above 0.80 indicate a high internal consistency. We performed principal factor analysis (eigenvalue >1, varimax rotation) and determined the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and the Bartlett’s test of sphericity. Convergent construct validity was assessed in terms of a Spearman rank correlation test between means of each item of the PACIC short

form and the mean for the 20-item PACIC overall score, a well as between the mean score of the PACIC short form and the mean score of the 20-item PACIC. In this context, correlations often range between 0.2 and 0.6, rarely above; correlations between 0.40 and 0.60 are regarded as good correlations.²¹ Frequency distributions and statistical moments were calculated as percentages of patients rating at each level of item. We determined means, standard deviations of means, and missing values on item level to identify potentially inadequate items. Finally, we examined with Spearman rank correlation test the relation between the mean overall score of the PACIC short form and the number of chronic conditions. An alpha level of P <.05 was used for tests of statistical significance.

Ethical Approval

The study was fully approved by the ethics committees of the medical faculties of the University of Heidelberg and the University of Jena.

RESULTS

In total, 264 out of 535 participants (49%) completed the PACIC short form. Details on sociodemographic characteristics and morbidity are given in Table 1. Almost every second participant (47.0%) has more than 3 chronic conditions. Hypertension (n = 184; 69.7%), back pain (n = 148; 56.1%), osteoarthritis (n = 113; 42.8%), and type 2 diabetes (n = 86; 32.6%) were the most frequently reported chronic conditions.

Table 2

Factor analysis revealed a 1-dimensional structure of the PACIC short form with explained variance of R² = 48.15% (KMO 0.90, Barlett's test of sphericity P <.001). The factor loadings ranged between 0.52 and 0.85. The Spearman rank correlation coefficients ranged between 0.38 for item 6 of the PACIC short form and 0.78 for item 8 of the PACIC short form. For the mean overall scores of the short version and 20- item PACIC, the correlation was 0.82 (P <.001). Details on factor loadings and Spearman rank correlation are given in . Furthermore, the PACIC short form showed an internal consistency reliability of 0.87 (Cronbach α).

Table 3

Frequency distributions and statistical moments of the PACIC short form are presented in . The participants tended to gravitate to both end points, 0% and 100%. The skewness of most of the items was within tolerable levels, tending to be fairly close to zero. A majority of the kurtosis values removed substantially from zero, which is acceptable.

Table 4

displays details on missing values on item level. We observed non-response rates which range from 4.2% and 12.5% on item level. The highest score of missing values showed item 7 of the PACIC short form with 33 nonresponders (12.5%). The lowest score was shown among item 2 of PACIC short form with 11 non-responders (4.2%).

No correlation between the mean overall score of the PACIC short form and the number of chronic conditions (r = 0.068; P =.273) was found.

DISCUSSION

The presented study describes good or even very good psychometric properties of the German version of the PACIC short form and shows reliable measures regarding patient assessment of receiving care congruent with the CCM. The results of our study found high internal consistency and good external validity of the 1-dimensional scale. Moderate missing rates, low floor, and ceiling effects on item level support these results.

Compared with the original version of the PACIC short form, the German version shows similar psychometric properties. The development of the original instrument based on data from 2 samples of patients with type 2 diabetes revealed an internal consistency of α = 0.95 and α = 0.96, respectively.¹³ The high correlation between the mean scores of both PACIC versions shows that the short form of the PACIC reflects the 20-item PACIC sufficiently. Moreover, regarding the fact that the mean score of the PACIC short form and the number of chronic conditions do not correlate, we can conclude that the PACIC measurement is independent of the number of chronic conditions.²² The development of a generic questionnaire to assess whether quality of care is in focus with the CCM is important for future studies focusing on the issue of multimorbidity.²³ The number and distribution of chronic conditions which we observed are comparable with other studies.^9,24,25

Three items (“given choices about treatment to think about” [PACIC 1 S]; “helped to make a treatment plan that I could carry out in my daily life” [PACIC 7 S]; and “helped to plan ahead so I could take care of my condition even in hard times” [PACIC 8 S]) had about 10% missing values. We could assume that these 3 items are not explicit enough, as there are 2 issues to assess in each question, and therefore they need reformulation.²⁶

Two items, “choices about treatment” and “treatment plan,” resulted in a relatively low response rate. From shared decision-making (SDM) studies in Germany we know that patient-doctor communication is an important subject for the individual patient’s treatment preference.^27,28 Additionally, we know that there is a need to improve these SDM skills.²⁸ Lack of SDM skills might be a possible explanation for the low response rate of these 2 items. However, this study and the described German short form of the PACIC make a valuable contribution to the requirements of future research on chronic disease management in German-speaking countries. It is important to have valid and less burdensome instruments that could be included either as intervention elements or evaluation measures.

Strengths and Weaknesses

We included a convenient sample of patients from 11 general practices throughout 2 different federal states located in eastern and western Germany. Our results have to be interpreted against the background of potential selection bias due to a moderate participation rate. Moderate participation rates in paper-based questionnaires are very common especially in the case of absent [financial] incentives for the participants.²⁹ Additional, future research should include follow-up to increase the response rate. However, the wide range of ages, as well as numbers and types of chronic conditions, are comparable to other studies.^24,25 The number of valid questionnaires appears to be sufficient for robust equations of the psychometric properties of the PACIC short form. Due to the study design, test-retest reliability and responsiveness to change could not be determined and should be targeted in further research.

CONCLUSIONS

The PACIC short form is a reliable instrument with good psychometric properties. Additionally, the short version of the PACIC presented a less burdensome instrument compared with the 20-item PACIC to measure patient assessment of receiving care congruent with the CCM. This is essential for improving quality of care and focusing on a patient-centeredness approach. Furthermore, due to its generic nature, it offers the opportunity to be used for patients with more than 1 chronic

condition. The availability of this instrument encourages further research in this field in German-speaking countries.Acknowledgments

Our thanks go to Cristian Gugiu for providing the original instrument.

We thank all participating practice teams and patients for their support. We thank Ingrid Gerlach (Jena) and Iven Fellhauer (Heidelberg) for supporting data collection. Drs Goetz and Freund contributed equally to this work.

Author Affiliations: From Department of General Practice and Health Services Research (KG, TF, AM, JS, JS), University Hospital Heidelberg, Germany; Department of General Practice (JG), Schiller University Hospital Jena, Germany.

Funding Source: This study was funded by the Federal Ministry of Art and Science Baden-Württemberg as a project of the Competence Centre General Practice Baden-Württemberg.

Author Disclosures: The authors (KG, TF, JG, AM, JS, JS) report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (KG, TF, JG, JS); acquisition of data (KG, TF, JS); analysis and interpretation of data (KG, TF, JG, AM, JS, JS); drafting of the manuscript (KG, TF, AM, JS, JS); critical revision of the manuscript for important intellectual content (JG, AM, JS); statistical analysis (KG, TF); provision of study materials or patients (KG, TF, JG, JS); obtaining funding (KG, TF, AM, JS, JS); administrative, technical, or logistic support (KG, TF); and supervision (JS).

Address correspondence to: Katja Goetz, PhD, Department of General Practice and Health Services Research, University Hospital Heidelberg, Vossstrasse 2, 69115 Heidelberg, Germany. E-mail: katja.goetz@med.uni-heidelberg.de.1. World Health Organization. Chronic disease report. Preventing chronic diseases: a vital investment. http://www.who.int/chp/chronic_disease_report/content/en/index.html. Accessed July 11, 2011.

2. Goetz K, Campbell S, Willms S, Rochon J, Klingenberg A, Szecsenyi J. How do chronically ill patients evaluate their medical care? an observational study with 46919 patients in 676 primary care practices. Int J Pers Cent Med. 2011;1:338-346.

3. Naessens JM, Stroebel RJ, Finnie DM, et al. Effect of multiple chronic conditions among working-age adults. Am J Manag Care. 2011;17(2):118-122.

4. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A. Improving chronic illness care: translating evidence into action. Health Aff (Millwood). 2001;20(6):64-78.

5. Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM. Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Med Care. 2005;43(5):436-444.

6. Szecsenyi J, Rosemann T, Joos S, Peters-Klimm F, Miksch A. German diabetes disease management programs are appropriate for restructuring care according to the chronic care model: an evaluation with the Patient Assessment of Chronic Illness Care instrument. Diabetes Care. 2008;31(6):1150-1154.

7. Schmittdiel J, Mosen DM, Glasgow RE, Hibbard J, Remmers C, Bellows J. Patient Assessment of Chronic Illness Care (PACIC) and improved patient-centered outcomes for chronic conditions. J Gen Intern Med. 2008;23(1):77-80.

8. Gensichen J, Serras A, Paulitsch MA, et al. The Patient Assessment of Chronic Illness Care questionnaire: evaluation in patients with mental disorders in primary care. Community Ment Health J. 2011;47(4): 447-453. doi:10.1007/s10597-010-9340-2.

9. Rosemann T, Laux G, Droesemeyer S, Gensichen J, Szecsenyi J. Evaluation of a culturally adapted German version of the Patient Assessment of Chronic Illness Care (PACIC 5A) questionnaire in a sample of osteoarthritis patients. J Eval Clin Pract. 2007;13(5):806-813.

10. Aragones A, Schaefer EW, Stevens D, Gourevitch MN, Glasgow RE, Shah NR. Validation of the Spanish translation of the Patient Assessment of Chronic Illness Care (PACIC) survey. Prev Chronic Dis. 2008; 5:A113.

11. Wensing M, van Lieshout J, Jung HP, Hermsen J, Rosemann T. The Patients Assessment of Chronic Illness Care (PACIC) questionnaire in The Netherlands: a validation study in rural general practice. BMC Health Serv Res. 2008;8:182.

12. Spicer J, Budge C, Carryer J. Taking the PACIC back to basics: the structure of the Patient Assessment of Chronic Illness Care. J Eval Clin Pract [published online ahead of print October 25, 2010]. doi:10.1111/j.1365-2753.2010.01568.x.

13. Gugiu PC, Coryn C, Clark R, Kuehn A. Development and evaluation of the short version of the Patient Assessment of Chronic Illness Care instrument. Chronic Illn. 2009;5(4):268-276.

14. Gugiu PC, Coryn CLS, Applegate B. Structure and measurement properties of Patient Assessment of Chronic Illness Care instrument. J Eval Clin Pract. 2010;16:509-516.

15. Jöreskog KG. On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika. 1994;59:381-389.

16. Wild D, Grove A, Martin M, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104.

17. German Joint Federal Board. Directive for the implementation of regulations for severely chronically ill patient according to § 62 SGB V, 2008. [in German], http://www.g-ba.de/informationen/richtlinien/8/. Accessed July 11, 2011.

18. Joos S, Rosemann T, Heiderhoff M, et al. ELSID-Diabetes studyevaluation of a large scale implementation of disease management programmes for patients with type 2 diabetes: rationale, design and conduct—a study protocol [ISRCTN08471887]. BMC Public Health. 2005; 5:99.

19. Glasgow RE, Whitesides H, Nelson CC, King DK. Use of the Patient Assessment of Chronic Illness Care (PACIC) with diabetic patients: relationship to patient characteristics, receipt of care, and self-management. Diabetes Care. 2005;28(11):2655-2661.

20. Cronbach LJ. Coefficient Alpha and the internal structure of tests. Psychometrika. 1951;16:297-334.

21. Nunnally JC. Psychometric Theory. 2nd ed. New York, NY: McGraw-Hill; 1979.

22. Ose D, Freund T, Urban E, Kunz CU, Szecsenyi J, Miksch A. Comorbidity and patient-reported quality of care: an evaluation of the primary care based German disease management program for type 2 diabetes. J Public Health. 2012;20(1):41-46. doi:10.1007/s10389-011-0429-z.

23. Coleman K, Austin BT, Brach C, Wagner EH. Evidence on the Chronic Care Model in the new millennium. Health Aff (Millwood). 2009;28(1):75-85.

24. Glynn LG, Valderas JM, Healy P, et al. The prevalence of multimorbidity in primary care and its effect in health care utilization and cost. Fam Pract. 2011;28(5):516-523. doi:10.1093/fampra/cmr013.

25. Schäfer I, von Leitner E-C, Schön G, et al. Multimorbidity patterns in the elderly: a new approach of disease clustering identifies complex interrelations between chronic conditions. PLoS One. 2010;5(12): e15941.

26. Sim J, Wright C. Research in Health Care: Concepts, Designs and Methods. Cheltenham: Nelson Thornes Ltd; 2000.

27. Müller-Engelmann M, Keller H, Donner-Banzhoff N, Krones T. Shared decision making in medicine: the influence of situational treatment factors. Patient Educ Couns. 2011;82(2):240-246.

28. Sonntag U, Wiesner J, Fahrenkrog S, Renneberg B, Braun V, Heintze C. Motivational interviewing and shared decision making in primary care. Patient Educ Couns [published online ahead of print August 26, 2011]. doi:10.1016/j.pec.2011.07.026.

29. Edwards PJ, Roberts IG, Clarke MJ, et al. Methods to increase response rates to postal and electronic questionnaires (Review). Cochrane Database Sys Rev. 2009;8(3):MR000008. 2007;(2); doi:10.1002/14651858.MR000008.pub4.

Download PDF: Adaptation and Psychometric Properties of the PACIC Short Form