The Patients Assessment Chronic Illness Care (PACIC) questionnaire in The Netherlands: a validation study in rural general practice
© Wensing et al; licensee BioMed Central Ltd. 2008
Received: 27 February 2008
Accepted: 01 September 2008
Published: 01 September 2008
Many patients with chronic illness receive health care in primary care settings, so a challenge is to provide well-structured chronic care in these settings. Our aim was to develop and test a Dutch version of the PACIC questionnaire, a measure for patient reported structured chronic care.
Observational study in 165 patients with diabetes or COPD from four general practices (72% response rate). Patients completed a written questionnaire, which included instruments for assessing chronic illness care (PACIC), evaluations of general practice (Europep), enablement (PEI), and individual characteristics.
The patients had a mean age of 68.0 years and 47% comprised of women. Twenty-two to 35% of responding patients did not provide answers to specific items in the PACIC. In 11 items the lowest answering category was used by 30% or more of the responders and in 6 items the highest answering category was used by this number of responders. Principal factor analysis identified the previously defined five domains reasonably well. Cronbach's alpha per domain varied from 0.71 to 0.83, and the intraclass coefficient from 0.66 to 0.91. Diabetes patients reported higher presence of structured chronic care for 14 out of the 20 PACIC items. The effect of patient evaluations of general practice on the PACIC score was positive (b = 0.72, p < 0.004), but the effect of patient enablement on the PACIC score was negative (b = -1.13, p < 0.000).
A translated and validated Dutch version of the PACIC questionnaire is now available. Further research on its validity is recommended.
The Chronic Care Model is a conceptual framework that supports the evidence-based proactive and planned care for chronic diseases [1, 2]. It has received widespread acceptance as a framework for improving the care of chronically ill patients. Measures of chronic care delivery are required to target efforts to improve chronic care and to monitor change of chronic care delivery over time. The Patient Assessment of Chronic Illness Care (PACIC) is a 20-item questionnaire for patients, which intends to measure chronic care delivery and which has been validated in USA for diabetes  and in Germany for osteoarthritis . A version for chronic care in general practice in The Netherlands was not yet available, although many chronic patients receive most of their health care in general practice. Therefore the aim of our study was to develop a Dutch version of the PACIC instrument and test it on patients with diabetes or COPD in general practice.
Design and setting
An observational study was performed in randomly sampled patients from four general practices, which were situated in a rural area in the south-eastern part of The Netherlands. Two practices were single-handed and two practices were group practices. All practices were involved in a program to enhance structured diabetes care, while no such program existed for COPD care. Ethical approval was received for this study from the Arnhem-Nijmegen ethical committee.
In each practice patients with diabetes mellitus and with COPD were sampled from the medical record system. An alphabetically ordered list of patient names was made, from which every second patients was included up to 30, except for one practice, which could provide only 20 COPD patients. A total of 230 patients was approached (120 diabetes patients and 110 COPD patients). Written questionnaires were sent by the practices, followed by postcard reminders three weeks later. Patients were invited to complete the questionnaire and return it anonymously in a prepaid envelope to the research institute. Informed consent was not explicitly asked, but implied when a patient returned the questionnaire.
The questionnaire included the following measures. Patient assessment of chronic illness care (PACIC) was measured with a 20 item questionnaire, which used a five point response scale (ranging from 1 = 'almost never' to 5 = 'almost always') . Higher scores mean more frequent presence of the aspect of structured chronic care. This instrument has five pre-defined domains: patient activation (3 items), delivery system/practice design (3 items), goal setting/tailoring (5 items), problem solving/contextual (4 items), follow-up/coordination (5 items). The English version was translated and culturally adapted in a structured procedure, including forward and backward translations, each by two independent researchers and then established in a consensus meeting with the four individuals involved. Next, face to face interviews were done with 15 elderly patients with chronic illness from one general practice. This led to substantial adaptations, mainly to simplify and clarify the questions, which were also discussed with the authors of the PACIC. Finally, we made two slightly different versions, one for patients with diabetes and one for patients with COPD (see Additional file 1). These versions different in two ways: the heading referred to diabetes or lung disease, and item 19 referred to lung physicians (for COPD patients) or internist, surgeon or ophmatologist (for diabetes patients). Aggregated mean scores for five domains and for the total instrument were calculated as described in previous research . No scores per domain were determined for patients with missing values on on more than one third of the items in the domain.
Patient enablement (PEI) was measured with a six-item questionnaire (with three response categories: 0 = 'same or worse', 1 = 'better', 2 = 'much better') . The range of the aggregated sum score was 0 to 12, with a higher score indicating a higher level of enablement. Respondents giving two or more missing values were excluded. Patient evaluations of general practice were measured with the Europep instrument, a 23-item internationally standardised and validated questionnaire (with a five point answering scale, ranging from 'poor' to 'excellent') . For this study, we determined the overall mean value on the 17 dichotomized (excellent versus other values) items focused on clinical performance (Cronbach's alpha = 0.97). Respondents with more than 5 missing items were excluded. Finally the questionnaire contained questions on patient age, gender, highest education, and overall health status (single item with five point response scale, 'excellent to 'poor).
The analysis of the measurement properties of the PACIC was based on published quality criteria for questionnaires . The content validity of the PACIC is based on the Chronic Care Model [1, 2]. The interpretability of the instrument (the extent that qualitative meaning can be assigned to the qualitative scores) was based on the percentage of chronic patients who provided valid responses on each of the items. In addition, we checked for floor and ceiling effects in terms of percentage of patients using the most extreme (upper or lower) response categories.
Principal factor analysis (PCA, factors with eigenvalue > 1, varimax rotation) was applied to examine the number and type of domains in the instrument . We determined the Kaiser-Meyer-Olkin Measure of sampling adequacy and the Bartlett's test of sphericity. Internal consistency (the extent to which items measure the same concept) was expressed in terms of Cronbach's alpha for each of five domains in PACIC and for the total PACIC instrument. Reliability was expressed as an intra class coefficient (ICC, absolute agreement), which was based on variation between patients divided by total variation (taking patients random and items fixed). Values > 0.70 for alpha and ICC were considered acceptable .
The analysis of construct validity was based on the following hypotheses. We expected that higher PACIC scores, reflecting patient perceived presence of structured chronic care, would be positively related to both patients' perceived enablement after the latest visit to the GP and to patients' overall evaluations of general practice. To verify this expectation, we used linear regression analysis  with PACIC scores as dependent factor, enablement or evaluation as independent factor, and patient age and gender also included in the model. All data-analysis was done with SPSS 14.
Patient characteristics (n = 165)
(n = 88)
(n = 77)
Mean age (SD)
Percentage with medium or high education
Percentage with good to excellent health status
Mean sum score on patient enablement (PEI)
Percentage of patients who evaluated clinical performance as 'excellent' (Europep)
Descriptive information on PACIC items
Item non-response in total study population
(n = 165)
Floor and ceiling effects in total study population
(n = 165)
% of responders reporting mostly/always present
% of responders in lowest response category (absence)
% of responders in highest response category (presence)
(n = 88)
(n = 77)
1 Asked about my ideas when we made a treatment plan
2 Given choices about treatment to think about
3 Asked to talk about any problems with my medicines or their effects
Delivery system/practice design
4 Given a written list of things I should do to improve my health
5 Satisfied that my care was well organized
6 Shown how what I did to talke care of my illness influenced my condition
7 Asked to talk about my goals in caring for my illness
8 Helped to set specific goals to improve my eating or exercise
9 Given a copy of my treatment plan
10 Encouraged to go to a specific group or class to help me cope with my chronic illness
11 Asked questions, either directly or on a survey, about my health habits
12 Sure that my doctor or nurse thought about my values and my traditions when they recommended treatments to me
13 Help to make a treatment plan that I could do in my daily life
14 Helped to plan ahead so I could take care of my illness even in hard times
15 Asked how my chronic illness affects my life
16 Contacted after a visit to see how things were going
17 Encouraged to attend programs in the community that could help me
18 Referred to a dietician, health educator, or counselor
19 Told how my visits with other types of doctors, like consultant or surgeon, helped my treatment
20 Asked how my visits with other doctors were going
Factor loadings in rotated factor solution
1 Asked about ideas
2 Given choices
3 Talk about problems
4 Given written list
5 Care well organized
6 Influence my condition
7 Talk about goals
8 Set goals
9 Given treatment plan
10 Go to group or class
11 Questions health habits
12 Values and traditions
13 Could do in daily life
14 Helped to plan ahead
15 How illness affects life
16 Contact after visit
17 Attend programmes
19 Other doctors
20 Asked about visits
Information on the PACIC domains and overall PACIC score
Number of items in the domain
Available sample for data-analysis
Patients with missing scores on more than one third of the items
(% of total number of patients)
Mean value (SD) for all responders
Mean value (SD) for diabetes patients
Mean value (SD) for COPD patients
Effect of aggregated Europep score (b coefficient, p-value)
(p < 0.004)
(p < 0.003)
(p < 0.002)
(p < 0.064)
(p < 0.011)
(p < 0.009)
Effect of aggregated enablement score (b coefficient, SD)
(p < 0.000)
(p < 0.801)
(p < 0.000)
(p < 0.001)
(p < 0.000)
(p < 0.030)
Cronbach's alpha for internal consistency
Intra class coefficient (absolute agreement)
This study showed that the Dutch version of the PACIC instrument had mixed measurement properties when applied for assessing diabetes care and COPD care in general practice in a rural setting. The five previously defined domains were confirmed and their internal consistency was good. The correlation with patient evaluations of general practice was positive, and diabetes patients reported higher presence of structured chronic care than COPD patients as expected. However, substantial numbers of patients did not provide answers to the PACIC questions, although they returned the questionnaires and completed other parts of the questionnaire reasonably well. Also, we found that a number of items might have floor or ceiling effects. A surprising finding was that better scores for chronic care were linked to lower patient reported enablement after the latest consultation in general practice.
The mean scores on the PACIC domains and total instrument were similar to those found in diabetes patients in the USA , but higher than those found in patients with osteoarthritis in Germany . The PACIC scores for diabetes patients in The Netherlands may be explained by the attention for enhancing structured diabetes care in recent years. For instance, there is no such attention for osteoarthritis, so we would expect similar scores for this condition compared to scores found in Germany. Despite the differences, our findings regarding measurement properties were similar to those found in Germany .
Obviously, the study had a number of limitations. The patient sample was relatively small, and only four general practices from a rural setting were involved, but it was not our aim to generalize the descriptive figures. It is difficult to speculate on how the validation results could be affected by the rural setting. Criterion validity, test-retest reproducibility and responsiveness to change could not be analysed. A substantial proportion of the patients used the lowest answering category, which may indicate a floor effect of the measure (inability to discriminate between patients). We suggest, however, that the scores might perfectly reflect reality – a complete absence of specific aspects of structured chronic care. The high number of non-responders was worrying. An explanation for the non-response may be a perceived lack of relevance of the aspects covered by the items. Some of the aspects covered in the PACIC instrument may be unknown or not relevant to many chronic patients in general practice in The Netherlands. A second explanation may be that the non-response actually implies absence of the aspects mentioned in the PACIC questionnaire – but we think we cannot be certain about such inferences. A final explanation for this is translation problems. A direct translation of the English questions into the Dutch language did not result in understandable language, so we had to rephrase the items quite substantially. Despite this, the final questionnaire might have remained too difficult for many patients.
We can only speculate about the (weak, but 5 out of 6 times highly significant) negative association between patient enablement with the latest visit in the practice and PACIC scores. Perhaps patients with a stronger internal health locus of control and better self-management 'ask' less for enablement, so that they do not need help in general practice regarding the aspects covered by PACIC. Enablement and structured chronic care (including patient activation) may be fundamentally different concepts, as opposed to our expectations beforehand. The finding might also suggest that structured chronic care could have some negative consequences, despite its intention to enhance self-management in patients.
A validated Dutch version of the PACIC instrument is now available. Further research into its validity is recommended, particularly with respect to the high number of non-responders and the counterintuitive finding regarding patient enablement. Also, the questionnaire needs to be tested in other settings than primary care, before using it in those settings.
We thank the participating patients and practices.
- Wagner EH, Austin BT, Von Korff M: Organizing care for patients with chronic illness. Milb Quat. 1996, 34: 511-544. 10.2307/3350391.View ArticleGoogle Scholar
- Bonomi AE, Wagner EH, Glasgow RE, Von Korff M: Assessment of Chronic Illness Care (ACIC): a practical tool to measure quality improvement. Health Serv Res. 2002, 37 (3): 791-820. 10.1111/1475-6773.00049.View ArticlePubMedPubMed CentralGoogle Scholar
- Glasgow RE, Nelson CC, Whitesides H, King DK: Use of the Patient Assessment of Chronic Illness Care (PACIC) with diabetic patients: relationship to patient characteristics, receipt of care, and self-management. Diabetes Care. 2005, 28 (11): 2655-61. 10.2337/diacare.28.11.2655.View ArticlePubMedGoogle Scholar
- Rosemann T, Laux G, Droesemeyer S, Gensichen J, Szecsenyi J: Evaluation of a culturally adapted German version of the Patient Assessment of Chronic Illness Care (PACIC 5A) questionnaire in a sample of osteoarthritis patients. J Eval Clin Pract. 2007, 13: 806-813. 10.1111/j.1365-2753.2007.00786.x.View ArticlePubMedGoogle Scholar
- Howie JG, Heaney DJ, Maxwell M, Walker JJ: A comparison of a Patient Enablement Instrument (PEI) against two established satisfaction scales as an outcome measure of primary care consultations. Fam Pract. 1998, 15: 165-171. 10.1093/fampra/15.2.165.View ArticlePubMedGoogle Scholar
- Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, Hjortdahl P, Olesen F, Reis S, Ribacke M, Szecsenyi J: Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract. 2000, 50: 882-887.PubMedPubMed CentralGoogle Scholar
- Terwee CB, Bot SDM, De Boer MR, Windt Van der DAWM, Knol DL, Dekker J, Bouter LM, De Vet HCW: Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007, 60: 34-42. 10.1016/j.jclinepi.2006.03.012.View ArticlePubMedGoogle Scholar
- Kim JO, Mueller CW: Factor analysis. Statistical methods and practical issues. 1976, Beverley Hills and London: SageGoogle Scholar
- Berry WD, Feldman S: Multiple regression in practice. 1985, Beverley Hills and London: SageView ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/182/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.