Skip to main content
  • Research article
  • Open access
  • Published:

Evaluating patient experiences in decentralised acute care using the Picker Patient Experience Questionnaire; methodological and clinical findings



Decentralised acute care services have, through the establishment of municipality acute wards (MAWs), been launched in Norway. The aim is to provide treatment for patients who otherwise would need hospitalisation. Currently there is a lack of studies investigating patient experiences in such services. The aims of this study were therefore to a) translate and validate the Picker Patient Experience Questionnaire (PPE-15) in Norwegian, and b) assess patient experiences in decentralised acute care, and potential factors associated with these experiences.


Patients were recruited from five municipal acute wards in southeastern Norway during the period from June 2014 to June 2015. Data on socio-demographics, length of stay and comorbidity (Charlson comorbidity index (CCI)) were collected. Patients completed the Picker Patient Experience Questionnaire (PPE-15) and the EuroQOL 5-dimension, 3-level version. Convergent validity of the PPE-15 was assessed by correlation of items in PPE-15 and the Nordic Patient Experience Questionnaire (NORPEQ). A retest of the PPE-15 was performed in a subgroup of patients approximately 3 weeks after baseline assessment. Test-retest agreement was assessed with Cohens’ unweighted Kappa.


A total of 479 patients responded, median age 78.0 years and 41.8% men. A total of 68 patients participated in the retest. Testing of convergent validity revealed an overall weak to moderate correlation. Kappa statistics showed from fair to good test-retest agreement. Most problems were related to continuity and transition, while fewest problems were related to respect for patient preferences. A higher Charlson comorbidity score was the only variable that was negatively associated with patient experience.


Patients reported problems in several items of the PPE-15 after discharge from decentralised acute wards. The findings from the current study may be helpful for planning ways to improve quality of care, e.g., by providing feedback to healthcare personnel or by using patient experience as a quality indicator.

Peer Review reports


The patient experience refers to how the patients, their families and other persons who participate in their care feel about the process and structure of care, as well as the outcomes of care [1]. There is a growing recognition that patients’ perspectives are essential in achieving high quality care [2, 3]. Integrating patients’ perspectives into the evaluation of healthcare delivery is important because they indicate ways to improve care, enhance strategic decision making, meet patients’ expectations, effectively manage and monitor healthcare performance, and document benchmarks for healthcare organisations [4, 5].

Due to the lack of a common definition, the measurement of patient experiences remains challenging [6]. Despite its different meanings, patient experience is often used interchangeably with terms such as patient satisfaction, perceptions or preferences [7, 8]. However, patient experiences are usually considered less subjective than patient satisfaction because patients may be satisfied with healthcare even though they have negative experiences and vice versa [9, 10]. Nevertheless, several factors have been found to influence these experiences, including age, gender, housing and employment status, ethnicity, self-rated health, multimorbidity, and care ward characteristics [11,12,13].

In the Nordic countries, regular assessment of patient experiences has been practiced for a long time [14]. For instance, as a part of the national quality indicators in specialist healthcare services, Norway implemented annual patient experience surveys in all hospitals in 2011 [15]. Although they are more commonly used in hospitals, municipalities are also obliged, according to the Norwegian National Health and Care Service Act, to collect patient experiences and to take these into account when planning and organising primary healthcare services [16].

To respond to future healthcare challenges, a national healthcare reform (the Coordination Reform (CR)) has been gradually implemented in Norway from 2012 to 2016 [16, 17]. One of the main objectives of the reform was to increase the total proportion of patients who accessed health services within their local community. As a direct consequence of the CR, all Norwegian municipalities were legislated to offer a 24-h acute healthcare service beginning in 2016 (Municipal Acute Wards, MAWs). Eligible patients are those who would be normally admitted to a hospital for a condition that can be managed in a general practice setting and within a timeframe of 72 h [18]. Throughout Norway, MAWs are organised differently, some being located in nursing homes, some in “houses of health, in local medical centres in relation to a casualty or a hospital. Some MAWs have employed their own doctors dedicated to the service on a 24-h basis, while other places have employed doctors only during daytime. All of the MAWs have daily doctors’ visits on weekdays. Moreover, the MAWs differ in terms of number of beds and services offered [19].

Information about patient experiences in primary healthcare, particularly in decentralised acute healthcare, are either limited or lacking [20,21,22,23]. Only two qualitative papers describing patients’ perspectives on MAWs have been published to date, finding for example that patients view MAWs as “almost a hospital”, but at the same time different with regard to person centeredness and diagnostic opportunities [24, 25]. Furthermore, the relative influence of socio-demographic variables, length of stay, self-rated health and comorbidity on these experiences remains unknown. Such knowledge could be useful when planning and improving alternatives to hospital treatment, such as MAWs.

The aims of this study were: a) to translate and validate the Picker Patient Experience Questionnaire (PPE-15) in Norwegian, and b) to assess patient experiences in decentralised acute care, and potential factors associated with these experiences.


Setting and participants

Østfold County, located in southeastern Norway, consists of approximately 300,000 inhabitants. The county is divided into 17 municipalities that belong to the same hospital catchment area. In addition to the hospital, a total of five MAWs have been established in different geographical locations throughout the county. The MAWs consist of four to eleven beds, dependent on the population size of their catchment area. Some laboratory services are offered in all of these facilities, while x-ray is accessible in three only.

Participants were recruited from all of the five MAWs during the period from June 2014 – June 2015 using a purposive, total population sampling method; all patients ≥18 years who had stayed in the MAW for a minimum 24 h and who were discharged alive were invited to participate.

Data collection

Socio-demographic variables were self-reported by the patients and included the following: gender, age, civil status (married, single, widow/widower, in a relationship), housing status (living alone or not), educational level (compulsory school, upper secondary school or university), and employment status (still working or not). Information concerning the length of stay was collected from medical records.

Comorbidity was collected using the Charlson comorbidity index (CCI) [26]. The risk of death associated with each of 19 predefined diseases included in the CCI, is expressed as weights with values of 1, 2, 3, or 6. Summing the weights for all contributing diseases gives the CCI score for each patient. The CCI is calculated based on codes in the International Classification of Diseases (ICD-10). Information about the patients’ comorbid conditions as ICD-10 codes was collected from the National Patient Registry (NPR), which includes data on all patients treated in Norwegian government-funded hospitals.


Patient Experience was measured using the Picker Patient Experience Questionnaire (PPE-15) [27]. The final choice of questionnaire was a result of several discussions between the researchers and collaborating physicians in the community over a one-year period. Even though other translated and validated instruments existed, they were viewed either as too extensive or too short (not covering all areas of interest). The PPE-15 was developed to elicit feedback from patients to highlight aspects of care that needed improvement and to monitor performance and care. It consists of 15 questions distributed to seven dimensions of care: respect, coordination, information/communication/education, physical comfort, emotional support, involvement of relatives, and transitions and continuity [27, 28] (Appendix). The questions have two (“yes” or “no”) to four response options (“yes”,” no”, “I did not need to”, or “yes, to some extent”). Neutral answers, such as “I did not need to”, and the most positive answer are coded as a “non-problem” (score = 0). The remaining responses are coded as “problems” (score = 1). The PPE-15 has previously been found to be valid and reliable [28].

The PPE-15 has not been translated into Norwegian, and forwards and backwards translation were consequently performed according to recommendations in the literature [29, 30]: Two professional bilingual translators with Norwegian as their mother tongue performed two independent translations into Norwegian. After comparing the translations and synthesizing these into one, the questionnaire subsequently underwent a backward translation to English by a translator with English as her mother tongue. Finally, three independent individuals evaluated the questionnaire by comparing the English and Norwegian versions with regard to semantic, idiomatic, experiential, and conceptual equivalence. Following this procedure and prior to statistical testing, the Norwegian PPE-15 underwent testing of face validity. This was done by distributing the questionnaire to 10 patients prior to the study period in order to assess the adequacy, appropriateness and understandability of the questionnaire, including language and scoring instructions [31]. Patient feedback did not reveal any problematic issues in any of these aspects. Following these procedures, a final version of the PPE-15 was approved and tested.

To assess patients’ self-reported health, we used the EuroQOL 5-dimension, 3-level version (EQ-5D-3 L) [32, 33]. The EQ-5D-3 L consists of the EQ-5D descriptive system that measures health-related quality of life on five dimensions: mobility, self-care, usual activities, pain/discomfort, anxiety/depression, and the EQ visual analogue scale (EQ VAS). Responses are scored according to three levels: 0 (no problem), 1 (some problems) to 2 (severe problems). The EQ-5D-3 L score was used as an overall EQ-5D-3 L index score by assigning weights to each level of each dimension according to the Europe VAS value set [32, 34].


The following standardised inclusion procedure was used: 1) Prior to discharge, the study nurses provided the patients with oral information about the purpose of the study. 2) At discharge, the patients received written information, the study questionnaires and a consent form. 3) The patients were asked to complete these forms at home and were instructed to return the completed questionnaires, along with the signed informed consent form, in a pre-stamped envelope. Non-responders were reminded once by a phone call from the first author approximately 2 weeks after discharge.

Statistical analysis

Summative statistics were used to present characteristics of the sample. Because the data (age, length of stay, comorbidity and self-rated health) were not normally distributed, the continuous variables are displayed as the median, mean and standard deviation. Because no method for calculating missing items exists for the PPE-15, and based on recommendations from statistical expertise at the Picker Institute (personal communication – available from the first author upon request), missing items were not included in the analysis. A Mann-Whitney U test was used to evaluate differences between responders and non-responders.

To assess convergent validity, Spearman’s correlation between of the PPE-15 items and items in the previously validated Nordic Patient Experience Questionnaire (NORPEQ) questionnaire was used. The NORPEQ consists of six questions that cover important aspects of healthcare encounters and is scored from the worst experience (0) to the best experience (100) [35]. We hypothesised that a generally moderate correlation would be found between the two questionnaires because the PPE-15 and NORPEQ do not capture exactly comparable aspects of the patient experience. We also hypothesised that the second item of the NORPEQ would have an overall low correlation with the PPE-15, since none of the items in the PPE-15 measure doctors’ professional skills.

Internal consistency of the PPE-15 was assessed by Cronbach’s alpha. In order to measure test-retest reliability of the PPE-15, a subgroup of responders was invited to fill out the PPE-15 a second time, approximately 3 weeks after the first completion. Invitation to participate in retest was sent consecutively as completed questionnaires were received. The literature suggests that only patients in a stable condition should be included in the re-test, since responses may be influenced by a change in health status [36]. Hence, patients were also asked to indicate whether their condition was unchanged, had deteriorated or improved. Test- retest was assessed using the Cohens’ unweighted Kappa statistics. Kappa less than 0.2 is defined as ‘poor agreement’, 0.2–0.4 as ‘fair agreement’, 0.4 to 0.6 as ‘moderate agreement’, 0.6 to 0.8 as ‘good agreement’, and a kappa =0.8 to 1.0 as ‘very good agreement’ [37].

The proportion of the PPE-15 items scored as a ‘problem’ (dependent variable, in long format) was estimated by a binomial linear mixed model [38,39,40] that used socio-demographic variables, length of stay, the Charlson score and the EQ-5D index score as covariates (independent variables). Some of the variation in the patients’ scores can be attributed to individual experiences as well as to aspects of the different locations (e.g., the staffing situation and the type of services they offer) [39]. The care wards and patients (identity) were consequently included as random effects to account for the inhomogeneity among patients and inhomogeneity among wards. Insignificant variables were removed from the model one at a time until only significant effects remained. The ICC was calculated to explore the proportion of random variation.

All tests were two-sided, used a 95% confidence interval (CI) and used a significance level of 0.05. All analyses were performed using Statistical Package for the Social Sciences (SPSS) version 21 [41].


During the inclusion period, 1235 (56.6%) patients, out of the 2182 patients who were admitted to the five MAWs, received a questionnaire. The proportion of discharged patients who received a questionnaire varied from 36.9% to 68.9% in the five MAWS. A total of 479 patients (38.8%) returned the questionnaires. Table 1 presents an overview of the responders’ socio-demographic characteristics, length of stay, Charlson comorbidity score and self-rated health.

Table 1 Study responders’ socio-demographic characteristics, length of stay, Charleson co-morbidity score and self-rated health

For non-responders, mean age was 78.1 years (median = 83, SD 15.2), 36.2% male. Compared with responders, non-responders were older (p < 0.001) and fewer were male (p < 0.001).

PPE-15, validity and reliability

The correlation between the NORPEQ and the PPE-15 items varied between −0.46 and 0.43 (Table 2). The internal consistency of the PPE-15 as assessed by Cronbach’s alpha was 0.83.

Table 2 Correlations (Spearman Rho) between the items in the NORPEQ and the PPE-15 (n = 68)a

Cohens’ unweighted Kappa statistics varied from 0.27 (fair agreement) - 0.7 (good agreement), except from on PPE-15 item 11, in which the agreement was 0.13 (poor) (Table 3).

Table 3 Test-retest of the PPE-15 using unweighted Kappa coefficient (n = 68)

Patient experiences and association with background characteristics

The number of respondents who reported each PPE-15 item as a problem and the item’s corresponding dimension are presented in Table 4. Fourteen of the respondents did not report any problems.

Table 4 Proportion of responders reporting problems on the items of Picker Patient Experience Questionnaire (PPE-15)

The largest proportion of problems was observed in the continuity and transition dimension of the PPE-15, particularly the item related to information about dangerous signals to watch for at home (73.3%). The smallest proportion of problems were observed for the item related to being treated with respect and dignity, and the item asking whether doctors talked in front of them as if they were not there (8.8%, respectively). Furthermore, 42.1% of patients reported problems related to their involvement in treatment and care. While 25.5% of patients reported problems related to whether explanations about the purpose of medicines were understandable, and 51.2% reported problems related to information about medication side effects.

The random effects analysis revealed a negligible variation among wards (ICC < 0.001), whereas the random variation between patients within wards contributed to 21% of the total random variation. In the binominal linear mixed model only the Charlson comorbidity score was statistically significantly associated with decreased patient experience, while factors such as gender, age, self-rated health, length of stay, educational background, employment status, housing status and civil status were not associated with patient experiences (Table 5).

Table 5 Results from the binomial linear mixed model of the PPE-15, using care wards (n = 5) and patients (n = 479) as random effects


Findings indicate that the Norwegian PPE-15 is a valid and reliable patient experience questionnaire. Furthermore, among patients admitted to MAWs, the highest proportion of problems were related to aspects of ‘continuity and transition’. Increased comorbidity was the only factor that was significantly negatively associated with patients’ experiences in this study.

PPE-15, validity and reliability

The PPE-15 has primarily been tested in hospitalised patients [28]. The dimensional structure is based on input from patients during it’s development and not on any statistical tests, such as e.g., factor analysis [27]. The latter might be because no dimensional score exist. Consequently, the dimensions reported only reflect what items are perceived by patients to be semantically related. Similar testing have been performed in Sweden [42]. As a consequence of the procedures used in prior studies and scoring instructions, a factor analysis was not used in the current study. However, during the assessment of face validity, patients did not report any specific issues related to neither the content of PPE-15 nor the items corresponding dimensions.

Testing of convergent validity revealed an overall weak to moderate correlation between the NORPEQ and the PPE-15 items. This may be because the specific content of each item differs between these two questionnaires. However, correlations were generally higher in those items hypothesised to be more closely associated, and lower in those not hypothesised as associated. This may indicate that the PPE-15 has an acceptable convergent and discriminant validity.

The Kappa values varied from fair to good agreement in the vast majority of items. In item 11, concerning whether family and close ones had the possibility to talk to a doctor if they were in need or wanted to, the agreement was poor. The potential reason for this finding is unclear and future studies using the PPE-15 should explore this more closely.

Patient experiences

Our results show that the vast majority of patients reported that they were treated with respect and dignity, which corresponds with findings from a qualitative study [25]. Nevertheless, many wanted greater involvement in decisions about their care and treatment. Although this finding is in line with a study of 34,000 hospitalised patients in Sweden [43], our results seem to contrast prior reports that have observed a positive association between respect/dignity and involvement in care [44]. Patient-centred care has been described as a partnership between patients and healthcare professionals to inform and involve patients in shared decision making [45]. Outcomes have been suggested to include patients’ feeling of respect, involvement, engagement and knowledge [46].

Patients in the current study also experienced problems related to continuity and transition. Prior studies have shown that poor communication, incomplete transfer of information, and inadequate education of the patients may have a negative impact on care transitions, leading to unplanned readmissions and adverse events [47, 48]. In particular, problems were reported concerning information about potential medication side effects to observe at home. Drug-related problems (DLPs) occur quite frequently after discharge, and factors such as a short stay and inadequate communication may influence the figure negatively [49]. Studies have however found that providing elderly patients with medication reports may reduce this amount [49, 50]. Interestingly, 24–24.8% of patients in our study reported problems related to receiving understandable answers to questions from either nurses or doctors, while 31.2% reported that they had received different answers to questions from different personnel. Studies have found that significant differences exist between what physicians think patients know and what patients actually know [51]. Because we do not have any specific information on how the discharge process was handled at each treatment location, it is beyond the scope of this paper to draw any firm conclusions in that regard. Nevertheless, the variation among care wards was negligible in the current study. However, given the importance of discharges and handovers, it seems important to target quality of patient and healthcare personnel communication to improve how patients experience their healthcare encounter [52].

The results from our study show that the random effect attributed to the ward to which patients were admitted had very little influence on patient experiences. This may be due to a relatively small sample size, or that participants were included from one county were the MAW routines were quite homogenous, due to joint collaborative efforts. Interestingly, prior studies have argued that for most quality aspects, including the ward into analyses of patient experiences is important for a number of quality indicators [53].

Factors influencing patient experiences

Our findings show that age, gender, civil status, employment status, educational status, housing status, self-rated health and length of stay had no significant effect on the number of problems. This is in contrast to earlier research reporting that increased age and decreased self-reported health status are the strongest predictors of negative patient experiences [54]. Additionally, one study found that at least 79% of the variance of all patient experience measures occurred at the patient level [9]. In a study of 34,000 Swedish hospitalised patients, poorer experiences were associated with greater healthcare utilisation, higher age, functional impairment and female gender. Previous studies have also suggested a positive association between older age and higher care satisfaction [54, 55]. However, studies on primary care services have also been inconsistent regarding the relative influence of health and socio-economic status, age, gender, ethnicity and self-rated health on patient reported experiences [56, 57]. It might be speculated if our findings occurred because the PPE-15 uses a dichotomised score.

A greater number of comorbid conditions was weakly, but significantly negatively associated with patient experiences. Comorbidity is associated with worse health outcomes, more complex clinical management, and increased health care costs [58]. In older adults, comorbidity is one of three factors (along with frailty and disability) that are commonly used to indicate vulnerability [59]. Vulnerable patients have also reported fewer positive care experiences than non-vulnerable patients in prior studies [43]. Healthcare professionals may view vulnerable patients as individuals who have more complex healthcare needs; thus, they may focus on meeting these needs rather than focusing on patient satisfaction or patient experiences [43].


Increasing time since discharge seems to result in poorer patient experiences scores, and patients who self-report their experiences at home following discharge may consequently have an increased risk of recall bias [60]. While collecting data before discharge from MAWs could have increased the number of respondents and reduced the risk of recall bias, the responses could also have been influenced by potential interruptions and influence of the personnel [61].

Sadly, not all patients who were discharged from the five MAWs were invited to participate, as planned. This could indicate a potential selection bias. Based on input from the various MAWs and study follow-up, there did not seem to be any pattern regarding who was invited and who was not. The reasons seemed to be lack of time due to many tasks, and forgetfulness. Another drawback is that we did not collect information on those who did not receive a questionnaire, which of course makes it impossible to assess any differences between those who were invited and those who were not. Moreover, the response rate is relatively low. It is therefore difficult to know if those not responding had worse experiences, which has been reported in the literature [62]. Comparison of non-responders and responders furthermore revealed significant differences in gender and age.

In retrospect, we could have invited a larger number of patients in the retest, yet the number of patients needed in these tests have been subject of debate. For instance some have advocated that a sample size of 50 could be sufficient or a starting point, while others have highlighted the need for larger sample sizes and more robust test-retest data [63, 64]. Indeed, a systematic review found that the median number included in retest analysis was 60 [36].

Patients may judge an organisation in light of personal expectations, and this may not adequately reflect actual care quality. Pre-existing higher expectations of care quality have been linked to increased critical evaluation of a healthcare service [65]. Our study did not include questions about patient expectations, and consequently, we cannot conclude whether worse experiences reflect suboptimal care or different expectations [66]. However, because MAWs are newly established health services, patients may not adequately know what to expect when they are treated in these units [24].


The PPE-15 displayed acceptable validity and reliability. While patients reported problems in all of the PPE-15 items, this was particularly evident in aspects related to discharge information. These findings may be helpful in planning ways to improve quality of care, e.g., by providing feedback to healthcare personnel, or by using patient experience as a quality indicator. Findings may also be important when developing new decentralised acute health care services.



Charleson comorbidity index score


EuroQol 5-dimension-3 level version index score


Municipality acute ward


Picker Patient Experience Questionnaire


  1. NHS Institute for Innovation and Improvement. The patient experience book. A collection of the NHS Institute for Innovation and Improvement guidance and support. Coventry: University of Warwick Science Park; 2013.

  2. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Rathert C, Brandt J, Williams ES. Putting the ‘patient’ in patient safety: a qualitative study of consumer experiences. Health Expect. 2012;15(3):327–36.

    Article  PubMed  Google Scholar 

  4. Bjertnaes O, Sjetne I, Iversen H. Overall patient satisfaction with hospitals: effects of patient-reported experiences and fulfilment of expectations. BMJ Qual Saf. 2012;21(1):39–46.

    Article  PubMed  Google Scholar 

  5. Garratt A. National and cross-national surveys of patient experiences. Oslo: Nasjonalt kunnskapssenter for helsetjenesten [Norwegian Knowledge Centre for the Health Services]; 2008.

    Google Scholar 

  6. LaVela S, Gallan A. Evaluation and measurement of patient experience. Pat Exp J. 2014;1(1):28–36.

    Google Scholar 

  7. Luxford K. What does the patient know about quality? Int J Qual Health Care. 2012;24:439–40.

    Article  PubMed  Google Scholar 

  8. Ahmed F, Burt J, Roland M. Measuring patient experience: concepts and methods. Patient. 2014;7(3):235–41.

    Article  PubMed  Google Scholar 

  9. Salisbury C, Wallace M, Montgomery A. Patients’experience and satisfaction in primary care: a secondary analysis using multilevel modelling. BMJ. 2010;341:5004.

    Article  Google Scholar 

  10. Grøndahl V, Wilde-Larsson B, Hall-Lord M, Karlsson I. A pattern approach to analysing patients' satisfaction and quality of care perceptions in hospital. Int J Pers Cent Med. 2011;1(4):766–75.

    Google Scholar 

  11. Johnson M, Rodriguez H, Solorio M. Case-mix adjustment and the comparison of community health center performance on patient experience measures. Health Serv Res. 2010;45:670–90.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Paddison C, Saunders C, Abel G, Payne R, Campbell J, Roland M. Why do patients with multimorbidity in England report worse experiences in primary care? Evidence from the general practice patient survey. BMJ Open. 2015;5(3):e006172.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Grøndahl VA, Karlsson I, Hall-Lord ML, Appelgren J, Wilde-Larsson B. Quality of care from patients’ perspective: impact of the combination of person-related and external objective care conditions. J Clin Nurs. 2011;20(17–18):2540–51.

    Article  PubMed  Google Scholar 

  14. Skudal K, Garratt A, Eriksson B, Leinonen T, Simonsen J, Bjertnaes O. The Nordic patient experiences questionnaire (NORPEQ): cross-national comparison of data quality, internal consistency and validity in four Nordic countries. BMJ Open. 2012;2(3):e000864. doi:10.1136/bmjopen-2012-000864.

  15. Wiig S, Storm M, Aase K, Gjestsen MT, Solheim M, Harthug S, et al. Investigating the use of patient involvement and patient experience in quality improvement in Norway: rethoric or reality? BMC Health Serv Res. 2013;13:206.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Norwegian department of health and care. Nasjonal helse- og omsorgsplan [national health and care plan] Oslo: 2010-11. Report No. 16.

  17. Norwegian department of health and care. Samhandlingsreformen. Rett behandling-til rett tid-på rett sted [ The coordination reform. Proper treatment in right time and place]. Report No. 47. Available from: Accessed 3 Feb 2016.

  18. Norsk forening for allmennmedisin [Norwegian Association for General Practice]. Medisinskfaglig veileder for kommunale akutte døgnplasser [ medical guidelines for municipal acute wards]. Oslo: Den norske legeforening [the Norwegian Medical Association]; 2014.

    Google Scholar 

  19. Skinner M. Øyeblikkelig hjelp døgnopphold. Oppsummering av kunnskap og erfaringer fra de første fire årene med kommunalt øyeblikkelig hjelp døgnopphold [acute care wards. Summarisation of knowledge and experiences from the four first years of municipal acute care units]. Gjøvik: Høgskolen i Gjøvik; 2015.

  20. Young J. The development of intermediate care services in England. Arch Gerontol Geriatr. 2009;49(Suppl 2):S21–5.

    Article  PubMed  Google Scholar 

  21. Primary Care Foundation. Urgent care centres: what works best? A discussion paper. 2012.

    Google Scholar 

  22. Li P, Schneider JE, Ward MM. Effect of critical access hospital conversion on patient safety. Health Serv Res. 2015; 42(6):2089–108.

    Google Scholar 

  23. Martinsen B, Norlyk A, Lomborg K. Experiences of intermediate care among older people: a phenomenological study. Br J Comm Nurs. 2015;20(2):74–9.

    Article  Google Scholar 

  24. Leonardsen A, del Busso L, Grøndahl V, Ghanima W, Barach P, Jelsness-Jørgensen L. A qualitative study of patient experiences with decentralised acute healthcare services. Scand J Prim Health Care 2016; 34(3): 317-324.

  25. Leonardsen A, Del Busso L, Grøndahl V, Jelsness-Jørgensen L. “It’s a whole human being”: a qualitative study of care experiences among patients treated in decentralised healthcare services. EJPCH. 2017;5(1):82–7.

    Article  Google Scholar 

  26. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40(5):373–83.

    Article  CAS  PubMed  Google Scholar 

  27. Jenkinson C, Coulter A, Bruster S. The picker patient experience questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care. 2002;14(5):353–8.

    Article  PubMed  Google Scholar 

  28. Jenkinson C, Coulter A, Reeves R, Bruster S, Richards N. Properties of the picker patient experience questionnaire in a randomized controlled trial of long versus short form survey instruments. J Pub Health Med. 2003;25(3):197–201.

    Article  Google Scholar 

  29. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005; 8(294–104).

  30. Mokkink L, Terwee C, Patrick D, Alonso J, Stratford P, Knol D, et al. The COSMIN checklist for assessing the methodological quality of studies in measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Polit D, Beck C. Nursing research principles and methods. 9th ed. Baltimore: Lippincott Williams & Wilkins; 2012.

    Google Scholar 

  32. EuroQol Group. EuroQol- a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208.

    Article  Google Scholar 

  33. Esser A, Gube M, Schettgen T, Kraus T, Lang J. QUALY as evaulation in a health surveillance program. Available from: Accessed 5 Nov 2015.

  34. Rabin R, de Charro F. EQ-5D-SD: a measure of health status from the Euroqol group. Ann Med. 2001;33(5):337–43.

    Article  CAS  PubMed  Google Scholar 

  35. Oltedal S, Garratt A, Bjertnaes O, Bjornsdottir M, Freil M, Sachs M. The NORPEQ patient experiences questionnaire: data quality, internal consistency and validity following a Norwegian inpatient survey. Scand J Pub Health. 2007;35(5):540–7.

    Article  Google Scholar 

  36. Paiva C, Barroso E, Carneseca E, de Padua SC, dos Santos F, Lopez R, et al. A critical analysis of test-retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review. BMC Med Res Methodol. 2014;14(8).

  37. McHugh M. Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276–82.

    Article  Google Scholar 

  38. Nelder J, Wedderburn R. Generalized Linear Models. J Royal Stat Soc. 1972;135(3):370–84.

    Article  Google Scholar 

  39. Roberts M, Campbell J, Abel G, Davey A, Elmore N, Maramba I, et al. Understanding high and low patient experience scores in primary care: analysis of patients’ survey data for general practices and individual doctors. BMJ. 2014;349:g6034.

    Article  PubMed  PubMed Central  Google Scholar 

  40. McCulloch C, Searle S. Generalized, linear and mixed models. New York: Wiley; 2001.

    Google Scholar 

  41. IBM Corporation. IBM SPSS statistics for windows, version 21.0. New York: Armonk; 2012.

    Google Scholar 

  42. Höglund A. Vad tycker patienten om vården? Delrapport från en provstudie med picker-metoden [what do patients think about their care? Part report from a test study with the picker method]. Lund: Sociologiska instituten; 1998.

  43. Wolf A, Olsson L, Taft C, Sweberg K, Ekman I. Impacts of patient characteristics on hospital care experience in 34,000 Swedish patients. BMC Nurs. 2012;11(8).

  44. Parlour R, Slater P, Mccormack B, Gallen A, Kavanagh P. The relationship between positive patient experience in acute hospitals and person-centred care. Int J Res Nurs. 2014;5(1):25–34.

    Google Scholar 

  45. O’Leary K, Killarney A, Hansen L, Jones S, Malladi M, Marks K, et al. Effect of patient-centred bedside rounds on hospitalised patients’ decision control, activation and satisfaction with care. BMJ Qual Saf. 2015;25(12):921–8.

    Article  PubMed  Google Scholar 

  46. Gerteis M, Edgman-Levitan S, Daley J, Delbanco T. Through the patient's eyes: understanding and promoting patient -centered care. San Fransisco: Jossey-Bass; 1993.

    Google Scholar 

  47. Naylor M. Nursing intervention research and quality of care: influencing the future of healthcare. Nurs Res. 2003;52(6):380–5.

    Article  PubMed  Google Scholar 

  48. Apel E, OḾalley T, Greenwald J. Hospital discharge and readmission. 2016. Available from: Accessed 6 June 2016.

  49. Garcia-Caballos M, Ramos-Diaz F, Jimenez-Moleon JJ, Bueno-Cavanillas A. Drug-related problems in older people after hospital discharge and interventions to reduce them. Age Ageing. 2010;39(4):430–8.

    Article  PubMed  Google Scholar 

  50. Midlov P, Holmdahl L, Eriksson T, Bergkvist A, Ljungberg B, Widner H, et al. Medication report reduces number of medication errors when elderly patients are discharged from hospital. PWS. 2008;30(1):92–8.

    PubMed  Google Scholar 

  51. Olson DP, Windish DM. Communication discrepancies between physicians and hospitalized patients. Arch Intern Med. 2010;170(15):1302–7.

    Article  PubMed  Google Scholar 

  52. Siemsen I, Madsen M, Pedersen L, Michaelsen L, Pedersen A, Andersen H, et al. Factors that impact on the safety of patient handovers: an interview study. Scand J Publ Health. 2012;40:439–48.

    Article  Google Scholar 

  53. Krol MW, DeBoer D, Sixma H, Van Der Hoek L, Rademakers JJ, Delnoij DM. Patient experiences of inhospital care: a department matter and a hospital matter. Int J Qual Health Care. 2015;27(1):17–25.

    Article  PubMed  Google Scholar 

  54. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess. 2002;6:1–244.

    Article  CAS  PubMed  Google Scholar 

  55. Xiao H, Barber J. The effect of perceived health status on patient satisfaction. Value Health. 2008;11(4):719–25.

    Article  PubMed  Google Scholar 

  56. Glynn L, Byrne M, Newell J, Murphy A. The effect of health status on patient's satisfaction with out-of-hours care provided by a family doctor co-operative. Fam Pract. 2004;21(6):677–83.

    Article  CAS  PubMed  Google Scholar 

  57. Danielsen K, Bjertnaes OA, Garratt A, Forland O, Iversen HH, Hunskaar S. The association between demographic factors, user reported experiences and user satisfaction: results from three casualty clinics in Norway. BMC Fam Pract. 2010;11:73-2296-11-73.

    Article  Google Scholar 

  58. Valderas J, Starfield B, Sibbald B, Salisbury C, Roland M. Defining comorbidity: implications for understanding health and health services. Ann Fam Med. 2009;7(4):357–63.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Fried L, Ferrucci L, Darer J, Williamson J, Anderson G. Untangling the concepts of disability, frailty and comorbidity: implications for improved targeting and care. J Gerontol A Biol Sci Med Sci. 2004;59(3):255–63.

    Article  PubMed  Google Scholar 

  60. Bjertnaes O. The association between survey time and patient-reported experiences with hospitals: results of a national postal survey. BMC Med Res Methodol. 2012;12(13).

  61. French K. Methodological considerations in hospital patient opinion surveys. Int J Nurs Stud. 2003;40(5):525–41.

    Article  PubMed  Google Scholar 

  62. Barron D, West E, Reeves R, Hawkes D. It takes patience and persistence to get negative feedback about patients' experiences: a secondary analysis of national inpatient survey. BMC Health Serv Res. 2014;14(153). doi:10.1186/1472-696-14-153.

  63. Stucky B, Periera C, de Vet H, Terwee C, Mokkink L, Knol D. Measurement in medicine: a practical guide. Qual Life Res. 2012;21(2):371–3.

    Article  Google Scholar 

  64. Polit D. Getting serious about test-retest reliability: a critique of retest research and some recommendations. Qual Life Res. 2014;23(6):1713–20.

    Article  PubMed  Google Scholar 

  65. Lyratzopoulos G, Elliott M, Barbiere J, Henderson M, Staetsky L, Paddison C, et al. Understanding ethnic and other socio-demographic differences in patient experience of primary care: evidence from the English general practice patient survey. BMJ Qual Saf. 2012;21:21–9.

    Article  CAS  PubMed  Google Scholar 

  66. Saunders C, Abel G, Lyratzopoulos G. What explains worse patient experience in London? Evidence from secondary analysis of the cancer patient experience survey. BMJ Open. 2016;4. doi:10.1136/bmjopen-293-004039.

Download references


The authors would like to thank all the study participants as well as study nurses and physicians at the five MAWs. MD Per Grunde Weydahl and MD Paul Barack are acknowledged for contributing to the planning of this project.


The study was funded by an unrestricted research grant from the Østfold Hospital Trust.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to local ownership of the data, but are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations



ACLL performed the data collection and scanning of the data. Together with LPJJ and RH did the analysis and interpretation of data. ACLL wrote a first edition of the full manuscript, of which all of the co-authors participated in the drafting and critical revision of the manuscript for important intellectual content, as well as approval of the final submitted version of the manuscript. All co-authors contributed in the conception and planning of the work that led to the manuscript, analysis and interpretation of the data. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Ann-Chatrin Linqvist Leonardsen.

Ethics declarations

Ethics approval and consent to participate

The study was performed according to the principles stated in the Declaration of Helsinki. The study was approved by the Regional Committees for Medical and Health Research Ethics (REC) in Norway (ref. no 2013/1276/REK sør-øst D) and the Norwegian Social Science Data Services (NSD) (ref. no 38585). Written, informed consent was obtained from all of the study participants.

Consent for publication

Not applicable.

Competing interests

All authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.



Table 6 Domains, items and scoring alternatives of the Picker Patient Experience Questionnaire

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Leonardsen, AC.L., Grøndahl, V.A., Ghanima, W. et al. Evaluating patient experiences in decentralised acute care using the Picker Patient Experience Questionnaire; methodological and clinical findings. BMC Health Serv Res 17, 685 (2017).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: