Research article | Open | Open Peer Review | Published:
Parent experiences of diabetes care questionnaire (PEQ-DC): reliability and validity following a national survey in Norway
BMC Health Services Researchvolume 18, Article number: 774 (2018)
Patient experiences are acknowledged as an important aspect of health care quality but no validated instruments have been identified for the measurement of either parent or patient experiences with outpatient paediatric diabetes care. The aim of the current study was to assess the psychometric properties of a new instrument developed to measure parent experiences of paediatric diabetes care at hospital outpatient departments in Norway.
The development of the questionnaire was based on a literature review of existing questionnaires, qualitative interviews with both parents and children/adolescents, expert-group consultations, pretesting of the questionnaire and a pilot study. The national pilot study included parents of 2606 children/adolescents aged 0–17 years with Type 1 Diabetes registered in The Norwegian Childhood Diabetes Registry, a nationwide, population-based registry. Levels of missing data, ceiling effects, factor structure, internal consistency, item discriminant validity and construct validity were assessed.
A total of 2606 patients were included in the survey, but 80 were excluded due to incorrect addresses. 1399 (55%) parents responded to the questionnaire. Low levels of missing or “not applicable” responses were found for 31 of the 35 items (< 10%), and 27 of 35 items were below the ceiling-effect criterion. Psychometric testing and theoretical considerations identified six scales: Consultation (six items), organisation (five items), equipment (three items), nurse contact (four items), doctor contact (four items) and outcome (five items). All six scales met the 0.7 criterion for Cronbach’s alpha (range: 0.71–0.90). As expected, each item had a higher correlation with its hypothesised scale than with any of the other five scales. The construct validity of the Parent Experiences of Diabetes Care Questionnaire (PEQ-DC) was supported by 17 out of 18 associations with variables expected to be related to parent experiences.
The psychometric testing of the PEQ-DC showed good evidence for data quality, internal consistency and construct validity. The instrument includes important aspects of diabetes care at paediatric outpatient departments from the perspective of the parent. The content validity of the PEQ-DC was secured by a rigorous development process, and the instrument was tested following a national survey in Norway, securing generalisability across Norway.
In the last few decades, patient satisfaction and patient experiences have drawn increasing interest and have been acknowledged as a critical component of quality assessment. Consistent associations have been found between patient experiences and clinical effectiveness and patient safety . The importance of incorporating patients’ needs and perspectives into care delivery has been highlighted, but when it comes to children under a certain age, parents often act as a proxy response. The information provided by patients or parents or caregivers can be used to inform quality improvement initiatives and improve services [2, 3].
The prevalence of Type 1 Diabetes (T1D) is increasing worldwide, but the prevalence varies between countries. Norway has one of the highest incidences of childhood onset T1D in the world [4, 5], but recent data show that the rise in incidence may have levelled off . According to calculations from the Norwegian Prescription Database, around 28,000 people (0.6% of the population) have T1D .
High quality of care is essential to avoid acute and long-term complications . In Norway, children have regular follow-up appointments at the local outpatient diabetes clinics every third months. If the treatment goals are not met, or there are other concerns, the consultation frequency is increased. Before designing the content and structure of the questionnaire we asked the different paediatric departments for information on how often the consultations only include a nurse or only a doctor, how often they include both, and how often patients have consultations with other patients (group consultations). 20 of 27 departments responded, and the feedback showed that the organisation of the consultations differed. Some departments always provide consultations with both a paediatrician and a diabetes nurse participating. At other departments the patient always sees the paediatrician alone, but are most of the times met by the diabetes nurse. Other departments have arranged for the patient to see the paediatrician and the diabetes nurse at every second appointment. In one department, due to staffing problems, the patients only meet the paediatrician once a year. Some of the clinics also provides age matched group consultation for patients in addition to individual consultations. Dieticians and psychologists can be consulted if requested. The degree of parental involvement is essential to disease management .
T1D is a demanding chronic disease and caring for a child with T1D has been described as an overwhelming experience, requiring constant vigilance [9, 10]. Type 1 diabetes usually develops in childhood or adolescence. Accordingly, parents play an important role in the day-to-day disease management of both pre-adolescent children and adolescents, and this responsibility places considerable demands on parents [11, 12]. The diabetes management can only be effective if the caregiver or family and the patient are able to implement it. Family involvement is a crucial component of optimal management throughout childhood and adolescence .
Parents of children with T1D prefer health care tailored to the individual needs of parents and child, for instance, to the developmental level of the child and to their history regarding the disease . The need for active emotional and practical support from professionals is the greatest in the period immediately after diagnosis . After this the needs are more related to support in aberrant situations and upon request . Parents want to be guided through the enormous amount of information available on the Internet, information about ongoing trials and study results, but also guidance on how this information can be applied to everyday diabetes management. The accessibility of health care professionals is also emphasised [11, 12], as well as the benefits of continuity of care . Parents of older children want opportunities to speak to health professionals alone because young children could be distracting and/or the parents would prefer not to raise distressing issues in front of their child .
To secure the quality and usefulness of studies of patient or parent experiences, valid and reliable measures are crucial. Instruments have been developed to characterise the transition experiences of emerging adults with T1D . However, a review of the literature identified no available psychometrically sound instruments that addressed parent satisfaction with paediatric diabetes care at outpatient hospital departments. Consequently, a new measure was developed.
The aim of the present study was to assess the data quality, internal consistency reliability and validity of the Parent Experiences of Diabetes Care Questionnaire (PEQ-DC). The PEQ-DC was developed and evaluated in accordance with the standard methodology of the national patient experience survey programme in Norway [16,17,18,19,20,21,22,23,24,25,26,27,28,29], also supported by criteria that have been defined as important in the measurement of patient satisfaction . The questionnaire was designed for application in a national survey of parents or primary caregivers with children/adolescents aged 0–17 years with T1D included in The Norwegian Childhood Diabetes Registry (NCDR) attending paediatric outpatient departments.
Questionnaire development and content
The development of the PEQ-DC was based on a process including several standardised steps carried out to ensure sound psychometric properties. A literature review was conducted to identify relevant questionnaires, but also to explore methodological issues relevant when addressing health care for children and adolescents with diabetes. An expert group was established comprising health personnel, researchers and representatives from patient organisations. We decided against the use of different versions for various age groups to ensure the questionnaire could be used for parents of children and adolescents with T1D of all ages. Semi-structured interviews were undertaken with 14 parents of children/adolescents aged 5 to 17 years who had received care from outpatient clinics. The interviews were undertaken by two senior researchers independently. The content of the first draft of the questionnaire was informed by the review of the literature, the qualitative interviews and consultations and meetings with the expert group. The first draft was pretested by cognitive interviews with 15 parents of children aged 4 to 17 years, and minor changes were made to the questionnaire before the national pilot survey.
The instrument tested in the pilot survey comprised 40 questions. We asked the parents about their experiences with the paediatric outpatient clinic where their child has diabetes controls. The questionnaire was divided into sections addressing questions about reception and waiting, organisation, nursing services, doctor services, the consultation, parent issues, information and training, access to care and usefulness. The questionnaire also included five items concerning sociodemographic characteristics. Five-point scales with response options that ranged from “not at all” (1) to a “very large extent” (5) were used for most items relating to the experience of care. Two items had a 5-point response format ranging from 1 “very dissatisfied” to 5 “very satisfied”. One item had a 3-point response scale ranging from 1 to 3; 1 = “too few”, 2 = “appropriate”, 3 = too many. Most items included a “not applicable” option. An open-ended question on the last page probed further comments on experiences with the outpatient clinic or feedback on the questionnaire. An English version of the questionnaire is available in a recent publication from this study .
The national survey was conducted by the Norwegian Institute of Public Health (NIPH), commissioned by the NCDR, a nationwide, population-based quality registry.
The questionnaire was tested in a national pilot survey including 2606 parents of children and adolescents aged 0–17 years with T1D registered in the NCDR and with at least one outpatient consultation the previous year. NCDR transferred data about the sample to the NIPH, including contact information as a basis for conducting the survey. Data from 27 paediatric outpatient departments in Norway was collected.
All patients registered in the NCDR were included, and their parents were contacted by mail and sent an invitation letter in November 2016. Non-respondents were sent up to two reminders. The sample was randomised into three groups to compare different data collection methods: Group A was posted questionnaires with only a pen-and-paper response option; group B were posted questionnaires with both an electronic and a pen-and-paper response option and, finally, group C were posted questionnaires with only an electronic response option. The three groups were compared on response rate, background variables about respondents, main study results and survey costs. Results from the randomised comparison are reported in another publication .
All 34 questionnaire items that addressed experiences of care and had 5-point response formats were considered relevant to include in initial factor analyses. Items were assessed for levels of missing data, ceiling effect and underlying structure of the items. Items with missing data or responses in the “not applicable” option > 10% were omitted from factor analysis to avoid an extensive loss of responses. Using this criteria for inclusion, 30 out of 34 items were included. A ceiling effect is when marks are clustered in a few boxes at one extreme, in patient experience surveys, often near the very top of the scale, making it almost impossible to detect any improvement or to distinguish between various grades of excellence . The criteria relating to ceiling effects vary between studies. In this study we set the cut off to 50%: If the item should be considered acceptable, fewer than 50% could tick the most favourable response option [27, 28, 33].
Exploratory factor analysis (EFAs) with principal axis factoring was conducted to assess the underlying factor structure of the items. Correlations among the factors were expected, and oblique rotation (promax) was chosen for interpretability of results. The criterion for the number of factors to be rotated was eigenvalues greater than 1. Items with factor loadings lower than 0.4 were excluded, and no cross-loadings over 0.30 were retained. However, the analysis and the subsequent scales were not only data-driven, but also founded on theoretical considerations underpinning the analyses.
The internal consistency reliability of the scales was assessed using item-total correlation and Cronbach’s alpha. Item-total correlation measures the strength of association between the individual item and the remainder of its scale, and a correlation of 0.40 was considered acceptable . An alpha value between 0.70 and 0.90 was considered satisfactory [34,35,36,37].
Item discriminant validity was tested by correlating all single items with the scales. Each item was expected to have a significantly higher correlation with its hypothesised scale than with the other scales . The associations between the items and the scales were assessed with Spearman’s rho.
The literature review did not identify any obvious variables for inclusion in the construct validity testing, neither did a literature review on patient satisfaction measurement in general , or results from previous parent experience studies in the national survey programme in Norway [19, 22, 40]. We did not have access to patient background data when the current study was carried out. Thus, we had no background information on the child. In the absence of gold standard measures, the construct validity testing adopted an exploratory approach, and background variables assumed to relate to the scales were tested. It was hypothesised that scale scores were associated with parent participation, education and considerations regarding the number of consultations at the paediatric department. Age, gender and whether the respondent was living with the child’s other parent were also explored. Correlations between background questions and the scales were assessed with t-tests and one-way ANOVA for categorical questions and by Spearman’s rank correlations (r) for continuous variables.
All statistical analyses were conducted using SPSS version 23.0.
A total of 2606 patients were included in the survey, but 80 were excluded because of incorrect addresses. 1399 (55%) parents or guardians responded to the questionnaire. Table 1 lists the characteristics of the respondents. The mean age was 43.4 years and 1090 (78.5%), were female. As shown in Table 1, 66.8% had a university or college education and 78.7% were living with the child’s other parent. Of the 1399 parents that responded, 1150 (82.8%) had participated in the consultation three or more times in the last year.
Levels of missing data, responses in the “not applicable” option, mean and ceiling effect for the items are shown in Table 2. Levels of missing data ranged from 0.1 to 4.0%. Responses in the “not applicable” category ranged from 0.4 to 38.3%. Rates of missing and “not applicable” < 10% were considered acceptable, and 31 of the 35 items were retained for further analyses. The lowest mean score on the scale of 1–5, where 5 is the best possible experience, was for the item relating to the child’s access to a psychologist (2.14). The highest score of 4.57 was for the item addressing the doctor’s knowledge of diabetes and diabetes care. The majority of items had a score in the range of 3–4 on the 1–5 scale. Furthermore, 27 out of 35 items were below the ceiling-effect criterion of 50%.
All 25 items relating to structure and process were included in the first factor analysis. However, after comprehensive testing, we decided to analyse the items addressing nurse and doctor contact separately to render results more useful for the purposes of local improvement. The organisation of outpatient clinics in Norway differs and integration of the nurse and doctor contact items could confuse the interpretability of the results; this assumption was supported by the empirical testing. Factor analysis including the remaining 17 items on structure and process resulted in three factors. Three items (5, 19 and 22) were excluded from the factor analysis, one by one, due to cross loadings > 0.30 and also low factor loadings on own factor (< 0.40). The results suggested three factors with an eigenvalue of > 1 accounting for 64.3% of the total variation: i) consultation ii) organisation and iii) equipment (Table 3). The second factor analysis included the four items addressing nurse contact and produced one factor with an eigenvalue of > 1 explaining 55.7% of the variation. The third factor analysis included the four items concerning doctor contact and produced one factor that explained 62.2% of the variation. Finally, the fourth factor analysis included the five outcome items (31–35) and produced one factor accounting for 69.5% of the variation.
Table 3 shows that the item-total correlations for the final six scales are acceptable, ranging from 0.40 to 0.78. All scales met the criterion of 0.70–0.90 for Cronbach’s alpha, ranging from 0.71 to 0.90.
The results of item discriminant validity testing are shown in Table 4. As expected, all items had a substantially higher correlation with its hypothesised scale than with any of the other scales in the questionnaire. All correlations were statistically significant (p < 0.001).
Table 5 shows that the associations between the scale scores and the tested variables were statistically significant in 17 out of 18 tests. Associations with age, gender and if the respondent were living with the child’s other parent were all insignificant or very low and are not reported. Participation in the consultation during the last year was significant for all scales (range 0.15–0.19): The higher the number of times parents had participated in the consultation, the higher the scores were on all six scales on a scale of 0 to 100 where 100 is the best possible experience of care. Associations between education and the six scales were significant for all except one (doctor contact). Higher education was associated with lower scores on all five scales. Significant differences (all at the < 0.001 level) were also found between the scores of parents who considered the number of consultations as too few or too many and those who considered the number of consultations as appropriate (range 9–14 points), with the latter group reporting better experiences.
This study was the first national survey undertaken to assess parent experiences of paediatric diabetes care at outpatient departments in Norway. The psychometric testing of the PEQ-DC showed good evidence for data quality, internal consistency and construct validity. The content validity of the instrument was secured by a rigorous development process, and the instrument was tested following a national survey, securing generalisability across Norway.
The PEQ-DC is multidimensional and comprised six scales based on both empirical and theoretical assumptions. The results can be used to monitor performance and enable hospitals to identify areas where the parent-perceived quality should be improved. The scales have excellent psychometric properties and are minimally burdensome for patients to complete. The PEQ-DC is also relevant to use as a basis for identifying quality indicators.
The low percentage of omitted answers suggests good acceptability and few responses in the “not applicable” option indicate that the questions are relevant to a majority of the parents. Four items had high levels of “not applicable” responses. The first related to having enough time for conversations without the child present (19), the second to necessary support to transfer more of the responsibility for diabetes treatment to the child (21). The third addresses access to a dietitian, the fourth access to a psychologist. The first and second are probably most relevant for parents with children above a certain age and missing responses to these questions most likely relate to parents of younger children. The probability of parents having such an experience presumably increases with their child’s age. When children grow older and gain more independence and autonomy, they prefer to take more responsibility for their own diabetes management . The third and fourth questions are not important or relevant for all respondents. Those who did not provide a valid answer to these items probably did not have such experiences, and were consequently not able to answer.
We had limited evidence of which variables to include in construct validity testing. Hypotheses based on previous research findings and theory were scarce and the literature review underpinning this work did not identify any relevant questionnaires. A systematic review found a consistent positive relationship between age and satisfaction, but evidence on other variables including gender and education was equivocal . However, the current study does not include the patient, but the parents or caregivers, and the age range is limited (SD = 6.4). The results showed significant associations with the scale scores and participation in the consultation during the last year, education and considerations regarding the number of consultations at the paediatric department, but not with age, gender and whether the respondent was living with the child’s other parent. The variables included self-reported data from the questionnaire, and further tests should also include administrative and clinical variables from the outpatient clinics. However, results from a study in a paediatric cohort of patients with T1D showed that patient and parent satisfaction were not related to other clinical outcome measures . Multiple assessments and analyses are needed to assess whether a construct appears to be valid .
The respondents in the current study included a high proportion of women (78.5%). Studies focus less on fathers’ experiences and their role and the influence of their management of T1D than the mothers’ [43, 44]. Research shows that mothers experience more responsibility for diabetes tasks than fathers [10, 44, 45]. Mothers also experience greater perceived burden of medical treatment tasks . Thus, mothers may be more involved in coordinating care and may also have more contact with the paediatric outpatient clinic.
The test-retest reliability was not assessed in this study, but results from previous patient experience surveys in the national programme have found acceptable levels of test-retest reliability for other questionnaires [16,17,18,19,20,21,22,23,24,25,26,27,28,29].
This study addressed the considerations of the parents, not those of the patient. Studies show that while parents’ and their adolescents’ reality perceptions might correlate well, this is not always the case for their views on the degree of importance of different aspects of paediatric diabetes care quality , or views and priorities about health and illness . However, parent involvement is necessary until their children are able to manage their diabetes independently. Thus, parents form an integral part of the treatment, and their experiences should be included when assessing the quality of care at the outpatient clinic. Consequently, it is important to understand how parents perceive their child’s health care. Understanding the challenges that parent’s face can inform quality improvement initiatives in outpatient clinics. Both the parents’ and the child’s experiences should be included when assessing the quality of care from the user’s perspective. Paediatricians communicate with both parents and patients and this may require varying levels of information and communication. Health care providers must be capable of evaluating the educational, behavioural, emotional, and psychosocial factors that impact implementation of a treatment plan and work with both the individual and the family to overcome barriers or redefine goals .
Methods to enhance response rates should be considered. However, previous studies among parents in the national patient experience survey programme have produced lower or similar response rates of 46% and 54% [19, 22].
Strengths and limitations
A strength of this study is the large national sample including all paediatric diabetes outpatient clinics in Norway. Also, the study was performed by a third party (the NCDR and the NIPH) not involved in providing health care to the patients. The domains and items were derived from a standardised, comprehensive process. The variables tested in this study included self-reported data from the questionnaire. Further analyses should include administrative and clinical variables from the outpatient clinics.
No background information is available on non-responders. This is a potential bias and future research should undertake more detailed comparisons of respondents and non-respondents to assess the extent of bias.
The PEQ-DC comprises six scales with excellent internal consistency reliability and validity. The instrument is recommended for future applications designed to access parent experiences of outpatient paediatric clinics in Norway. Further tests of construct validity including clinical data are recommended, to also evaluate the instrument’s usefulness as a basis for external quality indicators.
Exploratory factor analysis
The Norwegian Childhood Diabetes Registry
The Norwegian Institute of Public Health
Parent Experiences of Diabetes Care Questionnaire
Type 1 Diabetes
Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3:e001570. https://doi.org/10.1136/bmjopen-2012-001570.
Haugum M, Danielsen K, Iversen HH, Bjertnæs OA. The use of data from national and other large-scale user experience surveys in local quality work: a systematic review. Int J Qual Health Care. 2014;26(6):592–605.
Iversen HH, Bjertnæs ØA, Groven G, et al. Usefulness of a national parent experience survey in quality improvement: views of paediatric department employees. Qual Saf Health Care. 2010;19(5):e38.
Soltesz G, Dahlquist GG. Global trends in childhood type 1 diabetes. Diabetes Atlas, vol. 2012. 3rd ed. p. 154–8.
Skrivarhaug T, Stene LC, Drivvoll AK, Strøm H, Joner G. Incidence of type 1 diabetes in Norway among children aged 0–14 years between 1989 and 2012: has the incidence stopped rising? Results from the Norwegian childhood diabetes registry. Diabetologia. 2014;57(1):57–62.
Strøm H, Selmer R, Birkeland KI, Schirmer H, Berg TJ, Jenum AK, et al. No increase in new users of blood glucose-lowering drugs in Norway 2006–2011: a nationwide prescription database study. BMC Public Health. 2014;14(1):520.
Hanberger L, Birkebaek N, Bjarnason R, et al. Childhood diabetes in the Nordic countries: a comparison of quality registries. J Diabetes Sci Technol. 2014;8(4):738–44. https://doi.org/10.1177/1932296814531479.
Laffel LM, Vangsness L, Connell A, Goebel-Fabbri A, Butler D, Anderson BJ. Impact of ambulatory, family-focused teamwork intervention on glycemic control in youth with type 1 diabetes. J Pediatr. 2003;142(4):409–16.
Sullivan-Bolyai S, Deatrick J, Gruppuso P, Tamborlane W, Grey M. Constant vigilance: mothers’ work parenting young children with type 1 diabetes. J Pediatr Nurs. 2003;18(1):21–9.
Sullivan-Bolyai S, Rosenberg R, Bayard M. Fathers’ reflections on parenting young children with type 1 diabetes. MCN Am J Matern Child Nurs. 2006;31(1):24–31.
Rankin D, Harden J, Waugh N, Noyes K, Barnard KD, Lawton J. Parents’ information and support needs when their child is diagnosed with type 1 diabetes: a qualitative study. Health Expect. 2016;19(3):580–91. https://doi.org/10.1111/hex.12244.
Boogerd EA, Maas-van Schaaijk NM, Noordam C, Marks HJ, Verhaak CM. Parents’ experiences, needs, and preferences in paediatric diabetes care: suggestions for improvement of care and the possible role of the internet. A qualitative study. J Spec Pediatr Nurs. 2015;20(3):218–29.
American Diabetes Association. Diabetes Care 2016;39(Supplement 1): S86–93. https://doi.org/10.2337/dc16-S014.
Lawton J, Waugh N, Noyes K, Barnard K, Harden J, Bath L, et al. Improving communication and recall of information in paediatric diabetes consultations: a qualitative study of parents’ experiences and views. BMC Pediatr. 2015;15:67.
Sheehan AM, While AE, Coyne I. The experiences and impact of transition from child to adult healthcare services for young people with type 1 diabetes: a systematic review. Diabet Med. 2015;32(4):440–58.
Pettersen KI, Veenstra M, Guldvog B, Kolstad A. The patient experiences questionnaire: development, validity and reliability. Int J Qual Health Care. 2004;16(6):453–63.
Oltedal S, Garratt A, Bjertnaes O, Bjornsdottir M, Freil M, Sachs M. The NORPEQ patient experiences questionnaire: data quality, internal consistency and validity following a Norwegian inpatient survey. Scand J Public Health. 2007;35(5):540–7.
Iversen HH, Holmboe O, Bjertnæs OA. The Cancer Patient Experiences Questionnaire (CPEQ): reliability and construct validity following a national survey to assess hospital cancer care from the patient perspective. BMJ Open. 2012;2:e001437. https://doi.org/10.1136/bmjopen-2012-001437If.
Garratt AM, Bjertnaes OA, Barlinn J. Parent experiences of paediatric care (PEPC) questionnaire: reliability and validity following a national survey. Acta Paediatr. 2007;96(2):246–52.
Bjertnaes O, Iversen HH, Kjollesdal J. PIPEQ-OS--an instrument for on-site measurements of the experiences of inpatients at psychiatric institutions. BMC Psychiatry. 2015;15:234.
Garratt A, Bjorngaard JH, Dahle KA, Bjertnaes OA, Saunes IS, Ruud T. The psychiatric out-patient experiences questionnaire (POPEQ): data quality, reliability and validity in patients attending 90 Norwegian clinics. Nord J Psychiatry. 2006;60(2):89–96.
Garratt AM, Bjertnaes OA, Holmboe O, Hanssen-Bauer K. Parent experiences questionnaire for outpatient child and adolescent mental health services (PEQ-CAMHS outpatients): reliability and validity following a national survey. Child Adolesc Psychiatry Ment Health. 2011;5:18.
Garratt AM, Danielsen K, Forland O, Hunskaar S. The patient experiences questionnaire for out-of-hours care (PEQ-OHC): data quality, reliability, and validity. Scand J Prim Health Care. 2010;28(2):95–101.
Sjetne IS, Iversen HH, Kjollesdal JG. A questionnaire to measure women's experiences with pregnancy, birth and postnatal care: instrument development and assessment following a national survey in Norway. BMC Pregnancy Childbirth. 2015;15:182.
Bjertnaes OA, Garratt A, Nessa J. The GPs’ experiences questionnaire (GPEQ): reliability and validity following a national survey to assess GPs’ views of district psychiatric services. Fam Pract. 2007;24(4):336–42.
Sjetne IS, Bjertnaes OA, Olsen RV, Iversen HH, Bukholm G. The generic short patient experiences questionnaire (GS-PEQ): identification of core items from a survey in Norway. BMC Health Serv Res. 2011;11:88.
Haugum M, Iversen HH, Bjertnaes O, Lindahl AK. Patient experiences questionnaire for interdisciplinary treatment for substance dependence (PEQ-ITSD): reliability and validity following a national survey in Norway. BMC Psychiatry. 2017;17:73. https://doi.org/10.1186/s12888-017-1242-1.
Bjertnaes OA, Lyngstad I, Malterud K, Garratt A. The Norwegian EUROPEP questionnaire for patient evaluation of general practice: data quality, reliability and construct validity. Fam Pract. 2011;28(3):342–9.
Olsen RV, Garratt AM, Iversen HH, Bjertnaes OA. Rasch analysis of the psychiatric out-patient experiences questionnaire (POPEQ). BMC Health Serv Res. 2010;10:282.
Sitzia J. How valid and reliable are patient satisfaction data? An analysis of 195 studies. Int J Qual Health Care. 1999;11:319–28. https://doi.org/10.1093/intqhc/11.4.319.
Bjertnaes O, Iversen HH, Skrivarhaug T. A randomized comparison of three data collection models for the measurement of parent experiences with diabetes outpatient care. BMC Med Res Methodol. 2018;18(1):95.
Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 4th ed. New York: Oxford University Press; 2008.
Ruiz MA, Pardo A, Rejas J, Soto J, Villasante F, Aranguren JL. Development and validation of the “treatment satisfaction with medicines questionnaire” (SATMED-Q). Value Health. 2008;11(5):913–26.
Nunnally JC, Bernstein IH. Psychometric theory. 3rd ed. New York: McGraw-Hill; 1994.
Field A. Discovering statistics using SPSS. London: Sage; 2009. u.a
Kline RB. Principles and practice of structural equation modeling. New York: Guildford; 2005.
Lance CE, Butts MM, Michels LC. The sources of four commonly reported cutoff criteria – what did they really say? Org Res Methods. 2006;9(2):202–20. https://doi.org/10.1177/1094428105284919.
Gandek B, Ware JE Jr, Aaronson NK, Alonso J, Apolone G, Bjorner J, Brazier J, Bullinger M, Fukuhara S, Kaasa S, et al. Tests of data quality, scaling assumptions, and reliability of the SF-36 in eleven countries: results from the IQOLA project. International quality of life assessment. J Clin Epidemiol. 1998;51(11):1149–58.
Crow R, Gage H, Hampsom S, Hart J, Kimber A, Storey L, Thomas H. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess. 2002;6(32):1–244.
Holmboe O, Iversen HH, Hanssen-Bauer K. Determinants of parents’ experiences with outpatient child and adolescent mental health services. Int J Ment Heal Syst. 2011;5:22. https://doi.org/10.1186/1752-4458-5-22.
Dashiff C, Vance D, Abdullatif H, Wallander J. Parenting, autonomy and self-care of adolescents with type 1 diabetes. Child Care Health Dev. 2009;35(1):79–88.
Koves IH, et al. Satisfaction of care in a tertiary level diabetes clinic: correlations with diabetes knowledge, clinical outcome and health-related quality of life. J Paediatr Child Health. 2008;44(7–8):432–7.
Boman A, Povlsen L, Dahlborg-Lyckhage E, Hanas R, Borup I. Fathers’ encounter of support from paediatric diabetes teams; the tension between general recommendations and personal experience. Health Soc Care Community. 2013;21(3):263–70.
Dashiff C, Morrison S, Rowe J. Fathers of children and adolescents with diabetes: what do we know? J Pediatr Nurs. 2008;23(2):101–19.
Dashiff CJ. Self-and dependent-care responsibility of adolescents with IDDM and their parents. J Fam Nurs. 2003;9(2):166–83.
Haugstvedt A, Wentzel-Larsen T, Rokne B, Graue M. Perceived family burden and emotional distress: similarities and differences between mothers and fathers of children with type 1 diabetes in a population-based study. Pediatr Diabetes. 2011;12(2):107–14.
Hanberger L, Ludvigsson J, Nordfeldt S. Quality of care from the patient's perspective in paediatric diabetes care. Diabetes Res Clin Pract. 2006;72(2):197–205.
Jutras S, Morin P, Proulx R, Vinay MC, Roy E, Routhier L. Conception of wellness in families with a diabetic child. J Health Psychol. 2003;8(5):573–86.
We would like to thank Hilde Bjørndalen at the Department of Paediatric Medicine at Oslo University Hospital for organising patients and parents for the interviews, for participating in the expert group and for the contribution to financing the project through “Lillian and Werner Næss legat”. We would also like to thank Inger Opedal Paulsrud, Olaf Holmboe, Johanne Kjøllesdal and Nam Pham from the Knowledge Centre at the NIPH for their help in developing and conducting the survey, including administrative and technical tasks in the data collection. We would further thank Ann Kristin Drivvoll from the NCDR for data extraction from the register. Finally, thank you to the parents for participating in the survey.
This work was financed by the Knowledge Centre at the NIPH, the NCDR and “Lillian and Werner Næss legat”. NCDR is funded by the South-Eastern Norway Regional Health Authority. The researchers/authors at the NIPH and the NCDR were involved in the design of this study, during its execution, analyses, interpretation of the data and writing the manuscript. The Lillian and Werner Næss legat had no role in the design of this study or during its execution, analyses, interpretation of the data, or writing the manuscript.
Availability of data and materials
The datasets generated and/or analysed during the current study are not publicly available due to protection of personal data.
Ethics approval and consent to participate
The study was approved by the Data Protection Authority at Oslo University hospital. Registration in the NCDR is based on informed consent from the child (above 12 years of age), and/or the child’s parents. In the consent form, the patient and/or the parents are informed that consent may result in requests to answer questionnaires on patient experiences. Return of the questionnaire constituted parent consent in the survey, which is the standard procedure in all national patient experience surveys conducted by the NIPH.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.