Skip to main content
  • Research article
  • Open access
  • Published:

Determinants of caregiver satisfaction with child neurodevelopmental assessment in neuropaediatric clinics

Abstract

Background

In addition to patient evaluations, caregiver evaluations and experiences are important indicators of the quality of health services. The aim of this study was to examine determinants of caregiver satisfaction with and perceived benefit of child neurodevelopmental assessment in neuropaediatric clinics.

Methods

The study was conducted among caregivers of children and adolescents aged 4–18 years (N = 330) referred for neurodevelopmental assessment in two neuropaediatric clinics in the specialised health service in Northern Norway. The Generic Short Patient Experiences Questionnaire (GS-PEQ) for child psychiatric outpatient patients was distributed to caregivers immediately following the assessment, and two of its items were used as measurements of caregiver satisfaction with and perceived benefit of the assessment.

Results

Caregiver satisfaction with the assessment was correlated with a better general level of function in the child, higher socioeconomic status, Norwegian mother tongue, referral from a specialist, and the respondent being a woman. Higher perceived benefit of the assessment was correlated with higher socioeconomic status, Norwegian mother tongue, and younger age of the child. Regression analysis revealed that caregivers’ perception that the assessment was suited to their child’s situation and that there was good cooperation with other public services (e.g., primary care and social/educational services) seemed more fundamental to caregiver satisfaction with neuropaediatric clinics’ services than any background variable. Younger age of the child, in addition to caregivers’ perception that the assessment was suited to their child and receiving sufficient information about the child’s diagnosis/afflictions, were essential to the perceived benefit of the assessment.

Conclusions

Caregiver satisfaction with child neurodevelopmental assessment in neuropaediatric clinics partly depends on variables not related to the assessment experience per se. An assessment that was suited to the child, good cooperation with other public services such as primary health care and social/educational services, and giving sufficient information about the child’s diagnosis are essential to an overall positive caregiver evaluation of neurodevelopmental assessments.

Peer Review reports

Background

User experiences with health services can be viewed as reports on the quality of these services [1]. Indeed, patient experiences and satisfaction are associated with important quality aspects of health care, like patient adherence to treatment, patient safety, and clinical effectiveness [2,3,4]. Information on caregiver experiences with neuropaediatric health services or services for disabled children are increasingly sought [5,6,7,8] and are important indicators of the quality of health care delivered to children.

The concepts ‘experiences with health care’ and ‘satisfaction’ are positively related [9]. Measuring experiences with health services gives providers the opportunity to improve care, meet patients’ expectations, and effectively manage and monitor health care performance [9, 10]. Patient satisfaction is a complex concept that depends on several variables, such as social standards, context, needs, values, previous experiences, future expectations, information, education, health, medical care, treatment, and psychological factors [2, 11,12,13]. Satisfaction surveys are used to capture patient evaluations of many different services and are implicitly or explicitly based on the understanding of satisfaction as the fulfilment of expectations [14]. Reported high satisfaction does not necessarily indicate a good experience, and reported dissatisfaction may be used as an indicator of some negative experience [14]. A study by Norman and colleagues [15] reported that, even when treatment outcomes were poor, satisfaction with Child and Adolescent Mental Health Services (CAMHS) was still high. Collecting information about specific experiences with concrete aspects of health services is a more valid measure of satisfaction, and easier to interpret than satisfaction ratings [16]. Both user experiences and user satisfaction are increasingly employed as quality indicators in the health care sector [17, 18].

From a user perspective, the main components of the quality of service relate to access to information, respect, support, and good coordination and collaboration [2, 19,20,21,22,23,24]. Parents with children receiving a first-time diagnosis of developmental disability were more satisfied when a large amount of information was provided, and especially appreciated it when clinicians possessed good communication skills and had an understanding of their situation [25]. Previously reported user satisfaction following neuropsychological evaluation [8] was mostly related to clinicians’ concern and competence.

In a study of user satisfaction following paediatric neuropsychological evaluation, Bodin et al. [6] concluded that neither wait time nor referral source were associated with this variable. Holmboe and colleagues [24] found that the perceived wait time for a consultation was associated with parents’ experiences with mental health services, but found no association with the wait time recorded in patient journals. Other results indicated that patient satisfaction with child outpatient services may be related to shorter recorded wait times [26, 27].

Some demographic factors have been repeatedly related to user satisfaction and may be treated as proxies of expectations. Earlier studies have indicated that more positive parental evaluation of health services was related to younger children [19, 21,22,23, 28,29,30], and shorter parental education [6, 7, 24, 30,31,32]. No significant differences in evaluations by caregivers’ mother tongue [22, 24], and in most studies, no significant differences for the gender of the child was found [6, 21, 22, 29, 33]. The reported role of socioeconomic status [2, 34] and the gender of the respondent [18, 35] in satisfaction with health services have been inconsistent.

Generally, good health status of the respondent is associated with a positive evaluation of health services [2, 18, 32, 36, 37]. Parents with poorer health, a lower quality of life, and those who experience more everyday stress may have more negative views of their child’s treatment [38,39,40,41,42]. Therefore, it is important to examine to what extent parents’ mental health can influence their evaluation of neuropaediatric services.

Different results have been reported for the relationship between children’s diagnoses [6, 19, 21, 23, 24, 29, 43] or the number of child health problems and caregiver evaluations of health services [23, 24, 29, 31, 42]. Thus, it is still unclear whether caregivers are experiencing and assessing child rehabilitation services in the context of the severity of their child’s problems. Enhancing knowledge in this area would help clinicians and health services to identify those caregivers who need more information and support, as well as contribute to higher satisfaction with neuropaediatric services. Many earlier studies on user evaluations of health services for disabled children included a limited number of factors that could affect caregiver evaluations; they focused mostly on user experiences with health services and their relationship with demographic variables [6, 19, 21,22,23, 44].

The aim of the current study was to examine determinants of caregiver satisfaction with and perceived benefit of child neurodevelopmental assessment in neuropediatric clinics in Northern Norway. The outcomes caregiver satisfaction and perceived benefit of assessment were measured by a short-form survey, the Generic Short Patient Experiences Questionnaire (GS-PEQ) [1], to reduce the burden of collecting and analysing data [45]. Given the inconsistent results of other studies, we aimed to investigate the association between background variables (health service characteristics, caregiver characteristics, and child characteristics), as well as specific experiences with neuropaediatric services, and the outcome variables.

Methods

Participants

The study population consisted of caregivers of children referred by a general practitioner or a medical specialist to the neuropaediatric outpatient clinics at the University Hospital of North Norway or the Finnmark Hospital Trust in Norway for neurodevelopmental or neurological assessment [46,47,48]. These clinics are specialised health service units in Northern Norway serving children and adolescents with neurodevelopmental disorders, early-acquired disabilities or intellectual and developmental disabilities. The participants were included consecutively in the present study. In order to be included in the study, children had to be aged 4 to 18 years at the time of referral, and referred between October 2012 and July 2016 at the University Hospital of North Norway, or between January 2014 and July 2016 at the Finnmark Hospital Trust. A total of 518 children and adolescents met these criteria, of whom 153 (30%) were excluded due to lack of treatment in the clinics either because of time constraints, lack of caregiver motivation, or insufficient knowledge of the Norwegian language because several of the questionnaires were only available in Norwegian or a restricted number of additional languages [46]. The caregivers of the remaining 365 eligible children (247 referred by general practitioner, 118 referred by medical specialist) were invited to complete the GS-PEQ and participate in the study. Three hundred thirty caregivers agreed (90.4%) and were included in the final study sample.

The most frequent diagnostic groups among children in the sample were specific developmental disorders (41.5%), intellectual disability (21.8%), autism spectrum disorder (15.8%), and diseases/disorders of the central nervous system such as epilepsy and cerebral palsy (14.8%). The diagnoses were not mutually exclusive. A total of 12% children were not diagnosed with any neurological or neurodevelopmental disorder.

Measures

The generic short patient questionnaire

The GS-PEQ [1], created by The Norwegian Knowledge Centre for the Health Services, is a generic, 10-item, questionnaire that collects information on user experiences across a range of specialist health services. The GS-PEQ is based on previous testing of six, group-specific questionnaires, among them parents’ evaluation of their experiences with somatic inpatient services [49] and psychiatric outpatient services (i.e., CAMHS patients) [50]. The GS-PEQ includes items regarding specific experiences with clinical services, user involvement, information, organisation, accessibility (wait time), incorrect treatment, and outcome (satisfaction and perceived benefit) [1]. The questionnaire’s authors also added three items relevant to CAHMS patients: one regarding clinical services, one regarding information about the assessment, and one regarding cooperation [51]. This version is referred to as the “Generic short version – caregivers about CAMHS” of the GS-PEQ [51], which was used in this study. All 13 items in the applied version of the GS-PEQ are formatted as questions. Twelve of them are answered on a 5-point scale from “not at all” (0) to “to a great extent” [4], or as “not applicable”. One question regarding the wait time to get an appointment is answered on a 4-point scale from “no wait time” (0) to “way too long” [3]. Two of the questions about wait time and incorrect treatment did not correlate or correlated weakly with the other scores in the GS-PEQ, and they were used for comparison with administrative data, respectively in the Parent Assessment of Outpatient CAMHS [50, 52] and the Parent Experiences of Paediatric Care [49]. GS-PEQ is freely available without a license.

Background variables

Caregivers’ demographic data (age, gender, marital status, mother tongue, education, and employment) were collected from the appendix that was distributed with the GS-PEQ [51]. Children’s demographic data (age and gender) were taken from the Development and Wellbeing Assessment (DAWBA [53];). Information about referral source (general practitioner or medical specialist) and wait time for the assessment was taken from patient records. A single subscale of the Family Stress Scale part of the DAWBA, socioeconomic/housing score [53], was employed to assess subjective experience of socioeconomic status in the previous 12 months. The variable consisted of items about subjectively evaluated stressors connected to financial difficulties, unemployment, problems with neighbours/neighbourhood, and having home inadequate for family’s needs. Caregivers rated the items on a 3-point scale from “none, or doesn’t apply” (0) to “a lot” [2]. Scores equal to two or higher were assigned lower socioeconomic status (score = 0). We have license to use the DAWBA including the Everyday Feeling Questionnaire through the Youth in Mind: https://youthinmind.com/

Caregivers’ mental health

The mental health of the caregivers was assessed with the self-administered version of the Everyday Feeling Questionnaire (EFQ [54]), which is part of the DAWBA [53]. The EFQ contains 10 items estimating symptoms of distress (e.g. “stressed” or “very unhappy”), and psychological well-being (e.g. “positive about the future” or “calmed and relaxed”). Respondents rated the symptoms on a 4-point scale ranging from “none of the time” (0) to “all of the time” [4]. Lower total scores reflect lower levels of distress and higher levels of well-being. The EFQ has good internal consistency, with a Cronbach’s α reported between .87 and .90 [48, 54, 55].

Children’s global assessment scale

The CGAS [56] is a clinician-rated tool used to assess the global psychosocial functioning of children, taking into account all available information. The score on this scale reflects the lowest overall level of psychosocial functioning (i.e., at home, at school, and with peers) of the child or adolescent during the preceding month. Total CGAS score ranges from 1 (the most impaired level) to 100 (the best level of functioning), and the score is separated into 10-point intervals, each of which describes a specific level of functioning, followed by examples of matching behaviour and life situations adequate for children and adolescents. In a large Norwegian study of clinicians in outpatient CAMHS [57], the interrater reliability of the routine use of the CGAS was found to be moderate (intraclass correlation coefficient = .61). We have license to use the CGAS through The Norwegian Directorate of eHealth: https://ehelse.no/english

Procedure

Children underwent the interdisciplinary assessment of neurodevelopmental/neurological disorders and an additional assessment of the presence of coexisting behavioural and emotional disorders. The interdisciplinary assessment included specialists such as paediatricians, neuropsychologists, special education therapists, occupational therapists, and physiotherapists [46, 47]. The presence of a neurodevelopmental/neurological was examined by paediatricians using methods such as MRI Caput, EEG or genetic testing if indicated. A clinical psychologist/neuropsychologist assessed developmental level in all children using a standardised intelligence scale and the Vineland Adaptive Behaviour Scale II [58]. The GS-PEQ was distributed to and completed by caregivers immediately following child neurodevelopmental assessments. Informed consent was obtained from all individual participants included in the study. The study was approved by the appropriate ethics committee. The use of de-identified data was approved by the data protection officer at University Hospital of North Norway and Finnmark Hospital Trust.

Statistics

The data were analysed using SPSS Version 25. For categorical variables, dummy variables were used (e.g., gender: 0 – man/boy, 1 – woman/girl). Some dummy variables were created for variables with more categories (e.g., mother tongue: 0 – Norwegian, 1 – others; education: 0 – primary/secondary/high school; 1 – college/university). Sami mother tongue was combined with Norwegian mother tongue, as the three participants declaring Sami mother tongue also reported Norwegian mother tongue.

Missing response on items, ceiling effect, demographic characteristics of the respondents, and specific experiences with neuropaediatric services were assessed using descriptive statistics, based on the total number of participants (N = 330). An acceptable ceiling effect is generally defined as a maximum of 50% of respondents choosing the most positive response category [59]. Specific experiences with neuropaediatric services were categorised as positive (the two highest item scores), neutral (moderate item scores), and negative (the two lowest item scores). The question “Do you believe that your child was in any way given the wrong treatment?” was excluded from the analyses due to probable misunderstanding by the participants (high scores for wrong treatment were associated with positive experience and satisfaction, n = 15).

The relationships between the outcome variables (satisfaction and perceived benefit of the assessment), service characteristics (clinic site, wait time, referral source), caregiver characteristics (gender, mother tongue, socioeconomic status, education, mental health), and child characteristics (age, gender, psychosocial functioning) were examined using Pearson’s correlation coefficients. Hierarchical linear regression analyses were conducted to examine which variables could uniquely explain variation in the outcome variables. The predictive variables included service characteristics, caregiver characteristics, child characteristics, and specific experiences with neuropaediatric services. In correlation and regression analyses, only cases with data both on satisfaction and perceived benefit were used (N = 265) to assure that exactly the same participants were used both to predict satisfaction and perceived benefit. These 265 participants had missing values for the following variables: socioeconomic status (16.6%), caregiver gender (8.3%), CGAS score (3.4%), and specific experience with neuropaediatric services (between 0.4 and 13.4%). Missing values were not substituted and were dealt with in linear regression by pairwise deletion. We assessed the significance of change in explained variation (R2) by applying a conventional R2 change of 2% as a small effect, a change of 13% as a medium effect, and a change of 26% as a large effect [60]. The statistical methods were set on a 5% significance level.

Results

Caregiver characteristics

Of 365 caregivers invited to answer the GS-PEQ-CAMHS, 330 completed it (90.4%). Respondents were between 24 to 71 years of age (mean, M = 41.5; standard deviation, SD = 7.4). Children’s age ranged between 4 and 18 years (M = 10.2, SD = 3.8), and 34.2% were females. Detailed caregiver characteristics are included in Table 1. There were no significant differences in service or child or characteristics between caregivers who completed the GS-PEQ and those who did not.

Table 1 Demographic characteristics of caregivers (N = 330)

Ceiling effect, missing values, and not applicable responses

Only one question about perceived benefit of the assessment met the criterion of maximum 50% responses in the most positive category. All the other questions achieved a high ceiling effect. Missing values occurred in around 2% of answers, with a range 0.9–7% (Table 2).

Table 2 Caregivers’ specific experiences with neuropaediatric services. Questions from the Generic Short Patient Experiences Questionnaire (GS-PEQ)

Caregivers had the possibility to choose “not applicable” in response to all the questions about specific experiences, as well as for the outcome variables. They judged two questions as especially irrelevant to their situation: “Were you involved in any decisions regarding your child’s treatment?” (22.1% answered “not applicable”) and “Do you find that the clinic cooperated well with other public services?” (20.9% answered “not applicable”). Most of the questions that contained the word “treatment” had a high percentage of “not applicable” answers. Caregivers that chose “not applicable” to answer the question on information about diagnosis/afflictions were statistically significantly more likely to have a child without any diagnosis of neurodevelopmental/neurological disorder (χ2 = 6.6, p = .01). All the questions concerning communication with the clinician and confidence in his/her professional skills were highly applicable (only around 1% “not applicable”), with exception of confidence in the professional skills of other staff (8% responded “not applicable”).

Caregiver evaluation of the assessment

Most caregivers were highly satisfied with their child’s assessment (97%) (Table 2), and they answered positively to almost all questions about the relationship with clinicians (i.e., communication and confidence in their professional skills). Caregivers evaluated the assessment as highly beneficial (83.9% positive experiences). Most negative experiences were related to caregivers’ involvement in decisions regarding the child’s assessment (5.9%), the information they were given about their child’s diagnosis or afflictions (3.3%), and the clinic’s cooperation with other public services (3.2%).

Determinants of caregiver satisfaction with and perceived benefit of the neurodevelopmental assessment

Caregiver satisfaction with and perceived benefit of the child neurodevelopmental assessment were moderately correlated (r = .47, p < .001). Satisfaction with the assessment was weakly associated with referral from a specialist, being a woman, having Norwegian mother tongue, higher socioeconomic status, and having a child with higher psychosocial functioning (Table 3). Perceived benefit of the assessment was weakly related to having Norwegian mother tongue, higher socioeconomic status, and being caregiver to a younger child. Wait time, caregiver’s education and mental health, and child’s gender did not have any significant association with the outcome variables. Caregiver satisfaction with and perceived benefit of the assessment did not differ significantly between the two clinic sites (University Hospital of North Norway and the Finnmark Hospital Trust).

Table 3 Bivariate relationships between overall caregiver satisfaction with child neurodevelopmental assessment and background variables

The overall model predicting caregiver satisfaction with the neurodevelopmental assessment was significant (F(15,158) = 13.03, p < .001) and accounted for 55.3% of the variance in the satisfaction score (Table 4). Background variables (step 1) and specific experiences with neuropaediatric services (step 2) accounted for 13.7, and 41.6% of the variance in satisfaction, respectively, reflecting an effect of medium magnitude in step 1, and an effect of large magnitude in step 2. Specifically, satisfaction was significantly associated with two kinds of specific experiences with neuropaediatric services: perceived suitable assessment and cooperation with other public services (i.e., primary care and social and educational services).

Table 4 Hierarchical multiple regression analysis results for the prediction of caregiver satisfaction with and benefit of the assessment

The overall model predicting the perceived benefit of the assessment was significant as well (F(15,158) = 4.74, p < .001) and accounted for 31% of the variance in the way caregivers answered to the question about the benefit of the assessment. Background variables (step 1) and specific experiences with neuropaediatric services (step 2) accounted for 8.4, and 22.7% of the variance in benefit, respectively, reflecting an effect of small magnitude in step 1, and an effect of medium magnitude in step 2. Specifically, child’s lower age, caregiver’s perception of a suitable assessment, and getting sufficient information about the child’s diagnosis/afflictions significantly predicted the perceived benefit of the assessment.

Discussion

The overall purpose of this study was to examine determinants of caregiver satisfaction with and perceived benefit of the child neurodevelopmental assessment. We looked at specific experiences with neuropaediatric services as well. In general, most of the caregivers were satisfied with their child’s assessment in the two neuropaediatric clinics in Northern Norway; similar results have been reported in similar patient populations [6, 8, 29]. In addition, good cooperation with other public services and the assessment suited to the child’s situation seemed more fundamental to caregiver satisfaction with neuropaediatric clinics’ services than any background variable.

As user surveys tend to be positively skewed [13, 14], it was important to look at the few respondents who were not fully satisfied. A relatively high number of caregivers evaluated their involvement in the assessment, the cooperation with other services, and the provision of sufficient information about their child’s diagnosis/afflictions as either more negative or not relevant for them in relation to other specific experiences. Other studies on services for disabled children pointed out that caregivers gave the most negative evaluations for the amount of information received [19, 21,22,23,24,25] and the coordination of delivered services [21], which is in line with our study.

Most of the background variables were negligible in predicting caregiver satisfaction with and perceived benefit of the assessment, especially after specific experiences with neuropaediatric services were included in the regression analyses. These specific experiences explained more of the high overall satisfaction with the assessment than any other background variable. In our study, these specific experiences played a causal role in caregiver satisfaction; and a generic survey that includes single questions on specific indices of different experiences instead of full scales is a good method to identify predictive variables [61]. User experiences with health services were the most powerful determinants of patient overall satisfaction in other studies as well [9, 18, 34, 36]. In our study, two types of specific experiences with neuropaediatric services were especially crucial in the explanation of overall satisfaction, i.e., if the assessment was suited to the child’s situation, and cooperation with other public services, demonstrating that these factors are of high importance. The situation seemed different for the perceived benefit of the assessment. Specific experiences with neuropaediatric services explained this outcome to a smaller degree, and among the background variables, child’s age was still essential to the explanation of the variance in this outcome. Specific experiences that were crucial to the perceived benefit of the assessment were whether the assessment was suited to the child’s situation, and getting sufficient information about child’s diagnosis/affliction after the assessment.

Some background variables were clearly only weakly correlated with caregiver satisfaction and perceived benefit. Caregivers of children referred by general practitioners were less satisfied with the assessment than those of children referred by a medical specialist. This result could be due to the different health problems that may be present in patients referred from a medical specialist, as it is likely that these patients spent more time in specialist health services, and/or had more serious health problems, making the neurodevelopmental assessment an important step in the process of clarifying the child’s afflictions. Caregivers with Norwegian mother tongue were more satisfied with and perceived a higher benefit of the assessment. This difference could be caused by either communication problems or different expectations of health services related to cultural background. Higher socio-economic status was related to both higher satisfaction with the assessment and more perceived benefit of the assessment. Previous results on the relationship between socioeconomic factors and user satisfaction have been inconsistent [2, 34]. In a review, Willems and colleagues [62] concluded that patients from lower social classes could be disadvantaged due to a misperception of their needs on the part of their doctor, as well as their lower ability to participate in the care process. They pointed out that the communication between doctors and these patients was characterised by less information, fewer directions, and less socio-emotional and partnership building. Both our results and the results from the review indicate that clinicians should be aware of contextual differences in their communication patterns with patients/their caregivers. Finally, in our study, caregivers of younger children had a higher perceived benefit of the assessment, confirming other findings [19, 21,22,23, 28,29,30]. Younger children are new in the system, and an assessment can be a milestone in understanding the child and learning more about a condition. Another possible explanation is that older children may have more severe neurodevelopmental problems and a higher incidence of mental health difficulties [63]. Egilson [28] explained such results simply by assuming that parents become more critical of the services as their children grow older. Generally, the existence of small associations between demographics and service evaluations can have two explanations – different groups may have different response tendencies or different groups may be treated differently during the care process [12].

Higher child global psychosocial functioning was associated with higher caregiver satisfaction, and this should be taken into consideration when interpreting user satisfaction surveys. This result could indicate that the caregivers of these children needed less help. A study by Ezpeleta et al. [64] showed that parents of children with high functional impairment both more often admitted needing psychiatric help and more often sought such help. It is also possible that expectations of health care delivery did matter [2, 9, 65]. Ambiguous results about the severity of a child’s mental health and service evaluation exist in the literature [6, 19, 21, 23, 24, 29, 31, 43], but our results are in accordance with results that showed an association between higher caregiver satisfaction with services and better functioning [42] or less severe problems of the child [64]. Our finding of no significant relationship between caregiver’s mental health and satisfaction with the assessment disproves earlier findings of an association between the health status of a respondent and service evaluation [2, 18, 32, 36, 37, 40,41,42].

When the GS-PEQ was created, it was assumed that the number of questions inapplicable to any respondent would not exceed 20% [1]. In our study, as many as one-fifth of the caregivers evaluated questions about involvement and cooperation as inapplicable to their child’s situation. This may indicate that the caregivers did not recognise these areas as a responsibility of the clinics. Such an evaluation of the service could be influenced by the temporal characteristics of the assessment. A cooperative feedback meeting, where the results are communicated and clinical implications and further treatment is planned, takes place within 2 weeks of the assessment. Visible cooperation with other public services, like primary care and social/educational services, also starts then, whereas our caregivers completed their evaluation of health care delivery directly after the assessment. If our caregivers had completed the GS-PEQ after the cooperative feedback meeting, it may have led to a different evaluation.

Clinical implications

The evaluation of cooperation with other services as inapplicable by many of our caregivers could mean that they did not get clear information about the possibility for cooperation between neuropaediatric clinics and primary care and social/educational services, among others. At the same time, it is difficult to imagine that an assessment in a neuropaediatric clinic could be conducted in a vacuum, without any interaction with other important services. Thus, caregiver evaluations might indicate that The Coordination Reform, which was enacted in Norwegian health care system in the 2000s, and concerned cooperation and coordination across health care units [66], did not affect neuropaediatric services to the extent necessary.

Norwegian national guidelines for child neuropaediatric clinics emphasise the importance of user involvement as a prerequisite for patient and user safety, and a requirement for sound services [67]. Many caregivers in our study replied that the question on being involved in the assessment was inapplicable to their situation. We concluded that the involvement rates and knowledge of the possibility for involvement are definitely areas that require improvement in the clinics. In addition, we found that the use of the term “treatment” in the questionnaire might be problematic, as it could cause respondents to misunderstand the questions (among them the question about involvement); indeed, the health service delivered was primarily an assessment, not a treatment.

Strengths and limitations

Our study has some significant strengths. There are advantages to our close-to-real-time data collection [68]. We had a very good response rate, and the timing of our data collection prevented memory distortion in the participants. A generic survey has its advantages – it is time-saving, more motivating to complete, creates less burden on participants, is easier to interpret, and allows comparisons between different health care units [69]. Of course, our study has some limitations as well. The GS-PEQ is a survey that was created based on health service-specific surveys; it was meant to cover both adults and children, inpatients and outpatients, and short and long-lasting treatment. However, only two of these surveys refer specifically to children [49, 50], and one of them to outpatients [50]. None of these surveys are specific to child rehabilitation or neuropaediatric clinics. This could create a problem with applicability or suitability of the selected questions, and may have influenced the acceptance or understanding of these surveys by the users. The creators of the GS-PEQ [1] recommended it for the use in large samples to help strategic managers monitor quality of care, and to inform decision-making or service evaluation at the operational management level. In addition, our results were positively skewed, indicating the existence of a ceiling effect. Thus, interpretation of satisfaction can be problematic as the outcome of an active evaluation [14]. Another limitation is that we cannot exclude the possibility that the least satisfied caregivers refused participating in our study, and their lack of participation could influence the results.

Conclusions

The GS-PEQ contains questions related to a wide spectrum of specific experiences that explained significant proportions of the variation in satisfaction and perceived benefit of the assessment in our study. These specific experiences are indices of the perceived quality of health services. Caregiver satisfaction with neurodevelopmental assessment in neuropaediatric clinics in our study depended partly on variables not related to specific experiences with neuropediatric services per se. However, an assessment that was adapted to the child’s needs, good cooperation with other public services such as primary care and social/educational services, and giving sufficient information about the child’s diagnosis are experiences that are essential to an overall positive evaluation of child neurodevelopmental assessment. In addition, clinicians should be especially vigilant in including caregivers in decision-making and in discussing the possibilities for cooperation with other services.

Availability of data and materials

The datasets analysed during the current study are not publicly available due to ethical restrictions and personal data protection but are available from the authors on reasonable request and with permission of the Data Protection Official in the health trusts.

Abbreviations

CAMHS :

the Child and Adolescent Mental Health Services

CGAS :

Child Global Assessment Scale

DAWBA :

Development and Well-being Assessment

EFQ :

the Everyday Feeling Questionnaire

F :

ratio of the mean regression sum of squares divided by the mean error sum of squares

GS-PEQ :

the Generic Short Patient Experiences Questionnaire

M :

Mean

N :

total sample size

n :

subsample size

p :

probability of the data arising by chance

r :

Pearson’s product moment correlation coefficient

SD :

Standard Deviation

SDQ :

the Strengths and Difficulties Questionnaire

References

  1. Strømseng Sjetne I, Bjertnaes OA, Vegar Olsen R, Hestad Iversen H, Bukholm G. The Generic Short Patient Experiences Questionnaire (GS-PEQ): identification of core items from a survey in Norway. 2011;11(1).

  2. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess. 2002:1–244.

  3. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1).

  4. Ahmed F, Burt J, Roland M. Measuring patient experience: concepts and methods. The Patient - Patient-Centered Outcomes Research. 2014;7(3):235–41.

    Article  PubMed  Google Scholar 

  5. Chilvers R, Gratton S, Bernard SH. Satisfaction with a child and adolescent mental health services (CAMHS) intellectual disability service. Adv Ment Health Intellect Disabil. 2013;7(1):49–58.

    Article  Google Scholar 

  6. Bodin D, Beetar JT, Yeates KO, Boyer K, Colvin AN, Mangeot S. A survey of parent satisfaction with pediatric neuropsychological evaluations. Clin Neuropsychol. 2007;21(6):884–98.

    Article  PubMed  Google Scholar 

  7. Kirkwood MW, Peterson RL, Baker DA, Connery AK. Parent satisfaction with neuropsychological consultation after pediatric mild traumatic brain injury. Child Neuropsychology. 2017;23(3):273–83.

    Article  PubMed  Google Scholar 

  8. Farmer JE, Brazeal TJ. Parent perceptions about the process and outcomes of child neuropsychological assessment. Appl Neuropsychol. 1998;5(4):194–201.

    Article  CAS  PubMed  Google Scholar 

  9. Bjertnaes OA, Sjetne IS, Iversen HH. Overall patient satisfaction with hospitals: effects of patient-reported experiences and fulfilment of expectations. BMJ Quality & Safety. 2012;21(1):39.

    Article  Google Scholar 

  10. Al-Abri R, Al-Balushi A. Patient satisfaction survey as a tool towards quality improvement. Oman Medical Journal. 2014;29(1):3–7.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Nathorst-Böös J, Munck IME, Eckerlund I, Ekfeldt-Sandberg C. An evaluation of the QSP and the QPP: two methods for measuring patient satisfaction. Int J Qual Health Care. 2001;13(3):257–64.

    Article  PubMed  Google Scholar 

  12. Carr-Hill RA. The measurement of patient satisfaction. J Public Health Med. 1992;14(3):236–49.

    CAS  PubMed  Google Scholar 

  13. Sitzia J, Wood N. Patient satisfaction: a review of issues and concepts. Soc Sci Med. 1997;45(12):1829–43.

    Article  CAS  PubMed  Google Scholar 

  14. Williams B, Coyle J, Healy D. The meaning of patient satisfaction: an explanation of high reported levels. Soc Sci Med. 1998;47(9):1351–9.

    Article  CAS  PubMed  Google Scholar 

  15. Norman SM, Ford T, Henley W, Goodman R. A comparison of parent reported outcome with experience of services. Journal of Children's Services. 2016;11(2):157–69.

    Article  Google Scholar 

  16. Williams B. Patient satisfaction: a valid concept? Soc Sci Med. 1994;38(4):509–16.

    Article  CAS  PubMed  Google Scholar 

  17. Garratt AM, Solheim E, Danielsen K. National and cross-national surveys of patient experiences: a structured review. Oslo: Nasjonalt kunnskapssenter for helsetjenesten (Norwegian Knowledge Centre for the Health Services) 2008. Contract No.: Nr 7–2008.

  18. Danielsen K, Bjertnaes OA, Garratt A, Forland O, Iversen HH, Hunskaar S. The association between demographic factors, user reported experiences and user satisfaction: results from three casualty clinics in Norway. BMC family practice. 2010;11(1):73-.

  19. Granat T, Lagander B, Borjesson MC. Parental participation in the habilitation process - evaluation from a user perspective. Child Care Health Dev. 2002;28(6):459–67.

    Article  CAS  PubMed  Google Scholar 

  20. King GA, King SM, Rosenbaum PL. Interpersonal aspects of care-giving and client outcomes: a review of the literature. Ambulatory Child Health. 1996;2(2):151–60.

    Google Scholar 

  21. Arnadottir U, Egilson ST. Evaluation of therapy services with the measure of processes of care (MPOC-20): the perspectives of Icelandic parents of children with physical disability. Journal of Child Health Care. 2012;16(1):62–74.

    Article  PubMed  Google Scholar 

  22. Bjerre I, Larsson M, Franzon A, Nilsson M, Stromberg G, Westbom L. Measure of processes of care (MPOC) applied to measure parent's perception of the habilitation process in Sweden. Child Care Health Dev. 2004;30(2):123–30.

    Article  CAS  PubMed  Google Scholar 

  23. McConachie H, Logan S. Validation of the measure of processes of care for use when there is no child development Centre. Child Care Health Dev. 2003;29(1):35–46.

    Article  CAS  PubMed  Google Scholar 

  24. Holmboe O, Iversen HH, Hanssen-Bauer K. Determinants of parents' experiences with outpatient child and adolescent mental health services. International Journal of Mental Health Systems. 2011;5(1):22-.

  25. Hasnat M, Graves P. Disclosure of developmental disability: a study of parent satisfaction and the determinants of satisfaction. J Paediatr Child Health. 2000;36(1):32–5.

    Article  CAS  PubMed  Google Scholar 

  26. Bjørngaard J, Wessel Andersson H, Osborg Ose S, Hanssen-Bauer K. User satisfaction with child and adolescent mental health services. The International Journal for Research in Social and Genetic Epidemiology and Mental Health Services. 2008;43(8):635–41.

    Google Scholar 

  27. Kapp C, Perlini T, Jeanneret T, Stéphan P, Rojas-Urrego A, Macias M, et al. Identifying the determinants of perceived quality in outpatient child and adolescent mental health services from the perspectives of parents and patients. European Child & Adolescent Psychiatry. 2017;26(10):1269–77.

    Article  Google Scholar 

  28. Egilson ST. Parent perspectives of therapy services for their children with physical disabilities. Scand J Caring Sci. 2011;25(2):277–84.

    Article  PubMed  Google Scholar 

  29. Jansen SLG, Van Der Putten AAJ, Vlaskamp C. What parents find important in the support of a child with profound intellectual and multiple disabilities. Child Care Health Dev. 2013;39(3):432–41.

    Article  CAS  PubMed  Google Scholar 

  30. Terwiel M, Alsem MW, Siebes RC, Bieleman K, Verhoef M, Ketelaar M. Family-centred service: differences in what parents of children with cerebral palsy rate important. Child Care Health Dev. 2017;43(5):663–9.

    Article  CAS  PubMed  Google Scholar 

  31. Groleger Sršen K, Vidmar G, Sočan G, Zupan A. Parental evaluation of processes of care in relation to the child, parent and family characteristics. Int J Rehabil Res. 2014;37(3):220–8.

    Article  PubMed  Google Scholar 

  32. Rahmqvist M, Bara A-C. Patient characteristics and quality dimensions related to patient satisfaction. Int J Qual Health Care. 2010;22(2):86–92.

    Article  PubMed  Google Scholar 

  33. King S, King G, Rosenbaum P. Evaluating health service delivery to children with chronic conditions and their families: development of a refined measure of processes of care (MPOC-20). Children's Health Care. 2004;33(1):35–57.

    Article  Google Scholar 

  34. Batbaatar E, Dorjdagva J, Luvsannyam A, Savino MM, Amenta P. Determinants of patient satisfaction: a systematic review. London, England. 2017. p. 89–101.

  35. Hagen AK, Bjorbækmo WS. Parents evaluation of the processes of care in child rehabilitation: a reliability study of the Norwegian translation of MPOC-20. Child Care Health Dev. 2012;38(1):48–53.

    Article  PubMed  Google Scholar 

  36. Bleich SN, Ozaltin E, Murray CKL. How does satisfaction with the health-care system relate to patient experience? Bull World Health Organ. 2009;87(4):271.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Glynn L. The effect of health status on patients' satisfaction with out-of-hours care provided by a family doctor co-operative. Fam Pract. 2004;21(6):677–83.

    Article  CAS  PubMed  Google Scholar 

  38. Davis K, Gavidia-Payne S. The impact of child, family, and professional support characteristics on the quality of life in families of young children with disabilities. J Intellect Develop Disabil. 2009;34(2):153–62.

    Article  Google Scholar 

  39. Trute B, Hiebert-Murphy D. The implications of "working Alliance" for the measurement and evaluation of family-centered practice in childhood disability services. Infants Young Child. 2007;20(2):109.

    Article  Google Scholar 

  40. Bairati I, Meyer F, Gueye CBD, Desmarais C, Rouleau N, Sylvestre A. Factors influencing parent satisfaction with preventive health services for the early detection of speech and language delay in preschool children.(Report). Open Journal of Preventive Medicine. 2011;1(3):135.

  41. Mah JK, Tough S, Fung T, Douglas-England K, Verhoef M. Parents' global rating of mental health correlates with SF-36 scores and health services satisfaction. Quality of Life Research. 2006;15(8):1395–401.

    Article  PubMed  Google Scholar 

  42. King G, King S, Rosenbaum P, Goffin R. Family- centered caregiving and well- being of parents of children with disabilities: linking process with outcome. J Pediatr Psychol. 1999;24(1):41–53.

    Article  Google Scholar 

  43. Grøholt E-K, Nordhagen R, Heiberg A. Mestring hos foreldre til barn med funksjonshemninger (coping in parents of children with disabilities). Tidsskrift for Den norske legeforening. 2007;127(4):422–6.

    PubMed  Google Scholar 

  44. King G, Tucker MA, Baldwin P, Lowry K, Laporta J, Martens L. A life needs model of pediatric service delivery: services to support community participation and quality of life for children and youth with disabilities. Physical & occupational therapy in pediatrics. 2002;22(2):53–77.

    Google Scholar 

  45. Draper M, Cohen P, Buchan H. Seeking consumer views: what use are results of hospital patient satisfaction surveys? Int J Qual Health Care. 2001;13(6):463–8.

    Article  CAS  PubMed  Google Scholar 

  46. Halvorsen M, Mathiassen B, Myrbakk E, Brøndbo PH, Sætrum A, Steinsvik OO, et al. Neurodevelopmental correlates of behavioural and emotional problems in a neuropaediatric sample. Res Dev Disabil. 2019;85:217–28.

    Article  PubMed  Google Scholar 

  47. Halvorsen M, Aman MG, Mathiassen B, Brøndbo PH, Steinsvik OO, Martinussen M. Psychometric properties of the Norwegian aberrant behavior checklist and diagnostic relationships in a neuro-pediatric sample. J Ment Health Res Intellect Disabil. 2019;12(3–4):234–55.

    Article  Google Scholar 

  48. Kjærandsen KS, Handegård BH, Brøndbo PH, Halvorsen MB. Parental mental health screening in a neuropaediatric sample: psychometric properties of the Everyday Feeling Questionnaire and variables associated with parental mental health. J Appl Res Intellect Disabil. 2021;34(2);648–58. 

  49. Garratt AM, Bjertnæs ØA, Barlinn J. Parent experiences of paediatric care (PEPC) questionnaire: reliability and validity following a national survey. Acta Paediatr. 2007;96(2):246–52.

    Article  PubMed  Google Scholar 

  50. Garratt AM, Bjertnaes OA, Holmboe O, Hanssen-Bauer K. Parent experiences questionnaire for outpatient child and adolescent mental health services (PEQ-CAMHS outpatients): reliability and validity following a national survey. Child Adolesc Psychiatry Ment Health. 2011;5.

  51. Sjetne IS, Bjertnæs ØA, Iversen HH, Olsen RV. Pasienterfaringer i spesialisthelsetjenesten : et generisk, kort spørreskjema (Patient experiences in specialist health care: A generic, short questionnaire). Oslo: Nasjonalt kunnskapssenter for helsetjenesten (Norwegian Knowledge Centre for the Health Services); 2009.

  52. Bjertnæs ØA, Garratt A, Helgeland J, Holmboe O, Dahle KA, Hanssen-Bauer K, et al. Foresattes vurdering av barne- og ungdomspsykiatriske poliklinikker (parental evaluation of child and adolescent outpatient psychiatric clinics). Tidsskrift for Den norske legeforening. 2008;118(9):1041–5.

    Google Scholar 

  53. Goodman R, Ford T, Richards H, Gatward R, Meltzer H. The development and well- being assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiat. 2000;41(5):645–55.

    Article  CAS  PubMed  Google Scholar 

  54. Uher R, Goodman R. The everyday feeling questionnaire: the structure and validation of a measure of general psychological well-being and distress. Soc Psychiatry Psychiatr Epidemiol. 2010;45(3):413–23.

    Article  PubMed  Google Scholar 

  55. Mann J, Henley W, O’mahen H, Ford T. The reliability and validity of the everyday feelings questionnaire in a clinical population. J Affect Disord. 2013;148(2–3):406–10.

    Article  PubMed  Google Scholar 

  56. Shaffer D. A Children's global assessment scale (CGAS). Arch Gen Psychiatry. 1983;40(11):1228–31.

    Article  CAS  PubMed  Google Scholar 

  57. Hanssen-Bauer K, Aalen O, Ruud T, Heyerdahl S. Inter-rater reliability of clinician-rated outcome measures in child and adolescent mental health services. Adm Policy Ment Health Ment Health Serv Res. 2007;34(6):504–12.

    Article  Google Scholar 

  58. Sparrow S, Cicchetti D, Balla DA. Vineland adaptive behavior scales (2nd ed.). Norwegian manual supplement. Stockholm: Pearson Assessment; 2011.

  59. Bjertnaes OA, Lyngstad I, Malterud K, Garratt A. The Norwegian EUROPEP questionnaire for patient evaluation of general practice: data quality, reliability and construct validity. Fam Pract. 2011;28(3):342–9.

    Article  PubMed  Google Scholar 

  60. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. ed. Hillsdale, N. J: Laurence Erlbaum; 1988.

  61. Konerding U. Which kind of psychometrics is adequate for patient satisfaction questionnaires? Patient Prefer Adherence. 2016;10:2083–90.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Willems S, De Maesschalck S, Deveugele M, Derese A, De Maeseneer J. Socio-economic status of the patient and doctor–patient communication: does it make a difference? Patient Educ Couns. 2005;56(2):139–46.

    Article  CAS  PubMed  Google Scholar 

  63. Einfeld SL, Ellis LA, Emerson E. Comorbidity of intellectual disability and mental disorder in children and adolescents: A systematic review. Journal of Intellectual and Developmental Disability, 2011, Vol36(2), p137–143.

  64. Ezpeleta L, Granero R, Osa NDL, Doménech Massons JM, Guillamón CN. Perception of need for help and use of mental health services in children and adolescents. Do they share the same predictors? Psicothema. 2002;14(3):532–9.

    Google Scholar 

  65. Carr-Hill RA. The measurement of patient satisfaction. J Public Health Med. 1992;14(3):236.

    CAS  PubMed  Google Scholar 

  66. Services NMoHaC. The coordination reform. Proper treatment - at the right place and right time. In: services NMoHaC, editor. Oslo: Norwegian Directorate of Health; 2009.

  67. Health TNDo. Veileder om rehabilitering, habilitering, individuell plan og koordinator. [guidelines for rehabilitation, individual care plan and coordinator]. In: health TNDo, editor. Oslo: The Norwegian Directorate of Health; 2015.

  68. Robert G, Cornwell J. Rethinking policy approaches to measuring and improving patient experience. Journal of Health Services Research & Policy. 2013;18(2):67–9.

    Article  Google Scholar 

  69. Benson T, Potts HWW. A short generic patient experience questionnaire: howRwe development and validation. BMC Health Services Research. 2014;14(1).

Download references

Acknowledgements

The authors would like to thank the participants who contributed to the dataset.

Funding

This study was supported by the Northern Norway Regional Health Authority (PFP1178–14). The study was also supported in part by the Norwegian Directorate of Health and the Regional Centre for Child and Youth Mental Health and Child Welfare North at UiT The Arctic University of Norway, Tromsø, Norway.

Author information

Authors and Affiliations

Authors

Contributions

KSK planned and wrote the paper and performed statistical analyses. PHB critically revised the manuscript draft and approved the final version of the manuscript. MH is the project manager; she critically revised the manuscript draft, and approved the final version of the manuscript as well.

Corresponding author

Correspondence to Katarina Smejda Kjærandsen.

Ethics declarations

Ethics approval and consent to participate

The study was approved by the Data Protection Official in the health trusts. The participants were informed about the aims of the study, privacy, and the right to withdraw from the project. Written consent to participate was obtained from caregivers and adolescents. For children under the age of 12 years, only caregivers’ consent was collected. The Norwegian Regional Committees for Medical and Health Research Ethics evaluated the project and determined that their approval was not required.

Competing interests

The authors have no competing interests to declare.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kjærandsen, K.S., Brøndbo, P.H. & Halvorsen, M.B. Determinants of caregiver satisfaction with child neurodevelopmental assessment in neuropaediatric clinics. BMC Health Serv Res 21, 139 (2021). https://doi.org/10.1186/s12913-021-06153-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-021-06153-5

Keywords