Assessing the quality of care in sick child services at health facilities in Ethiopia

Abstract

Background

Quality of care depends on system, facility, provider, and client-level factors. We aimed at examining structural and process quality of services for sick children and its association with client satisfaction at health facilities in Ethiopia.

Methods

Data from the Ethiopia Service Provision Assessment Plus (SPA+) survey 2014 were used. Measures of quality were assessed based on the Donabedian framework: structure, process, and outcome. A total of 1908 mothers or caretakers were interviewed and their child consultations were observed. Principal component analysis was used to construct quality of care indices including a structural composite score, a process composite score, and a client satisfaction score. Multilevel mixed linear regression was used to analyze the association between structural and process factors with client satisfaction.

Result

Among children diagnosed with suspected pneumonia, respiratory rate was counted in 56% and temperature was checked in 77% of the cases. A majority of children (92%) diagnosed with fever had their temperature taken. Only 3% of children with fever were either referred or admitted, and 60% received antibiotics. Among children diagnosed with malaria, 51% were assessed for all three Integrated Management of Childhood Illnesses (IMCI) main symptoms, and 4% were assessed for all three general danger signs. Providers assessed dehydration in 54% of children with diarrhea with dehydration, 17% of these children were admitted or referred to another facility, and Oral Rehydration Solution was prescribed for 67% while none received intravenous fluids. The number of basic amenities in the facility was negatively associated with the clients’ satisfaction. Private facilities, when the providers had got training for care of sick children in the past 2 years, had higher client satisfaction. There was no statistical association between structure, process composite indicators and client satisfaction.

Conclusion

The assessment of sick children was of low quality, with many missing procedures when comparing with IMCI guidelines. In spite of this, most clients were satisfied with the services they received. Structural and process composite indicators were not associated with client’s satisfaction. These findings highlight the need to assess other dimensions of quality of care besides structure and process that may influence client satisfaction.

Background

In spite of global progress in child survival, 5.4 million children died before the age of 5 years in 2017, whereof most deaths occurred in low- and middle-income countries [1]. Ethiopia was one of few low-income countries achieving the fourth Millennium Development Goal by a two-thirds reduction of under-five mortality from 1990 to 2015 [2]. Nonetheless, Ethiopia remains one of the Sub-Saharan countries with the highest burden of child deaths [3].

Universal health coverage has been recommended as a strategy to improve health of a population. The success of this strategy is, however, also dependent on the provision of good-quality health care [4]. Poor quality of care provided at health facilities may contribute to child mortality [4, 5]. Poor-quality care can also lead to other adverse outcomes, including lack of trust and confidence in the health system [6].

In an effort to improve the quality of child health services, the World Health Organization (WHO) and the United Nations International Children’s Emergency Fund (UNICEF) developed the Integrated Community Case Management (iCCM) of childhood illnesses strategy [7]. This strategy aims to reduce morbidity and mortality among children under the age of 5 years through improved health workers’ skills by training and supportive supervision; improved health systems, including equipment, supplies, organization of work and referral systems; and improved key family practices and child care at community and household levels [8].

In Ethiopia, health centers and some hospitals provide Integrated Management of Newborn and Childhood Illness (IMNCI) and at health post level these services are referred to as the Integrated Community Case Management (iCCM). The Health Extension Workers (HEW) at health posts manage pneumonia, diarrhea, malaria, and malnutrition and refer severe cases [9].

As many countries attempt to improve service delivery, there is an increased need to assess the quality of care at health facilities, in order to identify problems and to identify factors that could lead to better care. The concept of quality of health care, in view of its subjective nature, is difficult to define and consequently difficult to measure. Donabedian proposed the triad of structure, process, and outcome to evaluate the quality of health care [10]. Based on Donabedian’s framework, this study examined the quality of sick child services and determinants of client satisfaction at health facilities in Ethiopia. Few studies have assessed the quality of care provided in Ethiopia [11, 12]. However, none of these studies examined the association between structural and process quality dimensions and client satisfaction. Thus, this study aimed at evaluating the quality of care for sick children at Ethiopian health facilities by assessing all three components of the Donabedian framework: structure, process, and outcome.

Methods

Data source

Data used in this study came from the Ethiopian Service Provision Assessment Plus survey (SPA+). Data were collected from March to July, 2014. The survey included observations of services provided to sick children to assess to what extent providers adhered to accepted service delivery and quality standards. Exit interviews were conducted with mothers or caretakers of sick children, whose consultations had been observed. The exit interviews included questions on the clients’ perception of the service delivery environment. Detailed methodology on SPA+ has previously been published [13].

In this survey, all hospitals and a representative sample of health centers, clinics, and health posts were selected. In each facility, clients were systematically selected for observation based on the number of clients visiting the facility for each of the sick child services on the day of the survey. A maximum of five clients were observed for a maximum of three providers of sick child services, with a maximum of 15 observations in a given facility. Interviewers were attempting to conduct an exit interview with all caretakers of observed sick children.

Measurements

Based on Donabedian’s quality of care framework, this study assessed three aspects of the quality of care: structure, process, and outcome [14]. In addition, descriptive statistics on adherence to IMCI guidelines for the assessment, physical examination, and treatment of sick children were presented. Structural data came from facility inventory interviews, while process measurements primarily depended on direct observations of client-provider interactions. Information about the clients’ satisfaction, which is viewed as reflecting outcome, was based on exit interviews with mothers or caregivers.

Information about structure

Table 1 shows structure and process indicators, which were selected based on the WHO Service Availability and Readiness Assessment (SARA) reference manual [15]. The structural indicators of quality of care included the facilities’ management system, service availability, infrastructure, and equipment.

Table 1 Items used in defining the structure and process indicators for quality in sick child services in Ethiopia. Ethiopian Service Provision Assessment Plus Survey 2014

Information about process

The process indicators of quality of care consisted of both interpersonal and technical features of the provider-client interactions. The interpersonal characteristics include information provided to a caregiver and the technical characteristics of the provider-client interactions included symptoms checked and physical examination conducted. These were based on the IMCI guidelines [16] and were derived from direct observation of the care provided during the consultations with sick children (Table 1).

Information about client satisfaction

Client satisfaction with sick child services, representing the outcome, was measured in the exit interview using 11 questions about mothers’ or caretakers’ perceptions of the quality of care. Satisfaction was rated as an index of problems not encountered during the visit.

Statistical analyses

Principal component analysis was used to construct several indices including the structural composite score, the process composite score, and the client satisfaction score. In doing so, we generated the indices with the highest Cronbach’s alpha, keeping the maximum number of common variables. The scores were computed based on the first principle component, which explained the largest proportion of the total variance. The structure, process, and outcome quality of care indicators and their respective composite indices were computed. The responses of caregivers were aggregated into an index to measure satisfaction using principal components analysis. Cronbach’s alpha of 0.82 revealed that items co-varied and probably measured the same underlying concept, i.e., satisfaction. In this study, a high reliability implies that it measures client satisfaction, while low reliability indicates that it measures something else (or possibly nothing at all).

Multilevel mixed linear regression was used to analyze if structural and process indices were associated with client satisfaction. The fixed effects, which is a measure of association, and random effects that is a measure of variation for client satisfaction, were determined by considering the region as the level of variation in the multilevel mixed regression. Four models were fitted, Model I included structural indicators, Model II included process indicators, Model III included each of the structure and process items, and Model IV included structural and process composite indicator. In the regression analysis for client satisfaction, we controlled for provider level covariates that could affect client satisfaction. The Akaike’s information criterion (AIC) and Bayesian information criterion (BIC) were used to measure the model fit and complexity. The model with the smallest value of the information criterion was considered to be better. The STATA Statistical Software version 14.2 (Stata Corp LP, College Station, TX, USA) was used to conduct these analyses.

Results

A total of 1908 clients were observed and interviewed at 1165 health facilities (hospitals, health centers, clinics, and health posts). Facility data were linked to data regarding sick children, whose consultations were observed and their mothers or caregivers, who were interviewed (Fig. 1).

Fig. 1

Study flow for the analysis of determinants of client satisfaction in sick child services in Ethiopia. Ethiopian Service Provision Assessment Plus Survey 2014

Client characteristics

The clients’ characteristics are shown in Table 2. The majority of the clients’ observations and exit interviews were performed in Oromia region (21%) followed by Amhara region (15%). The majority was done in urban areas (72%), primarily in hospitals (46%) and health centers (38%).

Table 2 Distribution of observed sick child consultations and exit interviews with mothers or caretakers by background characteristics. Ethiopian Service Provision Assessment Plus Survey 2014

Description of sick child service quality attributes

Table 3 presents the description and comparisons of sick child service quality attributes across the different types of facilities. About half (53%) of the providers in the facilities had received sick child care training in the past 2 years. Among the structural indicators, 91% had a system to conduct routine management meetings, 82% had a system to collect client opinions, but only 33% used the IMCI mother’s card. The purpose of that card was to provide the mother with reminders of the key messages she had received from the health worker. Only 2% of the sick child care providers had used visual aids during the consultations. More than two thirds (67%) of the clients reported that waiting time to see the provider was not a problem, and 74% of the clients reported that cost of the services was not a problem.

Table 3 Sick child services quality attributes by facility type. Ethiopian Service Provision Assessment Plus Survey 2014

Mean satisfaction score

Figure 2 describes the mean satisfaction score by facility type. At national level the mean satisfaction score was 76%. Compared to mothers/caregivers who sought care at other types of facilities, those that sought care at health posts reported the lowest mean satisfaction score (71%).

Fig. 2

Satisfaction score (mean percent, 95% CI) on sick child care by facility type. Ethiopian Service Provision Assessment Plus Survey 2014

Assessments, examinations, and treatment of sick children, classified by diagnosis or major symptoms

Table 4 describes other process indicators for the management of sick children. Among children ultimately diagnosed with pneumonia, respiratory rate was counted in 56% of the cases and temperature was checked in 77%. Overall, 19% of children diagnosed with pneumonia were either hospitalized or referred elsewhere. Eighty-four percent were given some form of antibiotics (9% received injectable antibiotics, and 78% an oral antibiotic). Among the children diagnosed with bronchial spasm or asthma, 43% had their temperature checked, only 13% were checked for their respiratory rate, and 89% received antibiotics. Providers prescribed any antibiotics for 63% of children diagnosed with cough or other upper respiratory illness.

Table 4 Adherence to IMCI assessment, physical examination, and treatment among sick children. Ethiopian Service Provision Assessment Plus Survey 2014

A majority of children (92%) diagnosed with fever had their temperature taken. Only 3% of children with fever were either referred or admitted, and 60% received antibiotics.

Among children diagnosed with malaria, 51% were assessed for all three IMCI main symptoms, and 4% were assessed for all three IMCI general danger signs. In addition, temperature was assessed in 91%, and anemia was assessed in 39% of the children. Overall, providers either hospitalized or referred the child elsewhere in 2% of these cases and gave some form of antimalarial medicine to 36%. The providers gave some antibiotics in half (52%) of these cases.

There were two categories of diarrhea: (1) any diarrhea without dehydration, and (2) any diarrhea with dehydration. The providers assessed dehydration in 56% of cases in the first category and in 54% of cases in the second category. Only 1% of children in the first category and 17% of children in the second category were admitted or referred to another facility. Oral Rehydration Solution (ORS) was prescribed for 76% of children in the first category (diarrhea without dehydration), while none received intravenous fluids. Among children in the second category (diarrhea with dehydration), 67% were given ORS while none were put on intravenous fluids. More children in the first category (diarrhea without dehydration) were given zinc (13%) than children in the second category (8%).

Determinants of mothers or caretakers’ satisfaction with the quality of care provided to their sick children

Results from the multivariable regression models for the associations between structural and process quality and client satisfaction are presented in Table 5. A few results were consistent across the analyses. The number of basic amenities in the facility was negatively associated with clients’ satisfaction. Private facilities combined with providers’ sick child care training in the past 2 years were positively associated with client satisfaction. Structural and process composite indicators were not associated with client satisfaction with sick child health services.

Table 5 Multivariable analysis to identify factors associated with client’s satisfaction with sick child services. Ethiopian Service Provision Assessment Plus Survey 2014

Discussion

The assessment of sick children in Ethiopia was of low quality, especially in clinics and health posts, but most clients were satisfied with the service they received. Health posts and clinics scored higher in some of the satisfaction attributes (waiting time, health worker’s ability to discuss, and explanation on treatment provided) compared to hospitals and health centers. The average satisfaction score did not vary across facility types. Structural and process composite indicators were not associated with satisfaction of clients in sick child services.

This study is the first effort to assess the quality of care provided to sick children in Ethiopia combined with an analysis of the association of client satisfaction with a range of process and structural characteristics based on a nationally representative sample. The study included a large number of observations. The sampling design assumed that hospitals, health centers and clinics played a more important role than health posts in the national health system. Therefore hospitals, health centers, and private clinics were over-sampled to improve the precision of the survey in the assessment at these levels.

The study involved observations of client-provider interactions during data collection, making it susceptible to the so-called Hawthorn effect. This implies that behavior might have changed when being observed resulting in better performance than usual. The health workers, who were selected as observers, had prior experience of the IMCI strategy and guidelines. They were trained for 4 weeks, including practical demonstrations of assessments, physical examination, and treatment.

The three main symptoms in the IMCI assessments (difficulty breathing, diarrhea, and fever), and the three general danger sign (inability to eat or drink anything, vomiting everything, and febrile convulsion) [17] were frequently neglected in the assessment of sick children. Similar gaps in following the steps of the IMCI guidelines were observed in a multi-country study conducted in Namibia, Kenya, Tanzania and Uganda [18]. The methodology used in these studies was similar to our study. A child with general danger signs should be considered as a serious problem. Most children with a general danger sign need immediate referral or admission to hospital [17]. They may also need lifesaving treatments with injectable antibiotics, oxygen or other treatments, which may not be available in the facility.

All children with respiratory problems should have had their respiratory rate counted [19]. We found inadequate physical examination of cases with suspected pneumonia. In general, none of the upper respiratory illnesses should be treated with antibiotics [19]. In this study, an even higher amount of antibiotic use was seen in cases with cough or other upper respiratory symptoms. Similarly, inappropriate antibiotics prescription was observed in a study in Benin [20]. With the worldwide growing problems of antibiotic resistance, an appropriate use of antibiotics should be encouraged to ensure that these drugs are not overused.

A recent study revealed that facility infrastructure was poorly associated with quality of child health services [21], which indicates that availability of equipment and supplies does not guarantee good quality of care and satisfaction. However, it is difficult to provide good quality care without access to basic equipment [21, 22]. These findings are in line with our results, where the composite structural index was unrelated to client satisfaction. We also found that the number of amenities in a facility was negatively associated with satisfaction. In addition, a relatively low alpha value for the structural and process indicators might contribute to the poor association. A study conducted on facility assessment tools indicated that a quarter of quality measures were not assessed by any of the tools including SPA [23]. Our study did not show an association between the number of physical examinations performed on the sick child and client satisfaction. A study conducted in Paraguay showed a positive association between the number of examinations performed and patient satisfaction [24].

Patient satisfaction is an increasingly common component in estimating the quality of healthcare. A recently published study in Nigeria revealed that patient satisfaction measured at the facility immediately after a visit tends to be high, irrespective of the objective quality of the facility [25]. Another study in Nigeria also indicated that patient satisfaction must be complemented with additional objective measures [26]. This points at the need of better measures when moving forward to improve quality of care. With the new sustainable development goals and commitments for universal health coverage, it is essential that the quality of health care dimension should be added to the agenda.

Another limitation of our study was that outcomes were measured solely on client satisfaction and did not consider other health-related outcomes of the children. Client satisfaction is often biased by people’s own expectations of what constitutes good quality of care. Populations with low expectations are more likely to be satisfied with poor quality of care, undermining the demand side efforts to deliver high quality of care [27].

Our results have a number of policy implications. There was a poor association between structure and process composite indicator and client satisfaction. Other dimensions of quality of care that may influence client satisfaction should be included. The context within which healthcare is delivered is important for its quality [21]. There is also a need to improve health providers’ skills to carefully take the patient’s history and perform physical examination. This could be achieved through targeted training.

Conclusions

Hospitals and health centers scored higher than health posts in most of the equipment and infrastructure indicators as well as adherence to clinical guidelines. However, health posts and clinics score higher in some of the satisfaction items. Quality gaps were observed in assessments, examinations, and treatment provided to sick children. The study also revealed that satisfaction of clients in sick child services in Ethiopia was not associated with composite indicators of equipment, infrastructure, and adherence to clinical guidelines. These findings highlight the need to include the context of the health system, in addition to structural factors and process indicators that may influence quality of sick child care and client satisfaction.