Identification of quality gaps in healthcare services using the SERVQUAL instrument and importance-performance analysis in medical intensive care: a prospective study at a medical center in Taiwan

Abstract

Background

Assessing patients’ expectations and perceptions of health service delivery is challenging. To understand the service quality in intensive care units (ICUs), we investigated the expected and perceived service quality of ICU care.

Methods

We conducted this study at an ICU of a university-affiliated medical center in Taiwan from April to September 2019. Admitted patients or their family members responded to a questionnaire survey adopted from the SERVQUAL instrument consisting of 22 items in five dimensions. The questionnaire was provided on ICU admission for expectation and before ICU discharge for perception. We analyzed the quality gaps between the surveys and applied important-performance analysis (IPA).

Results

A total of 117 patients were included (62.4% males, average age: 65.9 years, average length of stay: 10.1 days, and 76.9% survival to ICU discharge). The overall weighted mean scores for the surveys were similar (4.57 ± 0.81 and 4.58 ± 0.52, respectively). The ‘tangibles’ dimension had a higher perception than expectation (3.99 ± 0.55 and 4.31 ± 0.63 for expectation and perception, respectively, p < 0.001). IPA showed that most of the items in ‘reliability,’ ‘responsiveness’ and ‘assurance’ were located in the quadrant of high expectation and high perception, whereas most of the items in ‘tangibles’ and ‘empathy’ were located in the quadrant of low expectation and low perception. One item (item 1 for ‘tangibles’) was found in the quadrant of high expectation and low perception.

Conclusions

The SERVQUAL approach and IPA might provide useful information regarding the feedback by patients and their families for ICU service quality. In most aspects, the performance of the ICU satisfactorily matched the needs perceived by the patients and their families.

Background

Measurement of the quality of healthcare is essential for healthcare systems [1, 2]. While a variety of common structure, process, and outcome measurements on the quality of healthcare delivery have been applied [1, 3], one emerging focus is the assessment of the service quality of healthcare by determining how well it meets expectations [4,5,6]. The introduction of the concept of service quality [7, 8] into the healthcare sector has been an interesting topic. However, assessing the patients’ expectations and perceptions of health service delivery remains challenging. In the past decades, satisfaction surveys have been commonly conducted by healthcare systems to understand the quality from the viewpoint of the patients and visitors, allowing the provider to understand the association between medical service quality and patient satisfaction [9]. Researchers have suggested that the provider should compare the expected and perceived service quality to identify gaps between them [7], apply the feedback from patients to improve the quality of care [10], assess the real experiences of medical care [11], and perceptions of quality as provided by the patients [12]. As service quality has become an important corporate strategy for healthcare organizations [6], measuring perception may be an important approach [13], and understanding the gap between expectation and perception may provide additional insight into the background of traditional satisfaction surveys.

Several reports have discussed this gap measurement approach in assessing healthcare [5, 14, 15], and studies on the service quality of healthcare in Taiwan have also been reported [16,17,18,19]; however, reports on service quality in the critical care setting remain limited [20]. The clinical and care settings of intensive care units (ICUs) are characterized by the increased severity of the patients’ illness and the marked abundance of care and treatment modalities, making it difficult for the patients and their family members to provide their opinions about the experience during the patients’ stay. Beyond satisfaction, additional tools for approaching the gap assessment might be needed.

Two currently available tools used to understand expectation and perception in service research are the SERVQUAL instrument [21] and importance-performance analysis (IPA) [22]. The SERVQUAL model is a multi-dimension survey instrument consisting of multiple items proposed by Parasuraman et al. to understand the customers’ expectations and perceptions for service and identify service gaps to improve service [21]. IPA was originally proposed by Martilla and James [22] as a technique for measuring attribute importance and performance to identify service strengths and weaknesses and develop effective marketing programs. With a two-dimensional matrix, IPA uses importance as an X-axis and performance as a Y-axis to categorize items or attributes into four quadrants, and items located in different quadrants may represent different implications for managerial actions [22]. IPA has been used in general practice [23] and pediatric healthcare [12], and the SERVQUAL instrument has been used to measure patients’ opinions of nursing care [16, 24] as well as a variety of hospital services [25,26,27] and various healthcare environments [19, 28,29,30]. A newer hierarchical model of health service quality has also been developed [6], contributing to increased academic interest in satisfaction and service quality. There have been reports focusing on specific care settings about the service quality of healthcare; however, few reports from Western or East Asian countries have applied the combination of these two instruments to evaluate service quality for patients in a critical care setting [31, 32]. In our previous work [20], we adopted SERVQUAL and IPA for the settings of the Respiratory Care Center, a dedicated unit for caring long-term ventilator-dependent patients [20]. However, the care setting of ICU, with patients in more acute and critical conditions, might render different perspectives for the patients and their families. Therefore, this study aimed to understand the gaps in service quality in the medical critical care setting to explore opportunities for improvement further, with the research question of ‘What are the significant gaps in service quality of healthcare in the ICU?’

Methods

Study design and setting

This was a prospective cross-sectional study conducted from April to September 2019 at a medical ICU of National Taiwan University Hospital, a university-affiliated public medical center in Taiwan. This hospital is a public university-affiliated tertiary medical center consisting of about 2300 beds in northern Taiwan, with about 6400 full-time healthcare workers. The hospital is regulated by the National Health Insurance system, covering more than 95% of expenses. Accordingly, the patients come from all socioeconomic groups and are required to pay only a small portion of the healthcare fee, which has a fixed ceiling amount in the inpatient setting, even for admission to the ICU. The ICU in this study consists of 15 beds, with about 40 patients admitted every month, and cares for patients with a variety of critical medical illnesses. Visitors to the ICU, who are usually family members of the patients, are allowed to visit during three time periods every day in the morning, afternoon, and evening, with each period lasting for 1 h. During the visits, the family members can interact with the on-site healthcare workers and the patient and discuss their condition and planning related to care. The family members can also provide feedback or requests during the visits.

Participants

The investigators screened all patients upon admission to the ICU. Those who were able to provide consent to participate in the study within 48 h after admission were considered eligible for the study. Patients were excluded if they were younger than 20 years of age, or if they had no surrogate in case they could not participate in the questionnaire.

The questionnaire

The modified SERVQUAL scale has been shown to have adequate reliability and validity [21]. This instrument was generic, and adoption into the real-world setting was required [21]. In our previous study in another care setting, we had adopted this instrument and demonstrated its internal validity and reliability as translated into Chinese for a domestic setting [20]. In this study, forward translation of the scale into Chinese was performed by two of the study authors (SJL and JSJ) who are knowledgeable about healthcare terminology and the contents of the questionnaire in both English and Chinese. All experts involved in this study discussed the translated Chinese questionnaire to resolve any ambiguities and discrepancies until a general agreement was reached regarding the content. Backward translation of the scale into English was performed by another two physicians of the ICUs of this hospital who were blinded to the questionnaire. The study researchers then compared the original and translated versions, and the Chinese version was further revised and modified until the final version was agreed upon by consensus. The universal agreement score (scale-level content validity index divided by the number of items) was 1.0 [33]. The expectation and perception sections consisted of a structured questionnaire consisting of 22 statements scored using a Likert-type five-point scale in five dimensions (tangibles, reliability, responsiveness, assurance, and empathy). In addition, point-allocation questions were provided [21] (Table 1). In this study, the questionnaire was provided on admission to the ICU for the expectation section, and within 2 days before discharge from the ICU for the perception section and point-allocation for the service quality. In addition to the SERVQUAL items, the questionnaire also obtained the responder’s gender, age, profession, educational level, religion, relationship with the patient in the family, and their prior experience with family members treated at the ICU. The internal consistency of this instrument for each dimension was measured, with Cronbach’s values of 0.932, 0.931, 0.930, 0.930, and 0.930 for the tangibles, reliability, responsiveness, assurance, and empathy dimensions, respectively. In factor analysis, the Kaiser-Meyer-Olkin test value was 0.869, and Bartlett’s Test of Sphericity showed a chi-square of 1436.2 (p < 0.001), suggesting good sampling adequacy. Confirmatory factor analysis using the Varimax method [34] showed that the extracted factors contributed to 67.2% of the variance.

Table 1 Items of the SERVQUAL scale questionnaire used in this study^a

An investigator performed daily screening for the patients potentially eligible for inclusion in the survey. The patients or their family members were contacted by the study investigators to provide written informed consent once a patient has been identified as being eligible, and then responded to the survey. The responders then completed the questionnaire survey using a desktop computer or printed form by providing anonymous scores and de-identified information. It would be expected that most of the responders would not be the patients themselves; in such cases, the investigators contacted the family members who were familiar with the patient and had the most understanding of the services provided at the ICU. It was preferred, but not mandatory that the same person responded to both surveys.

In addition to the questionnaire, the investigators also collected the following data from the patients: gender, age, admission and discharge severity scores, co-morbidities, the status of ventilator use, length of stay, and outcome. Based on our experience from the prior study [20], given a potential population size of 120 with an estimated survey score of 4.5 ± 0.65 (95% confidence interval = 4.38–4.62), we calculated the sample size to be at least 96 to reach sufficient power of 0.8 to estimate the mean score.

Statistical analysis

We first performed a descriptive analysis of the patients and respondents regarding their demographics, clinical and social characteristics. Categorical data of the demographic, clinical, and socioeconomic features were presented as number and percentage, and continuous data were presented as mean and range. Data on SERVQUAL scores were expressed as mean and SD. The mean scores of each dimension were compared using the independent sample t-test. Mean scores of the dimensions in the survey were compared using analysis of variance (ANOVA).

The investigators plotted four-quadrant distributions using IPA [22] after having obtained the expectation and perception scales and assigned expectation and perception as the Y-axis of importance and X-axis of performance according to the relative mean scores of each item paired to the overall medians [35]. Item pairs in the upper right quadrant suggested high expectation and high perception, whereas those in the upper left quadrant suggested high expectation and low perception. The items in the lower left quadrant depicted low expectation and low perception, whereas those located in the lower right quadrant indicated low expectation but high perception. Any item located in the upper left quadrant, if found, was considered to be a priority for improvement with regards to service quality [22].

We then used binary logistic regression analysis to explore the factors associated with the gaps in service quality identified in the comparisons of survey data. Statistical analyses were performed using SPSS software (SPSS 22.0, NY, USA). A p-value < 0.05 was considered to be statistically significant.

Results

Demographic and clinical features

During the study period, 144 consecutive patients were admitted to the ICU, of whom 27 were excluded due to a lack of consent (n = 19), death within 24 h (n = 3), being transferred out within 24 h (n = 3), and readmission to this same ICU (n = 2). A total of 117 persons, including 16 admitted patients themselves and 101 members from their families, served as the respondents participating in the surveys for the on-admission expectation section; upon ICU discharge, 99 responders, including 20 patients, participated in the surveys for perception. Table 2 summarizes the demographic and clinical characteristics of the patients and the demographic and socioeconomic features of the respondents. The patients were old, more were male, and the majority (76.9%) survived to discharge from the ICU; the average length of stay at the ICU was 10.1 days. The respondents of the surveys were more commonly female (53.0%), and most were the children (39.3%) and spouse (28.2%) of the patients. Data regarding the demographics, clinical features, and questionnaire results are also available as a supplementary material (see the ‘Research Data’ file of the Supplementary Material).

Table 2 Demographic data and clinical features of the patients admitted to the intensive care unit (n = 117) and the respondents of the survey

Expectations and perceptions for service quality of healthcare

Table 3 shows comparisons between the perception on ICU admission and expectation scores upon ICU discharge. For the ‘tangibles’ dimension, the perception score was significantly higher than that of expectation (p < 0.001), whereas the scores of the two surveys were similar for the other four dimensions and the overall scores. The perception section had an overall score of 4.56 ± 0.52, and the ‘tangibles’ dimension (4.31 ± 0.63) had a significantly lower perception score among the five dimensions (ANOVA, p < 0.001). The other four dimensions had similar mean scores (ANOVA for the four dimensions other than ‘tangibles,’ p = 0.840), even though the ‘reliability’ score (4.65 ± 0.51) was the highest.

Table 3 Scores and gaps in the expectation and perception of service quality for each item based on the SERVQUAL scale

Table 4 summarizes the assessment of the data distribution of the questionnaire. All items had a left-skewed distribution in the two surveys, with slightly higher numbers of items having a skewness number lower than − 1 in expectation (13 of 22) than in perception (9 of 22). In expectation, the dimensions of reliability, responsiveness, and assurance were significantly left-skewed. In perception, the dimensions of reliability, responsiveness, assurance, and empathy were significantly left-skewed. More items with a kurtosis were in expectation (4 of 22) than in perception (5 of 22) (Table 4). For the dimensions, kurtoses were noted in reliability and assurance in expectation, and assurance and empathy in perception.

Table 4 Assessment of data distribution for the questionnaire

Importance-performance analysis

Figure 1 shows the diagram of the IPA results based on the mean scores of each item by on-admission expectations (the Y-axis) and on-discharge perception (the X-axis). The diagram shows that most of the items in ‘reliability,’ ‘responsiveness’ and ‘assurance’ were located in the upper right quadrant (high expectation and high perception). In contrast, most of the items in ‘tangibles’ and ‘empathy’ were located in the lower-left quadrant (low expectation and low perception). One item (item 1) was located in the upper left quadrant (high expectation and low perception); there were no items in the right lower quadrant (low expectation and high perception) (Fig. 1).

Fig. 1

Importance-performance analysis of the expectation on admission and perception upon discharge from the ICU

Factors associated with significant gaps in service quality

Table 5 summarizes the results of multivariate binary logistic regression analyses for the gaps in service quality related to ICU care. As shown in Table 3, there were no significant differences in the overall scores (4.49 ± 0.36 vs. 4.56 ± 0.52, p = 0.219) or weighted overall scores (4.57 ± 0.81 vs. 4.58 ± 0.52, p = 0.954) between expectation and perception; therefore, we decided not to perform regression analysis based on the overall score. For the ‘tangibles’ dimension, which had the lowest score for perception, most of the demographic characteristics were not associated with a tangibles score <  4.31 (the mean value in this survey) in perception in the multivariate analysis (Model 1). Further analysis including only variables with a p-value < 0.3 in Model 1 revealed that ‘responder older than 65 years’ was the only factor associated with a lower dimension score (tangibles score <  4.31) in perception (OR = 0.248, 95% CI = 0.091–0.678, p = 0.007) (Model 2).

Table 5 Multivariate logistic regression analysis for factors associated with a low tangibles score (< 4.31)

Discussion

In this study, to answer the research question, ‘What are the significant gaps in service quality of healthcare in the ICU?’, we used the SERVQUAL instrument to assess the service quality of healthcare in an ICU setting and understand the gaps between perception and expectation of the patients and their families. We also used IPA to explore the priorities for improvement in the service quality of healthcare. IPA showed excellent matching between performance and importance, with only one item pair of ‘tangibles’ located in the low performance-high importance quadrant. In general, the medical ICU in this study showed a performance that matched the needs perceived by the patients and their families.

Patient-centeredness has been increasingly advocated as an important domain of the assessment of healthcare performance in healthcare [36]; however, conventional satisfaction surveys lack an understanding of the expectations of the patients and their families. Although IPA has been shown to be able to assess the quality gaps and priorities for improvement [14], few reports have investigated the use of SERVQUAL and IPA in a critical care setting in Western or East Asian countries, such as quality improvement priorities in myocardial infarction care [37]. However, performance and priorities vary significantly across healthcare systems [37], and it remains unclear whether one model or instrument is better, even though other models have also been used to assess the service quality of healthcare [17, 38,39,40]. Our study provides further information regarding the application of these instruments and methods in healthcare services, especially the critical care setting. In addition, the ICU in this study was dedicated to caring for adult patients. The healthcare workers were not familiar with the care of patients younger than 20 years, who are mostly cared for at a children’s hospital in this healthcare system. Perceptions of responders related to younger patients may be quite different from those related to adult patients. Therefore, this study did not include any patients under 20 years of age.

Several implications of this study can be inferred. Traditionally, healthcare units use satisfaction surveys to identify areas that can be improved and target the dimensions with lower scores. In contrast, surveys using the expectation-perception model, and further deploying IPA, may provide more information regarding gaps in service quality. Therefore, the survey results may provide units with a more realistic understanding to prioritize goals to improve the service quality. The instrument used in this study was shown to have adequate validity; therefore, there is an opportunity to conduct a long-term survey with IPA and compare the results with the traditional satisfaction surveys as a research instrument for real-world healthcare service. Our data showed fair reliability and validity of the instrument with adequate sampling; therefore, we recommend the use of the SERVQUAL instrument and IPA to obtain feedback from family members in the critical care setting.

The perception of the healing environment may be important to the patient as well as the family [41]. In this study, the family members provided most of the responses to the questionnaire rather than the patients. Nevertheless, at least some of their opinions may still provide a valuable assessment of the service quality of a critical care setting. Several items, including the environment of care, the response to the need expressed by the family members, and the promise to keep planned tasks, may also be adequately reflected by family members. In Taiwan, family members are often highly involved in decision making and communication regarding the care process, and this is also shown by the high engagement in the participation in this survey by the family members. In addition, the severity of illness, indicated by the need for intubation and mechanical ventilation on admission to the ICU (about 75% of the patients), essentially precluded the possibility of eliciting valid expectation responses on admission. It has been suggested that family support may improve satisfaction in the critical care setting [42] and that the family may play an important role in patient participation in the ICU even when the patients are unable to participate [43].

Our finding that the quality gap found by SERVQUAL was not located in the upper left or lower right quadrant of IPA suggests that these two instruments may serve as complementary measures for monitoring the service quality of healthcare. A detailed understanding of unmet needs, as expressed by the patients and their families, might provide an opportunity to focus on important strategies to improve the service quality of healthcare. IPA has been advocated as a guide to improving service quality [44], and it has been suggested to be better than measuring the performance alone to predict healthcare quality and for further improvement [12]. The application of IPA has also been reported in hospitals in the East Asian region [45], and several studies that have used both SERVQUAL and IPA have been conducted in the healthcare setting [45,46,47,48,49], indicating a trend for greater focus on patient-orientedness. The results of SERVQUAL in this study were not compared with the results of a satisfaction survey in this unit, which was conducted periodically but not for every hospitalized patient. Therefore, we could not evaluate correlations between the measured gaps and satisfaction level regarding every dimension of the service quality of healthcare. For the ICU setting in this study, the services described in the items included in the survey questionnaire were a mixture of healthcare and administrative services, and the patients might be more likely to respond to the expectation of care and experience of received care, while the family respondents might be more likely to provide opinions about the administrative services.

In our multivariate analysis, we found that several patient and responder features were associated with quality gaps. Our finding that the older (age > 65 years) responders to the questionnaire tended to give a higher score in the tangibles dimension is comparable with previous studies. The outcome of death at ICU discharge was negatively associated with an increased expectation, suggesting that the family may care more about the convenience of visiting the patient when there was a favorable prognosis. Therefore, ICUs may need to respond to the need for family visits, especially when the patient has a favorable prognosis [50]. This study aimed to search for quality gaps, and this was measured by comparing perception with expectation. We found that the ‘tangibles’ dimension was the only one with a significant gap.

There are several limitations to this study. First, most (about 86%) of the responders of the surveys were family members of the patients. In addition, while there was only one responder per survey, it was not necessarily the same responder for both expectation and perception. The high likelihood of diversity of the responders and lack of paired responses may also have substantially increased the complexity of interpretation of the data. Although this was expected, as stated in the Methods section to reflect the real-world challenge in conducting surveys for ICU patients, there is a need for valid methods to understand better the expectation and perception of patients at an ICU. It is also necessary to further increase the sample size of respondents, especially the patients. Second, adequate responses to the survey questions require enough time and interaction with the healthcare workers and the care environment. However, family members may have had limited interactions in this regard. It may also be very difficult to obtain adequate responses from patients in an ICU because of their unstable physiological status, the use of sedatives and analgesics, and the use of mechanical ventilation in a bed-bound scenario. It would therefore be difficult for the patients to observe their care environment and have interactions with healthcare workers to provide adequate answers, and most of the responses may be the result of recollection. The number of responses from the hospitalized patients was limited (n = 16); therefore, to understand the expectations and perceptions of patients, further studies with larger patient populations are required. Similarly, most of the included patients in this study survived to discharge from the ICU, so that the family may have indicated a more satisfactory experience. Third, this study was conducted at a single center; therefore, further studies are needed to assess the generalizability of our results. Nevertheless, while we anticipate a substantial variation in the service models provided in critical care globally, even for different units within the same healthcare organization, we believe that this survey model can be generalized to other ICU settings based on its feasibility. Application of the SERVQUAL instrument in the healthcare sector remains limited, and a newer model has been developed to measure health service quality [6], which has been validated in the healthcare setting. Finally, although we applied IPA to the analysis of data obtained from SERVQUAL surveys, there is a need for a solid basis for expectation and perception of the SERVQUAL to be represented by the importance and performance of the IPA, even though there have been several reports of the combination of SERVQUAL and IPA in the area of healthcare [45,46,47,48,49]. Therefore, the interpretation of IPA findings should be limited to compare the expectation and perception expressed by the responders of the SERVQUAL survey. Further studies are needed to investigate and compare the practical implications of these instruments to understand the gaps in the service quality of healthcare.

Conclusions

In conclusion, this study shows that the application of the SERVQUAL instrument and IPA might provide useful information regarding the feedback by ICU patients and their families regarding the service quality of ICU. In this study, reliable, responsive, and assured care in the ICU were the most important aspects of quality from the viewpoint of the patients and their families.