Skip to main content

A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic

Abstract

Background

The balanced scorecard (BSC) has been implemented to evaluate the performance of health care organizations (HCOs). BSC proved to be effective in improving financial performance and patient satisfaction.

Aim

This systematic review aims to identify all the perspectives, dimensions, and KPIs that are vital and most frequently used by health care managers in BSC implementations.

Methods

This systematic review adheres to PRISMA guidelines. The PubMed, Embase, Cochrane, and Google Scholar databases and Google search engine were inspected to find all implementations of BSC at HCO. The risk of bias was assessed using the nonrandomized intervention studies (ROBINS-I) tool to evaluate the quality of observational and quasi-experimental studies and the Cochrane (RoB 2) tool for randomized controlled trials (RCTs).

Results

There were 33 eligible studies, of which we identified 36 BSC implementations. The categorization and regrouping of the 797 KPIs resulted in 45 subdimensions. The reassembly of these subdimensions resulted in 13 major dimensions: financial, efficiency and effectiveness, availability and quality of supplies and services, managerial tasks, health care workers' (HCWs) scientific development error-free and safety, time, HCW-centeredness, patient-centeredness, technology, and information systems, community care and reputation, HCO building, and communication. On the other hand, this review detected that BSC design modification to include external and managerial perspectives was necessary for many BSC implementations.

Conclusion

This review solves the KPI categorization dilemma. It also guides researchers and health care managers in choosing dimensions for future BSC implementations and performance evaluations in general. Consequently, dimension uniformity will improve the data sharing and comparability among studies. Additionally, despite the pandemic negatively influencing many dimensions, the researchers observed a lack of comprehensive HCO performance evaluations. In the same vein, although some resulting dimensions were assessed separately during the pandemic, other dimensions still lack investigation. Last, BSC dimensions may play an essential role in tackling the COVID-19 pandemic. However, further research is required to investigate the BSC implementation effect in mitigating the pandemic consequences on HCO.

Peer Review reports

Introduction

Evaluating the health care sector is quite challenging and complex. Unsatisfactory performance can result from long waiting times (WTs), inefficiency, dissatisfactory patients, and health care workers' (HCWs) burnout [1, 2]. Coronavirus disease 2019 (COVID-19) imposed further burdens on the health care system worldwide due to the limited capacity of hospital beds and the increased psychological stress of HCWs during the COVID-19 pandemic [3, 4]. There is still a lack of information that would help health care managers and policymakers in the era of COVID-19 to improve the delivery of health care quality and to learn for the future [5]. Higher pandemic burdens, such as HCW burnout and stress, will rise when health care organizations (HCO) lack plans and preparedness to strengthen their surge capacity and HCW resilience [6, 7].

Researchers have employed different tools for the performance evaluation (PE) of HCO. The most utilized PE tools were the International Organization for Standardization (ISO standards), Malcolm Baldrige National Excellence Model (MBNQA), European Foundation for Quality Management (EFQM) Excellence Model, Singapore Quality Award (SQA), Six Sigma, Data Envelopment Analysis (DEA), Pabon Lasso Model, and Balanced Scorecard (BSC) [8,9,10,11,12].

The World Health Organization (WHO) initiated the Performance Assessment Tool for Quality Improvement in Hospitals (PATH) in 2003. It aimed to develop a framework for the assessment of hospital performance. The resulting dimensions from this project were clinical effectiveness, efficiency, HCW orientation, responsive governance, safety, and patient-centeredness. However, studies have shown that there are still some gaps in this model and issues concerning the dimensions investigated [13, 14]. Additionally, the Organization for Economic Co-operation and Development (OECD) launched the Health Care Quality Indicator (HCQI) project in 2006; it aimed to develop key performance indicators (KPIs) to compare quality in health care at the international level and achieve international benchmarking. This project concluded that health care must be safe, effective, patient-centered, timely, efficient, equitable, acceptable, and accessible [15, 16].

Most of the abovementioned managerial tools mainly focused on the KPIs related to quality, efficiency, productivity, and timeliness dimensions [8,9,10,11,12, 17]. Each of these dimensions is considered a dimension at the internal perspective of the BSC, which consists of four perspectives: the internal process, customer, knowledge and growth perspectives, and financial perspectives [18]. Dimensions are described as collections of homogeneous or related KPIs. They are also referred to as diagnostic related groups (DRGs) [19], which have been proven to allow performance comparisons across hospitals and positively impact efficiency improvement [19].

The use of KPIs in the health care system before the pandemic has been beneficial for many reasons. First, the satisfaction rates of patients and HCWs were increased. Second, they lead to better efficiency, effectiveness, and financial performance and adapt to new technologies and ideas. Third, they lead to higher productivity and profitability [20,21,22]. In the pandemic, it is also crucial for HCO to track the performance of KPIs, which could draw faster attention to areas that require rapid responses and strengthening [6].

Most of the available PE models mainly focus on the internal perspective but lack coverage of the other dimensions or perspectives that are also important. BSC was considered different from the other managerial tools for two reasons. First, it offers a holistic approach to PE since it allows managers to highlight both financial and nonfinancial metrics. Second, the BSC is not only a planning or a PE tool. It is also a strategic managerial tool that assigns KPIs compatible with the HCO strategy [23, 24]. However, other PE tools, such as total quality management (TQM), lack these comprehensive properties [25]. The first generation of the BSC, unveiled by Kaplan and Norton in 1992, involved four perspectives: the financial, customer, internal process, and knowledge and growth perspectives, steered by the organizational vision and strategy [18]. See Fig. 1.

Fig.1
figure 1

Balanced Scorecard Perspectives [18]

Later, the second generation of BSCs was developed to include strategic maps, in which cause-effect cascades between perspectives or KPIs were inspected [23]. In the third generation of BSCs, a destination statement was incorporated, which evokes where the organization plans to go within a time horizon and the action plans to achieve each targeted objective [24]. In health care, Duke Children's Hospital in the United States of America (USA) was the first to implement the BSC in 1997. Figure 2 represents the strategic map of Duke University's health system. As a result, the hospital converted 11 million American dollar losses into four million profits after four years of implementation [25]. Since then, the BSC has gained increasing attention, and many HCOs in high-income countries and low- and middle-income countries have strategically utilized the BSC to develop their organizations [26,27,28,29,30].

Fig. 2
figure 2

Duke University Health System Strategic Map [31]

A recent systematic review [32] proved that BSC implementations were effective in improving the financial performance of HCOs, elevating patient satisfaction rates, and to a lesser extent improving HCW satisfaction rates. Another review [26] revealed that there had been a lack of engaging stakeholders in BSC implementations, such as engaging patients and HCWs. However, researchers have pointed to the importance of patient and HCW engagement in the process of PE and delivery improvement [27,28,29]. The rest of the BSC reviews [26, 30, 33,34,35,36,37,38,39,40,41,42] focused only on the general narration of the BSC perspectives and subdimensions used. Moreover, none of them summarized the perspectives or dimensions of BSC based on their importance or frequency of use by health care managers. In other words, all the previous systematic reviews lack a systematic methodological categorization of perspectives, dimensions, and KPIs.

In correspondence with this research gap, this review aims at a) finding and recategorizing all the perspectives, dimensions, and KPIs that were employed in BSC implementations for unification purposes, b) ranking dimensions according to their frequency of use by HCO worldwide, and c) ranking dimensions according to their importance from the health care managers perspective.

Methods

This systematic review is part of broad research. After assessing the impact of the BSC on stakeholder satisfaction [32] and before developing instruments to engage stakeholders in BSC implementations, we sought to accomplish the previously mentioned aims to summarize which dimensions were the most frequently used and essential as per health care managers in implementing the BSC. This review was conducted according to the 27-point checklist of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [43]; see Appendix (S1).

Eligibility criteria

The inclusion and exclusion criteria were set as shown in Table 1 based on the PICO (Population, Intervention, Comparison, and Outcome) tool [44]. Additionally, all study designs were included.

Table 1 Inclusion/Exclusion Criteria and Search Strategy for PubMed

Data sources, search strategy, and study selection

First, a search strategy was developed for the PubMed database (see Table 1). Then, the strategy design was adapted for the Embase, Cochrane CENTRAL, and Google Scholar databases. Furthermore, an additional search in the Google engine was performed to find gray literature or unpublished papers, including theses and conference abstracts. Additionally, the reference lists of all the eligible articles were checked. The databases were searched from inception until October 2020.

The search strategy was developed by the first, second, and fourth authors; the first two were experts in health care management and BSC, while the fourth author was an expert in systematic reviews and meta-analysis. It was initially developed for the PubMed database. Moreover, the search strategy was developed to include both Medical Subject Headings (MeSH) terms and keywords to widen the search frame and obtain more results. Then, appropriate truncation and relevant indexing terms were used. See Appendix (S2), which shows search strategies in all databases. Afterward, the first author conducted an electronic database search and removed duplicates using the EndNote X9.2 program.

The selection of eligible studies was performed independently by the first and second authors in all steps. Disagreements were resolved by discussion after each step or, if necessary, through arbitration by the fourth author. First, the articles' titles and abstracts were examined to eliminate irrelevant papers between November 2020 and February 2021. Then, full texts were carefully inspected to decide the final papers' inclusion list between February and June 2021. If different KPIs were used in more than one implementation in the same study, each was counted as a different implementation. In comparison, implementations using the same KPIs in other locations or times in the same research were considered one implementation. The authors of studies with no available full texts or with partially reported results were contacted for missing data.

Data extraction process

Data extraction was performed between June and July 2021 and then compared to discuss differences. The following data were extracted from the eligible studies: 1) author/s, 2) year of publication, 3) country of origin, 4) data collection duration, 5) data collection tool, 6) the number of perspectives, 7) the number of KPIs, 8) availability of weights/importance for perspectives or KPIs, and 9) outcome, which is represented in the KPIs that have been used and their weights/importance. The frequency of each KPI used at each implementation was plotted on Microsoft Excel, and the sum was calculated. In addition, the weight/importance assigned for each KPI at each implementation was reported on a scale of 100%. In the case of studies that did not give weights/importance explicitly, each KPI weight/importance was calculated by dividing one by the number of KPIs used in that study to assign an equal weight/importance for each KPI.

Consequently, we computed an average of the weights/importance assigned for each KPI. Next, we performed regrouping and coding for the KPIs to find the frequency of use and the set weights/importance percentages for each dimension. Then, the resulting major and subdimensions were listed and described between August and September 2021.

The research design of eligible studies was extracted directly from the studies. However, if the research design was not explicitly mentioned, we determined it based on the role of the investigator in that study. Specifically, the study was considered observational if the BSC exposures were naturally determined and the investigator had no part. On the other hand, the study was considered experimental if the investigator actively assigned the BSC intervention.

Quality assessment

The risk of bias (RoB) assessment was accomplished between August and September 2021 to assess the quality of the included studies. The Risk of Bias in Nonrandomized Intervention Studies (ROBINS-I) tool was used to evaluate the observational and quasi-experimental studies [45]. In comparison, the Cochrane (RoB 2) tool was used for the assessment of randomized controlled trials (RCTs), as per the Cochrane collaboration's guidelines [46]. The RoB was analyzed at the study level and across studies since authors should avoid summarizing the overall RoB as per the Cochrane Handbook guidelines [47, 48].

In the RoB 2 tool, five types of bias were assessed: bias arising from the randomization processes, bias due to deviations from intended interventions, bias due to missing outcome data, bias in the measurement of outcomes, and bias in the selection of the reported results. On the other hand, in the ROBINS-I tool, seven types of bias were assessed: bias due to confounding, bias in the selection of participants in a study, bias in the measurement/classification of interventions/exposures, bias due to deviations from intended interventions/exposures, bias due to missing data, bias in the measurement of the outcomes, and bias in the selection of the reported results.

Using the RoB 2 tool, each type of bias was assessed as low, high, or unclear. While using the ROBINS-I tool, each type of bias was evaluated into five categories: low, moderate, serious, critical, or no information. Figures for RoB were prepared using the ROBVIS (Risk Of Bias VISualization) tool [49]. Last, it was recommended not to advocate quality appraisal as a criterion for inclusion in reviews [50]. Therefore, the authors decided to include all studies in this systematic review regardless of their quality assessment. See Quality assessment below.

Results

Study selection

A total of 4028 studies resulted from running the search strategy in the four databases. In addition, another three studies were identified through a Google search. Therefore, a total of 4031 studies were included. Duplicates were removed (n = 1046) using the EndNote program, and then the remaining articles were screened based on their titles and abstracts (n = 2985). Irrelevant papers were excluded (n = 2794).

Consequently, the remaining 191 studies were examined by reading the full texts. Among these papers, 22 papers were written in non-English languages, including Spanish, German, French, Chinese, and Persian. A full-text translation was performed for each study to decide whether to include or exclude any of them. As a result of reading the full texts, 158 studies were excluded, and only 33 were eligible for this review, in which 36 full implementations of different BSC designs were actually applied. Table 2 shows a summary of the 36 implementations. Details of the study selection process are shown in the PRISMA flowchart [43]. See Fig. 3.

Table 2 Overview of Included Studies
Fig. 3
figure 3

PRISMA Flow Diagram

Study characteristics

Language and location

From the resulting 36 implementations, one was in Spanish [57], one was in Persian [79], and the rest were in English. The 36 implementations were performed in various countries: 19 in.

Asia [59,60,61, 64, 65, 67,68,69,70,71, 73,74,75, 78,79,80, 84], seven in North America [51, 53,54,55,56, 58, 62], six in Europe [52, 57, 66, 77, 82], three in Africa [72, 76, 81], and one without location information [83].

Settings

Twenty-one implementations were performed in hospitals (secondary and tertiary HCO) [41, 51,52,53, 55,56,57,58,59,60,61,62, 64, 66, 67, 71, 77, 79, 80, 82] and 15 in medical centers or health facilities (primary HCO) [54, 65, 68,69,70, 72,73,74,75,76, 81, 83, 84].

Implementations

Two studies [66, 74] included three and two implementations, respectively, with different KPIs per implementation. Thus, the 33 resulting studies contained 36 unique implementations. No BSC implementation in the COVID-19 era was found.

Study design

The 36 BSC implementations varied in their designs. However, most studies did not explicitly report their study design. We categorized the 36 implementations based on the active role of the investigator in BSC implementation and the time of data collection. Consequently, one sole study design was an RCT [72]. Moreover, 14 implementation designs were uncontrolled quasi-experiments. Specifically, six implementations had a posttest-only design [52, 60, 71, 78, 81, 82]. Five implementations in four studies had pretest–posttest designs [61, 66, 77, 79]. Finally, three implementations interrupted the time series design [54, 67, 85]. On the other hand, 20 implementations were observational; six implementations in four studies were cross-sectional [53, 73,74,75], one implementation was prospective [86], ten implementations were retrospective [58, 59, 62, 64, 65, 68,69,70, 76, 84], and two implementations were prospective and retrospective [55, 56]. Finally, one implementation did not have sufficient information or reported the study design [83].

Decision model

Some of the resulting studies integrated multiple-criteria decision analysis (MCDA) with BSC. One study [82] combined BSC with simulation and MCDA techniques with what was referred to as S-MEDUTA. Another study [60] integrated the BSC with fuzzy analysis. Two studies [67, 78] combined BSC with AHP, and one [78] used the TOPSIS technique. Studies explained that using these methodologies with the BSC would help them arrive at more informed and better decisions.

Perspectives frequency of use and importance

A total of 797 KPIs were extracted from the resulting implementations. These KPIs were categorized in the studies under 15 perspectives. The average number of perspectives used per study was 4.5, and for the KPIs, it was 22. The most frequently used perspectives were the internal, financial, patient, learning and growth, HCW, managerial, community, and stakeholder perspectives. The total use frequencies of these perspectives at the implementations were 29.6%, 17%, 12.6%, 12.6%, 9.4%, 6.3%, 5%, and 3.1%, respectively. On the other hand, the topmost important perspectives from the health managers' viewpoint were the internal, financial, learning and growth, patient, HCW, community, managerial, and stakeholder perspectives with a total weight/importance of 37.9%, 15.4%, 12%, 11.3%, 7.8%, 7.7%, 3.6%, and 2.8%, respectively.

Categorization and regrouping of KPIs into Dimensions/Subdimensions

The 797 extracted KPIs were plotted according to their frequencies and weights/importance in the categorization process. See Appendix (S3). Grouping and recategorizing KPIs resulted in Figs. 4 & 5 below, showing 13 major dimensions and 45 subdimensions based on their.

Fig. 4
figure 4

The BSC 45 subdimensions. Figure legend: After regrouping the 797 indicators, 45 subdimensions resulted. This figure shows the frequency and weight/importance for each subdimension

Fig. 5
figure 5

The BSC 13 major dimensions. Figure legend: Reassembling the 45 subdimensions resulted in 13 major dimensions. This figure shows the frequency and the weight/importance for each major dimension independently

Frequency of use and importance, respectively. After regrouping these KPIs into homogenous major dimensions and subdimensions, 13 major dimensions resulted, with 45 subdimensions. A summary of the resulting perspectives and their major and subdimensions contents are illustrated in Fig. 6. The description of each major and subdimensions is described further in Appendix (S4).

Fig. 6
figure 6

A summary of BSC perspectives in health care and their contents. Figure legend: Summary of BSC perspectives and the underlying major and minor subdimensions for the PE of HCOs

Quality assessment

Each study was evaluated in terms of RoB, as illustrated in Appendix (S5). The RoB 2 tool was utilized to assess the ROB in the sole RCT study [72], for which the assessment showed fair evaluation, except for performance bias. On the other hand, utilizing the ROBINS-I tool for assessing the RoB in observational and quasi-experimental studies revealed no information about confounders' adjustment methods except in three studies [51, 59, 67]. The confounding agents were apparent in the three studies; one study [51] performed confounders adjustments. On the other hand, another [59] adjusted for patient severity but not for the LOS and mortality rate. Last, one study [67] did not perform adjustment at all, which may have affected measurement precision.

The selection bias across studies reflected a serious RoB in five studies [59, 63, 67, 73, 80]. Therefore, the intervention and the follow-up did not coincide, and a potentially substantial amount of follow-up was missing in their analysis. Studies with a moderate risk of intervention/exposure measurement bias reflected a well-defined intervention status, but some aspects of the assignments of intervention status were determined retrospectively. Furthermore, bias in selecting the reported results was serious in one study that partially reported the results [80]. Studies that reported all results but did not have a preregistered protocol or whose outcome measurements were not defined in an initial plan were given a moderate risk. See Appendix (S5).

Discussion

Discussion of the main results

All the perspectives, dimensions, and KPIs employed in BSC implementations were collected to fulfill the research aims. Categorization and regrouping of the KPIs into major and subdimensions were performed. Then, the dimensions were ranked according to their frequency of use and their importance. The BSC tool can offer comprehensive planning, monitoring, evaluation, and improvement of HCO KPIs. Hence, their performance should be improved in the short and long term.

In general, studies had either no information or low or moderate ROB. At the same time, only a few of them had serious or critical ROB. However, studies that had only fully reported BSC indicator measurements were included. Many of them did not have a preregistered protocol or predefined measures in their plan. No information was found regarding confounders or deviation from unintended interventions.

Overall completeness and applicability of evidence

Analyzing the results shows that BSC implementations typically utilized four fundamental perspectives: financial, customer, internal, and knowledge and growth. However, the analysis of Fig. 5 revealed the frequent employment and the importance of other BSC perspectives in BSC implementations. Specifically, the external and managerial perspectives. This reflects the need for slight modifications of BSC design and corresponds with the findings of another study [87], which referred to the sustainability perspective of the BSC as the fifth pillar. Additionally, our findings reveal that focusing on both internal and external customers from the customer perspective is essential.

The variation among BSC implementations in the categorization of the same KPIs reflects the need for data standardization. HCW training-related KPIs, for example, were categorized under the learning and growth perspective in almost half of the resulting studies [58,59,60, 62, 64,65,66,67,68,69,70,71,72,73,74,75,76,77,78, 84]. Meanwhile, the rest of the studies categorized them under the perspectives of HCW [72, 76], quality [54], service capacity, provision/service capacity [64, 68, 70, 73, 76, 84], and healthcare facility functionality [75]. These results are consistent with a study [5] that referred to the lack of defining measures and the lack of data standardization. The differences in categorization prove our assumptions in the calculation imprecision in the previous reviews. Specifically, in the use frequency or the importance of the perspectives and KPIs. Our systematic review solved this calculation bias by uniformly forming the 797 KPI categorizations. Regrouping similar or semisimilar KPIs under the same category resulted in more precise results. Unification of dimensions can guide uniform future implementations of PE or BSC at HCO, allowing data sharing and comparability. Dimension unification can be why our findings are different from another systematic review [88] that did not consider unifying the classification of KPIs. According to HCO management, the average LOS, HAIs, patient satisfaction, bed occupancy, and bed turnover rate were the most useful KPIs.

Analyzing the results also shows a lack of BSC utilization in HCO during the pandemic. Additionally, there has been a lack of studies comprehensively examining the impact of COVID-19 on KPIs.

This review can guide healthcare managers and researchers since the resulting dimensions can be utilized to synthesize future BSC measurements. Specifically, the dimensions can direct the creation of new instruments to engage stakeholders in future BSC implementations. Moreover, this review can provide a road map for healthcare managers to perform a comprehensive PE of HCO during the COVID-19 pandemic. Since the COVID-19 pandemic may influence the BSC dimensions positively or negatively compared to prepandemic, analyzing the effect of the pandemic on the performance of the major- and subdimensions will allow HCO’s managers to better understand where to focus on their action plans to improve the overall performance of HCOs.

Practical assessment implications of the resulting dimensions in the COVID-19 era

Although this systematic review included ten months after the initiation of the COVID-19 pandemic, no research on BSC utilization in COVID-19 was found. Moreover, health policy experts stated that insufficient standardization of quality measurement approaches in the COVID-19 era challenged sharing purposes. As a result, the comparison between the performance of healthcare systems is disrupted [5]. Comparison is critical in cases where the optimal performance is not fully understood as in pandemics, and a comparison with other health systems would be informative and necessary [5]. Therefore, addressing the lack of data standardization was suggested to be overcomed by quickly defining measures, which could allow health systems, at least in the short term, to use standardized methods to better understand their performance [5].

We pursued further analysis in this paper based on independent studies per resulting dimension during the COVID-19 era to highlight how these dimensions can be utilized to monitor and improve HCO's performance during the COVID-19 pandemic.

The Financial major dimension

Due to COVID-19 hospitalizations at the beginning of the pandemic, health policy experts suggested that HCO in some regions will have more significant revenue and greater costs related to additional HCW and resources. In contrast, other hospitals will experience mostly sharp reductions in elective and outpatient payments, which will create unprecedented financial challenges for HCO [89]. However, in addition to the higher costs of HCWs and resources, researchers found higher costs of treatment due to extra diagnostic tests and isolation costs [90].

In the United Kingdom (UK), the total expenditure on the National Health System (NHS) has increased significantly during the pandemic [91]. The NHS made funding upgrades to expand waiting areas and treatment cubicles [92]. Some studies have focused on cost-effectiveness calculations. A study in South Africa indicated that purchasing intensive care unit (ICU) capacity from the private sector during COVID-19 surges may not be a cost-effective investment [93]. To date, there is still a lack of studies that handle the financial dimension or develop cost-saving strategies at the health organization level in COVID-19.

The efficiency and effectiveness major dimension

Analyzing the number of patient visits and admissions in the USA, [94] revealed a decrease in ER visits, increasing hospital admissions. However, another study in Alberta [95] perceived decreased admissions and ER visits to the hospital, despite the low volume of COVID-19 hospital admissions.

Many studies have been performed to analyze the efficiency, utilization, and productivity of HCO during the pandemic. A study [96] indicated that efficient hospitals under normal conditions lost their efficiency during COVID-19 and had to adapt to the new criteria. A systematic review [97] showed that healthcare utilization decreased by approximately one-third during the pandemic, with more significant reductions among people with less severe illnesses.

A study at an isolation hospital in Egypt [98] utilized the DEA tool to improve efficiency. This confirmed that the number of nurses and the number of beds impacted the operational efficiency of COVID-19, while the number of physicians had no significant effect on the efficiency. These results are compatible with a study in Mauritius [19] that found that nurses and beds are the most critical factors in hospital production; that is, a 1% increase in the number of beds and nurses resulted in an increase in hospital outputs by 0.73 and 0.51%, respectively.

The availability and quality of the supplies and services major dimensions

The supply and logistics management dimension was considered an important KPI in tackling COVID-19 [6]. This dimension includes evaluating the availability and quality of COVID-related medications, masks, personal protective equipment (PPE), detergents, medical services, supportive services, etc. Additionally, researchers viewed the availability of both clinical and supportive services at hospitals as essential in responding to the COVID-19 pandemic and the flow of COVID-positive patients [99]. The spectrum of supportive services to a hospital encompasses linen and laundry, diet, central sterile supply department, transport, consumables in large quantities at hospital stores, mortuary, and engineering services [99]. Some of the essential items were filtering face-piece respirators or N95 respirators and the availability of PPE kits [99]. The global challenge during this pandemic in terms of inadequate availability of PPE in HCO highlighted the vital role of CSSD. Centers for Disease Control and Prevention (CDC) suggested a method of decontamination, and reuse of filtering face respirators to overcome the shortage of these respirators is their extended use or reuse [99].

However, researchers have referred to the lack of studies on the quality of supplies and services at HCO in COVID-19 [5]. Lack of studies can be referred to as data lag in pandemics: the time between care provision and quality measurement reporting [5]. Policymakers suggested that measures should be less reliant on claims data, which by nature have a time lag, and focus on actions that can be generated from the electronic health record (EHR) [5] investigated during the pandemic.

The HCW scientific development major dimension

Due to its importance, many studies have aimed to evaluate HCW knowledge, attitudes, and practices (KAP) at the beginning of the pandemic [100]. HCW adherence to IC measures is affected by their KAP toward COVID-19 [101]. Some studies referred to insufficient knowledge about COVID-19 among nurses [102]. Surgeons were worried about losing their skills after months of lockdown due to paused practice [103]. However, HCWs were obliged to learn digital health skills and effectively communicate with patients during the pandemic [103].

A study [104] found that COVID-19 and non-COVID-19 publication productivity correlates with some factors. For example, epidemiologic, healthcare system-related, and pre-COVID publication expertise factors. Therefore, countries with a stable scientific infrastructure appear to maintain non-COVID-19 publication productivity nearly per year. More incentives must be drawn by HCOs to their HCWs to encourage research and scientific productivity related to COVID-19.

The managerial tasks and PE dimension

Standard policies, procedures, the availability of written standardized guidelines, and delivery in full and on time were considered essential in tackling COVID-19 [6]. A lack of standardization capability and conflicting or irrational managerial decisions were deemed dissatisfactory factors for HCWs in the pandemic [103].

Planning and preparedness are also crucial managerial tasks. The CDC developed a checklist to help hospitals assess and improve their preparedness for responding to COVID-19 [105]. Hospitals utilized a collection of some of the previously explained KPIs and dimensions to perform planning and internal assessment of their performance [106, 107].

Few studies [108, 109] have examined centralized governance's impact on HCO during the pandemic, which positively affected reactive strategies. Learning from past pandemics also positively influences proactive and reactive strategies. However, the role of PE internally, such as using BSC or MBNQA tools, or external assessments, such as Joint Commission International (JCI) accreditations, ISO certification, auditing, or peer review on HCO during the pandemic, still requires more investigation.

The HCW-centredness major dimension

Physicians referred to the importance of reliable acknowledgment and motivation both emotionally and financially, considering the sacrifices they provide every day [103]. In parallel, staffing and recruitment of an adequate number of medical and nonmedical HCWs were considered important KPIs for the PE of HCO at COVID-19 [6]. In the UK, the NHS employed strategies to facilitate the staffing process due to the shortage of HCWs. First, newly qualified/final year medicine and nursing students were deployed. Second, the return of the former HCW was made [110].

The HCW satisfaction rate and burnout have been evaluated in many studies during the pandemic. A study [103] showed that the physicians' burnout prevalence was 57.7% during the pandemic, which is considered high. HCWs who lack PPE reported lower occupational satisfaction than those who did not [103, 111]. HCW accomplishments during the pandemic were positively associated with higher occupational satisfaction rates [111]. Therefore, emphasizing HCW accomplishments leads to increased satisfaction rates.

Moreover, as mentioned earlier, better performance of the communication dimension, including psychological support, will raise HCW job satisfaction and lower the rates of burnout and stress [111, 112]. Some HCWs felt anxiety and fear mainly due to the possibility of transmitting the virus to their family members and the elderly living in their house [103]. A study in Canada [4] showed that HCW training and counseling services were perceived as helpful in reducing HCW stress. Despite that, they were underutilized in HCO.

On the other hand, although most nurses had to increase their workload due to staff shortages, a study [111] found that the elevation of the workload was not associated with lower occupational satisfaction. Additionally, another study in Singapore [113] found that HCW burnout was similar to the prepandemic rates. Nevertheless, the HCW vaccination, engagement, motivation, teamwork, and loyalty subdimensions and their impact are still not good.

The time major dimension

An "extra layer of processes" was added due to the donning and doffing protocols and cleaning requirements, which slowed all the operational processes down and increased the time required to accomplish serving the medical care to patients [91]. Patient WT was also influenced. In the UK, WT reached high levels in studies with a notable impact on elective surgery. The number of patients who waited for more than a year to receive NHS treatment in July 2020 was 81-fold greater than the previous year's number [92].

Moreover, the patient length of stage (LOS) also increased for another 2–3 days. A reason for this was the delays in COVID-19 testing results [114]. The LOS in the USA was two days more than that in Italy and five days less than that in Germany [115]. A systematic review for patient LOS in COVID-19 [116] concluded that LOS in China was longer than that in any other country, referring to differences in criteria for admission and discharge and different timings within the pandemic. Another study [105] found a negative association between the LOS and the case fatality rate. Therefore, LOS estimation can be introduced as a KPI to scale the success of the countries fighting the ongoing pandemic.

Moreover, LOS provides insights into when hospitals will reach capacity and predicts associated HCW or equipment requirements [116]. Discharge status should be considered when analyzing LOS since patients who are discharged alive have a longer LOS than those who died during their admission [116]. Hospitals reported that health insurance plans resisted paying for additional patient days in the hospital while awaiting COVID-19 test results [114].

However, complying with the CDC guidance on testing and disposition of patients was suggested to reduce the patient LOS, freeing up hospital beds for incoming COVID-19 patients [114]. Another study in the UK [114] indicated that due to the complexity and partiality of different data sources and the rapidly evolving nature of the COVID-19 pandemic, it is most recommended to use multiple LOS analysis method approaches on various datasets.

A combination of an accelerated failure time survival model and a truncation corrected method with the multistate survival model was found to be helpful in epidemic planning and management. Finally, the findings of a cohort study [117] concluded that a multimechanism approach effectively decreased the average LOS in the ICU by 5.4 days and up to nine days in older patients. This finding suggests that implementing this treatment protocol could allow a healthcare system to manage 60% more COVID-19 patients with the same number of ICU beds.

Major error-free and safety major dimension

This dimension includes monitoring, analyzing, and comparing mortality rates and investigating its determinants in HCO. Although mortality may not be directly related to errors, mortality rates higher than the average can reveal an underlying mistake. A cohort study in Mexico City [85] found that the mortality rates at the hospital's ICU and non-ICU departments were similar. The reason behind this finding was the ICU bed's unavailability. Approximately 45% of the patients who did not survive did not receive an ICU bed, which raised the mortality rate in the non-ICU admitted patients. However, this study revealed that the leading cause of non-ICU admission was acute respiratory distress syndrome. The leading cause of mortality for admitted patients was septic shock, followed by ARDS and multiorgan failure.

The WHO has provided clear guidelines for infection control (IC) during healthcare when COVID-19 is suspected or confirmed [118]. Patient safety was investigated in a systematic review of Indian-related studies [119]. Patient safety was negatively impacted during the COVID-19 pandemic due to inadequate preparation of the healthcare system, such as infrastructure and human and material resources. Additionally, researchers categorized diagnostic errors that could occur in the COVID-19 pandemic into eight types and suggested how to reduce them [120].

However, many studies have shown improvements in this dimension during the pandemic. A study in the UK [121] found a significant increase in the safety attitude questionnaire scores of doctors and other clinical HCWs and no change in the nursing group. It also showed a significant decrease in error reporting after the onset of the COVID-19 pandemic. Another study in Iran [122] found that health-associated infections (HAIs) during the pandemic were reduced, which could be referred to as the proper implementation of IC protocols. This finding is supported by a study in Ghana [123], which found that HCW compliance with IC measures was high during the pandemic.

The health waste (HW) management subdimension was intensively investigated due to the tremendous increase in HW volume during the pandemic [124]. A study in Iran [125] indicated that infectious waste increased by 121% compared with before the pandemic. Direct exposure of HCWs to virus-contaminated waste with inadequate safety measures and mismanagement of HWs may lead to their infection and facilitate the transmission of COVID-19 [124, 126]. The WHO has provided clear guidelines for managing healthcare waste during the pandemic [127]. Nevertheless, many studies worldwide [126, 128, 129] illustrated the existence of gaps and a flawed system for handling HWs during the pandemic.

A mini-review [124] of HWs during the pandemic showed that disinfecting waste, followed by proper segregation and on-site treatment, can also provide better and healthier HW management. It also revealed that surplus HW accommodation, mobile treatment, and temporary storage strategies might aid the sustainable management of healthcare waste without further spreading the virus. Another study in Brazil [128] proposed a model for the proper management of HWs. It focused not only on the operational management KPIs of HWs but also on environmental management, such as sustainable practices. Moreover, it highlighted the importance of employee training on HW guidelines since HW management was not considered an essential competence or a priority for every HCO.

The patient-centredness major dimension

Many studies have been conducted to evaluate patient satisfaction. A study [130] indicated no difference in patient satisfaction during the period spent in the emergency room before and during the pandemic. Another study [131] showed positive patient experience and satisfaction rates in Saudi Arabia's largest institutions during the pandemic. Moreover, many studies have focused on the psychological assessment of the impact of COVID-19 on the population in general. However, few studies have focused on specifically assessing the psychological effects on patients. For example, a study [132] found that COVID-19 patients with low education levels and females who have undergone divorce or bereavement tended to have a high prevalence of adverse psychological events. Another study [133] found that the psychological consequences of the pandemic were better handled by cancer patients 65 years of age or older, while younger cancer patients were more psychologically affected. Early psychological status identification and intervention should be conducted to avoid extreme events such as self-mutilating or suicidal impulsivity for patients [132]. Patient complaints and loyalty assessment during the pandemic and the psychological impact of COVID-19 on non-COVID-19 patients still need more investigation.

The responsiveness and communication major dimension

The main goal of HCO was considered to provide high-quality care to patients and meet their needs and expectations during an outbreak such as COVID-19 [96]. Moreover, dialog and listening to health demands in COVID-19-suspected patients was highlighted as the foremost step in the flows of care and guidance [134].

Communication among HCWs was also highlighted. A study [4] considered HCW reception of family support, colleagues, support, clear communication, and COVID-19 information as the most valuable resources in the pandemic. Lower HCW psychological distress symptoms, burnout, and intentions to quit were perceived when these communication resources were more available. Another study [112] indicated that gratitude in communication could reduce depression in HCWs by promoting social support and hope.

In addition, communication between HCWs and patients was also investigated during the pandemic. A study in Jordan [135] found that physician–patient communication positively affected the patient's psychological status in COVID-19. It recommended avoiding communication errors using jargon, not being available to patients, and not showing empathy in communication. Additionally, it emphasized the benefit of physicians as excellent listeners to patients. However, HCW-patient communication faced few obstacles during the pandemic. The protective equipment used by HCWs in the pandemic could have imposed a barrier to effective communication or eye contact with them [136]. Some pediatricians reported difficulty communicating with families and following up with patients, especially newly discharged neonates and infants, using the telephone [103].

However, more research is still needed to improve and evaluate patient education programs, patient guidelines, counseling and consultation services, and communication skills between HCWs and patients during the pandemic.

Community care and reputation major dimension

The external major dimension, including the social sustainability indicators for HCOs facing a crisis, can be ambiguous to define and apply, so SSIs have been organized under the broad categorical concerns of well-being, values, agency, and inequality [107]. Despite the doctor–patient confidentiality clause and the protection law for patient data privacy, the Department of Health and Social Care for England has relaxed the rules on sharing confidential patient data. It required HCO and the NHS to exchange patient information to help fight COVID-19 [137]. Moreover, COVID-19 patient data led to society breaching patient privacy in some countries [103, 138], which may have stigmatized those patients [103].

As mentioned earlier, a study [128] emphasized the importance of sustainable environmental practices for better HW management. The political situation was also considered an external influence during the pandemic. It was highlighted in a study [139] in the Palestinian territories, which referred to the COVID-19 situation in the presence of the Israeli military occupation to have a double epidemic effect, which eventually impacted the performance of the Palestinian health system HCO during the pandemic.

However, factors as exemptions offered by HCO for poor patients, social responsibility, patient privacy concerns, and HCO market shares in COVID-19 are still poorly investigated.

The HCO building major dimension

Design and infrastructure preparation were considered essential dimensions in some HCOs during the pandemic [6]. Healthcare systems made adaptations in HCO buildings after the COVID-19 pandemic. Examples include expanding waiting areas, increasing ICU capacity, establishing isolation areas, and building new hospitals [92]. In the UK, the NHS temporarily used private hospitals to provide public care, increasing the number of beds, ventilators, and all HCW categories. Moreover, nonhospital sites were temporarily turned into hospitals [110]. However, researchers did not sufficiently investigate the ease of access to HCO during the pandemic.

The technology and information system major dimension

Experts emphasized the role of technology and information (TI) in tackling COVID-19 as inevitable due to its importance in the response, prevention, preparedness, and recovery phases [140, 141]. TI system application varies from allowing HCO to maintain and share studies to producing different reports and follow-up with pandemic analysis. Telehealth is another example that proved helpful in the pandemic. It allowed HCWs to provide care for patients without direct physical contact, especially to patients at quarantine, while keeping them safe [142].

Researchers summarized the emerging technologies used to mitigate the threats of COVID-19 in the following categories: artificial intelligence/deep learning, big data analytics, high-performance computing infrastructures, robots, 3D printing technology, digital contact tracing technology, blockchain [113], bioinformatics systems, telemedicine, mobile phone, decision support system, the IC system in HCO, online interactive dashboard/geographic information system, Internet of Things, virtual reality, surveillance systems, and internet search queries [140, 141]

Governments, healthcare systems, and HCO need to keep updated with the emerging technologies in this field, allocate resources to invest in them, and develop the required skills in HCWs to utilize them properly.

Strengths and weaknesses

We believe that this paper has several strengths. First, this systematic review includes all types of studies with BSC implementations, such as books, theses, conference papers, and letters to the editor. Second, this review contains all implementations despite the country, language, or HCO administrative type, which gives an advantage of generalizing HCO results worldwide. Third, unlike other BSC reviews [35, 37], which included definitions of biobanks, pharmacies, laboratories, radiology, and medical colleges in HCO, this review limited the report to primary, secondary, or tertiary healthcare organizations. However, an initial assessment by top management to evaluate the importance of each dimension and KPI based on the health organizations' strategy could be needed. This strategy leads to the homogeneity of the resulting studies and leads to more valid comparisons among the results. Fourth, this review calculates the use frequency of perspectives and the weights/importance assigned to them. Fifth, the first review has uniform KPIs in homogenous major dimensions and subdimensions despite the categorization differences among implementations, yielding more precise results. The resulting KPIs and dimensions in this review can be generalized or replicable to other HCOs and hospitals. Finally, this study is the first to analyze the implications of BSCs in HCO during the pandemic based on the literature. This implication provides a guide for future theoretical implications, such as performing systematic reviews for each major dimension during the pandemic. It also provides a guide for practical implications of BSC dimensions to assess HCOs’ performance.

On the other hand, this systematic review has some limitations. First, unlike previous studies, it excludes some HCOs, such as laboratories, pharmacies, radiology departments, and biobanks, as specified in the inclusion/exclusion criteria. Therefore, our results cannot be generalized to such HCO types. However, we excluded them to arrive at more homogenous KPIs and dimensions that are directly related to HCO that offer primary, secondary, and tertiary medical services. Second, it includes only the articles that report the complete implementations of BSC while excluding studies that display only the BSC design without reporting the full implementation results. Third, we extracted the KPIs from all resulting implementations despite their RoB. However, we included an ROB assessment for each implementation.

Conclusion

In conclusion, our review shows that the most frequently used perspectives in BSC papers were internal, financial, patient, learning and growth, HCW, managerial, community, and stakeholder perspectives. The perspectives that had the highest importance were internal, financial, learning and growth, patient, HCW, community, managerial, and stakeholder.

Moreover, this review solves the dilemma of the KPI categorization difference between BSC implementations by dimension unification into 13 major dimensions. The financial, efficiency and effectiveness, availability and quality of supplies and services, managerial tasks, HCW scientific development, error-free and safety, time, HCW-centeredness, patient-centeredness, technology and information system, community care and reputation, HCO building, and communication. The proper utilization of the 13 major dimensions and the 45 subdimensions will serve as a planning, monitoring, evaluation, and continuous improvement tool for HCO, resulting in performance augmentation.

This research showed a lack of BSC utilization and any holistic PE approach in HCO during the COVID-19 pandemic. Additionally, some dimensions that are essential for PE are still poorly investigated. Our analysis reflects that most KPIs were negatively affected during the pandemic, except IC and safety measures, which improved in some cases. However, a comprehensive PE of HCO during the COVID-19 pandemic worldwide is still needed. Therefore, we recommend that future researchers perform a comprehensive practical PE for HCO during COVID-19 using the measurements of the resulting dimensions. This analysis will provide a better understanding of the causal relationships between dimensions. It will also allow comparability of the interventions' outcomes, which will boost the performance and mitigate the consequences of the pandemic on HCO. Moreover, researchers are encouraged to perform systematic reviews for each dimension, especially those that are already well investigated and the investigation of dimensions that are still poorly investigated but essential for PE. This theoretical implication will lead to performance enhancement and mitigate the consequences of the pandemic on HCO.

Availability of data and materials

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Abbreviations

BSC:

Balanced Scorecard

CDC:

Centers for Disease Control and Prevention

COVID-19:

Coronavirus Disease 2019

DEA:

Data Envelopment Analysis

DRG:

Diagnostic Related Groups

EFQM:

European Foundation for Quality Management

ER:

Emergency Room

HAIs:

Health-Associated Infections

HCO:

Health Care Organizations

HCQI:

Health Care Quality Indicator

HCW:

Health Care Workers

HMIS:

Health Management Information System

HW:

Health Waste

IC:

Infection Control

ICU:

Intensive Care Unit

ISO:

International Organization for Standardization

JCI:

Joint Commission International

KAP:

Knowledge, Attitude, and Practices

KPIs:

Key Performance Indicators

LOS:

Length Of Stay

MBNQA:

Malcolm Baldrige National excellence model

MBO:

Management by Objectives

MCDA:

Multiple-criteria decision analysis

MeSH:

Medical Subject Headings

NHS:

National Health System

OECD:

Organization for Economic Co-operation and Development

PATH:

Performance Assessment Tool for quality improvement in hospital

PE:

Performance Evaluation

PPE:

Personal protective equipment

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

ROA:

Return on Assets

RoB:

The Risk of Bias

ROI:

Return On Investment

SAQ:

Safety Attitude Questionnaire

SQA:

Singapore Quality Award

T.I.:

Technology and information

TQM:

Total Quality Management

UK:

United Kingdom

USA:

United States of America

WHO:

World Health Organization

WT:

Waiting Time

References

  1. Meena K, Thakkar J. Development of balanced scorecard for healthcare using interpretive structural modeling and analytic network process. J Adv Manag Res. 2014;11:232–56.

    Article  Google Scholar 

  2. Buathong S, Bangchokdee S. The use of the performance measures in Thai public hospitals. Asian Rev Account. 2017;25:472–85.

    Article  Google Scholar 

  3. Amer F, Hammoud S, Farran B, Boncz I, Endrei D. Assessment of countries’ preparedness and lockdown effectiveness in fighting COVID-19. Disaster Med Public Health Prep. 2021;15:e15-22. https://doi.org/10.1017/dmp.2020.217.

    CAS  Article  Google Scholar 

  4. Lou NM, Montreuil T, Feldman LS, Fried GM, Lavoie-Tremblay M, Bhanji F, et al. Evaluations of healthcare providers’ perceived support from personal, hospital, and system resources: implications for well-being and management in healthcare in Montreal, Quebec, during COVID-19. Eval Health Prof. 2021;44:319–22. https://doi.org/10.1177/01632787211012742.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Austin JM, Kachalia A. The state of health care quality measurement in the era of COVID-19. JAMA. 2020;324:333. https://doi.org/10.1001/jama.2020.11461.

    CAS  Article  PubMed  Google Scholar 

  6. Ali A, Owied MM, Virginia P, Ruth E, Abdulellah AT. Using logic model and key performance indicators to construct a field hospital for Coronavirus pandemic cases. BMC Emergency Medicine. 2020;:12–4. doi:https://doi.org/10.21203/rs.3.rs-102321/v1.

  7. Afulani PA, Gyamerah AO, Nutor JJ, Laar A, Aborigo RA, Malechi H, et al. Inadequate preparedness for response to COVID-19 is associated with stress and burnout among healthcare workers in Ghana. PLoS One. 2021;16 4 April:1–20. doi:https://doi.org/10.1371/journal.pone.0250294.

  8. Andersen HV, Lawrie G, Savič N. Effective quality management through third-generation balanced scorecard. Int J Product Perform Manag. 2004;53:634–45. https://doi.org/10.1108/17410400410561259.

    Article  Google Scholar 

  9. Ruiz U, Simon J. Quality management in health care: a 20-year journey. Int J Health Care Qual Assur. 2004;17:323–33. https://doi.org/10.1108/09526860410557570.

    Article  Google Scholar 

  10. Hojabri R, Manafi M, Eftekhar F, Ghassemzadeh H. Effective methods for health care organizations: an evaluation of excellence models. African J Bus Manag. 2013;7:2665–75.

    Google Scholar 

  11. Klassen A, Miller A, Anderson N, Shen J, Schiariti V, O’Donnell M. Performance measurement and improvement frameworks in health, education and social services systems: a systematic review. Int J Qual Heal Care. 2010;22:44–69. https://doi.org/10.1093/intqhc/mzp057.

    Article  Google Scholar 

  12. Sheykholeslam S, Sachin V. A study of performance evaluation models in health care sector with special reference to EFQM and BSC models. Int J Multidiscip Res Dev. 2015;2:378–85.

    Google Scholar 

  13. Veillard J, Champagne F, Klazinga N, Kazandjian V, Arah OA, Guisset AL. A performance assessment framework for hospitals: The WHO regional office for Europe PATH project. Int J Qual Heal Care. 2005;17:487–96.

    CAS  Article  Google Scholar 

  14. Carini E, Gabutti I, Frisicale EM, Di Pilla A, Pezzullo AM, de Waure C, et al. Assessing hospital performance indicators. What dimensions? Evidence from an umbrella review. BMC Health Serv Res. 2020;20:1–13.

    Article  Google Scholar 

  15. Busse R, Klazinga N, Panteli D, Quentin W, World Health Organisation. Improving healthcare quality in Europe: Characteristics, effectiveness and implementation of different strategies. Improving Healthcare Quality in Europe. 2019;:419. https://apps.who.int/iris/bitstream/handle/10665/327356/9789289051750-eng.pdf?sequence=1&isAllowed=y. Accessed 25 Jun 2021.

  16. Kelley E. Hurst J. Health care quality indicators project: Conceptual framework paper. Oecd Heal Work Pap; 2006. p. 1–37.

    Google Scholar 

  17. Goshtasebi A, Vahdaninia M, Gorgipour R, Samanpour A, Maftoon F, Farzadi F, et al. Assessing hospital performance by the Pabon Lasso model. Iran J Public Health. 2009;38:119–24.

    Google Scholar 

  18. Kaplan R, Norton D. The balanced scorecard–measures that drive performance. Harv Bus Rev. 1992;70:71–9.

    CAS  PubMed  Google Scholar 

  19. Nundoochan A. Improving public hospital efficiency and fiscal space implications: the case of Mauritius. Int J Equity Health. 2020;19:1–16.

    Article  Google Scholar 

  20. Aghaei Hashjin A, Kringos DS, Manoochehri J, Aryankhesal A, Klazinga NS. Development and impact of the Iranian hospital performance measurement program. BMC Health Serv Res. 2014;31:280–8.

    Google Scholar 

  21. De Vos M, Graafmans W, Kooistra M, Meijboom B, Van der Voort P, Westert G. Using quality indicators to improve hospital care: a review of the literature. Int J Qual Heal Care. 2009;21:119–29.

    Article  Google Scholar 

  22. Shaw C, Bruneau C, Baskia K, Jong DG, Sunol R. How can hospital performance be measured and monitored? Copenhagen, WHO Regional Office for Europe (Health Evidence Network report; http://www.euro.who.int/document/e82975.pdf, Accessed 9 Jan 2021).

  23. Olve, Nils-Göran; Roy, J.; Wetter M. Performance drivers : a practical guide to using the balanced scorecard. New York: John Wiley and Sons; 1999.

  24. Speckbacher G, Bischof J, Pfeiffer T. A descriptive analysis on the implementation of Balanced Scorecards in German-speaking countries. Manag Account Res. 2003;14:361–88.

    Article  Google Scholar 

  25. Meliones J. Saving money, saving lives. Harv Bus Rev. 2000;78:57–62 64, 66–7.

    CAS  PubMed  Google Scholar 

  26. Bohm V, Lacaille D, Spencer N, Barber CE. Scoping review of balanced scorecards for use in healthcare settings: development and implementation. BMJ open Qual. 2021;10:1–10. https://doi.org/10.1136/bmjoq-2020-001293.

    Article  Google Scholar 

  27. Gagliardi AR, Lemieux-Charles L, Brown AD, Sullivan T, Goel V. Barriers to patient involvement in health service planning and evaluation: an exploratory study. Patient Educ Couns. 2008;70:234–41. https://doi.org/10.1016/j.pec.2007.09.009.

    Article  PubMed  Google Scholar 

  28. Anderson NN, Baker GR, Moody L, Scane K, Urquhart R, Wodchis WP, et al. Approaches to optimize patient and family engagement in hospital planning and improvement: qualitative interviews. Heal Expect. 2021;24:967–77. https://doi.org/10.1111/hex.13239.

    Article  Google Scholar 

  29. Korlén S, Richter A, Amer-Wåhlin I, Lindgren P, von Thiele SU. The development and validation of a scale to explore staff experience of governance of economic efficiency and quality (GOV-EQ) of health care. BMC Health Serv Res. 2018;18:963.

    PubMed  PubMed Central  Article  Google Scholar 

  30. Zelman WN, Pink GH, Matthias CB. Use of the balanced scorecard in health care. J Health Care Finance. 2003;29:1–16.

    PubMed  Google Scholar 

  31. Duke-Children’s-Hospital. Duke Children ’ s Hospital Case Abstract. Balanced Scorecard Collaborative Hall of Fame. :1–13. https://thepalladiumgroup.com/download?file=AED_1551102784_116attachment_dubai_boot_camp_-_march_2019_opt.pdf%23Dubai_Boot_Camp_-_March_2019_opt.pdf. Accessed 15 May 2021.

  32. Amer F, Hammoud S, Khatatbeh H, Lohner S, Boncz I, Endrei D. The deployment of balanced scorecard in health care organizations: is it beneficial? A systematic review. BMC Health Serv Res. 2022;22:1–14. https://doi.org/10.1186/s12913-021-07452-7.

    Article  Google Scholar 

  33. Rouis S, Abdelaziz AB, Nouira H, Khelil M, Zoghlami C, Abdelaziz AB. Elaboration d’un tableau de bord stratégique pour le monitorage de la performance hospitalière dans les pays du gran. Tunisie Medicale. 2018;96:774–88.

    Google Scholar 

  34. Colbran R, Ramsden R, Stagnitti K, Toumbourou JW. Advancing towards contemporary practice: a systematic review of organisational performance measures for non-acute health charities. BMC Health Serv Res. 2019;19:132.

    PubMed  PubMed Central  Article  Google Scholar 

  35. Rabbani F, Jafri SMW, AbbasPappas FG, Brommels M, Tomson G. Reviewing the application of the balanced scorecard with implications for low-income health settings. J Healthc Qual. 2007;29:21–34.

    PubMed  Article  Google Scholar 

  36. Gurd B, Gao T. Lives in the balance: an analysis of the balanced scorecard (BSC) in healthcare organizations. Int J Product Perform Manag. 2007;57:6–21.

    Article  Google Scholar 

  37. Mcdonald B. A review of the use of the balanced scorecard in healthcare. BMcD Consult. 2012;2012:1–32.

    Google Scholar 

  38. Trotta A, Cardamone E, Cavallaro G, Mauro M. Applying the Balanced Scorecard approach in teaching hospitals: a Literature review and conceptual framework. Int J Health Plann Manage. 2013;28:181–201.

    PubMed  Article  Google Scholar 

  39. Behrouzi F, Shaharoun AM, Ma’Aram A. Applications of the balanced scorecard for strategic management and performance measurement in the health sector. Aust Heal Rev. 2014;38:208–17.

    Article  Google Scholar 

  40. Broccardo L. The balance scorecard implementation in the Italian health care system: some evidences from literature and a case study analysis. J Health Manag. 2015;17:25–41.

    Article  Google Scholar 

  41. Gao T, Gurd B. Meeting the challenge in performance management: the diffusion and implementation of the balanced scorecard in Chinese hospitals. Health Policy Plan. 2014;30:234–41.

    PubMed  Article  Google Scholar 

  42. Gonzalez-Sanchez MB, Broccardo L, Martins Pires AM. The use and design of the BSC in the health care sector: a systematic literature review for Italy, Spain, and Portugal. Int J Health Plann Manage. 2018;33:6–30.

    PubMed  Article  Google Scholar 

  43. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100.

    PubMed  PubMed Central  Article  Google Scholar 

  44. Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014;14:579.

    PubMed  PubMed Central  Article  Google Scholar 

  45. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919. https://doi.org/10.1136/bmj.i4919.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Higgins JP, Savović J, Page MJ, Elbers RG SJ. Chapter 8: Assessing risk of bias in a randomized trial. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane. 2019.

  47. Higgins JP, Altman DG. Assessing risk of bias in included studies. In: Cochrane handbook for systematic reviews of interventions. Chichester, UK: John Wiley & Sons, Ltd; 2008. p. 187–241. doi:https://doi.org/10.1002/9780470712184.ch8.

  48. Babic A, Vuka I, Saric F, Proloscic I, Slapnicar E, Cavar J, et al. Overall bias methods and their use in sensitivity analysis of Cochrane reviews were not consistent. J Clin Epidemiol. 2020;119:57–64. https://doi.org/10.1016/j.jclinepi.2019.11.008.

    Article  PubMed  Google Scholar 

  49. McGuinness LA, Higgins JPT. Risk-of-bias VISualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. 2021;12:55–61. https://doi.org/10.1002/jrsm.1411.

    Article  PubMed  Google Scholar 

  50. Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52:546–53. https://doi.org/10.1111/j.1365-2648.2005.03621.x.

    Article  PubMed  Google Scholar 

  51. Pink GH, McKillop I, Schraa EG, Preyra C, Montgomery C, Baker GR. Creating a balanced scorecard for a hospital system. J Health Care Finance. 2001;27:1–20.

    CAS  PubMed  Google Scholar 

  52. Zbinden AM. Introducing a balanced scorecard management system in a university anesthesiology department. Anesth Analg. 2002;95:1731–8.

    PubMed  Article  Google Scholar 

  53. Griffith RJ, Alexander AJ. Measuring comparative hospital performance. J Healthc Manag. 2002;47:41–57.

    PubMed  Google Scholar 

  54. Biro LA, Moreland ME, Cowgill DE. Achieving excellence in veterans healthcare- a balanced scorecard approach. J Healthc Qual. 2003;25:33–9.

    PubMed  Article  Google Scholar 

  55. Smith H, Kim I. Balanced scorecard at Summa health system. J Corp Account Financ. 2005;16:65–72.

    Article  Google Scholar 

  56. Devitt R, Klassen W, Martalog J. Strategic management system in a healthcare setting–moving from strategy to results. Healthc Q. 2005;8:58–65.

    PubMed  Article  Google Scholar 

  57. Martínez-Pillado M, Seco JM, Suárez A, González-Sanjuán R, Vázquez C. Hacia el cuadro de mando integral por el cuadro de mando posible. Rev Calid Asist. 2006;21:13–9.

    Article  Google Scholar 

  58. Goodspeed SW. Metrics help rural hospitals achieve world-class performance. J Healthc Qual. 2006;28:28–33.

    PubMed  Article  Google Scholar 

  59. Yang M-C, Tung Y-C. Using path analysis to examine causal relationships among balanced scorecard performance indicators for general hospitals. Health Care Manage Rev. 2006;31:280–8. https://doi.org/10.1097/00004010-200610000-00003.

    Article  PubMed  Google Scholar 

  60. Lin QL, Liu L, Liu HC, Wang DJ. Integrating hierarchical balanced scorecard with fuzzy linguistic for evaluating operating room performance in hospitals. Expert Syst Appl. 2013;40:1917–24.

    Article  Google Scholar 

  61. Chen XY, Yamauchi K, Kato K, Nishimura A, Ito K. Using the balanced scorecard to measure Chinese and Japanese hospital performance. Int J Health Care Qual Assur. 2006;19:339–50.

    Article  Google Scholar 

  62. Josey C, Kim I. Implementation of the balanced scorecard at Barberton citizens hospital. J Corp Account Financ. 2008;19:57–63.

    Article  Google Scholar 

  63. Chang W, Tung Y, Huang C. Performance improvement after implementing the balanced scorecard: a large hospital’s experience in Taiwan. Total Qual Manag Bus Excell. 2008;19:1143–54.

    Article  Google Scholar 

  64. Chang W-C, Tung Y-C, Huang C-H, Yang M-C. Performance improvement after implementing the balanced scorecard: a large hospital’s experience in Taiwan. Total Qual Manag Bus Excell. 2008;19:1143–54. https://doi.org/10.1080/14783360802323560.

    Article  Google Scholar 

  65. Hansen PM, Peters DH, Niayesh H, Singh LP, Dwivedi V, Burnham G. Measuring and managing progress in the establishment of basic health services: the Afghanistan Health Sector Balanced Scorecard. Int J Health Plann Manage. 2008;23:107–17.

    PubMed  Article  Google Scholar 

  66. Lupi S, Verzola A, Carandina G, Salani M, Antonioli P, Gregorio P. Multidimensional evaluation of performance with experimental application of balanced scorecard: a two year experience. Cost Eff Resour Alloc. 2011;9:1–5.

    Article  Google Scholar 

  67. Chu H, Wang C. A study of a nursing department using the balanced scorecard and the performance measurement system: analytic hierarchy process. Nurs Econ. 2009;27:401–8.

    PubMed  Google Scholar 

  68. Edward A, Kumar B, Kakar F, Salehi AS, Burnham G, Peters DH. Configuring balanced scorecards for measuring health system performance: evidence from 5 years’ evaluation in Afghanistan. PLoS Med. 2011;8:e1001066.

    PubMed  PubMed Central  Article  Google Scholar 

  69. Chen HF, Hou YH, Chang RE. Application of the balanced scorecard to an academic medical center in Taiwan: the effect of warning systems on improvement of hospital performance. J Chinese Med Assoc. 2012;75:530–5.

    Article  Google Scholar 

  70. Khan MM, Hotchkiss RD, Dmytraczenko T, Zunaid AK. Use of a balanced scorecard in strengthening health systems in developing countries: an analysis based on nationally representative Bangladesh health facility survey. Int J Health Plann Manage. 2013;28:202–15.

    PubMed  Article  Google Scholar 

  71. Ajami S, Ebadsichani A, Tofighi S, Tavakoli N. Medical records department and balanced scorecard approach. J Educ Health Promot. 2013;2:1–6.

    Article  Google Scholar 

  72. Mutale W, Stringer J, Chintu N, Chilengi R, Mwanamwenge MT, Kasese N, et al. Application of balanced scorecard in the evaluation of a complex health system intervention: 12 months post intervention findings from the BHOMA Intervention : a cluster randomised trial in Zambia. PLoS ONE. 2014;9:1–10.

  73. Rowe JS, Natiq K, Alonge O, Gupta S, Agarwal A, Peters DH. Evaluating the use of locally-based health facility assessments in Afghanistan: a pilot study of a novel research method. Confl Health. 2014;8:1–9.

    Article  Google Scholar 

  74. Edward A, Osei-bonsu K, Branchini C, Yarghal T, Arwal SH, Naeem AJ. Enhancing governance and health system accountability for people centered healthcare: an exploratory study of community scorecards in Afghanistan. BMC Health Serv Res. 2015;15:1–15.

    Article  Google Scholar 

  75. Rabbani F, Pradhan NA, Zaidi S, Azam SI, Yousuf F. Service quality in contracted facilities. Int J Health Care Qual Assur. 2015;28:520–31.

    PubMed  Article  Google Scholar 

  76. Teklehaimanot HD, Teklehaimanot A, Tedella AA, Abdella M. Use of balanced scorecard methodology for performance measurement of the health extension program in Ethiopia. Am J Trop Med Hyg. 2016;94:1157–69. https://doi.org/10.4269/ajtmh.15-0192.

    Article  PubMed  PubMed Central  Google Scholar 

  77. Catuogno S, Arena C, Saggese S, Sarto F. Balanced performance measurement in research hospitals: the participative case study of a haematology department. BMC Health Serv Res. 2017;17:1–11.

    Article  Google Scholar 

  78. Gao H, Chen H, Feng J, Qin X, Wang X, Liang S, et al. Balanced scorecard-based performance evaluation of Chinese county hospitals in underdeveloped areas. J Int Med Res. 2018;46:1947–62. https://doi.org/10.1177/0300060518757606.

    Article  PubMed  PubMed Central  Google Scholar 

  79. Ebrahimpour A, Alimohammadzadeh K, Maher A. Evaluating the health reform plan in Ayatollah Taleghani Hospital, Tehran in 2010–2017 using the balanced scorecard. Hakim Heal Sys Res. 2019;22:100–11.

    Google Scholar 

  80. Widyasari NLG, Adi NR. Balanced scorecard implementation in the government hospital. Russ J Agric Socio-Economic Sci. 2019;93:285–91.

    Google Scholar 

  81. Mabuchi S, Alonge O, Tsugawa Y, Bennett S, Bennett S. Measuring management practices in primary health care facilities – development and validation of management practices scorecard in Nigeria. Glob Health Action. 2020;13:1763078.

    PubMed  PubMed Central  Article  Google Scholar 

  82. Manolitzas P, Grigoroudis E, Christodoulou J, Matsatsinis N. S-MEDUTA: Combining balanced scorecard with simulation and MCDA techniques for the evaluation of the strategic performance of an emergency department. In: Vlamos P, editor. GeNeDis 2018 computational biology and bioinformatics. Switzerland: Springer; 2020. p. 1–22.

    Google Scholar 

  83. Gonzalez J, Jaber J, Barker PC, Markham MJ, Kittelson SM. Palliative care scorecard: an update for the outpatient setting. J Palliat Med. 2020;23:306–7.

    PubMed  Article  Google Scholar 

  84. Peters DH, Noor AA, Singh LP, Kakar FK, Hansen PM, Burnhama G. A balanced scorecard for health services in Afghanistan. Bull World Health Organ. 2007;85:146–51.

    PubMed  PubMed Central  Article  Google Scholar 

  85. Olivas-Martínez A, Cárdenas-Fragoso JL, Jiménez JV, Lozano-Cruz OA, Ortiz-Brizuela E, Tovar-Méndez VH, et al. In-hospital mortality from severe COVID-19 in a tertiary care center in Mexico City; causes of death, risk factors and the impact of hospital saturation. PLoS One. 2021;16(2):1–17.

    Article  CAS  Google Scholar 

  86. Baker GR, Pink GH. A Balanced Scorecard for Canadian Hospitals. Healthc Manag Forum. 1995;8:7–13.

    CAS  Article  Google Scholar 

  87. Kalender ZT, Vayvay Ö. The fifth pillar of the balanced scorecard: sustainability. Procedia - Soc Behav Sci. 2016;235:76–83.

    Article  Google Scholar 

  88. Rahimi H, Khammar-nia M, Kavosi Z, Eslahi M. Indicators of hospital performance evaluation : a systematic review. Int J Hosp Res. 2014;3:199–208.

  89. Khullar D, Bond AM, Schpero WL. COVID-19 and the Financial Health of US Hospitals. JAMA. 2020;323:2127–8. https://doi.org/10.1001/jama.2020.6269.

    CAS  Article  PubMed  Google Scholar 

  90. Raghuvanshi VP, Raghuvanshi SP. Implications and future strategies on cost management for hospitals during and after COVID-19. Int J Community Med Public Heal. 2020;7:2405.

    Article  Google Scholar 

  91. Coyle D, Manley A. Productivity in UK healthcare during and after the Covid-19 pandemic. 2021.

    Book  Google Scholar 

  92. Griffin S. Covid-19: waiting times in England reach record highs. BMJ. 2020;370:m3557.

    PubMed  Article  Google Scholar 

  93. Cleary SM, Wilkinson T, Tamandjou Tchuem CR, Docrat S, Solanki GC. Cost-effectiveness of intensive care for hospitalized COVID-19 patients: experience from South Africa. BMC Health Serv Res. 2021;21:1–10.

    Article  Google Scholar 

  94. Jeffery MM, D’Onofrio G, Paek H, Platts-Mills TF, Soares WE, Hoppe JA, et al. Trends in emergency department visits and hospital admissions in health care systems in 5 states in the first months of the COVID-19 Pandemic in the US. JAMA Intern Med. 2020;180:1328–33.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  95. Rennert-May E, Leal J, Thanh NX, Lang E, Dowling S, Manns B, et al. The impact of COVID-19 on hospital admissions and emergency department visits: a population-based study. PLoS ONE. 2021;16:e0252441. https://doi.org/10.1371/journal.pone.0252441.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  96. Shirazi H, Kia R, Ghasemi P. Ranking of hospitals in the case of COVID-19 outbreak: a new integrated approach using patient satisfaction criteria. Int J Healthc Manag. 2020;13:312–24. https://doi.org/10.1080/20479700.2020.1803622.

    Article  Google Scholar 

  97. Moynihan R, Sanders S, Michaleff ZA, Scott AM, Clark J, To EJ, et al. Impact of COVID-19 pandemic on utilisation of healthcare services: a systematic review. BMJ Open. 2021;11:11–7.

    Article  Google Scholar 

  98. Kamel MA, Mousa ME-S. Measuring operational efficiency of isolation hospitals during COVID-19 pandemic using data envelopment analysis: a case of Egypt. Benchmarking An Int J. 2021;ahead-of-p ahead-of-print. doi:https://doi.org/10.1108/BIJ-09-2020-0481.

  99. Sodhi J, Satpathy S, Arora P. Role of hospital supportive services in COVID-19. Int J Infect Control. 2020;16:19–20.

    Article  Google Scholar 

  100. Wen X, Wang F, Li X, Gu H. Study on the Knowledge, Attitude, and Practice (KAP) of nursing staff and influencing factors on COVID-19. Front Public Heal. 2021;8:1–6.

    Google Scholar 

  101. Limbu DK, Piryani RM, Sunny AK. Healthcare workers’ knowledge, attitude and practices during the COVID-19 pandemic response in a tertiary care hospital of Nepal. PLoS One. 2020;15 11 November:7–13. doi:https://doi.org/10.1371/journal.pone.0242126.

  102. Huang H, Zhao WJ, Li GR. Knowledge and psychological stress related to COVID-19 among nursing staff in a hospital in China: Cross-sectional survey study. JMIR Form Res. 2020;4:1–8.

    Article  Google Scholar 

  103. Alrawashdeh HM, Al-Tammemi AB, Alzawahreh MK, Al-Tamimi A, Elkholy M, Al Sarireh F, et al. Occupational burnout and job satisfaction among physicians in times of COVID-19 crisis: a convergent parallel mixed-method study. BMC Public Health. 2021;21:811.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  104. Müller SM, Mueller GF, Navarini AA, Brandt O. National publication productivity during the COVID-19 Pandemic—A preliminary exploratory analysis of the 30 countries most affected. Biology (Basel). 2020;9:271.

    Google Scholar 

  105. The Centers for Disease Control and Prevention (CDC). Comprehensive Hospital Preparedness Checklist for Coronavirus Disease 2019 (COVID-19). https://www.cdc.gov/coronavirus/2019-ncov/hcp/hcp-hospital-checklist.html. Accessed 25 Jun 2021.

  106. Members in Action. Four stages of planning and implementation during Covid-19: One rural Hospital’s preparations. American Hospital Association. 2020; April:1–3. https://www.aha.org/system/files/media/file/2020/04/rural-members-in-action-case-study-four-stages-planning-implementation-during-covid-19-one-rural-hospital-preparations-marshall-medical-center.pdf. Accessed 25 Jun 2021.

  107. Burlea-Schiopoiu A, Ferhati K. The managerial implications of the key performance indicators in healthcare sector: a cluster analysis. Healthc. 2021;9:1–20.

    Google Scholar 

  108. Martin N. Finding a new normal: hospital governance best practices during COVID-19. Healthc Q. 2020;23:24–8. https://doi.org/10.12927/hcq.2020.26337.

    Article  PubMed  Google Scholar 

  109. Sharma A, Borah SB, Moses AC. Responses to COVID-19: The role of governance, healthcare infrastructure, and learning from past pandemics. J Bus Res. 2021;122 January:597–607. doi:https://doi.org/10.1016/j.jbusres.2020.09.011.

  110. McCabe R, Schmit N, Christen P, D’Aeth JC, Løchen A, Rizmie D, et al. Adapting hospital capacity to meet changing demands during the COVID-19 pandemic. BMC Med. 2020;18:1–12.

    Article  CAS  Google Scholar 

  111. Savitsky B, Radomislensky I, Hendel T. Nurses’ occupational satisfaction during Covid-19 pandemic. Appl Nurs Res. 2021;59 January:151416. doi:https://doi.org/10.1016/j.apnr.2021.151416.

  112. Feng L, Yin R. Social support and hope mediate the relationship between gratitude and depression among front-line medical staff during the pandemic of COVID-19. Front Psychol. 2021;12:1–7.

    Google Scholar 

  113. Ng KYY, Zhou S, Tan SH, Ishak NDB, Goh ZZS, Chua ZY, et al. Understanding the psychological impact of COVID-19 pandemic on patients with cancer, their caregivers, and health care workers in Singapore. JCO Glob Oncol. 2020;:1494–509. doi:https://doi.org/10.1200/GO.20.00374.

  114. Olanipekun T. The impact of COVID-19 testing on length of hospital stay and patient flow in hospitals. J Community Hosp Intern Med Perspect. 2021;11:180–3. https://doi.org/10.1080/20009666.2020.1866249.

    Article  PubMed  PubMed Central  Google Scholar 

  115. Jamshidi B, Zargaran SJ, Bekrizadeh H, Rezaei M, Najafi F. Comparing length of hospital stay during COVID-19 pandemic in the USA, Italy and Germany. Int J Qual Heal Care. 2021;33:1–11. https://doi.org/10.1093/intqhc/mzab050.

    Article  Google Scholar 

  116. Rees EM, Nightingale ES, Jafari Y, Waterlow NR, Clifford S, Carl CA, et al. COVID-19 length of hospital stay: a systematic review and data synthesis. BMC Med. 2020;18:270.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  117. Pascua FV, Diaz O, Medina R, Contreras B, Mistroff J, Espinosa D, et al. A multi-mechanism approach reduces length of stay in the ICU for severe COVID-19 patients. PLoS One. 2021;16 1 January:1–15. doi:https://doi.org/10.1371/journal.pone.0245025.

  118. World Health Organisation (WHO). Infection prevention and control during health care when COVID-19 is suspected or confirmed: Interim guidance. Interim Guidance. 2020; 29 June 2020:1–5. https://apps.who.int/iris/rest/bitstreams/1272420/retrieve. Accessed 25 Jun 2021.

  119. Narwal S, Jain S. Building resilient health systems: patient safety during COVID-19 and lessons for the future. J Health Manag. 2021;23:166–81.

    Article  Google Scholar 

  120. Gandhi TK, Singh H. Reducing the risk of diagnostic error in the COVID-19 era. J Hosp Med. 2020;15:363–6.

    PubMed  PubMed Central  Article  Google Scholar 

  121. Denning M, Goh ET, Scott A, Martin G, Markar S, Flott K, et al. What has been the impact of COVID-19 on safety culture? A case study from a large metropolitan healthcare trust. Int J Environ Res Public Health. 2020;17:1–12.

    Article  Google Scholar 

  122. Jabarpour M, Dehghan M, Afsharipour G, HajipourAbaee E, MangolianShahrbabaki P, Ahmadinejad M, et al. The impact of COVID-19 outbreak on nosocomial infection rate: a case of Iran. Can J Infect Dis Med Microbiol. 2021;2021:Cdc.

    Article  Google Scholar 

  123. Ashinyo ME, Dubik SD, Duti V, Amegah KE, Ashinyo A, Asare BA, et al. Infection prevention and control compliance among exposed healthcare workers in COVID-19 treatment centers in Ghana: A descriptive cross-sectional study. PLoS One. 2021;16 3 March:1–13. doi:https://doi.org/10.1371/journal.pone.0248282.

  124. Das AK, Islam MN, Billah MM, Sarker A. COVID-19 pandemic and healthcare solid waste management strategy – A mini-review. Sci Total Environ. 2021;778: 146220. https://doi.org/10.1016/j.scitotenv.2021.146220.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  125. Kalantary RR, Jamshidi A, Mofrad MMG, Jafari AJ, Heidari N, Fallahizadeh S, et al. Effect of COVID-19 pandemic on medical waste management: a case study. J Environ Heal Sci Eng. 2021;19:831–6. https://doi.org/10.1007/s40201-021-00650-9.

    CAS  Article  Google Scholar 

  126. Mekonnen B, Solomon N, Wondimu W. Healthcare waste status and handling practices during COVID-19 Pandemic in Tepi General Hospital. Ethiopia J Environ Public Health. 2021;2021:6614565.

    PubMed  Google Scholar 

  127. World Health Organisation (WHO). Water, sanitation, hygiene, and waste management for SARS-CoV-2, the virus that causes COVID-19. Interim guidance. 2020; 29 July:1–11. https://www.who.int/publications/i/item/water-sanitation-hygiene-and-waste-management-for-the-covid-19-virus-interim-guidance. Accessed 25 Jun 2021.

  128. de Aguiar Hugo A, Lima R da S. Healthcare waste management assessment: challenges for hospitals in COVID-19 pandemic times. Waste Manag Res J a Sustain Circ Econ. 2021;:0734242X2110103. doi:https://doi.org/10.1177/0734242X211010362.

  129. Maalouf A, Maalouf H. Impact of COVID-19 pandemic on medical waste management in Lebanon. Waste Manag Res. 2021;39:45–55. https://doi.org/10.1177/0734242X211003970.

  130. Abdelhadi A. Patients’ satisfactions on the waiting period at the emergency units. Comparison study before and during covid-19 pandemic. J Public health Res. 2021;10:15.

    Article  Google Scholar 

  131. Bin Traiki TA, AlShammari SA, AlAli MN, Aljomah NA, Alhassan NS, Alkhayal KA, et al. Impact of COVID-19 pandemic on patient satisfaction and surgical outcomes: a retrospective and cross sectional study. Ann Med Surg. 2020;58:14–9. https://doi.org/10.1016/j.amsu.2020.08.020.

    Article  Google Scholar 

  132. Wang M, Hu C, Zhao Q, Feng R, Wang Q, Cai H, et al. Acute psychological impact on COVID-19 patients in Hubei: a multicenter observational study. Transl Psychiatry. 2021;11:133.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  133. Pigozzi E, Tregnago D, Costa L, Insolda J, Turati E, Rimondini M, et al. Psychological impact of Covid-19 pandemic on oncological patients: a survey in Northern Italy. PLoS One. 2021;16(3):1–13.

    Article  CAS  Google Scholar 

  134. Rodrigues MENG, Belarmino A da C, Lopes Custódio L, Lima Verde Gomes I, Rodrigues Ferreira Júnior A. Communication in health work during the COVID-19 pandemic. Investig y Educ en Enfermería. 2020;38:1–12.

    Google Scholar 

  135. Al-Zyoud W, Oweis T, Al-Thawabih H, Al-Saqqar F, Al-Kazwini A, Al-Hammouri F. The psychological effects of physicians’ communication skills on covid-19 patients. Patient Prefer Adherence. 2021;15:677–90.

    PubMed  PubMed Central  Article  Google Scholar 

  136. Samarasekara K. ‘Masking’ emotions: doctor–patient communication in the era of COVID-19. Postgrad Med J. 2021;97:406. https://doi.org/10.1136/postgradmedj-2020-138444.

    Article  PubMed  Google Scholar 

  137. Dyer C. Covid-19: Rules on sharing confidential patient information are relaxed in England. BMJ. 2020;369:m1378. https://doi.org/10.1136/bmj.m1378.

    Article  PubMed  Google Scholar 

  138. Zlobina A. Personal data of thousands of Covid-19 patients leaked in Moscow. Human Rights Watch. https://www.hrw.org/news/2020/12/15/personal-data-thousands-covid-19-patients-leaked-moscow. Accessed 25 Jun 2021.

  139. Hammoudeh W, Kienzler H, Meagher K, Giacaman R. Social and political determinants of health in the Occupied Palestine Territory (OPT) during the COVID-19 pandemic: who is responsible? BMJ Glob Heal. 2020;5:e003683.

    Article  Google Scholar 

  140. He W, Zhang Z (Justin), Li W. Information technology solutions, challenges, and suggestions for tackling the COVID-19 pandemic. Int J Inf Manage. 2021;57:102287. https://doi.org/10.1016/j.ijinfomgt.2020.102287.

    Article  PubMed  Google Scholar 

  141. Asadzadeh A, Pakkhoo S, Saeidabad MM, Khezri H, Ferdousi R. Information technology in emergency management of COVID-19 outbreak. Informatics Med Unlocked. 2020;21:100475. https://doi.org/10.1016/j.imu.2020.100475.

    Article  Google Scholar 

  142. Monaghesh E, Hajizadeh A. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. BMC Public Health. 2020;20(1):1193. https://doi.org/10.1186/s12889-020-09301-4.

Download references

Acknowledgements

We want to thank Dr. Vahideh Moghaddam, Diêgo Andrade, and Wang Zhe for their help in translating and data extraction of Persian, Spanish, and Chinese articles, respectively. Additionally, we would like to thank Dr. Duha Shellah for contributing to revising the manuscript. Last, special thanks to Mr. Amjad Amer for his supports in figures preparation.

Funding

The research was financed by the Thematic Excellence Program 2021 Health Sub-programme of the Ministry for Innovation and Technology in Hungary, within the framework of the EGA-10 project of the University of Pécs. Open access funding provided by University of Pécs.

Author information

Affiliations

Authors

Contributions

This systematic review was planned and designed by F. A, S. L, S.H., I. B & D.E. Specifically, F. A & S. H performed the processes of searching for eligible studies and data extraction. F. A performed the statistical analysis, the interpretation of the data, the preparation of figures, the RoB assessment, the implication of dimensions in COVID-19, and writing the manuscript. All of the authors participated in revising the final manuscript. S.L., I. B & D. E supervised this systematic review. Ultimately, all of the authors approved the final manuscript and agreed to be personally accountable for the author's contributions and to ensure that the accuracy and integrity of any part of the work were appropriately investigated and resolved.

Corresponding author

Correspondence to Faten Amer.

Ethics declarations

Ethics approval and consent to participate

Not applicable since it is a systematic review.

Consent for publication

Not Applicable.

Competing interests

The authors have declared that no competing interests exist.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Amer, F., Hammoud, S., Khatatbeh, H. et al. A systematic review: the dimensions to evaluate health care performance and an implication during the pandemic. BMC Health Serv Res 22, 621 (2022). https://doi.org/10.1186/s12913-022-07863-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-07863-0

Keywords

  • Balanced scorecard
  • Performance
  • Indicators
  • Health
  • Hospital
  • Evaluation
  • Assessment
  • COVID-19