This study has examined the utility of a comprehensive evaluation framework and its derived questionnaire for furthering the understanding and measurement of the component parts of effective hospital resilience. There are 8 key domains of hospital resilience, and as a result four key factors were extracted from them. Among these factors, emergency medical care is the most important capability, while others (hospital safety, management mechanism, and disaster resources) are supporting capability to guarantee its continuity and surging. This framework of hospital resilience provides a starting point for integrating these key components of hospital resilience together into a comprehensive disaster management framework (including prevention, preparedness, responsiveness, and recovery and adaptation phases). It also seeks to make an achievable goal of improving hospital pre-disaster strength (robustness) and promoting rapidity of response and recovery. This goal can be achieved through a wide range of management approaches including redundancy of processes and resources, and resourcefulness (or flexibility) of plans or strategies (i.e.: can be reflected by some key variables in the survey) [22–26].
Considerable variability in the scope of disaster resilience arrangement of hospitals in the Province was identified through a survey conducted using the self-report questionnaire. We have stratified our analyses by different level of hospitals. It was noticed that in some key areas (e.g., safety evaluation, planning and cooperation, MOUs, personal protective equipment, rescue, surge capacity and drills), there was a difference in disaster resilience arrangements between tertiary A and tertiary B hospitals. This may be due to different levels or types of hospitals having divergent functions in disasters. For example, most (92.3%) of the hospitals that have been assigned missions were tertiary A, and thus they should be more resilient to disasters for health service supply, and should have better arrangements in the above areas for disaster preparedness and response.
This paper offered a four-factor structure as a way of modelling the overall level of hospital resilience and the level of each factor independently. Thus the questionnaire can be used to provide a helpful and comprehensive instrument for assisting hospitals to assess their level of resilience at a regional or a district level in regard to disasters, and assist them in identifying areas for further strengthening their resilience capability through comparison with similar components of other hospitals. The evaluation framework and its key measures in the questionnaire may inform the development of hospital resilience evaluation in other countries.
Similar indicators in this study can be compared with other studies, especially on hospital disaster preparedness and management [13, 16]. One survey has been conducted in 2005 to evaluate secondary and tertiary hospitals of Shandong . Comparing its results, it was found that the percentage of most similar indicators in our study is reasonable higher, such as the percentages of: syndromic surveillance systems, single-hazard disaster plans, public and mass media communication protocol, stockpiles of emergency resources, and training programs and drills. Thus, to some extent, it was validated the representativeness of the sample in this study to reflect the status of the province. Additionally, it was found that hospital disaster preparedness in Shandong province is close to Beijing (the capital city), and it is above the average level of preparedness in China, due to economic factors and other factors . It is expected that hospital disaster ability in many other parts of China may lag well behind that of Shandong province. Thus, understanding the status of hospital disaster resilience in this province can be used as the first step in planning effective hospital resilience.
After the SARS crisis, the preparedness of hospitals in China especially for infectious diseases has improved significantly . Our survey revealed that these tertiary hospitals had devised disaster plans and command structures. Almost all of the surveyed hospitals possessed strategies to evacuate and protect existing patients when there is risk in hospitals. Most of them had syndromic surveillance systems. Many had different personal protective equipment and had relevant training programs. A large percentage of them had stockpiles of emergency drugs and resources and had the ability to accept more than 30 cases of infectious diseases within a short period.
These results also highlighted the following shortfall areas in current hospital disaster resilience in Shandong. Firstly, for disaster management mechanism, in US, nearly 67.9% had specific plans for all the essential individual hazards in 2008 . Comparatively, disaster plans in this Province of China had less considerable scope for improving their preparedness for natural disasters, biological, nuclear radiation and other terrorist attacks.
Secondly, for disaster resources, simply stockpiling materials fails to achieve adequate hospital surge capacity, especially in the aftermath of a catastrophic disaster. The community should have functional inter-hospital arrangements to share personnel and resources . In the US, nearly 87.8% of hospitals had MOUs with other hospitals to transfer general patients, 84.1% had contract with other agencies to share suppliers, and more than 70% of hospitals performed mass casualty drills with outside organizations . However, in the Province of China, less than half of the responding hospitals had signed contracts with drug-supplies, and less than one third had signed MOUs with other hospitals to share resources and staff. Also less than half had attended the local coordinating meeting, and only one fifth had attended community-wide drills. The lack of cross-institutional interaction and coordination would likely hinder the availability of resources in a community, and limit timely disaster response.
Finally, continuity of medical care is amongst the most important objectives for prompt and effective response to emergencies. As the experience from developed courtiers, on-site rescue can be enhanced either through dispatched rescue teams (be equipped with living supplies for 3 to 5 days and portable medical equipment) and advanced ‘portable hospitals’ (be equipped with various functional vehicles that can be used for operating surgery, accepting patients, on-site command and communication and etc.) . However, there is still insufficiency of on-site medical rescue, especially a lack of “portability” of critical care service (i.e., patient transport and bringing care to the patient). These two models of on-site rescue still need to be further developed, as they are scalable, mobile and can surge medical care service significantly even after catastrophic disasters . Additionally, medical care capability requires significant surge capacity during disasters, with a critical feature of hospital staffed beds . In US, most hospitals had plans and flexible procedures for surging staffed beds . Also it has been surveyed that in the hospitals of Kentucky, the surge capacity equal to 27% of licensed beds . However, in this study, only less than one fourth of responding hospitals had a wide variety of flexible procedures for surging their beds and emergency staff. The surge capacity within 24 hours is 12.52% of fixed beds, which is relatively low.
Cohesive approaches have been identified using the evaluation framework and its key variables. They can be used by hospital managers and health authorities to enhance general practices to achieve effective disaster resilient. It also can used to assess hospitals, so as to identify the vulnerabilities and improve disaster capability further. These approaches include:
Hospital safety: (1) Evaluation of locally prioritized hazards, and enforcement of safety standards that need to meet or exceed the local standards; [30, 31] (2) Evacuation plan in place and have special procedures to protect and evacuate vulnerable people when there is risk within the hospital.
Disaster mechanism: (3) The existence of disaster plans that have been developed in advance of a disaster, taking into consideration the communities’ resources, hazards and other unique factors; [2, 13] (4) The establishment of a specific department to take responsibility of incident command and control, crisis communication and cooperation, and after-event recovery;  (5) Incorporation of the hospital into the overall local disaster planning, including inter-facility cooperation and alternative plans to transfer patients to other hospitals if the hospital is partly destroyed or become unusable ;
Disaster resources: (6) Stockpile of self-sufficient resources and emergency drugs for at least 48 to 72 hours, so as to cope with major disasters initially;  (7) Establishment of MOUs with other hospitals for transferring patient and the sharing of staffing, equipment, and supplies;  A community-wide, integrated, inter-agency network should be built, with local hospitals working together to surge overall capacity collectively .
Emergency medical care: (8) Transportation of the medical staff or transferring patients to hospitals in a timely manner, and the provision of medical care service on site, which can be in the form of rescue teams or ‘portable hospital’ especially during catastrophic disasters;  (9) Disaster surge planning should be devised in advance by adoption of a wide variety of flexible strategies (e.g., disaster triage, ability to surge beds and staff, ability to transfer patients, early discharge of patients) for surging medical demands from wide-spread infectious diseases or mass casualty incidents; [27, 33, 34] (10) Development of hospital internal conditions (e.g., space, beds, treatment protocols and on-call specialists) for treating patients according to type and magnitude of event(s);  (11) Systematic and ongoing training and drill staff for emergency medical care skills, equipment usage and disaster management skills in high risk communities .
The current study has several limitations. Firstly, the likelihood of non-response bias was likely to exist. As a relatively larger percentage of tertiary A hospitals replied to the survey than the tertiary B and tertiary C hospitals. Although two reminders were sent to the hospital coordinators, there were still 9 hospitals who failed to attend the survey. The follow-up telephones demonstrated that they could not assign the responsible staff to fill the survey, or they are lack of relevant data. Thus it is very possible that the 41 participating hospitals may have relatively good backgrounds of disaster rescue than other 9 nonparticipation hospitals. Also, we suspect that the participating hospitals are better prepared in terms of disaster management than the non-participating hospitals. Secondly, due to the sample size (n = 41), it was possible that not all the significant difference of the mean score of each domain was tested statistically between different hospital categories (as illustrated in Table 1). Despite the limitation, the sample accounted for 52.1% of total hospitals that the study targeted, and did get a response rate over 80%. Thirdly, as the findings are self-reported by the respondents there may be a bias in their reporting. While the inclusion of official documents from Provincial Health Bureaus may have encouraged respondents to complete survey, this may have also been interpreted as an official assessment, thus leading some hospital representatives to overestimate their capability. Fourthly, due to ethical issues, the surveyed hospitals have to be anonymous which impedes comparison with their actual levels of preparedness. Finally, the study was undertaken only in one Province of China. And due to the limitation of funding and investigation time, stratified sampling was used in this study rather than investigating all the tertiary hospitals.