Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Assessing safety climate in prehospital settings: testing psychometric properties of a common structural model in a cross-sectional and prospective study

BMC Health Services Research volume 19, Article number: 674 (2019) Cite this article

Abstract

Background

Little research exists on patient safety climate in the prehospital context. The purpose of this article is to test and validate a safety climate measurement model for the prehospital environment, and to explore and develop a theoretical model measuring associations between safety climate factors and the outcome variable transitions and handoffs.

Methods

A web-based survey design was utilized. An adjusted short version of the instrument Hospital Survey on Patient Safety Culture (HSOPSC) was developed into a hypothetical structural model. Three samples were obtained. Two from air ambulance workers in 2012 and 2016, with respectively 83 and 55% response rate, and the third from the ground ambulance workers in 2016, with 26% response rate. Confirmatory factor analysis (CFA) was applied to test validity and psychometric properties. Internal consistency was estimated and descriptive data analysis was performed. Structural equation modelling (SEM) was applied to assess the theoretical model developed for the prehospital setting.

Results

A post-hoc modified instrument consisting of six dimensions and 17 items provided overall acceptable psychometric properties for all samples, i.e. acceptable Chronbach’s alphas (.68–.86) and construct validity (model fit values: SRMR; .026–.056, TLI; .95–.98, RMSEA; .031–.052, CFI; .96–.98). A common structural model could also be established.

Conclusions

The results provided a validated instrument, the Prehospital Survey on Patient Safety Culture short version (PreHSOPSC-S), for measuring patient safety climate in a prehospital context. We also demonstrated a positive relation between safety climate dimensions from leadership to unit level, from unit to individual level, and from individual level on the outcome dimension related to transitions and handoffs. Safe patient transitions and handoffs are considered an important outcome of prehospital deliveries; hence, new theory and a validated model will constitute an important contribution to the prehospital safety climate research.

Peer Review reports

Background

The prehospital environment is characterized by time pressure, high activity, uncertain situations, changing environments and a high dependency on the emergency medical providers’ teamwork, competence and communication abilities: a demanding mix, prone to errors. Undesirable events and near misses seems to be more recurrent within this environment than what is reported or shared [1]. Compared to the hospital context, little research exists on patient safety in the prehospital context [2,3,4] and on the relationship between organizational factors creating barriers and increased patient safety [5]. Safety culture is considered an important theme and premise for continued improvement of patient safety [6, 7]. There is mounting evidence on the relationship between improved safety culture and less occurrence of adverse events [8,9,10,11,12] and improved safety performance [13, 14].

Different from safety culture, safety climate is defined as “surface features of the safety culture from attitudes and perceptions of individuals at a given point in time” [15], i.e. safety climate research is viewed as a ‘snapshot’ of the safety culture [16]. A growing number of studies report on the value of safety climate assessments [14, 17]. Research also indicates a strong connection between safety climate and safety behavior [18,19,20], and that safety-related outcomes such as accidents may be predicted by safety climate assessments [21]. In health care such assessments are conducted to reveal, keep track of, and manage safety issues, in addition to evaluate interventions and trends [6, 18, 22]. An area viewed as critical in the eyes of the Emergency Medical Services’ (EMS) providers is transitions and handoffs, since this brief window provides the opportunity to influence the further course of their patients’ care [23]. To make improvements in transitions and handoffs, the first step is for the policy decision-makers to understand how the workers perceive their organization’s patient safety climate [24].

Survey methods are considered a good approach to perform safety climate assessments [25], and several instruments have been developed for application within health care services [15], some of which are also intended for use within acute hospital settings [26]. One such instrument is the Hospital Survey On Patient Safety Culture (HSOPSC), developed in 2004 by the Agency for Healthcare Research and Quality (AHRQ), and later translated to several languages [27]. The Norwegian version of the HSOPSC instrument was recently adjusted and validated for the prehospital environment [28]. Another instrument is a rebuilt shorter version, the HSOPSC-S, which has been validated for the hospital and the petroleum sector [29]. HSOPSC-S was developed with the help of exploratory factor analysis (EFA) to explore the possibility of a common structural model for measuring the safety climate in two different sectors: the health care sector and the petroleum sector. The HSOPSC-S is based on a multilevel theoretical framework, emphasizing that all levels in an organization have safety functions and influence performance at the individual level [29].

In the prehospital environment in Norway, car-, boat-, and air ambulances constitute the main part of the EMS activities. Although the different types of ambulances has similar goals and tasks, there are also substantial differences, such as team compositions, worker competence groups, care and treatment processes, and the physical environment [30]. This makes it valuable to provide a validated safety-climate instrument, suitable for application in these different subgroups.

Due to the fast-pace work culture in the prehospital environment, a shorter instrument seems beneficial, as it may be challenging to perform frequent assessments of the patient safety climate. There is also a focus in the Norwegian health sector on developing shorter tools for the measurement of health, safety and environment (HSE) and culture [31]. As the prehospital setting differs from the hospital setting, it is necessary to perform a new test of the psychometric properties for HSOPSC-S, preferably split into different subgroups. Such testing is beneficial, as research on psychometric properties, including the reliability and validity of replicated instruments, is continually needed [32,33,34,35,36]. It is also recommended to develop theoretical models to understand the safety climate impact on outcomes [26]. The aims of our study were to:

  1. 1.

    Test psychometric properties and validate an adjusted version of Norwegian HSOPSC-S performed on three samples from the prehospital environment: two samples from the Helicopter Emergency Medical Services (HEMS; air ambulance) retrieved at different points of time and one sample from the ground ambulance.

  2. 2.

    Test and further develop a theoretical model using a structured equational modelling (SEM) approach to create a fit for three prehospital samples, i.e. to test the network of relationships between the variables in a theoretical framework of HSOPSC-S in a prehospital setting.

Theoretical background

Patient safety is a broad endeavor that requires thinking beyond the individual patient to consider the characteristics of the whole system [37]. It is reasonable to consider the prehospital domain as part of the whole healthcare system, and in particular as part of the treatment chain. The importance of performing research in this domain is reflected by the limited amount of prehospital research on patient safety, compared to the amount of hospital research. Taking a multilevel perspective when performing assessments of safety climate in organizations has been suggested [18, 38]. Figure 1 is a simple illustration of a hypothetical prehospital multilevel system model, on which we build our theoretical structured framework.

Fig. 1
figure 1

Hypothetical prehospital multilevel model

Full size image

Top and unit management levels

It is well known that leadership has a strong cultural influence on an organization through its beliefs and values [39] and that effective leadership promotes better patient outcomes [1]. Top management’s safety-related attitudes and behaviors will form the basis for healthcare providers’ safety behavior and hence the organization’s safety performance [1]. Middle management, i.e. unit management, is responsible for transforming priorities and values into operating procedures and action guidelines [40] and the view that top management’s influence on unit management supervisory practices is aligned with a multilevel perspective on safety climate [41, 42]. Supervisors achieve higher levels of safety compliance from their subordinates if the subordinates perceive that the supervisors prioritize safety [43]. The decentralized nature of the prehospital setting, relative to the hospital setting, is an important aspect in this matter. One might speculate that prehospital unit management might have a more important role, compared to that of their hospital peers, in their influence on the integration of systems, patient safety interventions and staff performance. With the help of coaching, mentoring and the fostering of mutual respect, staff may incorporate management’s priorities and values into their daily work, helping them to develop their own leadership potential [7].

Different leadership styles seem to influence safety-relevant behavior [41]. Several studies suggest that there is a correlation between transformational leadership and organizational outcomes [43,44,45]. Transformational leadership can also function as an antecedent of climate strength [46]. Transformational leadership is conducted on organizational workers to raise their awareness of the importance of task outcomes, to meet the team members’ needs, and to induce them to prioritize the organizational goals [47]. Effective leadership promotes a setting in which healthcare providers are treated respectfully, consequently improving their performance [1]. It is also essential that unit leadership behavior creates an atmosphere in which team members feel they can communicate openly and participate in decision-making [48] and speak up if they have safety concerns [49]. In addition, the prehospital unit management conducts leadership in the local unit by assigning teams to the different unit transport vehicles, from which the emergency dispatcher allocates the designated teams’ medical emergencies, transport assignments or similar. In the prehospital environment, critical tasks are performed in teams without direct supervision from the unit management, which implies that the leadership task of putting together teams influences team outcomes.

Unit and individual level

For different high-stake contexts, a close relationship between teamwork and performance has been identified [50,51,52,53,54,55]. Teamwork is generally considered to be the basis for good patient care [56], and it is also valuable within fast-paced unpredictable environments [57]. Teamwork is also regarded as of critical importance in assuring patient safety [49, 58]. Poor teamwork and communication have emerged as contributions to medical errors [50, 51]; in particular, a lack of cross-monitoring of the actions of team members has been identified as a significant contributor to teamwork failure [54].

In acute healthcare settings, teams are hierarchical [1], and team leaders are responsible for executing operational procedures and action guidelines given by the unit management [42]. On one side, the positive attribute of hierarchy is the clear path of information and decision-making in critical situations. On the other side, hierarchy may hinder problem-solving, e.g. if team members leave everything to the team leader and do not contribute to collecting data or finding the best treatment option [1]. Although hierarchy, formal or informal, applies in the majority of the prehospital setting, HEMS is an exception to this, as the team members are responsible for different domains and tasks, according to their profession.

In addition to effective leadership and teamwork, key elements of a safety culture also include learning from errors and developing an open environment based on trust [7]. A recent study identified two effective interventions improving the governance of patient safety within organizations: simulation-based training is increasingly valued as an effective method to enhance safety knowledge and behavior, and, secondly, the use of well-designed incident reporting leads to increased learning, feedback and improvement within units [59]. Both are among many activities that could be undertaken to create organizational learning.

It is common to define organizational learning as change in the organization’s knowledge that occurs as a function of experience [60]. Organizational learning is a dynamic process [61], meaning that, in addition to occurring over time and between organizational, unit and individual levels, learning creates a tension between assimilating new learning (feed forward) and using what is already learned (feedback). For the purpose of assimilating new learning in the prehospital setting, it is important that sufficient resources for effective and efficient reporting are available [59]. It is also important to strive for an open and fair culture to achieve open discussions about things that went wrong and when, how, and in what context they occurred [62]. In addition, such a culture should encourage healthcare providers to report patient safety incidents [7].

The degree of risk for the patient in a decision-making situation depends on the healthcare providers’ (a) awareness and knowledge of threats, (b) experience and practice with similar situations, and (c) actual clinical competence [1]. Individual team members’ behavior is influenced by the team they work in, i.e. how they communicate, support and supervise each other [37]. Learning is also dependent on other individuals; e.g., inexperienced workers learn by observing more experienced colleagues’ actions and related consequences [1].

Outcome: transitions and handoffs

Patient transitions and handoffs in hospitals have the potential for errors [24, 63], and, recently, transitions and handoffs have gained greater attention, also in the prehospital domain [5, 23, 64,65,66,67]. Due to the characteristics of the prehospital context, transitions and handoffs are both challenging and complex as they involve a number of different people, such as both hospital and prehospital staff, patients, the public, and a range of communication technologies and formats [5]. Several factors could prevent effective and high-quality transitions and handoffs, e.g. lack of understanding between health care disciplines, inattention, variable quality and quantity of information exchanged, and busy and complex situations [68]. Some poor communication practices are rooted in individual behavior, such as not listening, relational issues, or misunderstandings [5]. Heavy workload in the emergency department may induce reduced “active listening” [64, 69,70,71]. Errors also tend to follow unsafe behavior, e.g. when workers do not follow procedures or rush to finish tasks [18]. A recent study found it disturbing that they observed no effective interventions to monitor or improve patient safety in the prehospital chain, considering the high number of patient transitions and handoffs between EMS organizations [59].

Good transitions and handoffs are associated with improved patient safety, continuity of patient care and improved decision-making [64, 71,72,73] and are clearly more than an exchange of information [65]. Quality of care is dependent on the transfer of patient information, and, since verbal transfer is often only 50% accurate, a technical solution is likely to improve patients’ outcome [67]. Nevertheless, due to the inherent complexity found in healthcare settings and communication tasks, this indicates that communication issues may not be solved by technical solutions alone [5]. The quality and quantity of information has been found to correlate with the ambulance personnel’s level of formal training and experience [71, 74]. To provide quality patient transitions and handoffs from prehospital to in-hospital, it is imperative to have ongoing formal learning in interdisciplinary teamwork, communication and a structured flexible framework in a supportive work environment [68]. Four potential improvements have been identified for transitions and handoffs from prehospital to in-hospital: direct communication between the EMS provider and the physician responsible for the patient’s care; interdisciplinary communication between hospital and prehospital staff; standardization of many of the aspects of transitions and handoffs; and improved information technology [23]. There seems to be a need for a broader conceptualization of patient transitions and handoffs, including social and organizational factors, some of which are embedded in the organization’s safety culture.

A recent study was performed on whether an organization’s safety culture factors influence effective handoffs within hospitals [24], especially the effect from an organization’s communication, teamwork, reporting and management cultures. They found that these safety culture composites influence different aspects of transitions and handoffs, i.e. information, responsibility and accountability. Viewing transitions of care as a more specific variant of patient safety culture attenuates ambiguity so that the stakeholders may more easily identify and align with the goals and processes of patient safety improvement programs [24]. Based on this, we made a major adjustment to the theoretical framework that constitutes the HSOPSC-S. The safety climate dimension “transitions and teamwork across units” was replaced with the safety climate dimension “transitions and handoffs” from the original HSOPSC, and, moreover, it was treated as an outcome dimension in our model.

Theoretical model and hypotheses

In summary, we hypothesize that the prehospital safety climate is aligned with a multilevel perspective, i.e. that the higher levels have an influence on the lower levels, as previously tested in the hospital sector and the petroleum sector [29]. Further, based on knowledge about the prehospital setting, theory and earlier research, we hereby argue that patient transitions and handovers must be considered an important outcome variable; handovers and transitions are the final ‘product’ of the prehospital chain. As it is found that an organization’s safety culture composites have an influence on patient transitions and handoffs [24], we hypothesize that all safety climate dimensions in HSOPSC-S have an influence on the outcome dimension “transitions & handoffs”. The theoretical, structured framework is based on the following hypotheses:

  1. a.

    Organizational management support for safety will enhance transitions and handoffs.

  2. b.

    Organizational management support for safety will enhance supervisor/manager expectations and actions promoting safety at the unit level.

  3. c.

    Manager expectations and actions promoting safety within units will enhance transitions and handoffs.

  4. d.

    Manager expectations and actions promoting safety within units will enhance learning, feedback, and improvement within units.

  5. e.

    Manager expectations and actions promoting safety within units will enhance teamwork within units.

  6. f.

    Learning, feedback, and improvement within units will enhance teamwork within units.

  7. g.

    Teamwork within units will enhance safety behavior.

  8. h.

    Learning, feedback, and improvement within units will enhance safety behavior.

  9. i.

    Teamwork within units will enhance transitions and handoffs.

  10. j.

    Learning, feedback, and improvement within units will enhance transitions and handoffs.

  11. k.

    Safety behavior will enhance transitions and handoffs.

Hypothesis a-h are consistent with the original theoretical framework [29], and hypothesis i-k are adjustments for this study. Figure 2 displays the hypothesized theoretical, structured framework.

Fig. 2
figure 2

Hypothetical structured framework. Note: Management support = “Management support for patient safety”, Management exp. & supp = “Manager expectations & actions promoting patient safety”, Learning, feedb & impr = “Learning, feedback & improvement within units”, Stop working in dangerous sit = “Stop working in dangerous situations”

Full size image

Methods

Population characteristics

In the prehospital environment in Norway, regional health trusts are responsible for the EMS activities (car-, boat-, and air ambulance). Regulations state that car ambulances should be staffed by at least two people; either two emergency medical technicians (EMT) or one EMT and another licensed health care worker with necessary EMS competence, e.g. a paramedic, a nurse or a physician [30]. For the boat ambulance, at least one person must be an EMT, in addition to the skipper. For both car- and boat ambulances, the transportation of critically ill patients may require special medical competence, often provided by accompanying healthcare personnel [30]. For the Helicopter Emergency Medical Services (HEMS; air ambulance), regulations state that the staff should consist of a physician/anesthesiologist and HEMS crew member (HCM), both with the required competences, in addition to the pilot. HEMS is a high-competence service and represents the sharp end in the prehospital chain [30]. The content of this paragraph is previously described in [28].

The Norwegian HSOPSC-S and pre-testing of the instrument

The original Norwegian HSOPSC-S consists of six dimensions and 21 items. The response format ranges from 1 (disagree strongly) to 5 (agree strongly) on a Likert scale. There are also seven items relating to the respondents’ work characteristics (work area, geographic location, field of competence, patient contact, work hours, seniority in the prehospital area, seniority in position). A pre-test of the instrument’s items was performed in collaboration with prehospital professionals to ensure correct terminology in the prehospital context. Although minor adjustments had to be implemented, and some items were pointed out as challenging in the prehospital context, no items were left out or conceptually changed before distribution of the survey. The HSOPSC-S items is retrieved from [28, 29], and minor adjustments and challenging items is thoroughly discussed in [28].

Data collection

The target group of this study consists of two main groups of employees: HEMS and ground ambulance personnel. The professional team composition of the ground ambulance is very different from that of HEMS. A HEMS team normally consists of a physician/anesthesiologist, a HCM and a pilot. One base in Norway operates with a nurse aboard, in addition to the aforementioned three-man crew. The three largest professional groups in the ground ambulance are EMTs, paramedics and nurses.

The HEMS samples were retrieved in 2012 and 2016, and the ground ambulance sample was retrieved in 2016 from questionnaires performed in 17 (of 18) health trusts.

The 2012 sample was retrieved from a survey among crew members in the civilian Norwegian HEMS. To maximize the response rate, a commentary on the upcoming study was published in the Norwegian Medical Journal. The survey was distributed via both e-mail, with a link to a web-based questionnaire (Questback), and an identical paper version along with prepaid stamped return envelopes. Data were collected between May and July 2012. All crew members received a follow-up phone call as a reminder and encouragement to answer 2–4 weeks after the survey was commenced.

For the 2016 samples we retrieved e-mail addresses for prehospital personnel in the Norwegian ground ambulance and HEMS from prehospital system leaders. We applied a web-based tool (SurveyXact) to conduct the survey, and an individual link to the questionnaire was distributed by e-mail to all personnel. Data were collected between October and December 2016, and non-responders received up to five reminders before the study was closed.

Statistical analyses

Psychometric properties for the assessment of validation were applied [75, 76], to evaluate the HSOPSC-S for the different samples. Negatively worded items were reversed, and confirmatory factor analysis (CFA) was conducted to analyze the construct validity and to determine the degree of fit between the constructed measurement instrument and the sample. Covariation between underlying dimensions was permitted.

The following indices were applied for global fit assessment: Standardized Root Mean Square Residual (SRMR), Tucker-Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA) and Comparative Fit Index (CFI). Values for TLI and CFI in the 0.90s are generally accepted as guidance values for an acceptable fit, and those above 0.95 reflect a good model fit [77, 78]. It has been suggested using a two-index strategy by reporting SRMR with one of the fit indices (e.g. CFI or RMSEA), with the guidance criteria CFI > 0.95, SRMR < 0.8 and RMSEA < 0.6 [79]. A value below 0.5 is considered a good fit for RMSEA [77]. Guidance values should be adjusted in regard to both complexity of the model and to sample size [76]; see Table 1. Chi Square (Χ 2) is not reported as it has been shown to be problematic in model fit assessments for larger samples [80]. The methodology described in this paragraph is previously described in [28].

Table 1 Guidance values for model fit indices for models consisting of 12–30 items
Full size table

Modification of the instrument to gain model fit for all samples was performed with the help of modification indices and standardized residuals [75]. Values above 3.84 are statistically significant for modification indices. Standardized residual covariances for the different items should be below |4.0| and preferably below |2.5|; i.e., values above |2.5| indicate a concern with an item and values above |4.0| indicate candidates for removal [76]. In addition, items with weak loadings were considered for removal. The reader should be aware that such post hoc modifications to models should normally be done sparingly and founded on theoretical and practical plausibility [81].

To assess configural variance, the GEMS and the two HEMS samples were entered as a grouping variable when CFA was conducted for the total sample of data. Configural invariance is an approach to test measurement invariance which is related to the concept validity of measures. Configural invariance is applied to test the samples together and freely without cross-group constraints [82].

To indicate discriminant validity, MANOVA (multivariate analysis of variance; Wilks’ Lambda) was performed to investigate whether the different work characteristics had an influence on the overall statistical variance of the safety climate dimensions. To demonstrate convergent validity for a latent construct, items with high loadings on a factor should be observed, as they would indicate convergence against a common point. All item loadings for a factor should be at least 0.5 or higher (ideally 0.7 or higher) for standardized estimates, in addition to being statistically significant [76]. On the other hand, several loadings at very high levels are not desirable, as this would indicate a lack of discriminant validity for the factor items. A range of loadings between 0.6 and 0.9 seems reasonable [75]. Inter-correlation between the dimensions was examined by the Spearman-Rho correlation. Cronbach’s alphas were estimated, to determine whether factor scales yielded acceptable internal consistencies with alpha coefficients between 0.70 and 0.90 [83]. The methodology described in this paragraph is previously described in [28].

After obtaining model fit of the measurement model for all subgroup samples, the second step was to perform SEM to test the hypotheses. For this study, we applied a specified criterion that structural relations between two latent factors should be significantly valid (p < .05) in at least two of the three samples. This approach has been adequately demonstrated in another study [84]. Following this criterion, structural relations that did not meet this criterion were removed so that a final robust model could be established. Lastly, the final structural model was compared to a model where all direct effects was estimated. This is conducted to compare the psychometric difference between the hypothetical and alternative model.

Confirmatory factor analysis (maximum likelihood) was estimated using AMOS 25.0. The other statistical analyses were performed using SPSS 25.0.

Ethics approval and consent to participate

For the 2012-data the Regional Committee for Medical and Health Research (REC) South-East Norway (Ref. number 2010/3326) reviewed and approved the study. Participating was voluntary, and consent was given by responding to the questionnaire. For the 2016-data approval was obtained from the Norwegian Social Science Data Services (NSD; project number 45723). All participants received information regarding the purpose of the study; they were assured that the digital questionnaires were to be treated in confidence and that no participants could be identified in the published material. Their written consent to participate in the study was given at the start of the survey.

Results

Response rates

The HEMS 2012 sample consisted of 172 (83% response rate) individuals participating, with 145 (70%) completed questionnaires. The HEMS 2016 sample consisted of 118 (55%) individuals participating, with 109 (51%) completed questionnaires. The Ground 2016 consisted of 1269 individuals participating (26%) with 1045 (21%) completed questionnaires.

A few responses in both HEMS samples reported their work area to be Search and Rescue Services (SAR) or Fixed Wing air ambulances (FW), but these were excluded since their mission profile and crew concepts differ substantially from HEMS.

For the analyses, only returned questionnaires with all items answered were used (listwise deletion). The majority of incomplete questionnaires was discontinued early in the survey, and we evaluated that replacing missing values was not expedient.

Sample characteristics

The sample sizes are considered representative, based on variation in demographic variables, e.g. distribution across professional groups, range in seniority, and geographic location (Table 2).

Table 2 Demographic and professional characteristics of the three samples in the study (2016 samples similar to [28])
Full size table

The collected data clearly reflects the professional groups in the Norwegian prehospital environment, and also the difference in team compositions between ground ambulance and HEMS. Respondents are evenly distributed geographically between the four healthcare regions, with a majority from the south-east region for all samples. Such even distribution is also observed for seniority in the prehospital environment, with a mean of at least ten years. A high number of the respondents worked directly with patients (97%).

Construct validity and post hoc modification of the instrument

CFA was applied to determine model fit. For the original instrument, acceptable model fit was observed for the ground ambulance sample, while both HEMS samples fell below this. Post hoc modification was performed to obtain acceptable model fit for both HEMS samples. One item was initially observed with a relative weak factor loading (< 0.5): item A11 (Ground 16: 0.46, HEMS 16: 0.15). A few items with at least five standardized residual covariances above |2.5| were observed: item A11 (HEMS 16, Ground 16), item C3 (Ground 16), and item C4 (Ground 16). A post hoc modification, removing from the instrument items A11 (when one area in this unit gets really busy, others help out), C3 (whenever pressure builds up, my manager wants us to work faster, even if it means taking shortcuts) and C4 (my local manager overlooks patient safety problems that happen over and over), resulted in improved model fit for all three samples. Based on modification indices, items D2 (staff will freely speak up if they see something that may negatively affect patient care) and D5 (in this unit, we discuss ways to prevent errors from happening again) were also removed, with the result that acceptable model fit was obtained for all three samples (Table 3).

Table 3 Model fit measurement model
Full size table

In addition, based on the two HEMS samples and the ground sample, a configural invariance test was conducted to reveal whether CFA achieved adequate fit among these samples when they were tested together and freely without cross-group constraints. The test indicated that the resultant model achieved good fit (SRMR: 0.048, TLI: 0.98, RMSEA: 0.020 and CFI: 0.98), and conclusively, that configural invariance was adequate among the samples.

The factor loading values for the items in the adjusted instrument were in the range 0.56 to 0.92 (Table 4). None of the dimensions had more than one value below 0.6 or above 0.9 for any of the samples. Following the reasoning that the values should preferably be between these two values, this indicated an overall acceptable convergent validity.

Table 4 HSOPSC-S dimensions and items (the selected items have same wording as in the full PreHSOPSC [28])
Full size table

The intercorrelations between the safety dimensions ranged from little/fair (0.24) to moderate (0.59) degree of relationship for the different samples (Tables 5 and 6); hence, no strong correlation (> 0.75) was observed between dimensions. One correlation between “management support for patient safety” and “stop working in dangerous situations” was observed as not significant in the HEMS 16 sample.

Table 5 Intercorrelation (Spearman’s Rho) of dimensions for the HEMS samples
Full size table
Table 6 Intercorrelation (Spearman’s Rho) of dimensions for the ground ambulance and the total 2016 sample
Full size table

MANOVA was conducted based on satisfactory data quality meeting different assumptions (normality, equality of variance, univariate outliers, and equality of covariance matrices). By utilizing MANOVA, Wilk’s Lambda was measured for all different employee characteristics (Table 7). Although not significant for all characteristics, overall, along with intercorrelations, acceptable discriminant validity is observed.

Table 7 Wilk’s Lambda
Full size table

Descriptive statistics and internal consistency

The mean statistics and standard deviation (SD) are presented in Table 8 for each of the measurement concepts of the modified instrument, for each of the sample groups. The patient safety climate dimensions with highest scores are “teamwork within units” and “manager expectations & actions promoting patient safety”. Overall, variance of items was considered adequate. Cronbach’s alpha coefficients varied from 0.68 (stop working in dangerous situations) to 0.87 (manager expectations & actions promoting patient safety).

Table 8 Means, standard deviations (SD) and Cronbach’s alpha coefficients
Full size table

Testing of structural model

SEM indicated satisfactory model fit for all samples, although slightly below the recommended values for the HEMS 2012 sample (Table 9). Following the criterion that paths should have at least two significant relationships, several non-significant paths were removed sequentially, starting with the highest probability (p). The final model fit remained relatively equal to the model fit of the original structure (Table 9).

Table 9 Model fit structure model
Full size table

The final structure model with its standardized path coefficients is presented in Fig. 3.

Fig. 3
figure 3

Standardized path coefficient. Note: HEMS 12 = HEMS sample 2012, HEMS 16 = HEMS sample 2016, Ground 16 = ground ambulance sample 2016, Total 16 = combined HEMS and ground ambulance 2016 samples. Management support = “Management support for patient safety”, Management exp. & supp = “Manager expectations & actions promoting patient safety”, Learning, feedb & impr = “Learning, feedback & improvement within units”, Stop working in dangerous sit = “Stop working in dangerous situations”. n.s. = not significant, **p < .01, ***p < .001

Full size image

After the non-significant path removal process, only one relationship remained non-significant. Surprisingly, despite our hypotheses on the safety climate in the prehospital environment, none of the dimensions “manager expectations & actions promoting patient safety”, “teamwork within units” and “learning, feedback and improvement within units” had two or more significant relationships on the outcome dimension “transitions and handoffs”. The path between “learning, feedback and improvement within units” and “teamwork within units” was also removed.

The path from the dimension “management support for patient safety” to “manager expectations & actions promoting patient safety”, and the paths from “manager expectations & actions promoting patient safety” to both of the unit dimensions “teamwork within units” and “learning, feedback and improvement within units” are all observed to have relatively high coefficients for all samples. This supports the importance of management levels influencing the other safety climate concepts at both team and individual levels.

A post hoc test was conducted to compare the final structural model with a model where all latent factor were specified to have direct effects on the outcome dimension “transitions and handoffs”. This alternative model indicated a good model fit (e.g. for the total 2016 sample; SRMR: 0.026, TLI: 0.98, RMSEA: 0.032 and CFI: 0.98). However, two of the direct paths on the outcome dimension proved non-significant; “manager expectations & actions promoting patient safety” and “learning, feedback and improvement within units”. Due to non-significant paths, and as the hypothesized model is built on a theoretical framework, the final structural model is preferred over the alternative model.

Discussion

This study produced three major findings. Firstly, the original HSOPSC-S measurement model provided overall acceptable model fit for the ground ambulance sample, implying that the original HSOPSC-S instrument is adequate for distribution among the ground ambulance. As it does not provide model fit for any of the HEMS samples, no further psychometric properties were tested on the original HSOPSC-S instrument in this article. Secondly, a trimmed version of HSOPSC-S was observed with overall acceptable psychometric properties for the measurement model for both the ground ambulance sample and the two HEMS samples, i.e. acceptable internal consistencies and construct validity. Thirdly, by testing the theoretical, structured framework, we could observe the dynamic relation between the management and unit levels, which culminates in a significant influence on the outcome dimension “transitions and handoffs”. This is beneficial for an improved understanding of the prehospital patient safety climate. The second and third aforementioned findings indicate that the final model, the PreHSOPSC-S, is appropriate for generalized application in the prehospital environment.

Modified HSOPSC-S

By performing a post hoc modification of HSOPSC-S, the results demonstrated a clearly better model fit for all samples in the measurement model. In addition, after removal of weak items, the results also indicated acceptable convergent and discriminant validity. It has been advocated a conservative approach when performing modifications, i.e. few modifications and clear interpretability [81]. The removal of five items may be argued to be a moderate modification relative to the original 22 items.

A recent article demonstrates that items A11 (when one area in this unit gets really busy, others help out) and C3 (whenever pressure builds up, my manager wants us to work faster, even if it means taking shortcuts) is challenging due to respectively not reflecting work processes and due to use of idioms, and should thus be considered candidates for removal [28]. For items C4 (my local manager overlooks patient safety problems that happen over and over), D2 (staff will freely speak up if they see something that may negatively affect patient care) and D5 (in this unit, we discuss ways to prevent errors from happening again), it may be argued that these items possibly reflect variance in local practices which are not sufficiently coherent within the prehospital environment. Such variance may stem from the lack of common explicit knowledge (procedures, routines etc.) or tacit knowledge (norms, common understanding etc.). In a hospital, it is reasonable that local practices (unit level) is more aligned with regional (hospital level). However, due to the scattered nature of the prehospital domain this is not granted as such. As the argument has not been sufficiently demonstrated, it may constitute a hypothesis for future research.

Normally, modification may benefit on random variation in an obtained sample and should be viewed as tentative until cross-validated with an independent sample [81]. In this study, the modification was performed with its basis in three independent samples, either within the same work area with a split of four years (the two HEMS samples) or in different work areas (the ground ambulance and HEMS samples). After modification of the original measurement model and the structural framework model, the CFA demonstrated acceptable model fit for all samples against the recommended values. The result of the post hoc modification is the PreHSOPSC-S, which is a shorter – but stronger – instrument suitable for application in a broader prehospital environment.

As a shorter instrument, the PreHSOPSC-S may be a significant contribution to performing safety climate assessments in the prehospital environment. The prehospital environment is, relative to hospitals, geographically scattered, and digital surveys are easier to distribute than paper surveys. An issue with digital surveys is that they may generate a relatively lower response rate [86], and a shorter version may increase the response rate and representativeness. In addition, the practical administering and analysis of a shorter survey makes it easier to perform more frequent measuring. It has been demonstrated that received feedback improves safety climate [87], which implies that providing feedback based on the measurement results to the EMS providers may also result in improved safety climate. An effective approach for more frequent measuring is to attach the survey to existing frequently performed measuring tools such as employee surveys or re-certification tests.

The original full version of the Norwegian HSOPSC comprises 13 safety climate dimensions, consisting of 46 items, in addition to two single-item outcome items [29, 35], and a relevant question is whether the PreHSOPSC-S is to be preferred over the original full HSOPSC. In study design, it has become more important to consider specific measurement instruments, level of analysis and the selection of outcome measures [88]. Whether the original or the short version should be applied for measurements depends on the objectives. If efficiency, speed and frequent measurement constitute the goal, a shorter version may be preferred. On the other hand, if an organization is conducting comprehensive improvement programs, and there is a desire to measure changes in the patient safety climate, a full version covering all dimensions may be preferred. In essence, different objectives require different measuring instruments, implying that several validated instruments with different perspectives and outcomes are beneficial for both assessments of and research on patient safety climate in the prehospital domain.

Theoretical and practical implications

A structural model is valuable for understanding the multilevel relationships in the prehospital domain. The structured framework for the prehospital environment based on our results is illustrated in Fig. 4.

Fig. 4
figure 4

Structured framework for the prehospital environment. Note: Management support = “Management support for patient safety”, Management exp. & supp = “Manager expectations & actions promoting patient safety”, Learning, feedb & impr = “Learning, feedback & improvement within units”, Stop working in dangerous sit = “Stop working in dangerous situations”

Full size image

The final model was observed with relational paths from top management via unit management and unit level to the individual level, supporting the basic principle of multilevel safety climate [38]. This observed hierarchical relationship between levels implies that the change of patient safety climate on one level would influence subordinate levels, ultimately affecting safety outcomes. All samples supported a strong positive influence from top management on unit management, and from unit management on the unit level. Based on the theoretical framework on which HSOPSC-S is based, several strategies may be chosen for how to use the structural components in improving safety in organizational settings [29]. An example of such a strategy is to define and implement a multilevel safety program, whose goal is to improve safety culture and safety behavior [89]. Top management may put patient safety on the agenda, integrating it into policies and agendas, ensuring that unit management adopts these into practice. Another strategy is leadership development initiatives to improve the safety climate [90], involving leaders on all levels.

Top management level influence on transitions & handoffs

The direct path from the top management level dimension “management support to patient safety” on the outcome dimension “transitions and handoffs” may be related to the EMS providers’ perception of the overall healthcare system. This implies that top management has an important direct role in patient safety in transitions and handoffs, through the quality of internal work processes, procedures and routines for patient transitions and handoffs in the prehospital domain. Patient transitions and handoffs are also organizational interfaces [91]. When commencing transitions and handoffs between the prehospital and hospital domains, EMS workers may perceive this as an interface, where both these cultural domains influence the outcome of patient transitions and handoffs. This may be seen in light of patient transitions and handoffs being considered an important area of contact between employees and the hospital [23]. An implication of this is that the better the EMS providers perceive the management support to patient safety, the better they perceive the hospital management effort to provide and maintain effective and safe systems for transitions and handoffs. In essence, top management’s expressed values and priorities are important for obtaining improved outcomes.

Unit level influence on transitions & handoffs

The lack of a direct path from the outcome dimension “transitions and handoffs” to both of the unit levels “teamwork within units” and “learning, feedback and improvement within units”, demonstrates a full mediation by individual safety behavior. An implication of this finding is that workers’ perception of the safety climate within the unit differs from their perception of the safety climate in transition and handoff situations. This observation was surprising, as safe transitions and handoffs have been linked to interdisciplinary team effort and team communication skills [69]. The explanation may be that transitions and handoffs often involve collaboration across units in the prehospital chain, which implies that the safety climate in this broader domain is different from the safety climate related to teamwork within the unit.

Another explanation may be related to the fact that two of the three samples applied in this study are retrieved from HEMS workers. In the ground ambulance, tasks related to patient diagnostics, treatment, transitions and handoffs are often shared by the team. The composition of a HEMS team has a higher degree of individual professionalism than that of other prehospital team compositions, with each professional having their defined set of tasks and areas for which they are responsible [30]. The physician in HEMS is mainly responsible for patient diagnostics and treatment, in addition to transitions and handoffs, i.e. transfer of patient information, responsibility and accountability. However, patient safety processes and climate within the unit influence the individual behavior, hence having an indirect influence on the outcome of transitions and handoffs. This implies the importance of the continued focus on teamwork and the focus on learning, feedback and improvement within the unit.

Individual level influence on transitions & handoffs

The dimension “stop working in dangerous situations” is a particular type of safety behavior [29], and in practice such behavior is likely to be primarily related to high-risk tasks in the prehospital domain such as driving [92] or clinical judgement and decision-making [2, 93]. An example of “stop working” behavior is cross-monitoring, i.e. the possibility to ask critical questions or voice concerns if one believes that an action may harm the patient [56, 63]. The direct path from the safety behavior dimension “stop working in dangerous situations” to the outcome dimension “transitions and handoffs” implies that, if individuals perceive that they can “stop working” when performing safety-related tasks, it also increases quality and safety in the individual tasks related to patient transitions and handoffs. Implementing different “stop working” techniques in training programs, simultaneously with promoting and cultivating such behavior, is likely to improve both the outcome of transitions and handoffs, as well as overall patient safety.

Limitations and future research

The response rate for the ground ambulance sample was low relative to other similar studies [32,33,34, 36, 94]. This may induce non-response bias, meaning that the group of non-responders differs from the group of responders. See [28] for an extended discussion on possible reasons for the low response for the sample retrieved in 2016. The two HEMS samples had higher response rates, but sample sizes are relatively small, thus increasing the likelihood for Type II errors. However, being able to test on all these three different samples outweighs the overall likelihood for Type II errors.

A disadvantage of a modified instrument is the lack of opportunity to compare with other studies undertaken. Although the PreHSOPSC-S is tested for a major part of the prehospital environment, further research should test and validate the instruments (both original and modified) for other parts, to obtain a more generalized instrument for measuring safety climate. Use of the instrument in a new context is a limitation in itself, and development of a new instrument targeted on an prehospital context may be a better solution. However, we have demonstrated both a multilevel structural relationship and a relational effect from safety climate dimensions on patient transitions and handoffs, which may be included when designing new instruments.

PreHSOPSC-S is based on self-reported data and has not been tested for predictive validity, i.e. not compared with an external criterion [75], such as degree of error reporting, degree of patient compensation due to harm, or effect of intervention. Before such testing against external criteria is undertaken, the impact on the prehospital safety climate is not fully known.

Conclusion

Researching and measuring safety climate may provide a valuable input in responding to potential patient safety issues in the prehospital domain. Normally, relative few dimensions are addressed at a time when performing actions to improve the safety climate [15]. Through use of a multilevel approach, safety programs may be designed to include several levels and safety climate dimensions, simultaneously. The literature suggests to start with surveys when initiating the process of measuring the current safety climate status [95, 96]. Hence, it is satisfying to provide a validated instrument for measuring the patient safety climate with a multilevel perspective in a broader prehospital context: the PreHSOPSC-S. The validity for use throughout the prehospital domain is utterly demonstrated by the uniqueness of the samples, as the HEMS differs substantially from the ground ambulance, and the two HEMS samples were retrieved at two points in time. This study also demonstrated that the multilevel patient safety climate is positively related to the outcome dimension related to patient transitions and handoffs. To our knowledge, this is the first prehospital patient safety climate study with a multilevel perspective and with the use of transitions and handoffs as an outcome. In the prehospital domain, safe patient transitions and handoffs are considered important, and a structured framework will constitute an important contribution to prehospital safety climate assessment and research.

Availability of data and materials

The data sets generated and analyzed during the current study are not publicly available, as further papers will be written based on the data sets; however, they are available from the corresponding author on reasonable request.

Abbreviations

AHRQ:

Agency for Healthcare Research and Quality

CFA:

Confirmatory factor analysis

CFI:

Comparative fit index

EFA:

Exploratory factor analysis

EMS:

Emergency medical services

EMT:

Emergency medical technician

FW:

Fixed wing

HCM:

HEMS crew member

HEMS:

Helicopter emergency medical services

HSE:

Health, safety and environment

HSOPSC:

Hospital Survey on Patient Safety Culture

MANOVA:

Multivariate analysis of variance

PreHSOPSC:

Prehospital Survey on Patient Safety Culture

RMSEA:

Root mean square error of approximation

SAR:

Search and rescue services

SD:

Standard deviation

SEM:

Structured equational modelling

SRMR:

Standardized root mean square residual

TLI:

Tucker-Lewis index

References

  1. St Pierre M, Hofinger G, Buerschaper C, Simon R. Crisis management in acute care settings: human factors, team psychology, and patient safety in a high stakes environment. Berlin: Springer; 2011.

    Book  Google Scholar 

  2. Atack L, Maher J. Emergency medical and health providers' perceptions of key issues in prehospital patient safety. Prehosp Emerg Care. 2010;14(1):95–102.

    Article  PubMed  Google Scholar 

  3. Patterson PD, Huang DT, Fairbanks RJ, Wang HE. The emergency medical services safety attitudes questionnaire. Am J Med Qual. 2010;25(2):109–15.

    Article  PubMed  Google Scholar 

  4. Bigham BL, Buick JE, Brooks SC, Morrison M, Shojania KG, Morrison LJ. Patient safety in emergency medical services: a systematic review of the literature. Prehosp Emerg Care. 2012;16(1):20–35.

    Article  PubMed  Google Scholar 

  5. Wood K, Crouch R, Rowland E, Pope C. Clinical handovers between prehospital and hospital staff: literature review. Emerg Med J. 2015;32:577–81.

    Article  PubMed  Google Scholar 

  6. Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care. 2003;12(suppl 2):ii17–23.

    PubMed  PubMed Central  Google Scholar 

  7. Kaufman G, McCaughan D. The effect of organisational culture on patient safety. Nurs Stand. 2013;27(43):50–6.

    Article  PubMed  Google Scholar 

  8. Hofmann DA, Mark B. An investigation of the relationship between safety climate and medication errors as well as other nurse and patient outcomes. Pers Psychol. 2006;59(4):847–69.

    Article  Google Scholar 

  9. Naveh E, Katz-Navon T, Stern Z. Treatment errors in healthcare: a safety climate approach. Manag Sci. 2005;51(6):948–60.

    Article  Google Scholar 

  10. Neal A, Griffin MA. A study of the lagged relationships among safety climate, safety motivation, safety behavior, and accidents at the individual and group levels. J Appl Psychol. 2006;91(4):946.

    Article  PubMed  Google Scholar 

  11. Singer SJ, Gaba DM, Falwell A, Lin S, Hayes J, Baker L. Patient safety climate in 92 US hospitals: differences by work area and discipline. Med Care. 2009;47(1):23–31.

    Article  PubMed  Google Scholar 

  12. Vogus TJ, Sutcliffe KM. The safety organizing scale: development and validation of a behavioral measure of safety culture in hospital nursing units. Med Care. 2007;45(1):46–54.

    Article  PubMed  Google Scholar 

  13. Singer SJ, Lin S, Falwell A, Gaba D, Baker L. Relationship of safety climate and safety performance in hospitals. Health Serv Res. 2009;44(2p1):399–421.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf. 2010;6(4):226–32.

    Article  PubMed  Google Scholar 

  15. Halligan M, Zecevic A. Safety culture in healthcare: a review of concepts, dimensions, measures and progress. BMJ Qual Saf. 2011;20(4):338–43.

    Article  PubMed  Google Scholar 

  16. Guldenmund FW. The use of questionnaires in safety culture research–an evaluation. Saf Sci. 2007;45(6):723–43.

    Article  Google Scholar 

  17. Hellings J, Schrooten W, Klazinga N, Vleugels A. Challenging patient safety culture: survey results. Int J Health Care Qual Assur. 2007;20(7):620–32.

    Article  PubMed  Google Scholar 

  18. Flin R. Measuring safety culture in healthcare: a case for accurate diagnosis. Saf Sci. 2007;45(6):653–67.

    Article  Google Scholar 

  19. Christian MS, Bradley JC, Wallace JC, Burke MJ. Workplace safety: a meta-analysis of the roles of person and situation factors. J Appl Psychol. 2009;94(5):1103.

    Article  PubMed  Google Scholar 

  20. Clarke S. An integrative model of safety climate: linking psychological climate and work attitudes to individual safety outcomes using meta-analysis. J Occup Organ Psychol. 2010;83(3):553–78.

    Article  Google Scholar 

  21. Huang YH, Chen JC, DeArmond S, Cigularov K, Chen PY. Roles of safety climate and shift work on perceived injury risk: a multi-level analysis. Accid Anal Prev. 2007;39(6):1088–96.

    Article  PubMed  Google Scholar 

  22. Mearns K, Whitaker SM, Flin R. Benchmarking safety climate in hazardous environments: a longitudinal, interorganizational approach. Risk Anal. 2001;21(4):771–86.

    Article  CAS  PubMed  Google Scholar 

  23. Meisel ZF, Shea JA, Peacock NJ, Dickinson ET, Paciotti B, Bhatia R, Buharin E, Cannuscio CC. Optimizing the patient handoff between emergency medical services and the emergency department. Ann Emerg Med. 2015;65(3):310–7.

    Article  PubMed  Google Scholar 

  24. Lee SH, Phan PH, Dorman T, Weaver SJ, Pronovost PJ. Handoffs, safety culture, and practices: evidence from the hospital survey on patient safety culture. BMC Health Serv Res. 2016;16(1):254.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Hopkins A. Studying organisational cultures and their effects on safety. Saf Sci. 2006;44(10):875–89.

    Article  Google Scholar 

  26. Alsalem G, Bowie P, Morrison J. Assessing safety climate in acute hospital settings: a systematic review of the adequacy of the psychometric properties of survey measurement tools. BMC Health Serv Res. 2018;18(1):353.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Sorra JS, Nieva VF. Hospital survey on patient safety culture. AHRQ publication no. 04–0041. Rockville: Agency for Healthcare Research and Quality; 2004.

    Google Scholar 

  28. Sørskår LIK, Abrahamsen EB, Olsen E, Sollid SJ, Abrahamsen HB. Psychometric properties of the Norwegian version of the hospital survey on patient safety culture in a prehospital environment. BMC Health Serv Res. 2018;18(1):784.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Olsen E. Exploring the possibility of a common structural model measuring associations between safety climate factors and safety behaviour in health care and the petroleum sectors. Accid Anal Prev. 2010;42(5):1507–16.

    Article  PubMed  Google Scholar 

  30. Official Norwegian Reports 2015:17 «Først og fremst - Et helhetlig system for håndtering av akutte sykdommer og skader utenfor sykehus». https://www.regjeringen.no/contentassets/477c27aa89d645e09ece350eaf93fedf/no/pdfs/nou201520150017000dddpdfs.pdf. Norwegian only. Accessed 18 Dec 2017.

  31. Pasientsikkerhetsprogrammet 2016. http://www.pasientsikkerhetsprogrammet.no/aktuelt/nyheter/tester-ut-samordnet-kultur-hms-unders%C3%B8kelse. Norwegian only. Accessed 18 Dec 2017.

  32. Olsen E. Reliability and validity of the Hospital Survey on Patient Safety Culture at a Norwegian hospital. Quality and Safety Improvement Research: Methods and Research Practice from the International Quality Improvement Research Network (QIRN). 2008;173:186.

    Google Scholar 

  33. Waterson P, Griffiths P, Stride C, Murphy J, Hignett S. Psychometric properties of the hospital survey on patient safety culture: findings from the UK. Qual Saf Health Care. 2010;19(5):e2.

    CAS  PubMed  Google Scholar 

  34. Pfeiffer Y, Manser T. Development of the German version of the hospital survey on patient safety culture: dimensionality and psychometric properties. Saf Sci. 2010;48(10):1452–62.

    Article  Google Scholar 

  35. Haugen AS, Søfteland E, Eide GE, Nortvedt MW, Aase K, Harthug S. Patient safety in surgical environments: cross-countries comparison of psychometric properties and results of the Norwegian version of the hospital survey on patient safety. BMC Health Serv Res. 2010;10(1):279.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Hedsköld M, Pukk-Härenstam K, Berg E, Lindh M, Soop M, Øvretveit J, Sachs MA. Psychometric properties of the hospital survey on patient safety culture, HSOPSC, applied on a large Swedish health care sample. BMC Health Serv Res. 2013;13(1):332.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Vincent C. Patient safety. 2nd ed. Oxford: Wiley-Blackwell; 2010.

    Book  Google Scholar 

  38. Zohar D. Safety climate: Conceptualization, measurement, and improvement: The Oxford handbook of organizational climate and culture: New York: Oxford University Press; 2014:317–334.

  39. Schein EH. Organizational culture and leadership. San Francisco: John Wiley & Sons; 2010.

    Google Scholar 

  40. Zohar D, Luria G. Group leaders as gatekeepers: testing safety climate variations across levels of analysis. Appl Psychol. 2010;59(4):647–73.

    Article  Google Scholar 

  41. Zohar D. Modifying supervisory practices to improve subunit safety: a leadership-based intervention model. J Appl Psychol. 2002;87(1):156.

    Article  PubMed  Google Scholar 

  42. Zohar D, Luria G. A multilevel model of safety climate: cross-level relationships between organization and group-level climates. J Appl Psychol. 2005;90(4):616.

    Article  PubMed  Google Scholar 

  43. Kapp EA. The influence of supervisor leadership practices and perceived group safety climate on employee safety performance. Saf Sci. 2012;50(4):1119–24.

    Article  Google Scholar 

  44. Judge TA, Piccolo RF. Transformational and transactional leadership: a meta-analytic test of their relative validity. J Appl Psychol. 2004;89:755.

    Article  PubMed  Google Scholar 

  45. Michaelis B, Stegmaier R, Sonntag K. Affective commitment to change and innovation implementation behavior: the role of charismatic leadership and employees’ trust in top management. J Chang Manag. 2009;9(4):399–417.

    Article  Google Scholar 

  46. Zohar D, Tenne-Gazit O. Transformational leadership and group interaction as climate antecedents: a social network analysis. J Appl Psychol. 2008;93(4):744.

    Article  PubMed  Google Scholar 

  47. Yukl GA. Leadership in organizations. Pearson: Upper Saddle River; 2010.

    Google Scholar 

  48. Reader TW, Flin R, Mearns K, Cuthbertson BH. Interdisciplinary communication in the intensive care unit. Br J Anaesth. 2007;98(3):347–52.

    Article  CAS  PubMed  Google Scholar 

  49. Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care. 2004;13(suppl 1):i85–90.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Barrett J, Gifford C, Morey J, Risser D, Salisbury M. Enhancing patient safety through teamwork training. J Healthcare Risk Manage. 2001;21(4):61–9.

    Article  Google Scholar 

  51. Morey JC, Simon R, Jay GD, Wears RL, Salisbury M, Dukes KA, Berns SD. Error reduction and performance improvement in the emergency department through formal teamwork training: evaluation results of the MedTeams project. Health Serv Res. 2002;37(6):1553–81.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Weaver SJ, Rosen MA, DiazGranados D, Lazzara EH, Lyons R, Salas E, Knych SA, McKeever M, Adler L, Barker M, King HB. Does teamwork improve performance in the operating room? A multilevel evaluation. Jt Comm J Qual Patient Saf. 2010;36(3):133–42.

    Article  PubMed  Google Scholar 

  53. Reader TW, Flin R, Mearns K, Cuthbertson BH. Developing a team performance framework for the intensive care unit. Crit Care Med. 2009;37(5):1787–93.

    Article  PubMed  Google Scholar 

  54. Risser DT, Rice MM, Salisbury ML, Simon R, Jay GD, Berns SD. MedTeams research consortium. The potential for improved teamwork to reduce medical errors in the emergency department. Ann Emerg Med. 1999;34(3):373–83.

    Article  CAS  PubMed  Google Scholar 

  55. Wheelan SA, Burchill CN, Tilin F. The link between teamwork and patients’ outcomes in intensive care units. Am J Crit Care. 2003;12(6):527–34.

    PubMed  Google Scholar 

  56. Reid J, Bromiley M. Clinical human factors: the need to speak up to improve patient safety. Nurs Stand. 2012;26(35):35–40.

    Article  PubMed  Google Scholar 

  57. Dietz AS, Pronovost PJ, Mendez-Tellez PA, Wyskiel R, Marsteller JA, Thompson DA, Rosen MA. A systematic review of teamwork in the intensive care unit: what do we know about teamwork, team tasks, and improvement strategies? J Crit Care. 2014;29(6):908–14.

    Article  PubMed  Google Scholar 

  58. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 2009;53(2):143–51.

    Article  CAS  PubMed  Google Scholar 

  59. Hesselink G, Berben S, Beune T, Schoonhoven L. Improving the governance of patient safety in emergency care: a systematic review of interventions. BMJ Open. 2016;6(1):e009837.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Argote L. Organizational learning research: past, present and future. Manag Learn. 2011;42(4):439–46.

    Article  Google Scholar 

  61. Crossan MM, Lane HW, White RE. An organizational learning framework: from intuition to institution. Acad Manag Rev. 1999;24(3):522–37.

    Article  Google Scholar 

  62. Milligan F, Dennis S. Building a safety culture. Nurs Stand. 2005;20(11):48–52.

    Article  CAS  PubMed  Google Scholar 

  63. Behara R, Wears RL, Perry SJ, Eisenberg E, Murphy L, Vanderhoef M, Shapiro M, Beach C, Croskerry P, Cosby K. A conceptual framework for studying the safety of transitions in emergency care. In: Henriksen K, Battles JB, Marks ES, et al., editors. Advances in patient safety: From research to implementation (Volume 2: Concepts and Methodology). Rockville: Agency for Healthcare Research and Quality (US); 2005.

    Google Scholar 

  64. Owen C, Hemmings L, Brown T. Lost in translation: maximizing handover effectiveness between paramedics and receiving staff in the emergency department. Emerg Med Australasia. 2009;21(2):102–7.

    Google Scholar 

  65. Jensen SM, Lippert A, Østergaard D. Handover of patients: a topical review of ambulance crew to emergency department handover. Acta Anaesthesiol Scand. 2013;57(8):964–70.

    Article  CAS  PubMed  Google Scholar 

  66. Shelton D, Sinclair P. Availability of ambulance patient care reports in the emergency department. BMJ Qual Improv Rep. 2016;5(1):u209478–w3889.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Cuk S, Wimmer H, Powell LM. Problems associated with patient care reports and transferring data between ambulance and hospitals from the perspective of emergency medical technicians. Issues Inform Syst. 2017;18(4):16–26.

  68. Bost N, Crilly J, Wallis M, Patterson E, Chaboyer W. Clinical handover of patients arriving by ambulance to the emergency department–a literature review. Int Emerg Nurs. 2010;18(4):210–20.

    Article  PubMed  Google Scholar 

  69. Bost N, Crilly J, Patterson E, Chaboyer W. Clinical handover of patients arriving by ambulance to a hospital emergency department: a qualitative study. Int Emerg Nurs. 2012;20(3):133–41.

    Article  PubMed  Google Scholar 

  70. Jenkin A, Abelson-Mitchell N, Cooper S. Patient handover: time for a change? Accid Emerg Nurs. 2007;15(3):141–7.

    Article  PubMed  Google Scholar 

  71. Thakore S, Morrison W. A survey of the perceived quality of patient handover by ambulance staff in the resuscitation room. Emerg Med J. 2001;18(4):293–6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Smith AF, Pope C, Goodwin D, Mort M. Interprofessional handover and patient safety in anaesthesia: observational study of handovers in the recovery room. Br J Anaesth. 2008;101(3):332–7.

    Article  CAS  PubMed  Google Scholar 

  73. Berkenstadt H, Haviv Y, Tuval A, Shemesh Y, Megrill A, Perry A, Rubin O, Ziv A. Improving handoff communications in critical care: utilizing simulation-based training toward process improvement in managing patient risk. CHEST Journal. 2008;134(1):158–62.

    Google Scholar 

  74. Bruce K, Suserud BO. Ambulance nursing: part three. Emergency Nurse. 2003;11(2):16–21.

    Article  PubMed  Google Scholar 

  75. Netemeyer RG, Bearden WO, Sharma S. Scaling procedures: issues and applications: Sage Publications: Thousand Oaks, California; 2003.

  76. Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 7th ed. Pearson: Essex; 2014.

    Google Scholar 

  77. Schumacker RE, Lomax RG. A beginner's guide to structural equation modeling: Mahwah, New Jersey: Lawrence Erlbaum Associates; 2004.

  78. Lei PW, Wu Q. Introduction to structural equation modeling: issues and practical considerations. Educ Meas Issues Pract. 2007;26(3):33–43.

    Article  Google Scholar 

  79. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model Multidiscip J. 1999;6(1):1–55.

    Article  Google Scholar 

  80. Hooper D, Coughlan J, Mullen M. Structural equation modelling: guidelines for determining model fit. Electronic J Bus Res Methods. 2008;6(1):53–60.

  81. MacCallum RC, Roznowski M, Necowitz LB. Model modifications in covariance structure analysis: the problem of capitalization on chance. Psychol Bull. 1992;111(3):490–504.

    Article  CAS  PubMed  Google Scholar 

  82. Milfont TL, Fischer R. Testing measurement invariance across groups: applications in cross-cultural research. Int J Psychol Res. 2010;3(1):111–30.

    Article  Google Scholar 

  83. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011;2:53.

    Article  PubMed  PubMed Central  Google Scholar 

  84. Olsen E. Influence from organisational factors on patient safety and safety behaviour among nurses and hospital staff. Int J Organ Anal. 2017. https://doi.org/10.1108/ijoa-05-2017-1170.

    Article  Google Scholar 

  85. Bodur S, Filiz E. A survey on patient safety culture in primary healthcare services in Turkey. Int J Qual Health Care. 2009;21(5):348–55.

    Article  PubMed  Google Scholar 

  86. Sorra J, Gray L, Streagle S, Famolaro T, Yount N, Behm J. AHRQ Hospital survey on patient safety culture: User’s guide. Rockville, MD: Agency for Healthcare Research and Quality; 2016. https://www.ahrq.gov/sites/default/files/wysiwyg/professionals/quality-patient-safety/patientsafetyculture/hospital/userguide/hospcult.pdf. Accessed 03 Jan 2018

    Google Scholar 

  87. Sexton JB, Adair KC, Leonard MW, Frankel TC, Proulx J, Watson SR, Magnus B, Bogan B, Jamal M, Schwendimann R, Frankel AS. Providing feedback following leadership WalkRounds is associated with better patient safety culture, higher employee engagement and lower burnout. BMJ Qual Saf. 2017;0:1. https://doi.org/10.1136/bmjqs-2016-006399.

    Article  CAS  Google Scholar 

  88. DiCuccio MH. The relationship between patient safety culture and patient outcomes: a systematic review. J Patient Saf. 2015;11(3):135–42.

    Article  PubMed  Google Scholar 

  89. Olsen E, Bjerkan AM, Nævestad TO. Modelling the effects of a large-scale safety culture programme: a combined qualitative and quantitative approach. J Risk Res. 2009;12(3–4):389–409.

    Article  Google Scholar 

  90. Zohar D, Livne Y, Tenne-Gazit O, Admi H, Donchin Y. Healthcare climate: a framework for measuring and improving patient safety. Crit Care Med. 2007;35(5):1312–7.

    Article  PubMed  Google Scholar 

  91. Foster S, Manser T. The effects of patient handoff characteristics on subsequent care: a systematic review and areas for future research. Acad Med. 2012;87(8):1105–24.

    Article  PubMed  Google Scholar 

  92. Brice JH, Studnek JR, Bigham BL, Martin-Gill C, Custalow CB, Hawkins E, Morrison LJ. EMS provider and patient safety during response and transport: proceedings of an ambulance safety conference. Prehosp Emerg Care. 2012;16(1):3–19.

    Article  PubMed  Google Scholar 

  93. Price R, Bendall JC, Patterson JA, Middleton PM. What causes adverse events in prehospital care? A human-factors approach. Emerg Med J. 2013;30:583–8.

    Article  PubMed  Google Scholar 

  94. Perneger TV, Staines A, Kundig F. Internal consistency, factor structure and construct validity of the French version of the hospital survey on patient safety culture. BMJ Qual Saf. 2014;23:389–97.

    Article  PubMed  Google Scholar 

  95. Huang DT, Clermont G, Sexton JB, Karlo CA, Miller RG, Weissfeld LA, et al. Perceptions of safety culture vary across the intensive care units of a single institution. Crit Care Med. 2007;35(1):165–76.

    Article  PubMed  Google Scholar 

  96. Pronovost P, Weast B, Rosenstein B, Sexton JB, Holzmueller CG, Paine L, et al. Implementing and validating a comprehensive unit-based safety program. J Patient Saf. 2005;1(1):33–40.

    Article  Google Scholar 

Download references

Acknowledgements

We thank the healthcare providers and local leaders in EMS and HEMS for participating. The authors are grateful to Stephen JM Sollid, to Kristen Rasmussen and to Lennart S Öhlund for their contribution to the data collection. We are also grateful to Knud Knudsen for his evaluation of the statistics.

Funding

LIKS received funding from the Norwegian Ministry of Education and Research. The funders had no part in the design of the study, collection, analysis, interpretation of data and in writing the manuscript.

Author information

Authors and Affiliations

  1. Institute for Safety, Economics and Planning, University of Stavanger, Kjølv Egelands hus, Kristine Bonnevies vei 22, 4021, Stavanger, Norway

    Leif Inge K. Sørskår, Eirik B. Abrahamsen & Håkon B. Abrahamsen

  2. Department of Innovation, Management & Marketing, UiS Business School, University of Stavanger, Elise Ottesen-Jensens hus, Kjell Arholms gate 37, 4021, Stavanger, Norway

    Espen Olsen

  3. Department of Global Public Health and Primary Care, University of Bergen, Kalfarveien 31, 5018, Bergen, Norway

    Gunnar Tschudi Bondevik

  4. National Centre for Emergency Primary Health Care, NORCE Norwegian Research Centre, Kalfarveien 31, 5018, Bergen, Norway

    Gunnar Tschudi Bondevik

  5. Department of Anesthesiology and Intensive Care, Stavanger University Hospital, Gerd Ragna Bloch Thorsens gate, Stavanger, 4011, Norway

    Håkon B. Abrahamsen

Authors
  1. Leif Inge K. Sørskår
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Espen Olsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eirik B. Abrahamsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gunnar Tschudi Bondevik
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Håkon B. Abrahamsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors participated in conceiving and designing the study. LIKS, HBA and GTB performed the data collections. LIKS performed the data analysis and EBA, HBA and EO contributed to the interpretation. LIKS drafted the manuscript. EBA, HBA and EO contributed to and revised the manuscript critically for intellectual content. All authors read and approved the final draft.

Corresponding author

Correspondence to Leif Inge K. Sørskår.

Ethics declarations

Ethics approval and consent to participate

For the 2012-data the Regional Committee for Medical and Health Research (REC) South-East Norway (Ref. number 2010/3326) reviewed and approved the study. Participating was voluntary, and consent was given by responding to the questionnaire. For the 2016-data approval was obtained from the Norwegian Social Science Data Services (NSD; project number 45723). All participants received information regarding the purpose of the study; they were assured that the digital questionnaires were to be treated in confidence and that no participants could be identified in the published material. Their written consent to participate in the study was given at the start of the survey.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sørskår, L.I.K., Olsen, E., Abrahamsen, E.B. et al. Assessing safety climate in prehospital settings: testing psychometric properties of a common structural model in a cross-sectional and prospective study. BMC Health Serv Res 19, 674 (2019). https://doi.org/10.1186/s12913-019-4459-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-019-4459-5

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us