Skip to main content

Advertisement

Psychometric properties of the Norwegian version of the hospital survey on patient safety culture in a prehospital environment

Article metrics

  • 802 Accesses

Abstract

Background

To develop a culture of patient safety in a regime that strongly focuses on saving patients from emergencies may seem counter-intuitive and challenging. Little research exists on patient safety culture in the context of Emergency Medical Services (EMS), and the use of survey tools represents an appropriate approach to improve patient safety. Research indicates that safety climate studies may predict safety behavior and safety-related outcomes. In this study we apply the Norwegian versions of Hospital Survey on Patient Safety Culture (HSOPSC) and assess the psychometric properties when tested on a national sample from the EMS.

Methods

This study adopted a web based survey design. The Norwegian HSOPSC has 13 dimensions, consisting of 46 items, in addition to two single-item outcome variables. SPSS (version 21) was used for descriptive data analysis, estimating internal consistency, and performing exploratory factor analysis. Confirmatory factor analysis (CFA) was applied to test the dimensional structure of the instruments using Amos (version 21).

Results

N = 1387 (27%) EMS employees participated in the survey. Overall, acceptable psychometric properties were observed, i.e. acceptable internal consistencies and construct validity. The patient safety climate dimensions with highest scores (number of positive answers) were “teamwork within units” and “manager expectations & actions promoting patient safety”. The dimension “hospital management support for patient safety” had the lowest score.

Conclusions

The results provided a validated instrument, the Prehospital Survey on Patient Safety Culture (PreHSOPSC), for measuring patient safety climate in an EMS setting. In addition, the explanatory power was strong for several of the outcome dimensions; i.e., several of the safety climate dimensions have a strong predictive effect on outcome variables related to employees’ perceptions on patient safety and safety-related attitude.

Background

Emergencies appears to constitute the most challenging situations in medicine. Prehospital emergency medical services (EMS) are sometimes called the ‘extended arm of the hospital’ and are characterized by high activity, time pressure, constantly changing environments, and uncertainty; a demanding mix for the providers, and prone to misconduct and errors [1].

Threats to patient safety in the prehospital setting consist of e.g. medication administration errors [2], communication problems [3], deviation from instructions [4], insufficient information [5], lack of training [6], intubation issues [7], patient condition and the related decision-making [8]. Some threats are technical, related to e.g. stretcher issues [9], crash related issues [10] or the introduction of new technology [11]. Frequent handovers between the different EMS organizations may also cause miscommunications and adverse events [12]. Near misses and adverse events appear to be common in the EMS setting, but the culture may suppress the reporting and sharing of such occurrences [1].

Since the famous report To Err is Human was published by the Institute of Medicine around the millennium shift [13], the amount of literature on understanding patient safety has grown – but in the context of EMS there is little research on patient safety, and thus little is understood [14, 15]. A literature study [16] pointed to knowledge gaps in the clinical handover of patients arriving by ambulance at the emergency department; e.g. handover information, transfer of responsibility, and staff perceptions and training. Another literature study [17] revealed several gaps compared to the established literature on patient safety, e.g. research into prehospital staffing, safety culture and climate, near-miss reporting, nosocomial infections, quality improvement techniques, and human factors engineering.

For the further improvement of patient safety in health care, safety culture is seen as an important issue and premise [18,19,20]. A commonly used definition for safety culture is “the product of individual and group values, attitudes, competencies and patterns of behavior that determine the commitment to, and the style and proficiency of, an organization’s health and safety programmes (sic). Organizations with a positive safety culture are characterized by communications founded on mutual trust, by shared perceptions of the importance of safety, and by confidence in the efficacy of preventive measures” [21]. Safety culture is developed in response to local conditions, past events, employees’ attitudes, and leadership’s safety-related attitudes and actions. The latter is especially crucial in the development of a good safety culture [1]. It exists several reports on the significant relationship between patient safety culture and specific patient outcomes [22], and improved safety culture is also related to safety performance and a lower incidence of adverse events [19, 23].

Safety climate is a term often used interchangeably with safety culture. Safety climate is commonly defined as “surface features of the safety culture from attitudes and perceptions of individuals at a given point in time” [21]. In other words, safety climate research is a ‘snapshot’ of the safety culture, and hence has less depth and is more transient than safety culture. Safety climate research concerns subjective perceptions and attitudes relating to a phenomenon and should not be mistaken for an objective view of the same phenomenon [24]. Safety behavior has been found to have a strong association with safety climate [25,26,27]. Research indicates that safety climate studies may predict safety behavior and safety-related outcomes such as harm or accidents [28]. Safety climate assessments have become a common practice in health care organizations, and the purposes are e.g. to conduct safety benchmarks and evaluate trends, to identify, monitor and proactively manage safety issues, to evaluate initiatives and interventions, and to meet regulatory requirements [18, 25, 29]. Such assessments have been made for over two decades, and a growing number of studies report on their value [23, 30].

Several instruments have been developed to assess patient safety climate in health care services [21]. Survey methods are regarded as a good way to study attitudes, values and perceptions, and this appears to be the dominant approach for assessing safety climate [31]. One of these is the Hospital Survey on Patient Safety Culture (HSOPSC), which was originally developed by the Agency for Healthcare Research and Quality (AHRQ) for use in hospitals. The dimensions of HSOPSC were chosen based on a literature review of the research, with a focus on safety, errors and misconducts, and on the existing instruments for measuring safety climate [32]. The HSOPSC has several positive attributes; it is one of the few safety climate measuring instruments in which initial psychometric properties are reported, it is designed for both clinical and non-clinical personnel, it distinguishes between organizational- and unit-level, there is increased use in different countries and contexts, and measuring the frequency of reported unwanted events may collaborate well with an organization’s wish for a better reporting climate [33].

Previous studies in Norway have examined the applicability of this instrument in a Norwegian setting, and the Norwegian translation has been validated for the hospital sector [34,35,36], nurses in intensive care units [37], and in an operating theatre setting [38]. However, applying the instrument in a prehospital setting would interfere with the contextual meaning of the items, affected by e.g. management style, team organization and tasks, and the implementation of reporting systems. The dimensions measured by the instrument, and the underlying model of patient safety climate may be incomplete, only partly applicable for the EMS setting. This requires a new test of the psychometric properties of the instrument in a prehospital context. There is a continued need for research into psychometric properties and the reliability and validity of replicated instruments [33, 34, 38,39,40]. The aim of our study was to test psychometric properties for HSOPSC performed in a prehospital context.

Method

Our testing of the HSOPSC in a prehospital context may be described as a three-stage process: (1) define the relevant population and retrieve necessary permissions and respondents’ contact information, (2) pre-test and adjust the instrument, (3) perform data collection and statistical analysis.

Population characteristics

Regional health trusts are responsible for the Norwegian EMS activities. Their main task is to maintain a state of medical emergency preparedness outside the hospitals and provide transport where acute medical treatment or monitoring is required. In the case of ground EMS (GEMS; car- and boat ambulance), cars are normally staffed by two persons: either two emergency medical technicians (EMT) or one EMT and another licensed health care worker with necessary EMS competence, e.g. a paramedic, a nurse or a physician. For the boat ambulance, the requirement is at least one EMT, in addition to the skipper. Some emergency missions in GEMS may require accompanying healthcare personnel with special medical competence, such as in the transportation of critically ill patients [40]. Norwegian EMTs have a high-school based vocational education, followed by a two-year apprenticeship working as an EMT, before gaining authorization. In addition to the EMT authorization, a paramedic has 60 to 180 European Credit Transfer and Accumulation System (ECTS) points [41]. Supplementing GEMS, helicopter EMS (HEMS) represents the sharp end of the prehospital chain, offering highly competent staff, consisting of an anesthesiologist, a rescuer (HEMS crewmember; HCM), and a pilot. HEMS is vital for providing patients with time-critical medical treatment, particularly in situations involving long distances to the relevant hospital [40]. Search and Rescue Services (SAR) and fixed wing (FW) air ambulances were excluded, since their mission profile and crew concepts differ substantially from HEMS, leading to an exclusion if such personnel were found among the respondents.

Questionnaire

The Norwegian version of the HSOPSC questionnaire was applied for this study. Prior research has translated the questionnaire into Norwegian and back-translated it by two different professionals [34]. Prior HSOPSC research for Norwegian hospitals [35, 36, 38] found that the outcome variable “number of events reported” proved to provide poor correlation with the safety dimensions;to compensate, the outcome dimension “stop working in dangerous situations” was amended. This outcome dimension reflects perceived individual safety behavior. It is based on items originally included as part of a questionnaire, called the Norwegian Offshore Risk and Safety Climate Inventory (NORSCI), developed through collaboration between the petroleum industry and various research environments during 2000 [42]. The Norwegian version of the HSOPSC instrument thus has 13 dimensions, 46 items and two single-item ‘outcome’ items [35, 36, 38]. The response format ranges from 1 (disagree strongly) to 5 (agree strongly) on a Likert scale. There are also seven items relating to the respondents’ work characteristics (work area, geographic location, field of competence, patient contact, work hours, seniority in the prehospital area, seniority in position).

Pre-test and adjustments of instrument

As the instrument was applied in a prehospital context, we checked the questionnaire on a test group of seven prehospital healthcare workers to ensure correct terminology. In addition, a prehospital patient safety professional helped in finding discrepancies between the hospital and the prehospital setting. The suggested changes are as listed in Table 1.

Table 1 Suggested adjustments of the instrument

We evaluated whether to include the option of “unknown/not applicable” to all or some of the items, similar to other studies [33, 35, 43]. The outcome variable “frequency of event reporting” was especially debated, as the average response may differ from the true (objective) value, and those personnel who do not know the frequency should have the option of stating so. The French HSOPSC study [43] experienced overall low missing score values, except for this outcome dimension (11%). The experience is similar for the German HSOPSC study [33], where items belonging to this outcome dimension have a relatively higher rate of “not applicable” answers than items belonging to the other HSOPSC dimensions. We believe the intention of this outcome dimension, as of other dimensions, is to gain the personnel’s perception of the reality. Therefore, it may be useful to force an answer to the items of this dimension (and other items). Consequently, the option of “unknown/not applicable” was not added, which is in accordance with e.g. the original HSOPSC questionnaire [44].

Considering the aims of the study, we believe it is important to keep the instrument as close to the original Norwegian HSOPSC as possible. Consequently, no items were left out or conceptually changed before distribution of the survey.

Data collection

E-mail addresses for prehospital personnel in the Norwegian GEMS and HEMS were retrieved from prehospital system leaders. We applied a web-based tool (SurveyXact) to conduct the survey, and an individual link to the questionnaire was distributed by e-mail to all personnel. Data were collected between October and December 2016, and non-responders received up to five reminders before the study was closed.

Statistical analysis

Psychometric assessment of validation was applied [45, 46] to evaluate the HSOPSC.

Construct validity

To determine the degree of fit between the sample and the constructed measurement instrument, a confirmatory factor analysis (CFA) was performed to analyze the construct validity, i.e. an assessment of the relationship between items, and between items and an underlying dimension. Negatively worded items were reversed, and covariation was allowed between the underlying dimensions.

The chi-square test is problematic for assessing model fit for large samples [47] and is thus not reported for this study. For assessing global fit, the following indices were applied: Standardized Root Mean Square Residual (SRMR), Tucker-Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA) and Comparative Fit Index (CFI). A good fit for RMSEA is a value below 0.5 [48]. Values for TLI and CFI in the 0.90s are generally accepted as guidance values for an acceptable fit, while values above 0.95 reflect a good model fit [48, 49]. It has been suggested to use a two-index strategy by reporting SRMR with one of the fit indices (e.g. CFI or RMSEA), with the guidance criteria CFI > 0.95, SRMR < 0.8 and RMSEA < 0.6 [50]. Guidance values for model fit may prove too strict for complex models with large samples, and the values for TLI and CFI should be reduced accordingly [46]; see Table 2.

Table 2 Guidance values for model fit indices

Items providing high loadings on a factor would indicate that they converge to a common point, demonstrating convergent validity for a latent construct. All factor loadings should be statistically significant and at least 0.5 or higher (ideally 0.7 or higher) for standardized estimates [46]. It is not desirable to have several loadings at very high levels, and a range of loadings between 0.6 and 0.9 seems reasonable [45].

Discriminant validity means that individual measured items should represent only one latent construct, and the presence of high cross-loadings potentially indicates a lack of discriminant validity. Inter-correlation between the dimensions was examined by Spearman’s Rho correlation: 0.0–0.25 little or no relationship; 0.25–0.50 fair degree of relationship; 0.50–0.75 moderate to good relationship; >0.75 very good to excellent relationship [51]. MANOVA (multivariate analysis of variance; Wilks’ Lambda) was performed to examine whether the different work characteristics had an overall influence on the overall statistical variance of the HSOPSC dimensions.

To evaluate possible other structures of safety climate dimensions, exploratory factor analysis (EFA) was applied. Varimax rotation was adopted to interpret the factor loadings independently. The latent root criterion (latent root > 1) was applied to identify factors and correlations between measured items [45]. The level for acceptable factor loading was set at ≥ 0.4 [46] and the level for (undesired) cross-loadings at ≥ 0.3. EFA was also forced to extract two factors to examine the grouping of system-level and unit-level dimensions.

To find evidence for criterion-related validity, associations between the safety climate dimensions and the outcome variables are developed by use of linear regression.

Internal consistency

Cronbach’s alpha was estimated for the different factors to determine whether they yielded internal consistency and acceptable alpha coefficients between 0.70 and 0.90 [52]. Alpha coefficients may understate reliability [46], but this is relatively inconsequential for practical applications such as meta-analysis [53].

Confirmatory factor analyses (maximum likelihood) were estimated using AMOS 21.0. The other statistical analyses were performed using SPSS 21.0.

Ethics approval and consent to participate

Approval was obtained from the Norwegian Social Science Data Services (NSD; project number 45723). The Regional Committee for Medical and Health Research West-Norway (REK west) evaluated this project as “not mandatory to submit” (Ref. number 2015/2249). The participants received information regarding the purpose of the study; they were assured that the digital questionnaires were to be treated in confidence and that no participants could be identified in the published material. Their written consent to participate in the study was given at the start of the survey.

Results

Sample characteristics

Individuals participating in the survey totaled 1387 (26% response rate from GEMS and 55% from HEMS; combined, 27% of the total population). The GEMS sample was retrieved from questionnaires conducted in 17 (of 18) health trusts. The sample was considered representative, based on variation in demographic variables, e.g. distribution in professional groups, range in seniority, and geographic location.

For the analyses, only returned questionnaires with all items answered were used. The majority of incomplete questionnaires was discontinued early in the survey, and we evaluated that replacing missing values was not expedient. Excluding responses with missing data (listwise deletion) provided 1154 full responses, consisting of the responses from 1045 GEMS employees and 109 HEMS employees. The sample size coincides with suggested criteria: >200 and at least 10 times the estimated parameters [54].

Of the 1154 respondents, a high number worked directly with patients (98%). As shown in Table 3, the largest professional group was EMTs (47%). Most respondents were from the South-East Regional Health Trust (38%), and the rest were evenly divided among the other three regional health trusts. Respondents were distributed evenly among the other seniority intervals, with a median of at least ten years of seniority.

Table 3 Demographic and professional characteristics of the 1154 employees in the study

Descriptive statistics

The mean statistics, standard deviation (SD) and confidence interval (CI) for each of the measurement concepts are presented in Table 4. Among the 1154 respondents, the safety grade was reported as ‘excellent’ by 53 (4.6%), ‘very good’ by 644 (55.8%), ‘acceptable’ by 389 (33.7%), ‘poor’ by 63 (5.5%) and ‘very poor’ by 5 (0.4%). The mean value was observed to be 3.59, where 5 represents ‘excellent’ and 1 represents ‘very poor’. The mean for the ten safety climate dimensions for the HSOPSC ranged from 3.03 to 4.03. The patient safety climate dimensions with highest mean scores, i.e. a higher number of positive answers, were “teamwork within units” (4.03) and “manager expectations & actions promoting patient safety” (3.79). For the single-item “number of events reported (last 12 months)”, 460 (39.9%) had filed no reports, 458 (39.7%) had filed 1-2 reports, 177 (15.3%) had filed 3-5 reports, and 59 (5.1%) had filed 6 reports or more. Overall, variance of items was considered adequate.

Table 4 Means, standard deviation (SD), 95% confidence interval (CI) and Cronbach’s alpha coefficients measured by the HSOPSC

Internal consistency

Cronbach’s alpha coefficients varied from 0.64 (teamwork across units) to 0.83 (manager expectations & actions for promoting patient safety) for HSOPSC (Table 4). Removing items from the dimensions with alpha value <0.7 proved to provide either no or marginal value increase. Compared with the dimensions of the original HSOPSC (retrieved from [43]), the median coefficients in our study are slightly lower than the original results. The greatest difference from the original is 0.64 vs. 0.80 (teamwork across units). Compared to published Norwegian studies [34, 37, 38, 55], a majority of coefficients in the current study are either within or higher than the interval of previous observed results. The dimension “staffing” has also been observed with a low coefficient in HSOPSC studies from other countries [43], and our value of 0.65 seems high, relative to these other studies (ranging from 0.44 to 0.65), including the original (0.63). The dimension “teamwork across units” had an observed value of 0.64, which is relatively low compared to both the Norwegian studies (0.65-0.73) [34, 37, 38] and those of other countries (0.59-0.80) [43].

Construct validity

CFA was applied to determine the model fit of the HSOPSC. Overall, compared to the guidance values in Table 5, it demonstrates good model fit values.

Table 5 Model fit

Factor analyses revealed three items with loadings below 0.5; item H2 (0.41), item A5 (0.43), and item A11 (0.47). The range of the other loading values was 0.56 to 0.87 (Table 6). All the items observed with relative weak factor loading (<0.50) had been pointed out as challenging during the pre-test of the instruments.

Table 6 HSOPSC dimensions and items

Although several factor loadings fell below 0.6, none of the factors had more than one value below 0.59. None of the factors had all loadings of high values (>0.80). Following the reasoning that the values should be between 0.6 and 0.9, both versions indicated an overall acceptable convergent validity.

As shown in Table 7, the inter-correlations ranged from 0.18 to 0.68 for the dimensions. Excluding the outcome dimensions, the inter-correlations between the safety climate dimensions ranged from 0.30 to 0.68 (fair to good degree of relationship). No values revealed a very good to excellent relationship between dimensions (>0.75).

Table 7 Inter-correlations (Spearman’s Rho) of the HSOPSC dimensions

In addition, by utilizing MANOVA, a significant Wilk’s Lambda (p < 0.001) was measured for all different employee characteristics, except for “seniority in position” (p = 0.060). Overall, acceptable discriminant validity is found.

EFA performed on the 46 items provided eight factors with latent root value greater than 1. The results in full are presented in Additional file 1: Appendix 1. The factors captured 56.2% of the total variance of all the items. The dimensions “Organizational learning - continuous improvement”, “Communication openness”, “Feedback and communication about error” and three of four items from “Manager expectations & actions promoting patient safety” loaded into factor 1. Dimensions “Teamwork across units” and “Handoffs and transitions” loaded into factor 2, and “Staffing” and “Overall perceptions of safety” loaded into factor 3. Of 16 cross-loadings (> 0.3), three cross-loadings were greater than 0.4 and also greater than the loading on its primary dimension: items A18 (“Our procedures and systems are good at preventing errors from happening”), D6 (“Staff are afraid to ask questions when something does not seem right”) and A2 (“We have enough staff to handle the workload”). Item A18 loaded into factor 1 as specified above, item D6 loaded into factor 4 alongside the dimension “Nonpunitive response to error”, and item A2 loaded into factor 5 alongside the dimension “Teamwork within units”. Two items showed overall loading below 0.4; items A11 (“When one area in this unit gets really busy, others help out”) and C3 (“Whenever pressure builds up, my manager wants us to work faster, even if it means taking shortcuts”).

EFA was also applied to confirm the second-order two-factor structure for the seven unit-level dimensions and three system-level dimensions. While most dimensions loaded into the designated factor in the postulated model, the dimension “Hospital management support for patient safety” loaded into the unit-level factor (loading 0.57), with a cross-loading on the system-level factor (loading 0.39). Of the total variance, 63.4% was captured by these factors. Evidently, we did not find full second-order level factors as in previous published results for HSOPSC [43].

A regression analysis was conducted for each of the outcome variables (Table 8). The safety climate dimensions had an overall positive effect on the outcome variables, except for the “number of events reported (last 12 months)”, which revealed negative influence from the safety dimensions. In addition, this dimension had low explanatory power, relative to the other outcome dimensions. The dimensions “nonpunitive response to error” and “teamwork across units” were both significant for only one outcome variable.

Table 8 Regression analysis testing the concurrent validity of HSOPSC

Discussion

This study produced two major findings. Firstly, the study provided overall acceptable psychometric properties, i.e. acceptable internal consistencies and construct validity. However, there were a few exceptions related to weak loadings for some items. Secondly, the explanatory power was strong for several of the outcome dimensions; i.e., it offers stronger predictions regarding which safety climate dimensions have an effect on which outcome variables. Based on these two findings, we provide the EMS environment with a suitable instrument for assessing the patient safety climate in prehospital settings – the Prehospital Survey on Patient Safety Culture (PreHSOPSC).

Validity of the PreHSOPSC

The observed Cronbach’s alphas were between the recommended limits of 0.70 to 0.90 for all but three dimensions (0.64, 0.65 and 0.69), but only the dimension “teamwork across units” had a relatively low alpha value, compared to those of other studies. EFA pointed towards an eight-factor construct, instead of the 13 dimensions that constitute the Norwegian HSOPSC. However, with a few exceptions, the results indicated acceptable convergent and discriminant validity, and the CFA demonstrated overall good model fit compared to the recommended values. The regression analyses showed that the outcome variables had explanatory power values in the range 0.26 to 0.59 (26-59%), except for the outcome dimension “Number of events reported (last 12 months)” at 0.03 (3%). The latter result is consistent with those of other HSOPSC studies [34, 56]. Rather than being a risk indicator for patient safety, this outcome variable serves better as a change measure to monitor the degree of reporting over time [57].

Implications

The HSOPSC instrument was primarily developed by AHRQ for hospitals [32]. Although the HSOPSC is tested for different contexts within the healthcare system, it is not applicable for all contexts in general. Further research should test and validate the instrument for other safety contexts to obtain a generalized instrument for measuring safety climate. An implication followed by the difference between the prehospital and the hospital context is to test the network of relationships between the variables; i.e. the existence of a “nomological network” [45]. Future research should investigate further the existence of such a network, and more evidence for nomological validity should be produced.

Another topic for future research is to take a closer look at the weak items identified by the CFA and the EFA, especially the items pointed out as challenging during the pre-test of the instrument. Still, post hoc modification, by means of e.g. modification indices and standardized residuals [45], should be carried out sparingly and based on theoretical and practical plausibility (e.g. [58]). The use of the HSOPSC instrument in a new context is a challenge in itself, and, instead of performing adjustments and modifications, the development of a new instrument targeted on an EMS context may be a better solution. In particular re-evaluating the position of the prehospital chain in relation to the unit level and hospital level, as indicated by both the lack of evidence of second-order level factors and the relatively low alpha value of the safety dimension “teamwork across units”, compared to other studies. A disadvantage of developing a new instrument is the lack of opportunity to compare it with other studies.

The dimensionality revealed by the EFA may also prove useful if developing a new instrument. Although testing within the prehospital domain, our results are similar to those of European hospital adaptations of HSOPSC, where the original postulated dimensions were not fully identified. Several studies support the factor combination of “Teamwork across units” and “Handoffs and transitions” [33, 35, 43, 59]. Other studies found a similar factor combination of the dimensions “Staffing” and “Overall perception of safety” [33, 39, 54]. The factor combination of dimensions “Communication openness”, “Feedback and communication about error” and “Organizational learning - continuous improvement” is similar to the findings of the Swedish version [54] and partly similar to the findings of several other studies [33, 35, 39, 43, 59, 60]. Our findings for the dimension “Manager expectations & actions promoting patient safety” added to the factor combination above did not support other European versions (to our knowledge); alongside the other factor combinations, it should be investigated further in future studies.

In adjusting the terminology of the original Norwegian HSOPSC for a prehospital context before performing the survey, the purpose was to perform as few adjustments as necessary and not to change the instrument conceptually. Based on this, the option of answering “unknown/not applicable” was not included for any of the items in the questionnaire. Although such an approach decreases the risk of missing score values for the items, it may increase the risk of missing other valuable data. Some aspects of patient safety may be less relevant for the prehospital domain compared to the hospital domain and, in ‘forcing’ respondents to provide an answer, there is a risk of not capturing items that either require an amended explanation in the survey or should be considered candidates for modification or removal. AHRQ is developing a new version of the HSOPSC, in which one of the concerns they are focusing on is to add a "does not apply/don’t know" response option [61]. Their argument is that respondents do not know how to answer if an item does not apply to them. In such cases, a “does not apply” option is reasonable, and adopting this option in future testing of the PreHSOPSC should be considered. However, a “don’t know” option may lead respondents to believe that they should objectively know how to respond, which may increase the risk of missing score values. If adding this option, the items of the questionnaire should be worded in such a way that they lead the respondent to answer according to their social-cognitive observation and evaluation of the environment.

A contextual challenge within acute healthcare is related to the outcome dimension “stop working in dangerous situations” [35]; employees are expected to continue working in order to e.g. rescue a patient. In general, this may follow three lines in this context, with increased risk for either the patient or the critical care provider/team – or for both. This may arise if the chosen approach to providing critical care is considered riskier, relative to alternative approaches. An example of this is to perform a rescue operation with a line from a helicopter in challenging terrain, due to e.g. elevations or tree height, while a possible option is to carry the patient out to a safer pick-up point. Another example is reckless driving of a car ambulance during an emergency response. A different view may be provided on this challenge; that safety and emergent care are not discordant concepts and EMS quality patient care can be administered in a safe manner [62]. Consequently, the results of this outcome dimension should be evaluated with the purpose of increasing safety for both patients and personnel.

Despite an adequate number of respondents, the response rate was at the lower end of satisfactory. One cause may be related to being distributed only digitally and not on paper. The majority of the email addresses were work email addresses, which may have caused technical difficulties in opening the questionnaires. In addition, if internal communication is not performed by email, a number of respondents may not have opened their email account during the sample period. Due to the scattered geographic nature of the prehospital environment, paper distribution would have been rather difficult to perform, but it would probably have increased the number of respondents. Another attribute in the prehospital environment is the embedded ‘fast pace working’ culture, and what is perceived as a time-demanding survey may cause the employee to not start or complete the questionnaire. This may explain why the majority of the respondents that did not complete the survey also stopped relatively early in the questionnaire. Another observation that may be related to this culture is the following; before starting the survey, the respondents were asked to provide their consent to participate – and nearly 200 responded negatively to this. Consequently, shorter surveys such as the Norwegian HSOPSC-Short [35] may be preferable. Another aspect of the low number of respondents, in addition to the health region not participating, may also be a cultural link to undesired ‘outside’ observations or that the survey is not prioritized due to ongoing staffing processes.

The aforementioned new version of HSOPSC (version 2.0) under development by AHRQ is based on some of the same considerations made in this article, e.g. issues regarding the use of idioms, alignment to other contexts, and length of survey [61]. Although the instrument is still mainly developed for hospitals, this article demonstrates the benefit of testing the suggested changes and a new safety climate instrument in the ongoing patient safety climate research in the prehospital domain.

Limitations

There are limitations to the data, which must be borne in mind. Firstly, as previously mentioned, the response rate was low relative to other HSOPSC studies (e.g. [33, 34, 39, 43, 54]). Low response rate may cause non-response bias, i.e. a discrepancy between the employees that responded and the those that did not.

Secondly, the study was limited to the main transport part of the prehospital environment (GEMS and HEMS), thus excluding other parties more or less linked to the prehospital chain (e.g. emergency rooms or emergency medical communications center). Hence, the safety climate for the full prehospital environment is not fully measured.

Thirdly, the instrument has not been tested for predictive validity, i.e. provided evidence of correlation with an external criterion separated in time [45], e.g. reporting of errors, degree of patient compensation, or other patient safety outcomes. Until the instrument has been tested against other external criteria in the prehospital setting, the impact on the EMS safety climate is not fully known.

Conclusion

Conducting safety climate research provides an opportunity to identify and address areas for improving patient safety. Often, an improved safety climate is accomplished through a number of interventions, targeting one or more dimensions at a time [21]. Using surveys to measure the current status is a suggested first step [63, 64]. To our knowledge, this is the first systematic study of patient safety climate in a Norwegian EMS environment by use of the HSOPSC. The HSOPSC has been previously validated for a Norwegian hospital setting, but, as the prehospital context is different, it generates a need to test the instrument for psychometric properties. Both threats to patient safety and new patient safety improvements/interventions require effective validated instruments to evaluate their impact on the prehospital patient safety climate. Hence, it is a satisfactory result of this study to provide the prehospital environment with a validated instrument, the PreHSOPSC, for measuring the prehospital patient safety climate. This is beneficial in the continuous work of improving patient safety, as the application of the PreHSOPSC may both indicate and predict safety behavior and safety-related outcomes.

Abbreviations

AHRQ:

Agency for Healthcare Research and Quality

CFA:

Confirmatory Factor Analysis

CFI:

Comparative Fit Index

CI:

Confidence Interval

ECTS:

European Credit Transfer and Accumulation System

EFA:

Exploratory Factor Analysis

EMS:

Emergency Medical Services

EMT:

Emergency Medical Technician

FW:

Fixed Wing

GEMS:

Ground Emergency Medical Services

HCM:

HEMS Crew Member

HEMS:

Helicopter Emergency Medical Services

HSOPSC:

Hospital Survey on Patient Safety Culture

MANOVA:

Multivariate Analysis of Variance

NORSCI:

Norwegian Offshore Risk and Safety Climate Inventory

PreHSOPSC:

Prehospital Survey on Patient Safety Culture

RMSEA:

Root Mean Square Error of Approximation

SAR:

Search and Rescue Services

SD:

Standard Deviation

SRMR:

Standardized Root Mean Square Residual

TLI:

Tucker-Lewis Index

References

  1. 1.

    St Pierre M, Hofinger G, Buerschaper C, Simon R. Crisis management in acute care settings: human factors, team psychology, and patient safety in a high stakes environment. Berlin: Springer; 2011.

  2. 2.

    Vilke GM, Tornabene SV, Stepanski B, Shipp HE, Ray LU, Metz MA, et al. Paramedic self-reported medication errors. Prehospital Emergency Care. 2007;11(1):80–4.

  3. 3.

    MacDonald RD, Banks BA, Morrison M. Epidemiology of adverse events in air medical transport. Academic Emergency Medicine. 2008;15(10):923–31.

  4. 4.

    Rittenberger JC, Beck PW, Paris PM. Errors of omission in the treatment of prehospital chest pain patients. Prehospital Emergency Care. 2005;9(1):2–7.

  5. 5.

    Burrell L, Noble A, Ridsdale L. Decision-making by ambulance clinicians in London when managing patients with epilepsy: a qualitative study. Emerg Med J. 2013;30:236–40.

  6. 6.

    Thies KC, Sep D, Derksen R. How safe are HEMS-programmes in Germany?: a retrospective analysis. Resuscitation. 2006;68(3):359–63.

  7. 7.

    Lossius HM, Røislien J, Lockey DJ. Patient safety in pre-hospital emergency tracheal intubation: a comprehensive meta-analysis of the intubation success rates of EMS providers. Critical Care. 2012;16(1):R24.

  8. 8.

    Price R, Bendall JC, Patterson JA, Middleton PM. What causes adverse events in prehospital care? A human-factors approach. Emerg Med J. 2013;30:583–8.

  9. 9.

    Wang HE, Weaver MD, Abo BN, Kaliappan R, Fairbanks RJ. Ambulance stretcher adverse events. Qual Saf Health Care. 2009;18(3):213–6.

  10. 10.

    Fournier M, Chenaitia H, Masson C, Michelet P, Behr M, Auffray JP. Crew and patient safety in ambulances: results of a personnel survey and experimental side impact crash test. Prehospital and Disaster Medicine. 2013;28(04):370–5.

  11. 11.

    Mytton OT, Velazquez A, Banken R, Mathew JL, Ikonen TS, Taylor K, et al. Introducing new technology safely. Qual Saf Health Care. 2010;19(Suppl 2):i9–i14.

  12. 12.

    Hesselink G, Berben S, Beune T, Schoonhoven L. Improving the governance of patient safety in emergency care: a systematic review of interventions. BMJ Open. 2016;6(1):e009837.

  13. 13.

    Donaldson MS, Corrigan JM. Kohn LT (Eds.). To err is human: building a safer health system (Vol. 6). Washington DC: National Academies Press; 2000.

  14. 14.

    Atack L, Maher J. Emergency medical and health providers' perceptions of key issues in prehospital patient safety. Prehospital Emergency Care. 2010;14(1):95–102.

  15. 15.

    Patterson PD, Huang DT, Fairbanks RJ, Wang HE. The emergency medical services safety attitudes questionnaire. American Journal of Medical Quality. 2010;25(2):109–15.

  16. 16.

    Bost N, Crilly J, Wallis M, Patterson E, Chaboyer W. Clinical handover of patients arriving by ambulance to the emergency department–a literature review. International Emergency Nursing. 2010;18(4):210–20.

  17. 17.

    Bigham BL, Buick JE, Brooks SC, Morrison M, Shojania KG, Morrison LJ. Patient safety in emergency medical services: a systematic review of the literature. Prehospital Emergency Care. 2012;16(1):20–35.

  18. 18.

    Nieva VF, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care. 2003;12(suppl 2):ii17–23.

  19. 19.

    Singer S, Lin S, Falwell A, Gaba D, Baker L. Relationship of safety climate and safety performance in hospitals. Health Services Research. 2009;44(2p1):399–421.

  20. 20.

    Kaufman G, McCaughan D. The effect of organisational culture on patient safety. Nursing Standard. 2013;27(43):50–6.

  21. 21.

    Halligan M, Zecevic A. Safety culture in healthcare: a review of concepts, dimensions, measures and progress. BMJ Quality & Safety. 2011;20(4):338–43.

  22. 22.

    DiCuccio MH. The relationship between patient safety culture and patient outcomes: a systematic review. Journal of Patient Safety. 2015;11(3):135–42.

  23. 23.

    Mardon RE, Khanna K, Sorra J, Dyer N, Famolaro T. Exploring relationships between hospital patient safety culture and adverse events. Journal of Patient Safety. 2010;6(4):226–32.

  24. 24.

    Guldenmund FW. The use of questionnaires in safety culture research–an evaluation. Safety Science. 2007;45(6):723–43.

  25. 25.

    Flin R. Measuring safety culture in healthcare: a case for accurate diagnosis. Safety Science. 2007;45(6):653–67.

  26. 26.

    Christian MS, Bradley JC, Wallace JC, Burke MJ. Workplace safety: a meta-analysis of the roles of person and situation factors. Journal of Applied Psychology. 2009;94(5):1103.

  27. 27.

    Clarke S. An integrative model of safety climate: linking psychological climate and work attitudes to individual safety outcomes using meta-analysis. Journal of Occupational and Organizational Psychology. 2010;83(3):553–78.

  28. 28.

    Huang YH, Chen JC, DeArmond S, Cigularov K, Chen PY. Roles of safety climate and shift work on perceived injury risk: a multi-level analysis. Accident Analysis & Prevention. 2007;39(6):1088–96.

  29. 29.

    Mearns K, Whitaker SM, Flin R. Benchmarking safety climate in hazardous environments: a longitudinal, interorganizational approach. Risk Analysis. 2001;21(4):771–86.

  30. 30.

    Hellings J, Schrooten W, Klazinga N, Vleugels A. Challenging patient safety culture: survey results. International Journal of Health Care Quality Assurance. 2007;20(7):620–32.

  31. 31.

    Hopkins A. Studying organisational cultures and their effects on safety. Safety Science. 2006;44(10):875–89.

  32. 32.

    Sorra JS, Nieva VF. Hospital Survey on Patient Safety Culture. AHRQ Publication No. 04-0041. Agency for Healthcare Research and Quality: Rockville MD; 2004.

  33. 33.

    Pfeiffer Y, Manser T. Development of the German version of the Hospital Survey on Patient Safety Culture: dimensionality and psychometric properties. Safety Science. 2010;48(10):1452–62.

  34. 34.

    Olsen E. Reliability and validity of the Hospital Survey on Patient Safety Culture at a Norwegian hospital. Quality and Safety Improvement Research: Methods and Research Practice from the International Quality Improvement Research Network (QIRN). 2008;173:186.

  35. 35.

    Olsen E. Exploring the possibility of a common structural model measuring associations between safety climate factors and safety behaviour in health care and the petroleum sectors. Accident Analysis & Prevention. 2010;42(5):1507–16.

  36. 36.

    Olsen E, Aase K. The challenge of improving safety culture in hospitals: a longitudinal study using hospital survey on patient safety culture. 11th International Probabilistic Safety Assessment and Management Conference and the Annual European Safety and Reliability Conference; 25–29 June 2012. Helsinki, Finland: Curran Associates, Inc.

  37. 37.

    Ballangrud R, Hedelin B, Hall-Lord ML. Nurses’ perceptions of patient safety climate in intensive care units: a cross-sectional study. Intensive and Critical Care Nursing. 2012;28(6):344–54.

  38. 38.

    Haugen AS, Søfteland E, Eide GE, Nortvedt MW, Aase K, Harthug S. Patient safety in surgical environments: cross-countries comparison of psychometric properties and results of the Norwegian version of the Hospital Survey on Patient Safety. BMC Health Services Research. 2010;10(1):279.

  39. 39.

    Waterson P, Griffiths P, Stride C, Murphy J, Hignett S. Psychometric properties of the Hospital Survey on Patient Safety Culture: findings from the UK. Quality and Safety in Health Care. 2010;19(5):e2.

  40. 40.

    Official Norwegian Reports 2015:17 Først og fremst - Et helhetlig system for håndtering av akutte sykdommer og skader utenfor sykehus. https://www.regjeringen.no/contentassets/477c27aa89d645e09ece350eaf93fedf/no/pdfs/nou201520150017000dddpdfs.pdf. Norwegian only. Accessed 27 July 2018.

  41. 41.

    Grusd E, Kramer-Johansen J. Does the Norwegian emergency medical dispatch classification as non-urgent predict no need for pre-hospital medical treatment? An observational study. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 2016;24(1):65.

  42. 42.

    Tharaldsen JE, Olsen E, Rundmo T. A longitudinal study of safety climate on the Norwegian continental shelf. Safety Science. 2008;46(3):427–39.

  43. 43.

    Perneger TV, Staines A, Kundig F. Internal consistency, factor structure and construct validity of the French version of the Hospital Survey on Patient Safety Culture. BMJ Qual Saf. 2014;23:389–97.

  44. 44.

    Hospital Survey on Patient Safety Culture. Content last reviewed March 2018. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/sops/quality-patient-safety/patientsafetyculture/hospital/index.html. Accessed 27 July 2018.

  45. 45.

    Netemeyer RG, Bearden WO, Sharma S. Scaling procedures: issues and applications. Sage Publications; 2003.

  46. 46.

    Hair JF, Black WC, Babin BJ, Anderson RE. Multivariate data analysis. 7th ed. Pearson: Essex; 2014.

  47. 47.

    Hooper D, Coughlan J, Mullen M. Structural equation modelling: guidelines for determining model fit. Articles. 2008;2.

  48. 48.

    Schumacker RE, Lomax RG. A beginner's guide to structural equation modeling. In: Psychology Press; 2004.

  49. 49.

    Lei PW, Wu Q. Introduction to structural equation modeling: issues and practical considerations. Educational Measurement: Issues and Practice. 2007;26(3):33–43.

  50. 50.

    Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: a Multidisciplinary Journal. 1999;6(1):1–55.

  51. 51.

    Bodur S, Filiz E. A survey on patient safety culture in primary healthcare services in Turkey. International Journal for Quality in Health Care. 2009;21(5):348–55.

  52. 52.

    Tavakol M, Dennick R. Making sense of Cronbach's alpha. International Journal of Medical Education. 2011;2:53.

  53. 53.

    Peterson RA, Kim Y. On the relationship between coefficient alpha and composite reliability. Journal of Applied Psychology. 2013;98(1):194–8.

  54. 54.

    Hedsköld M, Pukk-Härenstam K, Berg E, Lindh M, Soop M, Øvretveit J, Sachs MA. Psychometric properties of the hospital survey on patient safety culture, HSOPSC, applied on a large Swedish health care sample. BMC Health Services Research. 2013;13(1):332.

  55. 55.

    Olsen E, Aase K. A comparative study of safety climate differences in healthcare and the petroleum industry. BMJ Quality & Safety. 2010;19(Suppl 3):i75–9.

  56. 56.

    Etchegaray JM, Thomas EJ. Comparing two safety culture surveys: safety attitudes questionnaire and hospital survey on patient safety. BMJ Quality & Safety. 2012;21(6):490–8.

  57. 57.

    Sorra JS, Dyer N. Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture. BMC Health Services Research. 2010;10(1):199.

  58. 58.

    MacCallum RC, Roznowski M, Necowitz LB. Model modifications in covariance structure analysis: the problem of capitalization on chance. Psychological Bulletin. 1992;111(3):490–504.

  59. 59.

    Kapaki V, Souliotis K. Psychometric properties of the Hospital Survey on Patient Safety Culture (HSOPSC): Findings from Greece. In: Vignettes in Patient Safety-Volume 2: InTech; 2018.

  60. 60.

    Smits M, Christiaans-Dingelhoff I, Wagner C, van der Wal G, Groenewegen P P. The psychometric properties of the 'Hospital Survey on Patient Safety Culture' in Dutch hospitals. BMC Health Services Research. 2008;8(1):230.

  61. 61.

    Update on Version 2.0 of the Hospital Survey on Patient Safety Culture. Content last reviewed March 2018. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/sops/quality-patient-safety/patientsafetyculture/hospital/update/index.html. Accessed 27 July 2018.

  62. 62.

    Brice JH, Studnek JR, Bigham BL, Martin-Gill C, Custalow CB, Hawkins E, Morrison LJ. EMS provider and patient safety during response and transport: proceedings of an ambulance safety conference. Prehospital Emergency Care. 2012;16(1):3–19.

  63. 63.

    Huang DT, Clermont G, Sexton JB, Karlo CA, Miller RG, Weissfeld LA, et al. Perceptions of safety culture vary across the intensive care units of a single institution. Critical Care Medicine. 2007;35(1):165–76.

  64. 64.

    Pronovost P, Weast B, Rosenstein B, Sexton JB, Holzmueller CG, Paine L, et al. Implementing and validating a comprehensive unit-based safety program. Journal of Patient Safety. 2005;1(1):33–40.

  65. 65.

    The Working Environment Act. 2017. (Lov om arbeidsmiljø, arbeidstid og stillingsvern mv.) https://www.arbeidstilsynet.no/contentassets/e54635c3d2e5415785a4f23f5b852849/working-environment-act-october-web-2017.pdf. Accessed 6 Oct 2018.

Download references

Acknowledgements

We thank the healthcare providers and local leaders in EMS and HEMS for participating. The authors are grateful to Medical Director, Anne-Cathrine Braarud, at Oslo University Hospital, for her comments and suggestions for instrument adjustments.

Funding

LIKS received funding from the Ministry of Education and Research. The funders had no part in the design of the study, collection, analysis, interpretation of data and in writing the manuscript.

Availability of data and materials

The data sets generated and analyzed during the current study are not publicly available, as further papers will be written based on the data sets, but are available from the corresponding author on reasonable request.

Author information

LIKS, EBA, HBA and SJMS conceived of and designed the study. HBA and SJMS have been involved in performing changes to the instrument. LIKS performed the data collection and drafted the manuscript. LIKS performed the data analysis and EO contributed to the interpretation. EBA, HBA and EO contributed to and revised the manuscript critically for intellectual content. All authors read and approved the final draft.

Correspondence to Leif Inge K Sørskår.

Ethics declarations

Ethics approval and consent to participate

Approval was obtained from the Norwegian Social Science Data Services (NSD; project number 45723). The Regional Committee for Medical and Health Research West-Norway (REK west) evaluated this project as “not mandatory to submit” (Ref. number 2015/2249). The participants received information regarding the purpose of the study; they were assured that the digital questionnaires were to be treated in confidence and that no participants could be identified in the published material. Their written consent to participate in the study was given at the start of the survey.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Exploratory factor analysis of the Norwegian Prehospital Survey of Patient Safety Culture (PreHSOPSC). Rotated component matrix (DOCX 15 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Keywords

  • Prehospital
  • Emergency medical services
  • Patient safety culture
  • Patient safety climate
  • HSOPSC
  • Psychometric properties