Patient safety in surgical environments: Cross-countries comparison of psychometric properties and results of the Norwegian version of the Hospital Survey on Patient Safety

Design

This study was a cross-sectional survey examining perceptions of patient safety climate in operating theatre personnel using the validated Norwegian version of the HSOPS [1, 5, 6]. The Norwegian version of the HSOPS has previously been validated for paper distribution [7]. The questionnaire was translated into Norwegian before it was retranslated back to English, processed by two independent researchers. A pilot test was performed using health care personnel, to ensure that the concepts were correctly worded and conceptualized [8]. We performed the survey using a mixed distribution method, with a web and a paper version of the HSOPS. Before we distributed the survey, eight health care workers and research personnel pilot tested the readability and functionality of the web-based version. This pilot test resulted in splitting the first section of the HSOPS into two separate sections to improve readability. The Norwegian version has not previous been validated using a mixed distribution method.

Sample

The sample consisted of operating theatre personnel at Haukeland University Hospital, Bergen, Norway: surgeons, anaesthesiologists, operating theatre nurses, nurse anaesthetists and ancillary personnel (unit assistants, clerks and cleaning assistants) present at work during a four week study period in October and November 2009. The hospital is one of the largest in Norway with 1,100 beds and about 10,000 employees, serving a population of 950,000 as a referral hospital and 500,000 as an emergency hospital. The annual number of surgical procedures exceeds 24,000. The following surgical departments were included in the survey: orthopaedic; thoracic; neuro-; ear, nose and throat; maxillofacial; plastic; endocrine; urinary; gastrointestinal; and obstetric surgery. Of the eligible personnel 575 individuals were invited to participate. Thirty-one enlisted personnel were absent during the study period due to vacation, illness, working other places, education or specialist training and were not included.

Selection of the clinical setting of this sample presents a large number of physicians; surgeons and anaesthesiologists, compared to specialist nurses contrasting other patient safety climate studies which included all health care personnel of the hospitals [5–7, 10, 11]. The operating theatre personnel are located at three separate locations: the largest of them are the central operating unit with 19 operating theatres, the women's clinic with 5 operating theatres in a separate building, and the day surgical unit with 2 operating theatres being physical connected to the central operating unit.

Data collection

We distributed the web-based questionnaire to the operating theatre personnel through the hospital e-mail system. A paper version was sent to the personnel not responding to e-mail reminders. Physicians received two e-mail reminders before being sent the paper version. The operating theatre nurses, nurse anaesthetists and ancillary personnel received the paper version after one reminder. Some of the operating theatre personnel had logistical and technical difficulties in responding to the web-based questionnaire, such as being unable to gain access to the web version when using common log-on procedures and not having enough time in between daily routines. Identification numbers were assigned to or printed on each questionnaire to identify the working area or unit. We preserved the anonymity of data collection for the paper version by having respondents use closed envelopes addressed to the primary investigator (ASH) through the hospital's internal mail system. A consultant at the hospital research and development department administered the web-based questionnaire.

Data screening

We examined data and checked for errors. Respondent who answered less than half the questionnaire items were excluded. Five respondents had chosen two options in one item, and we allocated these to the most positive or negative value of the categories [27]. The highest number of missing values was in the factor "frequency of events reported", which had missing values in 5.3% of the items. In the remaining factors, missing values were present in 0.3% to 3.1% of the cases. We did not exclude any items based on these few missing values and replaced them by the mean scores of the item.

Questionnaire

Sample characteristics are included such as profession, clinical experience, working hours during the week and working area or unit. Results are compared with data from 885 United States hospitals, the Hospital Survey on Patient Safety: 2010 User Comparative Database Report, as well as data from three hospitals in the Netherlands and one university hospital in Norway [2, 5, 6, 8]. The results are presented as percent of average positive response (agree or strongly agree) in each item and factor, the highest percentage being the most positive. Sorra et al. [2] describe the method of calculation. In the previous psychometric evaluation study of the Norwegian version of the HSOPS, the twelve factors were classified as outcome variables (factor 1 and 2), measures at hospital unit level (factor 3-9) and at an overall hospital level (factor 10-12) [6].

Statistical analysis

Descriptive statistics was used to display the frequencies of sample characteristics and patient safety climate factors. Negatively worded items were reversed to ensure that positive answers indicated a high score. For the 12 factors of the HSOPS questionnaire inferential statistics were used. To analyse differences in the means of explanatory variables according to profession and surgical departments' one-way analysis of variance was used. To investigate whether the HSOPS would fit with the data from a surgical environment sample in Norway we performed factor analysis using Varimax rotation [28]. Bartlett's test was used to examine if the inter-item correlations were sufficient. The chi-square distribution should correspond with the significance level of P = 0.05 [29]. The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy, with value range from 0 to 1, should exceed 0.5 to meet Kaiser's criterion [30]. The internal consistency of the factors was assessed by intra-class correlations and by Cronbach's alpha. For the factors to be consistent the alphas should exceed 0.7 [31]. We measured correlations by Pearson's correlation coefficient and internal consistency by Cronbach's alpha. We used SPSS (version 17.0) for Windows for data analysis [32].

Ethics

The study was performed according to the ethical standards of the Helsinki Declaration [33]. The hospital research manager and the unit management leaders approved the study. The Committee for Medical Research Ethics of Western Norway reviewed the study and responded that approval was not necessary according to Norwegian law, since the study did not involve patients. The data privacy unit at Haukeland University Hospital consented to the project.

Sample

Background variables

Two of the 12 patient safety culture factors were considered outcome variables: "overall perception of safety" and "frequency of events reported". Seven of the remaining 10 factors were classified to be measured at the hospital unit level and the last three at the hospital level. The means of these factors were compared according to the background variables using one-way analysis of variance (ANOVA). Additional file 1: Table S1 presents the results for the various professions; the mean factor scores ranged from 2.80 to 3.55 between the five professional groups (ANOVA: P < 0.01). The mean factor scores of the two outcome variables differed between the professions (P < 0.01). In addition the mean factor scores differed within the variable "surgical departments" (ANOVA: P < 0.05) except for the factors "frequency of events reported" and "non-punitive response to error".

Reliability and validity

The internal consistency of the patient safety climate factors was confirmed when measured using Cronbach's alpha, ranging from 0.64 to 0.85, except for the factor "adequate staffing", the internal consistency was 0.59. The correlations found supported discriminate and construct validity. The unit-level factors had mutual correlations ranging from 0.20 to 0.61 (P < 0.01). The hospital-level factors had correlations varying from 0.26 to 0.62 (P < 0.01). The correlation between the outcome variables "patient safety grade" and "overall perception of safety" was 0.59 (P < 0.01). Additional file 2: Table S2 lists all correlations.

Factor analysis

Comparative results

The percent of average positive responses (agree, strongly agree) varied between 22% and 72% across the twelve patient safety climate factors of the HSOPS. The total average percent of the average positive responses in all the patient safety climate factors was 47% in our sample. Table 4 compares these results to comparative database results from hospitals in the United States and results from the Netherlands and Norway. Figure 1 illustrates the variation between the percent average positive responses in the twelve patient safety climate factors of this study compared with the results from the United States.

		United States [2]	Netherlandsa	Norway [8]	Norway
Patient safety climate factors of the HSOPSb		Hospital environment n= 338,607	Hospital environment n= 3,779	Hospital environment n= 1,919	Operating environment n= 358
		%	%	%	%
	Outcome variables
1.	Overall safety	65	52	-	57
2.	Frequency of events	62	38	28	31
	Unit-level factors
3.	Leaders' expectations	75	62	72	65
4.	Continuous improvement	72	47	50	46
5.	Teamwork within units	80	84	68	57
6.	Open communication	62	69	64	58
7.	Error feedback	63	49	40	37
8.	Non-punitive	44	67	72	72
9.	Adequate staffing	56	62	49	52
	Hospital-level factors
10.	Management support	72	32	25	22
11.	Teamwork across units	58	28	31	32
12	Handoffs and transitions	44	40	39	31
Total average sum score		63	53	49	47

^a Source: Wagner C, Smits M. Patient safety culture. Differences between professions and countries http://internationalforum.bmj.com/2010-forum/presentation-slides/wednesday/A7%20Wagner,%20Smits.pdf

^b Complete labels: 1: overall perceptions of safety; 2: frequency of events reported; 3; supervisors' or managers' expectations and actions promoting patient safety; 4: organizational learning - continuous improvement; 5: teamwork within units; 6: communication openness; 7: feedback and communication about error; 8: non-punitive response to error; 9: adequate staffing; 10: hospital management support for patient safety; 11: teamwork across hospital units; 12: hospital handoffs and transitions.

Variation in safety climate perception

The variation in the perception of patient safety climate factors between different surgical departments and between different professions was in accordance with previous HSOPS studies in Norway, which found significant differences (P < 0.001) both between clinical departments and professions [6, 8]. Here we found significant variations (P < 0.05) in mean scores between different surgical departments regarding the patient safety climate at the outcome and unit factor levels. Between the different professions, the variation found was significant (P < 0.001) for the patient safety climate at the outcome and unit factor levels. The anaesthesiologists and nurse anaesthetists had higher mean scores than the surgeons and operating theatre nurses, both in outcome variables and unit-level factors. The ancillary personnel had the lowest mean scores here. The group with less education and being most distant to the patients, the ancillary personnel, reports most negative as to the patient safety climate. This might imply that perceptions of the safety climate may vary between operating theatre personnel groups. However, this needs further investigation to be validated.

The HSOPS is assumed to measure patient safety climate within hospitals and hospital units [2]. This is supported by our results and by findings in hospitals in the Netherlands, with clustering of responses within hospitals and hospital units rather than between individuals [34]. Our results may indicate variation in safety climate perceptions within surgical units and between operating theatre professions. The findings of our study may also reflect a perception of distance between the management at the unit and hospital levels regarding involvement in patient safety issues. Involvement and support from hospital management are strongly associated with the success of patient safety initiatives [35, 36].

Factor analysis

We used explorative factor analysis to investigate differences between our results and the comparative database results in the United States and results from the Netherlands and Norway [5, 7, 37]. The internal consistency of our data on operating theatre personnel was in-between those of the comparative studies, but the factor "organizational learning - continuous improvement" (α = 0.64) was more satisfactory in this study than in previous studies in the Netherlands and Norway [5–7]. Combining the factor "organizational learning - continuous improvement" with "feedback and communication about error" into an 11-factor structure produced internal consistency (Cronbach's α = 0.78) that was as satisfactory as for the study in the Netherlands [5]. In the psychometric evaluation of the HSOPS within a large acute National Health Service trust in the United Kingdom, more than half the factors failed to achieve satisfactory internal consistency (α < 0.7). Their factor analysis with split-half sample validation was converted into nine dimensions [4]. The results from the United Kingdom are interesting when considering this necessary psychometric evaluation of the questionnaire. They contrast somewhat with our findings in Norway. The correlations and patterns in our study seem more consistent in the construct validity of the patient safety climate factors compared with the previously mentioned studies [4–7]. The Norwegian version of the HSOPS cannot be regarded as externally validated until more Norwegian surgical environments and hospitals have been surveyed and the results compared and validated against patient outcomes.

Comparative results

The health care personnel in United States hospitals generally seem to have a more positive perception of their hospitals' patient safety climate than operating theatre personnel in Norway. The largest difference in patient safety climate factors was found for the factor "hospital management support for patient safety", with a maximum 50 percentage-point difference in responses. These results, with the United States hospital personnel responding more positively, can be explained by cultural and organizational differences. Previous studies from Norway support our findings [6, 8]. One explanation of the excessive variation could be that the owners of hospitals in Norway measure hospital managers not as much on patient safety as on financial results. Another major deviation from the United States 2010 User Comparative Database Report results in our study is for the factor "non-punitive response to error". The difference in positive responses is 28 percentage points, this time with the operating theatre personnel in Norway responding more positively. According to the Institute of Medicine of the United States National Academies, achieving a patient safety climate in which individuals are not blamed for errors (a non-punitive climate) may accomplish an important goal towards a safer health system [38]. Our results suggest that the surgical environment in our hospital seems to have a more non-punitive climate, although the "frequency of events reported" is 31 percentage points lower than in United States hospitals compared. The patient safety climate factor that correlated most strongly with this factor was "feedback and communication about error". We interpret that this may indicate that health care personnel experiencing feedback and communication about the errors reported would benefit the patient safety climate, giving incentives to the health care personnel to report events more frequently.

Our hospital has been using an electronic error reporting system for a relatively short time (3 years), and altering systems of error reporting may influence the frequency of reports. The difference in events reported between the hospitals in the United States and the HSOPS studies in Norway could also indicate a difference in cultural patterns. Our findings of low scores on "hospital management support in patient safety issues" may indicate that such tools as event reports and feedback on such reports should be used more extensively to motivate reports even further.

The perception of patient safety climate of our surgical environment and the hospitals in the studies previously mentioned seem to differ from those of the hospitals in the United States. Although there are minor differences in the factor structure, the variation in average positive responses of the twelve HSOPS factors, indicate differences in perceptions of the climate [4–6]. In fact, the surgical personnel in our hospital may perceive that this particular surgical environment has a lenient attitude towards patient safety climate. Low hospital management support results in low reporting of errors and a subsequent low frequency of feedback to the surgical units and personnel. This, together with few or no punitive measures, may create a low standard of patient safety as a final result.

Limitations of the study

Several questionnaires are used worldwide to measure patient safety culture or climate, including the "Patient Safety Climate in Healthcare Organizations" [39], the "Culture of Safety Survey" [40], the "Safety Attitudes Questionnaire" [41] and the Hospital Survey of Patient Safety [1, 2]. Evaluation of the psychometric properties of safety culture instruments have been performed in various ways [24, 26]. Generally, these instruments measure abstract phenomena termed factors or dimensions from self reported perceptions of safety culture or safety attitudes. Such factors are by Byrne defined as indicators of the underlying construct they are presumed to represent. The use of sound psychometric instruments is then even more critical when the items measured are presumed to represent an underlying construct or factor [42]. When interpreting patient safety climate surveys one should have this limitation in mind.

This study is carried out in a single hospital, which limits the external validity of it even though the results are quite similar to the previous Norwegian studies [6, 7]. The largest respondent group was the surgeons, who also had the lowest overall response rate (56%). Although the investigators persisted in informing the personnel about the survey, several respondents may have missed out on the information. We have not performed an analysis on nonresponders and cannot rule out the possibility of bias of variations in the mean scores between the professions. The average numbers of respondents in the studies compared varied from 37% to 56%; our overall response rate was 62%, however a response rate exceeding 70% would have been favourable [2, 4, 6, 7].