Open Access

Psychometric properties of the hospital survey on patient safety culture, HSOPSC,applied on a large Swedish health care sample

  • Mats Hedsköld1Email author,
  • Karin Pukk-Härenstam1, 2,
  • Elisabeth Berg1,
  • Marion Lindh3,
  • Michael Soop4,
  • John Øvretveit1 and
  • Magna Andreen Sachs1
BMC Health Services ResearchBMC series ¿ open, inclusive and trusted201313:332

DOI: 10.1186/1472-6963-13-332

Received: 25 February 2013

Accepted: 17 August 2013

Published: 22 August 2013

Abstract

Background

A Swedish version of the USA Agency for Healthcare Research and Quality“Hospital Survey on Patient Safety Culture” (S-HSOPSC) wasdeveloped to be used in both hospitals and primary care. Two new dimensionswith two and four questions each were added as well as one outcome measure.This paper describes this Swedish version and an assessment of itspsychometric properties which were tested on a large sample of responsesfrom personnel in both hospital and primary care.

Methods

The questionnaire was mainly administered in web form and 84215 forms werereturned (response rate 60%) between 2009 and 2011. Eleven per cent of theresponses came from primary care workers and 46% from hospital care workers.The psychometric properties were analyzed using both the total sample andthe hospital and primary care subsamples by assessment of construct validityand internal consistency. Construct validity was assessed by confirmatory(CFA) and exploratory factor (EFA) analyses and internal consistency wasestablished by Cronbachs’s α.

Results

CFA of the total, hospital and primary care samples generally showed a goodfit while the EFA pointed towards a 9-factor model in all samples instead ofthe 14-dimension S-HSOPSC instrument. Internal consistency was acceptablewith Cronbach’s α values above 0.7 in a major part of thedimensions.

Conclusions

The S-HSOPSC, consisting of 14 dimensions, 48 items and 3 single-item outcomemeasures, is used both in hospitals and in primary care settings in Swedenfor different purposes. This version of the original American instrument hasacceptable construct validity and internal consistency when tested on largedatasets of first-time responders from both hospitals and primary carecentres. One common instrument for measurements of patient safety culture inboth hospitals and primary care settings is an advantage since it enablescomparisons between sectors and assessments of national patient safetyimprovement programs. Future research into this version of the instrumentincludes comparing results from patient safety culture measurements withother outcomes in relation to safety improvement strategies.

Keywords

Patient safety culture Hospital Survey on Patient Safety Culture S-Hospital Survey on Patient Safety Culture Primary care Psychometric properties Construct validity Internal consistency

Background

Assessments of safety culture are made in many industries outside of healthcare. Insome studies, an association between high safety culture scores and unsafe behaviorand accidents has been found [1, 2].

In health care, safety culture assessments have been made for almost a decade in theUS [3, 4]. A growing number of studies report on their value and use, both in theUS and internationally [5, 6]. A recent study suggested that improvements in clinical outcomescorrelated positively with improvement in safety culture as measured by SafetyAttitude Questionnaire [7]. However, researchers have noted the need for more standardized use ofterms and a greater understanding of how safety culture as measured is related toother features of healthcare as well as the need to develop theoretical models toexplain the influence of culture on patient safety outcomes [8, 9].

The Agency for Health Care Research and Quality (AHRQ) “Hospital Survey onPatient Safety Culture” (HSOPSC) is one questionnaire instrument commonly usedin the USA [3] and increasingly used internationally, with and without modifications.There is a growing body of literature on the HSOPSC and other tools such as theSafety Attitude Questionnaire for measuring safety culture [1012]. A lack of knowledge about the validity of the factor structure ofversions of these instruments has been noted which may limit their use andusefulness [13]. Erbes et al. 2004 proposed that an important consideration forevaluating a measure is the independence of the “factors” whichstructure the instrument–that the different dimensions are relativelyindependent [14].

There have been studies which have validated the factor structure of some of thenon-US HSOPSC-based instruments: a study of the Japanese version was shown to have agood fit with the factor structure of the original instrument [15]. In a UK study, confirmatory factor analysis, however, showed a weak fitcalling for a slight remodelling of the factor structure [16]. Also, in three other European studies, confirmatory factor analysis didnot fully replicate the structure of the original instrument [5, 13, 17]. One weakness with some of the above mentioned studies is the relativelysmall sample sizes used for the factor analyses since that may influence the resultof these analyses. Nevertheless, these findings have raised questions about theapplicability of the US HSOPSC in other countries and whether an instrument forpatient safety culture measurements can be exported across national borders andhealth care systems [16]. It is possible that there are significant differences between healthcare environments which weaken the validity and usefulness of the instrument. Thesefindings suggest that this and other safety culture or safety climate instrumentsrequire careful testing before being widely used or before drawing conclusions abouttheir meaning in countries or contexts other than those for which they weredeveloped.

The purpose of this study was further to investigate certain psychometric propertiesof the Swedish version of HSOPSC in order to contribute to knowledge aboutinternational examples of the HSOPSC and to guide its use within Sweden.

Why was HSOPSC chosen for a Swedish sample?

In 2007 a national network of safety practitioners and researchers in Swedenconcluded that measurement of safety culture could contribute to improvement andunderstanding of patient safety. The Hospital Survey on Patient Safety Culture(HSOPSC) was recommended by Medical Management Centre at Karolinska Institutetbecause this instrument had undergone extensive development and testing, waswidely used in the US and because comparisons with the US could be informative.This recommendation was also supported by a study describing the AHRQ instrumentas the only patient safety culture instrument which was based on a comprehensivescale development [18]. The US HSOPSC instrument includes 12 safety dimensions and 42 items,as well as two single-item outcome questions and additional backgroundquestions. An exploratory factor analysis had been performed to explore thedimensionality of the HSOPSC [19]. This study was later repeated on a larger dataset with confirmatoryresults [20]. Also, the European Society for Quality in Healthcare in a projectfunded by the European Commission has recommended the HSOPSC as one of threeinstruments for measuring patient safety culture in European countries [12].

When the instrument was chosen for use in Sweden it was regarded likely that itwould be suitable both for hospitals and primary care centres. The Swedishhealth care system is a tax based public system organised as 21 geographicalcounty health systems which are responsible for both primary and hospitalcare.

Two years after the instrument had been introduced in Swedish healthcare itbecame a governmental requirement, linked to reimbursement, for health careorganisations to measure patient safety culture and to issue reports onimprovement strategies [21]. The Swedish version of the instrument (S-HSOPSC) is now used by allcounty councils and findings from the surveys are published in annual reports bythe Swedish Association of local Authorities and Regions and the National Boardof Health and Welfare. The psychometric properties of this version of theinstrument (S-HSOPSC) have not until now been assessed.

The aim of this paper is to describe the S-HSOPSC for use in hospitals and inprimary care settings, report the results of examining its psychometricproperties on a large sample of responses and provide recommendations forfurther development in Sweden and elsewhere.

Methods

The Swedish version of the HSOPSC

The questionnaire was translated into Swedish by a professional translator. Thetranslation was checked by four health care and patient safety experts to ensurecorrect terminology and was then back-translated by another translator and minordiscrepancies between the versions were solved by the experts and the translatorin collaboration. The other key differences from the US AHRQ instrument are, soas to meet the Swedish Patient Safety Act [22], the addition of

  • one “outcome” question about the number of riskreports submitted (17/G2)

  • four questions about “Information and support topatients and family who have suffered an adverse event”, (the dimension13, items G3, G4, G5, G6).

Further, two questions about “Information and support to staff who havebeen involved in an adverse event” (Dimension 14, items G7, G8) were alsoadded to the S-HSOPSC.

These additional questions were formulated by three of the authors (ML, MS, MAS)using the wordings in the Act (G3, G4, G5, G6) and applying the same type ofwordings for the questions about staff information and support. Finally, sincethe instrument was meant to be used not only in hospitals but also in primarycare settings, the word hospital was either omitted or exchanged for a moregeneric term (e.g. unit or organisation).

The Swedish version of the instrument thus has 14 dimensions, 48 items and threesingle-item “outcome” questions (15/E, 16/G1 and17/G2), as shown inTable 1.
Table 1

Dimensions and items of the S-HSOPSC

1

Communication openness

C2

Staff will freely speak up if they see something that maynegatively affect patient care

C4

Staff feel free to question the decisions or actions of thosewith more authority

C6r

Staff are afraid to ask questions when something does notseem right

2

Feedback and communication about error

C1

We are given feedback about changes put into place based onevent reports

C3

We are informed about errors that happen in this unit

C5

In this unit, we discuss ways to prevent errors fromhappening again

3

Frequency of error reporting

D1

When a mistake is made, but is caught and corrected beforeaffecting the patient, how often is this reported?

D2

When a mistake is made, but has no potential to harm thepatient, how often is this reported?

D3

When a mistake is made that could harm the patient, but doesnot, how often is this reported?

4

Handoffs and transitions between units and shifts

F3r

Things “fall between the cracks” whentransferring patients from one unit to another

F5r

Important patient care information is often lost during shiftchanges

F7

Problems often occur in the exchange of information acrossunits

F11

Shift changes are problematic for patients in this unit

5

Executive management support for patient safety

F1

Executive management provides a work climate that promotespatient safety

F8

The actions of executive management show that patient safetyis a top priority

F9

Executive management seems interested in patient safety onlyafter an adverse event happens

6

Nonpunitive response to error

A8

Staff feel like their mistakes are held against them

A12

When an event is reported, it feels like the person is beingwritten up, not the problem

A16

Staff worry that mistakes they make are kept in theirpersonnel file

7

Organizational learning–continuous improvement

A6

We are actively doing things to improve patient safety

A9

Mistakes have led to positive changes here

A13

After we make changes to improve patient safety, we evaluatetheir effectiveness

8

Overall perceptions of safety

A15

Patient safety is never sacrificed to get more work done

A18

Our procedures and systems are good at preventing errors fromhappening

A10

It is just by chance that more serious mistakes don´thappen around here

A17

We have patient safety problems in this unit

9

Staffing

A2

We have enough staff to handle the workload

A5

Staff in this unit work longer hours (scheduled hoursincluding overtime) than is best for patient care

A7

We use more agency/temporary staff than is best for patientcare

A14

We work in “crisis mode”, trying to do too much,too quickly

10

Supervisor/manager expectations and actions promotingsafety

B1

My supervisor/manager says a good word when he/she sees a jobdone according to established safety procedures.

B2

My supervisor/manager seriously considers staff suggestionsfor improving patient safety

B3

Whenever pressure builds up, my supervisor/manager wants usto work faster, even if it means taking shortcuts

B4

My supervisor/manager overlooks patient safety problems thathappen over and over

11

Teamwork across units

F4

There is good cooperation among units that need to worktogether

F10

Units work well together to provide the best care forpatients

F2

Units do not coordinate well with each other

F6

It is often unpleasant to work with staff from otherunits

12

Teamwork within the unit

A1

People support one another in this unit

A3

When a lot of work needs to be done quickly, we work togetheras a team to get the work done

A4

In this unit, people treat each other with respect

A11

When one area in this unit gets really busy, others helpout

13

Information and support to patients and family who havesuffered an adverse event

G3

In this unit, apologies and regrets are given to patients andfamilies who have suffered an adverse event

G4

In this unit, patients and families who have suffered anadverse event are informed about the event, its causes andactions taken to prevent it from happening again

G5

In this unit, patients and families who have suffered anadverse event, receive help and support in order to managethe situation

G6

In this unit, patients and families who have suffered anadverse event, are informed about the possibility to applyfor economic compensation from the Patient Insurance

14

Information and support to staff who have been involvedin an adverse event

G7

In this unit, staff who have been involved in an adverseevent, receive information about actions taken to preventthe event from happening again

G8

In our unit, staff who have been involved in an adverseevent, receive help and support in order to manage thesituation

15

Patient safety grade

E

Please give your unit an overall grade on patient safety

16

Number of events reported

G1

In the past 12 months, how many event reports have you filledout and submitted?

17

Number of risks reported

G2

In the past 12 months, how many risk reports have you filledout and submitted?

Ethical considerations

Ethics approval was obtained from the Regional Ethics Committee of Stockholm(Number 2010/820-31/5).

Cognitive testing and first pilot testing

Focus groups were used for initial cognitive testing of the S-HSOPSC whichinvolved asking staff about how they perceived the questions. No changes wereneeded. A pilot testing was then carried out by letting a group of doctors andnurses working in primary and hospital care (n = 78) fill out thequestionnaire. The participants also answered questions on their reactions toand thoughts about the instrument. Minor amendments were made and the finalversion was approved by a second focus group without further remarks.

Further pilot testing

A validity assessment of the factor structure of the S-HSOPSC was made bydistributing questionnaires by mail or web based to all staff in primary carecentres and hospital departments that had volunteered to participate in a pilottesting of the instrument as part of their strategic, long-term patient safetyprograms in 2008 (n = 3114). Response rate in this pilot survey was56%. About half of the returned questionnaires had all items answered, i.e.could be used for the statistical analyses. The primary care sample, thussuitable for statistical analysis, turned out to be too small for assessments ofpsychometric properties.

Due to the spread and increased use of the instrument, mainly because of thegovernmental requirement to measure patient safety culture, the database ofreturned questionnaires grew substantially. At the beginning of 2012 theresearch group received permission from the owners of the material (the countycouncils) to use the database for the purpose of testing of construct validityand internal consistency.

Sample properties and response rates

The national data base includes 84 215 questionnaires (response rate 60%),returned between 2009 and 2011 (all first-time responders), and this data basewas used in this study. Less than 6% of the responses in this data base arepaper based questionnaires, the rest is web based. All county councils exceptone have provided data to the database and all returned questionnaires are thefirst measurements of safety culture using this instrument. The county councilsuse the survey as part of their strategic patient safety improvement work anddecisions about which organizations should be included in the survey were madeby them. The authors have had no influence on who received the questionnaire.Forty six per cent of the responders represent different types of hospitalsincluding university, larger regional and smaller rural hospitals and 11% ofresponders represent primary care centres. The work area for the respondents inthis data base is shown in Figure 1 and profession inFigure 2. For further analysis, two subsets ofthe total database were extracted for analyses: hospital and primary caresamples.
https://static-content.springer.com/image/art%3A10.1186%2F1472-6963-13-332/MediaObjects/12913_2013_Article_2773_Fig1_HTML.jpg
Figure 1

Respondents’ work area.

https://static-content.springer.com/image/art%3A10.1186%2F1472-6963-13-332/MediaObjects/12913_2013_Article_2773_Fig2_HTML.jpg
Figure 2

Respondents’ profession.

Statistical analyses

For the statistical analyses only returned questionnaires with all items answeredwere used. Since the number with all items answered was large both for thecomplete sample and the two sample subsets there was no need to replace missingvalues. We calculated the Kaiser-Meyer-Olkin measure of sample adequacy (KMO) toestablish the adequacy of the sample for factor analysis [23].

Construct validity

The analysis of the construct validity, i.e. assessing the links betweenitems and relations between items and an underlying dimension, was made byperforming confirmatory factor analyses (CFA) to determine the degree of fitbetween our sample and a hypothesized measurement model [24]. The following fit measures were used: Comparative Fit Index(CFI), Goodness of Fit index (GFI), Adjusted Goodness of Fit index (AGFI),Normalized Fit Index (NFI) and Non-normalized Fit Index (also known asTucker-Lewis Index) (NNFI). These measures range from 0 (poor fit) to 1(perfect fit) and 0.9 was chosen as acceptable level of fit [25]. The measure Root Mean Square Error of Approximation (RMSEA)(limit for acceptable fit: below 0.05) was also applied. Construct validityof the S-HSOPSC was further assessed by variance tests between items usingStandardized path coefficient (limit ≥0.5) and Squared multiplecorrelations (ItemR2) (limit ≥0.3). The proportion ofcommon item variance, i.e. communalities, was calculated in order to detectcommon underlying dimensions (limit ≥0.4) [23]. Based on these results average variance extracted (AVE) andconstruct reliability (CR) for each factor were calculated in order todetermine convergent validity. Acceptable values for AVE were: ≥ 0.5and for CR: > AVE [26].

To further assess the construct validity, an exploratory factor analysis(EFA) was performed. Different techniques are available. Due to the natureof the data material where correlations between factors are allowed we chosean oblique rotation method using Promax which is a procedure designed forvery large datasets [23]. Furthermore, factor analysis (principal axis factor analysis,PAF) was used to identify factors and correlations among measured items [27]. Level for acceptable factor loading was setat ≥ 0.4 [23]. Based on the EFA, the residual correlation matrix was alsocalculated, i.e. the differences between the observed correlationcoefficients and the correlations estimated from the model (should be<0.05) [23].

Finally, correlations between the dimensions 1–14 and the outcomequestions were studied by the non-parametric Spearman-Rho correlation(0.0–0.25 little or no relationship; 0.25–0.50 fair degree ofrelationship; 0.50–0.75 moderate to good relationship; >0.75 verygood to excellent relationship) [28].

Internal consistency

Internal consistency was established by Cronbach’s α (criterion:≥0.7 for each dimension) [25]. Cronbach’s α tests were performed separately on thecomplete sample, the hospital and the primary care samples where all itemswithin each dimension under study had been answered.

Statistical analyses were performed using SPSS 19 and AMOS 19.

Results

Response rates

The total number of returned questionnaires and the number of questionnaires withall items answered are shown in Table 2.
Table 2

Number of returned questionnaires with all items answered

Sample

Returned questionnaires

All items answered

Total sample

84215

39396

Hospital care sample

38812

21099

Primary care sample

9113

3518

Generally, response rates per item were satisfactory. Lowest values were 74%(total sample) and 78% (hospital sample) for item G6 (“In this unit,patients and families who have suffered an adverse event, are informed aboutthe possibility to apply for economic compensation from the PatientInsurance”) and 60% (primary care sample) for item F11(“Shift changes are problematic for patients in thisunit”).

Construct validity

KMO was 0.95 for all three samples confirming the adequacy for factor analysis.CFA of the complete sample and the hospital and the primary care samplesgenerally showed a good fit for our Swedish 14 dimension instrument. Only AGFIwas slightly below the set margin 0.9 for the primary care sample(Table 3).
Table 3

Summary of fit indexes

 

Complete sample

Hospital care sample

Primary care sample

The comparative fit index (CFI)

0.91

0.91

0.91

The goodness of fit index (GFI)

0.92

0.92

0.90

The adjusted GFI (AGFI)

0.90

0.91

0.89

The normalized fit index (NFI)

0.91

0.91

0.90

The non-normalized fit index (NNFI)

0.90

0.90

0.90

Root Mean Square Error of Approximation (RMSEA)

0.042

0.042

0.042

Acceptable level of fit for CFI, GFI, AGFI, NFI and NNFI ≥0.9and for RMSEA ≤0.05.

Further testing for construct validity by variance tests revealed that five itemswere below the 0.3 limit in Item R2 (A5, A7, A15, F6 and F11) ofwhich items A15 (“Patient safety is never sacrificed to get more workdone”) and A7 (“We use more agency/temporary staff thanis best for patient care”) had less than 20% of their variabilityexplained by the model in all samples. These two items also dropped below the0.5 cut off in standardized path coefficient calculations in all samples.Communality values were below the 0.4 level for 13 items in the total sample,and10 and 8 items in the hospital and primary care samples, respectively. Amongthese, those with the lowest values (< 0.2) were A7 and A15 (Table 4).
Table 4

Summary of variance testing

Dimension

Item

Item R2

Std path coefficient

Communalities

  

Total sample

Hospital care sample

Primary care sample

Total sample

Hospital care sample

Primary care sample

Total sample

Hospital care sample

Primary care sample

1

C2

0.45

0.45

0.47

0.67

0.67

0.68

0.39

0.38

0.42

 

C4

0.39

0.37

0.42

0.62

0.61

0.65

0.30

0.28

0.36

 

C6r

0.42

0.39

0.45

0.64

0.63

0.67

0.38

0.35

0.43

2

C1

0.41

0.41

0.48

0.64

0.64

0.69

0.43

0.43

0.46

 

C3

0.48

0.48

0.55

0.70

0.70

0.74

0.51

0.51

0.50

 

C5

0.65

0.64

0.67

0.81

0.80

0.82

0.61

0.60

0.66

3

D1

0.73

0.73

0.77

0.86

0.85

0.88

0.72

0.71

0.77

 

D2

0.75

0.74

0.78

0.87

0.86

0.88

0.79

0.78

0.81

 

D3

0.59

0.58

0.62

0.77

0.76

0.79

0.58

0.57

0.61

4

F3r

0.49

0.44

0.54

0.70

0.67

0.73

0.50

0.49

0.51

 

F5r

0.39

0.43

0.36

0.62

0.66

0.60

0.48

0.47

0.55

 

F7r

0.53

0.51

0.53

0.73

0.72

0.73

0.53

0.53

0.50

 

F11r

0.26

0.32

0.21

0.51

0.57

0.46

0.39

0.41

0.39

5

F1

0.66

0.68

0.61

0.81

0.82

0.78

0.60

0.62

0.55

 

F8

0.70

0.71

0.63

0,83

0.84

0.79

0.63

0.64

0.56

 

F9r

0.42

0.41

0.41

0.65

0.64

0.64

0.41

0.41

0.37

6

A8r

0.54

0.55

0.53

0.74

0.74

0.73

0.48

0.50

0.49

 

A12r

0.55

0.56

0.57

0.74

0.75

0.75

0.50

0.52

0.51

 

A16r

0.41

0.40

0.41

0.64

0.63

0.64

0.43

0.42

0.43

7

A6

0.53

0.54

0.55

0.73

0.73

0.74

0.46

0.47

0.46

 

A9r

0.30

0.32

0.29

0.54

0.56

0.54

0.28

0.30

0.27

 

A13

0.42

0.42

0.43

0.64

0.65

0.66

0.38

0.38

0.40

8

A15

0.19

0.19

0.16

0.43

0.44

0.41

0.19

0.20

0.16

 

A18

0.38

0.40

0.37

0.62

0.63

0.61

0.39

0.41

0.38

 

A10r

0.53

0.54

0.48

0.73

0.73

0.69

0.51

0.53

0.48

 

A17r

0.58

0.57

0.56

0.76

0.76

0.75

0.51

0.52

0.48

9

A2

0.48

0.52

0.50

0.69

0.72

0.71

0.45

0.46

0.46

 

A5r

0.25

0.24

0.21

0.50

0.49

0.45

0.26

0.24

0.23

 

A7r

0.15

0.15

0.14

0.39

0.39

0.37

0.18

0.18

0.18

 

A14r

0.59

0.57

0.61

0.77

0.75

0.78

0.52

0.51

0.53

10

B1

0.47

0.47

0.51

0.69

0.69

0.71

0.52

0.51

0.55

 

B2

0.68

0.69

0.66

0.82

0.83

0.82

0.72

0.73

0.66

 

B3r

0.33

0.35

0.32

0.58

0.59

0.57

0.39

0.42

0.40

 

B4r

0.52

0.53

0.49

0.72

0.73

0.70

0.53

0.55

0.50

11

F4

0.49

0.48

0.48

0.70

0.69

0.70

0.44

0.41

0.48

 

F10

0.48

0.48

0.46

0.69

0.69

0.68

0.44

0.43

0.47

 

F2r

0.45

0.45

0.44

0.67

0.67

0.66

0.45

0.45

0.44

 

F6r

0.23

0.20

0.30

0.51

0.49

0.55

0.28

0.24

0.41

12

A1

0.56

0.54

0.55

0.75

0.73

0.74

0.55

0.56

0.55

 

A3

0.43

0.43

0.52

0.65

0.66

0.72

0.49

0.45

0.57

 

A4

0.54

0.52

0.56

0.73

0.72

0.75

0.52

0.53

0.53

 

A11

0.31

0.31

0.43

0.56

0.56

0.66

0.34

0.32

0.46

13

G3

0.54

0.53

0.55

0.73

0.73

0.74

0.50

0.50

0.51

 

G4

0.70

0.69

0.72

0.84

0.83

0.85

0.68

0.67

0.69

 

G5

0.58

0.59

0.60

0.76

0.77

0.77

0.61

0.62

0.63

 

G6

0.41

0.42

0.46

0.64

0.65

0.68

0.42

0.43

0.46

14

G7

0.68

0.68

0.71

0.82

0.82

0.68

0.54

0.53

0.60

 

G8

0.59

0.59

0.65

0.77

0.77

0.81

0.47

0.46

0.56

Limits for standardized path coefficient ≥0.5 and for ItemR2 ≥0.3.

AVE showed values below the 0.5 level for almost half of all dimensions withlowest values for dimension 8 “Overall perceptions ofsafety” and 9 “Staffing” in all samples. CRvalues, however, were above the AVE values in all dimensions (Table 5).
Table 5

Summary of average variance extracted (AVE) and construct reliability(CR)

 

Total sample

Hospital care sample

Primary care sample

Dimension

AVE

CR

AVE

CR

AVE

CR

1

0.42

0.68

0.40

0.67

0.44

0.71

2

0.51

0.76

0.51

0.76

0.57

0.80

3

0.69

0.87

0.68

0.87

0.72

0.89

4

0.42

0.60

0.43

0.62

0.41

0.57

5

0.59

0.81

0.60

0.81

0.55

0.78

6

0.50

0.75

0.50

0.75

0.50

0.75

7

0.41

0.68

0.42

0.69

0.43

0.69

8

0.37

0.74

0.38

0.75

0.35

0.73

9

0.37

0.52

0.37

0.51

0.36

0.50

10

0.50

0.69

0.51

0.70

0.50

0.68

11

0.41

0.59

0.40

0.58

0.42

0.60

12

0.46

0.63

0.45

0.63

0.52

0.70

13

0.56

0.74

0.56

0.74

0.58

0.76

14

0.63

0.87

0.64

0.87

0.68

0.89

Acceptable levels for AVE ≥0.5 and forCR ≥ AVE.

Results of EFA by using principal axis factoring (PAF) as extraction method andPromax as rotation method are presented in Additional file 1: Appendix 1. The EFA indicated 9 factors in all three samples incontrast to the 14 dimensions of the instrument. The factors jointly explained56.4% of the total variance of all the items. Dimension 7“Organizational learning–continuous improvement”and dimension 2 “Feedback and communication about error”and two of three items from dimension 1 “Communicationopenness” as well as one item from dimension 8 “Overallperceptions of safety” all loaded onto factor 1. The remainingitems from dimension 8 loaded onto dimension 9 “Staffing”.The new Swedish dimensions 13 “Information and support to patients andfamily who had suffered an adverse event” and 14“Information and support to staff who have been involved in anadverse event” loaded together as did dimension 5“Executive management support for patient safety” anddimension 11 “Teamwork across units”.

Four items showed overall factor loading below 0.4, and so did one more item inthe hospital care sample and additional two more items in the total sample. Forall three samples these were C4 “Staff feel free to question thedecisions or actions of those with more authority”, C6“Staff are afraid to ask questions when something does not seemright”, A15 “Patient safety is never sacrificed to getmore work done” and A7 “We use more agency/temporarystaff than is best for patient care”. In addition, in both totaland hospital samples A18 “Our procedures and systems are good atpreventing errors from happening” and finally in the total sampleF10 “Units work well together to provide the best care forpatients” were below 0.4. Three items in all samples loaded ontoanother factor than the S-HSOPSC dimensions (1/C6, 8/A18 and 11/F6) and in theprimary care sample two more items in dimension 4 (4/F5 and 4/F11). Nine itemsfrom four dimensions loaded onto factor 1 and 7 items from two dimensions loadedonto factor 2 (Additional file 1: Appendix 1).

The non-redundant residuals were 25 (2.0%) with an absolute value above 0.05. TheSpearman-Rho correlation, based on the total sample, revealed a fair to gooddegree of relationship between most of the dimensions and also with the outcomequestion 15/E. There was no correlation between the two other outcome questions“Number of events reported” (16/G1) and“Number of risks reported” (17/G2) and the dimensions1–14 (Table 6).
Table 6

Correlation between dimensions and single-outcome questions bynon-parametric Spearman-Rho method

 

Dim1

Dim2

Dim3

Dim4

Dim5

Dim6

Dim7

Dim8

Dim9

Dim10

Dim11

Dim12

Dim13

Dim14

E1

G1

Dim2

.533**

               

Dim3

.331**

.446**

              

Dim4

.329**

.323**

.253**

             

Dim5

.318**

.380**

.315**

.376**

            

Dim6

.463**

.384**

.242**

.334**

.329**

           

Dim7

.422**

.567**

.404**

.304**

.429**

.348**

          

Dim8

.432**

.429**

.339**

.397**

.475**

.484**

.480**

         

Dim9

.304**

.274**

.164**

.309**

.392**

.441**

.281**

.536**

        

Dim10

.499**

.522**

.332**

.323**

.391**

.463**

.492**

.485**

.387**

       

Dim11

.331**

.344**

.268**

.605**

.512**

.316**

.358**

.421**

.315**

.342**

      

Dim12

.465**

.424**

.276**

.338**

.278**

.412**

.452**

.422**

.339**

.431**

.344**

     

Dim13

.346**

.379**

.339**

.268**

.312**

.243**

.407**

.333**

.186**

.343**

.311**

.308**

    

Dim14

.377**

.465**

.364**

.273**

.363**

.289**

.455**

.372**

.229**

.388**

.327**

.322**

.754**

   

E

.417**

.455**

.400**

.364**

.463**

.384**

.487**

.659**

.436**

.462**

.404**

.412**

.345**

.381**

  

G1

.023**

.070**

.060**

-.030**

-.070**

.047**

.030**

-.084**

-.055**

-.005

-.058**

.023**

-.065**

-.023**

.080**

 

G2

.024**

.061**

.102**

-.015**

-.037**

.019**

.063**

-.081**

-.058**

-.001

-.030**

.020**

.014**

.031**

.069**

.387**

**Correlation is significant at the 0.01 level (2-tailed).

0.0–0.25 little or no relationship; 0.25–0.50 fair degreeof relationship; 0.50–0.75 moderate to good relationship;>0.75 very good to excellent relationship.

Internal consistency

Results of internal consistency analysis are presented in Table 7. Of the 14 groupings of items into dimensions, 2dimensions, i.e. dimensions 7 “Organizationallearning–continuous improvement” and 9“Staffing” in all samples and dimension 1“Communication openness” in the total and hospital caresamples fell short of an adequate level of internal consistency, i.e. were below0.7.
Table 7

Internal consistency by test of Cronbach’s α

 

Dimension

Total sample

Hospital care sample

Primary care sample

1

Communication openness

0.68

0.67

0.70

2

Feedback and communication about error

0.76

0.76

0.80

3

Frequency of error reporting

0.87

0.87

0.88

4

Handoffs and transitions between units and shifts

0.74

0.75

0.73

5

Executive management support for patient safety

0.81

0.81

0.79

6

Nonpunitive response to error

0.74

0.74

0.75

7

Organizational learning–continuous improvement

0.66

0.66

0.68

8

Overall perceptions of safety

0.71

0.72

0.69

9

Staffing

0.67

0.67

0.64

10

Supervisor/manager expectations and actions promotingsafety

0.78

0.79

0.79

11

Teamwork across units

0.72

0.71

0.74

12

Teamwork within the unit

0.76

0.75

0.80

13

Information and support to patients and family who havesuffered an adverse event

0.83

0.83

0.84

14

Information and support to staff who have been involved in anadverse event

0.77

0.77

0.81

Criterion ≥0.7 for each dimension.

Discussion

In this study on psychometric properties of the Swedish version of theAHRQ-instrument (S-HSOPSC) for measurement of patient safety culture, a databasecontaining over 80 000 questionnaires from all sectors of Swedish health care, wasused. To our knowledge, a database of a similar size has only been used by Sorra andDyer in their 2010 examination of the multilevel psychometric properties of theoriginal instrument issued in 2004 [20]. In our study, psychometric tests were performed on the total sample andon two subsamples: the hospital and primary care samples.

The exploratory factor analysis pointed towards a 9-factor model for the total sampleand both subsamples in contrast with the 14 dimensions of the instrument. However,confirmatory factor analysis generally showed a good fit between our data in allsamples and the 14 dimension instrument with only 2 items, i.e. “We usemore agency/temporary staff than is best for patient care” and“Patient safety is never sacrificed to get more work done”having less than 20% of their variability explained by the model. Also, there wassatisfactory convergent validity and a fair to good degree of relationship betweenall dimensions and the single-item outcome measure “Patient safetygrade”.

Internal consistency was generally good in all samples with lowest Cronbach´sα values for “Communication openness” (total and hospitalsample), “Organizational learning–continuous improvement”(all samples) and “Staffing” (all samples).

Other researchers have reached similar results: low factor loadings for items“We use more agency/temporary staff than is best for patientcare”[16] and “Patient safety is never sacrificed to get more workdone”[17]. The latter item was excluded from the Dutch version of the instrument [17]. In contrast with our results, the confirmatory factor analysis carriedout on a UK sample by Waterson et al. [16] showed a poor fit and an optimal nine dimension-model was constructedinstead [16]. In other studies weak internal consistency has also been demonstratedfor the same dimensions as in our study: “Organizationallearning–continuous improvement”[13, 16, 17], and “Staffing”[13, 16, 17] Further studies are needed to investigate the possible linkage betweencertain dimensions and items.

At present, our opinion is that these items and dimensions should be kept since theysignify important aspects of patient safety and as such form a useful foundation forimprovement work.

Overall, the psychometric properties of the S-HSOPSC proved satisfactory and there issolid evidence for the 14 dimensions and 48 items of the S-HSOPSC. Thus, at presentno changes will be made to the instrument. Also, there is a general wish in Swedishhealth care to keep the instrument as close as possible to the original AHRQversion. These decisions will be reconsidered when repeating the analysis of thepsychometric properties of the S-HSOPSC on a dataset of second-time respondentswhich is now accumulating. The reason for repeating the analysis is that when thedatabase used in this study was collected, the patient safety movement in Sweden wasin its beginning and the general awareness of basic concepts of patient safety wasprobably low for many of the respondents of this questionnaire. This may haveaffected how the questions were understood and answered.

The instrument was originally designed for hospitals but with few changes of wordingsit is also in use within primary care in Sweden. To our knowledge, the use of theAHRQ instrument in primary care has only been reported in Turkey [29] and The Netherlands [30]. Some aspects of patient safety may not be as relevant for some primarycare offices as for hospital units, such as questions about handoffs, teamworkacross units and executive management support. Because of the absence of apossibility to give a “not applicable” reply, such items might be leftunanswered which in turn renders a lower response rate for these items as was thecase with the item “Shift changes are problematic for patients in thisunit” in our primary care sample. Another drawback with modificationof words, so as to suit both sectors of the system, may be the risk of a decrease inaccuracy of the question. The S-HSOPSC form starts with explanations regarding theinterpretation of certain of the words used in the survey to minimize this risk.

There are great advantages with one common instrument for patient safety culturemeasurements in both the hospital and the primary care sector. Not only does itsimplify measurements within the health care system which in Sweden includes primaryhealth care–the entrance to the system–and hospital care but it alsoprovides opportunities for comparisons and learning within the system andassessments of national programs for quality and patient safety improvement. On theother hand, by adapting an instrument designed for hospital care for use in primarycare, important aspects on patient safety in this sector of the health care system,might not be captured. This remains to be further studied.

Also, the criterion related validity of the S-HSOPSC needs to be explored bycomparing the results from safety culture measurements with other outcome measuresin relation to patient safety improvement strategies over time [18]. Future research will also consider how information from measurements ofpatient safety culture is used by Swedish health care organizations and units.

Limitations to the findings of our study are at least twofold. First, the sample usedfor validation of psychometric properties was a subset of the whole sample with allitems answered. It has been shown that responders who have all items answered aremainly those with direct patient interaction [13]. Thus, the factor analyses may mainly build on responses from staffworking with direct patient contact and the material not being representative forall staff members. Also, primary care responders may more often leave questionsunanswered if the item is not relevant to them. Secondly, proving correlationbetween an within-method outcome measure like self-estimated grade and thedimensions of the instrument has been questioned [9].

Conclusions

The Swedish version of HSOPSC, the S-HSOPSC, consisting of 14 dimensions, 48 itemsand 3 single-item outcome measures, is widely used both in hospitals and in primarycare settings. The assessment of its construct validity and internal consistency ina large dataset of first-time responders which is reported in this paper showedacceptable results both for the total sample and for the two subsamples: thehospital and primary care samples. This study suggests that the instrument can beused in both hospital and primary care settings after minor adjustments of wordings.There are advantages to one common instrument for measurements of patient safetyculture as it allows comparisons within the health care system and assessments ofnational patient safety improvement programs. The S-HSOPSC needs to be validated asa performance measurement tool by comparing the results from safety culturemeasurements with other outcome measurements over time and confirming its usefulnessas a tool for patient safety improvement work in Swedish health care.

Abbreviations

AHRQ: 

Agency for Healthcare Research and Quality

HSOPSC: 

Hospital Survey on PatientSafety Culture

S-HSOPSC: 

Swedish version of Hospital Survey on Patient SafetyCulture

EFA: 

Exploratory factor analysis

CFA: 

Confirmatory factor analysis

CFI: 

Comparative fit index

GFI: 

Goodness of fit index

AGFI: 

Adjusted goodness of fitindex

NFI: 

Normalized fit index

NNFI: 

Non-normalized fit index

RMSEA: 

Root meansquare error of approximation

AVE: 

Average variance extracted

CR: 

Constructreliability.

Declarations

Acknowledgements

We would like to thank Jessica Lindberg and Thomas Emilsson, Public HealthcareServices Committee Administration, Stockholm County Council, Stockholm, Swedenfor carrying out the initial pilot test.

Authors’ Affiliations

(1)
Medical Management Centre, Karolinska Institutet
(2)
Astrid Lindgren´s Childrens´ Hospital, Karolinska University Hospital
(3)
Public Healthcare Services Committee Administration, Stockholm County Council
(4)
National Board of Health and Welfare

References

  1. Christian MS, Bradley JC, Wallace JC, Burke MJ: Workplace safety: a meta-analysis of the roles of person and situationfactors. J Appl Psychol. 2009, 94 (5): 1103-1127.PubMedView Article
  2. Clarke S: An integrative model of safety climate: linking psychological climate andwork attitudes to individual safety outcomes using meta-analysis. J Occup Organ Psych. 2010, 83 (3): 553-578. 10.1348/096317909X452122.View Article
  3. Agency for Healthcare Research and Quality: Hospital survey on patient safety culture. http://​www.​ahrq.​gov/​qual/​patientsafetycul​ture/​usergd.​htm.
  4. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ: The Safety Attitudes Questionnaire: psychometric properties, benchmarkingdata, and emerging research. BMC Health Serv Res. 2006, 6: 44-10.1186/1472-6963-6-44.PubMedPubMed CentralView Article
  5. Hellings J, Schrooten W, Klazinga N, Vleugels A: Challenging patient safety culture: survey results. Int J Health Care Qual Assur. 2007, 20 (7): 620-632. 10.1108/09526860710822752.PubMedView Article
  6. Mardon RE KK, Sorra J, Dyer N, Famolaro T: Exploring relationship between hospital survey on patient safety culture andadverse events. J Patient Saf. 2010, 6: 7.View Article
  7. Vigorito MC, McNicoll L, Adams L, Sexton B: Improving safety culture results in Rhode Island ICUs: lessons learned fromthe development of action-oriented plans. Joint Comm J Qual Patient Saf. 2011, 37 (11): 509-514.
  8. Halligan M, Zecevic A: Safety culture in healthcare: a review of concepts, dimensions, measures andprogress. BMJ Qual Saf. 2011, 20 (4): 338-343. 10.1136/bmjqs.2010.040964.PubMedView Article
  9. Jackson J, Sarac C, Flin R: Hospital safety climate surveys: measurement issues. Curr Opin Crit Care. 2010, 16: 632-638. 10.1097/MCC.0b013e32833f0ee6.PubMedView Article
  10. Agency for Healthcare Research and Quality: Surveys on patient safety culture research reference list. 2012, Rockville, USA: Agency for Healthcare Research and Quality, [http://​www.​ahrq.​gov/​qual/​patientsafetycul​ture/​sopsrefs.​htm]
  11. Health Foundation: Research scan; measuring safety culture. [http://​www.​health.​org.​uk/​public/​cms/​75/​76/​313/​2600/​Measuringsafetyc​ulture.​pdf?​realName=​rclb4B.​pdf].
  12. European Union Network for Patient Safety: Patient safety culture instruments used in member states. Vol. 2010. 2010, DK-8200 Aarhus, Denmark: European Society for Quality inHealthcare-Office for Quality Indicators, ISBN 978‒87‒993779‒1‒6.[http://​ns208606.​ovh.​net/​~extranet/​images/​EUNetPaS_​Publications/​eunetpas-catalogue-use-of-psci-in-ms-april-13-2010.​pdf].
  13. Pfeiffer Y, Manser T: Development of the German version of the Hospital Survey on Patient SafetyCulture: Dimensionality and psychometric properties. Saf Sci. 2010, 48 (10): 1452-1462. 10.1016/j.ssci.2010.07.002.View Article
  14. Erbes C, Polusny MA, Billig J, Mylan M, McGuire K, Isenhart C, Olson D: Developing and applying a systematic process for evaluation of clinicaloutcome assessment instruments. Psychol Serv. 2004, 1 (1): 31-39.View Article
  15. Ito S, Seto K, Kigawa M, Fujita S, Hasegawa T: Development and applicability of Hospital Survey on Patient Safety Culture(HSOPS) in Japan. BMC Health Serv Res. 2011, 11: 28-10.1186/1472-6963-11-28.PubMedPubMed CentralView Article
  16. Waterson P, Griffiths P, Stride C, Murphy J, Hignett S: Psychometric properties of the Hospital Survey on Patient Safety Culture:findings from the UK. Qual Saf Health Care. 2010, 19 (5): e2-10.1136/qshc.2008.031625.PubMed
  17. Smits M, Christiaans-Dingelhoff I, Wagner C, Wal G, Groenewegen PP: The psychometric properties of the 'Hospital Survey on Patient SafetyCulture' in Dutch hospitals. BMC Health Serv Res. 2008, 8: 230-10.1186/1472-6963-8-230.PubMedPubMed CentralView Article
  18. Flin R, Burns C, Mearns K, Yule S, Robertson EM: Measuring safety climate in health care. Qual Saf Health Care. 2006, 15 (2): 109-115. 10.1136/qshc.2005.014761.PubMedPubMed CentralView Article
  19. Sorra JS, Nieva VF: Safety culture assessment; a tool for improving patient safety in healthcareorganizations. Qual Saf Health Care. 2003, 12: ii17-ii23.PubMedPubMed Central
  20. Sorra JS, Dyer N: Multilevel psychometric properties of the AHRQ hospital survey on patientsafety culture. BMC Health Serv Res. 2010, 10: 199-10.1186/1472-6963-10-199.PubMedPubMed CentralView Article
  21. Sveriges Kommuner och Landsting: Överenskommelsen om patientsäkerhet. 2013, [http://​www.​skl.​se/​vi_​arbetar_​med/​halsaochvard/​patientsakerhet/​overenskommelse-patientsakerhet-2013]
  22. Svensk Författningssamling: Patientsäkerhetslagen SFS 2010:659. 2010, [http://​rkrattsdb.​gov.​se/​SFSdoc/​10/​100659.​PDF]
  23. Field A: Discovering statistics using SPSS. 2009, London: SAGE Publications Ltd, 3
  24. Thompson B: Exploratory and confirmatory factor analysis: understanding concepts andapplications. 2004, Washington, D.C; London: American Psychological Association;Eurospan, 1.View Article
  25. Hair J: Multivariate data analysis. 1998, New Jersey, USA: Prentice-Hall Inc, 5
  26. Paswan A: Confirmatory factor analysis and structural equations modeling: anintroduction. [CFA-SEM–Intro-May 18 2009.ppt–University of North Texas][http://​www.​cob.​unt.​edu/​slides/​paswan/​BUSI6280/​CFA-SEM%20​-%20​Intro-May%20​18%20​2009.​ppt.
  27. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ: Evaluating the use of exploratory factor analysis in psychologicalresearch. Psychol Methods. 1999, 4 (3): 272-299.View Article
  28. Colton T: Statistics in medicine. 1974, Boston: Little, Brown and Company, 1
  29. Bodur S, Filiz E: A survey on patient safety culture in primary healthcare services inTurkey. Int J Qual Health Care. 2009, 21 (5): 348-355. 10.1093/intqhc/mzp035.PubMedView Article
  30. Zwart DL, Langelaan M, van de Vooren RC, Kuyvenhoven MM, Kalkman CJ, Verheij TJ, Wagner C: Patient safety culture measurement in general practice. Clinimetricproperties of 'SCOPE'. BMC Fam Pract. 2011, 12: 117-10.1186/1471-2296-12-117.PubMedPubMed CentralView Article
  31. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://​www.​biomedcentral.​com/​1472-6963/​13/​332/​prepub

Copyright

© Hedsköld et al.; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://​creativecommons.​org/​licenses/​by/​2.​0), whichpermits unrestricted use, distribution, and reproduction in any medium, provided theoriginal work is properly cited.