- Research article
- Open Access
Safety culture in the primary health care settings based on workers with a leadership role: the psychometric properties of the Slovenian-language version of the safety attitudes questionnaire – short form
BMC Health Services Research volume 18, Article number: 767 (2018)
Safety culture describes leader and staff interactions, attitudes, routines, awareness, and practices within an organisation. With this study, we aimed to determine the psychometric properties of the Slovenian-language version of the Safety Attitudes Questionnaire (SAQ) – Short Form in primary health care settings.
This was a cross-sectional study in the largest primary health care in Slovenia. We invited all employees with a leadership role to participate in the study (N = 211). We used the Slovenian-language version of the SAQ – Short Form.
There were 154 participants in the final sample (73.0% response rate), of which 136 (88.3%) were women. The mean age of the sample was 46.2 ± 10.0 years. Exploratory factor analysis put forward six factors: 1) Perceptions of Management; 2) Stress recognition; 3) Teamwork Climate; 4) Communication; 5) Safety Climate; 6) Working Conditions and Satisfaction. This model explained 61.7% of the variance of the safety culture in the primary health care setting. The reliability of the whole scale and of the six factors, assessed using Cronbach’s alpha, was all above 0.78.
The results of our study suggests that the Slovenian-language version of the SAQ – Short Form with six factors could be a reliable and valid tool for measuring the safety culture in the primary health care workers with leadership role In Slovenia. The Slovenian version differed from the original SAQ – Short Form and the majority of other translated versions. Also, the data was from one health centre only and therefore we cannot draw strong conclusions on its external validity.
Patient safety is an increasingly important part of quality assurance in health care. Within patient safety, a concept of a safety culture has developed which describes leader and staff interactions, attitudes, routines, awareness, and practices within an organisation . As safety culture is a group, and mainly an organisational phenomenon, it differs across the different levels and organisations of health care [2, 3], so appropriate tools should be employed in order to capture the true state of affairs. Safety culture differs from safety climate the latter referring to the measurable components of safety culture, a kind of a snapshot of safety culture at the given moment in time .
Several tools have been developed to measure the safety attitudes of healthcare providers [5,6,7]. Recent reviews of the Safety Climate Surveys in health care settings showed up to 17 [7, 8] available surveys, however only three were developed for a primary health care settings [7, 8]. Another systematic review on tools for safety culture in primary care  put forward the Safety Attitudes Questionnaire (SAQ)  as one of the widely used and also one of the most appropriate instruments for assessing patient safety culture. In 2007, the SAQ was adapted to outpatient (primary healthcare) settings . This adapted version, the SAQ – Ambulatory Version (SAQ-AV), was found to be a reliable tool for comparing attitudes across different professional groups of healthcare providers outside hospitals [5, 10, 11]. Its shorter version, the SAO Short Form is being increasingly used in many countries [9, 12,13,14,15,16,17,18].
In Slovenia, quality and safety at the primary health care level have been the focus of study over the past several years [19,20,21,22]. Patient safety features in primary healthcare have been investigated through a study on the Quality and Costs of Primary Care in Europe (QUALICOPC), dealing with the organisation and accessibility of primary healthcare services . In addition, safety culture has been measured by the Slovenian-language version of the SAQ-AV in out-of-hours primary health care settings [24,25,26]. Out-of-hours settings in Slovenia differ from primary healthcare centres in terms of organisation, location, and staff. Out-of-hours health services are combined with emergency medical services and are available both at the same place and time. Sometimes, they are located in hospitals and sometimes in the healthcare centres. Professionals that work in OOHC are family physicians, emergency physicians, and emergency nurses. Sometimes, there are also laboratory technicians and radiology technicians. Usually, emergency nurses work only in OOHC settings while family physicians work in their practice and in OOHC on the basis of rotation. An out-of-hours health centre has thus a unified leadership and can be seen as a “natural social unit”; which is a validation criteria for organizational climate measurements .
To date, the safety culture in primary health care settings such as primary health care centres has not been measured in Slovenia. Also, the SAQ – Short Form has not yet been validated in the Slovenian language.
Within this study we wanted to fulfil the following aims: 1) to test the reliability and validity of the Slovenian-language version of SAQ – Short Form in primary health care settings, and 2) to determine the factor structure of the Slovenian-language version of SAQ– Short Form in primary health care settings.
Research design and setting
We performed a cross-sectional study in the largest community health centre in Slovenia. This health centre provides healthcare services for the municipality of Ljubljana, which comprises around 280,000 people. It is divided into eight units, which are located in separate buildings in different parts of Ljubljana. It employs around 1500 employees of different medical and non-medical backgrounds.
We invited all the employees with a leadership role to participate in the study (N = 211). Employees with a leadership role come from different professional backgrounds (i.e. physicians, dentists, registered nurses, nurse assistants, administrative staff etc.). They are appointed to be leaders of different units within the health centre, such as chief of nurses, chief of physicians, chief of whole units, director of health centre etc. They work mostly within their professional fields, but have a certain amount of their working time dedicated to their leadership tasks. We thought that a homogenous group such as workers with a leadership role would be appropriate for a validation study as they are the ones responsible for the safety culture and their role in creating a proper safety environment is crucial.
We used the SAQ – Short Form  which consists of 36 items. Each item of the questionnaire is answered on a 5-point Likert scale by which the respondents indicate their level of agreement with the statement ranging from “disagree strongly” to “agree strongly”). In the analysis, the scores of negatively worded items were reversed so that higher scores always indicated a more positive evaluation of the safety culture. There are six factors in the original SAQ – Short Form: Teamwork Climate (items 1–6), Safety Climate (items 7–13), Job Satisfaction (items 15–19) Stress Recognition (items 20–23), Perceptions of Management (items 24–28), and Working Conditions (items 29–32). Items 14 and 33–36 are not included in any of the factors.
We were granted permission to use this questionnaire by the University of Texas at Houston-Memorial Hermann, Center for Health care Quality and Safety. The permission was given on June 3, 2016.
We also collected data on demographic characteristics (gender, age, role, work experience, working hours, and location of work).
The data was collected through an electronic survey. The link to the survey was sent to the email addresses of the participants in February 2017. The first reminder was sent after two weeks, and the second two weeks after the first. Participation was confidential, possible identifiers such as e-mail and IP addresses were removed by the administrative coordinator in the project. It was not possible for the researchers to link the participants to their responses.
Variability of the data set
We determined the skewness and kurtosis of the data set. Skewness was 0.210 and kurtosis was 0.332. We concluded that for a given sample size, the measures of the variability indicated a normal distribution of the data .
Confirmatory factor analysis
We performed a confirmatory factor analysis (CFA) of the SAQ – Short Form. We used the items that were part of the original factors (31 items). The following indices for the good CFA model fit were considered: 1) the chi-square goodness-of-fit: the model is acceptable if the p-value of chi-square is not significant, 2) relative chi-square, which is the chi-square divided by degrees of freedom (df), should range up to 3 ; 3) Comparative Fit Index (CFI) should lie within 0.90–1.00 for a fit model; 4) the Normed Fit Index (NFI) should range between 0 and 1 with a value of 0.90 or greater indicating of a good model fit; and 5) Root Mean Square Error of Approximation (RMSEA): a value of about 0.05 or less would indicate a close fit of the model in relation to df .
Exploratory factor analysis
Our CFA model did not provide a good fit and therefore the explorative factor analysis (EFA) was carried out with an Oblimin rotation (with Kaiser normalisation) on 31 items as in the original factor model . As the results was not acceptable, we decided to perform the EFA on all 36 items in the questionnaire. This yielded an acceptable result with good share of explained variance. A cut-off point for factors loadings was 0.300. We also performed a Kaiser-Meyer-Olkin analysis and a Bartlett test (both measures test how suited is the data for factor analysis). We determined that the Kaiser-Meyer-Olkin measure should be < 0.8 and that the Bartlett test should be < 0.001 . For each of the six factors, the scale scores were calculated by obtaining the mean of the item scores within one factor. Next, correlations between the scale scores were calculated to determine construct validity. Reliability (internal consistency) was assessed using the Cronbach’s alpha. Values of Cronbach’s alpha over 0.7 are considered acceptable, over 0.8 good, and over 0.9 excellent .
There were 154 participants in the final sample (73.0% response rate), of which 136 (88.3%) were women. Most of the sample consisted of physicians and registere nurses. All health centres’ units were represented in the sample (Table 1). The mean age of the sample was 46.2 ± 10.0 years; the mean time in current post was 13.6 ± 10.0 years, and the mean number of weekly working hours was 36.2 ± 10.4. The mean number of years of clinical experiences participants was 21.9 ± 10.1 years.
Confirmatory factor analysis
CFA model on the original six factors with the appropriate items (see Methods section) did not provide a good model fit on all test. P value was < 0.001, the relative chi-square was 1.636, CFI was 0.874, NFI was 0.737, and RMSEA was 0.064.
The on 36 items put forward six factors: 1) Perceptions of Management; 2) Stress recognition; 3) Teamwork Climate; 4) Communication; 5) Safety Climate; 6) Working Conditions and Satisfaction (Table 2). This model explained 61.7% of the variance. The Kaiser-Meyer-Olkin measure was 0.824 and the Bartlett test was significant (p < 0.001).
There was a significant difference in items belonging to the factors of the original scale and of our scale. Except for the factor Stress Recognition, all factors in our factor model had partially or completely different items (Table 3).
Reliability of the SAQ – Short form and its factors
The reliability of the whole scale, through Cronbach’s alpha, was 0.963 and the reliability of the individual factors was also good (Table 2).
All the factors except Stress recognition were significantly correlated with each other (Table 4), while Stress recognition was correlated just with the Communication.
The Slovenian-language version of the SAQ – Short Form used with the employees of the largest primary health care centre in Slovenia did not fit perfectly to the original factor structure . Our new factor model revealed six factors: 1) Perceptions of Management; 2) Stress recognition; 3) Teamwork Climate; 4) Communication; 5) Safety Climate; 6) Working Conditions and Satisfaction. Cronbach’s alphas of the factors were good, indicating no problems with the factors. The six-factor model of the Slovenian SAQ – Short Form covered 36 items.
Our factor structure differs from the original one  in the context of some factors. Factors Teamwork Climate, Perceptions of Management, Working Conditions, Safety Climate, and Stress Recognition are the same as in the original factor structure . However, the items belonging to these factors differ (see Table 3). Our model also revealed one new factor (Communication) and one partly new factor (Working Conditions and Satisfaction). The latter is actually a combination of two factors from the original SAQ – Short Form scale. The factor Communication has not been revealed within the original scale but has already been revealed as an individual factor of the Slovenian-language version of the SAQ-AV . Communication has also been recognised in other safety culture measurement tools as an important domain within the safety climate or culture [7, 10, 33].
Some items have negative loadings (all items in factors Teamwork Climate and Working Conditions and Satisfaction). This indicates that these factors are negatively correlated to the construct. It is a general agreement that the strength of a loading needs to be measured by its absolute value with the possibility that preconceptions about appropriate reverse coding might need modification based on the signs of factor loadings . Therefore, we decided to keep the original wording of the items.
All the factors measured the same construct except for Stress Recognition. This could be due to the fact that other factors related more to work and the working environment, and Stress Recognition more to personality traits. Stress Recognition was weakly correlated with Communication, which indicates that if the working environment stimulates communication about safety issues, the workers feel that they manage stress better.
The validation studies on SAQ – Short Form in Norwegian (, Italian , Portuguese , Chinese , Swedish [16, 17], and German  language confirmed the original factor structure. However, they were carried out in the secondary health care level which makes it difficult to compare the results of our study to them as our study was done in primary health care settings. Our result of a CFA not perfectly fitting to the original factor model indicates that there are differences in the safety culture between primary and secondary health care levels which has already been pointed out elsewhere [6, 33]. Namely, primary health care differs from hospitals in terms of organisational structure, administrative and clinical processes and the reasons for encounters . However, it could also indicate that the tool used (the SAQ – Short Form) was not specific enough for the primary health care settings and another tool could be used.
The safety climate and culture research in health care in general has been predominately conducted within acute care settings and hospital settings  therefore the existing measurements in primary health care settings are relatively new which makes our study an important one for this field of research.
The differences in factor structures between countries make cross-country comparisons of patient safety culture challenging. The structural differences may reflect cross-national variation in the nature and structure of primary care, or it may mean that item wordings trigger different connotations in different languages. For these reasons a wise option might be to compare countries at the item level rather than at the factor level .
The differences in the safety culture between the different levels of health care require different tools for measurements that would be developed for primary care or hospital safety culture measurement tools adapted for the use at the primary health care level. Since there is currently no consensus on which tool is the best to use , further research should focus on performing the \adaptations and validations of the existing tools for use at the primary health care level, with special emphasis on establishing the construct and criterion validity . Another option would also be a development of a new scale which would be based on the theoretical model of the safety culture at the primary health care level.
Our study has some limitations that have to be mentioned. We are aware of the fact that an EFA always produces a solution, but does not assess the risk that the EFA solution only describes the data set, and it may not be possible to generalise. Due to this fact, the results of our study may not be generalised to the whole population. The sample size in our study was limited to employees with a leadership role, which could produce a selection bias and therefore limits the validity of our results. However, the results were very similar to our previous study  and also to the Dutch study  (both performed at the primary health care level) and this gives us confidence in the validity of our results. Although the response rate was high, there is no information on the characteristics of the non-respondents, so this could also be a source of bias.
We used the SAQ – Short Form for the primary health care settings despite the fact that there are other specific primary health care tools available [6, 7] and might have been a better tool for our study. This is another limitation of our study.
The results of our study suggests that the Slovenian-language version of the SAQ – Short Form with six factors could be a reliable and valid tool for measuring the safety culture in the primary health care workers with leadership role. The factors and item loadings in our study differed from the original SAQ – Short Form and the majority of other translated versions.
Further studies should explore the safety culture of other primary health care organisations, and perhaps also use other tools for measuring it in order to recognise other factors that are also important. Possibly, another tool for a more comprehensive measurement of safety culture that would be based in the theoretical frameworks should be developed.
Confirmatory factor analysis
Comparative Fit Index
Degree of freedom
Exploratory factor analysis
The Normed Fit Index
Quality and Costs of Primary Care in Europe
Root Mean Square Error of Approximation
Safety Attitudes Questionnaire
Safety Attitudes Questionnaire-ambulatory version
Deilkas ET. Patient safety culture - opportunities for healthcare management. Oslo: University of Oslo, Norway; 2010.
Guldenmund FW. The nature of safety culture: a review of theory and research. Safety Sci. 2000;34(1–3):215–57.
Tomolo AM, Lawrence RH, Watts B, Augustine S, Aron DC, Singh MK. Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system- level quality improvement skills. J Grad Med Educ. 2011;3(1):49–58.
Hann M, Bower P, Campbell S, Marshall M, Reeves D. The association between culture, climate and quality of care in primary health care teams. Fam Pract. 2007;24(4):323–9.
Modak I, Sexton JB, Lux TR, Helmreich RL, Thomas EJ. Measuring safety culture in the ambulatory setting: the safety attitudes questionnaire - ambulatory version. J Gen Intern Med. 2007;22:1–5.
Parker D, Wensing M, Esmail A, Valderas JM. Measurement tools and process indicators of patient safety culture in primary care. A mixed methods study by the LINNEAUS collaboration on patient safety in primary care. Eur J Gen Pract. 2015;21:26–30.
Curran C, Lydon S, Kelly M, Murphy A, Walsh C, O'Connor P. A systematic review of primary care safety climate survey instruments: their origins, psychometric properties, quality, and usage. J Patent Saf. 2018;14(2):e9–e18.
Vasconcelos PF, Arruda LP, Sousa Freire VEC, Carvalho R. Instruments for evaluation of safety culture in primary health care: integrative review of the literature. Public Health. 2018;156:147–51.
Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ. The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;3:44.
Smits M, Keizer E, Giesen P, Deilkas EC, Hofoss D, Bondevik GT. The psychometric properties of the 'safety attitudes questionnaire' in out-of-hours primary care services in the Netherlands. PLoS One. 2017;12(2):e0172390.
Bondevik GT, Hofoss D, Hansen EH, Deilkas EC. The safety attitudes questionnaire - ambulatory version: psychometric properties of the Norwegian translated version for the primary care setting. BMC Health Serv Res. 2014;14:139.
Bulajic M, Bezic H, Plavec D. Factors associated with the differences in the safety attitudes questionnaire 2006 short form assessment of the management of Croatioan hospitals. Hrčak. 2017;11(2):9–20.
Nguyen G, Gambashidze N, Ilyas SA, Pascu D. Validation of the safety attitudes questionnaire (short form 2006) in Italian in hospitals in the northeast of Italy. BMC Health Serv Res. 2015;15:284.
Saraiva D, De Almeida A. Validation of the safety attitudes questionnaire - short form 2006 to Portugal. Int J Nurs. 2015;2(1):103–12.
Lee WC, Wung HY, Liao HH, Lo CM, Chang FL, Wang PC, Fan A, Chen HH, Yang HC, Hou SM. Hospital safety culture in Taiwan: a nationwide survey using Chinese version safety attitude questionnaire. BMC Health Serv Res. 2010;10:234.
Goras C, Wallentin FY, Nilsson U, Ehrenberg A. Swedish translation and psychometric testing of the safety attitudes questionnaire (operating room version). BMC Health Serv Res. 2013;13:104.
Norden-Hagg A, Sexton JB, Kalvemark-Sporrong S, Ring L, Kettis-Lindblad A. Assessing safety culture in pharmacies: the psychometric validation of the safety attitudes questionnaire (SAQ) in a national sample of community pharmacies in Sweden. BMC Clin Pharmacol. 2010;10:8.
Zimmermann N, Kung K, Sereika SM, Engberg S, Sexton B, Schwendimann R. Assessing the safety attitudes questionnaire (SAQ), German language version in Swiss university hospitals--a validation study. BMC Health Serv Res. 2013;13:347.
Klemenc-Ketis Z, Svab I, Poplas Susic A. Implementing quality indicators for diabetes and hypertension in family medicine in Slovenia. Zdr Varst. 2017;56(4):211–9.
Petek D, Mlakar M. Quality of care for patients with diabetes mellitus type 2 in ‘model practices’ in Slovenia – first results. Zdr Varst. 2016;55(3):179–84.
Zelko E, Svab I. Overcoming cultural cleavages: results from a health promotion intervention among Roma. Acta Medico-Biotechnica. 2016;9(1):33–41.
Ivetic V, Poplas-Susic T, Pašić K, Selič P. Beliefs and viewpoints of family medicine physicians on approaches to identify and treat medically unexplained symptoms. Acta Medico-Biotechnica. 2016;9(2):47–57.
Pavlic DR, Sever M, Klemenc-Ketis Z, Svab I. Process quality indicators in family medicine: results of an international comparison. BMC Fam Pract. 2015;16(1):172.
Klemenc-Ketis Z, Deilkas ET, Hofoss D, Bondevik GT. Patient safety culture in Slovenian out-of-hours primary care clinics. Zdr Varst. 2017;56(4):203–10.
Klemenc-Ketis Z, Maletic M, Stropnik V, Deilkås ET, Hoffos D, Bondevik GT. The safety attitudes questionnaire – ambulatory version: psychometric properties of the Slovenian version for the out-of-hours primary care setting. BMC Health Serv Res. 2017;17:36.
Klemenc-Ketis Z, Deilkas ET, Hofoss D, Bondevik GT. Variations in patient safety climate and perceived quality of collaboration between professions in out-of-hours care. J Multidiscip Healthc. 2017;10:417–23.
Zohar D. Safety climate: conceptual and measurement issues. In: Campbell Quick J, Tetrick L, editors. Handbook of occupational Health Psychology. Washington, D. C: American Psychological Association; 2003. p. 123–42.
Kim HY. Statistical notes for clinical researchers: assessing normal distribution (2) using skewness and kurtosis. Restor Dent Endod. 2013;38(1):52–4.
Carmines E, McIver J. Analysing models with unobserved variables: analysis of covariance structures. Beverly Hills, CA: Sage Publications; 1981.
Hooper D, Coughlan J, Mullen M. Structural equation modelling: guidelines for determining model fit. Electron J Bus Res Methods. 2008;6:53–60.
Field A. Discovering statistics using IBM SPSS statistics. London: Sage Publication; 2013.
Gliem JA, Gliem RR. Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales. Ohio: Ohio State University; 2003.
Kirk S, Parker D, Claridge T, Esmail A, Marshall M. Patient safety culture in primary care: developing a theoretical framework for practical use. Qual Saf Health Care. 2007;16(4):313–20.
Knafl GJ, Grey M. Factor analysis model evaluation through likelihood cross-validation. Stat Methods Med Res. 2007;16(2):77–102.
Deilkas ET, Hofoss D. Psychometric properties of the Norwegian version of the safety attitudes questionnaire (SAQ), generic version (short form 2006). BMC Health Serv Res. 2008;8:191.
Palacios-Derflingher L, O'Beirne M, Sterling P, Zwicker K, Harding BK, Casebeer A. Dimensions of patient safety culture in family practice. Healthc Q (Toronto, Ont). 2010;13:121–7.
We would like to thank all the participants in the study.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The study was approved by the National Ethics Committee (No. 107/07/16). Participants gave a verbal informed consent to participate in the study. As this was an electronic survey, a written consent was not applicable. The method of acquiring the consent was approved by the ethics committee.
Consent for publication
The authors declare they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Klemenc-Ketis, Z., Makivić, I. & Poplas-Susič, A. Safety culture in the primary health care settings based on workers with a leadership role: the psychometric properties of the Slovenian-language version of the safety attitudes questionnaire – short form. BMC Health Serv Res 18, 767 (2018). https://doi.org/10.1186/s12913-018-3594-8
- Patient safety
- Validation studies