- Open Access
Correlation between safety attitudes and early adoption of cognitive aids in the German culture sphere: a multicenter survey study
BMC Health Services Research volume 22, Article number: 1215 (2022)
Cognitive Aids (checklists) are a common tool to improve patient safety. But the factors for their successful implementation and continuous use are not yet fully understood. Recent publications suggest safety culture to play a key role in this context. However, the effects on the outcome of implementation measures remain unclear. Hospitals and clinics that are involved in cognitive aid development and research might have significantly different safety cultures than their counterparts, resulting in skewed assessments of proper implementation. Therefore, the objective of this study was to assess the correlation between cognitive aid implementation and safety attitudes of staff members in early adopting and later adopting clinics.
An online survey of the Safety Attitudes Questionnaire (SAQ) was carried out in German anaesthesiology departments during the initial implementation of a new checklist for emergencies during anesthesia (“eGENA” app). Subsequently an analysis between subgroups (“eGENA” app usage and occupation), with Kruskal–Wallis- and Mann–Whitney-U-Tests was carried out for the general SAQ, as well as it six subscales.
Departments that introduced “eGENA” app (Median 3,74, IQR 0,90) reported a significantly higher median SAQ (U (NeGENA = 6, Nnon eGENA = 14) = 70,0, z = 2,31, p = 0,02, r = 0,516) than their counterparts (Median 2,82, IQR 0,77) with significant differences in the dimensions teamwork climate, work satisfaction, perception of management and working conditions.
Early adopters of cognitive aids are likely to show a significantly higher perception of safety culture in the SAQ. Consequently, successful implementation steps from these settings might not be sufficient in different clinics. Therefore, further investigation of the effects of safety culture on cognitive aid implementation should be conducted.
Since Kohn’s landmark publication “To err is human”  patient safety became a growing concern and research topic in the last two decades. Several strategies and concepts for improvement have been developed and implemented. One of these are cognitive aids (e.g. checklists, medication dosing charts). Generally their aim is to reduce the risk of errors associated with high or low levels of cognitive strain . Their effects have been researched globally, showing a wide range of results, mostly supporting their use. Adequately designed and implemented cognitive aids have repeatedly, in different contexts, proven to significantly improve clinically relevant measurements including a reduction in mortality and complications  as well as staff centered measurements such as improving handover quality  and staff satisfaction . In spite of their low costs and success in other industries (e.g. aviation), implementation of cognitive aids in healthcare settings still has to overcome specific challenges  resulting in heterogenic quality of their design and commonness of their application.
Because of their benefits policymakers increasingly start to include cognitive aids in their considerations and guidelines . But these tools also have inherent risks (e.g. checklist fatigue) caused by improper implementation and design , as well as significant opportunity costs . This results in complex management challenges for healthcare leaders. While some research into the implementation of cognitive aids has already been conducted, the factors for the success of implementation are not yet fully understood . Safety culture repeatedly has been shown to be both a significant facilitator and hurdle for implementation success [10, 11]. While several studies have shown an increase in measurements of safety culture during implementation measures , differences between these early adopters and innovators and later adopting institutions has not been explored. But these differences might inhibit successful implementation processes in later adopting organizations. Consequently, this study investigates the interrelation between cognitive aid implementation and safety culture during the initial implementation of an electronic cognitive aid for crisis management in anesthesia (eGENA) through a multicenter survey in early and later adopting departments using the Safety Attitudes Questionnaire (SAQ).
Some studies into safety culture have found a significant difference in safety culture between different occupations in the same departments . Since these tools are most commonly used interprofessionally, and therefore rely on adoption by all occupations, a significant impact on cognitive aid implementation might stem from these differences. Therefore, this study aims to replicate differences between occupations.
Safety culture is generally understood as a multidimensional concept, with an ongoing debate of its definition. Some authors distinguish between safety culture and safety climate, while others use the term synonymously . Because of its importance not only in healthcare but also in other industries (e.g. aviation, nuclear) several instruments have been developed for its quantification . Of these the Safety and Attitudes Questionnaire (SAQ) has been used most commonly in the context of cognitive aids in healthcare [15,16,17,18]. Originally developed as the Flight Management Attitudes Questionnaire to asses safety culture in aviation, multiple versions of the SAQ have been developed for different clinical surroundings (e.g. intensive care, operating rooms and inpatient settings) . These all contain 30 unchanged core items measuring six dimensions of safety culture (teamwork climate, safety climate, job satisfaction, stress recognition, perception of management and working conditions). Zimmermann et al. have translated these into German and published a validation study, with satisfactory psychometric properties in all dimension but perception of management .
Since emergencies during anesthesia pose a significant challenge in terms of decision making and team management the use of cognitive aids seems reasonable [10, 21], the German Society of Anaesthesia and Intensive Care (DGAI) and the German Professional Association of Anaesthetists (BDA) decided to implement the German Cognitive Aid Working Group with the aim of develop a Cognitive Aid for crisis management in anesthesia. The working group developed an electronic cognitive aid called eGENA (elektronische Gedächtnis- und Entscheidungshilfe für Notfälle in der Anästhesiologie)  applying a user centered design process and a multistep implementation strategy. The working group developed not only the actual cognitive aid but also an editor, which can be used to adapt the checklist to the local context, as well as a training concept to adequately train staff . After the development several papers describing the design process, the functionality, as well as recommended steps for its implementation have been published [2, 22,23,24,25]. Currently, however, it is not mandatory for German anaesthesia departments to use the cognitive aid during intraoperative emergencies. Although the cognitive aid is distributed for free and can be downloaded without prior registration, departments that were interested in adopting the cognitive aid were encouraged to contact the working group for further information and support.
During the initial implementation phase some of these departments already adopted eGENA (early adopters) while others were still in various stages of planning or decided against implementation (late adopters). We reached out to twenty-five departments, that had inquired about eGENA in the past. All departments contacted were departments or clinics for anaesthesiology (some had an additional focus on critical care medicine). They agreed to be contacted by the German Cognitive Aid Working Group for research purposes and were asked to participate in an online survey (SoSci Survey Version 3.2.12 (Germany)). The survey entailed questions about the usage of eGENA at the participant’s hospitals, as well as the German language version of the SAQ (SAQ-GER) by Zimmermann et al. and sociodemographic questions (occupation, gender and relevant work experience). The survey was accessible from February until June 2021. Since a consecutive increase of measurements of safety culture after cognitive aid implementation has been demonstrated  and as we expected low response rates due to high workload in German anaesthesia departments during the COVID-19-Pandemic we opted for a single survey period.
Statistical analysis was carried out with IBM SPSS Statistics Version 184.108.40.206. (USA). Since the scale “perception of management” has not yet shown sufficient internal validity in the SAQ-GER we evaluated Cronbach’s ⍺ for the SAQ und it’s subscales, while accepting values between 0,70 and 0,95. To determine adequate selection of tests for comparing subgroups, both the Kolmogorov–Smirnov- and Shapiro–Wilk-test were performed assuming normal distribution for p > 0,05. Subsequently, an analysis between the subgroups was carried out with the Kruskal–Wallis- as well as the Mann–Whitney-U-test.
Of the twenty-five departments contacted, seven agreed to participate in the study. After the end of the survey period 24 data sets could be collected. The respondents were predominantly male (58,3%), use eGENA in their institution (62,5%) and work as physicians (58,3%). Table 1 shows the profession and the relevant work experience of the respondents.
The evaluation of Cronbach’s alpha showed sufficient internal validity with 0,923 for the SAQ (30 Items). This did not persist for the subscale “perception of management” with a value of 0,366 (4 Items). Further analysis showed that the Item “management does not knowingly compromise the safety of patients” had a negative inter-item-correlation with -0,596 which was therefore excluded from further analysis, resulting in sufficient internal validity in the SAQ and all subscales (Table 2).
A normal distribution was not present in Kolmogorov–Smirnov and Shapiro–Wilk-test for all subscales (abnormal in stress recognition and working conditions). Therefore, subgroups were analyzed with the Kruskal–Wallis- as well as the Mann–Whitney-U-test.
The Mann–Whitney-U-test comparing the responses of participants with or without eGENA in their departments showed a significantly higher median SAQ in the departments with eGENA implementation (U (N(eGENA) = 6, N(non-eGENA) = 14) = 70,0, z = 2,31, p = 0,02), as well as a large effect size (r = 0,516). We therefore performed a Mann–Whitney-U-test for all subscales. The Median eGENA departments was significantly higher with a moderate to large effect in 4 of the 6 subscales (Table 3). Surprisingly the subscale “stress recognition” showed by far the least differences between the subgroups.
Because of the small sample size, a Mann–Whitney-U-test for differences by occupation was performed for compiled subgroups of physicians (including consultants and residents; N = 14) as well as a subgroup of nurses (including nurses, specialized nurses and anesthesia technicians; N = 9) (Table 4). While the median SAQ was significantly higher with a large effect size for physicians only 3 (job satisfaction, perception of management and working conditions) of the 6 subgroups achieved statistical significance, with moderate to large effect sizes.
Kruskal–Wallis tests for differences by gender and relevant work experience did not achieve statistical significance in the SAQ as well as its subscales.
Statistically significant differences between eGENA and non-eGENA departments and between occupations could be attained although only a small sample size (n = 24) could be achieved. Overall participants from eGENA-Departments reported a significantly higher median SAQ-GER than their counterparts from non-eGENA departments indicating a higher SAQ in early adopters of this cognitive aid (Fig. 1). In this context, employees with high SAQ values can be a surrogate for high SAQ values in the entire department . Safety culture is therefore likely to differ significantly from settings where cognitive aids are developed and researched to later adopting settings. This might be one of several reason why, these potent, low-cost tools aren’t used as extensively as their possibilities might allow.
Statistical differences between early and later adopting anaesthesiology departments could be found in the subscales teamwork climate, job satisfaction, perception of management as well as working conditions. But the scale safety climate (U (NeGENA = 6, Nnon eGENA = 14) = 80,0, z = 1,94, p = 0,056, r = 0,414) did not achieve statistical significance. Therefore, multiple dimensions of safety culture are different in those early adopting departments. This could lead to extensive effects on the success of cognitive aid implementation. Experiences, insights and best practices from departments that are involved in development and research into cognitive aids therefore most likely must be adapted in different contexts. Since cognitive aids might be cost effective potent tools for improving patient safety the implication of these differences should be investigated further or else these tools might lose their impact and staff participation might fade off. Several strategies and techniques for the increase in safety culture measurements have been published in recent years. We suggest benchmarking the SAQ against a successful implementation to decide on the value of using one of these before cognitive aid implementation.
Surprisingly the subscale stress recognition showed very similar results in both subgroup assessments (U (NeGENA = 6, Nnon eGENA = 14) = 54,0, z = -0,39, p = 0,728, r = -0,081). Other studies, that did not focus on one specific specialty, typically reported lower values in this scale. Though this result can be perceived as favorable, since recognition of stress is an important step minimize its negative effects, the reasons for this discrepancy are unclear. The answer might lie in causes specific to the specialty, the departments or the selection of participants. Further research might provide significant insight in improving stress recognition for institutions.
Regardless of the small sample size respondents from different levels of work experience as well as occupations completed the survey. Therefore, we were able to again attain significant differences in perception of safety culture between occupations, suggesting a significant external validity. But due to the small sample size multiple occupations had to be compiled together. Comparing these compiled groups of physicians and nurses significant differences occurred for the SAQ in general, as well as for the scales job satisfaction, perception of management and working conditions. Noticeably these scales all relate to different aspects determined by management. In the German healthcare system physicians and nurses are typically under different management and work different shifts and hours, while typically caring for one patient in a nurse physician team. Additionally, poor working condition of nurses were and still are part of an ongoing debate in German society. Consequently, the differences by occupation might be caused be external effects, that do not influence cognitive aid implementation. These differences were nonetheless measurable in this study. Several studies have shown differences in cognitive aid acceptance in correlation with occupation . We therefore conclude that frameworks for cognitive aid implementation in German anaesthesiology departments should adequately assess and adjust for local circumstances from different occupations.
Though statistically significant differences between the departments as well occupations could be attained, we recognize several limitations to our study. First the sample size of this study was unexpectedly small. This might be explained by specific idiosyncrasies of the German health system where both anesthetists and anaesthesiology nurses through their training co-specialize for intensive care. During the survey period workload in German ICU due to COVID-19 was especially high and relocation of personnel from operating theaters was not uncommon. Another explanation might be the mode of contact for inclusion in the study, since it relied on high motivation from the participating departments to forward the survey. Likewise, the total size of the respective departments is not known. A response rate could therefore not be calculated. This secondly results in the possibility that single departments might be highly overrepresented, resulting in misinterpretation of the effects of the implementation of eGENA on the SAQ. Thirdly this study did not compare the SAQ with the use of eGENA, but only its availability, and therefore only describes the effects of the occurrence of implementation. Since no data was obtained before the implementation the subgroups of the departments might have shown significant differences before the use of eGENA, so the effect of the implementation can’t be assessed with this study. Yet the small sample size might have shown that small subgroups can be sufficient to generate statistically significant differences in the SAQ-GER. This Score therefore might even be of utility for benchmarking of smaller departments or hospitals, as well as ongoing surveillance.
Despite a small sample size (n = 24), this study likely demonstrates that members of departments that recently adopted the cognitive aid eGENA report a significantly higher median SAQ-GER than peers from departments without eGENA implementation, indicating a higher perception of safety culture in early adopters of cognitive aids. Since it has been shown to both be a significant facilitator and hurdle for implementation success, studying best practices from successful applications might not be enough, rather local leaders should carefully inspect requirements for implementation. Frameworks for implementation of cognitive aids should therefore include a reliable assessment of safety culture. For this benchmark from successful introductions might be of high utility.
Furthermore, some specific characteristics of the German language version of the SAQ could be shown. This includes a low Cronbach’s ⍺ for the subscale perception of management. Therefore, results of this and similar surveys should only be carefully compared to other versions of the SAQ. If the version by Zimmermann et al. would be used for benchmark development the item “Management does not knowingly compromise the safety of patients” from the scale perception of management should be excluded. This further complicates benchmarking against international samples.
Additionally, a significant difference in the SAQ-GER subject to the occupation could be demonstrated. Frameworks for cognitive aid implementation should therefore likely include measures to adequately address all occupations and their individual requirements in German anaesthesiology departments. The reasons for and the implications of these differences remain unclear and warrant further investigation.
Small sample sizes might be sufficient to draw significant conclusions about smaller subgroups. Therefore, the SAQ-GER might be used for ongoing surveillance of the safety climate even of smaller departments. Further investigation of the results of the subscale stress recognition in the participating departments might result in significant insights in improving the effects of stress on patient safety.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Institute of Medicine (US) Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System. US: National Academies Press; 2000.
Eismann H, Schild S, Neuhaus C, et al. Cognitive aids for crisis management in anaesthesiology. Principles and applications. Anasthesiologie & Intensivmedizin. 2020;61:239-47. (In German). https://doi.org/10.19224/ai2020.239.
Saxena S, Krombach JW, Nahrwold DA, Pirracchio R. Anaesthesia-specific checklists: a systematic review of impact. Anaesth Crit Care Pain Med. 2020;39:65–73. https://doi.org/10.1016/j.accpm.2019.07.011.
Wölfl C, Kotter J, Trupkovic T, Grützner PA, Münzberg M. „Trauma room time out“ (TRTO): Neues Sicherheitstool zur Verbesserung der Patientensicherheit und Mitarbeiterzufriedenheit im Schockraum. Unfallchirurg. 2014;117:83–5. https://doi.org/10.1007/s00113-013-2552-5.
Fletcher KA, Bedwell WL. Cognitive aids: design suggestions for the medical field. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care. 2014;3:148–52. https://doi.org/10.1177/2327857914031024.
Bundesausschuss G. Richtlinie des Gemeinsamen Bundesausschusses über grundsätzliche Anforderungen an ein einrichtungsinternes Qualitätsmanagement für Vertragsärztinnen und Vertragsärzte, Vertragspsychotherapeutinnen und Vertragspsychotherapeuten, medizinische Versorgungszentren, Vertragszahnärztinnen und Vertragszahnärzte sowie zugelassene Krankenhäuser (Qualitätsmanagement- Richtlinie/QM-RL). 2020. Available from: https://www.g-ba.de/downloads/62-492-2309/QM-RL_2020-09-17_iK-2020-12-09.pdf.
Grigg E. smarter clinical checklists: how to minimize checklist fatigue and maximize clinician performance. Anesth Analg. 2015;121:570–3. https://doi.org/10.1213/ANE.0000000000000352.
Semel ME, Resch S, Haynes AB, Funk LM, Bader A, Berry WR, Weiser TG, Gawande AA. Adopting a surgical safety checklist could save money and improve the quality of care in U.S. hospitals. Health Aff (Millwood). 2010;29:1593–9. https://doi.org/10.1377/hlthaff.2009.0709.
Alidina S, Goldhaber-Fiebert SN, Hannenberg AA, Hepner DL, Singer SJ, Neville BA, Sachetta JR, Lipsitz SR, Berry WR. Factors associated with the use of cognitive aids in operating room crises: a cross-sectional study of US hospitals and ambulatory surgical centers. Implementation Sci. 2018;13:50. https://doi.org/10.1186/s13012-018-0739-4.
Goldhaber-Fiebert SN, Pollock J, Howard SK, Bereknyei Merrell S. emergency manual uses during actual critical events and changes in safety culture from the perspective of anesthesia residents: a pilot study. Anesth Analg. 2016;123:641–9. https://doi.org/10.1213/ANE.0000000000001445.
Treadwell JR, Lucas S, Tsou AY. Surgical checklists: a systematic review of impacts and implementation. BMJ Qual Saf. 2014;23:299–318. https://doi.org/10.1136/bmjqs-2012-001797.
Fudickar A, Hörle K, Wiltfang J, Bein B. The effect of the WHO surgical safety checklist on complication rate and communication. Deutsches Aerzteblatt Online. 2012. https://doi.org/10.3238/arztebl.2012.0695.
Relihan E, Glynn S, Daly D, Silke B, Ryder S. Measuring and benchmarking safety culture: application of the safety attitudes questionnaire to an acute medical admissions unit. Ir J Med Sci. 2009;178:433–9. https://doi.org/10.1007/s11845-009-0352-2.
Lee SE, Scott LD, Dahinten VS, Vincent C, Lopez KD, Park CG. Safety culture, patient safety, and quality of care outcomes: a literature review. West J Nurs Res. 2019;41:279–304. https://doi.org/10.1177/0193945917747416.
Alomari A, Sheppard-Law S, Lewis J, Wilson V. Effectiveness of clinical nurses’ interventions in reducing medication errors in a paediatric ward. J Clin Nurs. 2020;29:3403–13. https://doi.org/10.1111/jocn.15374.
Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AHS, Dellinger EP, Dziekan G, Herbosa T, Kibatala PL, Lapitan MCM, Merry AF, Reznick RK, Taylor B, Vats A, Gawande AA, for the Safe Surgery Saves Lives Study Group. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Quality & Safety. 2011;20:102–7. https://doi.org/10.1136/bmjqs.2009.040022.
Hill MR, Roberts MJ, Alderson ML, Gale TCE. Safety culture and the 5 steps to safer surgery: an intervention study. Br J Anaesth. 2015;114:958–62. https://doi.org/10.1093/bja/aev063.
Kawano T, Taniwaki M, Ogata K, Sakamoto M, Yokoyama M. Improvement of teamwork and safety climate following implementation of the WHO surgical safety checklist at a university hospital in Japan. J Anesth. 2014;28:467–70. https://doi.org/10.1007/s00540-013-1737-y.
Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, Roberts PR, Thomas EJ. The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6:44. https://doi.org/10.1186/1472-6963-6-44.
Zimmermann N, Küng K, Sereika SM, Engberg S, Sexton B, Schwendimann R. Assessing the safety attitudes questionnaire (SAQ), German language version in Swiss university hospitals - a validation study. BMC Health Serv Res. 2013;13:347. https://doi.org/10.1186/1472-6963-13-347.
Huang J, Wu J, Dai C, Zhang X, Ju H, Chen Y, Zhang C, Ye F, Tan Y, Zong Y, Liu T. Use of emergency manuals during actual critical events in china: a multi-institutional study. Simul Healthc. 2018;13:253–60. https://doi.org/10.1097/SIH.0000000000000303.
Neuhaus C, Schild S, Eismann H, Baus J, Happel O, Heller AR, et al. Functionalities and operating instructions for eGENA, the German digital cognitive aid for crisis management in anaesthesiology. Anästhesiologie & Intensivmedizin. 2020;61(0708):340–51. https://doi.org/10.19224/ai2020.340.
Schild S, Sedlmayr B, Schumacher AK, Sedlmayr M, Prokosch HU, St Pierre M. A digital cognitive aid for anesthesia to support intraoperative crisis management: results of the user-centered design process. JMIR Mhealth Uhealth. 2019;7:e13226. https://doi.org/10.2196/13226.
Schild S, Gruendner J, Gulden C, Prokosch H-U, St Pierre M, Sedlmayr M. Data model requirements for a digital cognitive aid for anesthesia to support intraoperative crisis management. Appl Clin Inform. 2020;11:190–9. https://doi.org/10.1055/s-0040-1703015.
Richter T, Baus J, Eismann, H, Happel O, Heller A, Neuhaus C, Weinert M, St. Pierre M. Bereitstellung einer Smartphone-Variante von eGENA, der elektronischen Gedächtnis- und Entscheidungshilfe für Notfälle in der Anästhesiologie. Anasthesiologie und Intensivmedizin. 2021;62:V1-V8.
Haller G, Garnerin P, Morales M-A, Pfister R, Berner M, Irion O, Clergue F, Kern C. Effect of crew resource management training in a multidisciplinary obstetrical setting. Int J Qual Health Care. 2008;20:254–63. https://doi.org/10.1093/intqhc/mzn018.
O’Connor P, Reddin C, O’Sullivan M, O’Duffy F, Keogh I. Surgical checklists: the human factor. Patient Saf Surg. 2013;7:14. https://doi.org/10.1186/1754-9493-7-14.
Open Access funding enabled and organized by Projekt DEAL. Departmental funding only.
Ethics approval and consent to participate
All methods were performed under the relevant guidelines based on the Helsinki Statement. We obtained written informed consent from all study participants after a detailed explanation about the procedure of the study. They were informed that participation was voluntary, that they could withdraw at any time and their anonymity was assured. The ethics committee of the Hannover Medical School has approved this study (9444_BO_K_2020).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wegener, J., St.Pierre, M., Keil, O. et al. Correlation between safety attitudes and early adoption of cognitive aids in the German culture sphere: a multicenter survey study. BMC Health Serv Res 22, 1215 (2022). https://doi.org/10.1186/s12913-022-08581-3
- Cognitive Aid implementation
- Checklist implementation
- Safety Attitudes Questionnaire (SAQ)
- Elektronische Gedächtnis- und Entscheidungshilfe für Notfälle in der Anästhesiologie (eGENA)