- Research article
- Open Access
German translation and validation of the Reporting of Clinical Adverse Events Scale (RoCAES-D)
BMC Health Services Research volume 20, Article number: 689 (2020)
Reporting of adverse events is an important aspect of patient safety management in hospitals, which may help to prevent future adverse events. Yet, only a small proportion of such events is actually reported in German hospitals. Therefore, it is crucial to evaluate attitudes of clinical staff towards reporting of adverse events. The aim of this study was to translate the Reporting of Clinical Adverse Events Scale (RoCAES) developed by Wilson, Bekker and Fylan (2008) and validate it in a sample of German-speaking health professionals.
The questionnaire covers five factors (perceived blame, perceived criteria for identifying events that should be reported, perceptions of colleagues’ expectations, perceived benefits of reporting, and perceived clarity of reporting procedures) and was translated into German language according to translation guidelines. Within a cross-sectional study in a sample of 120 health professionals in German hospitals, internal consistency (omega) and construct validity (confirmatory factor analysis) of the German scale RoCAES-D was assessed.
The reliability was high (omega = 0.87) and the factor analysis showed a poor model fit (RMSEA: 0.074, χ2/df: 1.663, TLI: 0.690). Resulting from lower model fit of the original model (RMSEA: 0.082, χ2/df: 1.804, TLI: 0.606), one item was deleted due to low factor loadings and a low R2 (0.001), and two items were reallocated from the factor ‘perceived benefits’ to ‘perceived blame’.
The successful translation and initial validation of the RoCAES-D might be a good starting point for further research. A cultural adaptation of the scale needs to be done to initiate a large-scale usage of the questionnaire.
Patient safety indicates the absence of adverse events (AE), which has received increased interest as it is endangered by the increasing complexity of healthcare, the simultaneously increasing economic pressure and the shortages in the healthcare work force . An AE is defined as an unintended injury resulting from a medical intervention, regardless of the patient’s underlying medical condition [2, 3]. Prevention of AEs has been the focus of many improvement initiatives and studies across the world .
The occurrence of AEs varies between 3 and 17% of all hospital stays [1, 5, 6]. In Germany, AEs are expected to affect 5–10% of all patients in the inpatient sector, implying 950.000–1.900.000 AEs per year . Up to 60% of these AEs are considered to be preventable, resulting in approximately 19.000 avoidable deaths per year in Germany [3, 6, 7].
To understand structures and procedures in health care that might lead to an AE, it is crucial to analyse them objectively assuming AEs being reported at first . Hospital risk management, which is obligatory in German hospitals according to prevailing regulations of the Joint Federal Committee (gemeinsamer Bundesausschuss) since 2014, contains instruments to reporting of AEs and critical incidents in general [3, 9]. The critical incident reporting system (CIRS) is one of these instruments and comprises a platform where hospital employees can anonymously report AEs as well as other critical incidents that occur in the context of health care and increase the risk for AEs. These reports are then analysed by experts who give recommendations for the future avoidance of events of this kind .
However, there is a range of barriers that impede AE reporting. As possible reasons for underreporting of AEs, a lack of support from authorities or the person ‘causing’ the error, fear of consequences, lack of knowledge and high workload were named [11, 12]. There is strong evidence, that the number of unreported or unpublished AEs is much higher than the ones being reported and included in AE estimations leaving approximately up to 91% unreported [3, 13, 14].
Beneficial safety culture is necessary to achieve a supporting environment and thus encourage staff to report AEs . In Germany, within the last 15 years the ways of dealing with errors and AEs has changed. From avoiding this topic up to then, to approaches that aim to analyse factors that contribute to the occurrence of errors and also include an organisational perspective . However, still more than half of German hospitals are estimated to not ensure an optimally beneficial safety culture that includes the ability to learn from reported incidents . To promote a supportive safety culture in general, it is necessary to understand attitudes and values of employees towards adverse event reporting .
Yet, there is no German-language questionnaire assessing health professional’s attitudes to AE within German health services.
The Reporting of Clinical Adverse Events Scale (RoCAES) was developed and validated in the UK by Wilson, Bekker and Fylan in 2008 and assesses five-domains of staff attitudes to AEs: perceived blame (six items); perceived criteria for identifying events that should be reported (six items); perceptions of colleagues’ expectations (six items); perceived benefits of reporting (five items); perceived clarity of reporting procedures (two items) . It was translated and validated for use with health professionals in China , and may be a robust measure for use within a German healthcare system. The aim of this study was to translate this questionnaire into German and assess its reliability and validity for use to assess attitudes of health professionals to AEs in German hospitals.
The questionnaire was translated into German language according to international translation guidelines. Psychometric properties of this translated version were then examined in a cross-sectional study in German hospitals.
Forward translation: The instrument was translated from English into German by two independent translators whose mother tongue is German and that are fluent in English.
Comparison of the two translated versions: Within step a), two translated versions of the questionnaire were produced. A consensus meeting was held to discuss discrepancies and ambiguities of these two versions and to generate the preliminary initial translated version.
Blind back-translation: Two bilingual translators, whose mother tongue is English, were asked to independently translate the preliminary initial translated version back into the source language. For this step, translators with no initial knowledge of the instrument were chosen.
Comparison of the two back-translated versions: Within a second consensus meeting, discrepancies in sentence structures, meaning or other uncertainties have been discussed and an English consensus version has been merged. In addition to this step, the back-translated version has been sent to the original author of the RoCAES who compared this version with her initial intentions of the original questionnaire to ensure conceptional equivalence.
Pilot testing of the pre-final version: several independent experts were asked to review the German pre-final version of the questionnaire. Furthermore, five experts matching the inclusion criteria for the subsequent survey were involved in a cognitive debriefing evaluating instructions, items and response format clarity.
Study population and data collection
For data collection, the RoCAES-D was transferred into an electronic version. The link to the online survey was sent to approximately 1300 staff members in 18 different institutions (14 hospitals, a medical school, a nursing school, a school for therapists and a germanwide network of professionals interested in incident reporting) step by step in spring 2019. Study population included all clinical staff members and students that work in the inpatient care sector and have direct patient contact (e.g. nurses, physicians, therapists, …). We covered this interprofessional perspective in the German version contrarily to the eligibility criteria of Wilson et al. and Sun et al. [16, 17], who addressed doctors and nurses only, as interprofessional teamwork plays an important role in patient safety and prevention of clinical incidents .
The hospitals cover various clinical departments such as neurology, orthopedy, rehabilitation clinics, geriatrics or internal medicine. Within the hospitals, the survey was sent to the management, quality management and staff councils via e-mail. They were asked to pass on the mail to their employees. In addition, we offered to send paper-based questionnaires to reach staff without access to their e-mail accounts. Because management staff is not included in the target group addressed by the questionnaire, an initial filter question excluding unsuitable test persons was added. Furthermore, the clinical staff was also asked to send the link to their colleagues, former fellow students or friends matching the eligibility criteria (snowball system).
The RoCAES consists of 25 items rated with a 4-point-likert scale from ‘1’ (strongly agree) to ‘4’ (strongly disagree). 16 items required reversed scoring. As agreeing to item 4 (‘reporting adverse events protects patients’) results in a positive attitude and item 3 (‘It is not my responsibility to report adverse events involving colleagues’) is formulated negatively, we assumed that the scoring guideline for these two items was switched in the original study. Contrary to the recommendation, we therefore reverse scored item 3 instead of item 4 for our analysis.
Since the questionnaire was sent in an online version and included mandatory fields, the answer option ‘not specified’ had to be inserted.
An introductory definition of AEs and three additional items introduce the main scale focusing on whether the participant has ever witnessed or been involved in an AE, if he ever reported an AE and how likely the participant is to report an AE in the future. Closing the questionnaire, demographic data was collected including profession, gender and previous duration of employment. The student employee status was not assessed separately. The full German questionnaire can be found in the Additional file 1.
To assess reliability and validity of the RoCAES-D, only fully completed questionnaires were analysed. For further analysis, answers that chose ‘not specified’ were treated as missing data and have been imputed using the mean value of each item across participants . Internal consistency was determined by calculating coefficient McDonald’s omega which can be used for multidimensional tests as a more suitable alternative to Cronbach’s alpha . To find omega, a factor analysis is done by rotating the factors obliquely and performing a Schmid Leiman transformation. Cronbach’s alpha was assessed as well to provide better comparability and transparency. Alpha was calculated for the overall scale following the original authors and the Chinese validation paper [16, 17].
A CFA was selected for testing construct validity. For this, the factor model was based on data of original research by Wilson et al. and Xiao Sun et al. [16, 17] and was used to test its model fit for the empiric research data. Data analysis was done using R Version 3.5.1 and SPSS Version 25. In R we used the procedures ‘omega’ within the psych package and ‘cfa’ within the lavaan package.
The CFA was done using the original factor structure of the RoCAES. A one-factor model was used to compare model fit showing no relevant benefit for the existing five-factor model. For calculating reliability, the used coefficient McDonald’s omega estimates the general factor saturation of a questionnaire by doing an exploratory factor analysis (EFA) first. This exploratory approach was used, as well as residuals and covariances calculated within the CFA, to modify the existing factor model.
More precisely, items with a low factor loading (less than 0.25) and a low number of explained variance (R2 less than 0.1) were reallocated using the highest observed factor loadings in the exploratory analysis that was done to estimate omega.
The translation process did not reveal any noteworthy discrepancies or ambiguities. The pilot testing via cognitive debriefing has led to an adaption of two sentences, that had to be changed marginally due to incongruities in sentence structure. Observed associations triggered by the phrasing of the items agreed with each other and with the intention of the questionnaire. Together with the standardized introduction, and the written instructions, objectivity of application can be assumed as given for this scale.
Furthermore, data analysis is clearly defined by referring to a fixed scoring for each answer option. The objectivity of the evaluation is therefore also seen as given.
The psychometric testing of the pre-final German RoCAES-D was conducted in spring of 2019. A total of n = 120 health professionals in 26 hospitals completed the survey. Because the recruitment was conducted as a snowball system, response rate cannot be estimated accurately. The sample included heterogeneous professions and work experience. Sociodemographic characteristics of the sample are shown in Table 1.
Confirmatory factor analysis
For the CFA, as the answer options ‘not specified’ were treated as missing values, two items showed no missings. In all other items, missing values range from 0.8 to 16%. The two items with the highest missing rate include Item 11 (‘My colleagues expect me to report adverse events’, 16% missing values) and Item 13 (‘The procedures in this hospital are clear on what sort of adverse events should be reported’, 12.5% missing values). Other items showed missing rates lower than 7.5%.
Results of the initial CFA showed a poor model fit (Table 2) for the original structure in the German context. Using the one-factor model to compare model fit also showed no relevant benefit for the existing five-factor model. Therefore, the original model was modified as described previously. This modified model resulted in the lowest RMSEA*, SRMR*, and AIC* (Table 2). Hence, it is considered to be the one with the best model fit [19, 22].
Classifying the measures, the modified model showed a poor model fit for the German target group. Modifications in this model conclude the deletion of item 9 (‘I am not doing my job properly unless I report adverse events’) because of low factor loadings (0.024) and a low R2 (0.001). Item 1 (‘Reporting adverse events helps identify staff who need additional training’) and Item 24 (‘Reporting adverse events makes people accountable for their actions’) originally loaded on factor 4 (‘Perceived benefits of reporting’) and have now been relocated to factor 1 (‘Perceived blame’). Both items showed low factor loadings within the initial CFA and a correlation with other items within the EFA. Additionally, a content-related alignment lead to the described reallocation. Item 4 (‘Reporting adverse events protects patients’) also required a reallocation. However, a new assignment was not reasonable due to a bad model fit and a content-related match with the original factor 4 (perceived benefits). This resulting 24-item questionnaire can be seen in Table 3.
For the translated pre-final version, omega was 0.85, implying a high factor saturation. In order to ensure better comparability, Cronbach’s alpha was also calculated and was 0.79. Both coefficients were again estimated for the modified 24-item RoCAES-D. Omega increased to 0.87, alpha also increased to 0.81 (Factor 1: 0.8; Factor 2: 0.53; Factor 3: 0.45; Factor 4: 0.53; Factor 5: 0.68).
The resulting RoCAES-D shows high reliability and a poor validity for the German target population. Cronbach’s alpha was somewhat lower than but still comparable with the English RoCAES (0.83) and the Chinese C-RoCAES (0.85) [16, 17]. However, coefficient omega showed a high reliability (0.87). The even poorer model fit of the one factor model (Table 2) in the RoCAES-D underlines the multidimensional structure of the scale .
The original factor structure did not have a good fit with the data. Reallocating items 1 and 24 may result from cultural differences between German and English health professions. The two items were originally associated with benefits of reporting and now correlate with perceived blame. As risk management and handling adverse events is not an official part of the professional education in German health professions [26, 27], staff is not inevitably habituated with reporting processes, implying a perceived blame when actually practiced. In contrast, being aware of risks of medical interventions and being familiar with reporting procedures is part of the English competence model of nurses , resulting in a more neutral and professional point of view perceiving benefits instead of blame and implying a higher rate of incident reporting . The observed negative loadings of those two items (Table 3) might also support the cultural differences that have been identified throughout the reallocation. Furthermore, resulting from this different understanding of professional roles, German health care workers might not primarily associate their professional duties in health care with the responsibility of reporting adverse events, which can be seen in the CFA: item 9 (‘I am not doing my job properly unless I report adverse events’) showed particularly low factor loadings on any of the five factors and an average R2 of 0.001 and was therefore deleted for the RoCAES-D.
Due to the imputation of missing values, an impact on variance and therefore on the estimated psychometric properties leading to a miscalculation needed to be considered. Yet, the proportion of imputed values stayed relatively low with only two items higher than 7.5% (16% and 12,5%). In this order, no considerable impact is expected .
Besides the acceptable psychometric properties of the RoCAES-D, the usage of the scale in a multiprofessional target group has proven to be practical. This population differs from the original survey in England, which only included nurses and doctors. The German scale also addressed other health professions that are directly involved in daily patient care. In addition to actual evidence showing the necessity of involving an interprofessional team to prevent errors and adverse events, Wilson et al. also recommended an application to a more diverse clinical staff [16, 21]. The RoCAES-D can be carried out in German health care institutions without further restriction and hence, involve a larger number of clinical staff. Interventions that can be derived from performing these surveys will now be tailored more specifically and are therefore expected to be more effective as they address the whole multiprofessional team .
Limitations to our research concern the study population and the relatively low response rate as well as the model fit. In this survey, the response rate has been extremely low. However, the main purpose of this data collection was to gain a high variance and only use the data for the translation and validation process. For this, the sample size of n = 120, population structure and model fit are seen as appropriate [19, 22, 23]. To get an actual insight in the attitudes of health professionals, we recommend using the scale on institutional level. For this it is important to monitor the population structure and to achieve a higher response rate. On this basis, subgroup analysis, derivation of action plans or identification of needs for further investigation, e.g. employee interviews or process analyses, can be done properly. We also recommend a content-related revision of the scale after several years of use in different institutions. Possible additional cultural differences between the English, Chinese and German reporting culture have not been identified yet. The implementation of the scale for institutional use can help uncover cultural aspects and items that might need to be added for the German context. Even if the RoCAES-D showed a high reliability, the lower internal consistency on factor-level is reflected by the poor model fit of the modified model. This might support these cultural differences that require further investigation. Institutional usage with a data analysis on item-level, as well as additional psychometric testing to increase model fit is indicated in this regard.
The fact that a CFA had to be carried out instead of calculating divergent or convergent validity using a related survey underlines the existing research gaps. Nevertheless, translating and initially validating this scale might be a good starting point for further research and a progression for risk management in German health care institutions.
With 25 Items, the scale showed to be comprehensive. This comprehensiveness might explain the relatively low factor loadings. Yet, the purpose of this scale is to be an easy applicable screening device that identifies diverse starting points for further investigations or potential for further actions within a health care facility. As the scale is also short and concrete, we believe it can easily be embedded in a more extensive survey within institutions. It can for example be combined with a questionnaire addressing safety culture in general, or any other.
In conclusion, the successful translation and initial validation of the RoCAES-D might be a good starting point for further research whereby data can be collected for further training or further investigations regarding an internal AE management system in clinics. Based on these results, a cultural adaptation of the scale needs to be done to initiate a large-scale usage of the questionnaire and to gain a useful instrument. Usage of the scale can contribute to improve patient safety on a long-term basis and area-wide through tailored interventions, such as education of employees or promoting a supporting reporting culture.
Availability of data and materials
The datasets used and analysed during the current study are available from the corresponding author on reasonable request. The German questionnaire can be seen in the appendix and is free for use when cited adequately.
Clinical Incident Reporting System
Reporting of Clinical Adverse Events Scale
German Translation of the RoCAES
Chinese Translation of the RoCAES
Confirmatory Factor Analysis
Exploratory Factor Analysis
Root Mean Square Error of Approximation
Standardized Root Mean Squared Residual
Bentler’s Comparative Fit Index
Tucker Lewis Index
Akaike’s Information Criterion
Degrees of Freedom
Waeschle RM, Bauer M, Schmidt CE. Fehler in der Medizin. Ursachen, Auswirkungen und Maßnahmen zur Verbesserung der Patientensicherheit. Anaesthesist. 2015;64:689–704.
Kohn LT, Corrigan Janet M, Donaldson MS. To err is human: building a safer health system. Washington, D.C.: National academy press; 1999.
Schrappe M. Patient safety: a topic of the future, the future of the topic. Bundesgesundheitsbl Gesundheitsforsch Gesundheitsschutz. 2015;58(1):4–9.
Lawati MHA, Dennis S, Short SD, et al. Patient safety and safety culture in primary health care: a systematic review. BMC Fam Pract. 2018;19(1):104.
Hoffmann B, Rohe J. Patient safety and error management: what causes adverse events and how can they be prevented? Dtsch Arztebl Int. 2010;107(6):92–9.
Siman AG, Cunha SGS, Brito MJM. The practice of reporting adverse events in a teaching hospital. Rev Esc Enferm USP. 2017;51:e03243.
de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care. 2008;17(3):216–23. https://doi.org/10.1136/qshc.2007.023622.
Eisold C, Heller AR. Risk management in anesthesia and critical care medicine. Medizinische Klinik, Intensivmedizin und Notfallmedizin. 2017;112(2):163–76.
Rothmund M, Kohlmann T, Schmidt CE. Einführung und Beurteilung von Maßnahmen zur Fehlerprävention in chirurgischen Kliniken: Ergebnisse einer aktuellen online-Befragung. Z Evid Fortbild Qual Gesundh wesen. 2015;109:284–393.
Merkle W. Risikomanagement und Fehlervermeidung im Krankenhaus. Heidelberg: Springerverlag; 2014.
Poorolajal J, Rezaie S, Aghighi N. Barriers to medical error reporting. Int J Prev Med. 2015;6(97):1.
Luengo C, Paravic T, Valenzuela S. Causes of underreporting of occupational injuries and adverse events in Chile. Rev Panam Salud Publica. 2016;39(2):86–92.
Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Med. 2016;13(9):e1002127Published 2016 Sep 20. https://doi.org/10.1371/journal.pmed.1002127.
Farquhar C, Armstrong S, Kim B, Masson V, Sadler L. Under-reporting of maternal and perinatal adverse events in New Zealand. BMJ Open. 2015;5(7):e007970Published 2015 Jul 23. https://doi.org/10.1136/bmjopen-2015-007970.
Hansen S, Schwab F, Gropmann A, Behnke M, Gastmeier P. Infection control and safety culture in German hospitals. Bundesgesundheitsbl Gesundheitsforsch Gesundheitsschutz. 2016;59(7):908–15.
Wilson B, Bekker HL, Fylan F. Reporting of clinical adverse events scale: a measure of doctor and nurse attitudes to adverse event reporting. Qual Saf Health Care. 2008;17(5):364–7.
Sun X, Shi Y, Zhang S, et al. Reliability and validity of the Chinese version of reporting of clinical adverse events scale (C-RoCAES). Int J Clin Exp Med. 2014;7(10):3543–9.
Beaton DE, Bombardier C, Guillemin F, et al. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25(24):3186–91.
Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. J Eval Clin Pract. 2011;17(2):268–74.
Wild D, Grove A, Martin M, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. 2005;8(2):94–104.
Ginsburg L, Bain L. The evaluation of a multifaceted intervention to promote “speaking up” and strengthen interprofessional teamwork climate perceptions. J Interprof Care. 2017;31(2):207–17.
Moosbrugger H, Schermelleh-Engel K. Exploratorische (EFA) und Konfirmatorische Faktorenanalyse (CFA). In: Moosbrugger H, Kelava A, editors. Thesttheorie und Fragebogenkonstruktion. 2. Berlin Heidelberg: Springer-Verlag; 2012.
Wolf EJ, Harrington KM, Clark SL, et al. Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas. 2013;76(6):913–34.
Jackson DL, Gillaspy JA, Purc-Stephenson R. Reporting practices in confirmatory factor analysis: an overview and some recommendations. Psychol Methods. 2009;14(1):6–23.
Dunn TJ, Baguley T, Brunsden V. From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation. Br J Psychol (London, England : 1953). 2014;105(3):399–412.
Oelke U, Hundenborn G, Kühn C. Ausbildungsrichtlinie für staatlich anerkannte Kranken- und Kinderkrankenpflegeschulen in NRW; 2003.
Rausch M, Thelen K, Beudert I. Kompetenzprofil für die Logopädie. Frechen: Deutscher Bundesverband für Logopädie e.V. (dbl); 2014.
Nursing & Midwifery Council. Standards for competence for registered nurses. London; 2018.
Backhaus K, Blechschmidt B. Fehlende Werte und Datenqualität. Die Betriebswirtschaft. 2009;69(2):265.
Chapuis C, Chanoine S, Colombet L, et al. Interprofessional safety reporting and review of adverse events and medication errors in critical care. Ther Clin Risk Manag. 2019;15:549–56.
The authors thank Dr. Sarah Berger and Regina Poß-Doering, Department of General Practice and Health Services Research, University of Heidelberg, for their support in the translation and validation process. Dr. Hilary Bekker, Chair in Medical Decision Making, University of Leeds, UK (email@example.com), gave permission for RoCAES to be used and translated by the authors for this study, and was involved in clarifying issues around items and scoring during the translation process. We thank her for her support.
Open access funding provided by Projekt DEAL.
Ethics approval and consent to participate
The study was approved by the Ethics Committee of Medical Faculty Heidelberg (S049–2019) and complied with the Declaration of Helsinki. For the participation of clinical staff, permission of the respective staff council was asked. As the survey was conducted using an anonymous online questionnaire, participants gave their consent by actively sending their completed questionnaires to the study centre. All participants were informed about this procedure in writing and were given the opportunity to ask questions. The Ethics Committee of Medical Faculty Heidelberg confirmed this procedure.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Litke, N.A., Wensing, M., Miksch, A. et al. German translation and validation of the Reporting of Clinical Adverse Events Scale (RoCAES-D). BMC Health Serv Res 20, 689 (2020). https://doi.org/10.1186/s12913-020-05546-2
- Adverse events
- Incident reporting
- Patient safety
- Risk management
- Safety culture