Because we personally visited all hospital units to collect the data, we could observe that the questionnaire was met with interest – but generally with less enthusiasm from physicians than from others. The questionnaire was not regarded as threatening. Only two units of the 49 approached declined the invitation to participate, and only one of them because it did not want to go on record at this moment, the other was a laboratory unit which found the generic patient safety questionnaire irrelevant to their tasks. The response rate was relatively high (68% – among physicians, however, only 52%), and, as shown in Additional file 1, very few items produced a large number of missing responses. The outstanding exception was "I experience good collaboration with pharmacists in this clinical area", which had a missing rate of 20%. In our hospital, pharmacists do not participate in daily procedures in care-giving areas; their cooperation with the units is limited to more or less annual inspections. The reason why many have not responded to this item is probably that they found it irrelevant.
A number of respondents asked how to understand the item "Fatigue impairs my performance during emergency situations (e.g. emergency resuscitation, seizure)". Their comments have convinced us that the translation into Norwegian of this item should be reformulated and should not read "Slitenhet reduserer måten jeg opptrer på i krisesituasjoner (som resuscitering, anfall o.l.)" but "Jeg arbeider dårligere i krisesituasjoner (som resuscitering, anfall o.l.) når jeg er sliten".
The questionnaire was not very time-consuming. In all clinical units at the Akershus University Hospital we observed that most responders completed the questionnaire within the 10–15 minutes suggested by the SAQ technical reports, and all respondents finished within 20–25 minutes.
A data collection challenge was to ensure that all those who participated in patient care at the care-giving units were invited to participate in the data collection. The problem was that many physicians and physiotherapists were not employed by any specific unit and therefore did not attend unit staff meetings. These caregivers had to be reached in their own professions' group meetings.
The relatively high response rate, low number of missing data and the relatively short completion time testify to the acceptability of the SAQ in the Norwegian setting. One item, however, stood out as a candidate for removal, since not many Norwegian clinical workers cooperate directly with pharmacists – in fact, one may wonder why not many more than 20% of the responders left the question of the quality of their cooperation with pharmacists unanswered.
Responses were, for most – but not all – items skewed towards the positive end of the scale. But the response distributions did not suggest that any particular item or set of items should be removed for failing to reflect variation.
All items were, as they should be, more strongly correlated with their own factor than with any of the others.
The relatively high Cronbach alphas for all hypothesised factors demonstrates the internal consistency of the factors: all alphas were between 0.71 and 0.85 – except for the factor Teamwork climate, but its alpha of 0.68 was not much below the recommended limit of 0.70.
The stability of the questionnaire also proved acceptable: the test-retest intraclass correlation coefficients of the factors were relatively high – except for factors Stress recognition (0.55) and Perceptions of hospital management (0.44). A possible interpretation is that in the average clinical worker's eyes, the hospital's top management is so distant that it is difficult to maintain a stable perception of its qualities. The fact that the test-retest correlation for Perceptions of hospital management was practically zero for non-physicians, but quite high (0.83) for physicians lends credibility to that interpretation. The relatively low retest stability of the Stress recognition score, too, was due to the low correlation for non-physicians, whose stress load may feel much more variable and beyond control than the physicians'. The striking difference in the three-week test-retest intraclass correlation coefficients between physicians and others may indeed be seen as suggesting that checking a questionnaire's reliability by the stability of the responses to it is more appropriate among staff who are likely to feel reasonably in command of their work. The items made no reference to the length of the period to be taken into consideration when ticking the questionnaire, and for those more easily subject to the variable demands of those higher in the hospital hierarchy, work must be expected to be appear more variable. Users of the Norwegian translation might want to double-check the test-retest reliability of this factor, and interpret this factor score with due regard to its stability.
The construct validity of the questionnaire, as judged by the goodness-of-fit indicators from the confirmatory factor analysis, can be considered acceptable, but less than perfect. Some of the goodness-of-fit indices speak against the fit of the model to the data, namely the p-value of less than 0.001 and the AGFI of 0.871. But the χ2-value (2.583) was within the limits indicated by Wheaton et al and Carmines and MacIver. And the pclose (0.893) and the RMSEA (0.048) both exceeded the criteria suggested by Browne and Cudeck , and the Hoelter 0.05-value of 296 was above the critical value given by Hoelter.
The questionnaire cannot be regarded as externally validated until more hospitals have been surveyed and the results from similar units can be compared and related to patient outcomes. However, our informal impression from our feeding the results back to the clinical units and from our presentation of the results to the hospital's top management and to its Quality department is that the responding units seemed to feel not surprised by their SAQ-scores, and that the hospital top management and Quality department felt the scores were credible. Department average scores also correlated with the frequency of adverse hospital events (as determined by Global trigger tool revision of patient records) and with department average patient reports on general satisfaction with hospitalization, worries about possible maltreatment, and evaluation of the smoothness of hospital work.