Main findings and interpretation
Among the 498 adverse events identified by patient record review only 18 adverse events (3.6%) were identified by record review, meaning that the majority of 480 adverse events was not registered in one of the four registration systems.
Most of the adverse events (n = 17) were detected in either a professional's report (n = 9) or a patient's report (n = 8); only one adverse event was detected in both systems. Patient reports and healthcare professional reports contributed almost equally to the detection of adverse events. Since healthcare professionals overall made over five times more reports than patients did, we can conclude that there is a higher chance to find an adverse event in reporting systems of patient reports than in incident reporting systems for healthcare professionals. In incident reporting systems for professionals, most of the reports are made by nurses and, therefore, probably mainly concern nursing care. Adverse events, in contrast, often concern medical care provided by resident physicians and medical consultants. The fact that these professionals are the ones making few reports in reporting systems could explain the relatively low number of adverse event matches in incident reports. Patients, on the other hand, report about the whole sequence of care. Generally, both healthcare professionals and patients report about a broader group of incidents than mere adverse events. Often, their reports concern quality issues without patient harm.
Statistical calculations showed that the degree of preventability and severity of the adverse events does not influence the likelihood they are reported. The results showed no statistically significant differences. However the non-significance might be the result of the low power.
Patients and healthcare professionals reported relatively more preventable and severe adverse events compared to less preventable and severe adverse events. The degree of preventability of adverse events reported by healthcare professionals was relatively higher than of adverse events reported by patients. We found that healthcare professionals reported more preventable adverse events and patients more severe adverse events. The learning potential of the adverse event or its visible impact on the patient might contribute to reporting behaviour. But, unfortunately, not all highly preventable and severe adverse events are registered.
Because of the low number of adverse events detected in the reporting systems, under-reporting of especially preventable and severe adverse events remains to be a problem.
Patients made only a few medico legal claims. In the Netherlands, there is not a real claim culture. In addition, patients can only file a claim when they are informed about the possibility to financially redress the harm that is suffered.
On the whole, the results show that in order to detect the same adverse events identified by record review, one cannot rely on the existing reporting systems within hospitals at present.
Why are so few adverse events reported by healthcare providers or patients? Healthcare professionals might be embarrassed or afraid of condemnation by their colleagues or the hospital management after reporting adverse events they were involved in, especially those with severe consequences. Furthermore, there are other barriers mentioned in literature that influence event reporting, including: fear of disciplinary action or potential litigation, time pressure, no feedback, the perception that reporting is unnecessary, unclear reporting procedures and a lack of clear definitions as to what constitutes a reportable event [19–25]. Moreover, resident physicians and medical consults do not generally perceive (surgical) complications to be "reportable incidents". They address complications, among other incidents, in Mortality and Morbidity meetings (M&M) [26]. There are also a number of possible reasons why patients do not report adverse events [19]. Patients may not be aware they have sustained harm from medical care, while it is not easy to disentangle medical injury from the development of the underlying illness. Moreover, patients can be unaware of the possibilities of making a complaint or claim. Or they can be unwilling to do so, because they do not want to commit the time and energy needed to take action, they are concerned the report will bring tension into the relationship with their doctor or they do not feel the need to complain, because their doctor clearly explained the event to their satisfaction (disclosure). Moreover patients might not feel the need to file a claim because in the Netherlands the healthcare and social security systems are well developed and patients do not feel the need for financial compensation.
Finally, patients with grievances or concerns may choose to speak directly to their healthcare provider. Other patients that have experienced an adverse event can choose to step outside the hospital and submit an appeal to e.g. the medical disciplinary board or civil court.
The sensitivity and positive predictive value is low (3.6% and 4.6%), meaning that reports of patients and healthcare professionals are not useful as a predictive method to detect the same adverse events as a patient record study. Although specificity and negative predictive value seem high (93.3% and 91.5%), one has to bear in mind that an absence of a report does not imply that no adverse event occurred. We did not researched the types and consequences of the reports, therefore it might be possible that reports that did not match still concerns adverse events.
Although incidents, complaints and claims do not detect the same (number of) adverse events as the patient record review, reports can still be useful in identifying issues concerning patient safety. In the sample 441 patients were involved in one or more adverse events and 348 patients were involved in one or more report. With an overlap of 16 patients 332 patients were still involved in situations that healthcare professionals or patients found important enough to report.
Comparison with previous research
Different from our study design, other studies compared adverse events identified with record review with either incident reporting by healthcare professionals or patient complaints [13, 15, 16]. Olsen et al. compared local real-time record review with incident reporting and pharmacist surveillance [15]. As in our study, the authors found little overlap in events detected by the different methods. In a study by Sari et al., data of record review were compared with data submitted to a routine incident reporting system of the same patients [13]. They found that the reporting system missed most patient safety incidents that were identified by record review and detected only 5% of those incidents that resulted in patient harm (these incidents probably included adverse events). In our study, 2% of the adverse events that were identified by record review were reported by healthcare professionals. The authors of both studies concluded that incident reporting alone does not provide an adequate assessment of adverse events and recommend hospital staff and researchers to use more than one method at the same time [13, 15].
Bismark et al. compared results from a record review study with patient complaints: 0.4% of the adverse events resulted in complaints [16]. In our study, 1.8% of the adverse events were detected in complaints and claims. Bismark found that the odds of a complaint were higher for adverse events with higher preventability and severity [16]. Because of the low numbers we could not compare the results.
Strengths and limitations
No earlier study has compared record review with both registration systems of reports by healthcare professionals and patients (complaints as well as medico-legal claims). We studied a large number of adverse events and included multiple hospitals in our study design, increasing the likelihood that our results can be generalised to other hospitals.
Our study has, however, several limitations. Firstly, complaints and claims relating to episodes of care in 2004 may have been submitted later, outside our study period (January-December 2004), especially those that are submitted to the Complaint Committee and Liability Officer. Moreover, in some hospitals resident physicians and medical consultants also make reports of adverse outcomes of medical care during Morbidity and Mortality meetings (M&M). It was not possible to include this as a source in our study, because in some hospitals M&M reports are written down on notepads and are not formally registered. In other hospitals, there was only an M&M registration for surgery or not an M&M registration at all. Finally, patient record review has recognized limitations regarding the estimation of adverse event rates [27, 28]. Thomas et al. stated that estimates of adverse events rates from patient record review are highly sensitive to the degree of consensus and confidence among reviewers [28]. Therefore it is possible that reports contained adverse events that were not identified in the record review study. Because of these limitations, the number of adverse events that can be detected by patients and healthcare professionals might be underestimated.