To what extent are adverse events found in patient records reported by patients and healthcare professionals via complaints, claims and incident reports?

Incident reporting, complaints and claims in Dutch hospitals

In the Netherlands, healthcare professionals have been increasingly stimulated to report incidents or near misses within their own hospital. For the healthcare professional in 2004 reporting incidents (especially adverse events and calamities) was not mandatory by law but a consequence of healthcare quality and patients' rights legislation. It often was mandatory by employment contract. The incident reporting system is by definition meant for all incidents, including adverse events. An incident can be defined as "Any deviation from usual medical care that causes an injury to the patient or poses a risk of harm. Includes errors, preventable adverse events, and hazards." A near miss is defined as "Serious error or mishap that has the potential to cause an adverse event but fails to do so because of chance or because it is intercepted." [17]. This means that an adverse event is always an incident but an incident does not has to be an adverse event. After an incident/near miss has happened, it can be reported by filling out an electronic or paper-based report form containing, among other things, a description of the event, the time and place of occurrence and the people involved. The incident reporting committee will register and analyse the incidents.

Patients are encouraged to report their complaints without further definition.

In the Netherlands patients have various options to file a complaint or claim in the Netherlands when they are not satisfied with the care or cure they received. The choice of the path depends largely on the intention of the patient making the complaint or claim.

There are different reasons for patients to complaint, for example, patients want to have an explanation or an apology or initiate an investigation on the legitimacy of certain acts committed. Reasons to file a claim are for example to get financial compensation or to prevent recurrence of the incident to restore their sense of justice [18]. Patients can submit their grievances to the complaints officer or to the more formal complaint committee within the hospital. The purpose of an informal complaint is mediation or expressing a concern about the quality of care, whereas a formal complaint is made to instigate an investigation followed by a formal judgement about the legitimacy of the complaint (not juridical binding). A legal option outside the hospital is to submit a formal appeal to the medical board to obtain a verdict or when a financial compensation is wanted, to file a claim to the hospital board. The Netherlands does not have a no-fault system. Malpractice claims are judged by the insurance company of the hospital. If patients do not agree with the judgement of the insurance company on the liability or the financial compensation, patients can approach a civil court. It is possible for patients to use more than one path simultaneously or consecutively. Reporting systems are not set up with the intention to report adverse events in specific. Patient reports often concern the quality of care, whereas healthcare professionals frequently report about deviations from procedures.

Study design and setting

A retrospective study was performed using a database of reviewed patient records collected during a retrospective patient record review study examining the incidence of adverse events [1, 9]. This record review study was conducted in 21 hospitals, involving 20% of the acute care hospitals in the Netherlands. All 21 hospitals were invited to participate in our supplementary study on the completeness of incident reporting systems. Fourteen hospitals were willing to participate: two university hospitals, four tertiary teaching hospitals and eight general hospitals. Hospital sizes ranged from 201 to 985 beds. Seven hospitals declined because of practical reasons, such as simultaneous participation in other projects, time constraints, elimination of the reporting system after two years or having an anonymous reporting system (no patient information registered).

Four reporting systems per hospital were linked with the adverse events database in these fourteen hospitals: 1) the reporting system of the complaint officer: for informal complaints by patients; 2) the reporting system of the complaint committee: for formal complaints by patients; 3) the reporting system of the liability officer: for medico-legal claims by patients and 4) the reporting system of incident reports: for incident reports by healthcare professionals. For each adverse event identified in the patient records, the equivalent was sought in the four reporting systems of the same hospitals by comparing the date and descriptions of the events.

This study is a continuation of the Dutch patient record review study [1, 9], for which ethical approval was granted by the VU University Medical Center in Amsterdam. The participating hospitals formally consented to take part in this study.

Patient record review

In each hospital a stratified random sample was selected of 200 admissions of patients discharged from the hospital in 2004 (> 24 hours stay) and 200 (or less if the total of patients who died in 2004 was lower) admissions of patients deceased in the hospital in the year 2004, excluding admissions of psychiatry, obstetrics and children less than one year old.

Between August 2005 and October 2006, 55 trained physicians reviewed the medical, nursing and, if available, the outpatient record of all sampled admissions that contained triggers for adverse events, for example an unplanned readmission, unplanned return to the operating room or unexpected death. The presence of one or more of the 18 predefined triggers was judged in advance by trained nurses. For each patient record two physician reviewers determined independently the presence of one or more, consequences, and degree of preventability of the adverse events, based on a standardised procedure and review form. A preventable adverse event was defined as an adverse event resulting from an error in management due to failure to follow accepted practice at an individual or system level. Accepted practice was taken to be 'the current level of expected performance for the average practitioner or system that manages the condition in question'. The degree of preventability was measured on a six-point scale from "(Virtually) no evidence of preventability" to "(Virtually) certain evidence of preventability".

The degree of severity of the consequences was rated on a seven-point scale from "No physical impairment or disability" to "Death". The methods of determining adverse events were based on the well-known protocol of The Harvard Medical Practice Study and have been described in detail elsewhere [1].

Linking record reviews with reporting systems (finding patient identity matches)

Between March 2007 and March 2008 fourteen hospitals provided datasets from each of the four reporting systems, containing identification characteristics of all patients involved in an incident report, complaint or claim in the year 2004: patient registration number, date of birth and sex and hospital. Reports without sufficient patient identifiers or reports of events which occurred in 2003 but were reported in 2004 were non-eligible for matching. Because of the confidentiality of the information, the researchers had no access to the reporting systems in the hospitals. Sometimes exceptions were made when the incident reports, complaints or claims were not digitally registered. In these cases, the researchers got permission to access the reports to get the information required. A confidentiality agreement was signed by the researchers to maintain secrecy of the information. Data from the reporting systems were recorded anonymously and all information was kept confidential.

The patient identification data from the four sources were then linked with the database of the sample from the record review. A case matched when a patient involved in the complaint, claim or incident report matched on identity with a patient from the record review sample. These matches were classified as patient identity matches (see Figure 1).

Comparison of narratives (finding adverse event matches)

It is possible that a complaint, claim or incident report - hereafter together named "reports"- concerned another issue than the adverse event identified by record review. For example, relatives of a patient complaining about how the doctors communicate with the patient and not about the adverse event that involved a wrong dose of medication. To be able to examine whether the identity matches related to the same issue, we obtained more information about these events, including the original description of the event, the date of occurrence and consequences for the patient, by means of a questionnaire.

Furthermore we obtained information about the characteristics of the reporter (patient or healthcare professional). The questionnaires were sent to the officer of each reporting system in the hospitals. A researcher (ICD) compared the dates and descriptions of the adverse events from the record reviews with the dates and descriptions/narratives of the reports. The descriptions of the events did not need to correspond perfectly. The physicians who reviewed the patient records made a medical description of the adverse event, while patients generally do not use professional language when describing their dissatisfaction. Moreover, an adverse event can evolve from a chain of events in time, while a report often describes just a part of the situation. Allowing for the above mentioned differences, cases were classified as a match if it was plausible that: 1) the descriptions were about the same event or 2) the description in the report was about an event that was related to the adverse event (for example, the adverse event was a consequence of the reported event). The comparisons were discussed with other researchers (LZ, CW). In doubt and for the final verification a physician was consulted (MD).

Matched cases in this final stage-meaning that the topic of the report was related to an adverse event in the record review study-were classified as adverse event matches (see Figure 1).

Statistical analysis

Usefulness of the reporting systems for detection of adverse events was determined by the number of adverse event matches (absolute number and percentage of all possible matches). The extent of overlap between reporting systems was determined by counting the number of adverse events that were detected in more than one reporting source. The characteristics of the reporters of each report were described in percentages of the following categories: professionals (physician, resident/physician, nurse and student nurse) and patients (patient, child, spouse, child & spouse and legal adviser).

To test whether the degree of preventability and severity of the adverse events influence the likelihood they are reported, we performed t-tests. Results were considered statistically significant at p < 0.05. For the non-significant differences we calculated the power. Data were analysed using SPSS 15.0.

Each patient was identified as being positive or negative for an adverse event for one or more reports that matched an adverse event in the patient record review. Sensitivity and specificity interpret the reports results retrospectively whereas positive predictive values and negative predictive values establish the predictive properties of the reports in the future.

Main findings and interpretation

Among the 498 adverse events identified by patient record review only 18 adverse events (3.6%) were identified by record review, meaning that the majority of 480 adverse events was not registered in one of the four registration systems.

Most of the adverse events (n = 17) were detected in either a professional's report (n = 9) or a patient's report (n = 8); only one adverse event was detected in both systems. Patient reports and healthcare professional reports contributed almost equally to the detection of adverse events. Since healthcare professionals overall made over five times more reports than patients did, we can conclude that there is a higher chance to find an adverse event in reporting systems of patient reports than in incident reporting systems for healthcare professionals. In incident reporting systems for professionals, most of the reports are made by nurses and, therefore, probably mainly concern nursing care. Adverse events, in contrast, often concern medical care provided by resident physicians and medical consultants. The fact that these professionals are the ones making few reports in reporting systems could explain the relatively low number of adverse event matches in incident reports. Patients, on the other hand, report about the whole sequence of care. Generally, both healthcare professionals and patients report about a broader group of incidents than mere adverse events. Often, their reports concern quality issues without patient harm.

Statistical calculations showed that the degree of preventability and severity of the adverse events does not influence the likelihood they are reported. The results showed no statistically significant differences. However the non-significance might be the result of the low power.

Patients and healthcare professionals reported relatively more preventable and severe adverse events compared to less preventable and severe adverse events. The degree of preventability of adverse events reported by healthcare professionals was relatively higher than of adverse events reported by patients. We found that healthcare professionals reported more preventable adverse events and patients more severe adverse events. The learning potential of the adverse event or its visible impact on the patient might contribute to reporting behaviour. But, unfortunately, not all highly preventable and severe adverse events are registered.

Because of the low number of adverse events detected in the reporting systems, under-reporting of especially preventable and severe adverse events remains to be a problem.

Patients made only a few medico legal claims. In the Netherlands, there is not a real claim culture. In addition, patients can only file a claim when they are informed about the possibility to financially redress the harm that is suffered.

On the whole, the results show that in order to detect the same adverse events identified by record review, one cannot rely on the existing reporting systems within hospitals at present.

Why are so few adverse events reported by healthcare providers or patients? Healthcare professionals might be embarrassed or afraid of condemnation by their colleagues or the hospital management after reporting adverse events they were involved in, especially those with severe consequences. Furthermore, there are other barriers mentioned in literature that influence event reporting, including: fear of disciplinary action or potential litigation, time pressure, no feedback, the perception that reporting is unnecessary, unclear reporting procedures and a lack of clear definitions as to what constitutes a reportable event [19–25]. Moreover, resident physicians and medical consults do not generally perceive (surgical) complications to be "reportable incidents". They address complications, among other incidents, in Mortality and Morbidity meetings (M&M) [26]. There are also a number of possible reasons why patients do not report adverse events [19]. Patients may not be aware they have sustained harm from medical care, while it is not easy to disentangle medical injury from the development of the underlying illness. Moreover, patients can be unaware of the possibilities of making a complaint or claim. Or they can be unwilling to do so, because they do not want to commit the time and energy needed to take action, they are concerned the report will bring tension into the relationship with their doctor or they do not feel the need to complain, because their doctor clearly explained the event to their satisfaction (disclosure). Moreover patients might not feel the need to file a claim because in the Netherlands the healthcare and social security systems are well developed and patients do not feel the need for financial compensation.

Finally, patients with grievances or concerns may choose to speak directly to their healthcare provider. Other patients that have experienced an adverse event can choose to step outside the hospital and submit an appeal to e.g. the medical disciplinary board or civil court.

The sensitivity and positive predictive value is low (3.6% and 4.6%), meaning that reports of patients and healthcare professionals are not useful as a predictive method to detect the same adverse events as a patient record study. Although specificity and negative predictive value seem high (93.3% and 91.5%), one has to bear in mind that an absence of a report does not imply that no adverse event occurred. We did not researched the types and consequences of the reports, therefore it might be possible that reports that did not match still concerns adverse events.

Although incidents, complaints and claims do not detect the same (number of) adverse events as the patient record review, reports can still be useful in identifying issues concerning patient safety. In the sample 441 patients were involved in one or more adverse events and 348 patients were involved in one or more report. With an overlap of 16 patients 332 patients were still involved in situations that healthcare professionals or patients found important enough to report.

Comparison with previous research

Different from our study design, other studies compared adverse events identified with record review with either incident reporting by healthcare professionals or patient complaints [13, 15, 16]. Olsen et al. compared local real-time record review with incident reporting and pharmacist surveillance [15]. As in our study, the authors found little overlap in events detected by the different methods. In a study by Sari et al., data of record review were compared with data submitted to a routine incident reporting system of the same patients [13]. They found that the reporting system missed most patient safety incidents that were identified by record review and detected only 5% of those incidents that resulted in patient harm (these incidents probably included adverse events). In our study, 2% of the adverse events that were identified by record review were reported by healthcare professionals. The authors of both studies concluded that incident reporting alone does not provide an adequate assessment of adverse events and recommend hospital staff and researchers to use more than one method at the same time [13, 15].

Bismark et al. compared results from a record review study with patient complaints: 0.4% of the adverse events resulted in complaints [16]. In our study, 1.8% of the adverse events were detected in complaints and claims. Bismark found that the odds of a complaint were higher for adverse events with higher preventability and severity [16]. Because of the low numbers we could not compare the results.

Strengths and limitations

No earlier study has compared record review with both registration systems of reports by healthcare professionals and patients (complaints as well as medico-legal claims). We studied a large number of adverse events and included multiple hospitals in our study design, increasing the likelihood that our results can be generalised to other hospitals.

Our study has, however, several limitations. Firstly, complaints and claims relating to episodes of care in 2004 may have been submitted later, outside our study period (January-December 2004), especially those that are submitted to the Complaint Committee and Liability Officer. Moreover, in some hospitals resident physicians and medical consultants also make reports of adverse outcomes of medical care during Morbidity and Mortality meetings (M&M). It was not possible to include this as a source in our study, because in some hospitals M&M reports are written down on notepads and are not formally registered. In other hospitals, there was only an M&M registration for surgery or not an M&M registration at all. Finally, patient record review has recognized limitations regarding the estimation of adverse event rates [27, 28]. Thomas et al. stated that estimates of adverse events rates from patient record review are highly sensitive to the degree of consensus and confidence among reviewers [28]. Therefore it is possible that reports contained adverse events that were not identified in the record review study. Because of these limitations, the number of adverse events that can be detected by patients and healthcare professionals might be underestimated.