This article has Open Peer Review reports available.
Estimating the incidence of adverse events in Portuguese hospitals: a contribution to improving quality and patient safety
© Sousa et al.; licensee BioMed Central Ltd. 2014
Received: 14 September 2013
Accepted: 11 July 2014
Published: 18 July 2014
Several review studies have shown that 3.4% to 16.6% of patients in acute care hospitals experience one or more adverse events. Adverse events (AEs) in hospitals constitute a significant problem with serious consequences and a challenge for public health. The occurrence of AEs in Portuguese hospitals has not yet been systematically studied. The main purpose of this study is to estimate the incidence, impact and preventability of adverse events in Portuguese hospitals. It is also our aim to examine the feasibility of applying to Portuguese acute hospitals the methodology of detecting AEs through record review, previously used in other countries.
This work is based on a retrospective cohort study and was carried out at three acute care hospitals in the Administrative Region of Lisbon. The identification of AEs and their impact was done using a two-stage structured retrospective medical records review based on the use of 18 screening criteria. A random sample of 1,669 medical records (representative of 47,783 hospital admissions) for the year 2009 was analyzed.
The main results found in this study were an incidence rate of 11.1% AEs, of which around 53.2% were considered preventable. The majority of AEs were associated with surgical procedures (27%), drug errors (18.3%) and hospital acquired infections (12.2%). Most AEs (61%) resulted in minimal or no physical impairment or disability, and 10.8% were associated with death. In 58.6% of the AEs’ cases, the length of stay was prolonged on average 10.7 days. Additional direct costs amounted to €470,380.00.
The magnitude of these results was critical, reinforcing the need of more detailed studies in this area. The knowledge of the incidence and nature of AEs that occur in hospitals should be seen as a first step towards the improvement of quality and safety in health care.
Adverse events (AEs) occur with alarming frequency in healthcare. These events represent significant losses from a clinical, economic and social perspective [1–3]. To learn from these events and improve safety, they must be identified, measured and their causes found. Healthcare providers and researchers are searching for an accurate, reliable and low cost method to identify and measure AEs in hospital and other settings. The method developed in the Harvard Medical Practice Study, in the 90s, is the one most often used for national AE studies .
Many countries, like Portugal, have taken initiatives over the past decade to address safety problems in health care [3, 5, 7, 8]. The US report “To Err is Human” marked an acceleration in programmes and actions for increased patient safety initiated by health care policy makers, health care professionals and managers . As far as we know there have been no studies to date on the occurrence, nature, preventability and impact of AEs in Portuguese hospitals.
Studies based on the “Harvard method” have been carried out in several countries with different results, finding evidence of 3.4% to 16.6% of patients in acute care hospitals experiencing one or more AEs [10–17]. The results of these studies give data on a critical aspect of hospital performance and stimulate patient safety improvement of each country.
The aims of this study were to estimate the incidence, nature, preventability, cost and impact of adverse events in Portuguese hospitals. This was an exploratory study that also intended to examine the feasibility of applying to Portuguese acute hospitals the methodology previously used in other countries.
This work followed a retrospective cohort study design. The methods were based on the protocol used in the Harvard Medical Practice Study [4, 11, 18] with modifications introduced in subsequent studies undertaken in the United Kingdom, New Zealand, Canada, the Netherlands, Sweden, Brazil, and, more recently, in a sample of 26 hospitals from eight developing and transitional countries [10–12, 15–18].
Screening criteria and number of medical records showing evidence of one category of adverse event
Hospital-incurred patient injury
Unplanned readmission after discharge from index admission (12 months)
Unplanned admission related to previous healthcare management
Hospital-acquired infection or sepsis
Adverse drug reaction
Unplanned return to the operating room
Any other undesirable outcomes not covered in this list of criteria
Unplanned transfer from general care to intensive care
Other patient complications
Cardiac or respiratory arrest
Unplanned removal, injury or repair of an organ during surgery
Unplanned transfer to another acute care hospital
Development of neurological deficit not present on admission
Inappropriate discharge to home
Dissatisfaction or correspondence indicating litigation
Injury related to abortion or delivery
Dissatisfaction with care documented in the patient’s medical record
In the first stage, a group of six nurses (two from each hospital, with a minimum of five years experience in clinical audits) assessed the medical records, looking for the presence of, at least, one of the 18 criteria for the presence of a potential adverse event. In stage two, a group of five physicians (one Cardiologist, one Neurologist, two Surgeons, one Internal Medicine, with a minimum of five years experience in clinical codes and in clinical audits) reviewed each positive record in order to confirm the presence of an adverse event, to estimate its impact and determine its preventability, according to the definition established previously. The degree of agreement between the reviewers in each stage was calculated (assessed on a random sample of 10% of medical records) by using kappa coefficient. Neither nurses nor doctors knew the previous classification of their colleagues.
The timing of the AEs in relation to the hospital admission is an important methodological issue. We considered AEs that occurred during the index hospital admission and that were detected during either the index or subsequent hospital admissions over the following 12-month period (in the same hospitals).
Similarly to other studies, physicians estimated, based on evidence in the medical record and their professional judgment, the impact of AEs in two different ways: i) the degree of physical impairment or disability at discharge (minimal, moderate or permanent), or death; ii) the number of additional hospital days directly attributable to AEs.
The costs related to the additional length of stay were estimated based on official accounting data from all the NHS hospitals, providing information on daily costs. This value includes hospital daily costs, namely physicians' and nursing staff, lab tests and exams, medication, housing and overhead .
Using professional judgment and based on the information of medical records, the physician reviewers also classified the preventability of each AE using a six-point scale (1- virtually no evidence of preventability; 2- slight to modest evidence of preventability; 3- Preventability not quiet likely less than 50/50, but “close to call”; 4-preventability more than likely 50/50; 5- strong evidence of preventability; and 6 -virtually certain evidence of preventability. The preventability of an adverse event was considered with a score > = 4).
SPSS (version 19) was used for data processing and for statistical analysis. The study was approved by the Ethics Committee of participating hospitals.
Degree of physical impairment or disability at discharge
Minimal impairment or disability, recovery within 1 month
Moderate impairment or disability, recovery within 1-12 months
Permanent impairment or disability
Unable to determine
Total (of AEs)
We found that most of the patients, 109/186 (58.6%), who experienced AEs incurred extra bed days in hospital (a total of 1,166 extra days, an average of 10.7 days per patient, ranging from 1-70 days), with additional total costs of €470,380.00 for all three hospitals together.
Level of preventability of the AEs
Virtually no evidence of preventability
46.8% not preventable
Slight to modest evidence of preventability
Preventability not quite likely (less than 50/50, but “close call”)
Preventability more than likely (more than 50/50 but “close call”
Strong evidence of preventability
Virtually certain evidence of preventability
Reliability of assessment between reviewers in the first and the second screening
Presence of one or more positive criteria in the medical record analyzed
0.63 (83.5% agreement)
AE confirmed by physician
0.78 (89% agreement)
Level of preventability (scale 1–6)
0.58 (79% agreement)
We used retrospective medical record review following the HMPS methodology in order to assess the nature, incidence, and clinical and/or economic impact of adverse events and to provide some information on their causes. Studies in many countries have followed the same methodology and have come to broadly similar conclusions [10–12, 14–17]. Rates of AEs in most recent studies lie between 8% and 12%, a range now accepted as being common in the healthcare systems of developed countries.
In this study, we found an incidence of AEs of 11.1% of which 53.2% were considered as preventable. The assessment of preventability is one challenge for these studies, and indicates the potential gains to be achieved by improvements. For this reason, analysis of each case is needed to decide prevention strategies. On the other hand, the classification of preventable AEs, while using a clear criterion and standard, still involves a subjective element and may vary with the expertise, practical experience of the physician and the way the data is registered in the medical record. Some authors argue that knowing the outcome and its severity may influence the judgment of causation and preventability and that the bias element is likely to be in the direction of overestimation of the rate of “preventable” AEs (hindsight bias) [20–22]. Nevertheless, our results are similar to the findings of previous studies, particularly those of the UK (incidence10.8% and 52% preventable), New Zealand (incidence 11.3% and 61.6% preventable) and the Danish study (incidence 9.0% and 40.4% preventable) [10, 11, 23].
One finding was that 59.2% of patients who experienced an AE were 65 years old or older. Relatively little attention has been paid to patient safety in older people although they are particularly vulnerable to healthcare error and harm [24, 25]. Moreover, older people are more likely to suffer from multiple health conditions, receive multiple treatments, and stay longer in hospital. A longer stay increases the risk of all complications of hospitalization. In the last decades more people are living longer and this trend is likely to continue. For all these reasons, it is our opinion that safety for older patients should receive higher priority in research and in safety prevention strategies.
The majority of AEs (61%) were not documented as causing serious consequences for the patient. They did not result in any significant physical impairment or disability, and were resolved during the admission or, within one month from discharge. This is a similar percentage to that reported in other studies [3, 10, 12, 13, 17]. However, a meaningful proportion of patients died (10.8%) or experienced a permanent disability as a result of their AEs (5.4%).
The economical impact of adverse events has been gaining particular attention in the last years [26, 27]. The report of the Institute of Medicine estimates that, in the United States, the total national costs associated with adverse events represent approximately 4% to 6% of national health expenditures . In Britain, the cost of preventable AEs, in lost bed days alone, was estimated in one billion (Pounds) per year . In the Netherlands (the Dutch adverse events study) the authors concluded that 3% of all bed days and 1% of the total health budget could be attributed to preventable AEs (costs of direct hospital care, mainly additional time in hospital) . In our study, most of the patients (58.6%) who experienced AEs prolonged the length of stay in hospital on average 10.7 days, with additional direct costs of €470,380.00. Extrapolating this estimated value to the population of the study (all admission of the three hospitals in 2009) the costs varied between €1,290.310 and €1,691.643. The overall real costs are higher, as these estimates do not include additional treatments in ambulatory care or other hospitalizations (which are related to the same AE), costs to patients (e.g. medication in ambulatory) or any societal costs (e.g. absence from work, premature death). The costs of prevention, however, may be significant, especially if organizations or their health systems do not have effective implementation capacity .
In addition to the costs of unnecessary suffering caused to patients and families, the costs and savings of adverse events are a critical issue in Portugal, which will be experiencing a challenging economic situation for at least the next five years  . As with some other European countries, the economic rationale is increasing for selecting effective strategies and implementing them in a context which rewards safety improvement . Including an economic dimension to the knowledge gained from research into quality and safety is an important part of the health system reform that is being implemented in Portugal [5, 12, 29, 30]. Moreover, it could be a starting point for specific interventions in the improvement of patient safety and it may help to prioritise research areas for the near future [6, 20, 27, 31].
There are limitations to the study. These include those applying to retrospective studies, such as information bias and hindsight bias [15, 21, 22]. We observed that the quality of patient records could be better, although this did not significantly limit the study. Retrospective patient record studies still represent the “gold-standard” method for assessing incidence and monitoring the frequency of AEs [13, 20, 31, 32] and the use of similar methods to those used elsewhere allows comparisons to be made. However, for a more comprehensive assessment of safety, other methods also need to be used, including reporting and learning systems, root-cause analyses, failure mode effects analyzes and morbidity and mortality reviews [13, 33].
The reliability of the review process is another critical element in these studies [34, 35]. In our study, the level of agreement (Kappa statistic – k value) between nurses was considered good - substantial agreement (k value 0.63) and is related to the identification of the presence of at least one positive criterion in the first screening. This value is in line with those found in other studies [11–13, 15]. Among doctors (in the second screening), the reliability of determination of AEs was also considered good - substantial agreement with k value of 0.78 and fair concerning their preventability, with k value of 0.58. For the second screening, the k value is slightly higher than those found in other studies undertaken at national level (e.g. Canada, New Zealand, Sweden and The Netherlands). These results must be carefully interpreted, due to the small number of reviewers, cases and hospitals involved in this study [36, 37].
In Portugal, there is an overall awareness and a growing concern about patient safety issues. This study suggests that AEs in three Portuguese hospitals affect nearly one in ten patients and results in considerable avoidable suffering and economic costs. With local evidence of the size of the problem, staff is more motivated to act, especially if effective interventions to reduce adverse events can be selected and implemented to target those AEs that are prioritized, namely those resulting from surgical procedures, drug errors and health-acquired infections.
If interventions can be implemented and demonstrated to be cost-effective, then more healthcare managers and policy makers may begin to view improvements in safety as an investment rather than an expense. This can also speed changes in incentives so as to reward safety and quality. Local evidence of the size and nature of the problem and its clinical and cost impact is one of the first steps toward this desirable change.
We would like to thank everyone who contributed to the study, namely the staff of the three participating hospitals who facilitated the access to patient records and the nurses and physicians for reviewing the medical records. This work was supported by the Fundação Calouste Gulbenkian.
- Thomas E, Petersen L: Measuring errors and adverse events in health care. J Gen Intern Med. 2003, 18: 61-67. 10.1046/j.1525-1497.2003.20147.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Landrigan CP, Parry GJ, Bones CB, Hackbarth AD, Goldmann DA, Sharek PJ: Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2012, 363 (22): 2124-2134.View ArticleGoogle Scholar
- de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA: The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care. 2008, 17: 216-223. 10.1136/qshc.2007.023622.View ArticlePubMedPubMed CentralGoogle Scholar
- Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, Hebert L, Newhouse JP, Weiler PC, Hiatt H: The nature of adverse events in hospitalized patients: results of the Harvard medical practice study II. N Engl J Med. 1991, 324: 377-384. 10.1056/NEJM199102073240605.View ArticlePubMedGoogle Scholar
- Sousa P, Uva A, Serranheira F, Pinto F, Øvretveit J, Klazinga , Sunol R, Terris D: The patient safety journey in Portugal: challenges and opportunities from a public health perspective. Revista Portuguesa de Saúde Pública. 2009, Especial 25 anos: 91-106.Google Scholar
- Hauck K, Zhao X, Jackson T: Adverse event rates as measures of hospital performance. Health Policy. 2012, 104: 146-154. 10.1016/j.healthpol.2011.06.010.View ArticlePubMedGoogle Scholar
- Sousa P: Patient safety: a necessidade de uma estratégia nacional. Acta Med Port. 2006, 19: 309-318.PubMedGoogle Scholar
- Garrouste-Orgeas M, Philippart F, Bruel C, Max A, Lau N, Misset B: Overview of medical errors and adverse events. Annals of Intensive Care. 2012, 2: 2-10.1186/2110-5820-2-2. doi:10.1186/2110-5820-2-2View ArticlePubMedPubMed CentralGoogle Scholar
- Institute of Medicine (IOM): To err is human: building a safer health system. 2000, Washington, DC: National Academy PressGoogle Scholar
- Vincent C, Neale G, Woloshynowych M: Adverse events in British hospitals: preliminary retrospective record review. BMJ. 2001, 322: 517-519. 10.1136/bmj.322.7285.517.View ArticlePubMedPubMed CentralGoogle Scholar
- Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S: Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J. 2002, 115: 203-209.Google Scholar
- Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, Etchells E, Ghali WA, Hébert P, Majumdar SR, O'Beirne M, Palacios-Derflingher L, Reid RJ, Sheps S, Tamblyn R: The Canadian adverse events study: the incidence of adverse events among hospital patients in Canada. CMAJ. 2004, 170: 1678-1686. 10.1503/cmaj.1040498.View ArticlePubMedPubMed CentralGoogle Scholar
- Michel P, Quenon JL, de Sarasqueta AM, Scemama O: Comparison of three methods for estimating rates of adverse events and rates of preventable adverse events in acute care hospitals. BMJ. 2004, 328: 199-10.1136/bmj.328.7433.199. doi:http://dx.doi.org/10.1136/bmj.328.7433.199 View ArticlePubMedPubMed CentralGoogle Scholar
- Aranaz-Andrés JM, Aibar-Remón C, Vitaller-Murillo J, Ruiz-López P, Limón-Ramírez R, Terol-García E: Incidence of adverse events related to health care in Spain: results of the Spanish National Study of Adverse Events. J Epidemiol Community Health. 2008, 62 (12): 1022-1029. 10.1136/jech.2007.065227.View ArticlePubMedGoogle Scholar
- Zegers M, de Bruijne MC, Wagner C, Hoonhout LH, Waaijman R, Smits M, Hout FA, Zwaan L, Christiaans-Dingelhoff I, Timmermans DR, Groenewegen PP, van der Wal G: Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Safe Health Care. 2009, 18: 297-302. 10.1136/qshc.2007.025924.View ArticleGoogle Scholar
- Soop M, Fryksmark U, Köster M, Haglund B: The incidence of adverse events in Swedish hospitals: a retrospective medical record review study. Int J Qual Health Care. 2009, 22: 324-333.Google Scholar
- Mendes W, Martins M, Rozenfeld S, Travassos C: The assessment of adverse events in hospitals in Brazil. Int J Qual Health Care. 2009, 2: 279-284.View ArticleGoogle Scholar
- Wilson RM, Michel P, Olsen S, Gibberd RW, Vincent C, El-Assady R, Rasslan O, Qsous S, Macharia WM, Sahel A, Whittaker S, Abdo-Ali M, Letaief M, Ahmed NA, Abdellatif A, Larizgoitia I: Patient safety in developing countries: retrospective estimation of scale and nature of harm to patients in hospitals. BMJ. 2012, 324: e832-doi:10.1136/bmj.e832View ArticleGoogle Scholar
- da Saúde M: ACSS. Contabilidade analítica dos hospitais do SNS. 2011, Administração Central do Sistema de Saúde: LisboaGoogle Scholar
- Zegers M, De Bruijne MC, Spreeuwenberg P, Wagner C, Van Der Wal G, Groenewegen PP: Variation in the rates of adverse events between hospitals and hospitals departments. Int J Qual Health Care. 2011, 23 (2): 126-133. 10.1093/intqhc/mzq086.View ArticlePubMedGoogle Scholar
- Zegers M, de Bruijne MC, Wagner C, Groenewegen PP, Waaijman R, van der Wal G: Design of a retrospective patient record study on the occurrence of adverse events among patients in Dutch hospitals. BMC Health Serv Res. 2007, 7: 27-34. 10.1186/1472-6963-7-27.View ArticlePubMedPubMed CentralGoogle Scholar
- Henriksen K, Kaplan H: Hindsight bias, outcome knowledge and adaptive learning. Qual Saf Health Care. 2003, 12 (Suppl 2): ii46-ii50.PubMedPubMed CentralGoogle Scholar
- Schioler T, Lipczak H, Pedersen BL: Incidence of adverse events in hospitals: a retrospective study of medical records: Danish study of adverse events. Scan J Public Health. 2000, 32: 324-333.Google Scholar
- Vincent C: Patient Safety. 2010, Oxford: Wiley-Blackwell, 2View ArticleGoogle Scholar
- Tsilimingras D, Rosen AK, Berlowitz DR: Patient safety in geriatrics: a call for action. J Gerontol. 2003, 58: 813-819. 10.1093/gerona/58.9.M813.View ArticleGoogle Scholar
- Brown P, McArthur C, Newby L, Lay-Yee R, Davis P, Briant R: Cost of medical injury in New Zealand: a retrospective cohort study. J Health Serv Res Policy. 2002, 7 (Suppl 1): S29-S34. 10.1258/135581902320176449.View ArticlePubMedGoogle Scholar
- Øvretveit J: Does Improving Quality Save Money?: A Review of Evidence of which Improvements to Quality reduce Costs to Health Service Providers. 2009, London: The Health FoundationGoogle Scholar
- Commission E: The Economic Adjustment Programme for Portugal. June 2011, European Commission: BrusselsGoogle Scholar
- Parand A, Benn J, Burnett S, Pinto A, Vincent C: Strategies for sustaining a quality improvement collaborative and its patient safety gains. Int J Qual Health Care. 2012, 24 (4): 380-390. 10.1093/intqhc/mzs030.View ArticlePubMedGoogle Scholar
- Ovretveit J, Klazinga N: Learning from large-scale quality improvement through comparisons. Int J Qual Health Care. 2012, 24 (5): 463-469. 10.1093/intqhc/mzs046.View ArticlePubMedGoogle Scholar
- O'Leary KJ, Devisetty VK, Patel AR, Malkenson D, Sama P, Thompson WK, Landler MP, Barnard C, Williams MV: Comparison of traditional triggers tool to data warehouse based screening for identifying hospital adverse events. BMJ Qual Saf. 2012, 22 (2): 130-138.View ArticlePubMedGoogle Scholar
- Hogan H, Healey F, Neale G, Thomson R, Vincent C, Black N: Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual Saf. 2012, 21 (9): 737-745. 10.1136/bmjqs-2011-001159.View ArticlePubMedPubMed CentralGoogle Scholar
- Parry G, Cline A, Goldmann D: Deciphering harm measurement. JAMA. 2012, 307: 2155-2156.View ArticlePubMedGoogle Scholar
- Zegers M, de Bruijne MC, Wagner C, Groenewegen PP, van der Wal G, de Vet HC: The inter-rater agreement of retrospective assessment of adverse events does not improve with two reviewers per patient record. J Clin Epidemiol. 2010, 63: 94-102. 10.1016/j.jclinepi.2009.03.004.View ArticlePubMedGoogle Scholar
- Naessens JM, O'Byrne TJ, Johnson MG, Vansuch MB, McGlone CM, Huddleston JM: Measuring hospital adverse events: assessing inter-rater reliability and trigger performance of the global trigger tool. Int J Qual Health Care. 2010, 22 (4): 266-274. 10.1093/intqhc/mzq026.View ArticlePubMedGoogle Scholar
- Viera A, Garrett J: Understanding interobserver agreement:the Kappa Statistic. Fam Med. 2005, 37 (5): 360-363.PubMedGoogle Scholar
- de Vet HC, Mokkink LB, Terwee CB, Hoekstra OS, Knol DL: Clinicians are right not to like Cohen’s K. BMJ. 2013, 12: 346:f2125-doi:10.1136/bmj.f2125Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/14/311/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.