This article has Open Peer Review reports available.
Public health communications and alert fatigue
- Janet G Baseman†1, 2Email author,
- Debra Revere†2,
- Ian Painter†2,
- Mariko Toyoji†2,
- Hanne Thiede†3 and
- Jeffrey Duchin†1, 3
© Baseman et al.; licensee BioMed Central Ltd. 2013
Received: 10 December 2012
Accepted: 27 June 2013
Published: 5 August 2013
Health care providers play a significant role in large scale health emergency planning, detection, response, recovery and communication with the public. The effectiveness of health care providers in emergency preparedness and response roles depends, in part, on public health agencies communicating information in a way that maximizes the likelihood that the message is delivered, received, deemed credible and, when appropriate, acted on. However, during an emergency, health care providers can become inundated with alerts and advisories through numerous national, state, local and professional communication channels. We conducted an alert fatigue study as a sub-study of a larger randomized controlled trial which aimed to identify the most effective methods of communicating public health messages between public health agencies and providers. We report an analysis of the effects of public health message volume/frequency on recall of specific message content and effect of rate of message communications on health care provider alert fatigue.
Health care providers enrolled in the larger study (n=528) were randomized to receive public health messages via email, fax, short message service (SMS or cell phone text messaging) or to a control group that did not receive messages. For 12 months, study messages based on real events of public health significance were sent quarterly with follow-up telephone interviews regarding message receipt and topic recall conducted 5–10 days after the message delivery date. During a pandemic when numerous messages are sent, alert fatigue may impact ability to recall whether a specific message has been received due to the “noise” created by the higher number of messages. To determine the impact of “noise” when study messages were sent, we compared health care provider recall of the study message topic to the number of local public health messages sent to health care providers.
We calculated the mean number of messages that each provider received from local public health during the time period around each study message and provider recall of study message content. We found that recall rates were inversely proportional to the mean number of messages received per week: Every increase of one local public health message per week resulted in a statistically significant 41.2% decrease (p < 0.01), 95% CI [0.39, .87] in the odds of recalling the content of the study message.
To our knowledge, this is the first study to document the effects of alert fatigue on health care providers’ recall of information. Our results suggest that information delivered too frequently and/or repetitively through numerous communication channels may have a negative effect on the ability of health care providers to effectively recall emergency information. Keeping health care providers and other first-line responders informed during an emergency is critical. Better coordination between organizations disseminating alerts, advisories and other messages may improve the ability of health care providers to recall public health emergency messages, potentially impacting effective response to public health emergency messages.
“Information chaos”—various combinations of information overload, information underload, information scatter, information conflict and erroneous information—is an increasingly cited factor affecting the effectiveness, quality and safety of clinical care . By contributing to increased mental workload and decreased situational awareness in the work place, information chaos may also be a factor in alert fatigue; i.e., health care providers (HCPs) missing or ignoring important messages within the volume of information they have been conditioned to perceive as irrelevant . Previous alert fatigue studies have focused on clinical decision support systems (CDS) and computerized provider order entry (CPOE) systems which aim to improve efficiency and quality of care. These studies have investigated how CDS and CPOE systems alert providers to potential adverse medication events and interactions [3, 4], the frequency and factors associated with overriding CPOE alerts [2, 5], recommended design approaches for mitigating adverse drug events and alert fatigue [6, 7], and unintended consequences of these systems in contributing to information and mis-information overload, role confusion, excessive errors and alert fatigue . In addition to the clinical system messages, HCPs receive public health alerts and advisories, clinical guidelines and updates, and training notifications from professional associations, agencies and organizations  and through a system of national, state and local public health communication channels .
HCPs serve as partners with public health in surveillance and case investigation activities by providing case reporting and clinical management, preventing excess deaths, treating the injured and mitigating suffering . For example, HCPs were the first to recognize early cases of inhalation anthrax in 2001  and first to identify dengue outbreaks in Hawaii, Florida and Texas between 2001–2011 . In an emergency, public health agencies rely on HCPs as both frontline responders and as trusted and preferred communicators of health information to the public during an emergency [14–16].
Ensuring that timely, accurate, trustworthy and context-relevant information can be “heard” by HCPs within their information chaotic environments is essential to an effective public health communication program [17, 18]. However, while their intention is to reduce information chaos, the growing variety of public health agency and other messaging systems may be increasing, rather than reducing, communication challenges for HCPs . Poor coordination between agencies can contribute to inconsistent and/or conflicting information as well as redundant messaging between public health agencies and HCPs, as demonstrated by communication issues identified in the 2008 Salmonella Saint Paul outbreak response . Similar issues were recently documented by a process analysis and survey conducted with primary care providers in Utah following the first wave of the 2009 influenza pandemic. A majority of providers stated that: 1) they received an overwhelming amount of email from public health authorities as well as health care organizations and 2) they desired less frequent, more concise and locally relevant emergency communications during a public health emergency . Despite recognition of these problems, little is known about the potential for HCP alert fatigue from public health communications.
The reach (Rapid Emergency Alert Communication in Health) study is a randomized controlled trial to systematically evaluate and compare the effectiveness of mobile (SMS) and traditional (email, fax) communication strategies for sending public health messages to health care providers—physicians, pharmacists, nurse practitioners, physician assistants and veterinarians. The study aims to identify which communication modality is most effective for dissemination of health alerts and advisories between public health agencies and HCPs in order to improve emergency preparedness and response. The findings reported in this paper are a reach sub-study involving one study site. The objective of the analysis described in this paper is to measure the effects of public health message volume on recall of specific message content as a measure of HCP alert fatigue.
Study protocols were approved by the UW Institutional Review Board (Minimal Risk Category 7).
Design and setting
The reach study is a multi-site randomized control trial conducted by the University of Washington (UW) across three sites in partnership with local (King County, WA and Spokane County, WA) or state (State of Montana) public health agencies. The findings reported in this paper are a reach sub-study involving the King County, WA study site. Between October 2009 and January 2011, the reach study was conducted in King County, WA, the most populous county in Washington State, and the 14th most populous in the United States . HCPs were recruited and randomized to receive public health messages via email, fax, short message service (SMS delivered to a cell phone) or to a control group that did not receive messages. HCPs were blindly randomized regardless of ability to receive the group intervention; for example, HCPs who did not provide a fax number at enrollment could be blindly randomized into the fax group. Investigators, data analysts and interviewers were blinded to randomization groups.
Demographic characteristics of study participants in King County
“Noise” data collected
Listserv subscription of King County health care providers
Messages received per week from listserv*
Data analysis approach
All analyses were conducted using R v2.10.0 . Basic HCP demographic data were compiled based on HCP information provided at enrollment and interview questions regarding age and gender (Table 1). Proportions of HCPs recalling receipt of study messages and correctly recalling study message topics with and without prompting were calculated using responses from follow-up interviews.
Data analysis methods
To examine the main effect of message rate on alert fatigue, we analyzed the effects of message rate on the probability of correctly recalling content of study messages among those who remembered receiving messages. A HCP was classified as having correctly recalled the reach study message topic if the HCP: 1) Recalled receiving a message without prompting and the topic correctly matched the reach study message topic; or 2) Did not recall receiving the “fake” topic but did recall the reach study message topic. To discount the effects of “agreeability bias”, HCPs who recalled receiving a message on the “fake” topic and required prompting for the study message topic were not counted as correctly recalling receipt of the study message. All other responses were classified as not correctly recalling the message.
Analysis was conducted using generalized linear models (GLM) with Binomial outcomes and the logit link function with exchangeable autocorrelations within providers using generalized estimating equations (GEE) in using the function ‘gee’ in the ‘gee’ package. Robust standard errors were used to calculate z scores and p-values. Data from the first message were not included in this analysis because questions prompting HCP recall with the study message topic were only used on interviews for messages 2–5. Message number was included in the model to control for differences in recall or “memorability” of the study message topic.
Exploratory data analysis of the data indicated that one of the (blinded) randomization groups was obviously the control group due to a large failure rate in recall. For the alert fatigue analysis and results that follow, the control group was excluded since the alert fatigue analysis focused on an HCP’s ability to recall whether our message was received amidst the “noise” of other messages they received and there was no mechanism by which message rate could influence recall of the study messages. Two covariates other than message rate— message number and (blinded) randomization group—were included in the main analysis. Our analysis investigated: 1) HCP rates of message recall for both prompted and unprompted responses; 2) estimated effect of increases in message rate on prompted recall; and 3) differences in the effect of message rate on recall between provider types, gender, age and randomization group (message delivery method).
Rates of message recall by message and outcome type
Mean number of messages received per week by participants belonging to any listserv.
Recall content of study message (coded open ended question) among those that could recall content
“Fake”/Distractor topic recalled when study topic was recalled
Correct recall of study message by prompting or by open ended response among participants who recall receiving any message
Correct recall of study message by prompting or by open ended response, over all participants interviewed
Effect of message rate on alert fatigue
Analysis of prompted recall
weekly message rate
Within participant correlation: 0.10
The inclusion of the additional interaction term reduced the residual deviance by 0.38, 2.80 and 0.08 respectively on 4, 1 and 1 degrees of freedom; these reductions are all small in magnitude and none are statistically significant. An analysis of randomization group on alert fatigue demonstrated no effect of communication channel on alert fatigue.
To our knowledge, this is the first study to document the effects of alert fatigue on HCPs’ recall of information. The number of local public health messages significantly decreased the odds of HCPs correctly remembering the receipt and content of a study message, and suggests that alert fatigue and information overload may inhibit the ability of HCPs to respond effectively to messages during a public health emergency when volume of messages may be higher. The higher than expected recall of a “fake” topic, despite its irrelevance to any recent public health message, may indicate a degree of HCP role confusion or uncertainty around public health messages or it could be an outcome related to information chaos in the health care setting.
Information delivered too frequently and/or repetitively through multiple communication channels also may negatively affect the ability of HCPs to recall public health messages. It should be noted that six months prior to the date of this study, King County HCPs on local public health listservs were exposed to high volumes of health alerts and advisories during the initial onset of the 2009 H1N1 pandemic. This “noise” may have desensitized HCPs to information delivered through the public health emergency message system. We also did find a general trend of improvement in recall of message receipt and recall of content over time, which could be attributed to reduced message volume in the time period before the study message and could indicate that HCPs recovered from the high volume of messages sent during the H1N1 pandemic.
It might seem logical that an electronic health record (EHR) that provides contextually relevant information to clinicians would also provide a delivery mechanism for population-based information. However, it is far from clear that this information would not become part of the clinical “noise” and contribute to increased information chaos in the clinical environment. Alert fatigue studies conducted in clinical environments have documented that over 90% of medication safety/drug-drug interaction alerts are overridden in electronic decision support systems without a physician even looking at the alert ; pharmacists have also been found to override one-third of life-threatening drug-drug interactions . Low alert specificity, unclear information content, perceived lack of relevance, information overload and unnecessary disruptions to workflow are cited as reasons for ignoring alerts . While the opportunity to leverage EHRs to disseminate public health alerts and advisories is promising, more research is needed to identify the most effective means of delivering these messages to HCPs, as well as how to communicate with HCPs who do not work in an environment with an EHR.
Also more research is needed to identify public health message sources of greatest value to HCPs. Anecdotal evidence from open-ended interview responses suggested that at least some HCPs received messages from other sources, such as the CDC, FDA, FEMA, as well as professional organizations such as the Washington State Medical Association and internal employer communications during the study period. We were unable to accurately document these additional message sources, although source of message may impact the likelihood of message recall.
Our study has several limitations.
Study messages, while timely, were not emergency alerts and, thus, may have been perceived as less relevant, important or “memorable” than other public health agency messages HCPs received during the same time frame. In addition, we varied the day of the week that study messages were sent. It is possible that messages sent on certain days are better recalled by HCPs and that our messages were sent on lower recall days. To study this phenomenon would require communication of multiple study messages on each day of the week—a wholly different study than we conducted—thus we do not have data to determine whether HCPs experience lower or higher recall days.
The primary explanatory variable of interest, message rate, is an uncontrolled variable. Our ability to estimate the effect of message rate depends on our ability to separate out the effects of alert fatigue from the “memorability” effects of individual messages. This depends on the variability in the number of messages received between participants, and a large part of this variability comes from differences based on public health listserv membership. To the extent that systematic differences exist between providers who do and do not belong to public health listservs, biases may exist in our estimates. We identified HCPs who subscribed to public health listservs by comparing listserv and study contact information. It is possible that our listserv identification was not accurate as HCP contact information may have changed or HCPs may have subscribed to a listserv after the study began, which in turn means our estimates of the rate at which messages were received by participants may have been underestimated. The effect of such bias would be to underestimate the effect size of message rate on message recall.
We also are unable to document whether reach subjects actually received the local public health listserv communications or whether a listserv message was shared with other HCPs; we can only document that a public health listserv message was sent to the HCP. This highlights an issue for public health messaging through lists—whether email, fax or SMS—in general. Most public health listservs require HCPs to opt in to receiving messages and the burden of maintaining correct contact information rests on public health agency staff. A clinic, hospital or other health care organization that maintains a directory of provider contact information may be more a more efficient mechanism for disseminating public health messages and communicating emergency information to its HCPs. However, this raises the question of whether a public health message has the authority or novelty to be “heard” within the usual clinical noise and would need further investigation.
While this study contributes to the evidence base for improving public health agency emergency communications with HCPs, we cannot correlate failure to recall message topic with inability to act on the information received, such as HCPs communicating information further downstream by, for example, sharing email with colleagues or disseminating information to patients or the public. Recent disasters have highlighted effective communication as essential in public health agencies’ ability to protect populations during emergencies; these events can increase information chaos in the health care environment. More research is needed to understand the impacts of the method of message delivery, the source of the message, the type of information being disseminated and the health care organizational context to guide or improve the practice of communication between public health agencies and HCPs before, during and after a public health emergency.
Keeping health care providers and other first-line responders informed during an emergency is critical. Implementing measures to avoid redundant and conflicting messages from multiple agencies may help reduce alert fatigue. Delivering public health alerts and advisories too frequently and/or repetitively may have a negative effect on the ability of HCPs to effectively recall and, potentially, to respond to public health emergency messages. Our results suggest that public health agencies should consider the potential impacts of timeliness of information and frequency of communications in emergency planning to reduce the potential for alert fatigue and information chaos during public health emergencies.
We thank our collaborators at Public Health – Seattle & King County and the research core at the University of Washington’s Northwest Preparedness and Emergency Response Research Center (NWPERRC). The authors would like to thank the article reviewers for their insightful comments and suggestions. This work was made possible by the CDC Office of Public Health Preparedness and Response (OPHPR) Extramural Research Program Preparedness and Emergency Response Research Centers (PERRCs) award, Grant no. 5P01TP000297, to the University of Washington. The contents of this paper are solely the responsibility of the authors and do not necessarily represent the official views of the CDC.
- Beasley JW, Wetterneck TB, Temte J, Lapin JA, Smith P, Rivera-Rodriguez AJ, Karsh BT: Information chaos in primary care: Implications for physician performance and patient safety. J Am Board Fam Med. 2011, 24 (6): 745-751. 10.3122/jabfm.2011.06.100255.View ArticlePubMedPubMed CentralGoogle Scholar
- Van der Sijs H, Aarts J, Vulto A, Berg M: Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assn. 2006, 13 (2): 138-147. 10.1197/jamia.M1809.View ArticleGoogle Scholar
- Avery AJ, Savelyich BS, Sheikh A, Cantrill J, Morris CJ, Fernando B, Bainbridge M, Horsfield P, Teasdale S: Identifying and establishing consensus on the most important safety features of GP computer systems: e-Delphi study. Inform Prim Care. 2005, 13 (1): 3-12.PubMedGoogle Scholar
- Smithburger PL, Buckley MS, Bejian S, Burenheide K, Kane-Gill SL: A critical evaluation of clinical decision support for the detection of drug-drug interactions. Expert Opin Drug Saf. 2011, 10 (6): 871-882. 10.1517/14740338.2011.583916.View ArticlePubMedGoogle Scholar
- Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS: Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003, 163 (21): 2625-2631. 10.1001/archinte.163.21.2625.View ArticlePubMedGoogle Scholar
- Lee EK, Mejia AF, Senior T, Jose J: Improving patient safety through medical alert management: An automated decision tool to reduce alert fatigue. AMIA Annu Symp Proc. 2010, 417-421.Google Scholar
- Phansalkar S, Edworthy J, Hellier E, Seger DL, Schedlbauer A, Avery AJ, Bates DW: A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems. J Am Med Inform Assoc. 2010, 17 (5): 493-501. 10.1136/jamia.2010.005264.View ArticlePubMedPubMed CentralGoogle Scholar
- Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH: Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc. 2007, 26-30.Google Scholar
- Revere D, Nelson K, Thiede H, Duchin J, Stergachis A, Baseman J: Public health emergency preparedness and response communications with health care providers: A literature review. BMC Publ Health. 2011, 11: 334-10.1186/1471-2458-11-334.View ArticleGoogle Scholar
- Baker EL, Porter J: The management moment - The Health Alert Network: Partnerships, politics, and preparedness. J Public Health Man. 2005, 11 (6): 574-576.Google Scholar
- Gerberding JL, Hughes JM, Koplan JP: Bioterrorism preparedness and response - Clinicians and public health agencies as essential partners. Jama-J Am Med Assoc. 2002, 287 (7): 898-900. 10.1001/jama.287.7.898.View ArticleGoogle Scholar
- Jernigan JA, Stephens DS, Ashford DA, Omenaca C, Topiel MS, Galbraith M, Tapper M, Fisk TL, Zaki S, Popovic T, Meyer RF, Quinn CP, Harper SA, Fridkin SK, Sejvar JJ, Shepard CW, McConnell M, Guarner J, Shieh WJ, Malecki JM, Gerberding JL, Hughes JM, Perkins BA, Anthrax Bioterrorism Investigation Team: Bioterrorism-related inhalational anthrax: The first 10 cases reported in the United States. Emerg Infect Dis. 2001, 7 (6): 933-944. 10.3201/eid0706.010604.View ArticlePubMedPubMed CentralGoogle Scholar
- Adalja AA, Sell TK, Bouri N, Franco C: Lessons learned during dengue outbreaks in the United States, 2001–2011. Emerg Infect Dis. 2012, 18 (4): 608-614.View ArticlePubMedPubMed CentralGoogle Scholar
- Chen FM, Hickner J, Fink KS, Galliher JM, Burstin H: On the front lines: family physicians’ preparedness for bioterrorism. J Fam Pract. 2002, 51 (9): 745-750.PubMedGoogle Scholar
- Kahan E, Fogelman Y, Kitai E, Vinker S: Patient and family physician preferences for care and communication in the eventuality of anthrax terrorism. Fam Pract. 2003, 20 (4): 441-442. 10.1093/fampra/cmg419.View ArticlePubMedGoogle Scholar
- Markenson D, Reynolds S: The pediatrician and disaster preparedness. Pediatrics. 2006, 117 (2): e340-e362. 10.1542/peds.2005-2752.View ArticlePubMedGoogle Scholar
- Freimuth VS: Order out of chaos: The self-organization of communication following the anthrax attacks. Health Commun. 2006, 20 (2): 141-148. 10.1207/s15327027hc2002_5.View ArticlePubMedGoogle Scholar
- Lurio J, Morrison FP, Pichardo M, Berg R, Buck MD, Wu W, Kitson K, Mostashari F, Calman N: Using electronic health record alerts to provide public health situational awareness to clinicians. J Am Med Inform Assoc. 2010, 17 (2): 217-219. 10.1136/jamia.2009.000539.View ArticlePubMedPubMed CentralGoogle Scholar
- Taylor E, Kastner J, Renter D: Challenges involved in the Salmonella Saintpaul outbreak and lessons learned. J Public Health Manag Pract. 2010, 16 (3): 221-231.View ArticlePubMedGoogle Scholar
- Staes CJ, Wuthrich A, Gesteland P, Allison MA, Leecaster M, Shakib JH, Carter ME, Mallin BM, Mottice S, Rolfs R, Pavia AT, Wallace B, Gundlapalli AV, Samore M, Byington CL: Public health communication with frontline clinicians during the first wave of the 2009 influenza pandemic. J Public Health Manag Pract. 2011, 17 (1): 36-44.View ArticlePubMedPubMed CentralGoogle Scholar
- U.S. Census Bureau: State and County QuickFacts: Population Estimates, Counties. 2011, U.S: Census BureauGoogle Scholar
- R Development Core Team: R Foundation for Statistical Computing, Vienna, Austria. R: A language and environment for statistical computing, reference index version 2.2.1. 2009, http://www.R-project.org.Google Scholar
- Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW, Avery AJ: What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior?. J Am Med Inform Assoc. 2009, 16 (4): 531-538. 10.1197/jamia.M2910.View ArticlePubMedPubMed CentralGoogle Scholar
- Bates DW, Kuperman GJ, Wang S, Gandhi T, Kittler A, Volk L, Spurr C, Khorasani R, Tanasijevic M, Middleton B: Ten commandments for effective clinical decision support: Making the practice of evidence-based medicine a reality. J Am Med Inform Assoc. 2003, 10 (6): 523-530. 10.1197/jamia.M1370.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/13/295/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.