Skip to main content
Download PDF

Adaptation of the brainwriting premortem technique to inform the co-creation of COVID-19 testing strategies in underserved communities in South San Diego

BMC Health Services Research volume 24, Article number: 12 (2024) Cite this article

Abstract

Introduction

Meaningful engagement of partners in co-creating and refining health-related programs can increase the initial uptake, sustained implementation, broad reach, and effectiveness of these programs. This is especially important for underserved communities where resources are limited and need to be prioritized. Brainwriting premortem is a novel qualitative approach to partner engagement that combines the strengths of individual idea generation with the concept of premortem exercise that addresses failure points prior to the implementation of new programs.

Methods

An adapted form of brainwriting premortem was used to inform iterative refinements to a COVID-19 testing program at a Federally Qualified Health Center (FQHC) in San Diego. Patients and providers from the FQHC participated in interviews at two time points (early- and mid-implementation of the program). Interview data were transcribed, translated, and analyzed using a rapid qualitative approach. Key themes and sub-themes were identified and used to inform refinements to the program.

Results

A total of 11 patients (7 Spanish- and 4 English-speaking) and 8 providers participated in the brainwriting premortem interviews. Key themes related to possible reasons for COVID-19 testing program failure: advertising/sharing information; access to testing; handling of test results; staff and patient safety; patient beliefs and views regarding the SARS-CoV-2 virus; and COVID-19 testing options offered. Proposed solutions were offered for the key failures except for patient beliefs and views regarding the SARS-CoV-2 virus. Additional solutions offered were related to education, physical operations, and recruitment strategies. Real-time changes to the program flow and components were made in response to 7 suggestions from patients and 11 from providers. Changes related to the process of returning results were the most common, and included sending results via email with distinct workflows based on the test result.

Conclusion

The implementation of the adapted brainwriting premortem technique allowed us to incorporate the perspective of key partners in the delivery and iterative refinement of the COVID-19 testing program. This was an effective tool in the context of an FQHC and can be a promising and approach to incorporate iterative input from patients and providers to ensure successful program implementation. Future studies, particularly those requiring rapid response to public health emergencies, should consider the use of this technique.

Peer Review reports

Contributions to the literature

  • Meaningful engagement of partners in co-creating and refining programs to address important public health priorities can greatly increase the initial uptake, sustained implementation, broad reach, and effectiveness of these programs.

  • Our paper describes an adapted form of an innovative approach, brainwriting premortem to engage patients and providers in a federally qualified health center to refine the workflow for a COVID-19 testing program.

  • The adapted brainwriting premortem proved to be a feasible approach to implement at multiple time-points to gain perspectives from multiple partners and supported an iterative improvement of the program protocol.

Introduction

Widening health disparities among underservedcommunities such as Latino/a, Black, Indigenous, and people of color (BIPOC) were experienced with the COVID-19 pandemic [1]. These communities showed lower testing and vaccination rates compared to white individuals in the United States and were significantly more likely to experience mortality and morbidity from COVID-19 [2]. Further inflating the likelihood of ongoing transmission in these communities was reduced access to testing resources [3,4,5,6,7,8]. Meaningful engagement of partners in co-creating and refining programs to address important public health priorities can greatly increase the initial uptake, sustained implementation, broad reach, and effectiveness of these programs [9, 10]. Engagement is especially important when programs are offered in settings that serve historically under-represented and culturally diverse communities, often with limited resources that need to be prioritized [11, 12]. When responding to public health emergencies, the need for rapid action in the context of uncertainty and lack of definitive evidence means that program development is best undertaken through an iterative approach. The initial program is created based on the best available evidence and program elements are refined over time as additional knowledge is gained and feedback is received on implementation strategies from partners [13,14,15].

The Community-driven Optimization of COVID-19 testing to Reach and Engage underserved Areas for Testing Equity—in Women and Children (CO-CREATE) program is a partnership between investigators in an academic institution and a Federally Qualified Health Center (FQHC). The CO-CREATE program was designed to be responsive to the needs of the community and offers no-cost COVID-19 testing to patients and community members at the clinic site of the partnering (FQHC). Throughout our work we engaged with a multidisciplinary Community and Scientific Advisory Board to guide our program development and implementation [11, 16].

A number of approaches are available to support partner engagement [17] (e.g., intervention mapping and implementation mapping [18], human-centered design approaches [19], qualitative systems mapping [20], rapid process improvement workshops [21], etc. In many cases, the use of multiple engagement strategies is required to achieve the most comprehensive understanding of perspectives from all partners, which can then support the creation of programs with the best fit to local priorities and resources [22]. There are few innovative techniques that seek input from diverse partners at multiple time points and allow for iterative and rapid improvement of a program. These techniques support bringing programs into practice rapidly that highlight the stakeholder’s voice. Brainwriting and the premortem technique are two innovative engagement methods that are gaining attention in public health implementation [23].

Brainwriting was developed in the context of marketing to provide an alternative to the traditional group-based brainstorming approach [24]. This technique involves asynchronous brainstorming where people contribute ideas independently using a method of writing down all of their ideas on a topic in a short period of time while in a group setting. Individual ideas are then shared with others in the group to expand on or to add new ideas. Brainwriting combines the strength of individual idea generation with the strength of group exchange [24]. The premortem technique has been used in the creation of new products, technology, and programs to predict risk for failure and prevent such failures from happening prior to launch [25]. The process involves assuming that the new product, technology, or program failed and works backwards to identify factors that might have led to this failure. A novel technique that combines these two approaches, brainwriting premortem, builds on a combination of individual wisdom and group problem-solving to identify potential failure points for the implementation of a program and possible solutions for these failure points [23]. Brainwriting premortem has been used recently in the context of implementation science to inform the refinement of interventions based on input from multiple partners [26,27,28]. While traditional approaches to the brainwriting premortem include in-person group-based activities, this was not feasible during the early stages of the COVID-19 pandemic for members of underserved communities or frontline healthcare providers due to the increased care demands on families and on providers at clinics. Our team adapted the traditional brainwriting premortem approach to an individually engaged format, both in-person and virtually, where community members and frontline providers provided their perspectives on a COVID-19 testing program at two separate time points. This paper describes the approach, key findings from the process, and how this information was used to refine our testing program.

Methods

CO-CREATE is funded through the NIH Rapid Acceleration of Diagnostics-for Underserved Populations (RADx-UP) initiative to understand practices, barriers, and facilitators to the access and uptake of COVID-19 testing and follow-up for members residing in an underserved community in South San Diego near the U.S./Mexico border. The primary deliverable of CO-CREATE was the design and implementation of a no-cost COVID-19 testing program that was responsive to the needs of the community. Data for this manuscript were extracted from the larger CO-CREATE research study (described in the next section).

CO-CREATE COVID-19 testing program

The CO-CREATE program was established in May 2021 and continues to offer free COVID-19 testing to patients and community members at the clinic site of the partnering FQHC. Community members can decide to participate in the research component of the program or simply access the testing services. Participating in the CO-CREATE research component involves completing a survey and a COVID-19 test. Upon arrival to the testing site, interested participants register to be part of the study by providing a valid form of identification and their demographics (name, date of birth, phone number, email, address, race/ethnicity). The study is then described to them with an opportunity to ask questions, and informed consent is obtained. Participants can return for repeat COVID-19 testing up to a total of 55 times per calendar year with no appointments needed. Eligibility criteria include speaking English or Spanish, providing informed consent or having a surrogate provide consent. There are no age restrictions; for children 7 years and older, child assent is obtained. This study is conducted under a protocol approved by the University of California San Diego Institutional Review Board.

COVID-19 testing workflow

The initial workflow for COVID-19 testing was co-created by the research and clinical partners prior to the launch of the CO-CREATE testing program. The workflow was presented from two different perspectives to the two types of participants in the brainwriting process – one for patients (Fig. 1) and one for providers (Fig. 2). The two versions of the workflow contained information about the activities involved in the testing process which included sample self-collection, informed consent, completion of study surveys and return of results. However, they differed in how they presented the details of these activities, focusing on the patient or provider perspective and emphasizing the aspects of the program the specific participant would experience. A written narrative and narrated video describing each version of the workflow were also created in English (for providers) and in English and Spanish (for patients). The workflow was updated by the research team when major modifications were made based on feedback from research staff or patients and providers during interviews.

Fig. 1
figure 1

Workflow for COVID-19 testing program for patients

Full size image
Fig. 2
figure 2

Workflow for COVID-19 testing program for providers

Full size image

Brainwriting premortem interviews (present study)

Patients and providers were interviewed in two phases: Phase 1- the early implementation phase from February 2021 – June 2021 and Phase 2—the mid-implementation phase from October 2021 – December 2021. Patients and providers were recruited using a variety of methods, including presentations at clinic staff meetings, patient contact lists, and on-site recruitment at the COVID-19 testing site. The brainwriting premortem interview guide (Table 1) was developed based on methodology by Gilmartin et al [23]. and refined for this study with three main sections: 1) general background about the participant; 2) brainwriting premortem exercise to identify key reasons why the COVID-19 testing program as described would fail; and 3) solutions for the identified COVID-19 testing program failures. To be eligible as a patient participant, individuals were required to be adults (18 years or older), speak Spanish or English, and be either a clinic patient or someone who was a close contact, family member of a clinic patient. To be eligible as a provider participant, individuals needed to be adults (18 years or older) and employed as a clinical provider or administrator at any of the clinic’s primary care facilities. Participants were provided a $40 gift card for completing an interview. Interviews were conducted in-person or virtually using Zoom in Spanish or English, lasted 15–30 min, and were recorded. CO-CREATE study staff conducted the interviews after participating in a training led by experts in qualitative data collection methods. The training included brainwriting premortem literature review, interactive interview practice sessions, and developing detailed protocols. All interviewers observed the first interview (with a provider) as part of their training that was conducted by an experienced clinical research interviewer. The team met after to discuss and refine the process.

Table 1 Brainwriting Premortem Interview Guide
Full size table

After consenting, participants provided general background information and were then introduced to the appropriate version of the COVID-19 testing workflow (patient or provider; Figs. 1 and 2). The testing flow was presented visually while a pre-recorded video of the interviewer was played to verbally describe the process. Participants were asked to imagine that the testing program failed and to think about why it failed. They were asked to list all potential reasons for the program’s failure and then to identify what they thought were the top three reasons for failure. Interviewees were then asked to identify possible solutions to their reasons for failure, as well as anything else they wanted to share or general comments. Interviewers took notes during the interviews about actionable solutions that could be implemented immediately (e.g., posting location and time of testing offerings on social media sites). The COVID-19 testing program workflow was modified for Phase 2 based on solutions that were presented during Phase 1 interviews as well as research staff recommended changes from early implementation. Phase 2 interviews (October 2021 – December 2021) showed participants this updated workflow but the interview questions remained the same.

Analysis

After interviews were completed, the recordings were professionally translated and transcribed. Data cleaning was completed by research staff. A rapid thematic analysis approach was used to identify overarching themes from the interview transcripts. Rapid thematic analysis is an applied but still rigorous method that can be used to produce actionable and targeted information on a shorter and more pragmatic timeline than traditional thematic analysis. This approach can be used in circumstances when there are highly structured and defined deliverables and the information gained from the analysis is meant to inform specific actions (i.e., refining a program to address concerns raised by diverse partners) [29,30,31].

Coders were trained by an expert in this data analysis approach. Interviews were initially double-coded using pre-determined codes based on the interview guide and reviewed together to build consensus and resolve discrepancies between coders. A second round of coding was conducted by the lead coder (BR) who synthesized all codes and identified additional subthemes and topics which were then verified by secondary coders (LA and KC). Data were organized by respondent type (patient and provider) and time point (Phase 1: early implementation and Phase 2: mid implementation) to explore differences and similarities across individuals and time points.

Results

Characteristics of patient and provider participants

A total of 11 patient interviews (5 in the early-implementation phase; all conducted in Spanish) and 6 in the mid-implementation phase (2 conducted in Spanish, 4 in English) and 8 provider interviews (4 in each phase; all were conducted in English) were completed. Participants between the two phases did not overlap. Patients reported being clinic patients for a few months to 20 years, and 3 reported not being a clinic patient and visiting the clinic for the purpose of testing. Patients reported seeing a variety of providers and number of clinic visits ranged from rarely to as many as 10–11 per month. Approximately half of the patients reported a decrease in visits since the COVID-19 pandemic started, half reported an increase in visits, and one person reported no change. It was also noted that the types of visits changed with the COVID-19 pandemic (e.g., more telehealth visits compared to in person visits).

Providers reported their tenure as ranging from 1 to 35 years in Phase 1 (mean = 14 years) and 1 to 16 years in Phase 2 (mean = 6 years). Fifty percent of Phase 1 providers were in the Pediatrics department, while 25% were in OBGYN, and 25% were in another department. Seventy-five percent of providers in Phase 2 were in Nursing and 25% in Adult Medicine. Most providers in Phase 1 were Physicians (75%) and 25% were in Clinical Administration, while in Phase 2, most were Clinical Administration (75%) and 25% were Clinical Staff. Providers reported between 40–100 patient visits per week pre-pandemic with most providers noting a decrease in in-person patient visits since the start of the COVID-19 pandemic due to an increase in telehealth visits. It was also noted that adult in-person visits were more impacted compared to pediatric visits and that in-person visits with RNs were higher compared to provider visits.

Top reasons for COVID-19 testing program failure

Some of the themes and subthemes span across both implementation phases and participant groups. The 6 main themes that emerged from the reasons for the potential failures of the COVID-19 testing program in both early and mid-implementation were advertising/sharing information, handling of test results, staff and patient safety and testing options offered. In contrast, patient beliefs and views of the SARS-CoV-2 virus emerged in early implementation only and access to testing emerged in mid-implementation. (Table 2). Access to testing was the most commonly mentioned failure by patients and providers and emerged in Phase 2.

Table 2 Themes for Top Reasons for COVID-19 Testing Program Failure from Patient and Provider Interviews during Early Implementation and Mid-Implementation Phases
Full size table

Subthemes for advertising/sharing information included not promoting the testing program enough and the brochures and study information having too much to read/too many words. Subthemes that emerged for access to testing were limited hours/days and no weekends, inconvenient testing locations, unsafe testing locations, and lack of technology literacy/access to participate. Subthemes for handling of test results were concerns regarding the completion of forms without a U.S. home address or email address, mistrust of emailed test results, difficulty reaching patients with their test results, extra work for staff if results were delivered in person, lack of timely return of results, and not communicating positive results to patients’ providers. Subthemes for safety included patients not following COVID-19 safety protocols, patients being in the presence of potentially infected people at the testing site, the fear of being exposed because the testing location would be near the main entrance to the clinic, and insufficient personal protective equipment for staff. Both patients and providers listed patient beliefs and views of the SARS-CoV-2 virus as a reason for failure, and providers also noted that limited testing options offered would be a reason for failure, specifically parents being reluctant to collect samples for their children and the lack of rapid antigen testing since, at the time, CO-CREATE only offered polymerase chain reaction (PCR) tests. Other themes that emerged as reasons for program failure during Phase 1 only included misinformation from unreliable sources, culture, fears and beliefs of patients, lack of trust in the system and science, and religious beliefs. These themes did not emerge in Phase 2 and were replaced by more pragmatic themes related to access to testing and handling of test results.

Proposed solutions to COVID-19 testing program failures

The main themes that emerged from the proposed solutions to the COVID-19 testing program failures overlapped with the perceived failures in both early and mid-implementation included advertising/sharing information, access to testing, handling of test results, and testing options offered while education emerged in early implementation only and staff safety, physical operations, and recruitment strategies emerged in mid-implementation (Table 3).

Table 3 Proposed Solutions to COVID-19 Testing Program Failures from Patient and Provider Interviews during Early Implementation and Mid-Implementation Phases
Full size table

Specific strategies for the theme of advertising/sharing information included incorporating visual materials describing the program available at the testing site, website/social media showing the testing schedule, using multiple modes of advertising (TV, flyers, schools/parents), and clarity about when and where testing was offered. Solutions related to advertising/sharing information were the most mentioned and emerged in patients for both Phases and providers only in Phase 2.

Specific strategies for access were extended hours and weekend testing, walk-up testing, easy access to testing at the clinic, moving testing to a more private location indoors, and having the testing site in front of the clinic. Recommended strategies for handling test results were: sending results by email to patients and their providers to communicate positive results, emphasizing during testing the importance of accurate contact information so that participants could be reached with test results, sending results by text, and encouraging patients to set up an email account if they did not have one. Solutions related to access to testing and handling of test results were commonly mentioned and emerged in both phases for patients and providers. Strategies for testing options offered were: providing a saliva test for pediatric patients and having rapid antigen tests available. Solutions related to testing options offered emerged with providers only, in both phases. Strategies for education highlighted helping people understand the severity of the COVID-19 pandemic, education about testing and vaccination, and a hands-on demonstration of the testing process. Specific strategies for staff safety were: N95s for those in front-line contact with symptomatic patients; designated places for personal protective equipment donning and doffing; separate testing areas for symptomatic and asymptomatic test-seekers; and delaying research surveys for symptomatic patients until after their quarantine period ended. Strategies for physical operations were: using floor decals to designate 6 feet distance for lines; posting signs to indicate flow; and stocking up on testing supplies when a surge is expected. Finally, strategies for recruitment were: disclosing the content and purpose of the research survey at the start of the testing process; sharing how participation will help others; and speaking to people in respectful and appropriate ways based on their level of literacy and comfort. The safety, physical operations, and recruitment themes were only mentioned by providers in Phase 2 interviews.

Changes to program flow based on proposed solutions

A number of changes to the testing workflow were made in response to solutions proposed by patients and providers during the brainwriting premortem interviews (Table 4). Changes related to the handling of test results were the most common and included sending results via text with distinct workflows based on the test result. Specifically, all results were sent to patients by text, but a request to contact the team was sent only to those with positive results (patients with negative or invalid results were not asked to contact the team). Patients with positive results were also called by a physician or nurse on the research team, with three attempts made.

Table 4 Proposed Solutions to COVID-19 Testing Program Failures and Changes Made to Program Flow to Address Them
Full size table

Changes to the advertising/sharing of information and education were the second most common, resulting in the development and distribution of aesthetically pleasing flyers at the testing sites. Designated on-site staff managed social media and website updates instead of relying on off-site volunteers to help with updates. Changes were also made to solutions proposed for staff safety (e.g., requiring N95s for on-site staff instead of normal face masks), physical operations (e.g., using colored cones to help with social distancing when patients stand in line for tests), and testing options offered (e.g., provided rapid antigen tests starting in June 2022). Overall, changes were made in response to 7 suggestions from patients and 9 suggestions from providers.

General comments

Participants were prompted to share general feedback or comments and positive feedback was noted as a common theme by English-speaking patients. This included: viewing the testing program as easily accessible and a good resource for the community; noting that trust was built in the community by testing everyone (symptomatic and asymptomatic); agreeing with focusing on areas with high positivity rates; and noting that test results were delivered in a more reliable and efficient manner with CO-CREATE compared to other testing services. Spanish-speaking patients commented that incentives to participate in the study and the outdoor testing location were good, in addition to also viewing that the testing program as easy to access and acceptable. Providers commented in Phase 2 that the testing program was a good resource for the community, that CO-CREATE staff were helpful, courteous and communicative, that the testing site was located in a convenient location, and that being walk-up, no appointment, was helpful, although there were long wait times during COVID-19 surges.

Discussion

We used partner-engaged interviews based on an adapted version of the brainwriting premortem technique to inform the iterative refinement of a COVID-19 testing program at a FQHC in the San Diego community. Multiple perspectives from providers and patients were represented in two distinct implementation time points allowing for an iterative and multi-partner improvement of the testing protocol. The process allowed us to incorporate the perspective of key partners in the delivery and iterative refinement of the COVID-19 testing program. Multiple changes to the testing protocol were made as a result of this feedback.

The most common potential failure reasons reported by patients and providers proved to be the sharing of results from the COVID-19 test with patients in a timely manner using an approach that was meaningful, low burden, and did not overextend an already overburdened healthcare system due to the pandemic. The most commonly recommended solution was related to providing information to the community about the continuously changing testing guidelines and the pandemic, especially in regard to reaching out to patients promptly and reaching all who might be interested in services offered by the testing site. Revisions to the workflow were adopted based on these recommendations. Our findings in terms of specific barriers and general recommendations are similar to other recent publications on the implementation of COVID-19 testing programs [32,33,34,35]. A broader theme that seems to be especially relevant across studies is the need to consider the preferences and circumstances of the specific priority population in a dynamic manner (at multiple time points to account for the rapidly changing pandemic context).

Engagement of partners in the creation of public health solutions has been broadly recommended in the field. Equally, if not more important, is ongoing engagement of these partners (checking in more than one time) especially when dealing with public health emergencies where guidance and evidence rapidly evolve. In a recent publication, Eisman and colleagues emphasized that rapid adaptations responding to urgent public health crises are critical to implementing impactful solutions [36]. Øvretveit also emphasized the need to use implementation science approaches to address COVID-19 pandemic challenges in health care settings [37].

As noted in the introduction, there are multiple techniques that can be used to achieve meaningful engagement of partners to inform potential program barriers and facilitators and eventually lead to strategies to address these barriers before and during implementation [17,18,19,20,21]. Our paper intended to expand the implementation science toolkit with a novel, less frequently used approach that happened to be also rapid. We believe that researchers should consider various approaches and select the one that seems to fit best for their circumstances. The use of the brainwriting premortem exercise was feasible to complete in the context of our study, but it was not used as a stand-alone strategy to engage with our partners. We used brainwriting premortem in combination with monthly community and scientific advisory board meetings to develop a theory of change of COVID-19 disparities in our setting [11] and an iterative ethnographic assessment of the engagement process in the advisory board meetings [16]. Jolles-Perez and colleagues recently operationalized five principles of a co-created collaborative process to enhance implementation efforts that encompass equity, reflexivity, reciprocity and mutuality, transformative and personalized, and facilitating relationships [38].

Important limitations for this study included the use of an individual instead of a group-based approach to the brainwriting premortem process. Although we were not able to capitalize on the group-based strength of the method, conducting individual interviews allowed for this approach to be feasible in the context of the COVID-19 pandemic where bringing together groups of providers and community members presented a substantial challenge. We also conducted only two waves of the brainwriting premortem process, which limited us to input at two time points (pre-early implementation and mid-implementation). In future studies, more frequent use of this technique in conjunction with other methods of implementation and evaluation could provide real-time guidance from key partners on how the program is implemented. There is increasing interest in the field in using real-time input from partners to guide and adjust implementation. Recent publications describe methods to guide this process [39].

Conclusions

This study adds to the growing literature on pragmatic methods for rapid and meaningful partner engagement by presenting successful application of the brainwriting premortem approach to identifying and implementing revisions to a COVID-19 testing workflow during both pre- and post-launch of testing activities. Brainwriting premortem was an effective tool in the iterative refinement of a COVID-19 testing program in the context of an FQHC and can be a promising and approach to incorporate iterative input from patients and providers to ensure successful program implementation.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to the majority of the data being qualitative in nature, so there are restrictions to sharing to preserve the privacy of individuals. However, the co-author team will review data requests and that data will be made available as reasonably appropriate. Contact the corresponding author for data requests.

Abbreviations

CO-CREATE:

Community-driven Optimization of COVID-19 testing to Reach and Engage underserved Areas for Testing Equity—in Women and Children

RADx-UP:

Rapid Acceleration of Diagnostics-for Underserved Populations

FQHC:

Federally Qualified Health Center

PCR:

Polymerase Chain Reaction

References

  1. Price-Haywood EG, Burton J, Fort D, Seoane L. Hospitalization and mortality among black patients and white patients with Covid-19. N Engl J Med. 2020;382:2534–43. https://doi.org/10.1056/nejmsa2011686.

    Article  CAS  PubMed  Google Scholar 

  2. Pham O, Ndugga N, Hill L, Artiga S, Alam R, Parker N. Latest Data on COVID-19 Vaccinations Race/Ethnicity. Kaiser Family Foundation. https://www.kff.org/coronavirus-covid-19/issue-brief/latest-data-on-covid-19-vaccinations-race-ethnicity/. Accessed 2 July 2021.

  3. Covid-19 cases by geography of residence. Archive COVID-19 Cases by Geography of Residence. (2020, June). https://www.sandiegocounty.gov/content/sdc/hhsa/programs/phs/community_epidemiology/dc/2019-nCoV/status/COVID19_Cases_by_Geography_of_Residence.html.

  4. Graham G. Addressing the Disproportionate Impact of COVID-19 on Communities of Color. J Racial Ethn Health Disparities. 2021;8(2):280–2. https://doi.org/10.1007/s40615-021-00989-7.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Health & Human Services Agency, Public Health Services, & Community Health Statistic Unit. (2019, November). 2017 Demographic Profiles San Diego County . Public Health Services. https://www.sandiegocounty.gov/content/sdc/hhsa/programs/phs/.

  6. Holden TM, Simon MA, Arnold DT, Halloway V, Gerardin J. Structural racism and COVID-19 response: higher risk of exposure drives disparate COVID-19 deaths among Black and Hispanic/Latinx residents of Illinois, USA. BMC Public Health. 2022;22(1):312. https://doi.org/10.1186/s12889-022-12698-9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Hooper MW, Nápoles AM, Pérez-Stable EJ. COVID-19 and racial/ethnic disparities. JAMA. 2020;323(24):2466–7.

    Article  CAS  Google Scholar 

  8. Khanna N, Klyushnenkova EN, Kaysin A. Association of COVID-19 with race and socioeconomic factors in family medicine. J Am Board Fam Med. 2021;34(Suppl):S40–s47. https://doi.org/10.3122/jabfm.2021.S1.200338.

    Article  PubMed  Google Scholar 

  9. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27(8):985–91. https://doi.org/10.1007/s11606-012-2037-1.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94(2):392–429. https://doi.org/10.1111/1468-0009.12197.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Stadnick NA, Cain KL, Oswald W, Watson P, Ibarra M, Lagoc R, Rabin B, et al. Co-creating a Theory of Change to advance COVID-19 testing and vaccine uptake in underserved communities. Health Serv Res. 2022;57 Suppl 1:149–57. https://doi.org/10.1111/1475-6773.13910. Epub 2022 Mar 4.

    Article  PubMed  Google Scholar 

  12. Stadnick NA, Cain K, Oswald W, Watson P, Ibarra M, Lagoc R, et al. Engaging underserved communities in COVID-19 health equity implementation research: an analysis of community engagement resource needs and costs. Front. Health Serv. 2:850427. https://doi.org/10.3389/frhs.2022.850427.

  13. Nooraie RY, Shelton RC, Fiscella K, Kwan BM, McMahon JM. The pragmatic, rapid, and iterative dissemination & implementation (PRIDI) cycle: Adapting to the dynamic nature of public health emergencies. Res Sq [Preprint]. 2021:rs.3.rs-188929. https://doi.org/10.21203/rs.3.rs-188929/v1. Update in: Health Res Policy Syst. 2021;19(1):110.

  14. Glasgow RE, Battaglia C, McCreight M, Ayele RA, Rabin BA. Making implementation science more rapid: use of the RE-AIM framework for mid-course adaptations across five health services research projects in the veterans health administration. Front Public Health. 2020;27(8):194.

    Article  Google Scholar 

  15. O’Cathain A, Croot L, Duncan E, Rousseau N, Sworn K, Turner KM, et al. Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open. 2019;9(8):e029954. https://doi.org/10.1136/bmjopen-2019-029954.

  16. Rabin B, Cain KL, Salgin L, Watson P, Oswald W, Kaiser B, et al. Using ethnographic approaches to document, evaluate, and facilitate virtual community-engaged implementation research. BMC Public Health. 2023;23(1):409.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Villalobos A, Blachman-Demner D, Percy-Laurry A, et al. Community and partner engagement in dissemination and implementation research at the National Institutes of Health: an analysis of recently funded studies and opportunities to advance the field. Implement Sci Commun. 2023;4:77. https://doi.org/10.1186/s43058-023-00462-y.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Fernandez ME, Ten Hoor GA, van Lieshout S, Rodriguez SA, Beidas RS, Parcel G, Ruiter RAC, Markham CM, Kok G. Implementation mapping: using intervention mapping to develop implementation strategies. Front Public Health. 2019;18(7):158. https://doi.org/10.3389/fpubh.2019.00158.

    Article  Google Scholar 

  19. Beres LK, Simbeza S, Holmes CB, Mwamba C, Mukamba N, Sharma A, Munamunungu V, Mwachande M, Sikombe K, Bolton Moore C, Mody A, Koyuncu A, Christopoulos K, Jere L, Pry J, Ehrenkranz PD, Budden A, Geng E, Sikazwe I. Human-Centered Design Lessons for Implementation Science: Improving the Implementation of a Patient-Centered Care Intervention. J Acquir Immune Defic Syndr. 2019;82 Suppl 3(3):S230–43. https://doi.org/10.1097/QAI.0000000000002216. Erratum in: J Acquir Immune Defic Syndr. 2020;84(1):e10.

    Article  PubMed  Google Scholar 

  20. Kiekens A, Dierckx de Casterlé B, Vandamme AM. Qualitative systems mapping for complex public health problems: A practical guide. PLoS One. 2022;17(2):e0264463. https://doi.org/10.1371/journal.pone.0264463.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Pittman JOE, Rabin B, Almklov E, Afari N, Floto E, Rodriguez E, Lindamer L. Adaptation of a quality improvement approach to implement eScreening in VHA healthcare settings: innovative use of the Lean Six Sigma Rapid Process Improvement Workshop. Implement Sci Commun. 2021;2(1):37. https://doi.org/10.1186/s43058-021-00132-x.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Kwan B, Ytell K, Coors M, DeCamp M, Morse B, Ressalam J, et al. A stakeholder engagement method navigator webtool for clinical and translational science. J Clin Transl Sci. 2021;5(1):E180. https://doi.org/10.1017/cts.2021.850.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Gilmartin H, Lawrence E, Leonard C, McCreight M, Kelley L, Lippmann B, et al. Brainwriting premortem: a novel focus group method to engage stakeholders and identify preimplementation barriers. J Nurs Care Qual. 2019;34(2):94–100. https://doi.org/10.1097/NCQ.0000000000000360.

    Article  PubMed  Google Scholar 

  24. Paulus PB, Korde RM, Dickson JJ, Carmeli A, Cohen-Meitar R. Asynchronous brainstorming in an industrial setting: exploratory studies. Hum Factors. 2015;57(6):1076–94. https://doi.org/10.1177/0018720815570374. (Epub 2015 Feb 4).

    Article  PubMed  Google Scholar 

  25. Klein G. Performing a project premortem. Harv Bus Rev. 2007;85(9):18–9.

    Google Scholar 

  26. Leonard C, Gilmartin H, McCreight M, Kelley L, Mayberry A, Burke RE. Training registered nurses to conduct pre-implementation assessment to inform program scale-up: an example from the rural Transitions Nurse Program. Implement Sci Commun. 2021;2(1):28. https://doi.org/10.1186/s43058-021-00127-8.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Joshi A, Mehendale AM. Project Premortem using Prospective Hindsight: An Unexplored Tool to address Healthcare Projects Implementation barriers. Indian J Comm Health. 2021;33(2):415–6. https://doi.org/10.47203/IJCH.2021.v33i02.039.

    Article  Google Scholar 

  28. Albright K, Jones CD. Methodological progress note: the case for mixed methods in quality improvement and research projects. J Hosp Med. 2022;17(6):468–71. https://doi.org/10.1002/jhm.12806. (Epub 2022 Mar 16).

    Article  PubMed  Google Scholar 

  29. Lewinski AA, Crowley MJ, Miller C, Bosworth HB, Jackson GL, Steinhauser K, White-Clark C, McCant F, Zullig LL. Applied rapid qualitative analysis to develop a contextually appropriate intervention and increase the likelihood of uptake. Med Care. 2021;59(Suppl 3):S242–51. https://doi.org/10.1097/MLR.0000000000001553.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Gale RC, Wu J, Erhardt T, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implement Sci. 2019;14:11.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hamilton AB. Qualitative methods in rapid turn-around health services research. Health Services Research & Development Cyberseminar. 2013. Available at: https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780 . Accessed 6 Nov 2023 .

    Google Scholar 

  32. Kruse GR, Pelton-Cairns L, Taveras EM, Dargon-Hart S, Gundersen DA, Lee RM, Bierer BE, Lawlor E, LaRocque RC, Marcus JL, Davies ME, Emmons KM, RADx-MA Partnership. Implementing expanded COVID-19 testing in Massachusetts community health centers through community partnerships: Protocol for an interrupted time series and stepped wedge study design. Contemp Clin Trials. 2022;118:106783. https://doi.org/10.1016/j.cct.2022.106783.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Embrett M, Sim SM, Caldwell HAT, et al. Barriers to and strategies to address COVID-19 testing hesitancy: a rapid scoping review. BMC Public Health. 2022;22:750. https://doi.org/10.1186/s12889-022-13127-7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. McElfish PA, Purvis R, James LP, Willis DE, Andersen JA. Perceived Barriers to COVID-19 Testing. Int J Environ Res Public Health. 2021;18(5):2278. https://doi.org/10.3390/ijerph18052278.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Bonner C, Batcup C, Cvejic E, Ayre J, Pickles K, Copp T, Cornell S, Nickel B, Dhahir M, McCaffery K. Addressing Behavioral Barriers to COVID-19 Testing With Health Literacy-Sensitive eHealth Interventions: Results From 2 National Surveys and 2 Randomized Experiments. JMIR Public Health Surveill. 2023;9:e40441.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Eisman AB, Kim B, Salloum RG, Shuman CJ, Glasgow RE. Advancing rapid adaptation for urgent public health crises: using implementation science to facilitate effective and efficient responses. Front Public Health. 2022;24(10):959567.

    Article  Google Scholar 

  37. Øvretveit J. Implementation researchers can improve the responses of services to the COVID-19 pandemic. Implement Res Pract. 2020;1:2633489520949151.

    PubMed  PubMed Central  Google Scholar 

  38. Pérez Jolles M, Willging CE, Stadnick NA, Crable EL, Lengnick-Hall R, Hawkins J, et al. Understanding implementation research collaborations from a co-creation lens: recommendations for a path forward. Front Health Serv. 2022;2:942658. https://doi.org/10.3389/frhs.2022.942658.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Glasgow RE, Battaglia C, McCreight MS, Ayele R, Maw AM, Fort MP, et al. Use of the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework to guide iterative adaptations: Applications, lessons learned, and future directions. Front Health Serv. 2:959565. https://doi.org/10.3389/frhs.2022.959565.

Download references

Acknowledgements

We would like to extend our gratitude for the time and wisdom shared by the CO-CREATE Community Advisory Board members and support from our partners at the Global Action Research Center (Paul Watson, Bill Oswald, Marina Castelo, Raphael Lagoc) and the participating Federally Qualified Health Center (Fatima Munoz, Nikol Sarbaich).

Funding

This work was funded by the National Institutes of Health: P42 ES010337-19S2 RADx-UP Supplement (Laurent, Tukey); K23 MH110602 (Stadnick); R34 MH120190 (Stadnick), the UC San Diego ACTRI Dissemination and Implementation Science Center (Rabin, Stadnick), the UC San Diego Altman Clinical and Translational Center NIH UL1TR001442 (Rabin, Stadnick).

Author information

Authors and Affiliations

  1. Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla, CA, USA

    Borsika A. Rabin & Kelli L. Cain

  2. UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, University of California San Diego, La Jolla, CA, USA

    Borsika A. Rabin & Nicole A. Stadnick

  3. Department of Obstetrics, Gynecology, and Reproductive Sciences, University of California San Diego, La Jolla, CA, USA

    Lawrence O. Ayers, Angel Lomeli, Arleth Escoto, Maria Linda Burola, Melanie Aguilar, Stephenie Tinoco Calvillo, Breanna Reyes & Louise C. Laurent

  4. San Ysidro Health, San Diego, CA, USA

    Linda Salgin

  5. Superfund Research Center, University of California San Diego, La Jolla, CA, USA

    Robert Tukey

  6. Department of Pharmacology, University of California San Diego, La Jolla, CA, USA

    Robert Tukey

  7. Department of Psychiatry, University of California San Diego, La Jolla, CA, USA

    Nicole A. Stadnick

  8. Child and Adolescent Services Research Center, San Diego, CA, USA

    Nicole A. Stadnick

Authors
  1. Borsika A. Rabin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kelli L. Cain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lawrence O. Ayers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Angel Lomeli
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arleth Escoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maria Linda Burola
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Melanie Aguilar
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Stephenie Tinoco Calvillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Breanna Reyes
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Linda Salgin
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Robert Tukey
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Louise C. Laurent
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Nicole A. Stadnick
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

BAR conceptualized the study designs and methods, drafted the article, contributed to the data collection and analysis, contributed to the implementation of the project, interpretation of results, and manuscript revisions. KLC conceptualized the study designs and methods, drafted the article, contributed to the data collection and analysis, contributed to the implementation of the project, interpretation of results, and manuscript revisions. LA contributed to the implementation of the project, contributed to the data collection and analysis, and contributed to the interpretation of results and manuscript revisions. AL contributed to the implementation of the project, contributed to the data collection and analysis, interpretation of results, and manuscript revisions. AE contributed to the implementation of the project, contributed to the data collection and analysis, interpretation of results, and manuscript revisions. MLB contributed to the implementation of the project, contributed to the data collection and analysis, interpretation of results, and manuscript revisions. MA contributed to the implementation of the project, contributed to the data collection and analysis, interpretation of results, and manuscript revisions. STC contributed to the implementation of the project, contributed to the data collection and analysis, interpretation of results, and manuscript revisions. BR contributed to the implementation of the project, contributed to the data collection and analysis, interpretation of results, and manuscript revisions. LS contributed to the implementation of the project, interpretation of results, and manuscript revisions. RT contributed to the implementation of the project, interpretation of results, and manuscript revisions. LSL conceptualized the study designs and methods, contributed to the implementation of the project, interpretation of results, and manuscript revisions. NAS conceptualized the study designs and methods, drafted the article, contributed to the implementation of the project, interpretation of results, and manuscript revisions. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Borsika A. Rabin.

Ethics declarations

Ethics approval and consent to participate

This study obtained ethics approval and informed consent was obtained from all study participants and/or their legal guardian and was conducted under a protocol approved by the University of California San Diego Institutional Review Board (#210498). All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplemental Table.

General Information about Patients and Providers from Interviews.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rabin, B.A., Cain, K.L., Ayers, L.O. et al. Adaptation of the brainwriting premortem technique to inform the co-creation of COVID-19 testing strategies in underserved communities in South San Diego. BMC Health Serv Res 24, 12 (2024). https://doi.org/10.1186/s12913-023-10341-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-023-10341-w

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us