Skip to main content
Download PDF

Systematic surveillance of patient-reported symptoms of viral respiratory tract infectious Syndromes in diverse populations

BMC Health Services Research volume 22, Article number: 1591 (2022) Cite this article

A Correction to this article was published on 26 January 2023

This article has been updated

Abstract

Background

Patient reported outcome measures (PROM) can improve patient care and be crucial for symptom tracking especially during disease outbreaks. FLU-PRO Plus is a validated PROM used to track viral respiratory symptoms. Our study aimed to evaluate the feasibility of using FLU-PRO© Plus, to track symptoms across three healthcare systems.

Methods

The prospective, longitudinal study recruited adults between February-May 2021 from HealthPartners Institute (HP), Kaiser Permanente Georgia (KPGA), and Kaiser Permanente Mid-Atlantic States (KPMAS). Adult members were eligible if they had a positive lab or diagnosis for either COVID-19 or influenza-like illness (ILI) or exhibited 2 + viral respiratory symptoms. Descriptive statistics were calculated to describe the patient characteristics for participants that were eligible for FLU-PRO Plus, successfully contacted, attempted to log in to the FLU-PRO Plus website, and participants who completed FLU-PRO Plus Day 1. Bivariable and multivariable logistic regression using PROC GLIMMIXX investigated the patient characteristics associated with (1) successful contact and (2) FLU-PRO Plus Day 1 completion.

Results

We identified a total of 15,650 eligible participants during the enrollment period: 9,582 from HP, 1,740 from KPGA, and 4,328 from KPMAS. Among the total of 409 eligible adults who attempted to participate in FLU-PRO Plus, 317 completed FLU-PRO Plus Day 1. Among the 317 individuals that completed FLU-PRO Plus Day 1, 205 (67.5%) were diagnosed with COVID-19; 112 adults diagnosed with COVID-19 completed FLU-PRO Plus Day 14. Among adults successfully contacted, adults aged 35–64 (OR = 1.40, 95% CI 1.05, 1.87), females (OR = 1.77, 95% CI 1.38, 2.27), and adults diagnosed with COVID-19 (OR = 1.66, 95% CI 1.27, 2.17) had higher odds of completing FLU-PRO Plus Day 1; Asian adults (OR = 0.38, 95% CI 0.19, 0.76) and Black and African American adults (OR = 0.33, 95% CI 0.19, 0.76) had lower odds compared to White adults.

Conclusion

Our study reports on the feasibility of patients across three integrated healthcare systems utilizing FLU-PRO Plus to monitor their respiratory symptoms. Patient reported outcome measures (PROM) can improve patient care, quality of life, and reduce the strain of limited resources on healthcare systems. Future FLU-PRO Plus studies should develop an implementation strategy to fully integrate FLU-PRO Plus within clinical care and patient management.

Peer Review reports

Introduction

Patient centered care has become increasingly recognized as an integral component and ethical imperative of modern healthcare [1]. Patient reported outcome measures (PROMs) are patient-reported health outcomes without interpretation from healthcare professionals, enabling patients to report what they are experiencing. PROMs have been shown to be significantly predictive of patient health status and to improve patient care [2, 3]. PROMs have been developed and shown to be valuable for patient care in chronic diseases [4,5,6,7], cancer [2, 8,9,10,11,12,13,14], orthopedics [15], mental health [16], primary care [17], and infectious diseases [18,19,20,21]. PROMs used sequentially over a series of several days for patients undergoing cancer treatment enabled the patients’ clinical team to monitor the patient-reported symptoms, provided the participants continuous access to self-care advice, and lessened the physical symptom distress patients experienced while improving their emotional functioning, compared to the patients randomized to usual care [13].

The increased use of telemedicine, ownership of electronic smart devices and availability of telemedicine have led to an increased development of electronic PROMs [22,23,24,25]. Electronic PROMs may be administered within a clinic setting or remotely. PROMs can improve symptom management between visits and be a cost-effective options for a healthcare system by minimizing the frequency of costly diagnostic tests and scans [2, 26,27,28]. Remote use of electronic PROMs with built-in prompts to encourage completion may reduce recall bias by allowing the patient to report their symptoms or outcomes they are experiencing more accurately and promptly [29,30,31]. However, despite the growing evidence of PROM benefits to the patients and healthcare systems, PROMs have not been widely implemented across healthcare systems or disease spectrums [32].

Viral respiratory illnesses are common, although the symptoms, symptom intensity, and symptom duration can vary across illnesses [28, 31]. The onset of the novel coronavirus (COVID-19) brought about a variety of patient-reported surveys that individuals could use to report and track their symptoms [33,34,35,36]. FLU-PRO© is a validated PROM originally developed to standardize capture of patients’ symptoms associated with influenza, respiratory syncytial virus, rhinovirus, enterovirus, and endemic coronaviruses throughout a 14-day course, or until symptoms subside and participants return to their baseline health [28, 31]. The instrument was developed under the US Food and Drug Administration’s recommendations for evaluating content validity and used a two-stage qualitative methodology, concept elicitation and cognitive interviews [28]. The initial, 32-item, version of FLU-PRO contained six domains (nose, throat, eyes, chest/respiratory, gastrointestinal, and body/systemic) and demonstrated an overall comparative fit index = 0.92 [31]. FLU-PRO’s 32-items were revised in 2020 to incorporate a seventh domain of senses (loss of taste and smell) [36]. This PROM asks participants, “please rate the extent to which you had each symptom during the past 24 hours” with Likert Scale answer choices ranging from, “Not at all”, “A little bit”, “Somewhat”, “Quite a bit”, to “Very much”. Domain-specific questions were asked; for example, questions under the ‘eyes’ domain inquire about watery or painful eyes, and the ‘gastrointestinal’ domain contains items on nausea and stomachache. FLU-PRO Plus also requests individuals to report, overall, how their symptoms are today, how their symptoms are compared to yesterday, and the interference their symptoms have on daily activities [36]. Although FLU-PRO© and FLU-PRO Plus have been developed and validated in several clinical trials and prospective cohorts [28, 31, 36], the feasibility of implementing FLU-PRO Plus within healthcare systems has not been evaluated.

Our multi-site study across three geographically diverse integrated healthcare systems aimed to evaluate the feasibility of routine surveillance of respiratory viral syndromes in clinical practice by tracking the number and proportion of patients completing the FLU-PRO Plus symptom questions over a 14-day period. We described the recruitment process and recruitment rate across each healthcare system and compared the participants who completed Day 1 of FLU-PRO Plus to all eligible individuals and individuals who were successfully contacted to participate in FLU-PRO Plus.

Methods

Study design and overview

We conducted a prospective longitudinal study evaluating patient-reported respiratory symptoms. Adult (age ≥ 18 years) patients were eligible for recruitment if they (1) were enrolled as members in one of the three participating healthcare systems: HealthPartners Institute (HP), Kaiser Permanente Georgia (KPGA), or Kaiser Permanente Mid-Atlantic States (KPMAS), (2) were diagnosed with, or had a positive lab value for either influenza-like illness (ILI) or COVID-19, or (3) reported 2 + viral respiratory symptoms (cough, migraine, shortness of breath, fever, myalgia, or fatigue) captured during their medical visit. Patients were excluded if they were hospitalized or ventilated at the time of diagnosis or positive lab value, were diagnosed with dementia, or if they had actively opted out of research at their respective healthcare systems. Electronic informed consent was obtained for all participants prior to the participants being able to access the FLU-PRO survey. All methods and protocols were carried out in accordance with human subjects research guidelines. Protocols were approved by the HealthPartners Institute, Kaiser Permanente Georgia, and Kaiser Permanente Mid-Atlantic States’ Institutional Review Boards.

Recruitment

Each integrated healthcare system utilized their robust electronic medical record (EMR) to determine member eligibility based on the inclusion and exclusion criteria. Standardized eligibility criteria were disseminated in an EMR-based algorithm shared across sites. Recruitment occurred within 72-hours of meeting the eligibility criteria. The algorithm generated a daily report of eligible members. Rolling recruitment began in February 2021 and continued through May 15, 2021.

Recruitment methodology was optimized for each healthcare system’s member population. HP conducted phone recruitment on a randomized list of eligible members. HP relied only on phone recruitment which occurred Monday-Friday from February 1, 2021, through April 15, 2021. KPGA and KPMAS used a hybrid of email and phone recruitment for daily, Monday-Friday recruitment outreach. All eligible members with a registered email address received an initial recruitment email. Targeted phone calls were then completed based on the following prioritization: (1) no listed email address, (2) ILI lab positive or diagnosis, and (3) COVID-19 lab positive or diagnosis. KPGA began recruitment on March 8, 2021, and KPMAS began on March 15, 2021. Each Kaiser Permanente site completed 9-weeks of recruitment. All interested participants were directed to the online survey to provide online consent and begin completing the FLU-PRO Plus 14-day survey questionnaires. Figure 1 shows the site-specific flow diagram of participant recruitment, consent, and FLU-PRO Plus Day 1 completion.

Fig. 1
figure 1

Flow diagram of participant eligibility, recruitment, consent, and day 1 survey completion

Full size image

Data collection

FLU-PRO Plus is an electronic PROM developed to assess symptoms related to ILI and COVID-19. The survey is designed to be completed daily by the participant for 14-days and asks the participant to rate the frequency and intensity of 34 symptoms, in the past 24-hours using a five-point Likert scale [28, 36]. The FLU-PRO Plus survey was administered using the HIPAA compliant electronic data capture software REDCap hosted by HP. After eligible adults were recruited and received the link to the REDCap-based FLU-PRO Plus, the participants were instructed to complete electronic, informed consent and were subsequently asked if they were still experiencing symptoms. We ascertained basic patient characteristics from each integrated healthcare system’s administrative databases for all eligible adults to describe the demographic differences between adults that were eligible, were successfully contacted, completed consent, and completed FLU-PRO Plus Day 1.

Our outcomes were focused on describing the eligible population that were (1) successfully contacted and (2) completed Day 1 FLU-PRO Plus. Eligible adults were considered to have a successful contact (yes/no) if the telephone number on file was still working or an email was sent to the individual’s registered email address and an error message did not occur. Completion of FLU-PRO Plus Day 1 was defined as ‘yes’ or ‘no’ and was determined if the participant used the FLU-PRO Plus link to log in after completing consent. We examined the odds of being successfully contacted and the odds of completing FLU-PRO Plus Day 1 among all eligible adults. Among the successfully contacted individuals, we examined the odds of completing FLU-PRO Plus Day 1.

Patient characteristics were collected from the EMR and were defined at the time of eligibility. Age at the time of eligibility was classified as a mean (SD) and categorized as 18–34, 35–64, and ≥ 65 years. Participant race from the EMR was categorized as ‘Asian’, ‘Black or African American’, ‘White’, ‘Other’, or ‘Unknown race’. Other race was defined as a race outside of the named categories and for participants reporting multiple races. ‘Unknown race’ was assigned if a participant did not have a reported race in the EMR. Gender (‘male’ vs. ‘female’) and Hispanic (‘yes’ vs. ‘no’) ethnicity were self-reported in the electronic medical record. Recruitment mode was captured at each site at the time of recruitment. Diagnosis, initially defined at the time of enrollment, was confirmed and validated by lab records 30-days after initial diagnosis record and was defined as either COVID-19 or influenza-like illness positive.

Statistical analysis

Descriptive statistics were calculated to describe the patient characteristics for participants that were eligible for FLU-PRO Plus, successfully contacted, had attempted to log in to the FLU-PRO Plus website, and completed FLU-PRO Plus Day 1 across all three sites. We assessed the FLU-PRO Plus survey days by patient characteristics overall and the total of survey days completed across each site. Bivariable and multivariable logistic regression investigated the patient characteristics associated with (1) successful contact and (2) FLU-PRO Plus Day 1 completion. PROC GLIMMIXX accounted for site-level clustering and treated each healthcare system as a random effect in the logistic regression models. All analyses were performed using SAS 9.4 ©.

Results

We identified a total of 15,650 eligible participants during the enrollment period: 9,582 from HP, 1,740 from KPGA, and 4,328 from KPMAS. HP randomized half of their eligible population (4,157 adults) for phone recruitment and successfully contacted 3,074. A total of 258 eligible adults from HP visited the REDCap website and attempted to participate in FLU-PRO Plus. KPGA successfully contacted 1,583 participants, of whom 74 eligible adults visited the REDCap website and attempted to participate in FLU-PRO Plus. KPMAS successfully contacted 3,753 participants and had 77 eligible adults visit the REDCap website and attempt to participate in FLU-PRO Plus. Among the total of 409 eligible adults who attempted to participate in FLU-PRO Plus, 317 completed FLU-PRO Plus Day 1.

Descriptive statistics reported the patient characteristics by site that were (1) eligible to participate in FLU-PRO Plus, (2) successfully contacted, (3) visited the website, and attempted to participate in FLU-PRO Plus, and (4) completed FLU-PRO Plus Day 1 (Table 1). Adults meeting the inclusion criteria and eligible to participate in FLU-PRO Plus were aged 35–64 years old (HP = 54.4%, KPGA = 62.9%, KPMAS = 56.3%); similar to the age distribution of the participants completing FLU-PRO Plus Day 1 (HP = 67.0%, KPGA = 78.0%, KPMAS = 51.6%). A higher percentage of females completed FLU-PRO Plus Day 1 across all three sites (HP = 66.5%, KPGA = 76.0%, KPMAS = 73.4%). HP had a higher number of White adults who were eligible (n = 7,443; 77.7%), successfully contacted (n = 2,507; 81.6%), visited the website and attempted to participate (n = 219; 84.9%), and completed FLU-PRO Plus Day 1 (n = 178; 87.7%), compared to other race/ethnicity groups. Both KPGA and KPMAS had a higher percentage of Black and African American adults that were successfully contacted (KPGA = 50.7%, KPMAS = 51.6%) compared to other race groups. A higher percentage of Black and African American KPGA adults visited the website and attempted to participate (52.7%) and completed FLU-PRO Plus Day 1 (54.0%) compared to other KPGA race groups.

Table 1 Descriptive statistics of patient characteristics at time of eligibility across three integrated healthcare systems, February-May 2021
Full size table

Table 2 reports the frequency of adults that visited the website and attempted to participate, and completed FLU-PRO Plus Day 1, 3, 7, 10, and 14. There were 118 adults from HP that completed FLU-PRO Plus Day 14 compared to 20 and 32 adults from KPGA and KPMAS, respectively. Females and adults aged 34–65 years mostly completed FLU-PRO Plus Day 1 through 14. There were 122 (29.8%) participants that were recruited through email only and visited the FLU-PRO Plus website; 97 of those individuals completed FLU-PRO Plus Day 1. Among the 317 individuals that completed FLU-PRO Plus Day 1, 205 (67.5%) were diagnosed with COVID-19; 112 adults diagnosed with COVID-19 completed FLU-PRO Plus Day 14. Among all individuals completing FLU-PRO Plus Day 1, there was a gradual decline of percent completion of FLU-PRO Plus surveys throughout the 14-day survey period (Fig. 2).

Table 2 Descriptive statistics on the FLU-PRO Plus steps completed
Full size table
Fig. 2
figure 2

Daily progress of FLU-PRO Plus survey completion among all individuals who consented and completed Day 1, over the course of the 14-day survey period, overall and stratified by site

Full size image

Multivariable logistic regression evaluated the patient characteristics associated with adults being successfully contacted and completing FLU-PRO Plus Day 1. Table 3 reports that among those eligible for FLU-PRO Plus, adults aged 35–64 (OR = 1.15, 95% CI 1.05, 1.26) and aged ≥ 65 (OR = 1.15, 95% CI 1.02, 1.30) had higher odds of being successfully contacted compared to adults aged 18–34. Black and African American adults (OR = 0.74, 95% CI 0.66, 0.83) and adults diagnosed with COVID-19 (OR = 0.73, 95% CI 0.68, 0.80) had lower odds of being successfully contacted compared to their counterparts. Among those eligible for FLU-PRO Plus, adults aged 35–64 (OR = 1.59, 95% CI 1.20, 2.11), females (OR = 1.76, 95% CI 1.38, 2.25), and adults diagnosed with COVID-19 (OR = 1.40, 95% CI 1.09, 1.80) had higher odds of completing FLU-PRO Plus Day 1; Asian adults (OR = 0.36, 95% CI 0.18, 0.72) and Black and African American adults (OR = 0.46, 95% CI 0.33, 0.64) had lower odds compared to White adults. Similarly, among adults successfully contacted, adults aged 35–64 (OR = 1.40, 95% CI 1.05, 1.87), females (OR = 1.77, 95% CI 1.38, 2.27), and adults diagnosed with COVID-19 (OR = 1.66, 95% CI 1.27, 2.17) had higher odds of completing FLU-PRO Plus Day 1; Asian adults (OR = 0.38, 95% CI 0.19, 0.76) and Black and African American adults (OR = 0.33, 95% CI 0.19, 0.76) had lower odds completing FLU-PRO Pus Day 1, compared to White adults.

Table 3 Multivariable logistic regressiona investigating the odds of completing FLU-PRO Plus Day 1 among those who completed online consent
Full size table

Discussion

Our multi-site feasibility study aimed to evaluate the feasibility of routine surveillance of respiratory viral syndromes in clinical practice and attempted to contact 15,650 eligible adults across three integrated healthcare systems. Of the eligible adults successfully contacted, we had 3.7% complete FLU-PRO Plus Day 1. Among those who completed Day 1 of FLU-PRO Plus, 77.6% completed Day 7 and 53.6% completed Day 14 of the survey. Compared to their eligible counterparts that were successfully contacted, females and people aged 35–60 years were more likely to complete Day 1 of FLU-PRO Plus.

PROMs play an important role in patient-centered care that can lead to patient goal setting, improvement in patient engagement, improve effectiveness of patient-provider interaction, and increased patient self-efficacy [2, 19, 26, 32, 37,38,39]. However, despite their utility and benefits to both the patient and healthcare system, PROM efficacy and implementation in patient care has been mostly focused within oncology [2]. The web-based Symptom Tracking and Reporting ePROM system (STAR) tracked chemotherapy patients’ symptoms on a weekly basis and was shown to significantly reduce the number of emergency department visits and in-patient hospitalizations, improve health-related quality of life, and significantly improve overall survival in the group of patients randomized to use the PROM [9, 10, 14]. Increased odds of survival were also seen in the e-Follow-Up application PROM for adults undergoing lung cancer treatment [9,10,11,12]. FLU-PRO Plus is a reliable and valid PROM that provides standardized questions on seven symptom domains that could be used across patient populations. FLU-PRO Plus has also been translated in to > 23 languages including Spanish, Nepalese, Hindi, Zulu, Portuguese, Korean, French, Russian, and Vietnamese. The utilization of FLU-PRO Plus globally would provide healthcare providers and scientists the ability to complete more robust disease and symptom surveillance across populations, locally and internationally. A standardized mechanism of collecting symptom data would also enable more efficient tracking of virus mutations overtime, like we have seen with COVID-19 mutations and different strains increasing the odds of different symptoms [40].

PROMs may enable providers to more accurately differentiate between viral respiratory symptoms and track seasonal illnesses [31, 36]. The onset of the SARS-CoV-2 (COVID-19) pandemic highlighted the importance of standardized, systematic, and comprehensive measures that accurately capture patient-reported viral respiratory symptoms [33]. Healthcare provider-developed questions may miss important information about respiratory infections or inaccurately represent the frequency of duration of reported symptoms [33,34,35, 41]. FLU-PRO is a PROM that was developed and validated in patients with acute, laboratory-confirmed influenza and influenza like illness [28, 42]. FLU-PRO scores were reliable, reproducible, and reported high internal consistencies across the six original domains (nose, throat, eyes, chest/respiratory, gastrointestinal, body/systemic) [21]. The development of FLU-PRO Plus occurred at the onset of the COVID-19 pandemic and included a seventh domain of senses (loss of taste and smell) in May 2020 [36]. The authors enrolled COVID-19 positive patients from March 2020 through June 2021 that completed Day 1 of either FLU-PRO (March 2020-April 2020) or FLU-PRO Plus (May 2020-June 2021) [36]. The Cronbach’s alpha reliability across the seven FLU-PRO Plus’s domains remained above 0.86 and the responsiveness to the 14-days of survey showed a steady decrease across time [36], similar to the responsiveness our feasibility study showed.

PROMs have been shown to improve patient survival, quality of life, and symptom management [9, 12, 14, 27]. Healthcare systems also benefit from the use of PROM through more efficient use of limited healthcare resources, reduction of patient-reimbursement for travel, fewer emergency department visits, and improved treatment adherence [2, 26]. FLU-PRO Plus could provide a healthcare system the ability to monitor their patients’ ongoing symptoms remotely, leading to better patient care, improved allocation of healthcare resources, and reduced risk of disease transmission during viral respiratory disease outbreaks. In addition to providing better individual-level patient care, healthcare systems could pair the FLU-PRO Plus PROM with their robust EMR administrative database to aggregate symptom tracking across patients. Aggregating patient-reported viral respiratory symptoms across patients, would enable the healthcare system to track any viral respiratory infections, such as the influenza, rhinovirus, or COVID-19 across their patient population, overall, and geographically stratified to better assess local outbreaks and properly allocate resources to clinics in the most need. Integrating FLU-PRO Plus within a healthcare system would allow the healthcare system to compare and improve provider-level care and track the patient’s quality of life changes over time.

The COVID-19 pandemic facilitated a renewed interest among healthcare systems, patients, and policy makers to focus on telemedicine that may be sustained beyond the pandemic. U.S. policy makers have begun to address improving access to affordable, reliable, and high-speed internet to every American [43] as more healthcare systems implement telemedicine practices [24, 44,45,46]. As the use of telemedicine and eHealth interventions continue to increase [47], FLU-PRO Plus and other PROMS that are designed to ask lay questions in a short survey format may be the best option to provide remote surveillance while accounting for possible eHealth literacy disparities [48, 49]. The FLU-PRO Plus feasibility study, across three diverse sites, was able to show the proportion of people eligible, successfully contacted, and completed Day 1 remained constant across patient demographics except for age ≥ 65 that had a lower Day 1 completion compared to other ages. Integration of FLU-PRO Plus within the clinical workflow could potentially reduce the ‘digital divide’ or eHealth Literacy disparity by providing patients the opportunity to be trained to use and answer the prompts during their in-clinic, Day 1 visit. A follow-up study should focus on the implementation mechanisms that would enhance the uptake of FLU-PRO Plus, capture the utility across subpopulations, and gather feedback from patient- and clinical-stakeholders on the real-world use of FLU-PRO Plus.

While the results demonstrate the feasibility of using FLU-PRO Plus across three integrated healthcare systems, the study has notable limitations. First, because our recruitment occurred between February through May 2021, our sample does not account for the recent COVID-19 variants and has limited sample size for patients diagnosed with influenza or influenza like illnesses. Second, the generalizability of the findings is limited due to the small sample sizes, recruitment restrictions, short sampling timeframe, and limited geographic reach. In this feasibility study, each of the three integrated healthcare systems completed recruitment based on institutional guidelines and resources. While we were able to test different recruitment methodologies and determine that combined email and phone recruitment may be the most successful strategy, it made defining the recruitment rate challenging. Future studies should apply the lessons learned from this feasibility study and design a robust email and phone recruitment strategy. Future studies should also include a prolonged recruitment period to increase the recruitment size and diversity across viral respiratory diseases and variants. Third, our analysis was limited to Day 1 of FLU-PRO Plus completion because we noticed participants were no longer completing the daily survey once their symptoms subsided and they returned to baseline health. Therefore, we decided an analysis on Day 14 completion would not be accurate and a revision of FLU-PRO Plus should include an opening question asking the individual to report if their symptoms have subsided to accurately capture the information on symptom duration. Fourth, our study excluded patients hospitalized at the time of eligibility and limits our findings to patients with less severe viral respiratory infections, which comprised most of the viral respiratory infections in 2021 [50,51,52]. Fifth, our study was limited on the socioeconomic variables collected at the participant-level and could not fully assess if a digital or eHealth literacy disparities were present among participants. Future studies should focus on collecting more granular socioeconomic and neighborhood-level variables, in addition to e-skills capability, to improve utilization across population groups. Sixth, this assessment did not differentiate symptom patterns between viral respiratory infections such as COVID-19 and influenza-like illnesses. Additional studies are needed to evaluate if symptom scores could capture distinct patterns in intensity and duration across viral respiratory illnesses.

Conclusion

Patient-reported outcome measures can improve patient care and quality of life and reduce the strain of limited healthcare systems resources. FLU-PRO Plus is a PROM focused on tracking viral respiratory illnesses including influenza, respiratory syncytial virus, rhinovirus, enterovirus, and coronaviruses. Our study reports on the feasibility of patients across three integrated healthcare systems utilizing FLU-PRO Plus to monitor their respiratory symptoms for 14-days and the patient characteristics associated with completing FLU-PRO Plus Day 1. Future FLU-PRO Plus studies should develop an implementation strategy to fully integrate FLU-PRO Plus within clinical care and patient management. As the acceptability and utilization of telemedicine and smart devices continue to increase, healthcare systems should strongly consider broad implementation of PROMs to collect patient-reported information that may lead to improved patient care and resource allocation.

Availability of data and materials

The data underlying this article cannot be shared publicly due to the privacy of individuals and integrated healthcare system members that participated in the study. The derived data will be shared on reasonable request to the corresponding author.

Change history

Abbreviations

HP:

HealthPartners Institute

ILI:

Influenza-like-illness

KPGA:

Kaiser Permanente Georgia

KPMAS:

Kaiser Permanente Mid-Atlantic States

PROM:

Patient-reported outcome measures

References

  1. Reuben DB, Jennings LA. Putting goal-oriented patient care into practice. J Am Geriatr Soc. 2019;67(7):1342–4.

    Article  Google Scholar 

  2. Aiyegbusi OL. Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems. Qual Life Res. 2020;29(2):325–33.

    Article  Google Scholar 

  3. Centers for Medicare and Medicaid Services. Patient reported outcome measures. M.M. System, editor. 2021.

  4. Kappelman MD, et al., Evaluation of the patient-reported outcomes measurement information system in a large cohort of patients with inflammatory bowel diseases. Clin Gastroenterol Hepatol, 2014. 12(8): 1315–23. e2.

    Article  Google Scholar 

  5. Ju A, et al. Patient-reported outcome measures for fatigue in patients on hemodialysis: a systematic review. Am J Kidney Dis. 2018;71(3):327–43.

    Article  Google Scholar 

  6. Ware JE, et al. Improving CKD-specific patient-reported measures of health-related quality of life. J Am Soc Nephrol. 2019;30(4):664–77.

    Article  Google Scholar 

  7. Appel CW, et al. Telemedicine based on patient-reported outcomes in management of patients with inflammatory bowel disease in a real-life setting–a before and after cohort study. Scand J Gastroenterol. 2022;57(7):825–31.

    Article  Google Scholar 

  8. Calvert M, et al. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the SPIRIT-PRO extension. JAMA. 2018;319(5):483–94.

    Article  Google Scholar 

  9. Denis F, et al. Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA. 2019;321(3):306–7.

    Article  Google Scholar 

  10. Denis F, et al. Randomized trial comparing a web-mediated follow-up via patient-reported outcomes (PRO) vs. routine surveillance in lung cancer patients: final results. J Clin Oncol. 2018;36, no. 15_suppl:6500.

  11. Denis F, et al. Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients. JNCI. 2017;109(9).

  12. Denis F, et al. Improving survival in patients treated for a lung cancer using self-evaluated symptoms reported through a web application. Am J Clin Oncol. 2017;40(5):464–9.

    Article  Google Scholar 

  13. Fjell M, et al. Reduced symptom burden with the support of an interactive app during neoadjuvant chemotherapy for breast cancer–A randomized controlled trial. The Breast. 2020;51:85–93.

    Article  Google Scholar 

  14. Nipp R, et al. Differential effects of an electronic symptom monitoring intervention based on the age of patients with advanced cancer. Ann Oncol. 2020;31(1):123–30.

    Article  CAS  Google Scholar 

  15. Ayers DC, et al. Psychological attributes of preoperative total joint replacement patients: implications for optimal physical outcome. J Arthroplast. 2004;19(7):125–30.

    Article  Google Scholar 

  16. Yeung AS, et al. Clinical outcomes in measurement-based treatment (comet): a trial of depression monitoring and feedback to primary care physicians. Depress Anxiety. 2012;29(10):865–73.

    Article  Google Scholar 

  17. Weenink J-W, Braspenning J, Wensing M. Patient reported outcome measures (PROMs) in primary care: an observational pilot study of seven generic instruments. BMC Fam Pract. 2014;15(1):1–8.

    Article  Google Scholar 

  18. Franklin P, et al. Framework to guide the collection and use of patient-reported outcome measures in the learning healthcare system. eGEMs. 2017;5(1):17.

  19. Greenhalgh J, et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J patient-reported outcomes. 2018;2(1):1–28.

    Article  Google Scholar 

  20. Powers III. J.H., et al., Reliability, validity, and responsiveness of influenza patient-reported outcome (FLU-PRO©) scores in influenza-positive patients. Value in Health. 2018;21(2):210–8.

    Article  Google Scholar 

  21. Powers III. J.H., et al., Patient-reported outcome assessments as endpoints in studies in infectious diseases. Clin Infect Dis. 2016;63(suppl_2):S52–6.

    Article  Google Scholar 

  22. Berenguer A, et al. Are smartphones ubiquitous?: an in-depth survey of smartphone adoption by seniors. IEEE Consum Electron Mag. 2016;6(1):104–10.

    Article  Google Scholar 

  23. Oshima SM, et al. Association of smartphone ownership and internet use with markers of health literacy and access: cross-sectional survey study of perspectives from project PLACE (Population Level Approaches to Cancer Elimination). J Med Internet Res. 2021;23(6):e24947.

    Article  Google Scholar 

  24. Demeke HB, et al. Trends in use of telehealth among health centers during the COVID-19 pandemic—United States, June 26–November 6, 2020. Morbidity and Mortality Weekly Report. 2021;70(7):240.

    Article  CAS  Google Scholar 

  25. Garfan S, et al. Telehealth utilization during the Covid-19 pandemic: a systematic review. Comput Biol Med. 2021;138:104878.

    Article  CAS  Google Scholar 

  26. Calvert M, et al. Maximising the impact of patient reported outcome assessment for patients and society. BMJ. 2019;364:k5267

  27. Kyte D, et al. Development of an electronic patient-reported outcome measure (ePROM) system to aid the management of patients with advanced chronic kidney disease. J patient-reported outcomes. 2020;4(1):1–9.

    Article  Google Scholar 

  28. Powers JH, et al. Development of the Flu-PRO: a patient-reported outcome (PRO) instrument to evaluate symptoms of influenza. BMC Infect Dis. 2015;16(1):1.

    Article  Google Scholar 

  29. Van Berkel N, Ferreira D, Kostakos V. The experience sampling method on mobile devices. ACM Comput Surv (CSUR). 2017;50(6):1–40.

    Article  Google Scholar 

  30. Coons SJ, et al. Capturing patient-reported outcome (PRO) data electronically: the past, present, and promise of ePRO measurement in clinical trials. The Patient-Patient-Centered Outcomes Research. 2015;8(4):301–9.

    Article  Google Scholar 

  31. Powers JH 3rd, et al. Reliability, validity, and responsiveness of InFLUenza patient-reported Outcome (FLU-PRO(c)) scores in influenza-positive patients. Value Health. 2018;21(2):210–8.

    Article  Google Scholar 

  32. Rikkert MGO, et al. Using patient reported outcomes measures to promote integrated care. Int J Integr Care. 2018;18(2):8.

    Article  Google Scholar 

  33. Menni C, et al. Quantifying additional COVID-19 symptoms will save lives. The Lancet. 2020;395(10241):e107–8.

    Article  CAS  Google Scholar 

  34. Menni C, et al. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat Med. 2020;26(7):1037–40.

    Article  CAS  Google Scholar 

  35. Dreyer NA, et al. Self-reported symptoms from exposure to Covid-19 provide support to clinical diagnosis, triage and prognosis: an exploratory analysis. Travel Med Infect Dis. 2020;38:101909.

    Article  Google Scholar 

  36. Richard SA, et al. Performance of the inFLUenza patient-reported Outcome Plus (FLU-PRO Plus) instrument in patients with Coronavirus Disease 2019. Open Forum Infect Dis. 2021;8(12):ofab517. Oxford University Press US.

  37. van der Wees PJ, et al. Development of a framework with tools to support the selection and implementation of patient-reported outcome measures. J patient-reported outcomes. 2019;3(1):1–10.

    Google Scholar 

  38. Greenhalgh T, et al. Virtual online consultations: advantages and limitations (VOCAL) study. BMJ open. 2016;6(1):e009388.

    Article  Google Scholar 

  39. Greenhalgh J, et al. Functionality and feedback: a protocol for a realist synthesis of the collation, interpretation and utilisation of PROMs data to improve patient care. BMJ open. 2014;4(7):e005601.

    Article  Google Scholar 

  40. El-Shabasy RM, et al. Three wave changes, new variant strains, and vaccination effect against COVID-19 pandemic. Int J Biol Macromol. 2022;204:161–8.

    Article  CAS  Google Scholar 

  41. Drew DA, et al. Rapid implementation of mobile technology for real-time epidemiology of COVID-19. Science. 2020;368(6497):1362–7.

    Article  CAS  Google Scholar 

  42. Powers III. J.H., et al., Performance of the inFLUenza patient-reported Outcome (FLU-PRO) diary in patients with influenza-like illness (ILI). PLoS ONE. 2018;13(3):e0194180.

    Article  Google Scholar 

  43. Room TWHB, FACT SHEET: The American Jobs Plan. 2021: https://www.whitehouse.gov/briefing-room/statements-releases/2021/03/31/fact-sheet-the-american-jobs-plan/.

  44. Bokolo AJ. Exploring the adoption of telemedicine and virtual software for care of outpatients during and after COVID-19 pandemic. Irish Journal of Medical Science (1971-). 2021;190(1):1–10.

    Article  CAS  Google Scholar 

  45. Barbosa W, et al. Improving access to care: Telemedicine across medical domains. Annu Rev Public Health. 2021;42:463–81.

    Article  Google Scholar 

  46. Karimi M, et al. National Survey Trends in Telehealth Use in 2021: Disparities in Utilization and Audio vs. Video Services. Issue Brief. 2022. https://aspe.hhs.gov/sites/default/files/documents/4e1853c0b4885112b2994680a58af9ed/telehealth-hps-ib.pdf.

  47. Barnett ML, et al. Trends in outpatient telemedicine utilization among rural Medicare beneficiaries, 2010 to 2019. American Medical Association: In JAMA Health Forum; 2021.

    Book  Google Scholar 

  48. Aissaoui N. The digital divide: a literature review and some directions for future research in light of COVID-19 Global Knowledge, Memory and Communication. 2021.

  49. Chang JE, et al. Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-COVID era. Milbank Q. 2021;99(2):340–68.

    Article  Google Scholar 

  50. Christie A, et al. Decreases in COVID-19 cases, emergency department visits, hospital admissions, and deaths among older adults following the introduction of COVID-19 vaccine—United States, September 6, 2020–May 1, 2021. Morb Mortal Wkly Rep. 2021;70(23):858.

    Article  CAS  Google Scholar 

  51. Scobie HM, et al. Monitoring incidence of COVID-19 cases, hospitalizations, and deaths, by vaccination status—13 US jurisdictions, April 4–July 17, 2021. Morbidity and Mortality Weekly Report. 2021;70(37):1284.

    Article  CAS  Google Scholar 

  52. Centers for Disease Control and Prevention. Weekly U.S. influenza surveillance report. 2021–2022. https://www.cdc.gov/flu/weekly/.

Download references

Acknowledgements

We would like to acknowledge the research teams at HealthPartners Institute, Kaiser Permanente Georgia, and Kaiser Permanente Mid-Atlantic States. This feasibility study was a large undertaking that involved support from multiple authors at each of our institutions. The authors would also like to thank Arnold Ventures Foundation for their support of our project.

Ethical Considerations: Written, informed consent was obtained for all participants prior to the participants being able to access the FLU-PRO survey. All methods and protocols were carried out in accordance with human subjects research guidelines. Protocols were approved by the HealthPartners Institute, Kaiser Permanente Georgia, and Kaiser Permanente Mid-Atlantic States’ Institutional Review Boards.

Concept and design

Gander, Roblin, Powers, Martinson.

Acquisition, analysis, or interpretation of data

Gander, Chrenka, Cromwell, Truitt, Sesay, Hudgins, Kodthala, Roblin, Whiting, Powers, Martinson.

Drafting of the manuscripts

Gander, Chrenka, Cromwell, Truitt, Sesay, Segall, Amouzou, Hudgins, Kodthala, Roblin, Deneal, Whiting, Powers, Martinson.

adafff

Critical revision of the manuscript for important intellectual content

Gander, Chrenka, Cromwell, Truitt, Sesay, Segall, Amouzou, Hudgins, Kodthala, Roblin, Deneal, Whiting, Powers, Martinson.

Statistical analysis

Gander, Chrenka, Cromwell, Truitt, Hudgins, Kodthala, Roblin, Whiting, Martinson.

Obtained funding

Gander, Roblin, Powers, Martinson.

Administrative, technical, or material support

Gander, Chrenka, Truitt, Sesay, Segall, Amouzou, Kodthala, Roblin, Deneal, Whiting, Powers, Martinson.

Supervision

Gander, Chrenka, Cromwell, Truitt, Sesay, Segall, Amouzou, Hudgins, Kodthala, Roblin, Deneal, Whiting, Powers, Martinson.

Funding

This work was funded by the Arnold Ventures Foundation.

Author information

Authors and Affiliations

  1. Center for Research and Evaluation, Kaiser Permanente Georgia, Atlanta, GA, USA

    Jennifer C. Gander, Lee Cromwell, Musu Sesay, Marni Segall, Sandra A. Amouzou & Alexander F. Hudgins

  2. HealthPartners Institute, Bloomington, MN, USA

    Ella Chrenka, Anjali R. Truitt, Prasanthi Kodthala & Brian C. Martinson

  3. Mid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, MD, USA

    Douglas Roblin, Adrienne N. Deneal & Thomas Whiting

  4. George Washington University School of Medicine, Washington, DC, USA

    John H. Powers

Authors
  1. Jennifer C. Gander
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ella Chrenka
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lee Cromwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anjali R. Truitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Musu Sesay
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marni Segall
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sandra A. Amouzou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Alexander F. Hudgins
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Prasanthi Kodthala
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Douglas Roblin
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Adrienne N. Deneal
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Thomas Whiting
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. John H. Powers
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Brian C. Martinson
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have met the International Committee of Medical Journal Editors criteria for authorship. Drs. Gander, Roblin, Powers, and Martinson led the investigation and the development of the manuscript. All authors have made substantial contributions to the concept and design, drafted, and revised it for content, approved the final version for publication, and agree to be accountable for all aspects of the work. A more detailed list of each authors contribution is provided above.

Corresponding author

Correspondence to Jennifer C. Gander.

Ethics declarations

Consent for publication

Not applicable.

Competing interests

The authors of this paper have no competing interest to declare.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original version of this article was revised: an author name error and affiliation error were corrected.

Supplementary Information

Additional file 1:

 Table 1. Bivariablelogistic regression investigating the odds of completing FLU-PRO Plus Day 1among those who completed online consent.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gander, J.C., Chrenka, E., Cromwell, L. et al. Systematic surveillance of patient-reported symptoms of viral respiratory tract infectious Syndromes in diverse populations. BMC Health Serv Res 22, 1591 (2022). https://doi.org/10.1186/s12913-022-08991-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-08991-3

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us