Reliability of predictive models to support early decision making in the emergency department for patients with confirmed diagnosis of COVID-19: the Pescara Covid Hospital score

Abstract

Background

The hospital management of patients diagnosed with COVID-19 can be hampered by heterogeneous characteristics at entry into the emergency department. We aimed to identify demographic, clinical and laboratory parameters associated with higher risks of hospitalisation, oxygen support, admission to intensive care and death, to build a risk score for clinical decision making at presentation to the emergency department.

Methods

We carried out a retrospective study using linked administrative data and laboratory parameters available in the initial phase of the pandemic at the emergency department of the regional reference hospital of Pescara, Abruzzo, Italy, March–June 2020. Logistic regression and Cox modelling were used to identify independent predictors for risk stratification. Validation was carried out collecting data from an extended timeframe covering other variants of concern, including Alpha (December 2020–January 2021) and Delta/Omicron (January–March 2022).

Results

Several clinical and laboratory parameters were significantly associated to the outcomes of interest, independently from age and gender. The strongest predictors were: for hospitalisation, monocyte distribution width ≥ 22 (4.09; 2.21–7.72) and diabetes (OR = 3.04; 1.09–9.84); for oxygen support: saturation < 95% (OR = 11.01; 3.75–41.14), lactate dehydrogenase≥237 U/L (OR = 5.93; 2.40–15.39) and lymphocytes< 1.2 × 10³/μL (OR = 4.49; 1.84–11.53); for intensive care, end stage renal disease (OR = 59.42; 2.43–2230.60), lactate dehydrogenase≥334 U/L (OR = 5.59; 2.46–13.84), D-dimer≥2.37 mg/L (OR = 5.18; 1.14–26.36), monocyte distribution width ≥ 25 (OR = 3.32; 1.39–8.50); for death, procalcitonin≥0.2 ng/mL (HR = 2.86; 1.95–4.19) and saturation < 96% (HR = 2.74; 1.76–4.28). Risk scores derived from predictive models using optimal thresholds achieved values of the area under the curve between 81 and 91%. Validation of the scoring algorithm for the evolving virus achieved accuracy between 65 and 84%.

Conclusions

A set of parameters that are normally available at emergency departments of any hospital can be used to stratify patients with COVID-19 at risk of severe conditions. The method shall be calibrated to support timely clinical decision during the first hours of admission with different variants of concern.

Background

Between December 2019 and April 2021, the coronavirus SARS-CoV2 affected at least 130 Million people diagnosed with COVID-19 in 223 different countries, of which almost 2.9 Million died [1]. The range of complications associated with the disease generated enormous pressure on hospitalisation and intensive care [2,3,4].

Clinical symptoms range from pauci-symptomatic states presenting fever, cough and fatigue, to more severe forms including acute respiratory distress syndrome (ARDS) and/or other critically severe conditions [5]. Mortality is higher in the elderly, when the disease is at an initial stage and the virus replicates. In some cases, this activity triggers a second aggressive phase in which hyper-inflammation may require immediate urgent hospitalisation [6]. Patients admitted with these characteristics show a 20-fold increased risk of death compared to non-severe cases [7].

Timely identification of COVID-19 patients at higher risk of severe complications may enable early hospitalisation, risk-tailored clinical treatment and optimal allocation of human and technical resources [8]. In these circumstances, a predictive risk tool may conveniently identify subjects requiring immediate attention, serving as a reference in the relative lack of evidence-based guidelines. A variety of methods have been used to identify subjects with a higher risk of target outcomes e.g. COVID-19 diagnosis, hospital admission and negative prognosis. However, their direct applicability in clinical practice appears still limited [9,10,11].

The triage of COVID-19 patients requires multiple parameters to be assessed by a multidisciplinary team at presentation to the Emergency Department (ED) [12]. Characteristics that need to be routinely evaluated include clinical signs and symptoms known to be associated with subsequent prognosis [9] and various laboratory measurements e.g. decrease in albumin and increase in C-reactive protein (CRP), lactate dehydrogenase (LDH), lymphopenia and other haematological parameters that have been less investigated in clinical settings [9, 13].

In the framework of everyday care provided during the first wave of the outbreak, our aim was to identify patients diagnosed with COVID-19 with a higher risk of four key outcomes: hospital admission, mechanical ventilation, admission to Intensive Care Unit (ICU) and death.

The present study is a retrospective review of all consecutive cases presented at the ED over 3 months, addressing the following key research questions:

• Which demographic, clinical and laboratory parameters are associated with a higher risk for each of the key outcomes identified?

• How accurate are predictive models using only characteristics available at presentation to the ED?

Materials and methods

Study population

We performed a retrospective analysis of all consecutive patients presented with a confirmed diagnosis of COVID-19 at the ED of the General Hospital of Pescara (Abruzzo, Italy), between 1st march – 30th June 2020. The diagnosis was confirmed through a swab test performed by the hospital personnel in the same occasion. The time frame follows the start of the outbreak of COVID-19 in Italy [14]. At that time, the province of Pescara was the province most affected in the macro-area of Central/Southern Italy (+ 15.6% excess deaths compared to previous years) [15].

The Pescara General Hospital (PGH) is an urban 650-bedded tertiary facility of regional reference for adult traumas, acute diseases of neurosurgical interest, and COVID-19. It includes two ICUs: an 11-bedded facility, receiving critically ill patients from other EDs in the region and most of the other wards in the same hospital, and a 24-bedded COVID Unit, specifically opened to respond to the emergency of the pandemic.

Data were merged from records available at different sources, including the hospital discharge abstract database, the computerized hospital information system including personal health records of laboratory measurements, and paper-based clinical abstracts, from which other characteristics were manually extracted.

Two additional samples were collected for external validation, using a large subset of entries to the same ED between December 2020–January 2021 (corresponding to the transition from the “Wild” lineage of COVID-19 to the “Alpha” variant) [16] and January–March 2022 (covering the transition between the “Delta” variant and “Omicron”) [17]. All data were included in an Excel sheet accessed for statistical analysis.

The study was conducted according to the Declaration of Helsinki (amended version). The local Health Administrative Board reviewed and approved the study plan submitted by the Infectious Diseases Unit, the Emergency Department and the Laboratory Staff of PGH. The use of anonymised clinical and laboratory data for institutional research purposes was granted through signed informed consent upon hospital admission by all patients included in the study. Specific consent for the conduction of the study was not considered required, as confidentiality was guaranteed, and no specific interventions were performed beyond the ordinary good standard clinical practices.

Target characteristics

The study targeted four different outcomes after presentation to the ED: hospitalisation, oxygen support, admission to the ICU, and death (in or out of hospital). By definition, not all combinations were possible, e.g. a patient can be discharged at home and experience a fatal event or can progress through all transitions from hospitalisation. Some states directly implied others e.g. a patient receiving oxygen support or admitted to the ICU must be hospitalised.

Demographic and clinical characteristics, signs and symptoms and laboratory measurements were considered as potential predictors for the selected outcomes.

Age and gender were used as the only demographic characteristics of interest. For clinical variables, we considered a history of six comorbid conditions: diabetes, cardiovascular diseases (CVD), obesity, cancer, End Stage Renal Disease (ESRD), chronic obstructive pulmonary disease (COPD) and hypertension. These characteristics were recorded as part of routine clinical practice, using standardized definitions adopted by all hospitals in Italy.

Signs and symptoms related to COVID-19 were also considered, e.g. fever, cough, asthenia, diarrhea and dyspnea. Accurate recording was ensured by a standard national protocol applied routinely to monitor patients diagnosed with COVID-19.

Finally, eight device-assisted and laboratory parameters were considered as potential predictors of patients’ prognosis, based on high/low levels considered as potential correlates to the severity of the disease [18]: procalcitonin, lactate dehydrogenase (LDH), monocyte distribution width (MDW), oxygen saturation level, D-dimer, prothrombin time, C-reactive Protein (CRP) and lymphocyte counts. Among these, only the levels of procalcitonin were pre-assigned, based on a threshold considered valid for all outcomes (value = 0.2 ng/mL). The remaining seven parameters were transformed into binary variables by using optimal thresholds for each of the four outcomes, defined by the maximum Youden index in a univariate ROC analysis [19]. The procedure allowed identifying a set of low/high levels for measures without prior targeted cut-offs for COVID-19-related outcomes (see Table 1).

Table 1 Categorization of continuous variables identified by ROC analyses (max Youden index)

Statistical analysis

We investigated hospitalisation and death in/out of hospital through follow-up of all patients in the study population. Oxygen support and admission to intensive care included only those hospitalised with less than 70 years, to avoid potential bias of deaths occurring before either option as a competing risk [20]. The choice is consistent with the guideline of avoiding transfer to ICU for patients aged >75y. During the reference timeframe, only 3 patients died in the selected subgroup.

Descriptive analysis included the calculation of mean and standard deviation for continuous variables and absolute and relative frequencies for categorical variables. Optimal thresholds for hospitalisation and oxygen support were used to calculate frequencies for the reference population in the overall sample and among those aged<70y hospitalised. Logistic regression was used for univariate and multivariate odds ratios (OR) of non-fatal outcomes [21]. Survival analysis was used to consider censoring in the analysis of time to fatal events, calculated as the difference between the date of death and admission to the hospital. For survivors, the censoring time was defined as the lag between presentation at the ED and the earliest date between the first negative swab result (an indicator of full recovery) and the date of study closure (30th June 2020). Cox proportional hazards was used for the calculation of univariate and multivariate hazard ratios (HR) for time-to-event analysis [22]. An alpha level of 0.05 was used to present odds and hazard ratios together with their 95% confidence intervals (95%CI) and p values. Forest plots were used to visualize results.

Predictive factors using a fully automated four-step backward elimination process in all multivariate regressions. Age and gender were forced in all models, with all other variables sequentially excluded in three consecutive rounds using a p value ≥0.20, ≥0.10 and ≥ 0.05.

Predictive formulas were defined using the regression coefficients as follows [23]:

• a) odds/hazard ratios < 1 were turned into their reciprocal value and assigned a negative sign;

• b) odds/hazard ratios greater or equal to 1.5 were rounded to their next integer value (to reflect increased risk by a multiplicative factor);

• c) odds/hazard ratios between 1 and 1.5 were transformed into their difference from 1 (rounded to the second decimal, to reflect increased risk by a percentage).

Total risk scores for each patient in the database were computed separately for each outcome, adding up all coefficients indicated above for all significant variables. A ROC analysis was performed using a separate 2 × 2 “confusion matrix” for every possible threshold applied to the total score [24]. The best threshold was defined by the highest value of the Youden Index [19]. Point estimates and confidence intervals were computed using the bootstrap for sensitivity, specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV) and the De Long method for the Area Under the Curve (AUC) [25].

The same measures were produced for both the study sample and the additional datasets collected for external validation. Survival results for validation samples were only available as odds ratios, as Cox proportional hazards was not applicable due to the unavailability of time of negative swab test.

Reliability analysis of the predictive formula was summarized using ROC curves showing optimal thresholds and sensitivity, specificity, AUC with point estimates and 95% confidence intervals. All statistical analyses were carried out by developing ad hoc software in the R language [26].

Results

A total of 536 consecutive admissions with a positive molecular assay to SARS-CoV2 were recorded at the ED in the reference period. The samples collected for validation were equal to 224 out of 639 (35%) for December 2020 – January 2021 and 375 out of a total of 872 (43%) for January–March 2022.

Among subjects hospitalised in the baseline study sample, a total of 174 consecutive admissions involved patients aged less than 70 years. The samples used for validation were equal to 95 out of 226 (42%) for December 2020 – January 2021 and 56 out of a total of 135 (41%) for January–March 2022.

All details of the association including risk ratios and 95% confidence intervals for the total population admitted to the ED at baseline and those hospitalised below 70 are reported in Tables 2 and 3 respectively.

Table 2 Descriptive frequencies for the overall sample of patients admitted to the Emergency Department ^a

Table 3 Descriptive frequencies for the sample of patients hospitalised with age less than 70 years^a

Hospitalisation and deaths (in or out of hospital)

The mean age (standard deviation) was 63.2 (19.1) y, with 50.7% of males. The most frequent comorbidities were hypertension (36.2%), CVD (25.9%) and diabetes (16.0%). The most frequent symptom was diarrhea (87.1%), followed by fever (79.9%), cough (43.7%) and dyspnea (43.3%). The parameter that was most frequently abnormal was prothrombin time < 95% (68.5%), followed by saturation < 96% (46.1%), lymphocytes< 1 × 10³/μL (44.6%), D-dimer≥0.72 mg/L (43.3%), LDH ≥ 240 U/L (41.2%), CRP ≥ 21 mg/L (37.3%) and MDW ≥ 22 (28.2%).

A total of 365 subjects (68.1%) were hospitalised after presentation to the ED. Among the main baseline characteristics observed, only cough and asthenia were not significantly associated with hospitalisation. An additional year of age was associated with a 6% increased risk of hospitalisation, while males were almost 70% more likely to be hospitalised. An over ten fold association with increased risk was found for CRP ≥ 21 mg/L, obesity, ESRD and saturation < 96%. Associated risk was over five fold for MDW ≥ 22, D-dimer ≥0.72 mg/L, diabetes, hypertension, LDH ≥ 240 U/L and lymphocytes< 1 × 10³/μL. Moderate association was found for dyspnea, COPD, CVD, cough and cancer. Patients with diarrhea were over 50% less likely to be hospitalised.

A total of 136 subjects (25.4%) died in or out of hospital in the reference timeframe, following presentation to the ED with a confirmed diagnosis of COVID-19. Among the main characteristics, gender, obesity, fever and cancer were not significantly associated with survival. An additional year of age was associated with a 8% increased risk of death, and an over threefold increase for procalcitonin≥0.2 ng/mL, D-dimer≥1.04 mg/L, ESRD, saturation < 96%, CVD, CRP ≥ 43 mg/L and COPD. Risk was more than doubled for lymphocytes< 0.7 × 10³/μL, LDH ≥ 307 U/L, dyspnea, prothrombin time < 77% and hypertension. Moderate increased risk was found for diabetes and MDW. On the other hand, three conditions were found to be associated with survival after diagnosis with COVID-19: patients with diarrhea were 80% less likely to die, with almost a 50% risk reduction also observed for those presented with asthenia and cough.

Oxygen support and intensive care

The mean age (standard deviation) was 54.3 (10.1) y, with 64.4% of males. The most frequent comorbidities were hypertension (33.9%) and diabetes (18.4%). The most frequent symptoms were fever (92.5%) and cough (60.9%). The parameter found most frequently abnormal was LDH ≥ 237 U/L (74.7%), followed by lymphocytes < 1.2 × 10³/μL (67.8%), MDW ≥ 24 (67.2%), D-dimer≥0.55 mg/L (64.9%), CRP ≥ 42 mg/L (62.6%) and prothrombin time < 88% (55.7%). Less than half of the patients had a saturation < 95% (47.7%) and procalcitonin ≥0.2 ng/mL (24.7%).

A total of 129 subjects (74.1%) received oxygen after hospitalisation. None of the demographic and clinical characteristics were significantly associated with such treatment. Among relevant parameters, a level of increased risk between four and fifteen-fold was found for saturation < 95%, LDH ≥ 237 U/L, MDW ≥ 24, CRP ≥ 42 mg/L and lymphocytes< 1.2 × 10³/μL. Over three-fold association was found for procalcitonin≥0.2 ng/mL. D-dimer and prothrombin time were not associated with oxygen support.

A total of 51 subjects (29.3%) aged below 70 were admitted to the ICU after hospitalisation. Among subject characteristics, only age and CVD were associated with increased risk of intensive care. Among laboratory and device-assisted measurements, risk was at least five times higher for LDH ≥ 334 U/L and saturation < 92%, more than tripled for MDW ≥ 25 and CRP ≥ 68 mg/L, more than doubled for lymphocytes< 0.7 × 10³/μL. Procalcitonin, D-dimer and prothrombin time were not associated with admission to ICU.

Multivariate analysis

The results of multivariate analysis are shown along with those obtained from validation over different samples in Fig. 1.

Fig. 1

Forest plot showing results of multivariate analysis for multiple outcomes and validation periods (* Odds Ratios computed for validation samples)

Among adjustment terms, only age (unit increase) was associated with a higher risk of experiencing three of the four outcomes of interest, namely hospitalisation (OR = 1.02; 1.01–1.04), intensive care (OR = 1.05; 1.00–1.10) and death (HR = 1.06; 1.04–1.08).

Taking into account all potential confounders, further eight characteristics were significantly associated to an increased risk of hospitalisation. The risk was over four fold for patients with MDW ≥ 22 (OR = 4.09; 2.21–7.72), and over three fold for people with diabetes (OR = 3.04; 1.09–9.84). All other significant characteristics had a risk between two and three times higher than their reference category: saturation < 96% (OR = 2.96; 1.54–5.82), CRP ≥ 21 mg/L (OR = 2.83; 1.54–5.17), procalcitonin≥0.2 ng/mL (OR = 2.80; 1.20–7.23), D-dimer≥0.72 mg/L (OR = 2.26; 1.22–4.20), dyspnea (OR = 2.18; 1.19–4.05) and prothrombin time < 95% (OR = 2.09; 1.16–3.76).

Only one clinical characteristic was associated with an increased risk of death: ESRD (HR = 2.20; 1.30–3.72). Other four categories with abnormal levels were significantly associated with an increased risk: procalcitonin≥0.2 ng/mL (HR = 2.86; 1.95–4.19), saturation < 96% (HR = 2.74; 1.76–4.28), D-dimer≥1.04 mg/L (HR = 1.87; 1.24–2.81) and LDH ≥ 307 U/L (HR = 1.74; 1.19–2.53).

After hospitalisation, oxygen support was over ten times more likely for saturation < 95% (OR = 11.01; 3.75–41.14), almost six times higher for LDH ≥ 237 U/L (OR = 5.93; 2.40–15.39) and over four times higher for lymphocytes< 1.2 × 10³/μL (OR = 4.49; 1.84–11.53).

Admission to ICU recorded the highest level of association found for a clinical characteristic, being almost sixty times more likely for ESRD (OR = 59.42; 2.43–2230.60). Other independent risk factors were LDH ≥ 334 U/L (OR = 5.59; 2.46–13.84), D-dimer≥2.37 mg/L (OR = 5.18; 1.14–26.36), MDW ≥ 25 (OR = 3.32; 1.39–8.50) and lymphocytes< 0.7 × 10³/μL (OR = 2.58; 1.18–5.80).

Several variables among those presented above were not significant in the validation over the extended timeframe.

Risk scores

Risk scores were directly derived from predictive models, according to the method outlined above. The predictive accuracy of the algorithms is presented in Table 4. The validation results are available as Supplementary Data.

Table 4 Algorithms for the calculation of scores

For hospitalisation, an optimal cut point equal to 12 achieved sensitivity of 80% (95%CI: 75–83%) and specificity of 87% (81–92%). The overall performance was very high (AUC = 91%; 89–94%), suggesting that the predictive model may be suitable for regular use at ED.

The same performance was achieved with a cut-off equal to 10 by the mortality predictive model (AUC = 91%; 87–94%), with a higher sensitivity (89%; 82–93%), but lower specificity (79%; 74–83%).

Similar levels were obtained using a cut-off of 8 for oxygen support, achieving a sensitivity of 87% (80–91%) and a specificity of 77% (63–87%), with a slightly lower overall performance (AUC = 87%; 82–92%).

Finally, admission to ICU recorded the lowest performance with a cut-off equal to 10 and a sensitivity = 82% (70–90%), specificity = 71% (63–79%), and AUC = 81% (73–89%).

The external validation denoted an optimism in the accuracy measured by the AUC ranging between 3 and 19%.

An overall ROC analysis of the performance achieved by all predictive models is presented in Fig. 2.

Fig. 2

Overall ROC analysis showing the performances of algorithms over different validation samples

Discussion

The Italian National Health System (Servizio Sanitario Nazionale, SSN) delivers national guidelines for the standard provision of health services across the country. However, when Italy was first hit by the coronavirus, hospitals experienced a rapid increase of hospitalisations, followed by an exceptional shortage of medical equipment and a limited set of recommendations regarding best practices [14]. The situation required immediate measures to organise local practices and provide urgent care. The rationale for the present study originates from the practical needs emerged during this phase of the emergency. To carry out our investigation, we strived to collect a large dataset that could help respond to the research questions posed at the outset.

Our results are consistent with those obtained by other studies carried out on different outcomes, including mortality, progression to severe/critical status, recovery, length of hospital stay, admission to ICU, intubation, duration of mechanical ventilation, acute distress respiratory, cardiac injury and thrombotic complications. Evidence showed predictive factors including age, comorbidities, vital signs, image features, gender, lymphocyte count and C-reactive protein [9].

In this study, we considered specific predictors, e.g. optimal cutoffs for MDW, which have not been previously considered by algorithms for COVID outcomes using data available during the early phase of admission, to support clinical decision in the Emergency Department.

We identified eight known targets of hospital management to be significantly associated with hospitalisation independently from age and gender: diabetes, dyspnea, procalcitonin≥0.2 ng/mL, MDW ≥ 22, saturation < 96%, D-dimer≥0.72 mg/L, prothrombin time < 95% and CRP ≥ 21 mg/L. Among them, MDW deserves to be presented in more detail. MDW is a novel haematological parameter recently introduced for the diagnosis of sepsis [18, 27], which has been already targeted by recent investigations [28]. Our study confirmed a significant association between MDW and sepsis [29,30,31], consistently with other viral diseases [29, 30, 32]. Changes in morphology and volumetric size of white blood cells are a well-documented consequence of cellular activation upon early infection, as a part of innate immunity response [33]. Monocytes are involved in the early response to infection, acting as first interceptors of the invading microorganism, for phagocytosis and further immune processing. Further studies showed changes in the morphology during inflammation [34], differentiating into amoeboid cells, as assessed by microscopy after Giemsa staining, and increased expression of functional markers such as CD16 [35]. Another study found that monocytes homeostasis and morphology appear considerably perturbed in patients with SARS-CoV2 infection [13], with MDW showing significantly elevated values also in patients with COVID-19, compared to other upper respiratory tract infections [36, 37]. As a consequence, the possibility of monitoring monocyte size in parallel with routine blood cell counts and other clinical indicators at presentation to the ED may represent a convenient tool to identify patients at high risk among those diagnosed with COVID-19.

As a first relevant transition after hospitalisation, we focused on oxygen support among patients with COVID-19 aged less than 70 years. In this group of subjects, dyspnea, chest distress and respiratory rate were found to be highly associated with oxygen therapy, suggesting strict monitoring for parameters that can be highly associated with clinical deterioration and adverse outcomes [38]. In addition to saturation < 95%, we also found levels of LDH ≥ 237 U/L and lymphocytes< 1.2 × 10³/μL as adequate targets for oxygen therapy.

Hospitalized patients aged<70y had a higher risk of being admitted to ICU if one of the following cases applied: ESRD, LDH ≥ 334 U/L, MDW ≥ 25, D-dimer≥2.37 mg/L and Lymphocytes< 0.7 × 10³/μL. Other studies found heart disease, COPD and heart rate to be significantly associated with ICU, in addition to ESRD [39]. The dominant role found for the latter in our model can be explained by the organ damage noticed in many patients admitted to ICU, which is associated with a high prevalence of limited renal function [40]. A high value of LDH is an indicator of tissue/cell destruction that is frequently used to monitor tissue damage associated with a wide range of disorders, including liver and interstitial lung disease. Patients with severe pulmonary interstitial disease present an increased LDH as one of the most important prognostic markers of lung injury [41, 42]. Lymphopenia is a biological disorder in patients with COVID-19 frequently considered as predictor of severe infection and myocardial injury, ARDS, and mortality [43]. Lymphopenia is a common consequence of infection caused by cytokine-induced reaction [44]. Reduced CD4+ T-cell and CD8+ T-cell levels promote viral replication and predict worsening outcome [45]. T -cells appear lower and functionally exhausted, and patients with COVID-19 with T- cells≤800/μL may still require urgent intervention due to a higher risk of further deterioration of their condition [46].

Several characteristics at entry to the ED were predictive of mortality in or out of hospital, independently from age and gender, including ESRD, procalcitonin≥0.2 ng/mL, LDH ≥ 307 U/L, saturation < 96% and D-dimer≥1.04 mg/L. We did not find known any significant predictive factor associated with increased mortality, differently from other reports addressing male gender, symptoms < 10 days prior to hospital admission, diabetes, coronary heart disease, chronic liver disease [47], white cell count, temperature, respiratory rate, lymphocytes and platelets [48, 49]. The significant association found between procalcitonin and mortality could be attributed to bacterial co-infection rather than viral replication [50].

Regarding our second research question, we were able to identify accurate risk scores, based on significant coefficients extracted from predictive models. The risk scores showed an average performance ranging from moderate to very high. The performance was very high for hospitalisation, with AUC = 91% (89–94%) and PPV = 94% (90–96%), mortality prediction, with an AUC = 91% (87–94%) and a low PPV = 60% (53–66%), as well as for oxygen support, with AUC = 87% (82–92%) and PPV = 92% (86–95%). The performance was slightly lower for admission to ICU, with AUC = 81% (73–89%) and PPV = 54% (43–65%). Notably, the NPV was high for death (95%, 92–97%) and admission to ICU (91%, 83–95%), indicating that a value of the score below the threshold, particularly for ICU, may be effective in ruling out major complications and considering oxygen therapy as a viable solution.

The comparison between predictive models estimated in our study and other scientific reports may be challenging. Predictive models use different types of cohorts to investigate a variety of outcomes including confirmed diagnosis, disease severity, ICU and mortality. Studies are conducted in different hospital environments, using different laboratory standards and applying heterogeneous inclusion/exclusion criteria e.g. tuberculosis, influenza and bronchitis [9]. Variables included in predictive models are also heterogeneous, from vital signs to image features, contact with other infected individuals, lymphocyte count, liver enzyme and red distribution width [9]. In many cases, data of non-hospitalized patients are limited and do not include laboratory and imaging analyses [51], drawing conclusions only from a limited set of characteristics, with an AUC as high as 90% [52].

To better respond to our second research question, we evaluated the accuracy of our predictive models under real life conditions during the second wave in late 2020.

We found that the overall performance was fairly robust for oxygen support, hospitalisation and death. On the other hand, the results were less satisfactory for admission to ICU (ranging between 66 and 81%, which corresponds to a sensitivity drop between 51 and 82%). A possible explanation may be related to the heterogeneous characteristics of patients admitted during the second wave, which changed the prognosis and thus the predictive ability of models specified under different conditions.

In summary, we identified a set of key parameters that can be translated into risk scores to inform clinical practice. The advantage of this approach is that it can be directly applied to the next patient entering the ED, even with a pocket calculator. A substantial barrier for the continuous update and adaptation of this method is the limited interoperability of health databases in most European contexts, which makes the process of data acquisition particularly burdensome. Improving the digitalization and standardization of health information at hospitals across Europe will be paramount to strengthen our preparedness to future outbreaks and favour the adoption of research methods in clinical practice [53,54,55].

Finally, several limitations of our study are worth to be outlined, along with their consequences on the future use of the algorithms.

Firstly, this is a retrospective study carried out at a single hospital, enrolling a limited number of patients during the initial phase of the COVID-19 outbreak in Italy. Consequently, the results may not equally apply to other jurisdictions and/or institutions. However, our report provides a focused stratification of subjects entering the ED that can be informative for clinical practice on a global scale.

Secondly, among the outcomes identified, only death represents a clinically objective measure, while all others reflect decisions made by clinicians at the hospital. Therefore, we cannot infer on the validity of the same models under different settings and variable conditions. However, it is a specific feature of the approach to be able to identify factors orienting practices, so that pragmatic guidelines can be offered when they are not readily available. This feature implies a repeated application of the method to make it locally relevant.

Thirdly, we searched for predictive variables among measures readily available at presentation to the ED. As for all observational studies, we may have missed a subset of characteristics that could be potentially predictive either at entry or in the subsequent follow up in and out of hospital. Nevertheless, we kept our focus on measures that are used in normal practice, so that they can be realistically applied.

Fourthly, the statistical models adopted did not take into account transitions between states e.g. oxygen support followed by intensive care. Our choice was based on the need to facilitate interpretation, rather than enhancing the statistical properties of our methods.

Finally, the stability of the algorithms should be considered in the broader perspective of a continuously evolving virus, in which the continuous update of the vaccination status may blunt the clinical response. This consideration triggers important closing reflections.

The vaccination status of individuals in our samples could not be ascertained from the available data. However, vaccination in Italy ramped up only in February–March 2021, reaching over 90% for those 60+ in early 2022 (see ECDC data at: https://vaccinetracker.ecdc.europa.eu/public/extensions/COVID-19/vaccine-tracker.html#uptake-tab). Therefore, while obviously vaccination was not an issue in 2020, it is also fair to assume that all patients included in the “Wild/Alpha” sample in Dec 2020-Jan 2021 were not vaccinated, while the majority of those included in the “Delta/Omicron” sample could be considered vaccinated. On this ground, we can fairly assume that the vaccination status may have biased estimates obtained for the Omicron sample. On the other hand, the fact that this characteristic is not continuously available in routine hospital databases makes it a difficult candidate for risk evaluation in everyday practice (other than asking directly to the patient, which may be prone to information bias). Therefore, in practical usage, it could be appropriate to consider it as a non- observable confounder, thus incorporating its effect in the association found for other variables.

Overall, the predictive models appeared moderately accurate, irrespective of the variant dominating the reference period. The AUC did not fall below 70%, except for intensive care, for which no variable was significant under the “Delta/Omicron” variants, and only ESRD was significant under the “Wild/Alpha” variants. This slight deviation may show that the subgroup of those below 70 years may not reflect practices following the initial emergency of the “Wild” outbreak (which is partially true also for oxygen support, where only lymphocytes retain their significance). Therefore, the method seems still generally useful to pick subjects at significantly increased risk.

The significance of specific variables in predictive models may also highlight differences in their relevance according to the lineage of the virus. The measurement of MDW, D Dimer, and Prothrombin did not appear as relevant in the evolution of COVID-19 for matters related to hospitalisation. On the other hand, saturation retained its predictive value for hospitalisation and oxygen therapy, while it was not significant to predict survival in the “Delta/Omicron” sample. These aspects seem to indicate changes in the characteristics of the population affected by different variants, reflecting the higher incidence of oxygen therapy and intensive care in the Wild/Alpha, as opposed to a sharp decrease in the incidence of all events under the Delta/Omicron variants (nearly 50% hospitalisations and deaths compared to other variants). On the other hand, the Wild/Alpha sample showed an increased 50% admission to intensive therapy, as opposed to the baseline population. Only few variables could be considered relevant for the new variants of COVID-19, primarily for hospitalisation (dyspnea, procalcitonin, saturation, reactive C protein) and death (ESRD, procalcitonin, LDH, D Dimer).

Since the model was estimated on the original virus, we may assume that our scoring algorithms will continue to be particularly relevant for any future new SARS-type respiratory diseases, where severe cases of pneumonia are more frequent in the absence of vaccination. The same model could be possibly continuously updated, estimating the weights of significant variables repeatedly over time.

Conclusion

We identified demographic, clinical and laboratory parameters associated to hospitalisation, oxygen support, ICU admission and death in a population admitted to a large regional hospital with a confirmed diagnosis of COVID-19. Risk scores derived from multivariate models showed moderate to high predictive accuracy in flagging subjects with more severe prognosis, based upon the early evaluation of personal characteristics at the ED. The method can be conveniently applied to support clinical decisions under different conditions, using targeted data collection at a single point of entry. Recalibration of the scoring algorithms will be needed to cope with the continuous evolution of the virus in different contexts.