Cost-effectiveness of a whole-area testing pilot of asymptomatic SARS-CoV-2 infections with lateral flow devices: a modelling and economic analysis study

Abstract

Background

Mass community testing for SARS-CoV-2 by lateral flow devices (LFDs) aims to reduce prevalence in the community. However its effectiveness as a public heath intervention is disputed.

Method

Data from a mass testing pilot in the Borough of Merthyr Tydfil in late 2020 was used to model cases, hospitalisations, ICU admissions and deaths prevented. Further economic analysis with a healthcare perspective assessed cost-effectiveness in terms of healthcare costs avoided and QALYs gained.

Results

An initial conservative estimate of 360 (95% CI: 311–418) cases were prevented by the mass testing, representing a would-be reduction of 11% of all cases diagnosed in Merthyr Tydfil residents during the same period. Modelling healthcare burden estimates that 24 (16—36) hospitalizations, 5 (3–6) ICU admissions and 15 (11–20) deaths were prevented, representing 6.37%, 11.1% and 8.2%, respectively of the actual counts during the same period. A less conservative, best-case scenario predicts 2333 (1764–3115) cases prevented, representing 80% reduction in would-be cases. Cost -effectiveness analysis indicates 108 (80–143) QALYs gained, an incremental cost-effectiveness ratio of £2,143 (£860-£4,175) per QALY gained and net monetary benefit of £6.2 m (£4.5 m-£8.4 m). In the best-case scenario, this increases to £15.9 m (£12.3 m-£20.5 m).

Conclusions

A non-negligible number of cases, hospitalisations and deaths were prevented by the mass testing pilot. Considering QALYs gained and healthcare costs avoided, the pilot was cost-effective. These findings suggest mass testing with LFDs in areas of high prevalence (> 2%) is likely to provide significant public health benefit. It is not yet clear whether similar benefits will be obtained in low prevalence settings or with vaccination rollout.

Introduction

The aim of mass community SARS-CoV-2 (COVID-19) testing by lateral flow device (LFD) tests is to provide a significant and sustainable reduction in the prevalence of infection in an area. By offering testing to the whole community, it is possible to identify asymptomatic infections that would not otherwise be detected. Through prompt isolation and contact tracing of these newly identified asymptomatic or pre-symptomatic individuals, community transmission may be reduced. The LFD test is advantageous because of it’s speed and ease of deployment to large populations, compared to the polymerase chain reaction (PCR) test. LFD testing is deemed particularly beneficial for groups that are more likely to be asymptomatic, such as children and students, or for key workers who are in regular contact with vulnerable individuals.

The LFD test has been criticised for its poor sensitivity. Initial reports from the Liverpool pilot reported the specificity of LFD tests was 99.9%, but a sensitivity of 48.9% [1]. A more systematic analysis revealed a sensitivity of 40% [2]. A study of LFD tests in UK university students found sensitivity can be as low as 3% [3]. The sensitivity of the test is highly dependent on the expertise of who the test is administered by [4]. LFD tests sensitivity is significantly influenced by viral load [3]. Viral load peaks in asymptomatic cases are lower than for symptomatic cases, affecting the usefulness of LFDs as a screening tool. Furthermore, the false sense of reassurance a negative test result could provide may lead to more risky behaviours and reduced adherence to social distancing and lockdown restrictions. Clear information regarding the low sensitivity of LFD tests was not always readily available to the public [5]. Consequently, the use of LFD mass testing as a screening tool has provoked controversy in the medical community and the media [6, 7]. It has been argued that mass testing with LFD devices is not cost-effective, with one estimate of the cost of detecting a single asymptomatic case by LFD mass testing as much as £30,000 [8]. However, such estimates do not take into account the potential costs saved in terms healthcare burden and quality life years lost though onward transmission that would have occurred had mass testing not identified these cases.

There has been previous studies that have evaluated the merits of the LFD testing in terms of sensitivity and specificity [1], but not on their cost-effectiveness in the context of mass testing. Most existing studies assessing cost-effectiveness of mass testing focus on screening exercises with PCR (see [9] for a review). One recent modelling study by [10] assessed the cost-effectiveness of LFD testing in a range of hypothetical scenarios in the USA. Their results also suggest mass testing by LFD is highly cost-effective, with repeated testing over a 28-day increment with 2-week isolation for positive cases suggested as the most cost-effective. A cost–benefit analysis of mass testing in Barcelona, suggests that absolute cost–benefit ratio of asymptomatic mass testing was low (< 1), but when including monetised healthcare outcomes in the analysis, showing a high (> 1) cost–benefit ratio [11]. A comparable economic analysis has not yet been conducted on the Liverpool pilot. An interim analysis using Bayesian time-series modelling estimated a small non-significant reduction in cases and hospitalisations due to mass-testing [12]. However, a more recent synthetic control analysis estimated a 40% reduction in hospitalisations was achieved [13]. As illustrated by the Barcelona study, by not considering monetised healthcare outcomes, it is difficult to infer whether economic impacts were significant.

Prior to national roll out across the UK [14], pilots were conducted in Liverpool [12] and in the Merthyr Tydfil and the Lower Cynon Valley area of South Wales during November to December 2020. [15]. Merthyr Tydfil had, at the time, one of the highest incidence rates of COVID-19 in the UK along with associated illness and deaths (248.6 cases per 100,000 in the first week of the pilot [16]). The Welsh Index of Multiple Deprivation shows that Merthyr Tydfil has some of the most deprived areas in Wales [17]. In addition, smoking rates are high in both areas in comparison to the health board and the Welsh average, and there are high levels of overweight and obesity and higher prevalence of long term conditions, all factors associated with poor COVID-19 outcomes [17].

In the present study, we assess the impact of the whole area testing pilot in Merthyr Tydfil by modelling the number of onward infections prevented using the reproduction number (Rt) in Merthyr Tydfil at the time and applying a series of assumptions on the natural history of the infection and the performance of the test. These estimates are then applied to a time-lagged regression model to estimate the number of hospitalizations, ICU admissions and deaths that would have arisen from the prevented cases. We also perform sensitivity analysis on some parameters that underlie the assumptions made in the analysis and derived estimates from likely worst-case and best-case scenarios. Finally, the cost-effectiveness, in terms of incremental cost-effectiveness ratio (ICER) per QALY gained is estimated.

Method

Study design

Modelling and economic analysis was carried out from a health perspective. Epidemiological modelling, using data collected as part of a mass testing pilot, was used to estimate cases prevented ad time-series analysis used to estimate subsequent healthcare burdens prevented. Economic analysis was applied to the modelled burdens to estimate QALYs gained and healthcare costs averted.

Ethics, data privacy and information governance

The study was reviewed by the Public Health Wales Research and Development Office and determined to be usual practice in public health, and therefore did not require external NHS ethics committee approval. Investigation of communicable disease outbreaks and surveillance of notifiable disease is permitted under Public Health Wales Establishment Order. Data were held and processed under Public Health Wales information governance arrangements in compliance with the Data Protection Act, Caldicott Principles and Public Health Wales guidance on the release of small numbers. All methods were performed in accordance with the relevant guidelines and regulations. Informed consent was obtained from all individuals prior to their participation in the mass testing pilot. No data identifying protected characteristics of an individual were released outside Public Health Wales. Data used in the study was anonymised prior to analysis. Permission to use this data in this study was granted by the organisers of the mass testing pilot.

Sample data

Lateral flow device (LFD) tests were performed as part of the whole area testing pilot that took place between 21st November 2020 and 20th December 2020 in Merthyr Tydfil and lower Cynon Valley. Testing was offered to individuals aged 11 or over, living or working in the area and who were not experiencing symptoms of COVID infection at the time of testing. Full details of the pilot are provided in the pilot evaluation [15]. For ease, the present analysis was restricted to Merthyr Tydfil only as this is a whole self-contained local authority, whereas lower Cynon Valley comprises a sub-region of the Rhondda Cynon Taf local authority. Of 48,834 tests that took place during the pilot, 33,822 were taken at one of 14 test centres in Merthyr Tydfil. Of these, 712 positive LFD tests who reported being asymptomatic were used for the analysis. This represents 2.1% of tests performed. Testing was performed using the INNOVA SARS-CoV-2 Rapid Antigen test device, the same device used in the Liverpool pilot, which has a sensitivity of 40.0% and specificity of 99.9% [2].

Analysis

Estimates of cases prevented

All analysis was carried out using R software. Assumptions made in estimating cases prevented are summarised in Table 1. These assumptions have been reviewed by a consultant epidemiologist. LFD positive asymptomatic cases during the testing period were counted for each day. From this, the number of LFD positive tests that were confirmed by a PCR test, either on-site or by an independent test no more than 14 days later, were counted. From this number, the number of true positives were estimated. Firstly we identified all LFD test positives with a confirmatory PCR test result (classified as true positives). Secondly, we estimated the number of true positives in those who did not have a confirmatory PCR test from the positive predictive value (PPV) of the LFD test.

Table 1 Assumptions of the analysis

A proportion of asymptomatic cases will go on to develop COVID-19 symptoms. The number expected to remain asymptomatic was estimated by scaling the number of true positives by the proportion of asymptomatic cases that did not report having symptoms when interviewed coincidentally as part of a case–control study that took place during the testing pilot. From 198 LFD positive results, 87 (44%) reported symptoms when followed up [20]. Asymptomatic infections are assumed, had mass testing not taken place, to not self-isolate and therefore would contribute fully to the next generation of infections. A small proportion of secondary infections (0.2) were assumed for symptomatic cases, resulting from infectious contact made prior to onset of symptoms or as a failure of self-isolation. The sum of these two count constitutes the initial set of cases identified though the testing pilot but not directly prevented (also referred to as 0th generation infections). The process for obtaining this count is visualised in the flowchart in Fig. 1a.

Fig. 1

Flowchart of process for (a) estimating number of cases detected by mass testing that would not otherwise be detected and for (b) estimating number of cases prevented, and other healthcare outcomes. This process is followed iteratively for each day in the testing period

From the initial set of cases, the 1st generation infections that would have occurred from these cases is estimated by multiplying by the reproduction number (Rt—mean and 95% CI) for that day by the number true positive LFD test cases. A scaling factor of 0.58 is also applied to take into account the relative reduction of infectability of asymptomatic cases compared to symptomatic cases [18]. The daily reproduction number Rt was estimated using the “EpiEstim” R package [21] based on actual PCR-tested case counts in Merthyr Tydfil local authority area (see Fig. 2a).

Fig. 2

a Estimated Rt in Merthyr Tydfil over the testing period. b Positive lateral flow test results from community testing pilot in Merthyr Tydfil, with estimates for true positives and proportion remaining asymptomatic. Note that missing data in the weekend of 11th to 13th December is due to temporary suspension of the mass testing on these dates. c Daily estimates of 1st generation and 2nd + higher generation infections prevented

The number of subsequent (2nd and higher generation) infections were estimated by iteratively multiplying the estimated Rt by the estimated number of prevented infections from the previous day, each time scaling the number of cases from the previous generation by the proportions of asymptomatic and symptomatic-infectious cases, and scaling the asymptomatic portion by the relative infectability of asymptomatic cases, as done for the 0th generation.

The number of infections estimated prevented for a given day is the sum of the 1st generation and 2nd and higher generation infections. 0th generation infections are excluded from this sum as they occurred before the LFD test and therefore cannot be considered preventable.

The number of infections estimated for a given day is then distributed over an infection window of 14 days, which has a log-normal distribution $\left(\mu =1.63, \sigma =0.5\right)$, with a median of 5 days. The parameters for this distribution were taken from the meta-analysis of McAloon et al. [19]. For 1st generation infections (infections arising direct from the LFD positive cases) the infectious period is assumed to be centred around the LFD test date, to take into account the likely delay between the date of infection and the date of the LFD test. Only the infectious period after the LFD test date is considered as any new infections generated before the LFD test would not have been prevented. For 2nd and higher generation infections, the whole infection window is considered and the start of the infection window is the date the infection is seeded. A flowchart depicting this process is shown in Fig. 1b and a mathematical description is given in supplementary materials 1.

The proportion of symptomatic cases that contribute to new infections was estimated from contact tracing data (restricted to the period of the mass testing exercise and to the Merthyr Tydfil local authority area). The contacts for all cases that reported symptoms were cross referenced to identify those that also have a PCR-confirmed positive test result. The proportion of contacts of symptomatic cases that were themselves identified had a positive test results was 0.2.

Some of the parameters underlying the assumptions of the analysis have a high degree of uncertainty. Three parameters were highlighted for sensitivity analysis and used to produce worst-case and best-case scenarios for the number of cases prevented (see Supplementary materials 2 for details). These additional scenarios were included in the subsequent analyses.

Estimates of healthcare burdens prevented

The estimated number of cases prevented were further used to predict three healthcare outcomes: Hospitalisations, ICU admissions and deaths. A time-lagged log-linear regression model was trained on corresponding surveillance data for residents of Merthyr Tydfil Local Authority [16]. Separate models were built for each healthcare outcome with multiple lagged case counts treated as the predictor variables and the burden measure as the outcome measure. Hospitalisation data were restricted to those where a positive PCR test was obtained no later than 2 days after admission. This rules out likely nosocomial infections, as opposed to those that are community-acquired. All data were log transformed and smoothed with a LOESS filter (span = 0.2). Models are built from time-lagged case counts with delays of 12–50, 24–50 and 30–50 days for hospitalizations, ICU admissions and death, respectively. These lag ranges were determined to be optimal by testing multiple lag windows of 1–50 days and comparing the AIC/BIC values.

The estimated number of cases prevented computed in the previous section were then fed into the model to predict daily counts of each healthcare burden prevented. The counts are then summed across all days modelled to obtain a final count of each healthcare burden.

Economic analysis

Standard cost-effectiveness analysis (sometimes called cost-utility analysis) methods were used [22] with a health perspective. There have been several attempts to attach costs to COVID-19 outcomes [23]. For this work, we used a ‘bottom up’ method using Wales-specific medical costs and intervention costs collected as part of the mass testing pilot. The cases and associate hospital burdens were multiplied by estimated costs and QALYs lost. The incremental cost-effectiveness ratio (ICER) cost per QALY gained was also calculated. This was done for the main estimate and the best- and worst- case scenarios derived from the previous sensitivity analysis. The QALY losses from COVID-19 cases included cases, deaths, hospital admissions and a conservative (low) estimate of the lost QALYs from long COVID-19. The programme costs for the mass testing pilot in Merthyr Tydfil were estimated as £515,688 [15]. Costs were measured in GBP, at 2020 prices. QALYs lost from COVID-19 deaths were discounted at 1.5% per annum in line with UK Treasury Green Book. Other costs and QALYs were not discounted as they were assumed to occur within-year. Table 2 shows the estimated outcomes and unit costs and QALYs for each outcome. No costs were assumed for cases that do not end in hospital or dying. Further details of the cost and QALY estimates are provided below. In addition to cost-effectiveness, additional cost–benefit measures were also calculated. Net monetary benefit (NMB) and return on investment (ROI) were calculated for the intervention. These were calculated from a health perspective, based on the cost savings plus the value of QALYs gained, valuing discounted QALYs at UK Treasury rate of £60,000 [24].

Table 2 QALYs and monetary costs per unit

COVID-19 community cases

COVID-19 community cases cause a 0.000889 QALY loss, equivalent to asthma for 7 days [25]. With the model results, QALY losses for long COVID-19 are also applied to a percentage of cases.

Post-viral syndromes

We used 0.15 QALYs lost (0.3 utility score decrement for six months) which is the equivalent of moderate fibromyalgia [26] for six months [26], but also may be similar to other syndromes that are similar to ‘long COVID-19’ post-viral syndromes. We had clinical input into using fibromyalgia: as a similar syndrome but in reality long COVID-19 may be several distinct syndromes. We applied this to 10% of COVID-19 cases as ONS data suggests that around 10% of people in the COVID-19 infection survey report symptoms for 12 weeks or more [27].

Deaths

Deaths result in a QALY loss of 7.24 QALYs per death, based on data from the Secure Anonymised Information Linkage (SAIL) databank that estimated mean years of life lost of 9.97 when compared with age- and sex- specific life tables, multiplied by average health-related quality of life (HRQoL) utility index for people aged 75 and over in the UK (0.726) [28]. This may be a high estimate as it includes age but does not directly factor in co-morbidities in terms of mortality risk. Discounted at 1.5% per annum this is 6.78 QALYs. COVID-19 deaths results in an excess healthcare cost of £232 [29]. This is likely a low estimate.

Hospital and ICU admissions

We assume 0.01126 QALYs lost per admission and 0.274 HRQoL loss for 15 days, based on people in hospital for pneumonia [30]. Cost per admission of £7,085 is (NHS Wales Finance Delivery Unit). We assume 1.402 HRQoL loss for 9 days [30] (Note this means that the QALYs for this health state are ‘worse than death’). Cost per ICU admission is £22,198 (NHS Wales Finance Delivery Unit). Our hospital costs are higher than other estimates like NHS tariff/ reference costs as they include costs of PPE and other costs associated with the pandemic.

Results

Estimates of cases prevented

A total of 33,823 lateral flow device (LFD) tests were performed in Merthyr Tydfil County Borough between 21st November 2020 and 20th December 2020, of which 712 were positive and reported asymptomatic. Of these 697.4693878 were estimated to be true positives (528 confirmed by PCR and an additional 169 estimated from PPV). Of these, 391 were estimated to not subsequently develop symptoms and contribute to secondary infections. In addition, an estimated 62 of those developing symptoms would still contribute to secondary infections. Figure 2b shows the daily count of these numbers over the testing period. Summary counts are provided in Table 3. PPV was estimated to be (0.98, from the proportion of LFD positives that had follow-up PCR (539), that were confirmed true positives (528).

Table 3 Estimated cases prevented (with 95% CI where applicable)

Including 1st and higher generation infections, a total of 360 (95% CI: 311—418) infections are estimated to have been prevented in the period from 21st November 2020 to 3rd January 2021. This represents 12.4% (10.7%, 14.4%) of the actual count of clinical cases diagnosed in the same period (2,900), an effective reduction of 11% from the would-be case count without mass testing (3,260). This also represents a reduction in mean daily incidence rate of 13.9 (12—16.1) cases per 100,000 per day (actual mean daily incidence rate was 112 cases per 100,000 per day). This is the sum of 169 (156—182) estimated 1st generation infections and 192 (156—236) estimated 2nd and higher generation infections (see Table 3). Figure 2c shows daily estimates of infections prevented over the course of the mass testing and the following 14 days.

Table 4 shows the estimated cases prevented for worst- and best-case scenarios, the worst-case scenario being 160 (142, 180) cases prevented, which is 44% of the original estimate and 6% of the actual case count. The best-case scenario is 2333 (1764, 3115), is 648% of the original estimate and 80% of the actual case count.

Table 4 Total estimated cases prevented and associated healthcare outcomes for original, worst-case and best-case scenarios (with 95% CI)

Estimates of healthcare burdens prevented

The model estimates, from the cases prevented during the mass testing, a total of 24 (16—36) hospitalizations, 5 (3—6) ICU admissions and 15 (11—20) deaths were prevented in the period between 21st November 2020 and 10th February 2021. These represent reductions of 6.37%, 11.1% and 8.2% of the actual hospitalization, ICU admissions and deaths, respectively. The estimated time course of these three outcomes are shown in Fig. 3.

Fig. 3

Estimates of hospitalizations, ICU admissions and deaths prevented from cases prevented by mass testing

Table 4 shows the estimated hospital burdens prevented for worst- and best-case scenarios. Healthcare burdens prevented for the best-case scenario are 66 (49—92) hospitalizations, 9 (7—12) ICU admissions and 33 (26—42) deaths. These represent reductions of 17.6%, 22% and 18.4% of the actual hospitalization, ICU admissions and deaths, respectively.

Economic analysis

Results of economic analysis for the original, best- and worst- case scenarios are shown in Table 5. A breakdown of QALYs and cost savings associated with each outcome is shown in Table 6. The programme was highly likely to be cost-effective, with an ICER of £2,143 per QALY gained (95% CI: £860—£4,175). NMB was £6.2million (£4.5 m – £8.4 m), which represents an ROI of approximately £12 (£9—£16) per £1 invested in the programme. Even in worst-case scenario, ICER was £5,562 (£8,280—£3,339), NMB was 3.5 m (2.6 m—4.8 m) and ROI was £7 (£5-£9). In the best-case scenario, ICER was £-608 (£35—£-122), NMB was 15.9 m (12.3 m—20.6 m) and ROI was £31 (£24—£40). Additional sensitivity analysis on NMB estimates are shown in supplementary materials 3.

Table 5 Results of cost-effectiveness analysis for original, worst-case and best-case scenarios (with 95% CI)

Table 6 QALYs gained and cost savings for original scenario of the mass testing intervention (with 95% CI). Negative cost savings indicate net costs

Discussion

As a result of the whole area testing pilot in Merthyr Tydfil, a non-negligible number of cases, hospitalisations and deaths, that would have otherwise occurred, were likely to have been prevented. Over one tenth of cases that would have occurred over a 6 week period were prevented. This forecast translates into a predicted reduction of 6–12% in burden on the healthcare system relative to actual healthcare burden at the time. These results were obtained for a conservative, but probable scenario. In a less-conservative, best-case scenario, the number of prevented cases could be as much as three quarters of the actual case count, and a reduction of 18–23% in burden on the healthcare system.

Economic analysis shows the mass testing pilot was cost-effective. This compares favourably with cost-effectiveness thresholds such as the quoted National Institute for Health and Care Excellence (NICE’s) threshold of £30,000 per QALY gained, or the health production cost for the NHS in England, which is said to be around £15,000 per QALY gained [31]. Even with pessimistic assumptions, the intervention is likely to be cost-effective.

Strengths and weaknesses of the study

This study is the first to evaluate the effectiveness of LFD mass testing by modelling the number of cases and healthcare burdens prevented and quantifying the benefits in terms of quality life years gained and healthcare costs saved. A strength of the present study, is that it uses real-world data from a mass testing pilot and regional estimates of Rt from actual case counts, as opposed to simulations. This eliminates many of the assumptions relating to the population and the costs and strategy of the mass testing programme. The main assumptions of our model were on the dynamics of onward transmission for which we validated with regional data where available.

A comparable economic analysis has not yet been conducted on the Liverpool pilot. An interim analysis using Bayesian time-series modelling estimated a non-significant reduction in cases and hospitalisations due to mass-testing [12]. However, a more recent synthetic control analysis estimates a much larger percentage hospitalisations prevented [13]. Our analysis shows that even a very small number of cases prevented can prove substantially cost-effective. Applying our model to the Liverpool data will therefore be prudent to determine if this is the case in Liverpool.

The model underlying these estimates makes a number of assumptions about the transmission dynamics of SARS-CoV-2. For example, the transmissibility of SARS_CoV-2 and the level of adherence to testing and self-isolation in symptomatic cases. Where possible, we used existing regional data to back up these assumptions. However, there is considerable uncertainty surrounding the definition of some of the parameters used. Our sensitivity analysis (See supplementary materials 2) shows that the model is very sensitive to the relative infectability of asymptomatic cases, which is one parameter for which there is high uncertainty [18] and is inconsistently defined [32]. This makes the output of the model particular susceptible to errors in this parameter and should be considered when evaluating these estimates.

There are some important assumptions to consider in interpreting the results. Not all LFD positives had confirmatory PCR tests. Although we attempt to use PPV to estimate the proportion of additional true positives, this method could lead to overestimation of the number of cases prevented. The impact of contact tracing on behaviours of asymptotic cases was not considered. We assumed all asymptomatic cases would fail to isolate, but contact tracing would advise contacts of cases to self-isolate and be tested, so a proportion of asymptomatic cases would self-isolate without the mass testing intervention.

Another factor that was not considered in the model is differences in transmissibility between age and demographic groups. These groups will likely differ in their likelihood of transmitting and acquiring an infection. Such considerations would require generating Rt estimates for sub-populations for which sample sizes are likely to be too small. Also, the impact of new variants of concern and vaccine roll-out were not considered. These were not factors at the time of the pilot, but have subsequently become significant factors in the transmission dynamics of COVID-19. We have been quite conservative and not included costs of non-hospital cases or long COVID cases, and the QALY losses from long COVID cases are also quite conservative. The true costs and QALYs lost from COVID-19 are likely to be higher than these estimates which do not include post-hospital rehabilitation, the likely full implications of multi-organ damage or informal care costs, productivity and wages lost. The indirect opportunity costs for people whose NHS treatment is delayed because of treating COVID-19 patients is not included. Overall, if more costs were included, the benefits would be likely to be greater. There are other onward costs of the COVID-19 system that are not included in this such as the costs of contact tracing, but these would fall both onto the intervention costs side, and on the cost savings side of any cost-effectiveness equation, because preventing cases would reduce the need for future contact tracing. We did not include equity impacts in the calculation of costs and benefits; Merthyr Tydfil has a deprived population so adding an additional weighting for equity would likely mean that the benefits would be greater [33]. However, it may also mean that people are exposed to more competing risks of morbidity and mortality. We have not included the benefits to individuals in Merthyr Tydfil from employment in supporting the mass testing programme. There is evidence from another combined modelling and economic analysis study mass testing has economic benefits by reducing the number of uninfected workers having to isolate [34]. We did not carry out probabilistic sensitivity analysis or vary the economic parameters except in one-way sensitivity analyses; so the range of results is influenced only by the upper and lower estimates of the epidemiological modelling.

Implications for policymakers and future research

The central estimate of NMB was £5.8 million which means a benefit–cost ratio of around 11 for the £516,000 that the programme cost, or an ROI of around £12 per £1 spent. This is in line with the finding that public health interventions often have a high return on investment [35].

These encouraging results come despite the poor sensitivity of the LFD test, as evidenced in other studies [2]. We were unable to calculate sensitivity from this pilot due to no systematic PCR follow-up for LFD negative results. However, in a community setting, sensitivity is likely to be highly variable and can be as low as 3% [36]. The implication of our contrasting results is that consideration of mass testing programmes should be based on other factors, in addition to test sensitivity. It should also be noted that since the pilot, other manufacturers have developed LFDs with improved sensitivity [37]. It is likely that cost-effectiveness would be even higher in a mass testing programme using one of these newer devices.

While generalisability of these finding to other regions with different socio-demographic profiles and under different prevalence scenarios needs further investigation, it is clear that mass testing, in conjunction with other NPI measures, should be considered an important tool in control of COIVD19.

The cost-effectiveness of mass testing will likely be influenced by specific demographics and local healthcare policies. In particular, the school mass testing programme currently undertaken in the UK should be evaluated for cost-effectiveness. Of particular concern is the impact of disease prevalence. The Merthyr Tydfil pilot was conducted at a time of particularly highly prevalence (peak incidence during the pilot was 282 cases per 100,000 population). Had the pilot been conducted more recently, when prevalence was lower, the cost-effectiveness will likely be reduced. Supplementary simulation work suggest that the mass testing program would no longer be cost-effective if background prevalence had been lower than 2% (see supplementary materials 4). In addition to increasing the number of false negatives, the number of false positives will increase in a low prevalence scenario. This may have further detrimental societal impact beyond cost-effectiveness (e.g. lost wages, productivity etc.). The confounding effects of vaccine roll-out and variants of concern should be considered. Since the majority of the benefits are through prevented deaths, if vaccines reduce the infection fatality ratio, the cost-effectiveness of preventing transmission will decrease. Estimating cost-effectiveness in these different scenarios will be useful for deriving thresholds of community prevalence at which mass testing will become cost-effective, which will benefit local policy makers when planning future mess testing programmes.

Another aspect our analysis does not consider is the potential behavioural impact of a negative LFD test result. While a positive LFD test result will motivate an individual to self-isolate and reduce further transmission, a negative LFD test result may lead to a false sense of assurance and individuals engaging in more risky behaviour. Further research into the impact of such behavioural changes is therefore warranted.

Conclusions

Although prior evidence indicates the sensitivity of the LFD test is low, its use in mass testing appears to have helped prevent a number of secondary cases during the Merthyr Tydfil pilot, which occurred during a period of high infection prevalence. A non-negligible number of hospitalisations and deaths are also likely to have been prevented. Economic analysis demonstrates that the pilot was a highly cost-effective exercise. However, there are a number of parameters, in particular prevalence, which are likely to significantly influence cost-effectiveness. Future work to identify thresholds of cost-effectiveness across these parameters will provide policy makers valuable guidance in planning of future mass testing programmes.