Cost-consequence analysis of computer vision-based skin prick tests: implications for cost containment in Switzerland

Abstract

Background

Skin prick tests (SPTs), or intraepidermal tests, are often the first diagnostic approach for people with a suspected allergy. Together with the clinical history, SPTs allow doctors to draw conclusions on allergies based on the sensitization pattern. The purpose of this study is to investigate the potential cost consequences that would accrue to a Swiss University hospital after the adoption of computer vision-based SPTs.

Methods

We conducted a cost-consequence analysis from a hospital’s perspective to evaluate the potential cost consequences of using a computer vision-based system to read SPT results. The patient population consisted of individuals who were referred to the allergology department of one of the five university hospitals in Switzerland, Inselspital, whose allergology department averages 100 SPTs a week. We developed an early cost-consequence model comparing two SPT techniques; computer vision-based SPTs conducted with the aid of Nexkin DSPT and standard fully manual SPTs. Probabilistic sensitivity analysis and additional univariate sensitivity analyses were used to account for uncertainty.

Results

The difference in average cost between the two alternatives from a hospital’s perspective was estimated to be CHF 7 per SPT, in favour of the computer vison-based SPTs. Monte Carlo probabilistic simulation also indicated that SPTs conducted using the computer vision-based system were cost saving compared to standard fully manual SPTs. Sensitivity analyses additionally demonstrated the robustness of the base case result subject to plausible changes in all the input parameters, with parameters representing the costs associated with both SPT techniques having the largest influence on the incremental cost. However, higher sensitization prevalence rates seemed to favour the more accurate standard fully manual SPTs.

Conclusion

Against the backdrop of rising healthcare costs especially in Switzerland, using computer-aided or (semi) automated diagnostic systems could play an important role in healthcare cost containment efforts. However, results should be taken with caution because of the uncertainty associated with the early nature of our analysis and the specific Swiss context adopted in this study.

Background

Skin prick tests (SPTs) and/or blood tests are often the first diagnostic approach for people with a suspected allergy [1]. In addition to being simple, safe and quick, SPTs are cheap and very reliable for diagnosing IgE-mediated allergic disease in patients with rhino conjunctivitis, asthma, urticaria, anaphylaxis, atopic eczema and suspected allergy [2]. SPTs provide evidence for sensitization and can help to confirm the diagnosis of a suspected type I allergy. Together with clinical history, SPTs allow to draw conclusions on allergies based on the sensitization pattern [1]. As such, SPT results only indicate sensitization and cannot be used as standalone allergy diagnostic methods [3].

Standard fully manual SPT procedure consists of putting drops of commercial extracts of food or pollen on the skin (usually forearm) which is then pierced with a small lancet, allowing the allergen to come into contact with the skin mast cells [4]. A positive reaction is shown by the typical “wheal and flare” reaction, which results from the erythema, pruritus and oedema that develop within 10 to 20 min. The SPT result is consequently given by the wheal diameter [2]. A wheal diameter of 3 mm or greater than the negative control is often arbitrarily considered a positive SPT result [1, 5]. The first SPTs were performed by the English physician Blackley in 1865 with the technique itself having evolved very little as it continues to be performed entirely manually today [6]. This standard manual SPT technique has several drawbacks that limit the utility of the tests, in many cases restricting them to a semi-quantitative assessment. Some of the drawbacks of standard SPTs include the possibility of making mistakes when reading the test results and more importantly, possible difficulties in the manual measurement of the reaction wheal sizes [7, 8].

Over the last 9 years, a team of engineers and allergologists in Spain have been working on the development of an computer vison-based (semi) automated system for reading SPT results which has led to the development of an electromedical device, Nexkin digital skin prick test (Nexkin DSPT), that helps automatically locate and measure wheal reactions. Nexkin DSPT captures a three-dimensional images of the forearm and using computer vision software, locates and measures the surface of the wheals caused by the allergens [9]. The device’s intended use is to aid in (semi) automated wheal detection and in the reading of the skin prick allergy test results. Nexkin DSPT’s computer vision-based technology therefore has the potential of addressing some of the shortcomings associated with standard fully manual SPTs [9, 10]. Nexkin DSPT also has the potential of impacting costs in hospitals and allergology units where thousands of SPTs are carried out yearly. However, its potential is yet to be substantiated through studies estimating the cost implications of computer vision-based SPTs. Besides a clinical study that showed less variability in wheal size results read with the aid of Nexkin DSPT in comparison to those read manually [10], no other study - to the best of our knowledge - has attempted to comparatively evaluate automated and standard manual SPTs nor quantify potential cost consequences of adopting computer vision-based SPT systems.

In the current context of growing concerns on rising healthcare costs and spending in Switzerland [11, 12], the objective of this study is to evaluate - from a Swiss university hospital’s perspective – potential cost consequences of adopting a computer vision-based SPT system in comparison to standard fully manual SPTs.

Methods

Study design

We conducted a cost-consequence analysis (CCA) and developed an early cost-consequence model to estimate potential cost consequences of using a computer vison-based SPT system (Nexkin DSPT) to read SPT results. Our cost-consequence analysis was limited to an “early” model because of data limitations and the novel nature of the Nexkin DSPT’s computer vision-based technology which made it more difficult to obtain robust parametrical inputs on a range of outcomes and costs necessary for a broader analysis [10, 13,14,15].

The early cost-consequence model was developed with the aid of a decision-tree that compared the expected cost values for both the computer vision-based SPT technique and the standard fully manual SPT technique [13, 16]. The patient population in our study consisted of individuals who were referred to the allergology department of one of the largest University hospitals in Switzerland, Inselspital, whose allergology department averages 100 SPTs a week. The costs were therefore determined from a healthcare provider’s perspective (hospital), with the costs accruing to the allergology department at Inselspital. All costs were presented in 2023 prices using Swiss francs (CHF). The computer vison-based SPT (Nexkin DSPT) device’s purchase cost was provided in Euros and an exchange rate of CHF 1 = €1 was assumed. The time horizon was taken to be a 1-year duration, with the assumption that 5,200 SPTs are carried out at Inselspital annually. Due to the short time horizon duration considered, we did not apply any form of discounting. The cost-consequence model was based on a decision analytical model constructed as a decision tree using the software program TreeAge Pro 2023 v. R2.0.

Model structure

Fig. 1

Decision tree schematic

The decision tree in Fig. 1 illustrates our modelled patient pathway and begins with a decision node that represents the two alternatives: standard fully manual SPTs and computer vision-based SPTs. Subsequent branches are denoted with a chance node, illustrated as a circle, and represent the possible events for patients referred to Inselspital with suspected sensitization. All possible events were based on Inselspital’s head allergology clinician’s experience.

The decision tree schematic therefore modelled the pathway through which a patient referred to the Inselspital hospital’s allergology department went through. Patients entered the model after referral by respective general practitioners (GPs) and left after SPTs were conducted by either of the two SPT techniques. Using the decision-tree model, we calculated the expected costs based on the summation of pathway values weighted by the pathway probabilities for each one of the SPT alternatives. We programmed the pathway probabilities as mutually exclusive sequences of events along the pathways illustrated in the decision tree schematic [13, 16].

Based on the Inselspital’s head allergology clinician’s experience, we assumed that a patient whose terminal node ended with a false positive sensitization result would have no direct cost consequence from a hospital’s perspective. However, we also reasonably assumed that patients with false negative terminal node outcomes would eventually get an additional referral to Inselspital, which would result in a repetition of SPT related costs for Inselspital because they would have to reconduct the SPT. Since our analysis was conducted from the hospital’s perspective, we considered “further tests” related costs solely accruing to the hospital. These costs were imputed for every terminal node with a false negative sensitization outcome only. This implies that we only considered cost implications potentially accruing to the Inselspital’s allergology department and not those potentially accruing to the patient. We also did not consider other costs potentially associated with alternative pathways beyond those illustrated in our decision tree.

Model input parameters

Model parameters used in this study were obtained from two main sources, i.e., Literature (peer-reviewed literature, grey literature) and clinical expert input. Both the peer-reviewed and grey literature were retrieved from a pragmatic targeted literature search based on relevant key terms. Due to the limited number and quality of model inputs obtained from the targeted literature search, expert clinical input from the head of Inselspital’s allergology department and Nexkin Medical engineers was used to complement inputs retrieved from literature.

Pollen allergy (hay fever/pollinosis) in Switzerland

SPTs can be used to check for sensitization for multiple foods and inhalants [2], however, we chose to focus on pollen allergy SPTs since pollen allergy (hay fever/pollinosis) is the most common allergy in Switzerland [17]. While pollen allergy is a reaction to one or more types of pollen, the most common allergen positive sensitization is grass. In Switzerland, around 20% of the population suffers from a pollen allergy with the majority – 70% - sensitized to grass [17, 18]. For these reasons, our base case CCA considered sensitization for one of the most common types of pollen: timothy grass.

Hay fever (pollinosis) prevalence values were retrieved from the 1995 prevalence of atopy and pollinosis in the adult population of Switzerland (SAPALDIA¹) study [19]. This is the only study whose results we could use as model inputs due to the robust data collection used in the study. Indeed, this study collected data from a random sample of 9,651 subjects aged from 18 to 60 years (mean age 41.1 years, males 49.2%), with complete allergy test data available for atopy, pollinosis and atopic asthma. The study reported results for 8,357 adults, with males accounting for 51.3% of the reported adult sample. Despite the old age of the study, the reference study is the largest and only study - to the best of our knowledge - to have ever collected country level data in the adult Swiss population with the aim of examining the relationship between environment and respiratory symptoms and diseases in Swiss adults. Recent studies discussing prevalence in Switzerland have neither been methodologically equal nor superior because they did not collect data from a representative sample of the entire Swiss population [20,21,22]. Furthermore, the mean age (41.1 years) and gender distribution (males 49.2%) reported in the 1995 SAPALDIA study [19] was confirmed by the head allergology clinician at Inselspital to be indicative of the mean age and gender proportions of the patient population referred to Inselspital.

The highest rate of sensitization in SPTs (adjusted wheal diameters 3 mm) observed in the SAPALDIA study was for grass pollen (total 12.7%, males 14.5%, females 10.9%; p < 0.001), followed by house dust mite (8.9%, no difference between males and females) and birch pollen (total 7.9%, males 7.9%, females 8%) [19]. Since we focus on timothy grass induced hay fever pollinosis in our study, we used sensitization values reported for grass pollen (Table 1).

Diagnostic inputs

We obtained diagnostic accuracy values (sensitivity and specificity) for standard manual SPTs in a study whose objective was to identify the best cut-off level for SPTs [23]. Despite the study concluding that a cut-off level of over 0 mm was best at identifying those with allergen specific IgE, we used 3 mm accuracy values since these reflect current standards and guidelines [1, 5] followed at the allergology department of Inselspital.

Diagnostic accuracy for automated SPTs on the other hand have been known to be inferior to manual SPTs that are widely accepted as the gold diagnostic standard for SPTs [7, 10]. Unpublished data from a clinical trial conducted in Spain alluded to lower accuracy for computer-vision based Nexkin DSPT [10]. Another study that discussed automated SPT techniques also suggested slightly inferior accuracy for computer vison based SPT systems [7]. This led us to systematically assuming and applying 2.5% lower accuracy values for Nexkin DSPT.

Visual illustrations and further details on how we obtained accuracy values (sensitivity & specificity) for the two SPT techniques can be found in the supplementary materials file. These values are also reported in Table 1.

Table 1 Hay fever (timothy grass) based diagnostic model inputs

Costs

The costs (CHF) taken into consideration for this study -Table 2- reflect the costs associated with the two SPT techniques.

Table 2 Overview of cost related model inputs

Only direct SPT related costs accruing to the hospital and were factored in the base case analysis. The first cost category associated with both SPT alternatives was the cost of materials (allergen extracts, lancets, etc.) which according to the head allergology clinician of Inselspital, costs CHF 19.41 on average per test for both alternatives. Both alternatives therefore had similar material costs, with the main difference coming from the amount of time spent reading and recording the SPT results and the resulting human resource costs.

Consequently, the second cost category was the human resource costs (salary/employee costs) associated with the health practitioner involved in the SPT procedure. Since both the manual and the computer vision-based SPTs require a 15-min wait time for the allergen reaction to appear [24], the time considered in our cost calculations was 5.5 min and 3 min for the standard fully manual SPT computer vision-based Nexkin DSPT respectively. This is indeed the respective amount of time spent reading and recording the results after the 15-min wait. Time estimates were informed by unpublished clinical trial data used for a previous study that compared the variability of wheal size readings conducted by both SPT techniques [10]. We did not calculate human resource costs for the 15-min wait since nurses could reasonably multitask while waiting for the 15 min to elapse. Based on an average annual Inselspital nurse salary of CHF109,743.3, a 52-year week and a 42-hour contract, the average human resources component of the manual SPT and Nexkin DSPT was established at CHF 19.96 and CHF 10.89 respectively [25].

Since the Nexkin DSPT device is an asset, we also included depreciation costs which refers to the amount of capital that is ‘used-up’ in one year [26]. The simplest approach to accounting for depreciation is known as straight-line depreciation which assumes that the services from the capital item are evenly distributed over the useful life of the capital item [27]. NexkinMedical engineers suggested a 10-year useful life period for the Nexkin DSPT ^® device, and a capital value of CHF 10,400. The annual depreciation amounted to CHF 1,040 on top of which an annual service cost of CHF 1,040 was added. We assumed that an average of 5,200 SPTs were conducted every year at Inselspital’s allergology department, which led to the addition of an extra annual cost of CHF 0.4 per computer vison-based SPT (Table 2).

Further tests

As alluded to earlier, a false negative sensitization terminal node outcome could lead to “further tests” at Inselspital because of a second GP referral for a patient who experienced uncomfortable symptoms after a false negative sensitization outcome. Unlike false negative SPT results, patients with false positive outcomes are unlikely to undergo further tests because SPTs are not standalone diagnostic procedures [9] and need to be coupled with the patient’s clinical history for allergy diagnosis. We therefore assumed that “further tests” would lead to a repetition of the SPT related costs for the hospital, and only for patients with false negative sensitization outcomes. The parameters in the last row of Table 2 accounted for this by assuming false negative terminal node outcomes would lead to an average of two SPTs for concerned patients.

Decision rule

After assigning costs and probabilities to each corresponding event, the product of the costs and their matching probabilities was summed across the nodes for each branch of the decision tree in Fig. 1 to approximate the average cost of the branches for each of the two SPT alternatives [13, 16]. The incremental cost was estimated by finding the difference in expected average costs incurred when performing the SPTs manually and the expected average costs incurred when SPTs were performed with the aid of the computer vison-based system, Nexkin DSPT.

Sensitivity analysis

To check the robustness of the base case results, deterministic univariate sensitivity analyses were conducted. Analyses were performed by varying one parameter at a time within a plausible range while keeping the others constant, thereby estimating the impact of each parameter on the incremental cost [28].

The plausible ranges for all parameters reflected the range of uncertainty associated with our model input parameters and are illustrated in the “ranges” columns of Tables 1 and 2. The lower bound prevalence value of 0.10 was the lowest prevalence rate for timothy grass induced hay fever [19] recorded in our reference study while the upper bound 0.20 represents current Swiss hay fever prevalence rate among Swiss adults [17]. The ranges for sensitivity and specificity values on the other hand were based on the 95% confidence intervals (CI) provided in our reference study for SPT accuracy [23]. Cost parameter ranges were based on a 20% +- variation in prices observed by the head clinician of the allergology department at Inselspital. Finally, the number of further tests ranged from 1.6 to 2.4, which is also a +-20% range.

Probabilistic sensitivity analysis (PSA)

To account for the overall parameter uncertainty in our model, we conducted probability sensitivity analysis (PSA) by assigning probability distributions to each input parameter as depicted in the “distribution” column of both Tables 1 and 2. Cost parameters were assigned Gamma distributions while probability parameters were assigned Beta distributions [29]. We conducted PSA by drawing Monte Carlo simulations from the probability distributions. The probabilistic model simulations stabilised at approximately 10,000 Monte Carlo simulations which yielded 10,000 result estimates.

Results

Cost-consequence model

From the perspective of a Swiss university hospital’s (Inselspital) allergology department, the average expected value of costs saved by performing SPTs on patients using a computer vision-based system was approximately CHF 7 per test. This result is illustrated in the incremental active payoff tornado chart in Fig. 2.

Sensitivity analysis

Tornado diagrams show the extent to which uncertainty in individual parameters affect the incremental cost as the various input parameters vary within the ranges provided in Tables 1 and 2 [30]. Results from the deterministic tornado diagram shown in Fig. 2 showed that all the parameters remained within a negative incremental cost, thus demonstrating robustness of the base case result to plausible changes in all the input parameters. Blue bars show the impact of low values of the parameter on the incremental cost of using computer vision-based Nexkin DSPT as opposed to using standard fully manual SPT procedures. Red bars on the other hand show the impact of high values of the respective parameters on the incremental cost.

Fig. 2

Incremental active payoff tornado chart of base case analysis

Parameters representing the costs associated with the two SPT alternatives were the most influential because changes in these two parameters had the largest influence on the incremental cost. As expected, higher costs of fully manual (computer vision-based Nexkin DSPT) SPTs reduced (increased) incremental costs of conducting computer vison-based SPTs while lower costs of manual (Nexkin DSPT) SPTs increased (reduced) incremental costs.

Higher hay fever sensitization prevalence also favoured standard fully manual SPTs because higher hay fever sensitization prevalence levels increased incremental costs of conducting computer vison-based SPTs. A potential explanation behind this observation would be the lower accuracy (sensitivity and specificity values) associated with the computer vison-based SPT techniques [7]. In fact, higher allergen prevalence rates would result in a higher proportion of diagnostic inaccuracies for computer vison-based SPTs and potentially lead to worse cost consequences.

Probability sensitivity analysis (PSA)

As illustrated in Fig. 3, our Monte Carlo simulation which accounted for general uncertainty around our model input parameters also indicated that SPTs conducted using the computer vison-based SPT system were cost-saving compared to standard fully manual SPTs in the majority of the simulations, i.e., incremental cost of conducting manual SPTs in comparison to computer vison-based SPTs was close to the base case result of CHF 7 for most of the simulations. The simulations were normally distributed.

Fig. 3

Monte Carlo simulation distribution of incremental cost

Discussion

Findings

This study showed that performing SPTs using a computer vision-based system (Nexkin DSPT) in the allergology department of a Swiss university hospital results in average savings of CHF 7 per test compared to fully manual standard SPT techniques. Deterministic sensitivity analyses confirmed our model’s robustness because computer vision-based SPTs remained cost-saving regardless of plausible changes in input parameters. The robustness of the findings was further investigated using probabilistic sensitivity analysis (PSA) which demonstrated that computer vison-based SPTs were cost saving with a high degree of certainty. However, the computer vison-based system became less cost saving as; (i). the costs associated with the standard fully manual SPT declined and (ii) hay fever sensitization prevalence rates increased. Higher allergen prevalence rates would therefore favour the more accurate manual SPTs, a result in line with what some studies have shown regarding the importance of accuracy in minimising costs [31, 32].

We developed an early cost-consequence model and as such, our results provide an initial understanding of some of the costs that could be saved by using (semi) automated devices in diagnostic procedures. Our observations are particularly relevant in Switzerland where healthcare costs have been incessantly rising and where cost containment remains a prime concern for all major stakeholders of the Swiss healthcare system, including hospitals [11].

Comparison to existing literature

A recent study showed that the Nexkin DSPT device provided more consistent readings in comparison to standard manual SPTs [10]. However, this study focused on wheal size reading consistency and did not address its cost saving potential. Another study pointed out the need for the development of an automated system of conducting prick tests and duly spelled out the optimality and effectiveness of automated SPTs in terms of time and materials [9]. Again, this study did not quantify the cost saving potential of automated SPTs vs. standard manual SPT procedures. To the best of our knowledge no other studies have evaluated the potential cost savings associated with the use of (semi) automated devices to perform SPTs at a hospital. Our study therefore takes an important step forward in the advancement of the adoption of (semi) automated diagnostic techniques which if adopted at scale, could play a key role in healthcare cost containment efforts.

Limitations

Data limitations and the novel nature of the computer vison-based SPT technique precluded us from conducting a more elaborate and comprehensive cost effectiveness analysis (CEA) with a broader perspective that would have not only considered cost implications but also other important health outcomes [13, 16]. In this context, a comprehensive CEA would have been premature for this study thus making a cost-consequence analysis (CCA) ideal for the purposes of our study.

CCAs have the advantage of allowing cost and outcome disaggregation which affords readers and decision makers the possibility of forming their own opinion on the relevance and relative importance to their decision-making context [13, 14, 33] Furthermore, CCAs are a straightforward form of evaluation for new health technologies and interventions with the potential of having important consequences whose relevance may differ depending on perspective (healthcare, patient, etc.) [33, 34]. For these reasons, we developed an early cost-consequence model and evaluated the cost consequences solely from a hospital’s perspective.

Despite these advantages, CCAs have been criticised for their limited generalizability due to their context specific nature and subjective importance of cost components which vary depending on decision makers primary concerns [14]. Concerns have also been raised regarding transparency of costs and outcomes used in CCAs which may lead to cherry picking positive results [15]. The use of CCA may therefore present a limitation to the current study. Moreover, we did not develop an ideal CCA which similar to comprehensive CEAs ideally consider all key health/cost related outcomes in addition to analysing results from multiple perspectives [14, 33].

For instance, our early cost-consequence model was analysed from a hospital’s perspective which implies that we did not consider important health/cost outcomes that would have helped provide a more informed decision from a broader societal perspective. Notable omissions include eventual GP related costs and patient related quality of life outcomes for patients requiring further consultations after a false negative sensitization outcome in our modelled patient pathway. The limited (hospital only) evaluation perspective and early nature of our model also precluded us from considering, among other potential patient related costs, prescription costs for medication and/or complementary tests prescribed to patients with both true and false positive sensitization outcomes in our modelled patient pathway. Our failure to consider such costs and outcomes ignores the broader societal implications of computer vision-based SPTs and as a result, limits the generalizability of our results.

Additionally, it would be wrong to assume that the lower diagnostic accuracy values associated with the computer vison-based SPT device would have been without effect on health outcomes, had our analysis considered health outcomes. Studies exploring the impact of diagnostic accuracy on health outcomes and healthcare costs have shown that inferior or delayed diagnostic accuracy increased health risks, worsened patient outcomes and increased healthcare costs due to unnecessary interventions [31, 32]. Nexkin DSPT’s lower diagnostic accuracy would have most probably had a negative impact or at least lowered the cost savings demonstrated in this study.

It is therefore important to note that while we achieve our objective, our early cost-consequence model does not capture the entire spectrum of cost and health outcome consequences that could have been captured had we considered patient related (health) outcomes and other broader consequences. Arguably low-quality data constantly used in our analysis also adversely affects the robustness of the model. First, hay fever prevalence values in Switzerland we used are based on a study that dates back to 1995 [19]. Input values sourced from the study might therefore be considered as outdated, despite the head allergology clinician at the Inselspital hospital confirming similar observations in their allergology department’s day to day. Second, the use of non-empirical parameter estimates and assumptions based on clinical expertise potentially weakens our conclusions because it could result in over and/or underestimations. Our context specific assumptions could also limit the transferability of our findings to other healthcare institutions and health systems where the suspected allergy patients’ care pathway differs from that of patients referred to a Swiss university hospital.

Recommendations (future studies)

Considering the limitations discussed and the early nature of our analysis, it would be suitable to collect more robust model input data, particularly data on health outcomes in order to further the evidence our study generated. The collection of data would indeed facilitate a broader evaluation perspective integrating the view of all key societal player (patients, hospital, and other payer institutions) thus providing a better understanding of the entire spectrum of the consequences of computer vision-based systems in conducting SPTs. To this end, we are currently conducting a clinical trial at Inselspital whose data collection objectives include furthering the early CCA model developed in this study².

Future evaluation models should also consider extending the analysis to multiple healthcare settings.

Conclusion

Our study is the first of its kind to evaluate the potential cost savings associated with the use of (semi) automated devices to perform SPTs at a hospital. Despite our study’s various limitations that restricted our analysis to an early cost-consequence model, we achieve our objective and conclude that the use of computer vison-based SPTs has the potential of saving CHF 7 per test on average in comparison to standard fully manual SPTs. Against the backdrop of rising healthcare costs, especially in Switzerland, using similar computer vison-based diagnostic systems could contribute in cost containment efforts. By developing an early cost-consequence model that evaluates the consequences of adopting (semi) automated technologies in the care pathway, we also generate early evidence in healthcare economics and healthcare policy that future research can build on. Ultimately, our study contributes to literature exploring cost implications of digital technology adoption in healthcare institutions. However, results should be taken with caution due to the early nature of our model and the Swiss hospital specific setting in which our analysis was conducted.