Improving the evidence base of Markov models used to estimate the costs of scaling up antiretroviral programmes in resource-limited settings

Background Despite concerns about affordability and sustainability, many models of the lifetime costs of antiretroviral therapy (ART) used in resource limited settings are based on data from small research cohorts, together with pragmatic assumptions about life-expectancy. This paper revisits these modelling assumptions in order to provide input to future attempts to model the lifetime costs and the costs of scaling up ART. Methods We analysed the determinants of costs and outcomes in patients receiving ART in line with standard World Health Organization (WHO) guidelines for resource poor settings in a private sector managed ART programme in South Africa. The cohort included over 5,000 patients with up to 4 years (median 19 months) on ART. Generalized linear and Cox proportional hazards regression models were used to establish cost and outcome determinants respectively. Results The key variables associated with changes in mean monthly costs were: being on the second line regimen; receiving ART from 4 months prior to 4 months post treatment initiation; having a recent or current CD4 count <50 cells/µL or 50-199 cells/µl; having mean ART adherence <75% as determined by monthly pharmacy refill data; and having a current or recent viral load >100,000 copies/mL. In terms of the likelihood of dying, the key variables were: baseline CD4 count<50 cells/µl (particularly during the first 4 months on treatment); current CD4 count <50 cells/µl and 50-199 cells/µl (particularly during later periods on treatment); and being on the second line regimen. Being poorly adherent and having an unsuppressed viral load was also associated with a higher likelihood of dying. Conclusions While there are many unknowns associated with modelling the resources needed to scale-up ART, our analysis has suggested a number of key variables which can be used to improve the state of the art of modelling ART. While the magnitude of the effects associated with these variables would be likely to differ in other settings, the variables influencing costs and survival are likely to be generalizable. This is of direct relevance to those concerned about assessing the long-term costs and sustainability of expanded access to ART.


Introduction
With access to antiretroviral therapy (ART) now rapidly expanding in low and middle-income countries, attention is increasingly turning to the affordability and sustainability of these programmes [1]. Given the potential effectiveness of treatment coupled with the scale of the response needed, it is important that planning takes a long term perspective. While many studies have focussed on the effectiveness of ART in resource-limited settings, cost studies are limited, especially those documenting costs in routine and established programmes and over longer periods of time. In recent years, the management of ART programmes in low and middle income countries has increasingly conformed to the World Health Organization (WHO) guidelines for resource-limited settings [2]. These include guidelines for when to start ART based on the patient's CD4 count or WHO stage, guidelines for monitoring ART as well as guidelines regarding which antiretrovirals (ARVs) should be administered within distinct first and second line regimens. These guidelines therefore provide a good framework for understanding disease progression and the costs of patients in ART programmes.
Because ART has only recently been available in resource-limited settings, lifetime costsa key input into the costs of scaling up -are calculated through extrapolating primary data, with the Markov model being the most common framework used for this extrapolation. Many models include the baseline and current CD4+ cell count (i.e. the most recent test value), viral load and WHO staging, but other potential determinants of costs such as adherence have been excluded. This raises questions of the accuracy of the resulting estimates which could have implications for attempts to plan for expanded access to ART.
A Markov model consists of a number of mutually exclusive and collectively exhaustive Markov states, with at least one of these being an "absorbing state" (e.g. death). Patients remain in each state for an equal increment of time, called a Markov cycle, before being allowed the option of moving to a different state (or staying in the current state) as determined by one or more transition probabilities. In addition to time (or survival) increments, health care costs are attached to each state. Over a large number of cycles, lifetime costs and life expectancy is estimated [3,4].
To establish appropriate Markov states it is thus necessary to estimate which variables have a sizeable impact on the costs associated with being in a state together with the transition probabilities determining movements between states. While many types of transition probabilities are possible, the most important is the probability of dying as this determines overall life expectancy. Because the majority of the costs of ART are associated with ARV drugs, accurate calculation of life expectancy is crucial for the estimation of lifetime costs which in turn is a key input into calculations of the costs of scaling up [5].
This paper seeks to identify the variables that have an impact on direct health care costs and the likelihood of dying with a view to informing the development of Markov models for estimating lifetime costs and the costs of scaling up ART in resource-limited settings. We initially review the ART Markov model and cost determinant literature to establish the variables and variable categories that have been used to date. Thereafter, we assess the importance of these through the analysis of a large cohort from a private health care disease management programme. While this analysis would ideally be conducted using data from individuals receiving ART in a range of routine models of care, including those found in the public health care sector, these routine data are not available. We have attempted to improve the generalisability of our findings by restricting our analysis to those patients in the private disease management programme that receive ART in line with the WHO guidelines for resource-limited settings [2].

Literature review
Our literature review included all cost, cost-effectiveness and cost-utility analyses of HIV-treatment including ART in resource-poor settings. While most economic analyses of ART focus on the annual per patient cost or the cost per specified outcome measure (e.g. per patient virally suppressed), we restricted our review to studies that had used Markov modelling to extrapolate available data to calculate lifetime costs and life expectancy. The reason for this is that the life expectancy of a patient on ART is a key determinant of lifetime costs and of the number of patients surviving and remaining in care over any projection period; it is therefore one of the most important inputs into any estimation of the costs of scaling up. A previous study by our group showed that 20% more patients would be remaining in care after a scale-up period of 10 years if life expectancy on ART of 13 years were assumed instead of 8.5 years [6]. However, we also included studies that attempted to ascertain the variables influencing costs over shorter time frames given that these could provide input into the construction of Markov states. A Pubmed search using the keywords "cost", "resource-poor", "low-income country/ countries", "middle-income country/countries", "developing" and "antiretroviral" was conducted for all papers published before 1 May 2009.

Model comparison and refinement Data source
The determinants of costs and survival on ART were evaluated using a large database of patients enrolled with Aid for AIDS (AfA), a group that manages HIVrelated care for a number of medical insurance funds in the private health care sector in Southern Africa. Aid for AIDS does not manage patients directly, but rather provides guidelines for private medical practitioners' care of its participants and reimburses claims. Medical care is provided via patients' own general practitioner, therefore there are no formal "sites" but rather several thousand general practitioners and specialist practices taking care of patients, including those participating in Aid for AIDS. Treatment is funded by contracted companies or medical aid funds (composed of pooled monthly contributions from members) which substantially cover co-morbid conditions including those not related to HIV. Data collected by AfA include demographics and previous medical history, CD4+ cell count, viral load and claims (including ART dispensing data). These claims are captured monthly by AfA from the medical insurance funds, pathology laboratories or from the patients or treating doctors directly using routine electronic administrative systems. Claim reimbursement is subject to established AfA protocols, including protocols for ART initiation, change of ART regimen, and the treatment of certain opportunistic infections. Despite this being a private sector programme, antiretroviral guidelines are similar in many respects to WHO guidelines for resource-poor settings as well as the South African public sector guidelines [2,7]. The recommended initial regimen is a combination of two nucleoside reverse transcriptase inhibitors (NRTIs) and a non-nucleoside reverse transcriptase inhibitor (NNRTI). Second line therapy consists of a boosted protease inhibitor (PI) with two NRTIs. Health services provided for patients within AfA include additional primary care doctor visits for patients that have exceeded their routine medical insurance benefits, telephonic counselling services and antiretroviral drugs dispensed monthly from private pharmacies or delivered via courier to the patient's home. While ART can be initiated at CD4 <350 cells/µL rather than CD4 <200 cells/µL, we restricted the analysis to the latter group as this is the more common starting criterion in resource poor settings, including the South African public sector at the time of this study (these guidelines have recently been revised to recommend ART initiation at CD4 <350 cells/µL [8]). Furthermore, we only included patients starting ART with a NNRTI plus two NRTIs, as recommended by the WHO for resource-limited settings [2].
We determined average adherence to ART using monthly pharmacy refill data. This approach has been shown to correlate well with adherence assessment by therapeutic drug levels [9,10], and has been found to reliably predict virologic suppression [11], development of HIV drug resistance [12], and survival [13]. Previous analyses of these AfA data have in addition shown that this measure of adherence is a determinant of costs [14,15]. We expressed pharmacy claim adherence as a percentage and calculated it as the number of months with ART claims submitted divided by the number of complete months from ART initiation to death, withdrawal from the Aid for AIDS program, or study end.
Two of the medical insurance funds that contract AfA were selected on the grounds that they had large numbersof patients, similar treatment benefits, and no co-payment for ART. This allowed us to describe determinants of costs and outcomes without biases associated with the patient's ability to pay, which has been reported to influence access to health care and outcomes on ART [16,17]. Patients were included in the study if they were ART naïve at entry (the exception being women who had received prophylaxis for prevention of mother-tochild transmission), adult (19 years or older at the time of approval for ART) and if ART was started between November 1998 and November 2007.
Direct health care costs were analysed from the provider's perspective. The tariff amount was used as a proxy for these costs, as opposed to the amount charged by the provider. This is because providers may charge different rates for services with the same tariff code. The use of the tariff rate allows for the same cost to be assumed for the same type of service.
The prices of antiretroviral drugs have fallen dramatically over the past ten years. To account for this we deflated ARV prices to the April 2007 level. All other health care costs have increased; these were inflated to the April 2007 level using the Consumer Price Index net of mortgage payments (CPIX) [18]. The average South African Rand to United States Dollar (US$) exchange rate in April 2007 (R7.14 to US$1) was used to convert costs to US$ equivalents [19].

Establishment of Markov states
As the distribution of mean health care costs was rightskewed in our data, ordinary least squares regression was not appropriate [20,21]. Generalised linear regression models (GLM) have been proposed as they determine the impact of variables on the arithmetic mean and thus provide a method for identifying variables strongly associated with varied costs [20]. A GLM with a log-link function and a gamma distribution described the trends in the data well. To account for multiple measures within an individual as well as potentially strong correlations between the variables, we used generalised estimated equations with an unstructured correlation matrix. Variable coefficients and their 95% confidence intervals were determined using robust standard errors and the model fit was evaluated using deviance residuals [20].
Multiple Cox proportional hazard regression analysis was used to identify variables associated with a likelihood of dying. Based on the findings of both the cost and outcome analyses, a pragmatic decision on which Markov states to include in the final model is needed; as the number of states increases, so the model complexity increases exponentially. Data storage, basic calculations and data extraction was handled in Microsoft Sequel Server 2008. Statistical analysis was performed in Stata 10.

Ethics statement
The study was approved by the Research Ethics Committee, University of Cape Town and by the Board of Directors of Aid for Aids. All patients signed consent for their information to be entered into the AfA database.

Existing models in literature
Over 300 cost, cost-effectiveness and cost-utility analyses of ART were found via a Pubmed search, but these included only 6 different Markov models of ART in low and middle income countries, some of which were used in more than one publication [6,[22][23][24][25][26][27]. A number of variables were used in these studies to define Markov states, as outlined in Table 1. These included the baseline (i.e. pre-ART) CD4 count category, the current CD4 count category, baseline and current viral load categories, time on ART, being on a first or second line ARV regimen, opportunistic infections or WHO staging; and adverse events on ART. These variables could be combined in a variety of ways to create distinct Markov states depending on the model.

Model comparison and refinement Dataset
The characteristics of the cohort are described in Table 2. After exclusions, 5,177 patients met our eligibility criteria, with over 136,600 patient months of observation, about half of which were on ART. Median follow-up on ART was 19 months (IQR: 10 to 32). The proportion of patients who left the medical insurance fund was 34%. These patients either changed their employment, switched to a different medical insurance scheme or voluntarily stopped their contributions to the insurance scheme. The most common first line antiretroviral regimen was zidovudine/lamivudine/efavirenz (65 %). Lopinavir/ritonavir/zidovudine/didanosine was the most common second line regimen. CD4 and viral load monitoring was done 1.5 times per annum on average.

Markov states
To determine the most important variables on which to base Markov states, we assessed whether variables had a sizeable effect on costs or on the likelihood of dying. The literature review identified a number of differences in the ways that variables were categorised. Using the categories described in the literature as a starting point, we determined the most appropriate categories for the variables in our dataset guided by residual diagnostics, whether overall model fit improved with a changed categorisation, and whether the p-value was significant at the 95% confidence interval. The variables included in the analysis were: (1) baseline CD4+ cell count (categorised as 0-49, 50-199 cells/µL) (following Cleary et al [6,22] and Goldie et al [25]); (2) current or most recent (carried forward for up to 12 months) CD4+ cell count (0-49, 50-199, 200 to 349, 350 to 499, and ≥500 cells/µl) (similar categories as Bachman [24], Badri et al [23] and Goldie et al [25]); (3) baseline viral load (categorised below or above 100,000 copies/ml) (Goldie et al [25] include this variable, but use far more categories); (4) Current or most recent viral load (categorised as <400, 400-10,000, 10,000-100,000, >100,000 copies/ml) (Goldie et al [25] and Vijayaraghavan et al [26] include this variable, but use different categories); (5) ART regimen (either first line or second line) (used by all except Bachman [24]); and (6) time periods relative to ART initiation (-4 to 4, 4 to 12, 12 to 24, 24 to 48 months on treatment) (similar to Cleary et al [6,22]). In addition to the variables identified in the literature, we also considered: (7) overall adherence, as determined using monthly pharmacy claim data and divided into quartiles based on the observed distribution of adherence in the cohort (<42%, 42%-75%, 75%-92%, >92%); (8) the NNRTI included in the initial first line regimen (either efavirenz or nevirapine); (9) the duration of CD4 count monitoring prior to starting ART, a proxy for duration within pre-ART care (≤6 months and > 6 months); (10) age at starting ART (<25, 25 to 50 and >50 years old); and (11) sex. Our data did not allow us to include variables relating to WHO disease staging, opportunistic infections or adverse events. All the variables were included in the analysis exploring the determinants of costs and the likelihood of dying. In the baseline, the following initial parameter states were assumed: (1) baseline CD4 cell count between 50 and 199cells/µL; (2) current CD4+ cell count between 50 and 199cells/µL; (3) baseline viral load < 100,000 copies/ml; (4) current viral load < 400 copies/ml; (5) on first line ART; (6) time period 12 to 23 months on ART; (7) overall adherence in the upper quartile (>75%); (8) NNRTI = efavirenz; (9) >6 months of CD4 monitoring prior to starting ART; (10) age at starting ART 25 to 50; and (11) sex = female The results from the multiple regression analysis of costs (all variables were included in the model) are found in Table 3. Mean cost per patient month is presented for each variable or variable category and is compared to a reference or baseline which is the mean cost across all patients and time periods. We found higher current CD4 counts were associated with lower costs while having a current viral load above 100,000 copies/ml was associated with increased costs. Being on efavirenz was more costly than nevirapine, and being on second line was more costly than being on first line; these findings are likely to relate to the higher costs of the ARV drugs in these states as opposed to other health care costs. Lower adherence was an important driver of costs, as was time on treatment, with monthly costs in the period from 4 months before starting ART to 4 months on ART being almost double the monthly costs thereafter. After this time, the size and significance of the association between costs and time on treatment waned dramatically. Figure 1 illustrates the full set of variables and their influence on the mean total monthly direct health care costs relative to the baseline scenario.
The results from the multiple Cox proportional hazards regression analysis of the likelihood of dying are found in table 4. The relative likelihood of dying is compared with the same referent groups as in the cost analysis, but a separate model was used for each of the time periods. We found higher current CD4 counts, higher ART adherence, and current viral loads below 100,000 copies/ml were associated with lower likelihoods of dying across all periods, with baseline values for CD4 and viral load contributing very little additional effect after the first 4 months on treatment. Longer duration of monitoring prior to starting ART was associated with a lower likelihood of dying in earlier periods and being greater than 50 years or younger than 25 years at starting ART was associated with a higher likelihood of dying in later periods. Being on second line was associated with an increased likelihood of dying across all periods.

Discussion
We analysed determinants of direct health care costs and survival in 5,197 HIV-infected adults enrolled in a South African managed care ART programme with 136,672 patient months of follow-up, spanning -4 months before ART to 4 years on ART. If each Markov state is defined to include unique combinations of these variables, thousands of states could be defined. A correspondingly large dataset would then be needed to calculate the costs and transitions associated with each of these states. For this reason, it becomes necessary to focus only on variables that have the most marked effects on costs and outcomes within reasonable confidence intervals. The key variables associated with changes in mean monthly costs were: being on the second line regimen; receiving ART from 4 months prior to 4 months post treatment initiation; having a recent or current CD4 count <50 cells/µL or 50-199 cells/µl; having mean ART adherence <75% as determined by monthly pharmacy refill data; and having a current or recent viral load >100,000 copies/mL. In terms of the likelihood of dying, the key variables associated with changes in survival were: baseline CD4 count<50 cells/µl (particularly during the first 4 months on treatment); current CD4 count <50 cells/µl and 50-199 cells/µl (particularly during later periods on treatment); and being on the second line regimen. Being poorly adherent and having an unsuppressed viral load was also associated with a higher likelihood of dying. The relationships between these variables and costs and outcomes were consistent with trends described in the literature, though the scale did differ in some cases: lower CD4 count, higher viral load, lower adherence, and being on second line therapy was associated with higher costs and worse outcomes. In addition, sub-optimal adherence drives resistance to first line therapy, leading to second line therapy being initiated. There was a relative small and limited association between costs and outcomes and the baseline pathology results (CD4 count and the viral load), most likely due to the current or most recent CD4 and viral load results being dominant. The finding that lower adherence is associated with higher costs (in addition to its known effects on biological variables such as CD4 count and viral load and starting second line therapy) further supports the need to include this variable in Markov models.
Based on the above, one would anticipate that Markov models that use these variables as the bases of their Markov states may have superior accuracy. However, in our literature review, the only Markov model that specified costs and outcomes in relation to duration on ART was Cleary et al [6,22]. Most of the models separated first from second line ART to capture the cost differences, but were unable to estimate different survival transition probabilities because data on outcomes of patients on second line was limited. Bachmann [24], Goldie et al [25] and Badri et al [23] all included the current CD4 count but Cleary et al [6,22] only included the baseline CD4 count. Baseline and current viral load was only included by Goldie et al [25]. Opportunistic infections or WHO staging were included in all the models except Cleary et al [6,22]. Unfortunately these data were not available within the AfA cohort, and we were therefore unable to assess the importance of these for the construction of Markov states. None of the models included sex, ART adherence, age or duration of pre-ART care.
There are a number of limitations to this analysis. First, our cohort consisted of private sector patients when the majority of patients in resource-limited settings are treated in the public or NGO sector. However, the baseline characteristics of our cohort (CD4+ cell count, proportion female and age) are comparable with cohorts from low-income countries [28] and we restricted our analysis to patients receiving NNRTIbased first line ART regimens and starting ART at CD4<200 cells/µl, in keeping with WHO recommendations for resource-limited settings [2]. While we would not claim that our actual cost findings are generalisable to public sector settings or to other countries, we would argue that the variables that influence costs and outcomes are likely to be relevant even if the magnitude of their effects could differ. Second, the impact of specific opportunistic infections, disease staging or adverse Figure 1 The influence of variables on mean monthly costs events on costs and outcomes was not included in this analysis as these data were not available. We are therefore unable to comment on the validity of this aspect of some of the Markov models in the literature.

Conclusion
In conclusion, we have analysed the determinants of direct health care costs and outcomes in a private health care sector managed ART programme. Our focus has been to use statistical techniques to determine the key variables to include in Markov states and to use these findings to inform future modelling of the costs of scaling up ART. Our results suggest that important drivers of costs and outcomes include time on ART, being on first versus second line regimens, the current CD4 cell count, the current viral load, age at starting ART and adherence. The inclusion of these variables should be considered for future modelling of the costs of scaling up ART programmes.  Authors' contributions RL analyzed the data. MH collected and prepared the data used in the study and provided input into the data analysis methodology. SC, GM and RL contributed to the overall study design and methodology. SC wrote the first draft of the paper while SC and RL were responsible for revisions of the paper. All authors contributed to the writing of the paper and agreed with the manuscript's results and conclusions.