Many international HTA agencies require health economic evaluations [12, 17, 19, 21–23, 25, 26, 29, 30],. In the majority of agencies, economic evaluations are carried out to support reimbursement decisions [17, 19, 21, 25, 26, 29, 30, 35]. The results of the economic evaluation are either used to decide whether the HT is generally covered by the funder or for price negotiations with the industry (or both). They are more often used for general coverage decisions (e.g., is an HT covered?) in state-funded health care systems because their budgets can be regarded as mainly fixed. Some HTA agencies evaluate exclusively pharmaceuticals. The focus on pharmaceuticals is probably because drugs are the HT representing the highest proportion of total health care costs and the methods for the economic evaluation of drugs (pharmacoeconomics) are well established.
Most agencies in state-funded health care systems recommend or require exclusively cost-utility analyses in the base-case scenario [19, 21, 25, 26, 30]. This is probably because a cost-utility analysis enables comparison between different indications and types of HT; especially in state-funded health care systems, reimbursement decisions are commonly made encompassing all indications. Furthermore, CUA enables a higher level of standardization because the same denominator is used for all types of HT and the methods to determine it can be better standardized. In contrast, social insurance systems mostly prefer cost-utility analyses or cost-effectiveness analyses [12, 22, 23]. A cost-effectiveness analysis can be applied because the HT is commonly only compared with one indication. Thus, indication-specific outcomes, especially patient reported outcomes (e.g., specific quality-of-life measurements) or proofed surrogates (e.g., blood pressure), can be used for comparison. However, the consequences of full standardization of both methods for outcomes in cost-effectiveness analysis and the definition of additional benefit or a fixed cost-effectiveness threshold are difficult to predict because there are too many outcomes that can potentially be used. The decision regarding a HT is therefore complicated and transparent. The methods can be further harmonized because, apart from the methodological difficulties of cost-utility analysis (e.g., measuring quality of life, multiplication of quality of life and live years), there is no reason to prefer cost-effectiveness analysis. This is especially true considering that HTA agencies give no concrete descriptions of cases where a cost-effectiveness-analysis is accepted or recommended. Furthermore, methodological problems (e.g., measurement) can also be relevant in cost-effectiveness analysis. A cost-benefit analysis is not accepted in the base case scenario by almost all HTA agencies. This indicates that valuing health outcomes in monetary units (e.g., through the willingness-to-pay or willingness-to-accept approach) is not considered to be best option for economic evaluation of HTs. The main reasons that cost-benefit analysis is not recommended are likely ethical concerns about the low acceptance of valuing health in monetary units in many societies and, consequently, low acceptance of decisions. Furthermore, there are methodological difficulties in valuing health in a monetary fashion (e.g., a willingness-to-pay stated preference depends on income) .
Almost all HTA agencies recommend one main comparator. This comparator is mostly usual care (e.g., standard care) or existing practice. The different formulations were probably chosen very consciously by the HTA agencies although each basically means the same thing. Nevertheless, in some cases, slightly different formulations can result in different comparators for the same HT or imply different degrees of flexibility for the choice. For example, usual care can mean “all HTs that are usually applied in practice”. Standard care is an accepted treatment (e.g., described in clinical guidelines) but not necessarily the most used. Only the IQWIG requires comparisons with all these alternatives; this is necessary to estimate a cost-effectiveness frontier for multiple interventions .
Many HTA agencies recommend choosing a time horizon depending on duration of disease, but this is often only useful for a societal perspective. Most of the other HTA agencies recommend choosing a time horizon depending on relevant costs and benefits; this is vague, but at least implies recognition of the importance of perspective of analysis. In addition to a lifetime period for chronic conditions, there are no other exact specifications of time horizons for analysis. For example, there is no recommendation for vaccinations that could have longer (economic) effect than a disease, or even a lifetime period because of herd immunity. Thus, the scientific literature suggests that the time horizon of an economic evaluation should be long enough to capture all observable health outcomes and effects caused by a technology  independent of disease duration. The majority of HTA agencies in state-funded health systems recommend using the perspective of the funder or all national social services [19, 20, 25, 26, 30]. For outcomes, this implies mostly a societal perspective because health care systems in these countries embrace the entire society. Additionally, regarding cost, this perspective is relatively broad: social services mostly have substantial coverage. In social insurance systems, the societal, social insured or the funder perspective is recommended. Principally, the societal perspective is adopted. The state-funded health care systems are funded by taxes paid by the entire population. Consequently, the proper perspective is all who pay, i.e., the whole society. In social insurance systems, a societal perspective is desirable because the main proportion of the societies is covered and the systems are publically funded. Furthermore, to subtract out the corresponding proportion of persons not covered is practically impossible, especially for health effects concerning a population (e.g., infectious diseases). None of the HTA agencies suggest that the perspective of analysis can depend on the indication so that costs or outcomes do not differ between different perspectives (e.g., acute conditions). If all HTA agencies used the societal perspective, the adaptation of economic evaluations to different countries would become easier: all arising costs of the disease would be considered independently of who bears the costs.
Recommended costs that should be included vary depending on the chosen perspective. However, substantial unjustifiable differences exist. Some agencies include only direct medical costs, while others also include non-medical direct costs, such as patient and family time. Often, payments that must be made by the patient are explicitly mentioned. In the societal perspective, either all indirect costs or only productivity losses are recommended for inclusion. All HTA agencies should consider all relevant changes in cost caused by HT, irrespective of cost type. Focusing on certain cost types may lead to improper decisions because HTs can have different effects on different cost types. Furthermore, the restriction to a certain cost type can result in missing costs that would otherwise have had a strong impact on the evaluation, especially in a societal perspective. All differences caused by the compared technologies are relevant. This would also facilitate the transferability of results.
Measuring resource use in natural units seems to be generally accepted. This simplifies the transferability of results because it can be presumed that differences in cost of HTs are primarily attributable to differences in prices, not resource use . The measurement in natural units allows adaptation: only the used resources have to be newly priced in country specific unit prices. A completely new measurement of costs is not necessary. None of the HTA agencies included intangible costs in the denominator, reflecting that this cost type is seldom used because of methodological difficulties (the same cost influences the numerator as well as the denominator in the analysis). Especially in cost-utility analysis, such cost effects are implicitly integrated into the denominator. Unfortunately, a more detailed comparison of included costs is difficult because recommendations for the measurement and sources of costs vary in detail and addressed issues. One reason is perhaps that sources of resource use and costs depend on national circumstances, such as the availability of databases containing national data. However, there seems to be a tendency to identify resource use systematically, e.g., by a systematic literature search. Regarding the valuation of costs, most agencies recommend the use of opportunity costs. Market prices, as well as list prices, and administrative data are suggested for approximation. Some recommendations within one HTA agency are in conflict with the demand to measure resource use in natural units respectively a micro-costing approach. Recommendations are heterogeneous among the HTA agencies. However, the mechanism of price formation depends upon both reimbursement systems and HT. Thus, the variations seem mostly justifiable. A harmonization is not suggested because prices depend strongly on the national economy also, if market prices are exclusively used.
Because there are many possible outcomes that can be measured in natural units, it is logical that the measurement and the valuation of outcomes for health-related quality of life used to estimate QALYs are described. With few exceptions, generic direct measures taken from affected patients are recommended in estimating QALYs. To measure health-related quality of life, the SF-6D, EQ-5 and -15D are mostly recommended [17, 19, 25, 27, 30, 35]. Using the same quality of life measurements, these measurements can be transferred between HTA agencies. However, even in this case, the transferability of results can be limited due to cultural differences . Further research is therefore needed to investigate the differences of quality-of-life estimates between countries in order to assess whether QALYs can be transferred. Only willingness-to-pay is recommended for monetary valuation of outcomes. It can be presumed that for willingness-to-pay estimates, the differences between countries are even higher than for health-related quality of life . Recommendations for sources of outcomes are conflicting. Some agencies prefer observational studies while others prefer experimental studies (above all, randomized controlled trials). This indicates the still ongoing, controversial discussion about whether the benefit of an intervention should be based on effectiveness or efficacy data and whether either can only be measured by one of the study types. On the one hand, it should be noted that because HTA evaluates mostly new products, observational data (e.g., registry data) are often not available. On the other hand, performing randomized controlled trials is expensive and time consuming and can delay the diffusion of a new HT. Where both types of data are found, the alleged gap between efficacy and effectiveness can be analyzed via sensitivity analysis to yield insight into the particular problem. This could be why some agencies do not restrict data in this manner and allow or require the use of both types of data [7, 19, 22, 23, 26, 27]. Regardless of the study/data type recommended as a basis for analysis, performing a systematic review to obtain data is recommended by many agencies. Here, there is great potential for cooperation of HTA agencies to avoid duplicate work because a systematic literature search and selection of relevant effectiveness data is basically the same when the same HT is evaluated.
The divergences in recommended annual discount rates between HTA agencies are remarkable, which reflects the ongoing discussion about this issue [43, 44]. The discussion primarily concerns differences between discount rates and benefit rates that can result in undesirable decisions. More agencies recommend using the same discount rate for costs as for benefits; fewer agencies recommend higher rates for costs than for benefits. Additionally, the recommended discount level varies for both costs and benefits. Although the discount rate is simple to adapt, it should be further harmonized because, depending on the time horizon of the analysis, it can have a strong influence on results and lead to unjustifiable differences in decisions between countries.
Recommendations on modeling are either not made by the agencies or are extremely vague. It can be presumed that this is because there are many different types of models and different problems that require different approaches. Perhaps, therefore, an a priori model specification is consciously avoided by the HTA agencies to prevent the use of suboptimal models for certain decisions; the detailed description for any potential model would be too extensive and always affected by uncertainty. The uncertainty in model choice and structure is also reflected in the fact that nine HTA agencies recommend testing different structural assumptions [7, 19, 22, 25–27, 30, 32, 35]. Specific recommendations are only made for decision trees and Markov-models, suggesting that these are the most established and accepted approaches for decision modeling . Recommendations regarding the type of sensitivity analysis are partly contradictory. Some HTA agencies require exclusively either probabilistic or deterministic sensitivity analyses. There are also agencies that do not specify the type of sensitivity analysis, or allow both. It remains mostly unclear why some agencies prefer or accept only one of the two types of sensitivity analyses. The approaches to sensitivity analyses are based on different mathematical assumptions that can provide different insight on a decision. A restriction on a certain type of sensitivity analysis can be unrewarding. Thus, in the scientific literature no recommendation can be found in favor of using only one of the approaches  or using both  to test parameter uncertainty. An a priori specification of parameters that should be tested in a sensitivity analysis is not conducive because potentially all included results can influence sensitivity analysis . Consequently, most agencies require only varying values for “relevant” parameters to prove the robustness of results. It also means that standardization is nearly impossible. For ranges, mostly confidence intervals are used. Surprisingly, incorporated differences that are presented in scientific literature are seldom requested. The recommended ranges for variation of discount rates are heterogeneous and lie often inside a wide range. This once again reflects a lack of consensus on this issue. Further recommendations on parameter variation are rare.
Equity aspects are given altogether little attention, which makes clear the often unbalanced view of HTA agencies between clinical effectiveness, cost-effectiveness and other aspects (social, ethical and legal). More efforts are needed to incorporate equity aspects into economic evaluations that reflect social values, especially given the background that reimbursement decisions depend on, and given the results of the economic evaluations, which can have different consequences depending on the patient group.
Similar to guidelines in scientific publications , the HTA agencies recommend mostly that results should be presented in disaggregated form and as incremental ratios. The disaggregated results support transparent reporting and incremental ratios simplify interpretation and conclusion synthesis. Both are important for transparent and trustworthy reimbursement decisions.
This study has some limitations. First, there is a language bias because we included only English and German manuals. Second, the response rate of the e-mail inquiry was only 28% and we contacted the HTA agencies only once for information, with no further attempts if a reply was not made. However it can be assumed that the investigation is fairly representative because there is no obvious reason that the HTA agencies make their recommendations not accessible on their webpages and the webpages of all HTA agencies were searched by two reviewers independently. Third, the comparison of methods might be partially inconsistent because of the different terminology used in the health economic literature. Fourth, only the members of the HTA umbrella organizations were considered.