- Open Access
- Open Peer Review
Specific guidelines for assessing and improving the methodological quality of economic evaluations of newborn screening
BMC Health Services Researchvolume 12, Article number: 300 (2012)
Economic evaluation of newborn screening poses specific methodological challenges. Amongst others, these challenges refer to the use of quality adjusted life years (QALYs) in newborns, and which costs and outcomes need to be considered in a full evaluation of newborn screening programmes. Because of the increasing scale and scope of such programmes, a better understanding of the methods of high-quality economic evaluations may be crucial for both producers/authors and consumers/reviewers of newborn screening-related economic evaluations. The aim of this study was therefore to develop specific guidelines designed to assess and improve the methodological quality of economic evaluations in newborn screening.
To develop the guidelines, existing guidelines for assessing the quality of economic evaluations were identified through a literature search, and were reviewed and consolidated using a deductive iterative approach. In a subsequent test phase, these guidelines were applied to various economic evaluations which acted as case studies.
The guidelines for assessing and improving the methodological quality of economic evaluations in newborn screening are organized into 11 categories: “bibliographic details”, “study question and design”, “modelling”, “health outcomes”, “costs”, “discounting”, “presentation of results”, “sensitivity analyses”, “discussion”, “conclusions”, and “commentary”.
The application of the guidelines highlights important issues regarding newborn screening-related economic evaluations, and underscores the need for such issues to be afforded greater consideration in future economic evaluations. The variety in methodological quality detected by this study reveals the need for specific guidelines on the appropriate methods for conducting sound economic evaluations in newborn screening.
Where resources needed for the production of health benefits are scarce, decision makers need to consider both the effectiveness and the cost-effectiveness of health interventions. Economic evaluation is the comparative analysis of alternative interventions with regard to both their costs and outcomes . Although economic evaluations are used to inform decision making in health care, several studies have shown that economic evaluations differ markedly in quality [2–4]. Common methodological flaws in economic evaluations relate to study design, data collection and analysis, and interpretation or reporting of results . In order to assess the methodological rigour of economic evaluations, various instruments have been developed. To date, two of these instruments have received more scrutiny than others , namely the “Guidelines for authors and peer reviewers of economic submissions to the BMJ“  and the “Consensus on Health Economic Criteria (CHEC) list for assessment of methodological quality of economic evaluations“ . Whereas the latter is a specific list of criteria for conducting systematic reviews of trial-based economic evaluations, the former is a general checklist for assessing economic evaluations. However, both the study questions and the inclusion criteria generally require additional disease-specific guidelines in order to assess and improve the methodological quality of economic evaluations in a specific disease area . For instance, in order to support effective decision making, the Canadian Agency for Drugs and Technologies in Health (CADTH) published oncology guidelines which provide specific guidance for conducting high-quality economic evaluations of oncology products .
The uniqueness of child health and associated methodological challenges in conducting economic evaluations have already been pointed out . Specific methodological challenges also emerge in economic evaluations of newborn screening. For instance, there is a lack of adequate and robust long-term data from which useful evidence can be produced [12, 13]. Besides the dearth of long-term follow-up studies of newborn screening programmes and the associated extrapolation of short-term data to capture a lifetime time horizon, other challenges pertain to the problem of appropriate discounting, the use of quality adjusted life years (QALYs) in newborns, and which costs and outcomes need to be considered in a full evaluation of newborn screening programmes. Because of the increasing scale and scope of such programmes, a better understanding of the methods of high-quality economic evaluations may be crucial for both producers/authors and consumers/reviewers of newborn screening-related economic evaluations in order to assure high quality allocation decisions in newborn screening. The aim of this study was therefore to establish specific guidelines for assessing and improving the methodological quality of economic evaluations in newborn screening. These guidelines are designed for both producers/authors and consumers/reviewers of economic evaluations of newborn screening, and are suitable for trial-based and model-based economic evaluations. Additionally, the guidelines were tested with respect to various economic evaluations in the areas of newborn hearing screening and newborn screening for medium-chain acyl–CoA dehydrogenase deficiency (MCADD) using tandem mass spectrometry (MS/MS).
In order to identify and review existing guidelines for assessing the quality of economic evaluations, several databases and the internet were searched. Using a deductive iterative process, data from textbooks on health economic evaluation [1, 14, 15], principles of good practice published by different ISPOR Task Forces [16, 17], and from several guidelines, checklists and criteria lists [7, 8, 18–22] were extracted qualitatively based on general categories identified in previous work . After data extraction, the relevance of each category and item within it was discussed in relation to the development of specific guidelines to assess and improve the methodological quality of economic evaluations in newborn screening. The final set of guidelines was developed based on majority consensus.
In order to test the specific guidelines, they were applied to economic evaluations which were identified through a systematic review of the literature: economic evaluations of newborn hearing screening and economic evaluations of newborn screening for inherited metabolic disorders including MCADD by MS/MS. The following databases were searched from inception to April 2012 for relevant studies: HTA database, NHS Economic Evaluation Database, Pediatric Economic Database, PubMed, Web of Science. The literature search for economic evaluations of newborn hearing screening used the following terms: (“newborn” or “neonatal” or “infant”) combined with (“economic” or “cost”) and (“hearing” or “deaf”) and “screening”. The literature search for economic evaluations of newborn screening for inherited metabolic disorders including MCADD by MS/MS used the following terms: (“newborn” or “neonatal” or “infant”) combined with (“economic” or “cost”) and (“screening” or “tandem mass spectrometry” or “MS/MS screening”) and (“metabolic disorders” or “MCADD” or “MCAD” or “medium-chain acyl-CoA dehydrogenase deficiency”). The searches were limited by study design (cost analysis, cost-minimization analysis, cost-effectiveness analysis, cost-utility analysis, and cost-benefit analysis), year (2000 – 2012) and language of publication (English, French or German). In addition, the economic analyses should be fully described and transparent. Further studies were identified by examining the reference lists of several systematic reviews and health technology assessments in the areas of newborn hearing screening [23–25] and newborn screening for inherited metabolic disorders including MCADD by MS/MS [12, 13, 26, 27]. The full list of sources and the search strategy are available from the authors.
The guidelines for assessing and improving the methodological quality of economic evaluations in newborn screening are organized into 11 categories: “bibliographic details”, “study question and design”, “modelling”, “health outcomes”, “costs”, “discounting”, “presentation of results”, “sensitivity analyses”, “discussion”, “conclusions”, and “commentary”. The full set of guidelines is presented in Table 1.
A. Bibliographic details
In addition to the common bibliographic details (author(s), title, and source of publication), details on the publication type (e.g., peer-reviewed journal article, non-peer-reviewed journal article, abstract, technical report, book, or book section) should be provided in order to obtain information on the methods of quality control during the publication process. Furthermore, the institutional affiliation of author(s) and the source of funding should be stated because both items can act as an indicator for potential conflicts of interest.
B. Study question and design
First, a clearly defined and answerable study question should be posed. This should encompass a detailed description of the screening intervention under evaluation and its comparator. Furthermore, the target population of the screening intervention and the time horizon over which costs and health outcomes are likely to accumulate should be stated. The perspective and the time horizon of the analysis should be clearly stated, and the reasons underlying these choices should be made transparent. Fixing these boundaries of the study determines which elements of costs and benefits should be included in the economic evaluation. If the primary purpose of the economic evaluation is to assist health policy makers in making optimal societal decisions, a societal perspective should be chosen, i.e. all relevant short- (costs of screening and diagnostic follow-up) and long-term costs (costs for therapy, rehabilitation, education and social services, and lost productivity), and consequences such as morbidity or mortality, should be included. In the case of newborn screening programmes, this societal perspective usually implies that a lifetime time horizon should be adopted. To judge the generalizability and transferability of the results of economic evaluation, the setting, including time and place of intervention, should be described in detail, because the type of setting (e.g., inpatient or outpatient) can have a great influence on screening coverage. Finally, the study design (clinical study or decision-analytic model), the type of economic evaluation (cost-consequences analysis, cost-minimization analysis, cost-effectiveness analysis, cost-utility analysis, and cost-benefit analysis), and the primary outcome measure used in the economic evaluation should be stated, and the study population should be described in detail to assess the generalizability of economic evaluation results with regard to the target population of the screening intervention.
Economic evaluations based on decision-analytic models synthesize the available evidence systematically and provide projected estimates of long-term outcomes when long-term data are missing. Thus, decision-analytic models can support policy decisions for newborn screening programmes . First, the model type (cohort model or patient-level simulation model) should be specified. Furthermore, the model structure should be described in detail, including statement of decision problem and scope, and it should be appropriate for addressing the study question. In addition, all model assumptions and associated choices should be stated and justified, and all sources used to develop and/or populate the model should be stated and justified. Furthermore, in Markov models, information on cycle length, and health states and transitions should be provided. Finally, with regard to model validation, internal and external validations should be described. A more detailed framework for assessing the quality of decision-analytic models is available , which is based on a comprehensive review of the respective literature . It is recommended that producers/authors and consumers/reviewers of newborn screening-related economic evaluations consult these or comparable good modelling practice guidelines for a more detailed assessment of the transparency and quality of model-based economic evaluations.
D. Health outcomes
Initially, it is necessary to determine which type of health outcome measure was used in the economic evaluation to assess whether all relevant consequences of the condition under evaluation were considered. The choice of health outcome measure should be based on the condition(s), intervention, and health policy setting and audience. Common types of health outcome measure used in newborn screening-related economic evaluations are health outcomes measured in natural units (e.g., life-years gained or cases correctly diagnosed), and preference-based health outcomes such as the quality adjusted life year (QALY). In general, health outcome measures should be used that are relevant to the patient. Especially with regard to economic evaluations of newborn screening for inherited metabolic disorders, appropriate preference-based health outcomes which integrate mortality and morbidity in a single outcome measure should be used to value meaningful differences between MS/MS and alternatives in terms of health-related quality of life. Furthermore, in contrast to measuring outcome by the number of cases correctly diagnosed, preference-based health outcomes such as QALYs can be used to compare screening interventions that are targeted at different disorders. However, the use of QALYs in child health care poses serious methodological challenges which must be considered carefully in the search for an appropriate outcome measure [30, 31]. Other health outcomes in newborn screening-related economic evaluations are intermediate health outcomes, non-health outcomes, or health outcomes measured in monetary units. In all cases, the type of health outcome or effectiveness measure influences its measurement and valuation. The methods used to measure and value health outcomes should be the most appropriate ones for the condition under evaluation and the study question, and should be stated for reasons of transparency and comparability of economic evaluations. Finally, the source of data used to calculate the health outcomes should be provided as well, including the evidence of effectiveness. One of the main weaknesses in economic evaluations of newborn screening is usually in the area of epidemiology . There is a lack of data to directly compare health outcomes for both screened and unscreened cohorts. Furthermore, readers should be aware that some sources of data such as systematic reviews might be more valid than others such as expert opinions. To validate data and modelling assumptions, epidemiologists and clinicians should be consulted.
Contrary to many other health technologies, interventions in newborn screening generate costs at the administrative level and at the level of the individual patient. Patient-related costs consist of direct medical costs (screening costs, treatment costs, and downstream health costs, such as monitoring and managing of cases detected), direct non-medical costs (e.g., travel expenses and time costs of parents accompanying their child, or costs associated with institutional care and special education), and indirect costs (costs for lost productivity). With regard to indirect costs in economic evaluations of newborn screening, it is essential to value both the patient and the caregiver time. A critical review of existing valuation methods can be found elsewhere . Programme-related costs are all “costs incurred at the administrative levels outside the point of delivery of health care to beneficiaries“ , such as costs for the implementation, organization, administration, monitoring, and evaluation of the screening programme and costs for tracking. In general, all resources that are relevant to the study perspective should be identified, measured, and valued. For instance, additional health care costs prior to diagnosis may be included for conditions not screened for , or unrelated costs that accrue from the intervention during life-years gained may be considered as well, an issue which is controversial in the literature [34–39]. Both the costing and the valuation method (e.g., adjustment for inflation) used should be stated and justified, and the sources used to obtain data on resource utilisation and unit costs should be reported. In order to enhance generalizability and transferability, quantities and unit costs should be reported separately.
In newborn screening-related economic evaluations with a longer time horizon than one year, costs and health outcomes should be discounted. However, discount rates can have a great influence on the results of an economic evaluation [40, 41]. For instance, studies which discount costs more than they discount health outcomes will result in a relatively low ICER . Therefore, the choice of discount rates should be stated and justified: both a discount rate for costs and a discount rate for health outcomes should be provided. Furthermore, a justification of the discount rates should be given – i.e., the rationale for the choice of discount model and discount rates in the base case and in the sensitivity analyses, and for the use of different or common discount rates for costs and health outcomes should be provided.
G. Presentation of results
First, both the costs and the health outcomes should be presented per newborn, and in both disaggregated and aggregated forms. Furthermore, an incremental cost-effectiveness ratio (ICER) should be provided for the primary outcome measure used in the economic analysis. Present values and trends of costs and health outcomes at the population level should be reported as well. Additionally, in terms of costs, present values and trends should be differentiated by payer because in newborn screening costs are borne by different payers (e.g., health insurance, patient, or state and federal government). Finally, the coverage of screening should be reported because it is one of the key parameters driving the cost-effectiveness of alternative screening strategies.
H. Sensitivity analyses
In the area of newborn screening for inherited metabolic disorders, various factors, such as prevalence of the disorder, or frequency of adverse outcomes, influence the ICER , and therefore should be varied in sensitivity analyses in order to assess the impact on economic evaluation results and conclusions. Parameter uncertainty should be addressed, including a statement and justification of the ranges or distributions of parameters used for sensitivity analyses. In model-based economic evaluations, modelling uncertainty should be investigated as well because structure, methods, and assumptions of models are subject to uncertainty. Furthermore, methods (univariate or multivariate sensitivity analyses, probabilistic sensitivity analyses, threshold analyses and analyses of extremes or best/worst case analyses) and results of sensitivity analyses should be reported.
First, the limitations of the study should be discussed. For instance, in newborn screening-related economic evaluations, one of the main constraints and weaknesses refers to the lack of long-term data . Second, both generalizability and transferability of the study results should be discussed, which relate to the extent to which the economic evaluation results are applicable to different settings. Methodological (e.g., perspective, discount rate), health care system (e.g., absolute and relative prices in health care), and population characteristics (e.g., disease prevalence) are factors that influence the transferability of economic evaluation results  in general as well as in newborn screening. Finally, ethical and distributional issues should also be considered, because both are essential for economic evaluations with respect to rare and disabling conditions. In the case of economic evaluations of newborn screening, ethical issues which might be of concern to users include the principles of beneficence (“finding the evidence of benefit”), non-maleficence (“finding the harm”), autonomy (“the right to choose, and the protection of those with diminished autonomy”), and justice (“distributive justice”) . For instance, ethical challenges emerging from the use of QALYs for disabling conditions should be addressed . Other ethical issues are associated with the use of MS/MS: detection of diseases for which there is no (effective) treatment, detection of diseases whose natural history is not (well) understood, detection of maternal variance, detection of carriers, and finding the evidence of benefit . However, benefits from the early and correct diagnosis of a newborn with a largely untreatable disorder relate to the prevention of misdiagnosis, reassurance of parents, and enabling future reproductive choices . In economic evaluations of newborn screening for inherited metabolic disorders, ethical issues regarding antenatal screening may also be included . Furthermore, the distribution of costs and benefits should be considered .
The conclusions should follow from the results of the economic evaluation and should answer the study question.
The commentary of the consumer/reviewer of the economic evaluation should be a critical summary of the overall reliability and generalizability of the economic evaluation and include the following issues: Selection of comparators, validity of estimate of measure of effectiveness, validity of estimate of measure of health outcome, validity of estimate of costs, other issues, and implications of the economic evaluation. These issues were adapted from the NHS EED structured abstract format .
Application of the guidelines for assessing and improving economic evaluations in newborn screening
In the following, the experiences of applying the guidelines to economic evaluations in the areas of newborn screening for inherited metabolic disorders including MCADD by MS/MS and newborn hearing screening are reported.
Economic evaluations of newborn hearing screening
In the area of newborn hearing screening, the guidelines were applied to 21 economic evaluations [23–25, 48–65], of which an overview is provided in Additional file 1. These studies were heterogeneous with regard to target population (all newborns or newborns with risk factors), screening technology (otoacoustic emission (OAE) or automated auditory brainstem response (AABR)), screening strategy (universal screening or risk screening), number of stages in the initial screening process (one or two), detection of hearing loss (unilateral or bilateral), setting (inpatient or outpatient), cost categories considered (development/implementation/organization/monitoring of the screening programme, tracking, screening tests, diagnostic follow-up, treatment for detected cases of hearing disorder, patient transportation for diagnostic procedures and treatment, education, work time loss for parents/adults with hearing disorders, productivity loss due to hearing disorders/premature mortality), and comparators. Most of the studies evaluated different screening technologies [25, 48–51, 54–59]. Five economic evaluations compared different screening strategies [23, 24, 60–62]. Two studies analyzed different screening devices [53, 64]. Two other studies examined different settings [52, 65], and one study compared subjective and objective screening tests . Six of the economic evaluations were from the United States [51, 55–57, 60, 62], six from Germany [23, 48, 53, 61, 64, 65], three from the United Kingdom [52, 54, 63], two from Taiwan [58, 59] and Canada [25, 50], and one each from Australia  and the Netherlands .
The results of the application of the guidelines revealed that most of the economic evaluations met the following items:
Study question (B.1.)
In contrast, the following items were inadequately met or not considered at all:
Model validation (C.7.)
Health outcomes measured in natural units (D.1.), adjusted by utility weights or health state preference scores (D.2.), or measured in monetary units (D.3.)
Measurement (E.1.b. and E.2.b.) and valuation (E.1.c. and E.2.c.) of resource data
Justification of discount rates (F.3.)
ICER for the primary outcome measure (G.3.)
Modelling uncertainty (H.2.)
Generalizability and transferability of the economic evaluation results (I.2.)
Ethical and distributional issues (I.3.)
In economic evaluations of newborn hearing screening, one of the main problems refers to the time horizon over which relevant costs and consequences accrue. In most of the studies, the time horizon from the initial screening test up to and including diagnostic follow-up was considered. The time horizon of seven economic evaluations was even restricted to the initial screening test [50, 51, 53, 54, 58, 59, 64]. To assess the cost-effectiveness of newborn hearing screening vs. risk or no (regular) screening, a long time horizon and a transsectoral perspective are needed, because long-term cost savings due to early detection and treatment are related to other sectors than the health care sector – such as, for example, education and labour. Four studies [24, 25, 60, 63] used a societal perspective, three of which [24, 25, 60] considered a lifetime time horizon, but all long-term estimates heavily relied on assumptions rather than long-term empirical data. To date, the long-term effectiveness of newborn hearing screening programmes has not been adequately evaluated. In particular, valid data on patient-relevant parameters, such as social skills, language acquisition, educational development (e.g., school performance), professional career, and quality of life are sparse [66–69].
In general, the economic evaluations in newborn hearing screening lack a clear and detailed description of which cost categories were considered, how the associated resource quantities were measured, and how the resources were valued, which hinders the generalizability and transferability of economic evaluation results. Most of the studies only include the costs of screening and diagnostic follow-up. Costs for the implementation of the newborn hearing screening programme are only considered in some studies [48, 56, 63, 65]. Furthermore, costs of screening vary due to different factors, such as different screening technologies (OAE or AABR), different settings (inpatient or outpatient), number of stages in the initial screening process (one-stage or two-stage), different target populations (all newborns or newborns with risk factors), or detection of hearing loss (unilateral or bilateral). In addition, measurement and valuation of resources vary, which also hinders the comparability of cost-effectiveness between studies. Furthermore, most studies rely on resource estimates identified through literature review or collected retrospectively alongside effectiveness data, and report only average costs per child screened or per child detected.
Instead of identifying the effectiveness of newborn hearing screening in terms of a health outcome variable, the standard approach to measuring effectiveness in all but three studies simply was to count the number of children screened or detected. Two further studies [23, 61] used the number of child months detected to emphasize the importance of early detection and intervention. There is only one study  using a health outcome measure called “quality-weighted detected child months”. This QALY-like measure is based on the assumption that children with hearing losses detected earlier would experience an improved quality of life; however, empirical data in support of this assumption are missing. Finally, only eight studies reported incremental cost-effectiveness ratios [23–25, 55, 60, 62, 63, 65] which are relevant to decision making in health care.
Economic evaluations in newborn screening for inherited metabolic disorders including MCADD by MS/MS
In newborn screening of inherited metabolic disorders, the guidelines were applied to twelve economic evaluations [70–81], of which an overview can be found in Additional file 2. These studies were heterogeneous in terms of disease prevalence, frequency of adverse outcomes (e.g., morbidity and mortality), metabolic disorders considered under MS/MS screening, and cost categories considered. In these economic evaluations, one (MCADD) up to 21 inherited metabolic disorders were included. The prevalence of MCADD alone ranged from 1.19 to 9 per 100,000 newborns. Five studies were from the United States [73, 75–77, 80], two from Canada [71, 78], and one each from Australia , Finland , France , the United Kingdom , and the Netherlands . Ten of the twelve studies provided incremental cost-effectiveness ratios [70–73, 75, 77–81]. In the remaining two studies MS/MS screening was found to be cost saving [74, 76]. The incremental cost-effectiveness ratios varied widely due to different epidemiological and clinical parameters, different outcome measures (life-years gained vs. QALYs gained), and differences in the number of conditions screened under MS/MS and in the cost categories considered in the economic evaluations (costs of screening and diagnostic follow-up, costs of immediate and ongoing treatment, costs of non-medical care, costs of productivity losses). For instance, only five studies considered start-up costs of MS/MS screening [70, 76–79].
The results of the application of the guidelines revealed that most of the economic evaluations met the following items:
Study question (B.1.)
Target population (B.4.)
In contrast, the following items were inadequately met or not considered at all:
Model validation (C.7.)
Health outcomes measured in monetary units (D.3.) and non-health outcomes (D.5.)
Method of valuing health benefits (D.2.b)
Measurement (E.1.b. and E.2.b.) and valuation (E.1.c. and E.2.c.) of resource data
Justification of discount rates (F.3.)
Generalizability and transferability of the economic evaluation results (I.2.)
Ethical and distributional issues (I.3.)
In economic evaluations of newborn screening for inherited metabolic disorders, one of the main problems relates to the lack of epidemiological data, such as the frequency of adverse outcomes with and without screening in terms of morbidity and mortality [12, 13]. A further common weakness refers to the health outcomes used in the economic analysis. In half of the studies, life-years gained were used as a health outcome measure, but mortality does not attach great importance to the prevention of morbidity . In contrast, the quality adjusted life year (QALY) incorporates both mortality and morbidity in a single outcome measure. However, the validity of QALY estimates in newborn screening is limited [82–84]. In newborn screening-related cost-utility analyses of MS/MS, it is common to take published utility weights from studies of adults with neurological impairments caused by other disorders than metabolic ones , although parent preferences for pediatric outcomes are also available . The application of these parent preferences to published pediatric cost-utility analyses showed that more than a third of these analyses would change if such utilities were used . In a recent review it was found that QALY weights for neurological impairments vary widely in pediatric economic evaluations , an issue which is also related to the economic evaluations considered here. To date, appropriate health state classification instruments that consider the dynamics of child development are lacking, health state classification instruments suitable for use in children aged 5 years and younger have not been established, and the role of proxies for measuring and valuing children’s health-related quality of life is not fully understood . However, a child-friendly version of the EQ-5D is now available, namely the EQ-5D-Y , which proved to be feasible, reliable, and valid . Nevertheless, further research is needed in terms of the development of methods that consider the health benefits of parents, and the consequences of integrating different forms of utility measurement in children and adults  in order to address potential harms arising from early detection or false-positive test results.
To date, no specific guidelines for assessing and improving the methodological quality of economic evaluations in newborn screening have been developed. Only one study was found that established “a generalised model of cost-effectiveness appraisal“ in the area of newborn screening for rare metabolic conditions using MS/MS, in order to identify the key variables that need to be considered to estimate costs and effects of MS/MS and alternatives : these key variables comprise general items, such as prevalence of the conditions screened for with and without screening, or sensitivity and specificity of MS/MS and alternatives for the various conditions, screening costs, treatment costs, downstream health costs, societal costs, mortality, and morbidity. These newborn screening-specific guidelines were developed to highlight the need for greater standardization in the methodological quality of newborn screening-related economic evaluations and to ensure that future economic evaluations in newborn screening are conducted consistently and appropriately in order to enhance the comparability, generalizability and transferability of economic evaluation results in newborn screening. Furthermore, the high economic and human burden of diseases that can be detected early through newborn screening programmes gives such programmes a high priority in resource allocation, and underscores the need for specific guidelines. The application of the guidelines revealed methodological flaws regarding perspective, time horizon, discounting, use of assumptions rather than data, measuring and valuing health outcomes and resource data, modelling in terms of uncertainty and validation, transparency of reporting, generalizability, and ethical and distributional issues. Using the Pediatric Quality Appraisal Questionnaire , a quality appraisal of pediatric health economic evaluations published between 1980 and 1999 revealed that these economic evaluations were of good quality for “economic evaluation”, “comparators”, “target population”, “discounting” and “conclusions”, and only of poor quality for “perspective”, “incremental analysis”, and “conflict of interest” . In this study, it was found that the quality domains of the specific guidelines developed here are fulfilled differently by the economic evaluations in the areas of newborn screening for inherited metabolic disorders including MCADD by MS/MS and newborn hearing screening. For example, in opposite to the economic evaluations in newborn hearing screening, all economic evaluations of newborn screening for inherited metabolic disorders including MCADD by MS/MS provide incremental cost-effectiveness ratios.
The guidelines proposed have several limitations. First, they can be applied to all economic evaluations in newborn screening. Therefore, issues specific to certain diseases detectable by newborn screening or technologies of newborn screening are not addressed in these guidelines. For instance, there is controversy in the literature whether newborn screening for all detectable disorders with MS/MS is cost-effective [92, 93]. This question, from which methodological and ethical issues arise, is common for screening technologies that can be used to detect multiple conditions such as MS/MS or DNA-based screening. Cipriano et al.  suggest that variation in MS/MS sensitivity and specificity to detect other disorders, different treatment costs, and different potential benefits from early diagnosis and treatment may require the independent evaluation of each disease. Furthermore, researchers have still to elaborate a methodology which enables the economic evaluation of health technologies with a one-test, many-disorders approach – i.e., the simultaneous detection of a panel of disorders in a single individual. Second, the guidelines should be considered a starting point for assessing and improving the methodological quality of economic evaluations in newborn screening. As methodological issues advance, the guidelines should be regularly updated. Regarding issues of modelling and transferability, more detailed assessments are available [21, 43, 94, 95]. Furthermore, the guidelines do not result in a quality score whose value, however, may be critically assessed , but in a structured abstract which is an alternative method of communicating the information to decision makers. In a study by [Thurston et al. 97] it was found that decision makers prefer a more detailed structured abstract in addition to a very short summary.
These guidelines were provided in order to assess and improve the methodological quality of economic evaluations in newborn screening. The application to economic evaluations in the areas of newborn hearing screening, and newborn screening for inherited metabolic disorders including MCADD by MS/MS highlights important issues of economic evaluations in these two areas of newborn screening and underscores the need for their greater consideration in future newborn screening-related economic evaluations. The variation in methodological quality of the economic evaluations in newborn screening which is detected through application of the guidelines, and which limits the comparability, generalizability and transferability of economic evaluation results, reveals the necessity for specific guidelines regarding the methods for conducting sound economic evaluations in newborn screening.
Drummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL: Methods for the economic evaluation of health care programmes. 2005, Oxford University Press, New York, 3
Neumann PJ, Greenberg D, Olchanski NV, Stone PW, Rosen AB: Growth and quality of the cost-utility literature, 1976–2001. Value Health. 2005, 8 (1): 3-9. 10.1111/j.1524-4733.2005.04010.x.
Neumann PJ, Fang CH, Cohen JT: 30 years of pharmaceutical cost-utility analyses: growth, diversity and methodological improvement. Pharmacoeconomics. 2009, 27 (10): 861-872. 10.2165/11312720-000000000-00000.
Schwappach DL, Boluarte TA: HEE-GER: a systematic review of German economic evaluations of health care published 1990–2004. BMC Health Serv Res. 2007, 7: 7-10.1186/1472-6963-7-7.
Drummond M, Sculpher M: Common methodological flaws in economic evaluations. Med Care. 2005, 43 (7 Suppl): 5-14.
Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. Edited by: Higgins JPT, Green S. 2011, The Cochrane Collaboration, , Available from http://www.cochrane-handbook.org
Drummond MF, Jefferson TO: Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ. 1996, 313 (7052): 275-283. 10.1136/bmj.313.7052.275.
Evers S, Goossens M, de Vet H, van Tulder M, Ament A: Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care. 2005, 21 (2): 240-245.
Centre for Reviews and Dissemination: Systematic reviews: CRD's guidance for undertaking reviews in health care. 2009, University of York, York
Mittmann N, Evans WK, Rocchi A, Longo CJ, Au HJ, Husereau D, Leighl N, Isogai P, Krahn M, Peacock S, et al: Addendum to CADTH’s Guidelines for the Economic Evaluation of Health Technologies: Specific Guidance for Oncology Products. 2009, Canadian Agency for Drugs and Technologies in Health, Ottawa
Ungar WJ, Gerber A: The uniqueness of child health and challenges to measuring costs and consequences. Economic Evaluation in Child Health. Edited by: Ungar WJ. 2009, Oxford University Press, Oxford
Grosse SD: Economic evaluations of newborn screening. Economic Evaluation in Child Health. Edited by: Ungar WJ. 2009, Oxford University Press, Oxford
Norman R, Haas M, Wilcken B: International perspectives on the cost-effectiveness of tandem mass spectrometry for rare metabolic conditions. Health Policy. 2009, 89 (3): 252-260. 10.1016/j.healthpol.2008.08.003.
Luce BR, Elixhauser A: Standards for Socioeconomic Evaluation of Health Care Products and Services. 1990, Springer, Berlin, Heidelberg
Gold MR, Russell LB, Siegel JE, Weinstein MC: Cost-effectiveness in health and medicine. 1996, Oxford University Press, Oxford
Weinstein MC, O'Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, Luce BR: Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices–Modeling Studies. Value Health. 2003, 6 (1): 9-17. 10.1046/j.1524-4733.2003.00234.x.
Ramsey S, Willke R, Briggs A, Brown R, Buxton M, Chawla A, Cook J, Glick H, Liljas B, Petitti D, et al: Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force report. Value Health. 2005, 8 (5): 521-533. 10.1111/j.1524-4733.2005.00045.x.
Chiou CF, Hay JW, Wallace JF, Bloom BS, Neumann PJ, Sullivan SD, Yu HT, Keeler EB, Henning JM, Ofman JJ: Development and validation of a grading system for the quality of cost-effectiveness studies. Med Care. 2003, 41 (1): 32-44. 10.1097/00005650-200301000-00007.
Siebert U, Behrend C, Mühlberger N, Wasem J, Greiner W, Graf von der Schulenburg JM, Welte R, Leidl R: Entwicklung eines Kriterienkataloges zur Beschreibung und Bewertung ökonomischer Evaluationsstudien in Deutschland. Approaches and methods of economic evaluation in health care - an international perspective. Edited by: Leidl R, Graf von der Schulenburg JM, Wasem J. 1999, Nomos, Baden-Baden, 156-170.
Graf von der Schulenburg JM, Greiner W, Jost F, Klusen N, Kubin M, Leidl R, Mittendorf T, Rebscher H, Schoeffski O, Vauth C, et al: German recommendations on health economic evaluation: third and updated version of the Hanover Consensus. Value Health. 2008, 11 (4): 539-544. 10.1111/j.1524-4733.2007.00301.x.
Philips Z, Bojke L, Sculpher M, Claxton K, Golder S: Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics. 2006, 24 (4): 355-371. 10.2165/00019053-200624040-00006.
National Institute for Health and Clinical Excellence: Updated guide to the methods of technology appraisal - June 2008. 2008, National Institute for Health and Clinical Excellence, London
Schnell-Inderst P, Kunze S, Hessel F, Grill E, Siebert U, Nickisch A, von Voss H, Wasem J: Screening of the hearing of newborns - Update. GMS Health Technol Assess. 2006, 2: Doc20.
Merlin T, Hedayati H, Sullivan T, Buckley E, Newton S, Hodgkinson B, Bywood P, Jenner F, Moss J, Hiller JE: Universal neonatal hearing screening. MSAC reference 17 Assessment report. 2007, MSAC, Canberra
Schopflocher D, Corabian P, Eng K, Lier D: Screening Newborns for Hearing. 2007, Institute of Health Economics, Alberta
Tran K, Banerjee S, Li H, Noorani HZ, Mensinkai S, Dooley K: Newborn screening for medium chain acyl~CoA dehydrogenase deficiency using tandem mass spectrometry: clinical and cost-effectiveness [Technology report no 62]. 2006, Canadian Coordinating Office for Health Technology Assessment, Ottawa
Pandor A, Eastham J, Beverley C, Chilcott J, Paisley S: Clinical effectiveness and cost-effectiveness of neonatal screening for inborn errors of metabolism using tandem mass spectrometry: a systematic review. Health Technol Assess. 2004, iii (12): 1-121.
Prosser LA, Grosse SD, Kemper AR, Tarini BA, Perrin JM: Decision analysis, economic evaluation, and newborn screening: challenges and opportunities. Genet Med. 2012 Apr 5, Epub ahead of print PubMed PMID: 22481131-10.1038/gim.2012.24.
Philips Z, Ginnelly L, Sculpher M, Claxton K, Golder S, Riemsma R, Woolacoot N, Glanville J: Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess. 2004, 8 (36): iii-iv. ix-xi, 1–158
Ungar WJ: Challenges in Health State Valuation in Paediatric Economic Evaluation: Are QALYs Contraindicated?. Pharmacoeconomics. 2011, 29 (8): 641-652. 10.2165/11591570-000000000-00000.
Sung L, Petrou S, Ungar WJ: Measurement of health utilities in children. Economic Evaluation in Child Health. Edited by: Ungar WJ. 2009, Oxford University Press, Oxford
Tranmer JE, Guerriere DN, Ungar WJ, Coyte PC: Valuing patient and caregiver time: a review of the literature. Pharmacoeconomics. 2005, 23 (5): 449-459. 10.2165/00019053-200523050-00005.
Johns B, Baltussen R, Hutubessy R: Programme costs in the economic evaluation of health interventions. Cost Eff Resour Alloc. 2003, 1 (1): 1-10.1186/1478-7547-1-1.
Meltzer D: Accounting for future costs in medical cost-effectiveness analysis. J Health Econ. 1997, 16 (1): 33-64. 10.1016/S0167-6296(96)00507-3.
Nyman JA: Should the consumption of survivors be included as a cost in cost-utility analysis?. Health Econ. 2004, 13 (5): 417-427. 10.1002/hec.850.
Van Baal PH, Feenstra TL, Hoogenveen RT, de Wit GA, Brouwer WB: Unrelated medical care in life years gained and the cost utility of primary prevention: in search of a 'perfect' cost-utility ratio. Health Econ. 2007, 16 (4): 421-433. 10.1002/hec.1181.
Van Baal PH, Wong A, Slobbe LC, Polder JJ, Brouwer WB, de Wit GA: Standardizing the inclusion of indirect medical costs in economic evaluations. Pharmacoeconomics. 2011, 29 (3): 175-187. 10.2165/11586130-000000000-00000.
Rappange DR, Brouwer WB, Rutten FF, van Baal PH: Lifestyle intervention: from cost savings to value for money. J Public Health (Oxf). 2010, 32 (3): 440-447. 10.1093/pubmed/fdp079.
Liljas B, Karlsson GS, Stalhammar NO: On future non-medical costs in economic evaluations. Health Econ. 2008, 17 (5): 579-591. 10.1002/hec.1279.
Claxton K, Paulden M, Gravelle H, Brouwer W, Culyer AJ: Discounting and decision making in the economic evaluation of health-care technologies. Health Econ. 2011, 20 (1): 2-15. 10.1002/hec.1612.
Schad M, John J: Towards a social discount rate for the economic evaluation of health technologies in Germany: an exploratory analysis. Eur J Health Econ. 2012, 13 (2): 127-144. 10.1007/s10198-010-0292-9.
Grosse SD: Cost-effectiveness as a criterion for newborn screening policy decisions. Ethics and Newborn Genetic Screening: New Technologies, New Challenges. Edited by: Baily MA, Murray TH. 2009, John Hopkins University Press, Baltimore, 58-88.
Welte R, Feenstra T, Jager H, Leidl R: A decision chart for assessing and improving the transferability of economic evaluation results between countries. Pharmacoeconomics. 2004, 22 (13): 857-876. 10.2165/00019053-200422130-00004.
Wilcken B: Ethical issues in newborn screening and the impact of new technologies. Eur J Pediatr. 2003, 162 (Suppl 1): S62-S66.
Brock DW: Cost-Effectiveness and Disability Discrimination. Econ Philos. 2009, 25 (1): 27-47. 10.1017/S0266267108002265.
Baily MA, Murray TH: Ethics, evidence, and cost in newborn screening. Hastings Cent Rep. 2008, 38 (3): 23-31. 10.1353/hcr.0.0009.
Centre for Reviews and Dissemination: NHS ECONOMIC EVALUATION DATABASE HANDBOOK. 2007, University of York, York
Böttcher P, Gramss M, Euler HA, Neumann K: Cost analysis of a universal newborn hearing screening for clinics using the State of Hesse as an example. HNO. 2009, 57 (1): 21-28. 10.1007/s00106-008-1879-7.
Boshuizen HC, van der Lem GJ, Kauffman-de Boer MA, van Zanten GA, Oudesluys-Murphy AM, Verkerk PH: Costs of different strategies for neonatal hearing screening: a modelling approach. Arch Dis Child Fetal Neonatal Ed. 2001, 85 (3): F177-F181. 10.1136/fn.85.3.F177.
Dort JC, Tobolski C, Brown D: Screening strategies for neonatal hearing loss: Which test is best?. J Otolaryngol. 2000, 29 (4): 206-210.
Gorga MP, Preissler K, Simmons J, Walker L, Hoover B: Some issues relevant to establishing a universal newborn hearing screening program. J Am Acad Audiol. 2001, 12 (2): 101-112.
Grill E, Uus K, Hessel F, Davies L, Taylor RS, Wasem J, Bamford J: Neonatal hearing screening: modelling cost and effectiveness of hospital- and community-based screening. BMC Health Serv Res. 2006, 6: 14-10.1186/1472-6963-6-14.
Heinemann M, Bohnert A: Hearing screening in newborn infants. Comparative studies and cost analysis with different instruments. Laryngorhinootologie. 2000, 79 (8): 453-458. 10.1055/s-2000-5911.
Iley KL, Addis RJ: Impact of technology choice on service provision for universal newborn hearing screening within a busy district hospital. J Perinatol. 2000, 20 (8 Pt 2): S122-S127.
Kezirian EJ, White KR, Yueh B, Sullivan SD: Cost and cost-effectiveness of universal screening for hearing loss in newborns. Otolaryngol Head Neck Surg. 2001, 124 (4): 359-367. 10.1067/mhn.2001.113945.
Lemons J, Fanaroff A, Stewart EJ, Bentkover JD, Murray G, Diefendorf A: Newborn hearing screening: costs of establishing a program. J Perinatol. 2002, 22 (2): 120-124. 10.1038/sj.jp.7210618.
Vohr BR, Oh W, Stewart EJ, Bentkover JD, Gabbard S, Lemons J, Papile LA, Pye R: Comparison of costs and referral rates of 3 universal newborn hearing screening protocols. J Pediatr. 2001, 139 (2): 238-244. 10.1067/mpd.2001.115971.
Lin HC, Shu MT, Lee KS, Lin HY, Lin G: Reducing false positives in newborn hearing screening program: how and why. Otol Neurotol. 2007, 28 (6): 788-792. 10.1097/MAO.0b013e3180cab754.
Lin HC, Shu MT, Lee KS, Ho GM, Fu TY, Bruna S, Lin G: Comparison of hearing screening programs between one step with transient evoked otoacoustic emissions (TEOAE) and two steps with TEOAE and automated auditory brainstem response. Laryngoscope. 2005, 115 (11): 1957-1962. 10.1097/01.mlg.0000178323.06183.3e.
Keren R, Helfand M, Homer C, McPhillips H, Lieu TA: Projected cost-effectiveness of statewide universal newborn hearing screening. Pediatrics. 2002, 110 (5): 855-864. 10.1542/peds.110.5.855.
Hessel F, Grill E, Schnell-Inderst P, Siebert U, Kunze S, Nickisch A, von Voss H, Wasem J: Economic evaluation of newborn hearing screening: modelling costs and outcomes. Ger Med Sci. 2003, 1: Doc09.
Kemper AR, Downs SM: A cost-effectiveness analysis of newborn hearing screening strategies. Arch Pediatr Adolesc Med. 2000, 154 (5): 484-488.
Uus K, Bamford J, Taylor R: An analysis of the costs of implementing the National Newborn Hearing Screening Programme in England. J Med Screen. 2006, 13 (1): 14-19. 10.1258/096914106776179764.
Neumann K, Berger R, Euler HA, Ahr A, Gall V: Brainstem response methods for neonatal hearing screening]. Zeitschrift für Audiologie. 2004, 43 (1): 10-21.
Lenarz T, Reuter G, Buser K, Altenhofen L: Modellprogramm Verbesserung der Früherfassung von Hörstörungen im Kindesalter. Machbarkeits- und Evaluationsstudie zum Einsatz otoakustischer Emissionen bei Neugeborenen. 2007, Deutscher Ärzte-Verlag, Köln
Wolff R, Hommerich J, Riemsma R, Antes G, Lange S, Kleijnen J: Hearing screening in newborns: systematic review of accuracy, effectiveness, and effects of interventions after screening. Arch Dis Child. 2010, 95 (2): 130-135. 10.1136/adc.2008.151092.
Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen: Newborn hearing screening in the detection of hearing impairment. 2007, Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG), Köln
Nelson HD, Bougatsos C, Nygren P: Universal newborn hearing screening: systematic review to update the 2001 US Preventive Services Task Force Recommendation. Pediatrics. 2008, 122 (1): e266-e276. 10.1542/peds.2007-1422.
Krauth C: Health economic analysis of screening. GMS Curr Top Otorhinolaryngology - Head Neck Surg. 2008, 7: 1-14.
Autti-Rämö I, Makela M, Sintonen H, Koskinen H, Laajalahti L, Halila R, Kaariainen H, Lapatto R, Nanto-Salonen K, Pulkki K, et al: Expanding screening for rare metabolic disease in the newborn: an analysis of costs, effect and ethical consequences for decision-making in Finland. Acta Paediatr. 2005, 94 (8): 1126-1136. 10.1080/08035250510029497.
Tran K, Banerjee S, Li H, Noorani HZ, Mensinkai S, Dooley K: Clinical efficacy and cost-effectiveness of newborn screening for medium chain acyl-CoA dehydrogenase deficiency using tandem mass spectrometry. Clin Biochem. 2007, 40 (3–4): 235-241.
Van der Hilst CS, Derks TG, Reijngoud DJ, Smit GP, TenVergert EM: Cost-effectiveness of neonatal screening for medium chain acyl-CoA dehydrogenase deficiency: the homogeneous population of The Netherlands. J Pediatr. 2007, 151 (2): 115-120. 10.1016/j.jpeds.2007.03.013. 120 e111-113
Venditti LN, Venditti CP, Berry GT, Kaplan PB, Kaye EM, Glick H, Stanley CA: Newborn screening by tandem mass spectrometry for medium-chain Acyl-CoA dehydrogenase deficiency: a cost-effectiveness analysis. Pediatrics. 2003, 112 (5): 1005-1015. 10.1542/peds.112.5.1005.
Pandor A, Eastham J, Chilcott J, Paisley S, Beverley C: Economics of tandem mass spectrometry screening of neonatal inherited disorders. Int J Technol Assess Health Care. 2006, 22 (3): 321-326.
Schoen EJ, Baker JC, Colby CJ, To TT: Cost-benefit analysis of universal tandem mass spectrometry for newborn screening. Pediatrics. 2002, 110 (4): 781-786. 10.1542/peds.110.4.781.
Carroll AE, Downs SM: Comprehensive cost-utility analysis of newborn screening strategies. Pediatrics. 2006, 117 (5 Pt 2): S287-S295.
Insinga RP, Laessig RH, Hoffman GL: Newborn screening with tandem mass spectrometry: examining its cost-effectiveness in the Wisconsin Newborn Screening Panel. J Pediatr. 2002, 141 (4): 524-531. 10.1067/mpd.2002.128116.
Cipriano LE, Rupar CA, Zaric GS: The cost-effectiveness of expanding newborn screening for up to 21 inherited metabolic disorders using tandem mass spectrometry: results from a decision-analytic model. Value Health. 2007, 10 (2): 83-97. 10.1111/j.1524-4733.2006.00156.x.
Hamers FF, Scemama O, Rumeau-Pichon C: Evaluation de l’extension du dépistage néonatal à une ou plusieurs erreurs innées du métabolisme par spectrométrie de masse en tandem. 1er volet: déficit en MCAD. 2011, Haute Autorite de Sante (HAS), Saint-Denis La Plaine Cedex
Prosser LA, Kong CY, Rusinak D, Waisbren SL: Projected costs, risks, and benefits of expanded newborn screening for MCADD. Pediatrics. 2010, 125 (2): e286-e294. 10.1542/peds.2009-0605.
Norman R, Haas M, Chaplin M, Joy P, Wilcken B: Economic evaluation of tandem mass spectrometry newborn screening in Australia. Pediatrics. 2009, 123 (2): 451-457. 10.1542/peds.2008-0911.
Griebsch I, Coast J, Brown J: Quality-adjusted life-years lack quality in pediatric care: a critical review of published cost-utility studies in child health. Pediatrics. 2005, 115 (5): e600-e614. 10.1542/peds.2004-2127.
Petrou S: Methodological issues raised by preference-based approaches to measuring the health status of children. Health Econ. 2003, 12 (8): 697-702. 10.1002/hec.775.
Prosser LA, Hammitt JK, Keren R: Measuring health preferences for use in cost-utility and cost-benefit analyses of interventions in children: theoretical and methodological considerations. Pharmacoeconomics. 2007, 25 (9): 713-726. 10.2165/00019053-200725090-00001.
Carroll AE, Downs SM: Improving decision analyses: parent preferences (utility values) for pediatric health outcomes. J Pediatr. 2009, 155 (1): 21-25. 10.1016/j.jpeds.2009.01.040. 25 e21-25
Finnell SM, Carroll AE, Downs SM: Application of classic utilities to published pediatric cost-utility studies. Acad Pediatr. 2012, 12 (3): 219-228. 10.1016/j.acap.2011.09.003.
Grosse SD, Prosser LA, Asakawa K, Feeny D: QALY weights for neurosensory impairments in pediatric economic evaluations: case studies and a critique. Expert Rev Pharmacoecon Outcomes Res. 2010, 10 (3): 293-308. 10.1586/erp.10.24.
Wille N, Badia X, Bonsel G, Burstrom K, Cavrini G, Devlin N, Egmar AC, Greiner W, Gusi N, Herdman M, et al: Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res. 2010, 19 (6): 875-886. 10.1007/s11136-010-9648-y.
Ravens-Sieberer U, Wille N, Badia X, Bonsel G, Burstrom K, Cavrini G, Devlin N, Egmar AC, Gusi N, Herdman M, et al: Feasibility, reliability, and validity of the EQ-5D-Y: results from a multinational study. Qual Life Res. 2010, 19 (6): 887-897. 10.1007/s11136-010-9649-x.
Ungar WJ, Santos MT: The Pediatric Quality Appraisal Questionnaire: an instrument for evaluation of the pediatric health economics literature. Value Health. 2003, 6 (5): 584-594. 10.1046/j.1524-4733.2003.65253.x.
Ungar WJ, Santos MT: Quality appraisal of pediatric health economic evaluations. Int J Technol Assess Health Care. 2005, 21 (2): 203-210.
Wilcken B: Newborn screening for all identifiable disorders with tandem mass spectrometry is cost effective: the negative case. Ann Acad Med Singapore. 2008, 37 (12 Suppl): 36-33.
Therrell BL, Buechner C: Newborn screening for all identifiable disorders with tandem mass spectrometry is cost effective: supporting arguments. Ann Acad Med Singapore. 2008, 37 (12 Suppl): 32-34.
Boulenger S, Nixon J, Drummond M, Ulmann P, Rice S, de Pouvourville G: Can economic evaluations be made more transferable?. Eur J Health Econ. 2005, 6 (4): 334-346. 10.1007/s10198-005-0322-1.
Sculpher M, Fenwick E, Claxton K: Assessing quality in decision analytic cost-effectiveness models. A suggested framework and example of application. Pharmacoeconomics. 2000, 17 (5): 461-477. 10.2165/00019053-200017050-00005.
Stearns SC, Drummond M: Grading systems for cost-effectiveness studies: is the whole greater than the sum of the parts?. Med Care. 2003, 41 (1): 1-3. 10.1097/00005650-200301000-00001.
Thurston SJ, Craig D, Wilson P, Drummond MF: Increasing decision-makers’ access to economic evaluations: alternative methods of communicating the information. Int J Technol Assess Health Care. 2008, 24: 151-157.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/12/300/prepub
The authors are grateful to Prof. Dr. Reiner Leidl for valuable comments on earlier versions of this paper. Moreover, they gratefully acknowledge the financial support provided for this research by the Bavarian Health and Food Safety Authority.
The authors declare that they have no competing interests.
AL conducted the searches and the review of economic evaluations, participated in study appraisal, and took the lead in writing the manuscript. JJ initiated the project, developed the guideline, provided methodological supervision throughout the study and contributed to writing the manuscript. RH participated in the development of the guideline and contributed to methodological discussions. All authors read and approved the final manuscript.