Clearing up the hazy road from bench to bedside: A framework for integrating the fourth hurdle into translational medicine
© Rogowski et al; licensee BioMed Central Ltd. 2008
Received: 16 February 2008
Accepted: 24 September 2008
Published: 24 September 2008
New products evolving from research and development can only be translated to medical practice on a large scale if they are reimbursed by third-party payers. Yet the decision processes regarding reimbursement are highly complex and internationally heterogeneous. This study develops a process-oriented framework for monitoring these so-called fourth hurdle procedures in the context of product development from bench to bedside. The framework is suitable both for new drugs and other medical technologies.
The study is based on expert interviews and literature searches, as well as an analysis of 47 websites of coverage decision-makers in England, Germany and the USA.
Eight key steps for monitoring fourth hurdle procedures from a company perspective were determined: entering the scope of a healthcare payer; trigger of decision process; assessment; appraisal; setting level of reimbursement; establishing rules for service provision; formal and informal participation; and publication of the decision and supplementary information. Details are given for the English National Institute for Health and Clinical Excellence, the German Federal Joint Committee, Medicare's National and Local Coverage Determinations, and for Blue Cross Blue Shield companies.
Coverage determination decisions for new procedures tend to be less formalized than for novel drugs. The analysis of coverage procedures and requirements shows that the proof of patient benefit is essential. Cost-effectiveness is likely to gain importance in future.
There is a gap between scientific knowledge and daily medical practice. While there have been major scientific breakthroughs, e.g. in the field of genomics or stem cell research, this does not necessarily directly translate into a variety of new treatments available to patients. A term frequently used to describe approaches for bridging this gap is "translational medicine". Translational medicine faces two major obstacles: the first is the translation of basic science discoveries into clinical studies; the second is to translate clinical studies into medical practice . A large body of literature has focused on the first aspect [2, 3]. Yet the second obstacle, which usually depends on coverage by third-party payers, is also essential for the economic success of new products in clinical development.
Coverage determination may prove to be a hurdle as difficult as those encountered in demonstrating the product's efficacy, safety and quality. Due to rapidly increasing healthcare expenditures, numerous countries currently set up institutions that further evaluate new medical technologies after their market approval, before national health services or insurance systems provide coverage . The so-called fourth hurdle set up by third-party payers should thus be considered by those who are involved with earlier stages of medical innovation, e.g. when target conditions and countries are selected and clinical trials are planned.
Looking closely at procedures for coverage determination may demand more attention than a time-constraint clinician or manager can give – the fourth hurdle is highly heterogeneous and subject to frequent reform. Earlier studies have approached the fourth hurdle from a legal or institutional perspective [5, 6]. Yet these approaches are quite complex and do not fit easily within the context of the whole process of translation, frequently represented by value-chain steps from basic research to application in healthcare.
This study develops a process-oriented framework to describe fourth hurdle procedures from the perspective of a life sciences company. The framework fits into value-chain representations of technology development and allows for easy presentation and monitoring of these decision procedures. Guidance is derived on how to prepare in advance for the later stages of translational medicine.
The study is based on expert interviews, as well as a literature search in PubMed, CRD, Econlit and generally via Google. Relevant attributes of coverage decision-making procedures were extracted and consolidated to a limited number of categories, to fulfil the following, conflicting targets to the highest possible degree:
Small number, to be manageable
Conclusive, to describe all items relevant to a life sciences company
Universal, to facilitate an international comparison of procedures
Accessible, to enable an investigation of these items for a potential market.
The framework was developed as part of a research project on coverage of novel tissue-engineering (TE) treatments which investigated coverage and reimbursement processes for Autologous Chondrocyte Implantation (ACI) [7–9]. ACI is a novel TE treatment for cartilage defects, particularly in knee joints. Small samples of normal cartilage incorporating cartilage-producing cells (chondrocytes) are removed from the damaged joint, cultured in special laboratories, and re-implanted into the areas of cartilage damage. While ACI may be an effective and cost-effective treatment, the evidence of its long-term benefit is still weak . ACI is the first tissue-engineering treatment that has been subject to explicit coverage decisions in England, Germany and the USA. Most of the current literature is dedicated to the coverage of novel drugs ([e.g. [4, 11, 12]]), yet innovation in life sciences also encompasses other types of technologies such as procedures applied in hospital. Given the variety of fourth hurdle procedures, the fourth hurdle of these other technologies may not be represented appropriately by procedures aimed at dealing with new pharmaceuticals. The choice of a tissue-engineering procedure as reference case allowed for a framework that is also suitable for other areas of life sciences innovation.
In order to cover various aspects of reimbursement decisions for different types of healthcare funding, institutions in England (tax-based), Germany (social health insurance) and the USA (large share of private health insurance) were chosen. These countries are of outstanding importance to innovative life sciences companies due to their large healthcare expenses. Decisions by the following institutions were selected for comparison: for the English National Health Service, the National Institute for Health and Clinical Excellence (NICE) was selected as the NICE appraisals are relevant for the entire English population and NICE serves as an international reference for fourth hurdle institutions . For Germany, the decisions by the Federal Joint Committee ("Gemeinsamer Bundesausschuss", G-BA) were investigated, which are relevant to approximately 90% of the population who are covered by the statutory health insurance. For the USA, decisions by Medicare were investigated, as Medicare is the largest publicly funded healthcare payer  and its decisions strongly influence those of private payers . Private health insurance and out-of-pocket payments account for more than half of the total of USA healthcare expenditures . As an example of a private payer of significant size, the companies of the Blue Cross Blue Shield Association were included in the analysis. 44 websites of different payers in the USA were searched [for details, see: [8, 9]].
The selected institutions are highly relevant to life sciences companies due to the healthcare budget within the scope of their decisions. They additionally represent very different approaches to coverage determination. For this reason, it is important to understand the coverage procedures of these institutions before proceeding to a wider international comparison.
Institutions for coverage determination are frequently subject to tight regulation and supervision by government and social courts. This analysis is dedicated to decision-making procedures within the institutions; the legal basis and decisions shaping these institutions were excluded from this analysis. Fourth hurdle procedures are indirectly affected by the rigour of licensing decisions which can vary by type of technology and by jurisdiction. Due to its focus on reimbursement decision making, also the procedures of market approval are beyond the scope of this study.
Reimbursement is both highly heterogeneous across countries and quickly developing at the same time. Figure 1 displays a set of descriptors that serve best to characterize coverage decision-making procedures in international comparison. These descriptors can be seen as crucial steps in the value chain between market authorization and diffusion: (1) after market authorization, a company's product enters the scope of a healthcare payer. (2) The decision maker now has to decide whether, at what cost and under which conditions to cover the technology. As there may be different deciding bodies (e.g. on a national or regional level), the decision process needs to be considered relevant by one specific deciding body before this body's decision process starts. (3) The decision maker typically applies some method of assessing the technology under investigation (e.g. formal reviewing the literature or asking for expert opinion). (4) The information provided by the assessment feeds into an appraisal of whether or not to cover a technology should be covered. (5) After granting coverage, a reimbursement rate needs to be fixed. (6) Besides funding a technology's use, the payer may use other options to exert influence on service provision, e.g. by requiring pre-authorization for expensive technologies. (7) Typically, there are different stakeholders (e.g. doctors or patient groups) to the decision who may be involved formally or informally in the decision process. (8) The decision maker may provide information about the decision process or its outcome during different steps of the decision process. There may be cases where some or even all of these steps are taken simultaneously.
Overview of the fourth hurdle for ACI
USA, Blue Cross
Trigger of coverage decisions
All hospital services are covered unless excluded explicitly. Evaluation upon request of entitled parties (no manufacturers)
All ambulatory services are excluded unless included explicitly. Evaluation upon request of entitled parties (no manufacturers)
Technologies with significant health benefits, social/political effects, impact on NHS resources and added value through guidance by NICE
Initiation of National Coverage Determination: Internally by CMS, external request by interested or aggrieved parties. Only services with considerable impact on the program are evaluated
Initiation of Local Coverage Determination: Internally by the contractor in case of need and in the absence of a NCD
Initiation on company-level, e.g. with Anthem internally by Medical Directors; externally by physicians, manufacturers, HTA organizations
G-BA, Commission for hospital services 
G-BA, Commission for ambulatory services 
Independent Appraisal Committee on behalf of NICE 
Contractor's Medical Director 
Mostly committee or Medical Director
Cost-utility model based on all evidence available by contracted technology assessment team; threshold area about £20–30.000/QALY) 
Review of evidence of medical benefit; HTA potentially by Agency for Healthcare Research and Quality; by other HTA institution; or by Medicare Coverage Advisory Committee 
Review of evidence of medical benefit; HTA potentially by external institution 
Review of evidence of medical benefit; HTA potentially by Technology Evaluation Committee or other HTA institution 
Criteria for Appraisal
Expedience, necessity and efficiency
Effectiveness and cost-effectiveness
Reasonable and necessary
Reasonable and necessary
Medically necessary (Approval, evidence for net health benefits, as beneficial as established alternatives, attainable outside investigational settings)
Out-patient: Fee for service
Global budgets for PCTs; DRGs paid by PCTs for hospital services
Management of service provision
e.g. quality assessment of doctors
e.g. requirement of participation in clinical trial
Participation of company
Consideration of written comments
Comments about open issues, HTAs and provisional decision; submission of model, appeal possible
Defined periods for comments of all interested parties
Heterogeneous; contractors are required to permit participation
Heterogeneous; mainly participation of medical experts
Amongst others, current open issues, assessment reports and decisions are available through the internet
Process, assessment and appraisal available through the internet
Current open issues and decisions are available through the internet
Written communication in the jurisdiction of the contractor; also available through the internet
Heterogeneous; medical policies often available through the internet
Besides out-of-pocket payments, healthcare financing can for example be provided by national and local health services, statutory social insurance systems, private insurance companies, employer-based insurance or integrated service delivery systems with insurance function . The relevance of different types of healthcare payers differs substantially across healthcare systems. Seen from an economic perspective, healthcare payments in England are dominated by one central payer. Also in Germany, the sickness funds within the statutory health insurance system dominate health care financing. For the US healthcare sector, a variety of financing organizations have to be taken into account. To capture the whole relevant market, it is important for a life sciences company to consider all relevant payers for the target healthcare system.
To arrive at an estimate of the relevance of a product's market size covered by the healthcare payer, the number of covered individuals and the types of reimbursed services are of interest to a company. As an example, the German statutory health insurance covers about 90% of the German population. In 2007, its total expenses for health services amounted to £144 billion . While the German statutory health insurance covers individuals at all ages, Medicare in the USA mainly covers the elderly, at a total amount of approximately US$300 billion in 2004. To achieve coverage of technologies primarily for young patients, other payers like the BCBS companies would therefore have to be targeted by a life sciences company.
To be able to tailor its product to the purchaser's preferences, it is relevant for a life sciences company to know who is going to make the decision to finance its product. Coverage decisions are frequently associated with national institutions under the spotlights of politics and the media. Yet due to limited capacities of reviewing bodies, only a small portion of novel technologies is subjected to the resource-consuming formal procedures of national bodies. The vast majority of decisions are usually made at the meso-level of regions or healthcare organizations such as primary care trusts in England or providers' "local coverage determinations" for Medicare in the US. Usually, decisions at the meso-level are far less formalized and thus less suitable for systematic analysis of published documents .
Knowing the trigger for a decision-maker's formal evaluation process is, therefore, relevant to determine whether a decision-maker's formal criteria can be expected to be applied at all. Various systems for prioritizing technologies for further investigation can be found across decision makers. Criteria that new technologies have to fulfil to become subject of a formal appraisal process include high-budget impact or disagreement on the technology's effectiveness . In the case of NICE, only technologies with a significant health benefit, impact on other health-related government policies and/or impact on NHS resources are assessed, where the institute is likely to be able to add value by issuing national guidance.
For Germany, primarily the procedure of initiating an appraisal is made explicit: the committee's directives on evaluating technologies only determine the parties authorized to initiate an assessment . Ambulatory treatments are only reimbursed upon explicit acceptance by the Committee – thus, every novel treatment has to pass the committee. The opposite is the case for in-patient services: Here, any novel procedure is covered implicitly within the DRG framework unless it is explicitly excluded by the decision-maker.
Coverage decisions require medical, economic and legal expertise. Thus, they are usually made by interdisciplinary committees. For the German statutory health insurance, decisions are made on behalf of both the Federal Associations of Sickness Fund Physicians or the German Hospital Association and the Federal Associations of Sickness Funds by the G-BA . For the English National Health Service, NICE commissions appraisal committees consisting representatives of relevant stakeholders in the health care system to provide guidance on the technology under investigation . For the decisions on ACI made in the USA, the insurers' medical directors played a major role. Frequently, they rely on the support of an interdisciplinary committee of researchers and other independent experts.
Typically, new technologies undergo some form of assessment of patient benefit or cost-effectiveness prior to the decision. These assessments frequently differ from those used for market approval . In the case of formalized decisions, they often are commissioned out to assessment teams or institutions.
How patient benefit should be established is ambiguous and a topic of persistent controversy, particularly with regard to the use of surrogate endpoints in health-impact measurement and the appropriateness of measurement tools for health-related quality of life . Another point of continual debate is the grading of quality of evidence. Evidence-based medicine advocates the use of high-quality clinical evidence ; however, if studies are available, their results tend to reflect a technology's efficacy, while the coverage decision-makers are interested in its "real world" effectiveness in day-to-day medicine. The emphasis of health technology assessments is usually placed on patient-relevant therapy endpoints compared with the current standard of care – as opposed to the frequently used intermediate endpoints in placebo-controlled trials for market approval. Frequently, the full consideration of patient-relevant health outcomes as, for example, changes in residual life expectancy, is beyond the scope of clinical studies. Therefore, decision-makers have to specify their attitudes towards modelling approaches.
Economic evaluations add the issue of value for money and include the costs of a treatment, which implies the assessment of the cost of the respective treatment strategy as a whole, not only the product price. Depending on the type of outcome compared with the costs, different types of studies can be distinguished: cost-benefit analyses measure benefit in terms of willingness to pay; cost-utility analysis use utility values represented by quality-adjusted life years; and cost-effectiveness analyses compare the additional costs associated with a new treatment with the additional health benefit in terms of clinical endpoints [32, 33]. A variety of guidelines for economic evaluation have been developed ; open issues include, for example, the perspective of the studies or the discount rates for future costs and effects to be applied.
NICE, for example, uses cost-utility models as the state of the art to obtain data for a health technology's effectiveness and cost-effectiveness. Independent academic centres are commissioned to review the published evidence on the technology and synthesize this evidence in decision-analytic models.
The G-BA can commission technology assessments to the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG). This institute also is involved in the development of methods for the evaluation. IQWiG holds a critical attitude towards the use of models and rejected the use of quality-adjusted life years in its statement of methods published in December 2006 . In its statement of "Methods for Assessment of the Relation of Benefits to Costs in the German Statutory Health Care System" published in January 2008, an efficiency frontier approach to economic evaluation was recommended . The methods guidance provides little and partially inconsistent information about the use of decision analytic modelling for evidence synthesis or QALYs for the establishment of effectiveness . Also its revised statement of general methods published in May 2008 does not address the issue of decision-analytic modelling or QALYs and focuses on the establishment of patient benefit on the basis of clinical trials .
For companies in the Blue Cross and Blue Shield Association, the Technology Evaluation Center (TEC) is an important source of health technology assessment methods and reports. The use of formal cost-effectiveness analyses has been advocated in the USA  and standards for their use in drug coverage decisions have been proposed by the American Academy of Managed Care Pharmacy . Still, their current use in coverage decision-making is rare and assessment of new technologies is mainly based on clinical evidence to establish whether a product improves health outcomes .
These assessments feed into an appraisal of the new technology which may to a large extent be determined by social legislation. For example, according to Book Five of the German Social Code, the G-BA is to appraise whether technologies are "expedient, necessary and efficient" . On behalf of NICE, independent appraisal groups are to appraise the technology's effectiveness and cost-effectiveness; guidance for parts of the additional principles and reflections to account for in the appraisal is provided in a document on social value judgements for NICE appraisals [43–45]. Medicare is to assess whether the technologies are reasonable and necessary; and the BCBS companies are to assess medical necessity .
In the case of ACI, the decisions differed across deciding bodies. For example, the G-BA did not reject in-patient treatment with ACI in knee joints so that it is covered by the statutory health insurance . NICE rejected general coverage. However, ACI is being funded as part of a large multi-centre clinical trial to collect further effectiveness data . In the USA, one local coverage determination for Medicare denied coverage of ACI . Among the Blue Cross Blue Shield companies, medical policies regarding ACI differed considerably .
Market access in terms of coverage may not be sufficient to allow for appropriate funding of a novel healthcare technology. The determination of the level of funding provided by third-party payers is frequently a distinct step, which may involve different decision-making bodies. In some cases (e.g. for new, innovative drugs in Germany), the price requested by the manufacturer is reimbursed directly. Alternatively, insurers may enter into price negotiations with manufacturers as occurs for new drugs in the USA. Besides the amount reimbursed, technical requirements for reimbursement need to be established, e.g. new codes for procedures in fee schedules .
Reimbursement can take the form of global budgets, salaries, capitation, case-based payments, fee-for-service or mixed schedules . It has been shown that technology diffusion rates are higher in the case of retrospective fee-for-service reimbursement than for prospective global budgets [50, 51].
In a hospital setting, where products like ACI are typically applied, reimbursement is usually in diagnosis-related groups (DRGs) . These groups have been introduced recently in Germany, as well as in the UK. Here, innovations that are cost-saving from the hospital's perspective have a major competitive advantage. For cost-increasing innovation, the application of DRGs to higher expenses may be time-consuming or may fail despite formal coverage of the procedure.
Reimbursement mechanisms can be explicitly designed to accelerate or block the adoption of certain new and more expensive technologies. An example for drug reimbursement provides reimbursement of reference prices instead of manufacturer prices . Examples of mechanisms to accelerate technology adoption in the case of DRG payments are additional reimbursement components for selected technologies .
Reimbursement of a new healthcare technology may come along with specified rules for service provision or quality controls. In general, approaches to connect financing of healthcare to service provision have been termed "managed care" . Examples of tools of managed care are utilization review or standard treatment protocols [22, 55]. The stronger this connection, the better the payer can exert influence on the diffusion of new technologies [56, 57].
Managed care has played a particularly important role in the USA: most payers offer managed care products. Nevertheless, the number of fully integrated HMO plans has decreased considerably since the end of the nineties ; insurance products with traditional fee-for-service payment are available. A tool used frequently by private health insurance in the USA to control service provision is preauthorization, which is especially common with new therapies like tissue-engineering products.
Coverage of new technologies can also be linked with requirements for collecting further data on the technology's effectiveness or cost-effectiveness. This so-called "coverage with evidence development" (CED) balances the aim of offering patients promising new health technologies with the demands for prudent use of resources and the principled arguments that coverage should be based on solid evidence of clinical effectiveness [19, 20]. Value of information analysis provides tools to substantiate the decision of whether and which further evidence should be collected to confirm coverage decisions . In the case of ACI, CED has been applied by NICE.
Reimbursement decisions are made in the context of a variety of stakeholders' interests . Besides information rights, the major stakeholders may also be involved actively in the decision-making process. The company's formal role in coverage determination procedures may, for example, be the option to initiate a coverage decision, to provide additional information, to participate in public hearings or to appeal within a fixed period.
The decision-making process by NICE is exceptional in terms of stakeholder participation: a manufacturer is offered the status of formal consultee in the appraisal process. This implies that he/she can participate in the "scoping" of the assessment, comment on various documents, submit a report on clinical and cost-effectiveness and appeal against the final appraisal. In decisions by the German G-BA, stakeholder participation beyond doctors and statutory health insurance funds is mainly reduced to comments from medical experts, self-help groups and patient organizations, as well as from umbrella associations of manufacturers.
Usually, national bodies publish decision memorandums on websites or in official government documents. In the case of national decisions, published material can go far beyond the decisions themselves. Frequently, decision bodies are additionally obliged to meet standards of procedural transparency and to publish themes of future consultations or documents from ongoing processes. Information on decisions and their rationale is less accessible for decisions at the micro- or meso-level. As far as they are available, these publications provide valuable information for companies both on how to learn best to fulfil requirements for reimbursement and how to keep trace of their competitors.
The English NICE has high standards regarding transparency in decision-making: not only the appraisals and the evidence they are based on, but also a number of documents in the decision-making process are published online. The German G-BA publishes their decisions and the related health technology assessment reports on their website [see e.g. ]. The Centers for Medicare and Medicaid services established a database, from which both local and national coverage determinations are accessible http://www.cms.hhs.gov/MCD/search.asp.
Table 1 provides an example for a comparative snapshot of the fourth hurdle in Germany, England and the USA. It displays the decision processes including examples of management of services provision for the innovative medical procedure of ACI.
This study presents a simple framework for monitoring fourth hurdle procedures, which consists of eight key steps. Compared with alternative approaches, this framework allows for easy monitoring of these key steps in the process of coverage determination and for easy communication to decision-makers in research and development.
It additionally allows for a comparison of very different processes, e.g. those in English primary care trusts. Even though the majority of coverage decisions are made at this sub-national level, these processes are frequently ignored by the literature.
The framework is best suited to describe formal stages of coverage decisions, based on published documents. Yet usually, health policy decisions are subject to diverse and potentially conflicting political interests [11, 24, 60, 62], and stakeholders like doctors' associations, patient groups or industry representatives may be involved in an informal manner. Frequently, the decisions are, at least to some extent, affected by informal considerations  and based on colloquial rather than scientific evidence . Especially for the stages "Trigger", "Appraisal" and "Participation", studies of fourth hurdle procedures can therefore benefit from an inclusion of both formal and informal elements. A substantial part of the variance of health technology decision-making, and thus the probability of success for companies, will remain unexplained by an analysis of overt documents and decision criteria only.
While, on the one hand, the framework is primarily for a descriptive analysis of fourth hurdle procedures, it can also serve as a starting point for normative analysis. Daniels and Sabin propose four criteria for coverage decision procedures in order to meet the standard of "accountability for reasonableness" : (1) Decisions regarding coverage for new technologies and their rationales must be publicly accessible (publicity condition); (2) These rationales must rest on evidence, reasons, and principles that all fair-minded parties can agree are relevant to deciding how to meet the diverse needs of a covered population under necessary resource constraints (relevance condition); (3) There must be a mechanism for challenge and dispute resolution regarding limit-setting decisions (appeals condition); and (4) There is regulation of the process to ensure that the first three conditions are met (enforcement condition). Applying the framework presented in this study to coverage decision processes additionally delivers information on their accountability for reasonableness, because it includes descriptive information for criteria (1)–(3) as proposed by Daniels and Sabin.
Relation of process and outcome
The coverage decisions for the reference case study are heterogeneous: Coverage is provided by the German statutory health insurance for hospital care and by a number of private payers in the USA if certain criteria are met. Coverage was declined by NICE (yet the development of further evidence is supported), as well as by the one Medicare local coverage determination.
One major reason for this heterogeneity may be the ambiguous evidence both for the effectiveness and cost-effectiveness of ACI: there are studies that suggest ACI is effective, but the long-term effectiveness and cost-effectiveness remain to be established . The differences may also be explained partly by the payers' scope: as a major potential benefit of ACI is presumed to be sustainability as compared with implants, it is of higher relevance for younger patients than for individuals over the age of 65. Therefore, Medicare would be more likely to decline coverage, than would benefit schemes for younger patients.
Interestingly, the German G-BA arrived at different conclusions for different conditions: while coverage for ACI in finger and shoulder joints was denied, the coverage decision for ACI in knee joints was postponed until 2014, with no conclusive evidence of long-term effectiveness for either condition. The explanatory statement justified this decision according to anticipated new evidence, particularly from studies in England. This can be taken as an example of the substantial amount of unexplained variance in coverage decisions, yet may also partially be explained by cooperation among coverage decision-makers; apparently, the G-BA is waiting for the results of further clinical studies conducted on behalf of NICE.
How to pave the way from bench to bedside
To pave the way for a research project from bench to bedside, project managers should prepare for the stages through which a novel technology will have to pass before being applied in healthcare. This framework provides background information and can help identifying important issues to address. These include, for example, which decision procedure is likely to be triggered by the product. If formal decisions on a macro-level are likely, clinical trials should be designed to include the endpoints and potentially the treatment alternatives requested by the relevant fourth hurdle processes.
The level of detail in which these questions can be answered will depend on the stage of project development . Early in research, it may be sufficient to do an explorative search of HTA reports for potential target conditions. This provides a first overview of potential medical need, practice patterns and competitors. Closer to the clinical trial stage, it may be worthwhile to develop early health technology assessments. A number of analytical tools are available for purposes like early market assessment, R&D portfolio management and first estimations of pricing and reimbursement scenarios. Nevertheless, high uncertainty of clinical and economic data in early stages restricts the validity of modelling results [67, 68]. As soon as sufficient clinical data are available, formal health economic models can be used to establish information needs for further clinical trials or scenarios for cost-effective pricing strategies.
Limitations of this analysis
Any overview of coverage decisions can only be a snap-shot, as institutions and requirements for coverage of new medical technologies are subject to frequent, considerable changes. This is illustrated by recent reforms in all countries included in this review, e.g. the introduction of formal cost-effectiveness analysis for novel drugs in Germany which was passed in parliament in February 2007. In June 2008, the methods of cost-effectiveness analysis were still subject to debate . Therefore, the results presented in this article's case study, as well as for any company's target health technologies, warrant regular updating.
The US healthcare system is highly heterogeneous and hosts a multitude of payers, many of which have different processes and requirements. Even though the case study presented above investigated major players, findings can hardly be generalized.
Besides the processes described in Table 1 of this study, further decision processes exist – in the case of Germany for example for ambulatory drug reimbursement or dental care; in England for example the procedures within primary care trusts. Even if the framework is generic, the overview in Table 1 had to focus on those processes which made decisions about ACI. It therefore does not provide a conclusive overview but an example of fourth hurdle processes.
The EMEA and the FDA have different practices for licensing [69, 70], and for other technologies like medical products other procedures and criteria for market approval exist . More rigorous licensing practices may filter out technologies that otherwise have to be assessed by fourth hurdle institutions. Therefore, these procedures which are not in the scope of this study have an influence on procedures for coverage and reimbursement.
In general, the procedures involved in the translation of clinical research from bench to bedside are internationally very heterogeneous and complex. Therefore, any simple framework can be claimed to lead to over-simplification and there are research questions for which the framework is not the best choice. For a detailed institutional analysis of coverage decision-makers, the framework developed by Hutton and colleagues would be better recommended because it can provide more precise answers . The framework also has its limitations if rather than one specific technology, an overview of coverage of very different types of technologies, e.g. within one country, is to be provided. Within this process-oriented framework, these decisions would all be described as a multitude of single decision processes. This might jeopardize the framework's simplicity. It is, therefore, better suited for a technology-specific description of the fourth hurdle – which corresponds with the need of a project manager to have the technology life cycle of a single project in mind, rather than a country's system for coverage determination. To describe entitlements to health benefits for a country, an approach like the one chosen by the Health Basket Project may be more appropriate .
The diffusion of health technology is a dynamic process between patients, physicians, hospitals, healthcare systems and technology. Apart from a product's effectiveness or cost-effectiveness, important impact factors in this process include purchasing power in the target country  or commercial marketing efforts [72, 73]. Some of these impact factors will also appear in an application of this framework, e.g. as informal decision criteria (e.g. affordability) or informal participation (e.g. influence by the industry). Nevertheless, researchers interested in that area should draw upon models and approaches developed in the literature on health technology diffusion [50, 72–74].
Implications for researchers and life sciences companies
All over the world coverage decision-makers are developing procedures to prioritize healthcare spending. Project managers involved in the translation of research projects from bench to bedside should keep track of the relevant fourth hurdle procedures in their target countries as part of their marketing activities. The framework presented above provides a tool to describe and compare these procedures in a transparent and manageable manner.
The focus of a decision-maker's interest is the evidence-based establishment of a technology's effectiveness in terms of patient-relevant endpoints. Of the three countries under investigation in the case study, the use of formal cost-effectiveness analysis was to a large extent restricted to NICE in England – yet due to increasing need for prioritization as a result of escalating healthcare costs, the high international perception of NICE and the evolving use of formal cost-effectiveness analysis for drug treatments [4, 11, 39, 75], it can be assumed that its importance will increase in future for other healthcare technologies.
Life sciences companies should include final health outcomes in their clinical trials for market approval, to provide information on patient benefit in order to achieve coverage. They should additionally collect resource consumption data, so that they can provide health economic evidence if required.
Implications for policy-makers
As shown in Figure 2, project managers are confronted with a variety of heterogeneous fourth hurdle procedures that lead to different outcomes. To enhance translational medicine, it would be desirable to increase the consistency of fourth hurdle procedures across payers and healthcare systems.
It has been reported that the criteria for establishment of effectiveness and value for money are not always applied consistently [15, 76]: decisions may be inconsistent with stated requirements, as for ACI-coverage in Germany, or they may be inconsistent between different deciding bodies, as for ACI coverage by Blue Cross Blue Shield companies in the USA. It is important that efforts to standardize fourth hurdle procedures do enforce, rather than jeopardize, their accountability for reasonableness , and lead to the adoption of those health services that are most desired by patients and the population.
To enhance the translation of research findings, it would be desirable if institutions for research funding would increase awareness of later stages in the project development process. This could be conducted, for example, if as a part of advertisements for research opportunities, information on targeted markets and potential reimbursement strategies was requested.
Implications for further research
There are a variety of open research questions on the different process steps. The framework presented here can assist researchers in identifying and distinguishing different areas of research at the important intersection of translational medicine and health services research. Two major areas of further research on the whole process of translation should be addressed: first, fourth hurdle institutions have been subject to continual reform. Thus, the framework presented above should be applied on a regular basis to keep track of changing conditions of market entry and diffusion. The framework was applied only to one technology and a limited number of decision-makers in three countries. It would, therefore, be of additional interest to include further decision-makers and reference technologies in future analysis. Second, coverage is a crucial target for the marketing activities of life sciences companies. It would be of value to develop further guidance on how requirements for coverage can be integrated into innovation management in a more systematic manner. Given the escalating healthcare costs in industrialized countries and the rising importance of fourth hurdle procedures to prioritize healthcare resources , it can be expected that the need for tools and guidance is likely to grow in the future.
We are grateful to Mike Drummond, Katharina Fischer, Reiner Leidl and Jonas Schreyögg for helpful comments on previous versions of this manuscript.
- Sung NS, Crowley WF, Genel M, Salber P, Sandy L, Sherwood LM, Johnson SB, Catanese V, Tilson H, Getz K, et al: Central challenges facing the national clinical research enterprise. Jama. 2003, 289 (10): 1278-1287. 10.1001/jama.289.10.1278.View ArticlePubMedGoogle Scholar
- Plebani M, Marincola FM: Research translation: a new frontier for clinical laboratories. Clin Chem Lab Med. 2006, 44 (11): 1303-1312. 10.1515/CCLM.2006.238.View ArticlePubMedGoogle Scholar
- Wehling M: Translational medicine: can it really facilitate the transition of research "from bench to bedside"?. Eur J Clin Pharmacol. 2006, 62 (2): 91-95. 10.1007/s00228-005-0060-4.View ArticlePubMedGoogle Scholar
- Taylor RS, Drummond MF, Salkeld G, Sullivan SD: Inclusion of cost effectiveness in licensing requirements of new drugs: the fourth hurdle. BMJ. 2004, 329 (7472): 972-975. 10.1136/bmj.329.7472.972.View ArticlePubMedPubMed CentralGoogle Scholar
- Schreyögg J, Stargardt T, Velasco-Garrido M, Busse R: Defining the "Health Benefit Basket" in nine European countries. Evidence from the European Union Health BASKET Project. Eur J Health Econ. 2005, Suppl: 2-10. 10.1007/s10198-005-0312-3.View ArticlePubMedGoogle Scholar
- Hutton J, McGrath C, Frybourg JM, Tremblay M, Bramley-Harker E, Henshall C: Framework for describing and classifying decision-making systems using technology assessment to determine the reimbursement of health technologies (fourth hurdle systems). Int J Technol Assess Health Care. 2006, 22 (1): 10-18. 10.1017/S0266462306050781.View ArticlePubMedGoogle Scholar
- Rogowski W, John J, Langer A, Hartz S, Leidl R: Beitrag des GSF-Instituts für Gesundheitsökonomie und Management im Gesundheitswesen. Regenerationstechnologien für Medizin und Biologie – Beiträge für ein strategisches Förderkonzept (details available from the authors upon request). 2007, Berlin: Bundesministerium für Bildung und ForschungGoogle Scholar
- Hartz S, Rogowski W: Entscheidungsprozesse über die Kostenerstattung für neue medizinische Verfahren in den USA. TEIL 1: Medicare. Gesundheitsökonomie und Qualitätsmanagement.Google Scholar
- Hartz S, Rogowski W: Entscheidungsprozesse über die Kostenerstattung für neue medizinische Verfahren in den USA. TEIL 2: Private Versicherungsträger. Gesundheitsökonomie und Qualitätsmanagement.Google Scholar
- Clar C, Cummins E, McIntyre L, Thomas S, Lamb J, Bain L, Jobanputra P, Waugh N: Clinical and cost-effectiveness of autologous chondrocyte implantation for cartilage defects in knee joints: systematic review and economic evaluation. Health Technol Assess. 2005, 9 (47): iii-iv. ix-x, 1–82.View ArticlePubMedGoogle Scholar
- Mossialos E, Mrazek M, Walley T: Regulating pharmaceuticals in Europe: striving for efficiency, equity and quality. 2004, Maidenhead: Open Univ. PressGoogle Scholar
- Wasem J, Greß S, Niebuhr D: Marktzugang und Preisbildung auf Arzneimittelmärkten im internationalen Vergleich. Diskussionsbeiträge aus dem Fachbereich Wirtschaftswissenschaften Universität Duisburg-Essen, Campus Essen. 2005, Essen: Univ. Duisburg-Essen Standort Essen Fachbereich WirtschaftswissenschaftGoogle Scholar
- Rawlins MD: 5 NICE years. Lancet. 2005, 365 (9462): 904-908. 10.1016/S0140-6736(05)71050-3.View ArticlePubMedGoogle Scholar
- A DATA BOOK. Healthcare Spending and the Medicare Program. [http://www.medpac.gov/publications/congressional_reports/Jun06DataBook_Entire_report.pdf]
- Niebuhr D, Rothgang H, Wasem J, Greß S: Die Bestimmung des Leistungskataloges in der gesetzlichen Krankenversicherung.Verfahren und Kriterien zur Konkretisierung des Leistungskatalogs in der gesetzlichen Krankenversicherung vor dem Hintergrund internationaler Erfahrungen. 2004, Düsseldorf: Hans-Böckler-Stiftung, 2.Google Scholar
- National Center for Health Statistics: Health, United States, 2005. With Chartbook on Trends in the Health of Americans. 2005, Hyattsville, Maryland: U.S. Department of Health and Human Services, National Center for Health StatisticsGoogle Scholar
- Demonaco HJ, Ali A, Hippel E: The major role of clinicians in the discovery of off-label drug therapies. Pharmacotherapy. 2006, 26 (3): 323-332. 10.1592/phco.26.3.323.View ArticlePubMedGoogle Scholar
- Pigeot I, Windeler J: [Post-marketing surveillance studies]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2005, 48 (5): 580-585. 10.1007/s00103-005-1033-6.View ArticlePubMedGoogle Scholar
- Tunis SR, Pearson SD: Coverage options for promising technologies: Medicare's 'coverage with evidence development'. Health Aff (Millwood). 2006, 25 (5): 1218-1230. 10.1377/hlthaff.25.5.1218.View ArticleGoogle Scholar
- Hutton J, Trueman P, Henshall C: Coverage with evidence development: an examination of conceptual and policy issues. Int J Technol Assess Health Care. 2007, 23 (4): 425-432. 10.1017/S0266462307070651.View ArticlePubMedGoogle Scholar
- Carino T, Williams RD, Colbert AM, Bridger P: Medicare's coverage of colorectal cancer drugs: a case study in evidence development and policy. Health Aff (Millwood). 2006, 25 (5): 1231-1239. 10.1377/hlthaff.25.5.1231.View ArticleGoogle Scholar
- Kutzin J: A descriptive framework for country-level analysis of health care financing arrangements. Health Policy. 2001, 56 (3): 171-204. 10.1016/S0168-8510(00)00149-4.View ArticlePubMedGoogle Scholar
- Kennzahlen und Faustformeln zur gesetzlichen Krankenversicherung. [http://www.bmg.bund.de/cln_117/nn_1193098/SharedDocs/Downloads/DE/Statistiken/Gesetzliche-Krankenversicherung/Kennzahlen-und-Faustformeln/Kennzahlen-und-Faustformeln.html]
- Wirtz V, Cribb A, Barber N: Reimbursement decisions in health policy – extending our understanding of the elements of decision-making. Health Policy. 2005, 73 (3): 330-338. 10.1016/j.healthpol.2004.12.002.View ArticlePubMedGoogle Scholar
- Noorani HZ, Husereau DR, Boudreau R, Skidmore B: Priority setting for health technology assessments: a systematic review of current practical approaches. Int J Technol Assess Health Care. 2007, 23 (3): 310-315. 10.1017/S026646230707050X.View ArticlePubMedGoogle Scholar
- Verfahrensordnung des gemeinsamen Bundesausschusses. [http://www.g-ba.de/downloads/62-492-83/VerfO_2006-04-18.pdf]
- Busse R, Riesberg A: Health Care Systems in Transition: Germany. 2004, Copenhagen: WHO Regional Office for Europe on behalf of the European Observatory on Health Systems and Policies, 6.Google Scholar
- Guide to the Methods of technology appraisal. [http://www.nice.org.uk/aboutnice/howwework/devnicetech/technologyappraisalprocessguides/guidetothemethodsoftechnologyappraisal.jsp]
- Papanicolaou S, Sykes D, Mossialos E: EMEA and the evaluation of health-related quality of life data in the drug regulatory process. Int J Technol Assess Health Care. 2004, 20 (3): 311-324. 10.1017/S0266462304001138.View ArticlePubMedGoogle Scholar
- Sawicki PT: Patientenrelevante Endpunkte – Stand der Diskussion im Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG). Dtsch Med Wochenschr. 2006, 131 (Suppl 1): S16-20. 10.1055/s-2006-941735.View ArticlePubMedGoogle Scholar
- Moher D, Schulz KF, Altman D: The CONSORT Statement: revised recommendations for improving the quality of reports of parallel-group randomized trials 2001. Explore (NY). 2005, 1 (1): 40-45.View ArticleGoogle Scholar
- Drummond MF: Methods for the economic evaluation of health care programmes. 2005, Oxford: Oxford Univ. Press, 3Google Scholar
- Trueman P, Drummond M, Hutton J: Developing guidance for budget impact analysis. Pharmacoeconomics. 2001, 19 (6): 609-621. 10.2165/00019053-200119060-00001.View ArticlePubMedGoogle Scholar
- Hjelmgren J, Berggren F, Andersson F: Health economic guidelines – similarities, differences and some implications. Value Health. 2001, 4 (3): 225-250. 10.1046/j.1524-4733.2001.43040.x.View ArticlePubMedGoogle Scholar
- Methoden. Version 2.0 vom 19.12.2006. [http://www.iqwig.de/download/Methoden_IQWiG_V-2-0.pdf]
- Methods for Assessment of the Relation of Benefits to Costs in the German Statutory Health Care System. [http://www.iqwig.de/download/08-01-24_Methods_of_the_Relation_of_Benefits_to_Costs_Version_1_0.pdf]
- Krauth C, John J, Aidelsburger P, Brüggenjürgen B, Hansmeier T, Hessel F, Kohlmann T, Moock J, Rothgang H, Schweikert B, et al: [Comments by the Working Group for Methods of Economic Evaluation in Health Care (AG MEG) to IQWiG's Draft Guidelines "Methods for Assessment of the Relation of Benefits to Costs in the German Statutory Health Care System"]. Gesundheitswesen. 2008, 70 (6): e1-16. 10.1055/s-2008-1077059.View ArticlePubMedGoogle Scholar
- Allgemeine Methoden V. 3.0. [http://www.iqwig.de/download/IQWiG_Methoden_Version_3_0.pdf]
- Neumann PJ, Palmer JA, Daniels N, Quigley K, Gold MR, Chao S: A strategic plan for integrating cost-effectiveness analysis into the US healthcare system. Am J Manag Care. 2008, 14 (4): 185-188.PubMedGoogle Scholar
- Sullivan SD, Lyles A, Luce B, Grigar J: AMCP Guidance for Submission of Clinical and Economic Evaluation Data to Support Formulary Listing in U.S. Health Plans and Pharmacy Benefits Management Organizations. Journal of Managed Care Pharmacy. 2001, 7 (4): 272-282.View ArticleGoogle Scholar
- Garber AM: Cost-effectiveness and evidence evaluation as criteria for coverage policy. Health Aff (Millwood). 2004, W4-284-296. Suppl Web ExclusivesGoogle Scholar
- Busse R, Stargardt T, Schreyögg J: Determining the "Health Benefit Basket" of the Statutory Health Insurance scheme in Germany: methodologies and criteria. Eur J Health Econ. 2005, Suppl: 30-36. 10.1007/s10198-005-0316-z.View ArticlePubMedGoogle Scholar
- Social value judgements. Principles for the development of NICE guidance. [http://www.nice.org.uk/media/873/2F/SocialValueJudgementsDec05.pdf]
- Culyer A: NICE's use of cost effectiveness as an exemplar of a deliberative process. Health Economics, Policy and Law. 2006, 1 (3): 299-318. 10.1017/S1744133106004026.View ArticleGoogle Scholar
- Rawlins MD, Culyer AJ: National Institute for Clinical Excellence and its value judgments. BMJ. 2004, 329 (7459): 224-227. 10.1136/bmj.329.7459.224.View ArticlePubMedPubMed CentralGoogle Scholar
- Garber AM: Evidence-based coverage policy. Health Aff (Millwood). 2001, 20 (5): 62-82. 10.1377/hlthaff.20.5.62.View ArticleGoogle Scholar
- Gemeinsamer Bundesausschuss: Bekanntmachung eines Beschlusses des Gemeinsamen Bundesausschusses über Maßnahmen zur Qualitätssicherung bei autologer Chondrozytenimplantation am Kniegelenk. Bundesanzeiger. 2007, 6979-131Google Scholar
- The use of autologous chondrocyte implantation for the treatment of cartilage defects in knee joints. Technology Apraisal 89. [http://www.nice.org.uk/nicemedia/pdf/TA089guidance.pdf]
- Casto AB, Layman E: Principles of Healthcare Reimbursement. 2006, Chicago, Illinois, USA: American Health Information Management AssocationGoogle Scholar
- Oh EH, Imanaka Y, Evans E: Determinants of the diffusion of computed tomography and magnetic resonance imaging. Int J Technol Assess Health Care. 2005, 21 (1): 73-80. 10.1017/S0266462305050099.View ArticlePubMedGoogle Scholar
- Bantle R: Determinanten der Innovation und Diffusion des medizinisch-technischen Fortschritts. 1996, Bayreuth: Verl. PCOGoogle Scholar
- Lungen M, Lauterbach KW: [Applicability of diagnostic related groups (DRG) in an international comparison]. Chirurg. 2000, 71 (10): 1288-1295. 10.1007/s001040051218.View ArticlePubMedGoogle Scholar
- Ess SM, Schneeweiss S, Szucs TD: European healthcare policies for controlling drug expenditure. Pharmacoeconomics. 2003, 21 (2): 89-103. 10.2165/00019053-200321020-00002.View ArticlePubMedGoogle Scholar
- Schreyögg J, Stargardt T, Tiemann O, Busse R: Methods to determine reimbursement rates for diagnosis related groups (DRG): a comparison of nine European countries. Health Care Manag Sci. 2006, 9 (3): 215-223. 10.1007/s10729-006-9040-1.View ArticlePubMedGoogle Scholar
- Amelung VE, Schumacher H, Amelung A: Managed care: neue Wege im Gesundheitsmanagement; mit 12 Fallstudien aus den USA, der Schweiz und Deutschland. 3., vollst. überarb. und erw. Aufl edition. 2004, Wiesbaden: Gabler u.aView ArticleGoogle Scholar
- Pearson SD, Miller FG, Emanuel EJ: Medicare's Requirement for Research Participation as a Condition for Coverage: Is it ethical?. Jama. 2006, 296 (8): 988-991. 10.1001/jama.296.8.988.View ArticlePubMedGoogle Scholar
- Busse R: Relevance of health technology assessment for benefit decisions in Germany now and in the future. Gesundh Okon Qual Manage. 2005Google Scholar
- Cooper PF, Simon KI, Vistnes J: A closer look at the managed care backlash. Med Care. 2006, 44 (5 Suppl): I4-11.PubMedGoogle Scholar
- Claxton K, Sculpher M, Drummond M: A rational framework for decision making by the National Institute For Clinical Excellence (NICE). Lancet. 2002, 360 (9334): 711-715. 10.1016/S0140-6736(02)09832-X.View ArticlePubMedGoogle Scholar
- McGuire A, Drummond M, Rutten F: Reimbursement of pharmaceuticals in the European Union. Regulating pharmaceuticals in Europe: striving for efficiency, equity and quality. Edited by: Mossialos E, Mrazek M, Walley T. 2004, Maidenhead: Open Univ. Press, 131-143.Google Scholar
- Autologe Chondrozytenimplantation (ACI). [http://www.g-ba.de/downloads/40-268-262/HTA-ACI.pdf]
- Chinitz D: Health technology assessment in four countries: response from political science. Int J Technol Assess Health Care. 2004, 20 (1): 55-60. 10.1017/S0266462304000789.View ArticlePubMedGoogle Scholar
- Sassi F: Setting priorities for the evaluation of health interventions: when theory does not meet practice. Health Policy. 2003, 63 (2): 141-154. 10.1016/S0168-8510(02)00061-1.View ArticlePubMedGoogle Scholar
- Weighting up the evidence. [http://www.chsrf.ca/other_documents/pdf/weighing_up_the_evidence_e.pdf]
- Daniels N, Sabin J: The ethics of accountability in managed care reform. Health Aff (Millwood). 1998, 17 (5): 50-64. 10.1377/hlthaff.17.5.50.View ArticleGoogle Scholar
- Rogowski W: Current impact of gene technology on healthcare A map of economic assessments. Health Policy. 2007, 80: 340-357. 10.1016/j.healthpol.2006.03.009.View ArticlePubMedGoogle Scholar
- Hartz S, John J: The contribution of economic evaluation to decision-making in early phases of product development: a methodological and empirical review. Int J Technol Assess Health Care.Google Scholar
- Hartz S, John J: Public health policy decisions on medical innovations: What role can early economic evaluation play?. Health Policy. 2008Google Scholar
- Permanand G, Mossialos E, McKee M: Regulating medicines in Europe: the European Medicines Agency, marketing authorisation, transparency and pharmacovigilance. Clin Med. 2006, 6 (1): 87-90.View ArticleGoogle Scholar
- Redmond K: The US and European regulatory systems: a comparison. J Ambul Care Manage. 2004, 27 (2): 105-114.View ArticlePubMedGoogle Scholar
- Schwarz JA: [Regulatory aspects of clinical trials for medical devices and in vitro diagnostics]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2005, 48 (5): 556-561. 10.1007/s00103-005-1036-3.View ArticlePubMedGoogle Scholar
- Booth-Clibborn N, Packer C, Stevens A: Health technology diffusion rates. Statins, coronary stents, and MRI in England. Int J Technol Assess Health Care. 2000, 16 (3): 781-786. 10.1017/S0266462300102053.View ArticlePubMedGoogle Scholar
- Berndt ER, Bhattacharjya A, Mishol DN, Arcelus A, Lasky T: An analysis of the diffusion of new antidepressants: variety, quality, and marketing efforts. J Ment Health Policy Econ. 2002, 5 (1): 3-19.PubMedGoogle Scholar
- Hashimoto H, Noguchi H, Heidenreich P, Saynina O, Moreland A, Miyazaki S, Ikeda S, Kaneko Y, Ikegami N: The diffusion of medical technology, local conditions, and technology re-invention: a comparative case study on coronary stenting. Health Policy. 2006, 79 (2–3): 221-230. 10.1016/j.healthpol.2006.01.005.View ArticlePubMedGoogle Scholar
- Laupacis A: Incorporating economic evaluations into decision-making: the Ontario experience. Med Care. 2005, 43 (7 Suppl): 15-19.PubMedGoogle Scholar
- Huang AJ, Gemperli MP, Bergthold L, Singer SS, Garber A: Health plans' coverage determinations for technology-based interventions: the case of electrical bone growth stimulation. Am J Manag Care. 2004, 10 (12): 957-962.PubMedGoogle Scholar
- Daniels N, Sabin J: The ethics of accountability in managed care reform: recent efforts at reforming managed care practices have one thing in common; a call for accountability to consumers. 1998Google Scholar
- Centers for Medicare and Medicaid Services: Federal Register Notice: Revised Process for Making Medicare National Coverage Determinations. Federal Register. 2003, 86 (187).Google Scholar
- Devlin N, Parkin D: Does NICE have a cost-effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 2004, 13 (5): 437-452. 10.1002/hec.864.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/194/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.