Skip to main content
  • Research article
  • Open access
  • Published:

Health care priority setting in Norway a multicriteria decision analysis



Priority setting in population health is increasingly based on explicitly formulated values. The Patients Rights Act of the Norwegian tax-based health service guaranties all citizens health care in case of a severe illness, a proven health benefit, and proportionality between need and treatment. This study compares the values of the country's health policy makers with these three official principles.


In total 34 policy makers participated in a discrete choice experiment, weighting the relative value of six policy criteria. We used multi-variate logistic regression with selection as dependent valuable to derive odds ratios for each criterion. Next, we constructed a composite league table - based on the sum score for the probability of selection - to rank potential interventions in five major disease areas.


The group considered cost effectiveness, large individual benefits and severity of disease as the most important criteria in decision making. Priority interventions are those related to cardiovascular diseases and respiratory diseases. Less attractive interventions rank those related to mental health.


Norwegian policy makers' values are in agreement with principles formulated in national health laws. Multi-criteria decision approaches may provide a tool to support explicit allocation decisions.

Peer Review reports


In all health care systems, choices in the allocation of resources are necessary. Public resources, in both low- and high-income settings, are insufficient to provide all possible services to the entire population at all times. Priority setting in the allocation of health interventions across ranges of health services or target groups becomes inevitably and at best should be explicit[1, 2].

Scarcity of resources and rational priority setting lead to identification of national packages of health services and to explicit reimbursement decisions [3]. In health, policy makers often may have to confront many groups with different and conflicting interests [4]. Rational approaches to guide policy makers become desirable and allow for navigation of complex social processes and to handle multifaceted information. The rise of evidence-based health policy has lead to a wealth of information on the nature and distribution of the disease burden and related societal costs as well on the effectiveness and efficiency of health interventions, and on their budget impact [5]. Existing approaches may concentrate on one or two criteria like safety and effectiveness, or also efficiency [6, 7] with real world decision makers having to handle multidimensional information [2, 8]. So far, priority setting tends to focus primarily on effectiveness and efficiency and less on equity-related social values [9].

Norway has adopted a well-developed comprehensive system in which complex problems and scientific support mechanisms such as health technology assessment and value-based priority-setting criteria were brought together to generate transparent solution [10]. As one of the richest countries it ranks second following the United States in health care spending and it enjoys one of the highest population health levels in the world. (OECD 2011[11]) The key elements of its health policy include provisions of health services for all based on needs regardless of personal income, local and regional accountability, public commitment and political interest in improving the health system [12]. Its health system is described as a decentralized national health service with universal coverage of primary care under the responsibility of the municipalities and with specialized care under national governance.

In 2007 a Council for Quality Improvement and Priority Setting was established "to debate, implement, regulate, and supervise" the national health plan under the Patient Rights Act of 2001. As such, the Norway situation provides a unique opportunity to compare formal health goals to actual policy makers preferences in priority setting.

Our study objective is to assess if policy-makers' actual value preferences agree with those formulated in the legal domain through the Patient Rights Act (PRA). The Act identifies three prioritizing rules: severity of the condition (those most in need must be helped first), expected health outcome, and, lastly, proportionality between need and treatment (with cost-effectiveness analysis providing this information). We used discrete choice experiments (DCE), a methodology that allows simultaneous assessment of multiple policy preferences in multi-criteria decision analysis (MCDA) for policy as we reported earlier in other case studies[9, 1315].


MCDA consists of four steps: (I) identification of policy criteria and metrics, (II) identifying series of alternative package vignettes based on various combinations of policy criteria, (III) measuring performance of alternatives by criteria (this is often done through a DCE questionnaire but can also be completed in various other ways), and (IV) determining the preferred selection through scoring intervention options against those criteria. The last step provides a broad classification and ranking of interventions within a chosen context or specific disease area (a composite league table). Although the league table is not an integral part of the MCDA methods but rather an illustrative application, it is often seen as the most important contribution of a MCDA.

Discrete choice survey and criteria attributes

A DCE is a stated-preference technique that uses an attribute-based survey method for measuring preferences for multiple benefits (utility). The survey elicits respondents' preferences based on their stated preferences in hypothetical and forced choices between two vignettes or, in our study, two sets of specified policy criteria. The choice vignettes are drawn, a priori, from all possible choice sets according to statistical design principles [16]. In stating a preference for one of the two vignettes, the individual is assumed to make trade-offs and choose the alternatives that yield the highest personal preference or benefit or utility [17].

The first step is the identification of context-relevant criteria and their levels. In general, criteria must be relevant to the stated policy choices and usually include some combination of equity-related and/or efficiency-related factors. Here, equity factors may be viewed as related to the distributional impact and often relate to whom in a more vulnerable state receives the expected health benefits like those with severe illness, of a high or low age or low income status. Efficiency factors may refer to achieving the largest impact at the lowest cost, the total beneficiaries and hence the total population impact, or the individual benefits. We used an existing questionnaire, applied in various settings and reported earlier for other countries, including The Netherlands [14, 15, 18]. The criteria (Table 1) are selected to assure completeness, feasibility, and mutual independence [3, 9], as based in the earlier country experiences and review [6, 7].

Table 1 Definition of attributes and levels

The survey questionnaire is detailed in Table 2 with the varying levels of criteria attributes that are grouped and assigned into sets of vignettes. Vignettes contain always the same attributes yet with differing levels across vignettes. A full factorial list, including all possible combination of levels and attribute is often impossible to use, yet not needed. We applied a fractional factorial design considering a sample selection of the possible alternatives. A computer-based software program was used to create choice sets regarding consideration on the properties of orthogonality, level balance, and minimum overlap [2]. Our fractional factorial design in the study includes a sub-set of 32 vignettes, paired in 16 sets of choices.

Table 2 Pair of competing interventions example in the DCE questionnaire scenarios.

The DCE consists of this set of discrete choices. It measures preferences between intervention options by counting the extent to which specified objectives are preferred most [2, 3]. We used conditional logistic regression model in STATA © to analyze all the response data together. The independent variables (the defined policy criteria) are either categorical with two or three levels and the dependent variable is the preferred choice i.e. the selected vignette. In case of a three levels attribute, the young age group is used as the reference category. The weighing results are presented as regression coefficients. These indicate the size of the effect of a criterion on the selection probability of an intervention option fulfilling the criterion [16]. Finally, the calculated sum weight over all criteria for each intervention option leads to ranking of the options in a composite league table based on the probability of selection [14].

Data collection

The DCE survey questionnaires were completed by 34 health professionals selected among all the senior members of the Norwegian Directorate of Health and senior public health academic officials, using a snow-balling technique. The selection of respondents was based on the number of years with relevant experience in actual decision-making, or their familiarity with economic evaluation defined as being actively involved in policy research in this field.

Data analysis

All levels for criteria were qualitative and data are dichotomous choice ('1' represents the option being chosen, '0' where not chosen). The observed sources of utility can be defined as a linear expression (main-effects additive probability model assumption) in which each attribute is weighted by a unique parameter to account for that attribute's marginal utility. Regression coefficients indicate the sign of the effect of a variable on the probability of selection of an intervention. The absolute value of the coefficients denote the relative importance of particular levels of a criterion in comparison to other levels of all other criteria so that the criterion which has higher probability will have more possibility to be used as the important one in selection of the interventions than others [16, 17].

Composite league table

The last phase in MCDA includes the ranking of interventions of interest in the country where the research is performed. In an actual priority-setting context, the aim of the composite league table is to rank-order the interventions to identify those that should be included or excluded from a health initiative (low-income countries) or public reimbursement (high-income settings). We selected the interventions in the Norway context to provide a broad presentation of existing and possible interventions across several disease areas, to demonstrate how MCDA may support rational decisions [7].

In the five major disease burden groups (WHO, 2011[19]) we selected 21 interventions. We consulted the updated National Clinical Guidelines as issued by the Directorate for Health and other sources to find the most appropriate context-given interventions addressing the five disease groups. The 21 interventions addressed neuropsychiatric conditions, cardio-vascular diseases, malignant neoplasms, respiratory diseases, and sensory organ diseases.

We defined a "composite index" (CI) that represents the relative priority of each intervention as a function of their characteristics, based on the criteria weights. The "probability of selection" is estimated for each intervention using the regression model and a rank ordering of all intervention on the basis of this composite index results in a composite league table [15].

Our main effect model uses the following specification:

L o g i t ( P ) = L n ( P / 1 - P ) = β 0 + β i V i + β i V i + + ε (1)

P is the probability of an intervention to be selected as the most qualifying intervention based on the joint inclusion of all relevant criteria and their relative weights. The β0 is the constant term or intercept, the βi + n are the indexed coefficients for each the criteria included in the model, while ε is the unobservable error term. V has the value of '1' in case the criterion is present and '0' when it is absent. The use of this linear additive utility model is based on the assumption of mutual independence of the criteria selected.

Computation of the CI involves several steps. The set of interventions to be prioritized has to be mapped, or rated, in which way they fit the selected policy criteria. This fit is indicated by '1' or'0' to denote levels of the various criteria, so that the matrix elaborated will indicate, e.g. for cost-effectiveness '1' when the intervention is CE and '0' if it is not. To illustrate computation, in Table 3 matrix, Atrial fibrillation intervention is rated as '0' for Disease severity criteria that is "no severe", '1' as providing significant individual benefit, '0' as not indicated for middle age group, '1' as indicated for high age group, '1' as the intervention is cost effective... when the mapping for all criteria is completed, the CI is calculated, for all interventions, based on the main effect additive utility model (formula 1), so that Atrial fibrillation will have as CI: 0*1.08 + 1*1.2638+0* - 0.2004+1*-1.117+1*1.5045+0*0.3714+1*0.3566 = 2.0079. Asthma no control will have as CI:1*1.08+1*1.2638-1*0.2004-1*1.117+1*1.5045+1*0.3714+1*0.3566 = 3.2589. The selection probability for each interventions is calculated, based on the general equation: P = EXP (CI)/1+ EXP (CI), the exponential (EXP) function returns e raised to the nth power, where e = 2.71828183. The selection probability for Atrial fibrillation is computed as:

ε 2.0079 / 1 + ε 2.0079 or 2.71 2.0079 / 1 + ( 2.71 2.0079 ) = 7.4476 / 8.4476 = 0.88.
Table 3 Composite League Table (COPD: Chronic obstructive pulmonary disease, K/cancer)

The research proposal was submitted for ethical approval to the IRS at the Johns Hopkins Bloomberg School of Public Health, and has been waived from requiring any formal ethics approval.


All 34 respondents completed and returned the whole survey questionnaire. Table 4 shows the results of the random effects regression model. A positive sign for a particular level implies a positive impact on the probability of choosing interventions with that level. As mentioned, the absolute values of the regression coefficients indicate their relative importance in priority setting, in that respect, cost-effectiveness, individual benefits and severity of diseases are the most important criteria. Except for middle age group and the willingness-to-subsidize criterion, all coefficients showed to be significant, and their signs had the expected direction.

Table 4 Results of the random effects regression

Table 3 shows the composite league table results. The interventions with the highest priority are those addressing cardio-vascular diseases, and next the respiratory diseases. Those receiving the lowest attractively are related to mental health.


Norwegian policy makers consider cost effectiveness, individual benefits and severity of disease as important criteria for priority setting of interventions. The findings are in line with those that one a priori may expect for a high-income country with universal coverage where a willingness to subsidize is less relevant. While using generic criteria based on other settings, our study confirms that the policy makers are consistent and accurately weigh criteria that the Patients Rights Act prescribes. These are all criteria that patients need to meet to qualify for needed specialized health care and to receive the status of 'entitled patient'.

The results from composite league table can be considered indicative. It provides an insight in intervention priorities they are giving or neglecting to address chronic diseases. Only if cardiovascular diseases and respiratory diseases are properly addressed, mental health conditions - such as depression, alcohol disorders and Alzheimer's disease - may become attractive. This may be in conflict with already existing central government initiatives that aim to promote control of neuropsychiatric conditions, in particular dementia [20].

A limitation in our study is that we did not use well defined and easily quantifiable criteria based on locally formulated standards. Recognizing this, we do not avoid a realistic debate on policy setting in Norway as the criteria selected ex ante are close to the country specific and country-relevant, as well as the levels of attributes. Our present full set of criteria chosen describes the most important generic aspects of a health intervention as also is found in the two large reviews [6, 7]. Also, our findings are comparable to the preferences of Dutch policy makers [9]. These two countries seem to favor efficiency to equity. Additionally, the trends in weights among Dutch health care groups are similar. Moreover, interventions will certainly have different perceived or measured characteristics in other countries. Some of criteria may not be relevant in other countries, yet this is accounted for by the country-specific weights that policy-makers in other settings may give to those criteria.

The second constraint i.e. the assumed independence of the criteria as they are defined in the DCE is important. Our approach is not only based on assumed independence but also assumed comprehensiveness and exclusiveness of the selected criteria, especially in the ranking of interventions. This is reflected and has been tested in the multi-variate regression approach, yet we cannot not exclude other relevant additional criteria in real-life settings. In addition, we also need to acknowledge that this type of MCDA provides a way to account for quantifiable elements of the decision-making process, yet other elements like feasibility, transaction costs, historical or local context, among others, may be difficult to incorporate into the approach [21]. The latter are much more related to implementation as to an informed and explicit priority setting. The present findings are important as they confirm current policies and do not lead to new insights in the approach of the policy makers. So, the study and the matching of criteria provides policymakers with an explicit structure to assess the extent to which current and future investments in health interventions in Norway may serve the country's societal objectives.

Health decision-makers make difficult decisions on the use of public funds either in a context-specific priority setting for reimbursement of interventions, prioritization between a few options within a constrained budget or, in a general policy setting where no specific choice has to be made in a specific resource allocation. At the moment, Norway is given only an average score in the OECD's new health system report [22]. A more formalized and explicit approach that includes methodological guidance, the use of health information and a transparent debate on policy formulation may improve the policy process [23]. Here, to balance policy makers taking ad-hoc or biased decisions, an approach like MCDA, while using a composite ranking method may be a valuable tool[2]. Ghana is a positive example and has successfully used explicit criteria to set its interventions priorities through the Strategic Plan for health 2007-2012 while other very divers countries with (almost) universal access systems continue to develop policy processes that includes both equity and efficiency factors such as in the UK, The Netherlands, Thailand, China, and others [15].


Although this study used general priority criteria, three criteria showed to be relevant to Norwegian policymakers. The existing implementation of appropriate, transparent, and fair national guidelines is in line with the legal, and hence societal values and reflects the Patients Rights of Act. Priority rankings based on the same criteria show how MCDA can be used and may invite a discussion on priority policies across major disease areas. The findings allow for a prioritization based on existing features of the modern health care complexities that policy makers are facing.


  1. Peacock SRD, Mitton C, Donaldson C, Bate A, Murtagh M: Using economics to set pragmatic and ethical priorities. BMJ. 2006, 332 (7539): 482-485. 10.1136/bmj.332.7539.482.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Rob Baltussen LN: Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effectiveness Resource Allocation (CERA). 2006, 4: 14-10.1186/1478-7547-4-14.

    Article  Google Scholar 

  3. Esther w, De Bakker-Grob MR, Karen Gerard: Discrete choice experiments in health economics: a review of the literature. Health Econ. 2012, 2: 145-172.

    Google Scholar 

  4. Paul Scuffham JW, Matthew Taylor, Ruth Saxby: Health system choice-a pilot discrete-choice experiment eliciting the preferences of British and Australian citizens. Appl Health Econ Health Policy. 2010, 8: 89-97. 10.2165/11531170-000000000-00000.

    Article  PubMed  Google Scholar 

  5. Niessen LW GE, Rutten FF: The evidence-based approach in health policy and health care delivery. Soc Sci Med. 2000, 51 (6): 859-869. 10.1016/S0277-9536(00)00066-6.

    Article  PubMed  Google Scholar 

  6. Noorani HZ HD, Boudreau R, Skidmore B: Priority setting for health technology assessments: a systematic review of current practical approaches. Int J Technol Assess Health Care. 2007, 23: 310-315.

    Article  PubMed  Google Scholar 

  7. Sitaporn Youngkong LK, Rob Baltussen: Setting priorities for health interventions in developing countries: a review of empirical studies. Trop Med Int Health. 2009, 14: 930-939. 10.1111/j.1365-3156.2009.02311.x.

    Article  PubMed  Google Scholar 

  8. Ducan Mortimer LS: Is the value of a life or life-year saved context specific? Further evidence from a discrete choice experiment. Cost Effectiveness and Resource Allocation(CERA). 2008, 6: 8-10.1186/1478-7547-6-8.

    Article  Google Scholar 

  9. Baeten Stefan A, Baltussen Rob, Uyl-de Groot Carin A, Bridges John, Niessen Louis W: Incorporating equity-efficiency interactions in cost-effectiveness analysis--three approaches applied to breast cancer control. Value Health. 2010, 13: 573-579. 10.1111/j.1524-4733.2010.00718.x.

    Article  CAS  PubMed  Google Scholar 

  10. Morland Berit, Ringard Anen, Rottingen John-Arne: Supporting tough decisions in Norway: a healthcare system approach. Int J Technol Assess Health Care. 2010, 26: 398-404. 10.1017/S0266462310001030.

    Article  PubMed  Google Scholar 

  11. OECD: OECD. StatExtracts. Health Status. 2011

    Google Scholar 

  12. Health systems in transition: Norway. Edited by: Johnsen J. 2006, Copenhagen: WHO regional Office for Europe on behalf of the European Observatory on health Systems and Policies

    Google Scholar 

  13. Baeten SA BR, Uyl-de Groot C, Bridges JFP, Niessen LW: Reducing Disparities in breast cancer survival - the effect of large-scale screening of the uninsured. Breast J. 2011, 17: 548-549. 10.1111/j.1524-4741.2011.01135.x.

    Article  PubMed  Google Scholar 

  14. Rob baltussen AtA, Koolman X, Shrestha N, Bhattari P, Niessen LW: Priority setting using multiple criteria: should a lung health programme be implemented in Nepal?. Health Policy Plan. 2007, 22: 178-185. 10.1093/heapol/czm010.

    Article  Google Scholar 

  15. Rob Baltussen ES, Dan Chisholm, Moss Aikins: Towards a multi-criteria approach for priority setting: an application to Ghana. Health Econ. 2006, 15: 689-696. 10.1002/hec.1092.

    Article  PubMed  Google Scholar 

  16. Mangham KH, Lindsay J, Barbara McPake: How to do (or not to do)... designing a discrete choice experiment for application in a low-income country. Health Policy Plan. 2009, 24: 151-158. 10.1093/heapol/czn047.

    Article  PubMed  Google Scholar 

  17. Mandy Ryan KG, Mabel Amaya-Amaya: Using Discrete Choice Experiments to Value Health and Health Care. 2008, Dordrecht: Springer, 1

    Book  Google Scholar 

  18. Marc A, Koopmanschap EAS, Xander Koolman: Dear policy maker: have you made up your mind? A discrete choice experiment among policy makers and other health professionals. Int J Technol Assess Health Care. 2010, 26 (2): 198-204. 10.1017/S0266462310000048.

    Article  Google Scholar 

  19. W.H.O. Disease and injury country estimates. Burden of diseases. []

  20. 2011-2015 MeldSt.16: National plan for health and care services (2011-2015). Nasjonal helse- og omsorgsplan. 2011, Oslo: (2011-2015) Helse-OG-Omsorgsdepartementet, 2010-2011, -2015

    Google Scholar 

  21. Kapiriri L, Norheim OF, Martin DK: Fairness and accountability for reasonableness. Do the views of priority setting decision makers differ across health systems and levels of decision making?. Soc Sci Med. 2009, 68: 766-773. 10.1016/j.socscimed.2008.11.011.

    Article  PubMed  Google Scholar 

  22. Kunnskapssenteret NKcfths-: OECD Press release, Oslo Dec 2009. 2009

    Google Scholar 

  23. Colin Green KG: Exploring the social value of health care interventions: a stated preference discrete choice experiment. Health Econ. 2009, 18 (8): 951-976. 10.1002/hec.1414.

    Article  PubMed  Google Scholar 

Pre-publication history

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Thierry Defechereux.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

TD: Drafted the manuscript, carried out data extraction and analysis. FP: designed, conceived and coordinated the study, reviewed the draft. AM: contributed to the extraction, analysis and interpretation of the data, reviewed the draft. SY: contributed to the extraction, analysis and interpretation of the data. GB: contributed to the analysis and interpretation of the data. TH: Drafted the manuscript, collected the data, contributed to the analysis and interpretation of the data. LN: designed and conceived the study, contributed to the analysis and reviews. All authors read and approved the final manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Defechereux, T., Paolucci, F., Mirelman, A. et al. Health care priority setting in Norway a multicriteria decision analysis. BMC Health Serv Res 12, 39 (2012).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: