Skip to main content
  • Research article
  • Open access
  • Published:

Evaluating an integrated care pathway for frail elderly patients in Norway using multi-criteria decision analysis



To provide value-based care for patients with multi-morbidity, innovative integrated care programmes and comprehensive evaluations of such programmes are required. In Norway, a new programme called “Holistic Continuity of Patient Care” (HCPC) addresses the issue of multi-morbidity by providing integrated care within learning networks for frail elderly patients who receive municipal home care services or a short-term stay in a nursing home. This study conducts a multi-criteria decision analysis (MCDA) to evaluate whether the HCPC programme performs better on a large set of outcomes corresponding to the ‘triple aim’ compared to usual care.


Prospective longitudinal survey data were collected at baseline and follow-up after 6-months. The assessment of HCPC was implemented by a novel MCDA framework. The relative weights of importance of the outcomes used in the MCDA were obtained from a discrete choice experiment among five different groups of stakeholders. The performance score was estimated using a quasi-experimental design and linear mixed methods. Performance scores were standardized and multiplied by their weights of importance to obtain the overall MCDA value by stakeholder group.


At baseline in the HCPC and usual care groups, respectively, 120 and 89 patients responded, of whom 87 and 41 responded at follow-up. The average age at baseline was 80.0 years for HCPC and 83.6 for usual care. Matching reduced the standardized differences between the groups for patient background characteristics and outcome variables. The MCDA results indicated that HCPC was preferred to usual care irrespective of stakeholders. The better performance of HCPC was mostly driven by improvements in enjoyment of life, psychological well-being, and social relationships and participation. Results were consistent with sensitivity analyses using Monte Carlo simulation.


Frail elderly with multi-morbidity represent complex health problems at large costs for society in terms of health- and social care. This study is a novel contribution to assessing and understanding HCPC programme performance respecting the multi-dimensionality of desired outcomes. Integrated care programmes like HCPC may improve well-being of patients, be cost-saving, and contribute to the pursuit of evidence based gradual reforms in the care of frail elderly.

Peer Review reports


The global phenomenon of increasing life expectancy and proportion of elderly [1, 2] is also seen in Norway. The 2020 national population projections in Norway predict more elderly people in the population than previous projections. The share of persons aged 65 years or over will increase from 18 to 30% by 2070 [3]. Even though general health status of the elderly population in Norway has improved (for example, better self-reported health, being more physically active, and having less anxiety and depression), the demand for health care has not decreased during the last twenty years. There is little change in the number of people living with chronic disease or requiring assistance in activities of daily living [4] and resources spent in elderly care have increased [5]. The Organisation for Economic Co-operation and Development (OECD) projections predict that budgetary pressures in the coming decades are likely to come mainly from rising long-term care expenditure due to population ageing [6].

In Norway, hospitals are state-owned enterprises governed by appointed administrative boards. Municipalities are responsible for providing primary care and social services. Well-coordinated healthcare programmes can help the elderly with chronic conditions smoothly navigate the health and social care system. Within this endeavour, in 2012 the Norwegian Care Coordination Reform [7] was launched. An important intention of the reform was to develop coordinated clinical pathways across primary care and specialist care. The three primary objectives of the Coordination Reform are: (i) a more cohesive and coordinated approach to health and care services; (ii) a greater proportion of health and care services to be provided in the local communities; (iii) greater focus on preventative measures and improving public health.

Through the Norwegian Care Coordination Reform, the municipalities were given increased responsibility for community-based treatment, care and rehabilitation, to substitute specialized care by more affordable primary care [8]. A key challenge which the Care Coordination Reform intends to address, is that the common disease-specific approach in specialist care suffers from fragmentation and lack of person-centeredness when applied to people with multi-morbidity [9, 10]. As a result, generic clinical pathways for patients with multiple chronic diseases and frail elderly were developed, based on the involved municipalities’ experiences, as well as empirical research. Based on a previous model in the municipality of Trondheim by Røsstad and Grimsmo [11], in 2013 the Norwegian Ministry of Health and Care Services, the Norwegian Directorate of Health, the Norwegian Institute of Public Health and the Norwegian Association of Local and Regional Authorities developed generic clinical pathways for patients with multiple chronic diseases and frail elderly through the programme Learning network of “Holistic Continuity of Patient Care” (HCPC) [12]. Learning networks as a method is based on Breakthrough Series, developed in the USA in 1995 [13], in which teams from different health services gather in collaboratives for learning and improvement.Footnote 1 The HCPC programme aims at improving patient pathways for the frail elderly by:

  1. i)

    Moving attention from “what is the matter with you” to “what is important to you” so as to strengthen the service receiver’s role and contribute to equity and empowerment of the patient.

  2. ii)

    More systematic collaboration between municipalities and hospitals.

  3. iii)

    Follow-up of patient pathways using standardised measures.

The programme is targeted at frail elderly people living at home that have recent functional deterioration requiring additional municipal care services. The program involves early assessment, patient-centered follow-up, early involvement of the patient’s general practitioner (GP), and assigning a designated primary care contact.

There is a strong need for integrated care models [15, 16] for elderly people with multi-morbidity, as well as evaluations of such interventions and measures, in particular regarding the meso organisational and macro system-level care integration strategies [17,18,19,20]. Integrated care models for people with multi-morbidity are complex multi-faceted interventions that aim to improve a wide range of outcomes, often referred to as the Triple Aim (i.e. improving health/wellbeing, experience with care, costs [21]. In order to evaluate such models, a broad evaluation framework is required, in which a wide range of outcome measures are included. The current evidence of integrated care for frail elderly is inconsistent; the quality of studies is weak and they are inconclusive [22,23,24]. In particular, much of the previous studies evaluated the effectiveness and cost-effectiveness of these interventions for frail elderly by considering health outcomes such as functional limitations and health-related quality of life [24, 25]. These outcomes might not be appropriate for frail elderly people whose health is deteriorating [26, 27]. Hence, traditional cost-effectiveness studies seem unable to capture outcomes relevant to frail older people. Multi-Criteria Decision Analysis (MCDA) can provide a framework for evaluating integrated care on a broader set of outcomes [28, 29]. In MCDA the outcomes (i.e. criteria) can be reviewed separately but also integrated into one score by applying a relative weighting of importance to the outcomes. In addition, MCDA can consider multiple perspectives by using weights reflecting the preferences of different stakeholders [28, 30].

To our knowledge, no thorough scientific evaluation of patient outcomes related to introducing HCPC has so far been completed. Within the HCPC context, a qualitative exploration was conducted by Nilsen et al. [31]. They studied home care nurses’ experiences with implementation and systematic use of functional and wellbeing checklists developed for improving continuity of care and quality of care in the pathways provided to old and chronically ill patients in the communities. The checklists aimed to be person-centred and function-based as suggested by the findings from the Patient Trajectory for Home-dwelling elders (PaTH) project [32]. Using MCDA, this study aims to investigate the impact of introducing a specific model of integrated care for frail elderly patients, HCPC. We investigate specifically whether the HCPC programme contributes to improved health and well-being, experience of care and resource utilisation. The programme is evaluated as part of the European Union -financed SELFIE project (Sustainable integrated care models for multi-morbidity, delivery, financing and performance).Footnote 2



The three core differences between the HCPC programme (i.e. integrated care programme) and usual care (control group) are first the initial and follow-up (6 weeks) assessment of the patient’s level of functioning by validated tools; second the “everyday-rehabilitation” informed by the patient’s own goals for activities of daily living; and third the early involvement of the patient’s GP. As part of the programme a new professional role has been developed; a designated primary contact person (coordinator), notably a nurse or a social worker, responsible for individual patient follow up. The designated primary care contact works in the municipal care service. Focus is on functional ability rather than on disease and impairment. A patient’s GP is involved within 2 weeks after enrolment, i.e. through consultation at GP surgery or a home visit. The primary care teams comprise at least the patient’s coordinator and the GP. Other primary care professionals (e.g. physiotherapist, occupational therapist, social worker) are involved when appropriate. The HCPC and patient care involved are financed through the participating municipalities’ general budgets. There are no direct financial incentives towards municipalities.

Population and data collection

The study population comprised frail elderly patients with multi-morbidity starting or extending their use of municipal home care services, or having a short-term stay in a nursing home, because of functional deterioration. While many of these patients were discharged from hospital recently, hospitalization was not mandatory for inclusion. The intervention group was recruited from municipalities participating in HCPC, and the usual care from municipalities not (yet) enrolled to this programme.

The HCPC group was derived from 12 municipalities in South- and Mid-Norway. Two of these municipalities were very small (< 2000 inhabitants), eight were small (5000 to 26,000 inhabitants) and two were mid-sized (44,000 to 46,000 inhabitants). The programme owner, The Norwegian Association of Local and Regional Authorities, introduced and recommended the study to the municipalities enrolled in HCPC through their contact network and during network meetings. Members of the research team attended several meetings, provided written and oral information to the municipalities and signed collaboration contracts. Primary contact persons in the enrolled municipalities recruited participants consecutively, i.e. they conducted home visits to eligible patients, provided study information, and collected informed consent. Within the eligible study population, there were no exclusion criteria. For patients with dementia or other conditions interfering with competence to give consent or answer survey questions, next of kin was invited instead. Three questionnaires at baseline and three at follow-up (for the same patients) were, and one questionnaire only at follow-up was completed by proxy. The enrolled patients’ primary contact person completed the “SELFIE-questionnaire for frail elderly” (we will refer to this questionnaire as the “SELFIE-Questionnaire” hereafter) based on a face-to-face patient interview in the patient’s home, at baseline (enrolment) and after 6 months. Data collection was conducted from September 2017 to June 2019.

The usual care group was derived from four municipalities in West-Norway not yet enrolled in the learning network programme of HCPC. Two of the municipalities were small (8000 to 16,000 inhabitants), one was mid-sized (30,000 inhabitants) and one was large (284,000 inhabitants). In the three small and mid-sized municipalities, research assistants (nurses/assistant nurses) conducted home visits to eligible patients, provided study information, and collected informed consent. In the large municipality, none of several attempts to recruite eligible home-dwelling people were successful. Instead, research assistants (employees/students at the University of Bergen, Norway) recruited eligible patients during short-term rehabilitation stays in nursing homes, before discharge to their own home. As with the HCPC group, there were no exclusion criteria in the usual care group. Next of kin was invited to fill out the questionnaire in cases where patients were unable to answer the questions by themselves. However, none of them filled out the questionnaire to a sufficient extent to be included in the study. For the enrolled patients, the research assistants completed the “SELFIE-Questionnaire” based on a face-to-face patient interview in the nursing home at baseline (enrolment) and in the patients’ home (after 6 months). Data collection was conducted from September 2018 to October 2019.

Multi-criteria decision analysis (MCDA)

The SELFIE MCDA framework was developed based on established guidelines and follows the seven recommended steps: i) establish the decision-context, ii) identify and structure criteria, iii) determine the performance on criteria, iv) determine the weights of the criteria, v) create an overall value score, vi) perform sensitivity analyses, vii) interpret results [28, 29]. In the first step the aim is to establish what the likely decisions are that need to be made and thus how the MCDA will be used [28, 29]. Earlier qualitative study including document analyses and interviews with programme-initiators, managers, representatives of payer organisations and care providers of the HCPC programme has shown that the programme needs to provide evidence on the effectiveness of the intervention to help establish the long-term sustainability and potentially wider implementation of the programme throughout Norway [30, 33]. The aim of the MCDA was to inform these decisions quantitively by comparing the HCPC programme to usual care on the improvement in health and well-being, experience of care and resource utilisation. Moreover, it was important to identify the relevant stakeholders in this decision-making process. The stakeholders that were considered pertinent were five groups: Patients, Partners and other informal caregivers, Professionals, Payers, and Policy makers [33].

Outcomes/criteria (the second step)

The decision criteria that were used in the MCDA were the outcome variables that were measured with the “SELFIE-Questionnaire” specially developed for the integrated care programmes targeting elderly populations (see Appendix 1). Table 1 illustrates the validated items from well-established and widely used “instruments” for defining the outcomes included in the “SELFIE-Questionnaire”. All outcomes were related to one of the three domains of the Triple aim, i.e. health/well-being, experience of care and resource utilisation/costs (Berwick et al., 2008). A core set of outcomes included physical functioning, psychological well-being, enjoyment of life, social relationships and participation, resilience, person-centeredness, continuity of care and total health and social care costs.Footnote 3 In addition, for the elderly patients, programme-specific outcomes included autonomy, informal care costs, long term institutional admission, and falls leading to hospital admissions. The distinction between a core set of outcomes and programme-specific outcomes was made within the SELFIE project, where the core set was measured in all evaluation studies as part of that project.

Table 1 Outcome measures included in the SELFIE-Questionnaire and their scale range

To estimate and compare total health and care cost between HCPC and usual care, the cost components were identified and quantified from a societal perspective. The health and social care costs included the costs arising from the consequences of treatment (i.e. costs incurred with primary care services, e.g. GP and nurse, drug, and hospital costs) and formal social care costs. Costs related to informal care were estimated separately. Data on health, social care and informal care use were collected with an adapted version of the iMTA Medical Consumption Questionnaire and included within the “SELFIE- Questionnaire” [34].Footnote 4 The questionnaire includes questions about contacts with GPs, primary care nurses, GP assistants, physiotherapists, dieticians, psychologists, dentists, social workers, welfare workers, and medical specialists, as well as hospital inpatient- and outpatient admissions, home care services, residential care and nursing homes, and informal care services during the past 3 months.

To quantify the total health and social care costs in monetary terms (Norwegian kroner - NOK) and to get the unit price of the relevant components, we used several different published documents and articles. In particular, national tariffs for GP services come from the Normal tariff for General Practitioners and Out-Of-Hours Emergency medical services 2018–19 document [36]. We converted all costs into 2019 prices using Consumer Price Index (changes in the price level of a weighted average market basket of consumer goods and services purchased by households) provided by Statatistics Norway (see [37]). The consultation fee for a GP was calculated as the weighted average of the fee for a non-specialist GP (NOK 165 (16.7 EURFootnote 5)) and an approved GP specialist (NOK 257 (26 EUR)). In estimating actual health care provider cost, out-of-pocket costs (co-payments) were assumed to be 30% of actual provider cost [39]. Per-diem in-patient hospital cost information was gathered from a recent published study from Norway [40]. Using their estimation, inflated by the Consumer Price Index, we estimated the average costs of a general ward bed day in Norwegian hospitals at NOK 8400 (853 Euros in 2019 price).

To estimate the medication costs we have collected the Defined Daily Dose, i.e. the assumed average maintenance dose per day for a drug used for its main indication in adults, by using the Defined Daily Dose Index 2020 [41]. The medication’s price was obtained using the joint directory for drugs [42]. The unit cost for social care and home care inpatient and outpatient components were gathered from reports published by the Norwegian Health Directorate [43, 44]. To estimate the informal care, we used relevant unit cost for home care services. The details on unit costs for social care and home care services by different municipalities are provided in Table A1 in Appendix 2.

Statistical analysis of performance scores (the third step)

The performance scores on the decision criteria of the MCDA were defined as the scores on the outcome measures described above. When estimating causal effects of using observational data, it is crucial to reduce systematic differences in the empirical distribution of the baseline (pre-intervention) confounders [45]. We performed a quasi-experimental approach called inverse probability of treatment weighting (IPTW) using the propensity score, to minimise the impact of any potential selection bias between HCPC and usual care at baseline. Logit regression was used to estimate the propensity score in which treatment status was regressed on selected observed baseline socio-demographic characteristics: age, gender, living condition, smoking status, multi-morbidity status (number of health problems≥2), and the baseline values of two core-set outcome variables, namely physical health, and social relationships and participation. The IPTW assigns a weight to each UC patient based on the similarity of the patient to the HCPC patients. In the IPTW the weights of the HCPC patients were set to 1 and the weights of the usual care patients were calculated with the formula Weight = propensity score/(1–propensity score).

The baseline differences between the HCPC and usual care patients were assessed before and after the IPTW. Overall matching results were assessed by examining and reporting three test statistics. First, the mean (median) absolute standardised bias (i.e. the mean/median of the ratios of the difference of the sample means in the HCPC and usual care groups over the square root of the average of the variances in both groups); second, Rubin’s B, defined as the standardized difference of the means of the linear index of the propensity score in HCPC and usual care group, and third, Rubin’s R, defined as the ratio of HCPC and usual care variances of the linear index of the propensity score [46]. Statistical significance of the standardised differences have also been presented for all covariates before and after matching.

We estimated the “average treatment effect on the treated” for all outcome measures on the IPTW data. To analyse the outcomes we used repeated measurement models with individual level random intercept effects (i.e we assumed that the coefficients were fixed but the intercept varied randomly). We used models assuming continuous outcomes for ease of interpretation. Formally, we estimated the following equation:

$$ {y}_{jt}={\beta}_0+{\beta}_1{I}_j+{\beta}_2{T}_t+{\beta}_3{I}_j\times {T}_t+\chi {X}_{jt}+{\psi}_j+{\varepsilon}_{jt} $$

where yji is the outcome for individual j at time t; Ij is a dummy variable categorising intervention group (variable equals to 1 if the observation is from the HCPC cohort) and Tt is a dummy variable for time (equals to 1 if the observation is from T1 period), respectively. The coefficient for \( {\hat{\beta}}_3 \) describes the treatment effect. Xjt includes individual jth age at time period t. ψj is the random error term for the jth individual and εjt is the remaining error term for jth individual observed in the tth period.

To calculate performance scores we predicted the mean score of the HCPC group at 6 months based on the regressions results. In addition we calculated the mean score of the usual care group assuming they had the same baseline score as the HCPC group. In this way the calculated performance scores could be directly compared between the HCPC and usual care group. This was done separately for each outcome.

Weghting the criteria (the fourth step)

Relative weights for the different criteria (i.e., outcomes) among stakeholders were elicited in an online weight elicitation study among Norwegian patients, informal caregivers, professional care providers, payers and policy makers. In the weight elicitation study, a discrete choice experiment (DCE) was used to obtain weights for the core set of outcomes (for full details of how these preference weight were obtained see [30, 47]. Table 2 gives the relative weights of the outcomes included in the MCDA, and as shown, all five stakeholder groups put relatively high weights on enjoyment of life and the lowest weight on cost.

Table 2 Relative DCE weights of the core set of outcomes used in the Multi-Criteria Decision Analysis by type of stakeholder

Overall value calculation (the fifth step)

In the MCDA, the mean predicted outcome scores at T1 were first standardised on a 0–1 scale. This was done using relative standardisation with the equation:

$$ {S}_{aj}=\frac{x_{aj}}{{\left({x}_{aj}^2+{x}_{bj}^2\right)}^{1/2}} $$

Where xajis the raw performance score in terms of mean predicted values for outcome j (on the natural scale) for the HCPC group and; \( {x}_{aj}^2 and\ {x}_{bj}^2 \) are the square of mean predicted values for outcome j for the HCPC group and the usual care group respectively. For all outcomes, the standardised score was set so that a higher score indicates better performance. To achieve this, in the above-mentioned equation, x was replaced by 1/x for reversely coded outcome measures (i.e outcomes where a higher score on the natural scale indicates a worse performance). We implemented an additive MCDA model where the standardised outcomes were weighted and subsequently summed to obtain a single overall value score for the HCPC group and a single overall value score for the usual care group.

Sensitivity analysis (the sixth step)

Probabilistic sensitivity analysis using Monte Carlo simulation was performed to evaluate the joint uncertainity of preference weights and performance scores. Cholesky decomposition was conducted for 10,000 replications to obtain a single overall value score. We calculated confidence (uncertainty) intervals around the overall value scores for each stakeholder group. The difference between the overall value score of the HCPC and the usual care is statistically significant if the confidence intervals do not overlap.

In the deterministic sensitivity analyses, we additionally conducted a swing weighting method for eliciting preferences [30, 48]. Within this analysis a wider range of outcomes (along with the eight outcomes in the core set), namely the programme-specific outcomes for frail elderly care programmes, were also included.Footnote 6

All statistical analyses were performed with STATA 16.


Sample characteristics

The study comprised information from 120 and 89 patients at baseline from the HCPC and usual care group respectively, and 86 patients in the HCPC group and 41 patients from usual care group at follow-up after 6 months. After scrutinizing the data, we found that there were some missing observations (varied from minimum 1 to highest 10) for a few outcome variables. To be pragmatic and to consider all available patients responding at baseline, we imputed these missing data on the outcome variables by their mean values and by the HCPC and usual care groups. However, we omitted four patients who responded at follow-up only but not at baseline.

Table 3 provides descriptive statistics on baseline characteristics of patients for two groups, and for pre-and post-matching data. In particular, before matching, the average age of the elderly patients at baseline was around 80.0 years for the HCPC group, while the corresponding average age was higher for the usual care group, at 83.6 years. Overall, as illustrated in the matching statistics given in the lower panel of Table 3, matching led to an improvement in the comparability of the two groups. The statistics satisfied the required criteria with recommended ranges (Rubin [46] suggested that the value of B < 25; 0.5 < R < 2 for sufficient balance). In particular, mean (median) standardised bias was 27.9 (28.0) in the unmatched sample, and after implementing the IPTW using the propensity score approach in the matched sample it was reduced to 6.3 (3.5); Rubin’s B was 88.6 in unmatched sample and reduced to 22.8 for the matched sample, and Rubin’s R showed 0.79 in unmatched sample and changed to 0.91 in matched sample.

Table 3 Baseline characteristics before and after propensity score matching

Figure 1 shows the distribution of the core outcome variables between the HCPC and usual care groups before and after matching. As is visualized, after matching the distributions are quite overlapping for all core-set outcomes provided in the figure.

Fig. 1
figure 1

The distribution of the core outcome variables between HCPC (treated) and usual care groups (untreated) before and after matching

Effects on outcomes

Table 4 illustrates the results of the statistical analyses of all core set outcomes included in the MCDA. The estimated “treatment effects on the treated”, \( {\hat{\beta}}_3 \) (coefficients of the interaction term between time and intervention), is provided in the last column of Table 4. The five health and wellbeing outcomes (the first 5 items/rows) consistently favour HCPC, and the experience outcomes favour UC, but there is no statistically significant effect (at the 5% level) of HCPC for any of these core-set outcome variables. A weak positive effect of the intervention is observed for the outcome variable resilience. A significant effect is observed for the programme-specific outcome autonomy (“remaining in charge and making own decisions on how one lives his/her own life”). The estimated coefficient indicates that HCPC negatively affects patients’ autonomy as measured by the Pearlin Mastery Scale (see Table 1), which is reduced by 2.9 points for the patients belonging to HCPC compared to usual care. A weak (at the 10% level) effect is observed for the informal care cost outcome in favour of HCPC.

Table 4 Health/Wellbeing and Experience Outcomesd: within and between group differences at 6 months

Table A2 (in Appendix 2) describes the health and social care costs differences for the last 3 months prior to interview date) by HCPC versus usual care at follow-up. The average total health and social care costs and informal care costs were lower for HCPC than usual care at follow-up. However, when categorising the total costs, we found that the costs of the care provided by the the GP, medical specialist and psychologist were higher for HCPC than usual care. Costs related to nurses, physiotherapists, hospital emergency care, hospital stay, inpatient and outpatient social care were substantially lower for HCPC than usual care.

As shown in Table 4, the treatment effect is also negative for health and social care costs (Euro − 2999), indicating that HCPC is less costly than usual care, as is the case also for costs of informal care. The estimate indicates a reduction in informal care costs of HCPC by Euro 2097 compared to usual care.

MCDA overall scores

Table 5 reports the overall value score in the Multi-Criteria Decision Analysis (MCDA) for five Norwegian stakeholder groups (5Ps): Patients, Partners, Professionals, Payers and Policy makers. For All stakeholder groups, HCPC performed better than usual care, although the difference was small. Regarding the individual outcomes, the standardized performance score was higher for HCPC than usual care, except for continuity of care.

Table 5 Stakeholders’ value scores in the Multi-Criteria Decision Analysis

Sensitivity analysis

The robustness of the results was investigated by considering new sets of coefficients for treatment effect and importance weights using the Cholesky decomposition approach. The lower panel of Table 5 shows the uncertainty around the value score calculated with Monte Carlo simulation. For all five stakeholders, the MCDA results were very similar to our base analyses, indicating that the overall value scores were higher for integrated care than usual care. The proportion of the 10,000 simulations showing a preference for integrated care was between 94 and 99% for all stakeholders. The 95% confidence intervals did not overlap.

Table A4 in Appendix 3 also presents the value scores in the MCDA with the swing weights. The weights differed slightly, and the estimated score for the programme-specific outcome autonomy also here favoured usual care rather than HCPC. Overall, the MCDA results were highly consistent with our base analyses. The weighted scores were of similar magnitudes, and higher for integrated care than usual care for all five stakeholder groups.


Summary of findings (the seventh step)

HCPC is an integrated care programme targeting frail elderly patients with multi-morbidity that have just started or extended their use of municipal home care services or have a short-term stay in a nursing home. The results from an MCDA, where a large set of outcomes corresponding to the ‘Triple aim’ (i.e. health/well-being, experiences with care, and costs) was taken into consideration, indicated that elderly patients in the municipalities participating in learning networks of HCPC fare better than elderly patients in the municipalities offering usual care. The result held for the perspectives of patients, partners, professionals, policy makers and payers. The differences in total MCDA scores between HCPC and usual care were constant over the five stakeholder groups. The main driver of this result seemed to be the difference in performance of the HCPC along the dimension enjoyment of life. Here HCPC improved patients’ experienceconsiderably more than usual care, although the effect was not statistically significant. There was little difference of opinion between the stakeholders with respect to enjoyment of life being the highest valued outcome measure. Furthermore, it appeared that the HCPC programme increased cost incurred for primary health care, but reduced comparatively costly components attributed to hospital stay and nursing home care, and cost associated with informal care. The municipalities cover the establishment costs for the HCPC programme, however, we did not have data on this intervention cost component (e.g. costs associated with intensified collaboration between sectors, costs of planning individual care pathways) at our disposal and it was not included in this analysis. Moreover, it is difficult to make any sensible assumptions on the costs related with collaboration between sectors and cost related with planning individual care pathways. We have been in contact with some of the municipalities to have them indicate costs associated without them being able to give us precise estimates. But they are not likely to be very high, since also individual pathways have to planned both in the HCPC and the usual care groups. Besides, when it comes to costs borne by the municipalities, who run the programmes, there is no strong indication of favourable or adverse financial incentives.

An interesting difference was observed when weighting outcomes with swing weights instead of DCE weights. Enjoyment of life was attached a lower weight, while autonomy was weighted quite highly. Furthermore, usual care patients fared better than HCPC patients when considering how they feel about mastering their own life. This may reflect that involving more health personnel in HCPC may provide better care but at the cost of more decision making left with professional care providers, which may negatively affect the patients’ feeling of autonomy.

Strengths and limitations

The study is a novel contribution to investigating and understanding how an integrated health care programme performs when taking into consideration the multi-dimensionality of relevant outcomes. The SELFIE MCDA provides a structured and explicit evaluation framework for assesing an integrated programme with multiple health and wellbeing outcomes relevant to frail elderly patients. Even though we are aware of the methodological challenges discussed in the literature, particularly regarding which outcomes to include and how to assess uncertainty [49], the study provides some evidence that pure cost-effective analyses may miss important aspects of relevant outcome space. An international valuation study has shown that different stakeholder groups - patients, partners, professionals and policy makers - both within and between countries appreciated the enjoyment of life, social relationships and participation, and personal autonomy in addition to traditional health outcomes [47]. This study showed that on certain outcomes HCPC performed better than usual care and on others it did not. MCDA allowed us to use an explicit framework to aggregate the various outcomes and to determine an overall value score. Furthermore, we showed overall value scores from the perspectives of different stakeholders. The latter is important because aligning the preferences of different stakeholders is likely to contribute to the success of an integrated care approach. There is good reason to have confidence in the weights attached to the outcomes, obtained in a discrete choice experiment among a total of 776 stakeholders (~ 150 per stakeholder group) (Rutten-van Mölken et al., 2020).

Another strength is that the performance scores were estimated using a quasi-experimental framework (i.e. IPTW) by including a control group which provided better estimates of effectiveness than many previous studies in this field that are before-after studies. Nevertheless, the uncertainty around the overall value scores was formally incorporated using probabilistic sensitivity Monte-Carlo simulations on both weights and scores.

Methodologically, we have shown benefits from using an MCDA framework in evaluating care programmes aimed at patients that are in a situation with several, and often conflicting, demands. Hoverver, there are some methodological limitations with the MCDA framework discussed in the literature [49]. In particular, the creation of a composite measure of benefit without knowing the threshold value for the willingness to pay or the opportunity costs of one additional unit of that benefit (as researchers do know for QALYs); and whether cost should be consider as one of the criteria/outcomes in the MCDA. Questions could also be raised on the utility of the value scores from the different perspectives. Is there a perspective that should prevail? Or, should analysts average the perspectives? In our study we assumed that the performance scales and weight scales were linearly associated. Simplicity, transparency and implementational convenience favour the additive MCDA models. To avoid inappropriate double counting of value, the outcomes in an additive model should not overlap and the outcomes should be preferentially independent [49]. The potential violations of these key assumptions may not be ruled out in our additive MCDA framework. To study this may involve, for example, mapping between the discrete choice experiment levels and the performance scale. However, changing our assumption of linearity is unlikely to affect our MCDA results because the performance scores of the HCPC and usual care do not differ very much. Although the SELFIE MCDA framework has made efforts to overcome several challenges facing the approach (see [30]), future work could be attained on the actual use of the MCDA tool by decision makers to clarify how stakeholders want to engage with MCDA, and precisely when and how it contributes the most.

Further limitalions are connected with the patient recruitment and the impact evaluation. Patient recruitment in both HCPC and usual care municipalities was difficult for several reasons. It was particularly difficult to establish the usual care municipalities. The Norwegian Association of Local and Regional Authorities made efforts to recruit municipalities that were planning to join the HCPC programme, expecting these might have an interest in a comparative study. However, several municipalities declined, often due to a lack of resources to assist in collecting data, or general overload of work also related to planned merging of municipalities. Furthermore, although the HCPC group was easier to establish, patient recruitment here proved difficult since so many of the municipalities were small with few eligible patients. Another issue to consider was that frail elderly patients in general may have problems answering the lengthy questionnaire that was used for data collection on health and wellbeing, experience of care, and individual costs. For these patients, the follow-up interview was particulary problematic, since their health condition might worsen. Recall bias associated with health and social care, and informal care resource use could be a potential concern for the study using a retrospective self-reported questionnaire. However, such a bias would not be systematically different between the HCPC and usual care groups. Attrition rate as a result of losses to follow-up was relatively higher for the usual care group. At the 6-month follow-up, 41 patients from the usual care group (more than 46% sample at the baseline) responded, whereas 86 patients (around 72% of baseline sample) responded from the HCPC group. Ifloss to follow-up was systematically related to the patients’ underlying characteristics such as age, multimorbidity status etc., differences in the attrition rates for usual care and HCPC group could result in selection biases. High attrition rates are common in studies on elderly patients [50]. The HCPC group comprised exclusively home-dwelling frail elderly people, while the UC group was mainly recruited from short-term nursing home rehabilitation stays (93%) and mostly from one large municipality. The difficulty to recruit home-dwelling patients in this control municipality was attributable to the lack of dedicated primary care contact persons. This strategy may have introduced bias which we attempted to overcome through a successful implementation of the IPTW approach at baseline by controlling for the living condition attribute in estimating the propensity score. Moreover, different persons were involved in recruiting and interviewing patients in the different municipalities. Even though they were given instructions on patient selection and how to assist patients in answering, it is difficult to rule out reporting bias.


The study population was derived from 16 municipalities of various sizes in different parts of the country, supporting transferability of the results to similar municipalities in other regions. The generic clinical pathways for patients with multi-morbidity in the HCPC programme ensured that the included patients have many different chronic conditions and thus comprised a representative population with multi-morbidity in need of municipal care services. Transferability of results to other countries depends on organisation and cooperation of health and social services, and how different outcomes would be weighted by different stakeholders.

Comparison with existing literature

It is difficult to directly compare the MCDA results of the HCPC programme to other integrated care programmes. Many integrated primary care approaches targeting frail older persons have emerged over the years. As mentioned in the onset, however, the quality of previous studies was not good and evidence for their effectiveness and cost-effectiveness remained mixed [22,23,24, 51]. The integrated care programmes were highly multifaceted interventions and as such comparing them to each other introduced difficulties due to differences in interventions, outcomes and populations [52]. Nevertheless, in a recent comprehensive review, Baxter and colleagues [52] concluded that integrated care programmes/models might enhanced patient satisfaction, increased perceived quality of care, and enabled access to services, although the evidence for other outcomes including health and wellbeing outcomes and service costs remained uncertain.

Implications for research and/or practice

The numbers of patients with multiple chronic diseases are increasing, which is a challenge to the public purse. To establish relevant and cost-efficient care it is vital to investigate the main needs and care demands of these patients. This project highlights the importance of providing care along different dimensions. The Norwegian HCPC programme for frail elderly patients is an integrated programme that has received much attention from the municipalities in Norway. The results of this study are important contributions to better understand the effects of integrated care programmes and may be useful for decision-makers at the municipal and central level for prioritisation of often-costly initiatives. Future research should focus on developing stronger links between outcomes and weights in the MCDA. Our results support furthering resources for a randomised controlled trial that may provide stronger conclusions on causality than possible in a quasi-experimental design.


This study shows the importance of evaluating reforms and new initiatives for chronically ill and frail elderly patients, often with multi-morbidities, in a broad framework. Cost-effectiveness analyses may give a first-hand insight into the acceptability of spending resources on pure health and cost dimensions. However, the pure health state dimension does not capture all relevant features of what is relevant to these patients. The results clearly show the importance of a broad perspective such as the MCDA framework when considering care delivery in a transparent way. A national programme like HCPC, with moderate changes, may improve well-being of the patients, in the long term be cost saving, and overall be worthwhile pursuing in gradual care reforms for the frail elderly patients.

Availability of data and materials

The stakeholder data used for weighing is anonymous (provided in Table 2), and can be available on request from the corresponding author (MKI). The data that support the findings provided in Tables 3, 4 and 5 are not publicly available due to them containing information that could compromise research participant privacy/consent.


  1. Learning networks as a method has been internationally used and evaluated for a range of issues, evidence summary can be accessed [14]. In Norway, three national programmes have the short name “Learning network” (for children, for drug abuse/psychiatric care and for elderly).

  2. Among other aims, the SELFIE project evaluates impact of 17 different integrated chronic care (ICC) programmes for patients with chronic conditions and multi-morbidities in eight European countries [30].

  3. In the SELFIE project, several different integrated programmes were investigated. Some outcomes were common for all programmes, termed ‘core outcomes’, whereas other outcomes were specific for a smaller groups of treatment programs, like those involving frail elderly. The latter are termed ‘program specific outcomes’. The selection of the outcomes was based on a literature review, workshops with representatives from the five stakeholder groups, and focus groups with individuals with multi-morbidity [33].

  4. The link to where people can order the questionnaires at [35].

  5. Annual exchange rate for 1 euro = NOK 9.8527 in 2019 [38].

  6. Table A3 in Appendix 3 shows the swing weights for five Norwegian stakeholders. We found some difference in the ranking of the weights over attributes and by stakeholders. For example, in swing weight, patients put relatively highest weight on physical functioning and then autonomy, while other stakeholders put relatively higher weights on enjoyment of life and autonomy.



Multi Criterion Decision Analyses


The Organisation for Economic Co-operation and Development


Holistic Continuity of Patient Care


Discrete Choice Experiment


Propensity Score Matching


Inverse Probability Treatment Weighting


  1. Rijken M, Struckmann V, Dyakova M, Gabriella Melchiorre M, Rissanen S, van Ginneken E. ICARE4EU: improving care for people with multiple chronic conditions in Europe. Eurohealth. 2013;19(3):29–3.

    Google Scholar 

  2. Palladino R, Tayu Lee J, Ashworth M, Triassi M, Millett C. Associations between multimorbidity, healthcare utilisation and health status: evidence from 16 European countries. Age Ageing. 2016;45(3):431–5.

    Article  Google Scholar 

  3. Syse A, Thomas MJ, Gleditsch RF. Norway’s 2020 population projectionsNational level results, methods and assumptions. Statistics Norway (SSB). 2020;ISSN:0806–2056.

    Google Scholar 

  4. SSB (Statisrics Norway). Levekårsus 1998–2008, SSB, SSB statistikkbankens levekårsundersøkelse, 2009, accessed 13 september, 2020.

    Google Scholar 

  5. SSB (Statisrics Norway). Eldres bruk av helse- og omsorgstjenester, 2013. ISSN 0804-322 Oslo, Norway.

    Google Scholar 

  6. OECD/European Observatory on Health Systems and Policies. Norway: Country Health Profile 2019, State of health in the EU, OECD publishing, Paris/European Observatory on Health Systems and Policies, Brussels. ISBN 9789264432161(PDF), Series: State of Health in the EU; SSN 25227041 (online).

  7. Norwegian Ministry of Health and Care Services. Report No. 47 (2008–2009) to the Storting. The coordination reform. Oslo:; 2008 [Summary in English]. Downloaded from: . Accessed: 7th April 2020.

  8. Grimsmo A. The Norwegian Care Coordination Reform was launched on 1 January 2012. How well has it worked, and where does it go from here? Tidsskr Nor Legeforen. 2015;135:528.

    Google Scholar 

  9. Danielsen K K, Nilsen E R, Fredwall TE, 2017, Pasientforløp for eldre med kronisk sykdom, [patient trajectories in elderly with chronic illness] in Norwegian. Evidence summary. Pdf. Downloaded from: accessed: 7th April 2020.

    Google Scholar 

  10. Grimsmo A, Løhre A, Røsstad T, Gjerde I, Heiberg I, Steinsbekk A. Disease-specific clinical pathways - are they feasible in primary care? A mixed-methods study. Scand J Prim Health Care. 2018;36(2):152–60.

    Article  Google Scholar 

  11. Røsstad T, Salvesen Ø, Steinsbekk A, Grimsmo A, Sletvold O, Garåsen H. Generic care pathway for elderly patients in need of home care services after discharge from hospital: a cluster randomised controlled trial. BMC Health Serv Res. 2017;17(1):275.

    Article  Google Scholar 

  12. de Vibe M, Udness E, Vege A. Gode pasientforløp i kommunene for eldre og kronisk syke, en beskrivelse av to læringsnettverk, 2011–2015. In: Holistic continuity of patient care in the communities for older and chronically ill, a description of two learning networks, 2011–2015: Notat om kvalitetsutvikling nr. 4; 2016. (Note about quality development no. 4). Retrieved from

  13. The Breakthrough Series. IHI’s Collaborative Model for Achieving Breakthrough Improvement. In: IHI Innovation Series white paper. Boston: Institute for Healthcare Improvement; 2003. (Available on

    Google Scholar 

  14. Learning Network for good patient care in the municipalities for the elderly and chronically ill. Available at: Accessed 28 June 2019.

  15. Leijten FRM, Hoedemakers M, Struckmann V, et al. Defining good health and care from the perspective of persons with multimorbidity: results from a qualitative study of focus groups in eight European countries. BMJ Open. 2018;8:e021072.

    Article  Google Scholar 

  16. World Health Organisation (WHO) 2016, Integrated care models: an overview. (working document) by the health services delivery Programme division of health systems and public health. (pdf) downloaded from: accessed: 7th April 2020.

    Google Scholar 

  17. Kirst M, Im J, Burns T, Baker GR, Goldhar J, O'Campo P, et al. What works in implementation of integrated care programs for older adults with complex needs? A realist review. Int J Qual Health Care. 2017;29(5):612–24.

    Article  Google Scholar 

  18. Damery S, Flanagan S, Combes G. Does integrated care reduce hospital activity for patients with chronic diseases? An umbrella review of systematic reviews. BMJ Open. 2016;6(11):e011952.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Threapleton DE, Chung RY, Wong SYS, Wong E, Chau P, Woo J, et al. Integrated care for older populations and its implementation facilitators and barriers: A rapid scoping review. Int J Qual Health Care. 2017;29(3):327–34.

    Article  Google Scholar 

  20. Briggs AM, Valentijn PP, Thiyagarajan JA, Araujo de Carvalho I. Elements of integrated care approaches for older people: a review of reviews. BMJ Open. 2018;8(4).

  21. Berwick D, Nolan T, Whittington J. The triple aim: care, cost, and quality. Health Aff. 2008;27(3):759–69.

    Article  Google Scholar 

  22. Blom J, et al. Effectiveness and cost-effectiveness of proactive and multidisciplinary integrated care for older people with complex problems in general practice: an individual participant data meta-analysis. Age Ageing. 2018;47:705–14.

    Article  PubMed Central  Google Scholar 

  23. Eklund K, Wilhelmson K. Outcomes of coordinated and integrated interventions targeting frail elderly people: a systematic review of randomised controlled trials. Health Soc Care Community. 2009;17:447–58.

    Article  Google Scholar 

  24. Looman WM, Huijsman R, Fabbricotti IN. The (cost-) effectiveness of preventive, integrated care for community-dwelling frail older people: a systematic review. Health Soc Care Community. 2019;27:1–30.

    Article  Google Scholar 

  25. Evers SM, Paulus AT. Health economics and integrated care: a growing and challenging relationship. Int J Integr Care. 2015;15:e024.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Comans TA, Peel NM, Gray LC. Scuffham PAQuality of life of older frail persons receiving a post-discharge program. Health Qual Life Outcomes. 2013;11:58.

    Article  Google Scholar 

  27. Makai P, Looman W, Adang E, Melis R, Stolk E, et al. Cost-effectiveness of integrated care in frail elderly using the ICECAP-O and EQ-5D: does choice of instrument matter? Eur J Health Econ. 2015;16:437–50.

    Article  Google Scholar 

  28. Thokala P, Devlin N, Marsh K, et al. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19(1):1–13.

    Article  Google Scholar 

  29. Marsh K. M IJ, Thokala P, et al. multiple criteria decision analysis for health care decision making--emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health. 2016;19(2):125–37.

    Article  Google Scholar 

  30. Rutten-van Mölken M, Leijten F, Hoedemakers M, Tsiachristas A, Verbeek N, et al. Strengthening the evidence-base of integrated care for people with multi-morbidity in Europe using multi-criteria decision analysis (MCDA). BMC Health Serv Res. 2018;18:576.

    Article  Google Scholar 

  31. Nilsen ER, Söderhamn U, Dale B. Facilitating holistic continuity of care for older patients: home care nurses’ experiences using checklists. J Clin Nurs. 2019;28:3478–91.

    Article  PubMed  Google Scholar 

  32. Røsstad T, Garåsen H, Steinsbekk A, Sletvold O, Grimsmo A. Development of a patient-centred care pathway across healthcare providers: a qualitative study. BMC Health Serv Res. 2013;13:121.

    Article  Google Scholar 

  33. Leijten FRM, Struckmann V, van Ginneken E, Czypionka T, Kraus M, Reiss M, et al. SELFIE consortium. The SELFIE framework for integrated care for multi-morbidity: Development and description. Health Policy. 2018;122(1):12–22.

    Article  Google Scholar 

  34. Bouwmans C, De Jong K, Timman R, Zijlstra-Vlasveld M, Van der Feltz-Cornelis C, Tan SS, et al. Feasibility, reliability and validity of a questionnaire on healthcare consumption and productivity loss in patients with a psychiatric disorder (TiC-P). BMC Health Serv Res. 2013;13:217.

    Article  Google Scholar 

  35. Questionnaires for the measurement of costs in economic evaluations available at:

  36. Norske legeforening. Normal tariff for General Practitioners and Emergency medical services 2018–19 ( in Norwegian Normaltariff for fastleger og legevakt 2018–2019. Den norske legeforening. Retrived from Accessed on June 28, 2019.

  37. SSB (Statisrics Norway)

  38. Bank Norway, 2019. Retrived from .

  39. Lindahl, Anne Karin, 2015. The Norwegian Health Care System by, Norwegian Knowledge Centre for the Health Services (accessed on January 20, 2020).

  40. Lindemark, et al. Costs and expected gain in lifetime healthfrom intensive care versus general ward care of 30,712 individual patients: a distribution weighted cost-effectiveness analysis. Crit Care. 2017;21:220.

  41. Daiily Dose Index 2020: Retrived from

  42. Felleskatalogen: Medisin: Retrived from

  43. Health Directoret, Forsøk med statlig finansiering av omsorgstjenester, 12/2018. Retrived from

  44. Health Directoret, Kostnadsnivå og produktivitet i psykisk helsevern og TSB 2014–2018, SAMDATA spesialisthelsetjeneste September 2019. Retrived from

  45. Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25:1–21.

    Article  Google Scholar 

  46. Rubin DB. Using propensity scores to help design observational studies: application to the tobacco litigation. Health Serv Outcome Res Methodol. 2001;2:169–88.

    Article  Google Scholar 

  47. Rutten-van Mölken M, Karimi M, Leijten F, Hoedemakers M, Looman W, Islam K, et al. Comparing patients’ and other stakeholders’ preferences for outcomes of integrated care for multimorbidity: a discrete choice experiment in eight European countries. BMJ Open. 2020;10:e037547.

  48. Tervonen T, Gelhorn H, Sri Bhashyam S, Poon JL, Gries KS, Rentz A, et al. MCDA swing weighting and discrete choice experiments for elicitation of patient benefit-risk preferences: a critical assessment. Pharmacoepidemiol Drug Saf. 2017; 26(12):1483-1491.

  49. Marsh KD, Sculpher M, Caro JJ, et al. The use of MCDA in HTA: great potential, but more effort needed. Value Health. 2018;21:394–7.

    Article  Google Scholar 

  50. Chatfield MD, Brayne CE, Matthews FE. A systematic literature review of attrition between waves in longitudinal studies in the elderly shows a consistent pattern of dropout between differing studies. J Clin Epidemiol. 2005;58:13–9.

    Article  Google Scholar 

  51. Wodchis WP, Dixon A, Anderson GM, et al. Integrating care for older people with complex needs: key insights and lessons from a seven-country cross-case analysis. Int J Integr Care. 2015;15:e021.

    PubMed  PubMed Central  Google Scholar 

  52. Baxter S, Johnson M, Chambers D, et al. The effects of integrated care: a systematic review of UK and international evidence. BMC Health Serv Res. 2018;18:350.

    Article  Google Scholar 

  53. Laan W, Zuithoff NP, Drubbel I, Bleijenberg N, Numans ME, de Wit NJ, et al. Validity and reliability of the Katz-15 scale to measure unfavorable health outcomes in community-dwelling older people. J Nutr Health Aging. 2014;18:848–54.

    Article  CAS  Google Scholar 

  54. Berwick DM, Murphy JM, Goldman PA, Ware J, Barsky AJ, Weinstein MC. Performance of a five-item mental health screening test. Med Care. 1991;29:169–76.

    Article  CAS  Google Scholar 

  55. Cardol M, de Haan RJ, van den Bos GA, de Jong BA, de Groot IJ. The development of a handicap assessment questionnaire: the impact on participation and autonomy. Clin Rehabil. 1999;13:411–9.

    Article  CAS  Google Scholar 

  56. Coast J, Peters TJ, Natarajan L, Sproston K, Flynn T. An assessment of the construct validity of the descriptive system for the ICECAP capability measure for older people. Qual Life Res. 2008;17:967–76.

    Article  Google Scholar 

  57. Smith BW, Dalen J, Wiggins K, Tooley E, Christopher P, Bernard J. The brief resilience scale: assessing the ability to bounce back. Int J Behav Med. 2008;15:194–200.

    Article  Google Scholar 

  58. Pearlin LI, Schooler C. The structure of coping. J Health Soc Behav. 1978;19(1):2–21.

    Article  CAS  Google Scholar 

  59. Measures for Person Centred Coordinated Care. Person centred coordinated experience questionnaire (formerly know as the modified long term conditions questionnaire 6). ; Available at: . Accessed November, 2016.

  60. Uijen AA, Schellevis FG, van den Bosch WJ, Mokkink HG, van Weel C, Schers HJ. Nijmegen continuity questionnaire: development and testing of a questionnaire that measures continuity of care. J Clin Epidemiol. 2011;64:1391–9.

    Article  Google Scholar 

  61. McGuiness C, Sibthorpe B. Development and initial validation of a measure of coordination of health care. Int J Qual Health Care. 2003;15:309–18.

    Article  Google Scholar 

  62. Krska J, Morecroft CW, Rowe PH, Poole H. Measuring the impact of long-term medicines use from the patient perspective. Int J Clin Pharm. 2014;36(4):675–8.

    Article  Google Scholar 

Download references


The authors are grateful to two anonymous referees for helpful comments and suggestions. We are also thankful to many different organisations and individuals for their valuable support during study initiation and data collection process. Our sincere thanks go to Sigrid Askum and Torun Risnes from Norwegian association of local and regional authorities; Anders Vege from Norwegian Institute of Public Health; Anders Grimsmo from Norwegian Health Network; research assistants, Tord Lauvland Bjornevik and Nina Lunde; to all participating municipalities.


This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 634288. The funder had no role in the design of the study; in the collection, analysis, and interpretation of data; and in writing the manuscript. The content of this report reflects only the SELFIE groups’ views and the European Commission is not liable for any use that may be made of the information contained herein.

Author information

Authors and Affiliations



MKI, SR, MRvM and JEA contributed to the study conception and design. MKI wrote the first draft of the manuscript. All authors made substantial contributions to the revision of the manuscript for important intellectual content. SR obtained approvals. SR, RF, KJ and JEA acquired data. MKI conducted the statistical analyses. MKI and JEA contributed to the interpretation of the results. All authors reviewed, edited, read and approved the final version of the manuscript.

Corresponding author

Correspondence to M. Kamrul Islam.

Ethics declarations

Ethics approval and consent to participate

The Regional Committee for Medical and Health Research Ethics has approved this study (2017/632–3). The study was conducted in accordance with the provisions of the Declaration of Helsinki (1996) and Good Clinical Practice guidelines. All patients provided written informed consent before any study procedure was performed. The collected data have been stored on a separate SAFE server (“Clinton”) at the University of Bergen, with quality assured firewall and restricted, password-protected access. Only researchers associated with the project have access to these data. No one has access to the data in identifiable form.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Islam, M.K., Ruths, S., Jansen, K. et al. Evaluating an integrated care pathway for frail elderly patients in Norway using multi-criteria decision analysis. BMC Health Serv Res 21, 884 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: