Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Consensus on quality indicators to assess the organisation of palliative cancer and dementia care applicable across national healthcare systems and selected by international experts

BMC Health Services Research volume 14, Article number: 396 (2014) Cite this article

Abstract

Background

Large numbers of vulnerable patients are in need of palliative cancer and dementia care. However, a wide gap exists between the knowledge of best practices in palliative care and their use in everyday clinical practice. As part of a European policy improvement program, quality indicators (QIs) have been developed to monitor and improve the organisation of palliative care for patients with cancer and those with dementia in various settings in different European countries.

Method

A multidisciplinary, international panel of professionals participated in a modified RAND Delphi procedure to compose a set of palliative care QIs based on existing sets of QIs on the organisation of palliative care. Panellists participated in three written rounds, one feedback round and one meeting. The panel’s median votes were used to identify the final set of QIs.

Results

The Delphi procedure resulted in 23 useful QIs. These QIs represent key elements of the organisation of good clinical practice, such as the availability of palliative care teams, the availability of special facilities to provide palliative care for patients and their relatives, and the presence of educational interventions for professionals. The final set also includes QIs that are related to the process of palliative care, such as documentation of pain and other symptoms, communication with patients in need of palliative care and their relatives, and end-of-life decisions.

Conclusion

International experts selected a set of 23 QIs for the organisation of palliative care. Although we particularly focused on the organisation of cancer and dementia palliative care, most QIs are generic and are applicable for other types of diseases as well.

Peer Review reports

Background

Europe faces a huge challenge with a population that is rapidly aging in the coming decades. It is estimated that the incidence and prevalence of cancer will increase by about 20% and the prevalence of dementia will double before 2050 [1–4]. Although it concerns two totally different diseases with different care needs and disease trajectories, they do have a lot in common: they are often unnecessarily hospitalized, [5] have a high need for a multidisciplinary approach [6] and many suffer from symptoms which are partly the same, like pain [7, 8]. Higher survival rates of people with life-threatening and progressive chronic diseases will result in a larger number of patients that have multiple and complex health-threatening problems. Therefore, a growing number of patients will be in need of palliative care. However, a wide gap exists between the knowledge of best practice in palliative cancer and dementia care and its application in every day clinical practice [9].

As a first step in bridging this gap, it is important to assess current performance of palliative care in relation to its desired performance. Such an assessment of health care can be achieved by using quality indicators (QIs). QIs are evidence based, explicitly defined and measurable items that evaluate and describe structures, processes and outcomes of health care [10]. As such, they reflect the core elements of good clinical care. In day-to-day terms QIs can, for example, show whether pain is regularly being assessed using a validated tool; or whether a general practitioner is timely informed about a patient’s situation before or directly after discharge from hospital [11, 12]. QIs can help trace potential problems or confirm good quality of care and can be used to guide quality improvement processes [10]. They have been used effectively to assess and improve hospital care, [13] primary care, [14] and dementia care [15, 16]. Several studies have also developed QIs to improve the structures and process needed for the delivery of good quality palliative cancer or dementia care [11, 12, 17, 18]. However, these studies were performed five or more years ago, developed large sets of QIs, ranging from 56 to 142 QIs. Furthermore, none of these sets were widely implemented in everyday clinical practice.

The objective of this study was to integrate existing sets of QIs into one generic set that can be used to assess and improve the organisation of palliative care in different services and countries. The study was conducted within the framework of the European IMPACT project (IMplementation of quality indicators for PAlliative Care sTudy) [19].

Methods

A modified RAND Delphi procedure was used to develop a set of QIs, [20] which is considered an accepted methodology to develop QIs [10]. Typically, a RAND Delphi procedure consists of a written and a face-to-face round [20]. In this study, four written rounds and one face-to-face round were performed to reach consensus about the essential aspects regarding the organisation of palliative care.

Panellists

The IMPACT consortium consists of experts of the pan-European research group on detection and timely INTERvention in DEMentia (Interdem) and the European Association for Palliative Care (EAPC), all of whom are stakeholders in their country in palliative cancer and/or dementia care. The research team was invited to use their networks to purposefully select panellists for the modified RAND Delphi procedure. A key selection criteria was that the expert had to have extensive knowledge about palliative care, cancer care or dementia care. Additionally, experts had to be able to communicate in English (both verbally and non-verbally) and were planning to attend the 2012 Congress of the European Association of Palliative Care (EAPC) in Trondheim, Norway. Project partners nominated national and international experts in palliative cancer and dementia care. All nominated experts were approached via email (n = 50), forty experts from twelve countries agreed to participate in the modified RAND Delphi procedure (Table 1). All participants provided written informed consent. About half of them were experts in palliative cancer care and the other half in dementia care. Twenty-two panellists were active clinicians in this field (e.g. physician, nurse, psychologist, etc. currently involved in direct-patient care), the others were researchers.

Table 1 Panellists per country
Full size table

Selecting a preliminary set of QIs

A search for existing sets of QIs was conducted in PubMed. The search strategy was limited to English literature and consisted of various search terms that referred to subject-specific keywords describing palliative care (combined using “or”), as well as (“and”) the assessment of care using QIs (combined using “or”). Synonyms and medical subheading terms were used to fully include relevant literature (see Table 2).

Table 2 Overview of search terms
Full size table

Subsequently, references of key papers describing sets of QIs were hand searched. Additionally, consortium members of the IMPACT project were asked to nominate national and international sets of QIs on palliative cancer and dementia care they considered important. Two researchers (YE & JvRP) subsequently reviewed all of the identified QIs independently to determine if the QIs assessed the structure and process of palliative care and to structure them according to the domains of the recommended framework for the organisation of palliative care of the Council of Europe [21].

First written Delphi-round

For the first written Delphi-round (April 2012), panellists received a personal invitation for an online questionnaire. To reduce the large number of identified QIs, panellists were asked to nominate one QI per domain of palliative care [22]. Those QIs that were nominated by the panellists were included in the second round of the modified RAND Delphi procedure.

Second written Delphi-round

In the second round (May 2012), also via an online questionnaire, panellists were asked to rate QIs on a 9-point Likert scale for clarity (1 = not clear at all; 9 = very clear), usefulness (1 = not useful at all; 9 = very useful), to rephrase unclear and to add missing QIs. They were instructed to rate a QI high on usefulness if it: 1) corresponded with a basic quality level; 2) referred to a higher quality level that would be met only in very good practices; or 3) was associated with an innovative quality level which is exceptional at the moment, but could become the optimal quality level in the near future [23]. They were asked to give a low rating on usefulness if a proposed QI: 1) was too ambiguous or represented an unrealistically high quality level; 2) did not correspond with the material, social or cultural conditions of the situation in their country; or 3) was not in accordance with the regulations of palliative cancer and dementia care in their country [23].

Third interactive Delphi-round

A consensus meeting was organised during the EAPC Congress June 7, 2012 in Trondheim, Norway. The meeting was chaired by an independent researcher with the aim to reach consensus on the QIs on which there was disagreement or where the median score was between 4 and 6 in the second Delphi round. Participants received a rating sheet on which the median ratings of the second Delphi-round of all experts were visible. Participants were given 30 minutes to rate the adapted QIs for usefulness. Next, per QI, participants were asked to raise their hand if they had rated usefulness 6 or less. If at least nine (30%) participants raised their hand, [20] the QI was discussed until consensus was reached.

Fourth written Delphi-round

After the consensus-round, the remaining indicators were fed back to the panellist with the purpose to validate the changes that were made (September 2012).

Fifth written Delphi round

In the final step of the QI development process, QIs were operationalised by the research team into questions that could be used by healthcare professionals to assess their organisation of palliative care and identify areas for improvement. During this process, it appeared that some QIs were inappropriate or not measurable (e.g. too time consuming to answer them appropriately). All QIs were therefore rated for necessity by the IMPACT research team (October 2012), representing both clinicians and researchers that also took part in the modified RAND Delphi procedure. QIs that were considered not necessary after this round, were omitted from the list.

Analysis

QIs with a median rating on the usefulness scale of 7, 8 or 9 without disagreement were considered to have face validity. Disagreement was defined as: 30% or more of the panellists rated a single QI in the 1–3 tertile and more than 30% in the 7–9 tertile. QIs scored with a median of 1–3 without disagreement were not considered to have face validity. Because panellists had rated QIs high on usefulness, only QIs with median ratings of 8 or 9 were considered face valid for the second Delphi-round. Only QIs that were rated valid by all panellists were included in the final set [20].

Ethical considerations

The Medical Ethics Committee of the district Arnhem-Nijmegen has declared that this study doesn’t fall within the remit of the Medical Research Involving Human Subjects Act (WMO) (registration number 2012/075). This means that this study can be carried out without an approval by an accredited medical ethics committee.

Results

650 QIs were selected from literature [11, 16, 18, 22, 24–54]. After having assessed these QIs, 554 were excluded because they were not about the organisation of palliative care or because of overlap; the remaining 96 QIs were included in a preliminary set of QIs (Figure 1). Of the 40 experts invited as panellists, 25 (63%) participated. In the first Delphi round, 65 of the 96 QIs were selected and 13 missing QIs were suggested in an open question in which panellists were asked if they missed any relevant QI. This resulted in an adapted list of 78 QIs, which were included in the second Delphi round.

Figure 1
figure 1

Modified RAND Delphi procedure. 1Equivocal is defined as all QIs on which there was no agreement: e.g. QIs with 30% or more of ratings in both the 1–3 tertile and the 7–9 tertile and all indicators with a median rating in the 4–6 tertile. 2At the end of round three, panelists agreed that 11 QIs could be merged.

Full size image

In the second Delphi-round, 27 (67,5%) of the 40 invited panellists participated. Sixteen QIs were considered to have face validity, 28 were invalid, and six QIs were added to the list based on suggestions made by the panellists. The 40 QIs on which there was no agreement on, were included into the third Delphi round.

In the third Delphi-round, a consensus meeting, 29 (72,5%) of the 40 panellists participated. After having rated 40 QIs, one was excluded and 10 were discussed. Of the QIs that were discussed, six were accepted and four were excluded. Panellists also agreed to merge 11 QIs. Round three therefore resulted in 24 accepted QIs. The total list of QIs (QIs considered to have face validity in round two and three) comprised 40 QIs.

In the fourth Delphi-round, panellist provided feedback to the remaining set of QIs. This resulted in minor linguistic changes and the exclusion of three QIs because they were considered inappropriate by the majority of panellists. The resulting set of QIs, therefore, consisted of 37 QIs. This list was critically assessed by members of the IMPACT consortium for their necessity (round five). Fourteen QIs were considered to be overlapping, inappropriate, or not measurable in palliative care. The final list of QIs, therefore, consisted of 23 QIs, covering seven domains (Table 3). Key findings can be summarized as follows:

Table 3 Overview of QIs
Full size table

Access to palliative care

The availability of a dedicated palliative care team was considered important by almost all panellists in the Delphi procedure. They explicitly stated that palliative care services should not only be available during office hours, but at all times (day, evening, night and weekends). Furthermore, specific elements of palliative care were considered important, such as the availability of opioids and anticipatory medications for symptom control, as well as the availability of bereavement support.

Almost all panellists also rated important the accessibility of the medical record to health care professionals, timely transfer of information between settings (including when transferring or discharging patients). Panellists also considered an assigned contact person, who maintains regular contact with patients and their families, useful.

Infrastructure

The infrastructure of the place where palliative care is provided, such as a single bed hospital room, was rated important. Access to equipment (such as anti-decubitus mattresses, suction equipment, etc.), required to provide palliative care, was considered important. Panellists also rated high consensus for facilities for relatives to visit, stay overnight, and a private area for saying goodbye to the deceased. There was no agreement on QIs that aimed to control waiting time or waiting list, i.e. these aspects were not considered to be unique for palliative care and therefore not important.

Assessment tools

Regular assessment of pain and other symptoms was rated as a valid quality criterion, though it was commented that a validated instrument might not always be available, particularly for specific patient groups (e.g. for persons with advanced dementia).

Personnel

There was agreement on the need for a multidisciplinary team, which should consist of at least a physician and nurse, and have access to a range of supporting disciplines, such as: physiotherapist, psychologist, occupational therapist, social worker, dietician, and chaplain. Panellists also rated a regular multidisciplinary team meeting important.

Documentation of clinical data

Panellists recognized the importance of having a well-structured medical record. However, a QI about the structure of the medical record was not considered important by the panellists. Panellists only considered the inclusion of a medication regimen in the medical record important. They also considered a timely assessment (within 48 hours) of pain and other symptoms, psychosocial and spiritual needs, patient preferences, wishes and needs, and the patient's capacity to be involved in the decision making process as important.

Furthermore, almost all panellists rated the documentation of communication on the medical condition, goals of treatment, physical, psychosocial, and spiritual needs of the patient and their relatives, intention to return home, advanced directive, and end-of-life decisions as important.

Quality and safety

Panellists rated the QI about assessing the experiences of care givers with the palliative care service important. Secondly, a QI about the quality of care, assessing the use of an end-of-life care pathway within the last three days of life, was also considered useful.

Education

QIs about the staff’s learning objectives and a program for specialised and/or continuing medical education about the physical, psychosocial, and spiritual needs of a patient in need for palliative care were not rated important. There was also no agreement on disease-specific education for staff members, but panellists considered palliative care training specified to the professiona’ls background important.

Discussion

With the help of a modified five-round RAND Delphi-procedure, we were able to develop an internationally validated set of QIs for the organisation of palliative care with high face validity as judged by experts in the field of cancer and dementia care. The final set provides 23 quality aspects regarding the accessibility of the service, its infrastructure, the use of symptom assessment tools, management of personnel, documentation of clinical data, quality of care, and education. Of these 23 QIs, one was identical to the original one (Family members and friends are able to visit the dying person without restrictions of visiting hours), [12] two were new (Family and caregiver experiences of the palliative care service are assessed/evaluated/recorded and An end-of-life care pathway (such as the Liverpool Care Pathway) was used for the last 3 days of life of a person in need of palliative care) and 20 were rephrased QIs. Panellist agreed not to formulate disease-specific QIs for the organisation of palliative care, since our set of QIs provide information about the organisation of services and not about the care provision itself. This might explain why so many QIs were rated face-valid for as well the organisation of cancer as dementia palliative care: regarding aspects as access to and 24 h availability of specialist palliative care or transferring information between settings, the specific condition of the patient (advanced cancer, dementia, or even COPD or heart failure) is not relevant, making our set of QIs much more broadly applicable. Therefore, these QIs are generically applicable and can be used in different settings. Thereby, they can also be used for (cross-)national comparisons and to identify best practices regarding the organisation of palliative care in other services and countries.

In the recent literature several sets of QIs for palliative care have been identified [11, 12, 17, 38, 55]. For example, Pastrana et al. used a nominal group technique to identify indicators for the assessment and evaluation of palliative care [38]. However, they primarily focused on the German health care system, which makes this set difficult to apply in an international context [38]. Pasman et al. conducted a literature review, and identified 142 QIs in 16 studies [11]. However, this set also has not been developed within an international context and it does not focus on the organisation of palliative care [11]. An update of this review, published in 2013, included a further 187 QIs, bringing the total to 326 QIs, with still few QIs about the organisation of palliative care [55]. Around the same time, Woitha et al. developed a set of 56 QIs [12, 17]. Woitha et al. conducted two written Delphi rounds, leaving little room for discussion, while we conducted a consensus round with extensive opportunity to discuss the QIs. Secondly, they included professionals from different European countries only, while we also included professionals from countries such as Canada, Australia and the USA, making the set of QIs presented here globally applicable. Thirdly, they focused on the organisation of palliative care in general and did not specifically consider the organisation of palliative care for patients with dementia, like was done in the present study.

A recently published White paper defining optimal palliative care in older people with dementia, [56] described several recommendations on palliative care treatment for persons with dementia. All of their recommendations that can be translated to the organisation of care, like the use of assessment tools, multidisciplinary meetings, bereavement support and about specialist palliative care teams, are represented in our QI set.

Another quality indicator, suggested by the World Health Organization as part of the framework programme on non-communicable diseases has been the focus of attention recently. This QI is being proposed to describe access to palliative care by assessing morphine equivalent consumption per death from cancer [57, 58]. However, this QI has been criticized as it might have provided flawed information due to inaccuracies in the underlying data base and the unavailability of national cancer registries [59]. Instead of assessing palliative care on a global level, we aim to assess whether palliative care services meet a basic quality level or higher quality level that would be met only in very good practices. Our set of QIs can therefore be used as internal QIs by health care providers (professionals and managers) to monitor and improve their service. They can also be used to describe and rank services according to performance, but this should not lead to a quality rating, as there may be good reasons for the differences in performance with the QI (e.g. different organisational structure).

Using our QIs as an external quality assessment tool will therefore make them unfit for their task [60]. Berwick et al. summarized this as ‘measuring for improvement is not measuring for judgement’ [61]. An ongoing intervention in 40 palliative care services in Europe, including hospitals, hospices, nursing homes and primary care settings, performed as part of the IMPACT project, in which this set of QIs is used as starting point to assess the organisation of palliative care, will evaluate the feasibility and discriminatory power of the QIs in relation to improving the organisation of palliative care in the participating services.

Strengths and limitations

Strengths of this study were that we used a large international group of panellists for our Delphi procedure, who were actively involved in palliative care (such as members of the European Association for Palliative Care). Secondly, by organising our consensus meeting during the EAPC Congress in Trondheim, Norway, key persons active in the fields of palliative care and dementia care were able to contribute extensively to the discussion of the Delphi procedure. Thirdly, the multidisciplinary character of palliative care was represented by the panellists (e.g. physicians, nurses, psychologists, etc.) involved in the Delphi procedure. Furthermore, half of them were professionals active in dementia care. Fourth, combining QIs for the organisation of services that provide care to palliative patients with cancer and those that provide such care to patients with dementia is unique. Our QIs can therefore be used in different settings.

A limitation of this study is that this set of QIs might not be comprehensive. Because an international, generic set of QIs was developed, some QIs that were important in only one or a few countries were excluded from the list. For that reason, important national or setting-specific QIs must be added when the set is used in a specific country. Secondly, this set of QIs is only related to the organisation of palliative care. Outcome and patient-related outcome measures were not included because they address a distinct purpose in measuring quality of palliative care. Thirdly, participants of the modified RAND Delphi procedure were selected because of their knowledge about palliative care, cancer care or dementia care. Because some experts of two large European networks (EAPC and InterDem) are part of the IMPACT consortium, they were also selected as participant for the modified RAND Delphi procedure (n = 18). Although not all countries were represented (like France) and others were overrepresented (like the Netherlands), the experts covered 12 countries from three continents, covering different health care systems and types of organisation of palliative care. Pilot testing the set of QIs in those countries and continents that were not represented in this study will reveal whether they are applicable in these countries too. Fourthly, unfortunately, there were no patient representatives involved as panellist. Testing the final set of QIs will therefore also have to incorporate their views on the basic quality level or higher quality level that would be met only in very good services.

Conclusion

International experts selected a set of 23 QIs for the organisation of palliative care that can be implemented in daily practice in order to demonstrate that organisations are providing high quality and effective palliative care or to identify areas for improvement.

References

  1. Lynn J, Adamsom D: Living Well at the End of Life: Adapting Health Care to Serious Chronic Illness in Old Age. Rand Health White Paper WP137 edition. 2003, Santa Monica: RAND

    Google Scholar 

  2. van den Beuken-van Everdingen MH, de Rijke JM, Kessels AG, Schouten HC, van Kleef M, Patijn J: High prevalence of pain in patients with cancer in a large population-based study in The Netherlands. Pain. 2007, 132: 312-320. 10.1016/j.pain.2007.08.022.

    Article  PubMed  Google Scholar 

  3. EuroCode: Dementia in Europe Yearbook 2008. 2008, Luxembourg: Alzheimer Europe

    Google Scholar 

  4. Coebergh JWW, van de Poll-Franse LV, Alers JC, van de Berg FA, van Duijn NP: Kanker in Nederland. Trends, prognoses en implicaties voor de zorgvraag. [Cancer in The Netherlands. Trends, prognosis, and implications for the demand of care - in Dutch]. 2004, Oisterwijk: KWF Kankerbestrijding, 53-61.

    Google Scholar 

  5. De Roo ML, Miccinesi G, Onwuteaka-Philipsen BD, Van Den Noortgate N, Van den Block L, Bonacchi A, Donker GA, Lozano Alonso JE, Moreels S, Deliens L, Francke AL: Actual and preferred place of death of home-dwelling patients in four European countries: making sense of quality indicators. PLoS One. 2014, 9: e93762-10.1371/journal.pone.0093762.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Vissers KC, van den Brand MW, Jacobs J, Groot M, Veldhoven C, Verhagen C, Hasselaar J, Engels Y: Palliative medicine update: a multidisciplinary approach. Pain Pract. 2013, 13: 576-588. 10.1111/papr.12025.

    Article  PubMed  Google Scholar 

  7. Teunissen SC, Wesker W, Kruitwagen C, de Haes HC, Voest EE, de Graeff A: Symptom prevalence in patients with incurable cancer: a systematic review. J Pain Symptom Manage. 2007, 34: 94-104. 10.1016/j.jpainsymman.2006.10.015.

    Article  PubMed  Google Scholar 

  8. Sachs GA, Shega JW, Cox-Hayley D: Barriers to excellent end-of-life care for patients with dementia. J Gen Intern Med. 2004, 19: 1057-1063. 10.1111/j.1525-1497.2004.30329.x.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003, 362: 1225-1230. 10.1016/S0140-6736(03)14546-1.

    Article  PubMed  Google Scholar 

  10. Campbell SM, Braspenning J, Hutchinson A, Marshall MN: Research methods used in developing and applying quality indicators in primary care. BMJ. 2003, 326: 816-819. 10.1136/bmj.326.7393.816.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Pasman HR, Brandt HE, Deliens L, Francke AL: Quality indicators for palliative care: a systematic review. J Pain Symptom Manage. 2009, 38: 145-156. 10.1016/j.jpainsymman.2008.07.008.

    Article  PubMed  Google Scholar 

  12. Woitha K, Beek K, Ahmed N, Hasselaar J, Mollard JM, Colombet I, Radbruch L, Vissers K, Engels Y: Development of a set of process and structure indicators for palliative care: the Europall project. BMC Health Serv Res. 2012, 12: 381-10.1186/1472-6963-12-381.

    Article  PubMed  PubMed Central  Google Scholar 

  13. de Vos M, Graafmans W, Kooistra M, Meijboom B, Van Der Voort P, Westert G: Using quality indicators to improve hospital care: a review of the literature. Int J Qual Health Care. 2009, 21: 119-129. 10.1093/intqhc/mzn059.

    Article  PubMed  Google Scholar 

  14. Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, Elwyn G, Vodopivec-Jamsek V, Gerlach FM, Samuelson M, Grol R: Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract. 2006, 23: 137-147.

    Article  PubMed  Google Scholar 

  15. Draskovic I, Vernooij-Dassen M, Verhey F, Scheltens P, Rikkert MO: Development of quality indicators for memory clinics. Int J Geriatr Psychiatry. 2008, 23: 119-128. 10.1002/gps.1848.

    Article  PubMed  Google Scholar 

  16. Vasse E, Moniz-Cook E, Rikkert MO, Cantegreil I, Charras K, Dorenlot P, Fumero G, Franco M, Woods B, Vernooij-Dassen M: The development of quality indicators to improve psychosocial care in dementia. Int Psychogeriatr. 2012, 24: 921-930. 10.1017/S1041610211002523.

    Article  PubMed  Google Scholar 

  17. Woitha K, Van Beek K, Ahmed N, Jaspers B, Mollard JM, Ahmedzai SH, Hasselaar J, Menten J, Vissers K, Engels Y: Validation of quality indicators for the organization of palliative care: a modified RAND Delphi study in seven European countries (the Europall project). Palliat Med. 2013, 28: 121-129.

    Article  PubMed  Google Scholar 

  18. Claessen SJ, Francke AL, Belarbi HE, Pasman HR, van der Putten MJ, Deliens L: A new set of quality indicators for palliative care: process and results of the development trajectory. J Pain Symptom Manage. 2011, 42: 169-182. 10.1016/j.jpainsymman.2010.10.267.

    Article  PubMed  Google Scholar 

  19. van Riet Paap J, Engels Y, Iliffe S, Radbruch L, Kaasa S, Chattat R, Vissers K, Vernooij-Dassen M: Improving the organization of palliative care by implementing quality indicators and national and setting-specific interventions: study protocol of the IMPACT project. Progress in Palliative Care. 2014, 22: 201-205. 10.1179/1743291X13Y.0000000080.

    Article  Google Scholar 

  20. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lázaro P, van het Loo M, McDonnell J, Vader JP, Kahan JP: The RAND/UCLA Appropriateness Method User’s Manual. 2001, Santa Monica, CA: RAND

    Google Scholar 

  21. Council of Europe: Recommendation Rec (2003) 24 of the Committee of Ministers to member states on the organisation of palliative care. 2003, http://www.coe.int/t/dg3/health/Source/Rec%282003%2924_en.pdf.

    Google Scholar 

  22. Engels Y, van Beek K, Woitha K, Ahmed N, Ahmedzai S, Gómez-Batiste X, Jaspers B, Leppert W, Menten J, Mollard JM, Vissers K: Development of an indicator set for the organisation of palliative care in Europe. Assessing Organisations to Improve Palliative Care in Europe. Edited by: Ahmedzai S, Gómez-Batiste X, Engels Y, Hasselaar J, Jaspers B, Leppert W, Menten J, Mollard JM, Vissers K. 2010, Nijmegen: Vantilt Publishers, 73-112.

    Google Scholar 

  23. Engels Y, Campbell S, Dautzenberg M, van den Hombergh P, Brinkmann H, Szecsenyi J, Falcoff H, Seuntjens L, Kuenzi B, Grol R: Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract. 2005, 22: 215-222. 10.1093/fampra/cmi002.

    Article  PubMed  Google Scholar 

  24. D'Angelo D, Mastroianni C, Vellone E, Alvaro R, Casale G, Latina R, De Marinis MG: Palliative care quality indicators in Italy. What do we evaluate?. Support Care Cancer. 2011, 20: 1983-1989.

    Article  PubMed  Google Scholar 

  25. Feil DG, MacLean C, Sultzer D: Quality indicators for the care of dementia in vulnerable elders. J Am Geriatr Soc. 2007, 55 (Suppl 2): S293-301.

    Article  PubMed  Google Scholar 

  26. de Vries N, Spijker A, Vasse E, Jansen I, Koopmans R, Vernooij-Dassen M, Schoonhoven L: Ketenindicatoren voor dementie. 2009, Nijmegen: Scientific Institute for Quality of Healthcare (IQ healthcare)

    Google Scholar 

  27. Kroger E, Tourigny A, Morin D, Cote L, Kergoat MJ, Lebel P, Robichaud L, Imbeault S, Proulx S, Benounissa Z: Selecting process quality indicators for the integrated care of vulnerable older adults affected by cognitive impairment or dementia. BMC Health Serv Res. 2007, 7: 195-10.1186/1472-6963-7-195.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Miyashita M, Nakamura A, Morita T, Bito S: Identification of quality indicators of end-of-life cancer care from medical chart review using a modified Delphi method in Japan. Am J Hosp Palliat Care. 2008, 25: 33-38. 10.1177/1049909107307376.

    Article  PubMed  Google Scholar 

  29. Miyashita M, Morita T, Ichikawa T, Sato K, Shima Y, Uchitomi Y: Quality indicators of end-of-life cancer care from the bereaved family members' perspective in Japan. J Pain Symptom Manage. 2009, 37: 1019-1026. 10.1016/j.jpainsymman.2008.05.015.

    Article  PubMed  Google Scholar 

  30. Perry M, Draskovic I, van Achterberg T, van Eijken M, Lucassen P, Vernooij-Dassen M, Olde Rikkert M: Development and validation of quality indicators for dementia diagnosis and management in a primary care setting. J Am Geriatr Soc. 2010, 58: 557-563. 10.1111/j.1532-5415.2010.02726.x.

    Article  PubMed  Google Scholar 

  31. Raijmakers N, Galushko M, Domeisen F, Beccaro M, Lundh Hagelin C, Lindqvist O, Popa-Velea O, Romotzky V, Schuler S, Ellershaw J, Ostgathe C: Quality indicators for care of cancer patients in their last days of life: literature update and experts' evaluation. J Palliat Med. 2012, 15: 308-316.

    PubMed  Google Scholar 

  32. Saliba D, Solomon D, Rubenstein L, Young R, Schnelle J, Roth C, Wenger N: Feasibility of quality indicators for the management of geriatric syndromes in nursing home residents. J Am Med Dir Assoc. 2005, 6: S50-59. 10.1016/j.jamda.2005.03.023.

    Article  PubMed  Google Scholar 

  33. Sato K, Miyashita M, Morita T, Sanjo M, Shima Y, Uchitomi Y: Reliability assessment and findings of a newly developed quality measurement instrument: quality indicators of end-of-life cancer care from medical chart review at a Japanese regional cancer center. J Palliat Med. 2008, 11: 729-737. 10.1089/jpm.2007.0227.

    Article  PubMed  Google Scholar 

  34. Lorenz KA, Rosenfeld K, Wenger N: Quality indicators for palliative and end-of-life care in vulnerable elders. J Am Geriatr Soc. 2007, 55 (Suppl 2): S318-326.

    Article  PubMed  Google Scholar 

  35. Mularski RA, Curtis JR, Billings JA, Burt R, Byock I, Fuhrman C, Mosenthal AC, Medina J, Ray DE, Rubenfeld GD, Schneiderman LJ, Treece PD, Truog RD, Levy MM: Proposed quality measures for palliative care in the critically ill: a consensus from the Robert Wood Johnson Foundation Critical Care Workgroup. Crit Care Med. 2006, 34: S404-411. 10.1097/01.CCM.0000242910.00801.53.

    Article  PubMed  Google Scholar 

  36. Nelson JE, Mulkerin CM, Adams LL, Pronovost PJ: Improving comfort and communication in the ICU: a practical new tool for palliative care performance measurement and feedback. Qual Saf Health Care. 2006, 15: 264-271. 10.1136/qshc.2005.017707.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Ostgathe C, Voltz R: Quality indicators in end-of-life care. Curr Opin Support Palliat Care. 2010, 4: 170-173. 10.1097/SPC.0b013e32833add10.

    Article  PubMed  Google Scholar 

  38. Pastrana T, Radbruch L, Nauck F, Hover G, Fegg M, Pestinger M, Ross J, Krumm N, Ostgathe C: Outcome indicators in palliative care–how to assess quality and success. Focus group and nominal group technique in Germany. Support Care Cancer. 2010, 18: 859-868. 10.1007/s00520-009-0721-4.

    Article  PubMed  Google Scholar 

  39. Yabroff KR, Mandelblatt JS, Ingham J: The quality of medical care at the end-of-life in the USA: existing barriers and examples of process and outcome measures. Palliat Med. 2004, 18: 202-216. 10.1191/0269216304pm880oa.

    Article  PubMed  Google Scholar 

  40. Wenger NS, Rosenfeld K: Quality indicators for end-of-life care in vulnerable elders. Ann Intern Med. 2001, 135: 677-685. 10.7326/0003-4819-135-8_Part_2-200110161-00006.

    Article  CAS  PubMed  Google Scholar 

  41. Twaddle ML, Maxwell TL, Cassel JB, Liao S, Coyne PJ, Usher BM, Amin A, Cuny J: Palliative care benchmarks from academic medical centers. J Palliat Med. 2007, 10: 86-98. 10.1089/jpm.2006.0048.

    Article  PubMed  Google Scholar 

  42. Seow H, Snyder CF, Mularski RA, Shugarman LR, Kutner JS, Lorenz KA, Wu AW, Dy SM: A framework for assessing quality indicators for cancer care at the end of life. J Pain Symptom Manage. 2009, 38: 903-912. 10.1016/j.jpainsymman.2009.04.024.

    Article  PubMed  Google Scholar 

  43. Seow H, Snyder CF, Shugarman LR, Mularski RA, Kutner JS, Lorenz KA, Wu AW, Dy SM: Developing quality indicators for cancer end-of-life care: proceedings from a national symposium. Cancer. 2009, 115: 3820-3829. 10.1002/cncr.24439.

    Article  PubMed  Google Scholar 

  44. Johannsen P, Jorgensen K, Korner A, Elmo EG, Lauesen LB, Utzon J: Development of a dementia assessment quality database. Aging Ment Health. 2011, 15: 40-46. 10.1080/13607863.2010.508769.

    Article  PubMed  Google Scholar 

  45. Grunfeld E, Urquhart R, Mykhalovskiy E, Folkes A, Johnston G, Burge FI, Earle CC, Dent S: Toward population-based indicators of quality end-of-life care: testing stakeholder agreement. Cancer. 2008, 112: 2301-2308. 10.1002/cncr.23428.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Earle CC, Park ER, Lai B, Weeks JC, Ayanian JZ, Block S: Identifying potential indicators of the quality of end-of-life cancer care from administrative data. J Clin Oncol. 2003, 21: 1133-1138. 10.1200/JCO.2003.03.059.

    Article  PubMed  Google Scholar 

  47. Earle CC, Neville BA, Landrum MB, Souza JM, Weeks JC, Block SD, Grunfeld E, Ayanian JZ: Evaluating claims-based indicators of the intensity of end-of-life cancer care. Int J Qual Health Care. 2005, 17: 505-509. 10.1093/intqhc/mzi061.

    Article  PubMed  Google Scholar 

  48. Clarke EB, Curtis JR, Luce JM, Levy M, Danis M, Nelson J, Solomon MZ: Quality indicators for end-of-life care in the intensive care unit. Crit Care Med. 2003, 31: 2255-2262. 10.1097/01.CCM.0000084849.96385.85.

    Article  PubMed  Google Scholar 

  49. Keay TJ, Fredman L, Taler GA, Datta S, Levenson SA: Indicators of quality medical care for the terminally ill in nursing homes. J Am Geriatr Soc. 1994, 42: 853-860.

    Article  CAS  PubMed  Google Scholar 

  50. Ministere de la Sante et de la Protection Sociale: Guide des bonnes pratiques d’une démarche palliative en établissements. 2004, Paris: Ministere de la Sante et de la Protection Sociale – Direction de l’Hospitalisation et de l’Organisation des Soins

    Google Scholar 

  51. Directorate of Health: National action program with guidelines for palliation in cancer care [report in Norwegian]. 2010, Oslo: Directorate of Health

    Google Scholar 

  52. Quality indicators. [http://www.helsedirektoratet.no/kvalitet-planlegging/kvalitetsindikatorer/somatikk/Sider/default.aspx]

  53. Norwegian Association for Palliative Medicine: Standard for palliation [report in Norwegion]. 2004, Oslo: The Norwegian Medical Association

    Google Scholar 

  54. Medicine NAfP: Quality indicators in palliative medicine. Letter to The Norwegian Medical Association’s Quality Improvement Committee [letter in Norwegion]. 2004

    Google Scholar 

  55. De Roo ML, Leemans K, Claessen SJ, Cohen J, Pasman HR, Deliens L, Francke AL: Quality indicators for palliative care: update of a systematic review. J Pain Symptom Manage. 2013, 46: 556-572. 10.1016/j.jpainsymman.2012.09.013.

    Article  PubMed  Google Scholar 

  56. van der Steen JT, Radbruch L, Hertogh CM, de Boer ME, Hughes JC, Larkin P, Francke AL, Junger S, Gove D, Firth P, Koopmans RT, Volicer L: White paper defining optimal palliative care in older people with dementia: a Delphi study and recommendations from the European association for palliative care. Palliat Med. 2013, 28: 197-209.

    Article  PubMed  Google Scholar 

  57. Payne S, Leget C, Peruselli C, Radbruch L: Quality indicators for palliative care: debates and dilemmas. Palliat Med. 2012, 26: 679-680. 10.1177/0269216312450123.

    Article  PubMed  Google Scholar 

  58. Junger S, Brearley S, Payne S, Mantel-Teeuwisse AK, Lynch T, Scholten W, Radbruch L: Consensus building on access to controlled medicines: a four-stage Delphi consensus procedure. J Pain Symptom Manage. 2013, 46: 897-910. 10.1016/j.jpainsymman.2013.01.014.

    Article  PubMed  Google Scholar 

  59. De Lima L, Wenk R, Krakauer E, Ferris F, Bennett M, Murray S, Bruera E, Radbruch L: Global framework for noncommunicable diseases: how can we monitor palliative care?. J Palliat Med. 2013, 16: 226-229. 10.1089/jpm.2012.0493.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Klazinga N, Stronks K, Delnoij D, Verhoeff A: Indicators without a cause. Reflections on the development and use of indicators in health care from a public health perspective. Int J Qual Health Care. 2001, 13: 433-438. 10.1093/intqhc/13.6.433.

    Article  CAS  PubMed  Google Scholar 

  61. Berwick DM: The NHS's 50 anniversary. Looking forward. The NHS: feeling well and thriving at 75. BMJ. 1998, 317: 57-61. 10.1136/bmj.317.7150.57.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Pre-publication history

Download references

Acknowledgement

This study is part of the larger IMPACT project (IMplementation of quality indicators in PAlliative Care sTudy). The research leading to these results has received funding from the European Union's Seventh Framework Programme FP7/2007-2013 under grant agreement n°258883.

The set of palliative care QIs could not have been established without the support and extensive feedback received from a large international panel of professionals in palliative care for patients with cancer and dementia, both in research as in clinical practice. Therefore, we would like to thank (in alphabetical order): Sam Ahmedzai, Marcel Arcand, Lieve van den Block, Rabih Chattat, Jacqueline Crowther, Rose-Marie Dröes, Yvonne Engels, Anneke Francke, Manuel A. Franco, Marie-José Gijsberts, Dagny F. Haugen, Marianne Hjermstad, Josephine Hockley, Steve Iliffe, Birgit Jaspers, Stein Kaasa, Raymond Koopmans, Carlo Leget, Wojciech Leppert, Jill Manthorpe, Franka Meiland, Johan Menten, Esme Moniz-Cook, Nele van den Noortgate, Christophe Ostgathe, Deborah Parker, Tania Pastrana, Sophie Pautex, Lukas Radbruch, Daniel Ramos, Claud Regnard, Julia Riley, Roman Rolke, Elizabeth Sampson, Rob van de Sande, Jenny van der Steen, Florian Strasser, Joan Teno, Myrra Vernooij-Dassen and Kris Vissers. We would especially like to acknowledge Marieke Groot for her contributions to the Delphi procedure by chairing the consensus meeting. Furthermore, we thank all participating researchers involved in the IMPACT consortium for their support.

Author information

Authors and Affiliations

  1. Scientific Institute for Quality of Healthcare (IQ healthcare), Radboud university medical center, P.O. Box 9101, 6500, HB, Nijmegen, The Netherlands

    Jasper van Riet Paap & Myrra Vernooij-Dassen

  2. Kalorama Foundation, Nijmegen, The Netherlands

    Myrra Vernooij-Dassen

  3. Department of General practice & Elderly care medicine and Department of Psychiatry, VU University Medical Centre, P.O. Box 7057, 1081, BT, Amsterdam, The Netherlands

    Rose-Marie Dröes

  4. Department of Palliative Medicine, Universitätsklinikum Bonn, and Department of Palliative Care, Malteser Hospital Bonn, Sigmund-Freud-Street 25, 53127, Bonn, Rhein-Sieg, Germany

    Lukas Radbruch

  5. Department of Anaesthesiology, Pain and Palliative Medicine, Radboud university medical center, P.O. Box 9101, 6500, HB, Nijmegen, The Netherlands

    Kris Vissers & Yvonne Engels

  6. the IMPACT research team, Spain

    Jasper van Riet Paap

Authors
  1. Jasper van Riet Paap
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Myrra Vernooij-Dassen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rose-Marie Dröes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lukas Radbruch
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kris Vissers
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yvonne Engels
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jasper van Riet Paap.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

YE and MVD are responsible for the design of the study and provided extensive feedback to multiple versions of the manuscript. JvRP is responsible for the analysis and manuscript. KV was involved in the organisation of the modified RAND Delphi procedure and provided extensive feedback to multiple versions of the manuscript. LR is work package leader in the IMPACT project. As such, LR is responsible for the intervention study in which the QIs are tested for their feasibility. Therefore, LR provided feedback to multiple versions of the manuscript. RMD provided feedback to multiple versions of the manuscript. All authors approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

van Riet Paap, J., Vernooij-Dassen, M., Dröes, RM. et al. Consensus on quality indicators to assess the organisation of palliative cancer and dementia care applicable across national healthcare systems and selected by international experts. BMC Health Serv Res 14, 396 (2014). https://doi.org/10.1186/1472-6963-14-396

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1472-6963-14-396

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us