Study design
To develop a set of relevant KIs and QIs, a RAND modified Delphi technique [16] was used with a predefined number of rounds to stop the Delphi process and a threshold value for consensus [17]. After selection of an extensive set of KIs and QIs from the literature and composition of a multidisciplinary expert panel, three anonymous questionnaire rounds and one physical meeting were performed to achieve panel consensus about the relevance of the proposed KIs and relevance and feasibility of the proposed QIs. Questionnaires were conducted through LimeSurvey®, an open-source software tool to conduct online surveys [18]. E-mail reminders were sent at 2 weeks following the initial email of each round. The consensus procedure took place between March and June 2016.
Composition of expert panel
The objective was to generate a multidisciplinary Delphi panel of physicians and nurses involved in the donation process after brain death in Belgium in order to guarantee relevance for clinical practice and generalizability of results [17, 19]. The main eligibility criteria consisted of a longstanding experience in the field of organ donation, preferably for a minimum of 10 years, and a minimum of 3 organ donors throughout 2015 in the donor hospital, in which the expert was professionally active.
All Belgian donor coordinators (n = 196), the board members of the Belgian Society of Intensive Care Medicine (n = 8), and the members of the Transplant Coordinators Section (n = 28) and the Belgian Organ Procurement Committee (n = 19) of the Belgian Transplantation Society were invited to join this study by an information letter (Additional file 1) sent by e-mail by the first author (PH), describing the criteria required to be involved in this Delphi panel.
Selection of key interventions and quality indicators
The selection of KIs and QIs consisted of 8 steps: (1) Delphi questionnaire preparation with extraction of KIs and QIs, (2) first Delphi round, (3) data analysis of the first round, (4) second Delphi round, (5) data analysis of the second round, (6) third Delphi round, (7) data analysis of the third round, and (8) physical consensus meeting.
Step 1: Delphi questionnaire preparation with extraction of key interventions and quality indicators
To develop a Delphi questionnaire including all possible relevant and feasible KIs and QIs, an extensive literature review was conducted by the first author (PH). For the review of guidelines on the management of a potential DBD, the following resources were explored: (I) Websites of national European transplantation organizations or societies: Agence de la biomédecine, British Transplantation Society, Deutsche Stiftung Organtransplantation, Nederlandse Transplantatie Stichting, NHS Blood and Transplant, and Organización Nacional de Trasplantes; (II) Websites of European transplantation or intensive care medicine organizations or societies: European Directorate for the Quality of Medicines and HealthCare, European Society of Intensive Care Medicine, European Society of Organ Transplantation, Eurotransplant, and Scandiatransplant; (III) Websites of international transplantation societies: International Liver Transplantation Society, International Transplant Nurses Society, The International Society for Heart & Lung Transplantation, and The Transplantation Society; (IV) Public resources for evidence-based clinical practice guidelines: Guidelines International Network, National Guideline Clearinghouse, National Institute for Health and Clinical Excellence, and Scottish Intercollegiate Guidelines Network; (V) Process flow diagrams based on evidence-based medicine: Map of Medicine and National Institute for Health and Clinical Excellence; and (VI) Electronic databases: MEDLINE, CINAHL and EMBASE.
For the first 5 resources, the following search terms were used: ‘organ donation’ and ‘brain death’. For the electronic database MEDLINE, the Medical Subject Headings (MeSH) terms ‘brain death’, ‘donor selection’, ‘tissue and organ harvesting’, ‘tissue and organ procurement’ or ‘tissue donors’ were used in combination with ‘guideline’ or ‘practice guideline’, both as publication type. The strategy was translated for the other databases. Search limit parameters included: (I) published between 2009 and 2015, and (II) written in English, Dutch or French.
Only few of these guidelines included KIs for donor management [20,21,22]. Therefore, an additional search was performed in the electronic databases, MEDLINE, CINAHL, EMBASE and The Cochrane Library, to include recent review articles, using the search term ‘donor management’. In addition to the QIs listed in the guidelines and review articles, the QIs identified in the organ donation process of the Organ Donation European Quality System (ODEQUS) project were also analyzed. These were developed by a consortium involving associated and collaborating partners from 16 European countries [23].
A two-phase screening evaluation of publications from these resources was applied. In the first phase, publications were appraised for relevance based on appropriateness of the title and abstract. If relevance was unclear, or if the abstract was unavailable, the full text of these publications was assessed. In the second phase, the full text of the selected guidelines or process flow diagrams were reviewed. Following inclusion criteria were applied: (I) descriptions of KIs or QIs regarding an adult patient with a devastating brain injury or lesion with evolution to imminent brain death until post procurement, and (II) underpinning by in-text references of evidence to support their practice. The guidelines selected after full text review were appraised using the validated AGREE II-Global Rating Scale (AGREE II-GRS) quality assurance tool for clinical practice guidelines. This instrument consists of 4 items assessing the quality of guideline reporting. Each item is scored on a seven-point scale [24]. Guideline quality was independently rated by three reviewers (PH, KV and PF). A consensus meeting was held between these reviewers to determine the mean score of the overall guideline quality. Disagreements between reviewers during quality rating were resolved through discussion until consensus was reached. Only clinical practice guidelines with a mean score of 5–7 points on the overall guideline quality were included.
After the extensive literature review, potential KIs and QIs were selected by PH, EH and PF. These KIs and QIs were integrated in an internet-based Delphi questionnaire, consisting of three main parts: demographic questions (name and type of hospital or organization, number of intensive care beds, number of organ donors, professional group, function, years of experience in organ donation, age and gender), KIs and QIs. The demographic questions are included in the Additional file 1. The provisional Delphi questionnaire was pretested by three intensivists, who were not eligible to participate in the expert panel.
Step 2: Delphi round 1
During the first round, the participants received an e-mail with a link to the internet-based Delphi questionnaire. In addition to the demographic information, experts were asked to provide comments on the listed KIs and QIs or add new ones.
Step 3: Data analysis of Delphi round 1
Based on the comments in Delphi round 1, adjustments with regard to the description of the KIs and QIs were made and KIs or QIs were deleted. Newly identified KIs or QIs suggested by the expert panel were included in the questionnaire.
Step 4: Delphi round 2
In preparation for the second round, the participants received feedback of all the first-round panel members’ comments, deleted KIs and QIs, and the additionally proposed KIs and QIs. In the first part of the second Delphi round, experts were asked to rate on a 9-point Likert rating scale (score 1 indicating “strongly disagree”; score 9 “strongly agree”), to what extent each KI would contribute to the quality of care for the management of a potential donor (or the donor family, recipient or graft) and similarly to which extent each QI could be considered relevant and/or feasible to be implemented. The KIs & QIs of the Delphi round 2 are presented in the Additional file 1.
Step 5: Data analysis of Delphi round 2
The results of the second round were analyzed using predefined consensus criteria based on a systematic review about the use and reporting of the Delphi method for selecting health care QIs [17]. A KI was considered valid if it had a median score of 7 or more with 75% or more of the ratings in the highest tertile (Likert score: 7–9). A QI was accepted with agreement if the attribute relevance had a median score of 7 or more with 75% or more of the ratings in the highest tertile (Likert score: 7–9) and the attribute feasibility had a median score of 7 or more.
Step 6: Delphi round 3
In round 3, feedback on the quantitative panel results was provided to all members of the panel, presented by the following summary statistics: central tendencies (median, minimum, maximum, and mode), frequency of ratings in each tertile Likert category (1–3, 4–6, and 7–9), rating of contribution (ratio of “sum of ratings on the intervention given by participants” to “sum of ratings on the intervention if all respondents rated the interventions as ‘strongly agree’”), and the respondent’s own responses. Using this information, respondents were asked to re-rate the KIs and QIs in case they would like to change their previous answers.
Step 7: Data analysis of Delphi round 3
The same predefined consensus criteria as in step 5 were applied to the analysis of the results of the third Delphi round. If participants of round 2 did not respond in round 3, their answers of round 2 were considered as final.
Step 8: Physical consensus meeting
A face-to-face consensus meeting (June 2016) was organized to discuss and re-rate the KIs and QIs without consensus after the third round [17]. The nominal group technique was used as consensus method [25]. One author (DV) moderated this meeting in order to contain the influence of dominant personalities. Another author (PH) presented the available literature concerning the ‘no consensus’ KIs and QIs. Subsequently, the experts had the possibility to discuss the literature, followed by the opportunity for re-rating previous individual scores using the same Likert rating scale.