Skip to main content
Download PDF

Identifying key items to be addressed by non-clinical operators to manage out-of-hours telephone triage services for older adults seeking non-urgent unplanned care in Belgium: an e-Delphi study

BMC Health Services Research volume 24, Article number: 189 (2024) Cite this article

Abstract

Background

This study aimed to achieve expert consensus regarding key items to be addressed by non-clinical operators using computer-software integrated medical dispatch protocols to manage out-of-hours telephone triage (OOH-TT) services for calls involving older adults seeking non-urgent unplanned care across Belgium.

Methods

A three-part classic e-Delphi study was conducted. A purposive sample of experts specialized in out-of-hours unplanned care and/or older persons across Belgium were recruited as panelists. Eligibility criteria included experts with at least 2 years of relevant experience. Level of consensus was defined to be reached when at least 70% of the panelists agreed or disagreed regarding the value of each item proposed within a survey for the top 10 most frequently used protocols for triaging older adults. Responses were analyzed over several rounds until expert consensus was found. Descriptive and thematic analyses were used to aggregate responses.

Results

N = 12 panelists agreed that several important missing protocol topics were not covered by the existing OOH-TT service. They also agreed about the nature of use (for the top 10 most frequently used protocols) but justified that some modifications should be made to keywords, interrogation questions, degree of urgency and/or flowcharts used for the algorithms to help operators gain better comprehensive understanding patient profiles, medical habits and history, level of support from informal caregivers, known comorbidities and frailty status. Furthermore, panelists also stressed the importance of considering feasibility in implementing protocols within the real-world setting and prioritizing the right type of training for operators which can facilitate the delivery of high-quality triage. Overall, consensus was found for nine of the top 10 most frequently used protocols for triaging older adults with no consensus found for the protocol on triaging patients unwell for no apparent reason.

Conclusion

Our findings show that overall, a combination of patient related factors must be addressed to provide high quality triage for adults seeking non-urgent unplanned care over the telephone (in addition to age). However, further elements such as appropriate operator training and feasibility of implementing more population-specific protocols must also be considered. This study presents a useful step towards identifying key items which must be targeted within the larger scope of providing non-urgent out-of-hours telephone triage services for older adults seeking non-urgent unplanned care.

Peer Review reports

Introduction

It is well established that the use of out-of-hours telephone triage services (OOH-TT) services has large potential for managing unplanned and OOH healthcare needs presented by the general population [1, 2]. However, little is known with regards to the quality of these services in more vulnerable segments of the population such as older adults [1, 3, 4]. Specifically, it is well known that older adults (over the age of 65) are especially difficult to triage over TT services as they frequently present with atypical symptoms of common health concerns. At the same time, these segments of the population not only make up a large proportion of all unplanned care users [5, 6] but they are also known to be at the highest risks for experiencing adverse health events and safety incidents due to inappropriate care transfers [7] and in cases of inappropriate triage, mis-triage [8, 9].

As a first step towards addressing this issue, a recent publication by Wardlow et al., (2022) aimed to better understand how telehealth-based services should be delivered to appropriately meet the needs of older adults seeking care [10]. However, existing findings have largely focused on provider-focused principles and guidelines for promoting age-friendly telehealth practices [10]. Furthermore, some additional recent efforts also found that developing or adapting existing protocols to better fit population-specific criteria for triage in older adults may be effective for reducing rates of mis-triage in older adults. To date, these efforts have only been examined in specific and in-person triage tools [11,12,13] and it is not yet clear how well they may apply within the larger context of medical dispatch protocols used by OOH-TT services.

In Belgium, there is a national OOH-TT service which aims to provide medical assistance to patients during evenings, weekends and holidays. Depending on the region, patients calling this number are re-directed to the local on-call general practitioner (GP) for an appointment or to a non-medical operator specialized in triage using the Belgian Manual for Medical Regulation (BHMR) for guidance on further medical assistance (more information available at: www.1733.be (Dutch, French, German and English) [3, 14, 15]. Overall, the use of these medical dispatch protocols have been shown to be safe and effective for use in the general population [16,17,18]. However, lower quality of triage (particularly related to concerns of safety assessed via the frequency of adverse events, errors, and patient hospitalization rates) has been shown to be associated with triaging older adults over the telephone [19,20,21]. One potential problem associated with the use of existing protocols, such as the BHMR is the generic nature of the protocols. It is typically the case that the same protocols are used for triaging all segments of the population (i.e. including children and older adults who are known to be at higher risks for lower quality triage) with some limited considerations of age and medical history for vulnerable segments of the population. However, by incorporating key population-specific items within the protocols, the potential effectiveness, efficiency and overall quality of managing calls involving older adults in the population may be improved. As such, this Delphi study aims to achieve expert consensus regarding key items to be addressed by non-clinical operators using the BHMR medical dispatch protocols to manage OOH-TT services for calls involving older adults seeking non-urgent unplanned care across Belgium.

Methods

Study design

A three-part classic e-Delphi study was conducted to address the aim of our research through an iterative process which seeks to combine diverse expert opinions into group consensus based on the guidance of Keeney et al. (2011) and recommendations for Delphi study reporting guidelines in the health science sector by Spranger et al. (2022) [22,23,24].

Expert panel recruitment and selection

A purposive sample of N = 16 experts specialized in out-of-hours unplanned care and/or older persons across Belgium were recruited for participation as panelists in our study via the network of the research team. Identified experts were first contacted via the telephone and were given some brief information regarding the study [22]. Panelists who agreed to participate were then contacted electronically (via e-mail) with a formal invitation letter and a complete study information sheet. Panelists were also informed that their participation is voluntary and that they were free to withdraw from the study at any time.

Eligibility criteria

Eligibility criteria for panelists included experts with at least 2 years of prior professional experience relevant to unplanned care for older adults and/or telephone triage including emergency physicians, nurses, GPs, geriatricians, telephone triage operators, patient representatives and medical directors of OOH-TT services across Belgium.

BHMR protocols

All calls received by the 1733 service are managed by non-clinical operators who are trained to use the BHMR protocols; a set of computer-software integrated medical protocols (more info: https://www.health.belgium.be/nl/belgische-handleiding-voor-de-medische-regulatie#anchor-25165) [15, 25]. The use of the BHMR protocols by 1733 service operators is mandatory. The protocols examined in this study includes version 4.0 (published on 5 June, 2019) which consists of 56 different protocols that can be prompted by entering specific key words and descriptions of medical symptoms [25]. Although these protocols are largely generic, some considerations are made for specific cases involving children between the ages of 0–14, adults aged 75 and over and patients with chronic illnesses.

Data collection procedures

This Delphi study consisted of three parts. The first two parts of the study were held electronically and the third part was held via a face-to-face consensus meeting with panelists. All surveys were pilot tested before administered to panelists. During part one and part two, panelists were asked to fill out online surveys using Qualtrics software.

Part one (survey one)

Following consent to participate, participants were sent an e-mail link to complete a first online survey. The first part aimed to collect demographic information of panelists. The remainder of the survey was composed of open- and closed- ended information regarding panelists beliefs surrounding current protocols used by the 1733 service for triaging older adults seeking OOH unplanned non-urgent care. Specifically, panelists were presented with the list of protocols that are available at the 1733 service and were asked whether they believe the protocol topics considered in the existing list to be sufficient for triaging older adults as well as to self-report (based on their expert experiences) on any relevant protocol topic they believed to be missing from the existing list. Panelists were then presented with the 10 most frequently used protocols for triaging older adults via the 1733 OOH-TT service between January 1 to December 31, 2019. Complete algorithms, flowcharts, and frequency distributions were made available to enable a detailed analysis of each of the individual protocols (see Table 1) [25].

Table 1 Summary of protocol topics available for use within the 1733 OOH-TT service
Full size table

After review of each protocol, panelists were asked to provide their opinion regarding the frequency of use and content of protocols for triaging calls involving older adults seeking unplanned care. Panelists were asked whether each protocol was used as frequently as they would expect and whether they believed that any modifications should be made to the content (i.e. the keywords, interrogation questions, degree of urgency, flowchart, or other factors related to the algorithm) within a specific protocol to improve the management of calls involving older adults.

Part two (survey 2)

Panelists who completed part one of the survey were invited to partake in the second survey. The results of the first survey were used to form the basis of the second survey. Panelists were presented with a summary of (de-identified) responses that were submitted. Using a four-point Likert scale, they were asked to rate each of the submitted responses (considered as items) as “not relevant at all”; “somewhat relevant”; “quite relevant”; or “extremely relevant.” An open comments box was provided for panelists to further specify each of their responses (i.e. to elaborate on responses, or to add further suggestions and/or comments). Once all the responses were submitted, ratings for each item were analyzed for consensus and were re-submitted to panelists as necessary.

Part three (consensus meeting)

The consensus meeting aimed to achieve final consensus on items identified for each topic rated during part one and part two. The consensus meeting was held using a hybrid format and all panelists were invited to join the meeting in person or online based on their availability.

Predefined criteria for consensus

In line with expert recommendations, level of consensus was defined a priori to be reached when at least 70% of the expert panel members agreed (positive consensus defined as the item being generally scored as “quite relevant” or “extremely relevant”) or disagreed (negative consensus defined as the item being generally scored as “not relevant at all” or “somewhat relevant”) regarding the relevance of each item proposed by panelists within a survey [22]. If consensus was not reached following first-time analysis of responses obtained for an item, the survey was re-administered for a second round and panelists were requested to re-submit their responses based on group results obtained during the previous round. If consensus was not reached by the second round, then the item was to be further discussed for a final time during the round table consensus meeting.

Data analyses

Responses were quantitively and qualitatively analyzed over several rounds until expert consensus was found. Descriptive and thematic analyses were used to aggregate responses. Qualtrics was used for data gathering and analysis while SPSS Statistics (version 24.0.0.0) was used for analysis purposes.

Ethical considerations

This project received ethical approval from the Medical Ethics Committee UZ KU Leuven (study reference number S64473).

Results

Data was collected between March 31, 2022 until December 8, 2022. A total of N = 16 panelists were invited to complete survey one, out of which 12 responses were obtained followed by 10 responses for survey two. Eight panelists were present for the consensus meeting (see Fig. 1). The expert panel was comprised of n = 1 medical director of a call center, n = 2 GPs, n = 2 emergency physicians, n = 2 geriatricians, n = 3 telephone triage operators, n = 2 nurses, n = 1 other (telephone triage center coordinator) (panelists could select more than one category for multiple professional backgrounds and/or area of work). The majority of panelists worked in the field of academia (n = 6), or in a clinical setting (n = 7) and half (n = 6) had between 11 to 30 years of work experience. (see Table 2).

Fig. 1
figure 1

Timeline of Delphi study procedure

Full size image
Table 2 Expert panel characteristics
Full size table

Survey 1

In total, n = 8 panelists responded that they believe there to be several self-reported missing protocol topics which are not currently covered by the existing BHMR protocols but that should be added to improve the quality of triage for calls involving older adults ≥65 years of age seeking unplanned care. The following categories of missing protocol topics were reported to include: Skull Trauma, Generalized Fever In Adults, Medication, Neurological Problems Other than stroke, fever without an infection focus, Fall-Related, Neglect, Acute Confusion, Non-Traumatic Joint Pains, Reduced Functionality and Decline of General Conditions. Overall, at least half or more panelists also reported that all of the top 10 most frequently used protocols for triaging older adults via the 1733 OOH-TT service were used as frequently as they would expect (see Table 3). At the same time, at least half or more panelists felt that some modifications should be made to the content of all the 10 protocols to improve the overall management of calls involving older adults (see Table 4 and Table 5).

Table 3 Summary of 10 most frequently used protocol topics presented to panelists and frequency of use
Full size table
Table 4 Summary of panelist responses on whether content related modifications to be made for each protocol
Full size table
Table 5 Summary of panelist responses supporting content related modifications to be made for each protocol
Full size table

Survey two and consensus meeting

The top 10 most frequently used protocols for triaging older adults were assessed and discussed by panelists during survey two and the consensus meeting (see Table 6 below as an example with complete details available in Supplementary Tables 1 (survey two results) and 2 (consensus meeting results)). In total, consensus (positive or negative) was found for all items on two of the protocols (namely “Trauma” and “Cardiac arrest- deceased”) during survey two. Consensus was not found for the remaining eight protocols.

Table 6 Example of findings for protocol on “Breathing Difficulties”
Full size table

The remaining eight protocols were further discussed during the consensus meeting. One protocol (namely, the protocol on “Non-traumatic back pain”) was excluded from discussion because several panelists commented that the items addressed in this protocol were already addressed in the updated version of the BHMR protocols (version 5.0) which was released for use by the 1733 service during the data collection period of this study [26]. Full consensus was found for all items on the remaining protocols except the protocol “Unwell for no apparent reason.” Overall, panelists stressed that a combination of patient related factors must be addressed to provide high quality triage for adults seeking non-urgent unplanned care over the telephone (in addition to age) including patient profiles, medical habits and history, level of support from informal caregivers, known comorbidities and frailty status. Additionally, panelists also highlighted that further elements such as appropriate operator training and feasibility of integrating revised protocols (i.e. more population-specific protocols) must also be considered before implementation of protocols within the call center. The following specific outcomes were obtained for each protocol item (see details in Supplementary Tables 1 and 2):

Protocol on “Trauma”

A total of four items were addressed for the protocol on “Trauma”. Positive consensus was found for all items including fall in older adults, location of injury, reason for fall, and level of urgency.

Protocol on “Cardiac arrest- deceased”

A total of two items were addressed for the protocol on “Cardiac arrest – deceased”. Positive consensus was found for all items including nature of death and practical questions to better evaluate dispatch to the Mobile Emergency Group.

Protocol on “Breathing Difficulties”

A total of five items were addressed for the protocol on “Breathing Difficulties”. Two items including fever and symptoms related to heart failure were excluded from the round table discussion as these suggestions were already included in the new version of the BHMR protocols [26]. Positive consensus was found for the remaining items including symptoms specific for older adults/geriatric patients, coughing and coloured sputa.

Protocol on “Non-Traumatic Abdominal Pain”

A total of six items were addressed for the protocol on “Non-Traumatic Abdominal Pain”. Four items including abnormal aorta, aneurism alertness, diarrhea and pain severity were excluded from the round table discussion as these suggestions were already included in the new version of the BHMR protocols [26]. Positive consensus was found for the item hydration level and negative consensus was found for the item history about previous aneurysm.

Protocol on “Urogenital Problems”

A total of five items were addressed for the protocol on “Urogenital Problems”. Positive consensus was found for the items on problems with use of other medical devices in older adults ≥ 65, urinary incontinence, urinary overflow and possibility to send patient a GP for home visit. Negative consensus was found for the item removal of questions related to temperature and urinary retention from this protocol and added to “confusion” protocol instead.

Protocol on “Cardiac Problem Other Than Thoracic Pain”

A total of nine items were addressed for the protocol on “Cardiac Problem Other Than Thoracic Pain”. Two items including addition of swollen legs complaint to protocol for hot/cold limbs and possibility to push dimple into swollen leg (possibly indicating heart failure) were excluded from the round table discussion as these suggestions were already included in the new version of the BHMR protocols [26]. Of the seven remaining items, positive consensus was found for the items adapting current 1733 protocol for “syncope” to broader guidelines for “transit loss of consciousness and possibility to push dimple into leg given shortness of breath”. Negative consensus was found for the items on swollen leg in older adults ≥ 65, shortness of breath in older adults ≥ 65, irregular heart palpitations in older adults ≥ 65, respiratory problems in older adults ≥ 65, and orthostatic hypotension.

Protocol on “Nose-Throat-Ear-Tooth”

A total of four items were addressed for the protocol “Nose-Throat-Ear-Tooth.” Positive consensus was found for the items problem with swallowing foods and clarification of GP’s role for patients with tooth problem. Negative consensus was found for the items tooth problems for all ages (separate from nose-throat-ear protocol) and location of pain.

Protocol on “Hot or Cold Limb”

In total, three items were addressed for the protocol “Hot or cold limb.” Two items including swollen leg joints for all ages and integration of swollen legs or joints complaint intro current protocol for hot or cold limb were excluded from the round table discussion as these suggestions were already included in the new version of the BHMR protocols [26]. Positive consensus was found for the remaining item on pain severity and changes in pain severity when lifting leg up and down.

Protocol on “Unwell for No Apparent Reason”

In total, eight items were addressed for the protocol “Unwell for no apparent reason.” Positive consensus was found for the items on unwellness (for no apparent reason) in older adults ≥ 65, fever without focus in older adults ≥ 65, acute confusion in older adults ≥ 65, pain and mobility in older adults ≥ 65 and temperature and urinary retention, and for the item atypical symptoms in older adults ≥ 65 and dysregulated blood pressure. No consensus was found by panelists for the item on voluntary stopping of eating and drinking.

Discussion

This study presents one of the first scientific efforts to better understand and find expert consensus regarding key items to be addressed by non-clinical operators using computer-software integrated medical dispatch protocols to manage OOH-TT services for older adults seeking unplanned care. Our interdisciplinary expert panel found strong consensus for the majority of the 10 most frequently used BHMR protocols for triaging older adults within the larger 1733 service in Belgium. Findings showed that the majority of experts felt that most protocols used for managing calls involving older adults were used as frequently as they would expect. However, panelists also agreed that some content related modifications should be made for keywords, interrogation questions, degree of urgency and/or flowcharts used for the algorithms of the top 10 most frequently used protocols to improve the overall quality of triage for older adults.

Overall implications of our research suggest that in addition to age of patients, a combination of patient related factors must be addressed to provide high quality triage for adults seeking non-urgent unplanned care over the telephone. Additionally, some further elements such as appropriate operator training and feasibility of implementing more population-specific protocols must also be considered within the call center setting. Such considerations must be thoroughly evaluated before implementation of any new or revised protocols within practice.

Specifically, our findings shows that overall, a combination of patient related factors must be addressed to provide high quality triage for adults seeking non-urgent unplanned care over the telephone (in addition to age). However, further elements such as appropriate operator training and feasibility of implementing more population-specific protocols must also be considered. This study presents a useful step towards identifying key items which must be targeted within the larger scope of providing non-urgent out-of-hours telephone triage services for older adults seeking non-urgent unplanned care.

Despite known concerns of triaging older adults via OOH-TT services, currently existing methods are still largely catered for providing remote medical assistance to less vulnerable segments of the population. It is not yet clearly known how protocols used across various services may be adapted and/or optimized for use in older adults [3, 17, 18, 22, 23, 25, 27]. One plausible explanation for this concern as discussed during the round table consensus meeting is the understanding that it is especially difficult (and perhaps not possible) to create age specific protocols for older adults in a way that is similar to children in which specific ages are often lined with specific pathologies [28]. The importance of addressing triage for older adults by using methods in which operators prioritize assessing a combination of factors (including patient profiles, medical habits and history, level of support from informal caregivers, known comorbidities and frailty status) rather than solely evaluating differences in patient age was embedded throughout the discussion.

Another key point that was addressed by panelists is the feasibility of implementing new/revised protocols within real world settings. Panelists were largely opposed to the creation of new protocols for providing OOH-TT services to older adults due to time and clinical restrictions of the telephone operators. While the importance of providing adequate training for operators was largely agreed upon, the key focus mentioned by panelists was to consider the addition and integration of specific questions that may capture different dimensions of triage-related questions for calls involving older adults within currently existing protocols. Despite a lack of existing literature, [29] discussions on evaluating quality of training in OOH-TT operators were also highlighted to be important and in line with the surrounding literature base which has shown that operator related characteristics (such as clinical background, length of experience, type of previous experience and flexibility of decision-making regarding level of urgency) may be key factors which influence triage outcome [3, 4, 30,31,32,33]. It is therefore crucial to ensure that operators are given adequate training which can in turn, ensure consistency and high-quality dispatch of patients using OOH-TT services. Supplementing the right questions alongside well-trained operators who are able to gather the right information during a call was stressed by panelists to be a key combination for enabling the delivery of high-quality OOH-TT services for older adults.

Several strengths and limitations must be considered in interpreting the overall study results. First, it is important to note that our conclusions provide insight into discussions and consensus drawn by one particular expert panel and do not necessarily reflect objectively correct answers which must be used for delivering OOH-TT services to older adults. It is also important to note that different reflections and ideas regarding quality of triage may be produced by panelists consisting of other experts [27, 34]. As such, the results of this study must be interpreted with caution. Furthermore, level of consensus in this study was defined to be reached when at least 70% of the expert panel members agreed or disagree regarding the value of a specific item. Although this is within the standard range of measuring consensus [22], it must be noted that an alternative definition of level of consensus may yield different results. A key strength of this Delphi study is that the process inherently allows flexibility and enables panelists to assess their judgements through several rounds of the research process [30]. Furthermore, it also plays a vital role for linking together existing knowledge and areas of agreement and disagreement within this field of research through a structured process that can allow us to ultimately have an impact on improving the quality of OOH-TT protocols for use in older adults [31, 35]. A limitation that must also be considered is that no views of patient representatives were included. Despite several attempts to reach out to various eligible contacts, we were not able to include any panelists who could represent the views of older adults who have experience with the use of OOH-TT services. Finally, it is important to mention that although we intended for a national representative Delphi panel (and has therefore reached out to panelists from three regions in Belgium – Flanders, Wallonia, and Brussels), we were presented with a high dropout rate of French speaking panelists leaving us with a small expert panel of 8 participants for the final consensus round. Furthermore, the study team moderating the development of the questionnaires and consensus meeting were all based in a university and/or clinical settings located in Flanders. As such, the results are therefore biased towards expert views of panelists from the Flemish region (i.e. Flanders) of Belgium. However, given the rather flexible nature of a Delphi study to determine sample size and heterogeneity based on the larger study aim and possible range of opinions, we believe that that concern of mathematical/statistical bias is limited.

Conclusion

This Delphi study presents one of the first scientific efforts targeting the improvement of TT protocols used for triaging older adults seeking non-urgent out-of-hours unplanned care. Overall, this research is a first step towards addressing current gaps in the existing topics of the BHMR and for identifying important content related benchmarks which must be targeted within the larger scope of OOH-TT services. Our findings showed that overall, a combination of patient related factors must be addressed to provide high quality triage for adults seeking non-urgent unplanned care over the telephone (in addition to age). However, further elements such as appropriate operator training and feasibility of implementing more population-specific protocols must also be considered. This is a useful step towards identifying key items which must be targeted within the larger scope of providing non-urgent out-of-hours telephone triage services for older adults seeking non-urgent unplanned care. We hope that these findings will eventually be used for guiding quality improvements within the BHMR protocols used within the context of the 1733 service but also for similar OOH-TT settings internationally.

Availability of data and materials

For any requests about data generated or analyzed during this study, please contact the corresponding author.

Abbreviations

OOH:

Out-of-hours

TT:

Telephone triage

ED:

Emergency department

BHMR:

Belgian Manual for Medical Regulation

References

  1. Bunn F, Byrne G, Kendall S. The effects of telephone consultation and triage on healthcare use and patient satisfaction: a systematic review. Br J Gen Pract. 2005;55(521):956–61.

    PubMed  PubMed Central  Google Scholar 

  2. Campbell JL, Fletcher E, Britten N, et al. Telephone triage for management of same-day consultation requests in general practice (the ESTEEM trial): a cluster-randomised controlled trial and cost-consequence analysis. Lancet. 2014;384(9957):1859–68.

    Article  PubMed  Google Scholar 

  3. Campbell JL, Fletcher E, Britten N, et al. The clinical effectiveness and cost-effectiveness of telephone triage for managing same-day consultation requests in general practice: a cluster randomised controlled trial comparing general practitioner-led and nurse-led management systems with usual care (the ESTEEM trial). Health Technol Assess. 2015;19(12):1–212.

    PubMed  PubMed Central  Google Scholar 

  4. Lake R, Georgiou A, Li J, et al. The quality, safety and governance of telephone triage and advice services - an overview of evidence from systematic reviews. BMC Health Serv Res. 2017;17(1):614.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Blank L, Coster J, O'Cathain A, et al. The appropriateness of, and compliance with, telephone triage decisions: a systematic review and narrative synthesis. J Adv Nurs. 2012;68(12):2610–21.

    Article  PubMed  Google Scholar 

  6. Lemoyne SE, Herbots HH, De Blick D, et al. Appropriateness of transferring nursing home residents to emergency departments: a systematic review. BMC Geriatr. 2019;19(1):17.

    Article  PubMed  PubMed Central  Google Scholar 

  7. World Health Organization (WHO). Transitions of care. 2016. https://iris.who.int. Accessed 7 Feb 2024.

  8. Burke RE, Rooks SP, Levy C, et al. Identifying potentially preventable emergency department visits by nursing home residents in the United States. J Am Med Dir Assoc. 2015;16(5):395–9.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Ouslander JG, Perloe M, Givens JH, et al. Reducing potentially avoidable hospitalizations of nursing home residents: results of a pilot quality improvement project. J Am Med Dir Assoc. 2009;10(9):644–52.

    Article  PubMed  Google Scholar 

  10. Wardlow L, Leff B, Biese K, et al. Development of telehealth principles and guidelines for older adults: a modified Delphi approach. J Am Geriatr Soc. 2022;71(2):371–82.

    Article  PubMed  Google Scholar 

  11. Alshibani A, Singler B, Conroy S. Towards improving prehospital triage for older trauma patients. Z Gerontol Geriatr. 2021;54(2):125–9.

    Article  PubMed  Google Scholar 

  12. Brown JB, Gestring ML, Forsythe RM, et al. Systolic blood pressure criteria in the National Trauma Triage Protocol for geriatric trauma: 110 is the new 90. J trauma acute care surg. 2015;78(2):352.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Nishijima DK, Gaona SD, Waechter T, et al. Out-of-hospital triage of older adults with head injury: a retrospective study of the effect of adding “anticoagulation or antiplatelet medication use” as a criterion. Ann Emerg Med. 2017;70(2):127–38. e6.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Islam F, Heeren P, Sabbe M, et al. A descriptive analysis of a national out-of-hours telephone triage service in Belgium. In: Oral presentation at the 18th European Geriatric Medicine Society (EuGMS) Congress. London, UK; 2022.

  15. Islam F, Milisen K, Gellens M, et al. Changes in the use and uptake of a national out-of-hours telephone triage service by younger and older patients seeking non-urgent unplanned care surrounding the COVID-19 pandemic in Flanders (Belgium). Acta Clin Belg. 2022;1-9.

  16. Van der Mullen C, Quintens H, Van Baelen S, et al. De patiënt met een niet-planbare zorgvraag naar het gepaste zorgniveau verwijzen: nieuwe 112-1733 geïntegreerde telefonische triage- en regulatieprotocollen. Tijdschrift voor geneeskunde. 2017;73(5):241–7.

    Google Scholar 

  17. Schoenmakers B, Delmeiren L, Pietermans S, et al. The implementation of the nationwide out-of-hours phone number 1733 in Belgium: analysis of efficiency and safety. Prim health care res develop. 2021;22(7):1–6.

    Google Scholar 

  18. Morreel S, Colliers A, Remmen R, et al. How accurate is telephone triage in out-of-hours care? An observational trial in real patients. Acta Clin Belg. 2022;77(2):301–6.

    Article  PubMed  Google Scholar 

  19. Huibers L, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: a systematic review. Scand J Prim Health Care. 2011;29(4):198–209.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Lake R, Georgiou A, Li J, Li L, Byrne M, Robinson M, et al. The quality, safety and governance of telephone triage and advice services - an overview of evidence from systematic reviews. BMC Health Serv Res. 2017;17(1):1–10.

    Article  Google Scholar 

  21. Carrasqueiro S, Oliveira M, Encarnação P. Evaluation of telephone triage and advice services: a systematic review on methods, metrics and results. User Centred networked. Health Care. 2011:407–11.

  22. Keeney S, McKenna H, Hasson F. The Delphi technique in nursing and health research. Croydon (UK): John Wiley & Sons; 2011.

    Book  Google Scholar 

  23. Lynn MR, Layman EL, Englebardt SP. Nursing administration research priorities: a National Delphi Study. JONA. J Nurs Administr. 1998;28(5):7–11.

    Article  CAS  Google Scholar 

  24. Spranger J, et al. Reporting guidelines for Delphi techniques in health sciences: a methodological review. Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen. 2022;172:1–11.

    Article  PubMed  Google Scholar 

  25. Federal Public Service (FPS) Health Food Chain Safety Environment. Belgian Manual for Medical Regulation. 2019. https://www.health.belgium.be. Accessed 7 Feb 2024.

  26. Federal Public Service Health. Belgian Manual for Medical Regulation. 2021. https://www.healthybelgium.be. Accessed 7 Feb 2024.

  27. Walker S, Scamell M, Parker P. Standards for maternity care professionals attending planned upright breech births: a Delphi study. Midwifery. 2016;34:7–14.

    Article  PubMed  Google Scholar 

  28. Jácome M, Rego N, Veiga P. Potential of a nurse telephone triage line to direct elderly to appropriate health care settings. J Nurs Manag. 2019;27(6):1275–84.

    Article  PubMed  Google Scholar 

  29. Islam F, Sabbe M, Heeren P, et al. Consistency of decision support software-integrated telephone triage and associated factors: a systematic review. BMC Med Informat Decis Making. 2021;21(1):107.

    Article  Google Scholar 

  30. Donohoe HM, Needham RD. Moving best practice forward: Delphi characteristics, advantages, potential problems, and solutions. Int J Tour Res. 2009;11(5):415–37.

    Article  Google Scholar 

  31. Fink-Hafner D, Dagen T, Doušak M, et al. Delphi method: strengths and weaknesses. Adv Methodol Statist. 2019;16(2):1–19.

    Google Scholar 

  32. O'Cathain A, Jon N, Sampson F, et al. Do different types of nurses give different triage decisions in NHS direct? A mixed methods study. J Health Serv Res Policy. 2004;9(4):226–33.

    Article  PubMed  Google Scholar 

  33. O'Cathain A, Webber E, Nicholl J, et al. NHS direct: consistency of triage outcomes. Emerg Med J. 2003;20(3):289–92.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Baker J, Lovell K, Harris N. How expert are the experts? An exploration of the concept of ‘expert’ within Delphi panel techniques. Nurse res. 2006;14(1).

  35. Iqbal SaP-Y L. The Delphi method. Nurs Res. 2009;46(2):116–8.

    Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Didier Desruelles for his help in translating and pilot testing the surveys conducted in the study. We also thank Marc Gellens and the 1733 team for their guidance throughout our research. Finally, would like to thank our panelists. This research would not be possible without their participation and valuable input!

Funding

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 81265. PH holds a post-doctoral fellowship of KU Leuven (PDMT2/21/079). The funders have no role in the design of the study, data collection, analysis, interpretation, or in writing of the manuscript.

Author information

Authors and Affiliations

  1. Department of Public Health and Primary Care, KU Leuven Kapucijnenvoer 35, 3000, Leuven, Belgium

    Farah Islam, Pieter Heeren, Kelu Yang, Koen Milisen & Marc Sabbe

  2. Department of Geriatric Medicine, University Hospitals Leuven, Herestraat 49, 3000, Leuven, Belgium

    Pieter Heeren & Koen Milisen

  3. Department of Emergency Medicine, University Hospitals Leuven, Herestraat 49, 3000, Leuven, Belgium

    Marc Sabbe

Authors
  1. Farah Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pieter Heeren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kelu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Koen Milisen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marc Sabbe
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors participated in the conceptualization and design of the study. FI completed all statistical analyses and wrote the draft of the manuscript. PH, KY, MS and KM provided oversight and consultation during all aspects of the study. All authors read and approved the final manuscript. KM and MS supervised the study.

Corresponding author

Correspondence to Koen Milisen.

Ethics declarations

Ethics approval and consent to participate

Ethics approval for this study was obtained by the Research Ethics Committee UZ/KU Leuven (study reference number S64473). The study was performed in accordance with relevant guidelines and regulations (including the Declaration of Helsinki and GDPR). Informed consent for research participation was obtained from all panelists.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Islam, F., Heeren, P., Yang, K. et al. Identifying key items to be addressed by non-clinical operators to manage out-of-hours telephone triage services for older adults seeking non-urgent unplanned care in Belgium: an e-Delphi study. BMC Health Serv Res 24, 189 (2024). https://doi.org/10.1186/s12913-024-10657-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-024-10657-1

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us