Skip to main content
  • Research article
  • Open access
  • Published:

The quality, safety and governance of telephone triage and advice services – an overview of evidence from systematic reviews

Abstract

Background

Telephone triage and advice services (TTAS) are increasingly being implemented around the world. These services allow people to speak to a nurse or general practitioner over the telephone and receive assessment and healthcare advice. There is an existing body of research on the topic of TTAS, however the diffuseness of the evidence base makes it difficult to identify key lessons that are consistent across the literature. Systematic reviews represent the highest level of evidence synthesis. We aimed to undertake an overview of such reviews to determine the scope, consistency and generalisability of findings in relation to the governance, safety and quality of TTAS.

Methods

We searched PubMed, MEDLINE, EMBASE, CINAHL, Web of Science and the Cochrane Library for English language systematic reviews focused on key governance, quality and safety findings related to telephone based triage and advice services, published since 1990. The search was undertaken by three researchers who reached consensus on all included systematic reviews. An appraisal of the methodological quality of the systematic reviews was independently undertaken by two researchers using A Measurement Tool to Assess Systematic Reviews.

Results

Ten systematic reviews from a potential 291 results were selected for inclusion. TTAS was examined either alone, or as part of a primary care service model or intervention designed to improve primary care. Evidence of TTAS performance was reported across nine key indicators – access, appropriateness, compliance, patient satisfaction, cost, safety, health service utilisation, physician workload and clinical outcomes. Patient satisfaction with TTAS was generally high and there is some consistency of evidence of the ability of TTAS to reduce clinical workload. Measures of the safety of TTAS tended to show that there is no major difference between TTAS and traditional care.

Conclusions

Taken as a whole, current evidence does not provide definitive answers to questions about the quality of care provided, access and equity of the service, its costs and outcomes. The available evidence also suggests that there are many interactional factors (e.g., relationship with other health service providers) which can impact on measures of performance, and also affect the external validity of the research findings.

Peer Review reports

Background

The provision of after-hours primary medical care services has been changing in many countries in response to increasing demand and pressures on existing services [1,2,3]. Telephone triage and advice services (TTAS) are one approach to expanding the provision of out-of-hours medical care. TTAS involves people with a health problem receiving assessment and advice over the telephone [4]. TTAS advice may include a recommendation to visit an Emergency Department (ED), make an appointment with a general practitioner (GP) or other healthcare provider, or the administration of home or self-care [5, 6]. The most common TTAS delivery models are either nurse- or physician-led, very often involving the use of call centre technology, incorporating clinical decision support systems to aid in the evaluation of patients’ health conditions [7].

One of the major drivers for TTAS has been the need to reduce the burden on GPs and EDs. Some estimates suggest that more than half of out-of-hours calls can be handled by telephone advice alone [8]. With this in mind it would seem reasonable to assume that access to health information and advice could relieve pressures on GP services and EDs particularly through reductions in “inappropriate” or “avoidable” attendances [9,10,11]. For many countries TTAS has become an important part of the organisation and delivery of out-of-hours care [12]. A number of healthcare call centres have been established in the United Kingdom (UK) [13], Australia, Sweden, Denmark, the Netherlands, Canada, along with the United States of America (USA), which does not have government-sponsored national telephone triage access but does have some localised systems [6, 11, 14].

With the expansion of TTAS over the last decade there has been a corresponding increase in its popularity and utilisation across many jurisdictions [11, 15]. This growth has in turn stimulated interest in the potential to incorporate WebRTC (real time communications) technologies [16], to provide accessible and easy-to-use video capabilities to expand the type of advice and care provided. These capabilities may have the potential to address the rising demands of over stretched health services [13], and enable access for people living in remote areas. TTAS involve many medical, technological and social/organisational challenges [17, 18] around how the service is run and monitored [19], along with issues related to the service’s accessibility, quality and appropriateness [20, 21]. The challenges and opportunities afforded by major telehealth care innovations underscore the need for a comprehensive assessment of available evidence related to their quality, safety and governance.

The existing evidence base on the topic examined aspects of TTAS from different perspectives (e.g., clinical, management and patients) using a wide range of measures that are not consistently applied or assessed. This makes it difficult for policymakers to use existing evidence to inform policy decisions. Systematic reviews represent the highest level of evidence synthesis. We aimed to undertake an overview of such reviews to determine the scope, consistency and generalisability of findings in relation to the governance, safety and quality of TTAS [22].

Methods

Inclusion criteria

Our overview included any systematic review focused on telephone based triage and advice services, available in English; and published since 1990. We decided to commence our search at 1990 as the types of telehealth services under consideration commenced operation around this time [1].

We excluded reviews which were related only to the use of telephone advice for a specific population or demographic (e.g. ethnic minority), condition (e.g., depression) medical specialty (e.g., asthma); and ongoing or chronic conditions (e.g., diabetes); technical assessments not related to patient or healthcare outcomes; and general health education. Table 1 lists the inclusion and exclusion criteria used in this overview.

Table 1 Inclusion and exclusion criteria

To maximise our search criteria we used the PICO (Population, Intervention, Comparison, Outcome) search strategy form. The patient or problem targeted were presenting patients, requiring triage or consultation. The intervention was telephone based triage, advice or consultation by telephone. Most of the studies included in the reviews compared telephone advice with face-to-face consultations, but some also compared telephone advice given by different healthcare professionals. Key indicators included those performance measures related to the safety, quality or governance of TTAS and/or other relevant patient/health-related outcomes.

Search methods

We sourced our results from the following databases: PubMed, MEDLINE, EMBASE, CINAHL, Web of Science and the Cochrane Library. The search terms used included ‘tele’ terms, with ‘tele’ truncated to elicit all related terms (telehealth, telemedicine, telephone, etc.); terms relating to the type of care: triage, advice, consultation; and other related terms such as after-hours, out-of-hours and primary care. The full search strategy is outlined in Additional file 1.

The title and abstract of the reviews identified through our searches was scrutinised, and potentially relevant papers were downloaded for further consideration. Following the search, the full text of these reviews was examined and those which fit our criteria were added to our results. We also scrutinised the reference lists of all relevant papers to identify reviews for inclusion or exclusion.

To ensure reliability, the search strategy was devised by three researchers (AG, LL and RL), the search was completed by one team member (RL) and then replicated by another (LL), and all three researchers discussed the identified texts to arrive at a consensus on the reviews for inclusion.

Data collection and analysis (selection, data extraction and management)

The final group of texts was read in full and the data extracted. We abstracted descriptive details of the included systematic reviews: the authors, title, year published and country. Data were collected about the studies reviewed in each text: the number and type of reviewed studies, the date range of the studies and the study populations (Table 1). Information was compiled about the indicators or performance measures used, the purpose of the review, the design, search methods, the performance of a grey literature search, and results (see Additional file 2 for a detailed summary table). An appraisal of the methodological quality of the systematic reviews was independently undertaken by two researchers (AG, JL) using A Measurement Tool to Assess Systematic Reviews (AMSTAR) [23]. Discrepancies in quality rating between researchers were reconciled through discussion.

Results

Search results

A total of 291 papers were identified and subjected to title/abstract review. Of these, 237 were excluded and 20 papers were downloaded and subjected to full text review, along with a further 26 papers identified through hand searching of reference lists. Ultimately, ten systematic reviews were selected for inclusion. Our search flow diagram is depicted in Fig. 1.

Fig. 1
figure 1

Search flow diagram

Description of included systematic reviews

The ten selected systematic reviews examined TTAS either alone, or in the context of primary care service models or interventions to improve primary care. The systematic reviews were all published within the period, 2003–2013. Four were published between 2003 and 2006, and six between 2011 and 2013. The reviews came from researchers in the UK, Australia, Canada, the Netherlands and Portugal. Each of these countries had a TTAS service. The UK National Health Service (NHS) established their national TTAS, NHS Direct in 1998, which was replaced with NHS 111 in 2014 [24]. Australia established the healthdirect helpline service in 2006 [25]. Portugal has Health Line 24 [26], and both the Netherlands and Canada have various services. Table 2 details the characteristics and aims of the included systematic reviews.

Table 2 Description of reviews

Study designs within systematic reviews

Nine of the ten systematic reviews allowed us to identify the studies they included, while one provided an outdated link to their reference list, which proved inaccessible. Our attempts to contact the authors to obtain this information were unsuccessful [7]. From the remaining nine reviews we identified 127 individual papers, 42 (33%) of which were included in more than one review. Thirty-one studies were included in two of the reviews, nine studies featured in three reviews, one study was included in four and another one in five of the reviews.

The studies ranged in date from 1978 to 2011 and originated from the UK (53), the USA (36), Australia (12), Canada (6), The Netherlands and Denmark (both 4), New Zealand and France (both 3), Switzerland (1), Germany (1), Sweden (1), Italy (1), India (1), and Israel (1). There were nine Randomised Controlled Trials (RCTs). The vast majority of studies were observational, including before and after and interrupted time series designs, as well as several surveys. The reviews also included some case studies, evaluations and reports.

Quality appraisal

To assess the quality of the systematic reviews we: i) conducted a quality appraisal of the systematic reviews against the AMSTAR checklist; ii) evaluated the methodological quality of the studies reported in the review; and iii) considered the comments made in the review regarding the overall quality of the evidence presented.

Out of a maximum score of 11, quality ratings of the systematic reviews against the AMSTAR checklist ranged from 2 to 8. Five reviews achieved a rating greater than or equal to 5 [6, 14, 15, 27, 28]. Three reviews provided no evidence of having assessed the quality of included studies. Of the other seven reviews, a range of methods was used for quality assessment. One used a version of the checklist developed by the Cochrane Effective Practice and Organisation of Care Review Group [29]. Another used a modified checklist for assessing economic evaluations [7]. Two reviews used a hierarchy of evidence method [15, 30] and two reviews used a revised version of the Critical Appraisal Skills Programme [6, 11, 31]. One review used a modified version of the Scottish Intercollegiate Guidelines Network (SIGN) checklist [32]. Eight of the reviews commented on the limitations of the evidence, citing poor study designs, poorly reported studies and low checklist scores. Overall, some of the evidence appears reliable, but much of it is not strong [27].

Publication bias

Six of the systematic reviews did not include a search of the grey literature [6, 7, 12, 14, 28, 30], while four did [11, 15, 27, 33]. This overview conducted a broad Google search to identify other existing reviews but did not elicit any new results. All searches conducted in this overview were restricted to English language systematic reviews only.

Quality, safety and governance dimensions

We identified the following nine quality, safety and governance dimensions from the available evidence: access, appropriateness, patient compliance, patient satisfaction, cost, safety, health service utilisation, clinical workload and clinical outcomes. Most of the systematic reviews examined between one to five measures, one review investigated eight, but no review considered all nine. We proceeded to group the review results together based on these indicators and then considered the ways in which they investigated and compared the results.

Access

Expanding healthcare access to marginalised communities or populations is an often-cited driver of the introduction or expansion of TTAS services [13]. However, the impact of TTAS on improving access was not apparent from the two systematic reviews that examined this issue. Carrasqueiro et al. considered health care access, but did not state how many studies they found that explored this issue. The authors reported that in some instances healthcare access for some people with severe symptoms was expedited [7], they nevertheless concluded that there was no consistent improvement in access based on TTAS availability. Chapman et al. reported on one study which found that the NHS Direct service was easily accessible, but callers appeared to be those already utilising other healthcare services [15].

Appropriateness

In their review, Blank et al. [14] noted the lack of consistency in the definition of “appropriateness” of telephone triage decisions by the studies included in their rapid evidence scan. The five systematic reviews which considered evidence about the appropriateness of TTAS used one of two definitions, examining either the number of calls which could be managed with telephone advice alone, or the appropriateness of the advice given.

Two reviews looked at the percentage of calls able to be handled with telephone advice alone. Both the Cochrane review by Bunn et al. [27] and the review by Fry [11] concluded that approximately 50% of calls handled by doctors or nurses could be handled with telephone advice alone. Bunn et al. also compared the call handling abilities of doctors and nurses and concluded that doctors handled 62% of calls with telephone advice alone, while nurses managed 59% with telephone advice alone [27].

The review by Blank et al. examined studies in which the advice given over the phone was compared with “appropriate” advice – either the “gold standard,” professional advice, subsequent treatment or diagnosis, patient validation or measure of adverse events. This review found a 44–98% rate of accuracy/appropriateness with a median of 75% [14]. The review also compared doctors’ and nurses’ handling of calls, but found that results were not consistent. This review and the review by Fry did however note there may be issues with under-referral and under-estimation of urgency [11, 14].

The review by Leibowitz et al. reported on an RCT which compared the number of deaths in seven days between those whose calls were handled by doctors or nurses. This RCT found no difference between the two groups. This paper also noted that studies using simulated patients reported variability and inadequacies in the advice given over the phone. Alternatively, studies which used real patients reported that the majority received appropriate advice [30].

The review by Carrasqueiro et al. examined studies in which audits of medical records were conducted to determine the adequacy of the advice given over the phone. The authors concluded that they could not demonstrate high rates of advice appropriateness or adequacy, but did not specify exactly how this conclusion was drawn [7]. The impact of other provider characteristics (e.g., age, length of experience) on the appropriateness of the advice given was not discussed by any systematic review.

Patient compliance

When advice is offered to patients via a new medium, such as the telephone, it is imperative to assess patients’ willingness to adhere to the advice. Services that provide advice with which patients are unwilling to comply are not likely to prove worthwhile. The three systematic reviews which investigated compliance all compared compliance rates by the advice given, which they grouped into three categories: emergency or urgent care, office/physician care, and home or self-care. Blank et al. reported that the median rate for compliance with advice to seek physician care was 66%, urgent care 75%, and self-care 77% [14].

The Purc-Stephenson and Thrasher review also found that callers were more likely to comply with advice to use emergency services or self-care than office care, but did not provide figures [6]. This review also commented that patient perceptions and expectations, along with the quality of the provider communication influenced caller compliance [6]. The third review by Carrasqueiro et al. [7] failed to provide compliance rates but did report that compliance varied significantly depending on the advice given, and that the variance was due to patient characteristics – original intention, complaint, age, income. This review, as well as the review by Blank et al. [14], also noted that rates of patient compliance were slightly higher when patient self-reporting data were examined, compared with provider data.

Two of the systematic reviews provided overall compliance rates. Blank et al. estimated an overall compliance rate of between 56 and 98%, with a median of 77% [14] while Purc-Stephenson and Thrasher reported an overall compliance rate of 62% [6]. Both of these reviews included some self-reported data, which may be potentially misleading.

Patient satisfaction

Five systematic reviews examined patient satisfaction of TTAS with most reporting satisfactory levels. Ismail et al. reported that their analyses found generally positive patient satisfaction rates, although one of the examined studies found low rates [28]. In their Cochrane review, Bunn et al. found patient satisfaction rates to be no different from other forms of care, and sometimes higher [27]. Fry considered patient satisfaction of the (no longer existing) UK NHS Direct service and found several studies reporting good levels of patient satisfaction [11].

Carrasqueiro et al. likewise found that most studies presented positive levels of patient satisfaction, but they, along with Leibowitz et al. observed that satisfaction rates decreased when patients’ initial expectations were not met [7, 30]. Patients were dissatisfied when telephone-based care represented a barrier to traditional forms of care, for example when patients rang physicians requesting a home-visit but were offered a telephone consultation instead [7, 30].

Cost

The findings from the four systematic reviews which examined the costs of TTAS, point to the potential for cost-effectiveness, but also suggest a lack of thorough research. Brebner et al. stated simply that all the studies which examined the telephone services found them to be cost-effective and did not elaborate further [33]. Carrasqueiro et al. found no thorough cost analysis studies [7]. Bunn et al. concluded that there was little cost difference between telephone advice and traditional care, but noted a study in which doctors made telephone advice calls to patients instead of granting same day appointments. In this case, the doctors’ phone bills increased 26% [27]. Finally, Ismail et al. concluded that while some cost reductions were possible, analyses suggested that the likelihood of savings across urgent care were low [28].

Safety

Two systematic reviews considered the issue of TTAS safety, primarily by investigating the frequency of adverse events, errors and hospitalisation rates. Carrasqueiro et al. reviewed studies which used patient surveys and medical records and concluded that the evidence suggests that TTAS was reasonably safe [7]. Huibers et al. [12] reported on 13 observational studies which showed that on average triage was safe in 97% of patients. However, 10 studies using high-risk simulated patients showed that on average 46% were safe [12].

Health service utilisation

Four systematic reviews investigated changes in health service utilisation brought about by the availability of TTAS. Carrasqueiro et al. reported that no clear pattern of change was evident [7]. The review by Ismail et al. reported conflicting results which produced no clear pattern, but acknowledged that reductions in downstream workloads were a possibility [28].

The systematic review by Chapman et al. included one study finding no change in service utilisation, while another study observed a decrease in use of GP services and an increase in out-of-hours consultations and ED attendances [15]. Two reviews reported a reduction in demand for GP consultations, particularly after-hours [11, 27]. One of these found no change in hospital use [27], while the other reported that ED activity could be reduced by up to two thirds [11].

Bunn et al. [27] reported that when telephone consultation was compared with traditional care, two of four studies reported no difference in the number of ED visits, while two studies reported increases in ED visits, one of which was a statistically significant increase, the other was not. Regarding hospital admissions, telephone consultation by doctors did not lead to any change in admission rates, while a phone service run by clinic clerks did lead to a reduction in admissions at 12 months. Bunn et al. also reviewed three studies which compared one type of health care professional with another [27]. Two RCTs compared nurse telephone advice with advice from a doctor in an out-of-hours deputising service [34, 35], and one Controlled Clinical Trial (CCT) compared telephone advice by a health assistant with telephone advice from a nurse or doctor [36]. All three observed a non-statistically significant rise in visits to the ED in the intervention groups. The two RCTs also examined hospital admission rates and found no changes.

Clinical workload

Changes in health service utilisation could also produce workload changes for health professionals. GPs may be especially susceptible to such changes. Four reviews considered how TTAS may affect GPs workloads. All four systematic reviews found evidence that telephone advice services had the potential to reduce GPs workloads [11, 15, 28, 30].

Clinical outcomes

Two systematic reviews investigated clinical outcomes and clinical effectiveness. The first concluded that ED tele-consultation was clinically effective [33]. Carrasqueiro et al. reviewed studies which utilised patient survey data to consider clinical outcomes. They determined that there were no long term studies which could lead to a conclusion one way or the other [7].

Discussion

This systematic overview collated the available evidence related to the governance, safety and quality of TTAS across nine key performance indicators – access, appropriateness, compliance, patient satisfaction, cost, safety, health service utilisation, physician workload and clinical outcomes. Some important indicators, such as the acceptability of TTAS were not specifically reported in the reviews we examined, but they may be considered in evaluations of performance conducted by the services themselves, or their funding bodies. These evaluations could potentially provide another source of information about these services.

The indicators of appropriateness and patient satisfaction received the most attention in the reviews. Appropriateness was evaluated in different ways ranging from the adequacy of triage advice, to the ability of doctors and nurses to handle enquiries over the telephone [11, 27]. The majority of reviews that investigated patient satisfaction with TTAS reported that it was generally comparable to, or occasionally even greater than satisfaction with traditional care [11, 27, 28].

The systematic reviews which examined safety compared the frequency of adverse events, errors and hospitalisation rates, and reported that triage was safe in most cases [7, 12]. However, findings from simulated high risk patients showed that on average 50% of patients received advice that was considered unsafe, presenting a higher risk of adverse events [12]. Safety can be very difficult to measure, as those who are already considered unwell or at risk, may be more prone to adverse events, through no fault of the service. Care should be taken about mistakenly using hospitalisation as a measure of safety. If TTAS provides advice that the caller should attend the hospital, this should be seen as advantageous for the patient’s safety, rather than an adverse event related to the use of the TTAS. A clearer definition of adverse events or safety issues related to the service would be of benefit to further research in this area.

The most reliable findings came from studies about clinician workload and patient compliance. The reviews which evaluated clinician workload reported that there was some potential for telephone triage and advice to reduce clinician workload, particularly for GPs. The reviews which investigated patient compliance mostly concluded that callers were more likely to comply with advice to seek emergency care or provide self-care, than advice to visit a GP or other health service [6, 7, 14].

The indicators which revealed the most variable and weakest evidence related to the utilisation of TTAS, service access, and costs. Reviews about access and the cost of TTAS showed no consistent evidence of improvement or gain [7, 15, 27, 28, 33]. Neither of the two reviews which considered clinical outcomes provided strong findings.

Taken as a whole, the available evidence does not provide definitive answers to questions about the quality of care provided, access and equity of the service, its costs and outcomes. The available evidence suggests that there are many interactional factors which can impact on measures of performance [6], and also affect the external validity or generalisability of the research findings [37]. For instance, TTAS is one (albeit very important) organisational model for dealing with out-of-hours care. Other models include individual general practices, primary care centres or even GP cooperatives [38]. These models are not necessarily exclusive to one distinct area or location, quite often they co-exist across municipalities, primary health care networks, regions or nations [39]. Decisions about the operation of TTAS are also influenced by a large number of demographic (e.g., rural and remote), cultural (e.g., language), finance (e.g., health care costs) and governance (e.g., policy priorities) factors [39]. Moreover, many of the quality and safety dimensions of TTAS are shaped by factors related to the integration and accessibility of care (e.g., the organisation of primary care) or technical infrastructure (e.g., patient access to information), specific to a community [40]. This suggests that TTAS safety and quality dimensions are intrinsically linked to properties of a larger and broader system [41].

We have in this overview bought together the available evidence and provided a list of measures by which to evaluate TTAS. There is not a simple yes or no answer to questions about the overall effectiveness and benefit of TTAS. Much of the evidence is contingent on the different circumstances and context of different systems. There remain many gaps in the evidence base. This overview provides a framework which can inform future research directions about TTAS and provide guidance relating to their implementation and expansion.

Conclusion

The diffusion of new technologies (e.g., telehealth care) has provided additional scope to expand and diversify health care services and provide opportunities for greater consumer engagement [42]. In the face of these developments, it is important that researchers continue to seek answers to the many unresolved questions about TTAS [43], albeit with a renewed and greater diversification of qualitative and quantitative research along with strategies that engage with patients as a means of enhancing the patient-centredness of TTAS [44]. Such research endeavours can inform decision making about the governance, quality and safety of TTAS into the future.

Abbreviations

A&E:

Accident and Emergency

CASP:

Critical Appraisal Skills Programme

CCT:

Controlled Clinical Trial

CT:

Controlled Trial

ED:

Emergency Department

EPOC:

Effective Practice and Organisation of Care

GP:

General Practitioner

PICO:

Population, Intervention, Comparison, Outcome

RCT:

Randomised Controlled Trial

SIGN:

Scottish Intercollegiate Guidelines Network

TTAS:

Telephone Triage and Advice Services

WebRCT:

Web Real Time Communication

References

  1. Jessopp L, Beck I, Hollins L, Shipman C, Reynolds M, Dale J. Changing the pattern out of hours: a survey of general practice cooperatives. Br Med J. 1997;314(7075):199.

    Article  CAS  Google Scholar 

  2. Hallam L. Out of hours primary care. Br Med J. 1997;314(7075):157.

    Article  CAS  Google Scholar 

  3. Salisbury C. The demand for out-of-hours care from GPs: a review. Fam Pract. 2000;17(4):340–7.

    Article  CAS  PubMed  Google Scholar 

  4. Lattimer V, George S. Nurse telephone triage in out-of-hours primary care. Prim Care Manag. 1996;6(5):3–6.

    Google Scholar 

  5. Rolland E, Moore KM, Robinson VA, McGuinness D. Using Ontario's Telehealth health telephone helpline as an early-warning system: a study protocol. BMC Health Serv Res. 2006;6(1):10.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Purc-Stephenson RJ, Thrasher C. Patient compliance with telephone triage recommendations: a meta-analytic review. Patient Educ Couns. 2012;87(2):135–42.

    Article  PubMed  Google Scholar 

  7. Carrasqueiro S, Oliveira M, Encarnação P. Evaluation of telephone triage and advice services: a systematic review on methods, metrics and results. Stud Health Technol Inform. 2011;169:407–11.

    PubMed  Google Scholar 

  8. Christensen MB, Olesen F. Out of hours service in Denmark: evaluation five years after reform. Br Med J. 1998;316(7143):1502.

    Article  CAS  Google Scholar 

  9. Driscoll P, Vincent CA, Wilkinson M. The use of the accident and emergency department. Arch Emerg Med. 1987;4(2):77–82.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Martin A, Martin C, Martin PB, Martin PA, Green G. Eldridge S: 'Inappropriate' attendance at an accident and emergency department by adults registered in local general practices: how is it related to their use of primary care? J Health Serv Res Policy. 2002;7(3):160–5.

    Article  PubMed  Google Scholar 

  11. Fry MM. A systematic review of the impact of afterhours care models on emergency departments, ambulance and general practice services. Australas Emerg Nurs. 2011;14(4):217–25.

    Article  Google Scholar 

  12. Huibers L, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: a systematic review. Scand J Prim Health Care. 2011;29(4):198–209.

    Article  PubMed  PubMed Central  Google Scholar 

  13. McLean S, Sheikh A, Cresswell K, Nurmatov U, Mukherjee M, Hemmi A, Pagliari C. The impact of telehealthcare on the quality and safety of care: a systematic overview. PLoS One. 2013;8(8):e71238.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Blank L, Coster J, O’Cathain A, Knowles E, Tosh J, Turner J, Nicholl J. The appropriateness of, and compliance with, telephone triage decisions: a systematic review and narrative synthesis. J Adv Nurs. 2012;68(12):2610–21.

    Article  PubMed  Google Scholar 

  15. Chapman JL, Zechel A, Carter YH, Abbott S. Systematic review of recent innovations in service provision to improve access to primary care. Br J Gen Pract. 2004;54(502):374–81.

    PubMed  PubMed Central  Google Scholar 

  16. Jang-Jaccard J, Nepal S, Celler B, Yan B. WebRTC-based video conferencing service for telehealth. Computing. 2016;98(1-2):169-93.

  17. Clarke A. Evidence-based evaluation in different professional domains: similarities, differences and challenges. In: Shaw IF, Greene JC, Mark MM, editors. Handbook of evaluation: policies, programs and practices. London: Sage; 2006. p. 559–81.

    Chapter  Google Scholar 

  18. Ekeland AG, Bowes A, Flottorp S. Methodologies for assessing telemedicine: a systematic review of reviews. Int J Med Inf. 2012;81(1):1–11.

    Article  Google Scholar 

  19. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inf. 2010;79(11):736–71.

    Article  Google Scholar 

  20. Salk ED, Schriger DL, Hubbell KA, Schwartz BL. Effect of visual cues, vital signs, and protocols on triage: a prospective randomized crossover trial. Ann Emerg Med. 1998;32(6):655–64.

    Article  CAS  PubMed  Google Scholar 

  21. McKinstry B, Hammersley V, Burton C, Pinnock H, Elton R, Dowell J, Sawdon N, Heaney D, Elwyn G, Sheikh A. The quality, safety and content of telephone and face-to-face consultations: a comparative study. Qual Saf Health Care. 2010;19(4):298–303.

    Article  CAS  PubMed  Google Scholar 

  22. National Health Performance Authority. The Framework's conceptual underpinning. http://www.aihw.gov.au/health-performance/performance-and-accountability-framework/4-conceptual-underpinning/. Accessed 28 Aug 2017.

  23. Bruyère Research Institute. AMSTAR – assessing the methodological quality of systematic reviews. https://amstar.ca/. Accessed: 19 June 2017.

  24. National Health Service. NHS choices. http://www.nhs.uk/NHSEngland/AboutNHSservices/Emergencyandurgentcareservices/Pages/NHS-111.aspx. Accessed 23 December 2015.

  25. Ng JY, Fatovich DM, Turner VF, Wurmel JA, Skevington SA, Phillips MR. Appropriateness of healthdirect referrals to the emergency department compared with self-referrals and GP referrals. Med J Aust. 2012;197:498–502.

    Article  PubMed  Google Scholar 

  26. Ministry of Health (Portugal). Saúde 24. http://www.saude24.pt/PresentationLayer/home_00.aspx. Accessed 23 December 2015.

  27. Bunn F, Byrne G, Kendall S. Telephone consultation and triage: effects on health care use and patient satisfaction. Cochrane Database Syst Rev. 2004;4:Cd004180.

    Google Scholar 

  28. Ismail SA, Gibbons DC, Gnani S. Reducing inappropriate accident and emergency department attendances: a systematic review of primary care service interventions. Br J Gen Pract. 2013;63(617):e813–20.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Cochrane Effective Practice and Organisation of Care. Effective practice and organisation of care (EPOC) Reources for review authors. http://epoc.cochrane.org/epoc-specific-resources-review-authors. Accessed 19 April 2016.

  30. Leibowitz R, Day S, Dunt D. A systematic review of the effect of different models of after-hours primary medical care services on clinical outcome, medical workload, and patient and GP satisfaction. Fam Pract. 2003;20(3):311–7.

    Article  PubMed  Google Scholar 

  31. Critical Appraisal Skills Programme (CASP). Making sense of evidence; CASP checklists. http://www.casp-uk.net/#!casp-tools-checklists/c18f8. Accessed 19 Apr 2016.

  32. Scottish Intercollegiate Guidelines Network: Critical appraisal: notes and checklists. http://www.sign.ac.uk/checklists-and-notes.html. Accessed 28 Aug 2017.

  33. Brebner JA, Brebner EM, Ruddick-Bracken H. Accident and emergency teleconsultation for primary care--a systematic review of technical feasibility, clinical effectiveness, cost effectiveness and level of local management. J Telemed Telecare. 2006;12(Suppl 1):5–8.

    Article  PubMed  Google Scholar 

  34. Lattimer V, George S, Thompson F, Thomas E, Smith H, Moore M, Bond H, Mullee M, Turnbull J, Glasper A. Safety and effectiveness of nurse telephone consultation in out of hours primary care: randomised controlled trial. BMJ. 1998;317(7165):1054–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Thompson F, George S, Lattimer V, Smith H, Moore M, Bond H, Turnbull J, Mullee M, Thomas E, Glasper A. Overnight calls in primary care: randomised controlled trial of management using nurse telephone consultation. BMJ. 1999;319(7222):1408.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Strasser PH, Levy JC, Lamb GA, Rosekrans J. Controlled clinical trial of pediatric telephone protocols. Pediatrics. 1979;64(5):553.

    CAS  PubMed  Google Scholar 

  37. Black A, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, McKinstry B, Procter R, Majeed A, Sheikh A. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011;8(1):e1000387.

  38. Huibers L, Giesen P, Wensing M, Grol R. Out-of-hours care in western countries: assessment of different organizational models. BMC Health Serv Res. 2009;9(1):105.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Mossialos E, Wenzl M, Osborn R, Anderson C. International profiles of health care systems, 2014. The Commonwealth Fund: New York; 2015.

    Google Scholar 

  40. Georgiou A, Marks A, Braithwaite J, Westbrook JI. Gaps, disconnections, and discontinuities—the role of information exchange in the delivery of quality long-term care. Gerontologist. 2013;53(5):770–9.

    Article  PubMed  Google Scholar 

  41. Georgiou A, Westbrook JI, Braithwaite J. An empirically-derived approach for investigating health information technology: the elementally entangled Organisational communication (EEOC) framework. BMC Med Inform Decis Mak. 2012;12(1):68.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Hordern A, Georgiou A, Whetton S, Prgomet M. Consumer eHealth - an overview of the research evidence and the implications for future policy. HIMJ. 2011;40(1):6–14.

    PubMed  Google Scholar 

  43. McKinstry B, Sheikh A. Unresolved questions in telephone consulting. J R Soc Med. 2006;99(1):2–3.

    Article  PubMed  PubMed Central  Google Scholar 

  44. The National Patient Safety Foundation’s Lucien Leape Institute. Safety is personal; partnering with patients and families for the safest care. http://patientcarelink.org/improving-patient-care/patient-family-engagement/. Accessed 28 Aug 2017.

Download references

Acknowledgements

Not applicable.

Funding

This review was supported by Healthdirect Australia.

Availability of data and materials

The datasets supporting the conclusions of this article are included within the article and its additional files.

Author information

Authors and Affiliations

Authors

Contributions

RL and AG made substantial contributions to conception and design of the review. RL collected the data and worked with AG, LL and JL to interpret the data and outcomes. RL drafted the manuscript. All authors were involved in the critical revision of the manuscript for important intellectual content. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Andrew Georgiou.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Search strategy. (DOCX 12 kb)

Additional file 2:

Results table. (DOCX 16 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lake, R., Georgiou, A., Li, J. et al. The quality, safety and governance of telephone triage and advice services – an overview of evidence from systematic reviews. BMC Health Serv Res 17, 614 (2017). https://doi.org/10.1186/s12913-017-2564-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-017-2564-x

Keywords