Skip to main content
Download PDF

Effect of an educational intervention for telephone triage nurses on out-of-hours attendance: a pragmatic randomized controlled study

BMC Health Services Research volume 23, Article number: 4 (2023) Cite this article

Abstract

Background

Telephone triage has been established in many countries as a response to the challenge of non-urgent use of out-of-hours primary care services. However, limited evidence is available regarding the effect of training interventions on clinicians’ telephone consultation skills and patient outcomes.

Methods

This was a pragmatic randomized controlled educational intervention for telephone triage nurses in 59 Norwegian out-of-hours general practitioners’ (GPs) cooperatives, serving 59% of the Norwegian population. Computer-generated randomization was performed at the level of out-of-hours GP cooperatives, stratified by the population size. Thirty-two out-of-hours GP cooperatives were randomized to intervention. One cooperative did not accept the invitation to participate in the educational programme, leaving 31 cooperatives in the intervention group. The intervention comprised a 90-minute e-learning course and 90-minute group discussion about respiratory tract infections (RTIs), telephone communication skills and local practices.

We aimed to assess the effect of the intervention on out-of-hours attendance and describe the distribution of RTIs between out-of-hours GP cooperatives and list-holding GPs.

The outcome was the difference in the number of doctor’s consultations per 1000 inhabitants between the intervention and control groups during the winter months before and after the intervention. A negative binomial regression model was used for the statistical analyses. The model was adjusted for the number of nurses who had participated in the e-learning course, the population size and patients’ age groups, with the out-of-hours GP cooperatives defined as clusters.

Results

The regression showed that the intervention did not change the number of consultations for RTIs between the two groups of out-of-hours GP cooperatives (incidence rate ratio 0.99, 95% confidence interval 0.91–1.07). The winter season’s out-of-hours patient population was younger and had a higher proportion of RTIs than the patient population in the list-holding GP offices. Laryngitis, sore throat, and pneumonia were the most common diagnoses during the out-of-hours primary care service.

Conclusions

The intervention did not influence the out-of-hours attendance. This finding may be due to the intervention’s limited scope and the intention-to-treat design. Changing a population’s out-of-hours attendance is complicated and needs to be targeted at several organizational levels.

Peer Review reports

Background

Out-of-hours primary care service is a crucial part of a well-functioning health care system, which provides health care when general practitioners’ (GPs) offices are closed. The member countries of the Organisation for Economic Co-operation and Development have selected different models to provide this kind of service to their populations [1]. GP cooperatives are the dominant model used in Europe [2].

The non-urgent use of out-of-hours service, both in primary care and emergency department setting, is of concern, because it may crowd out patients with the highest need and lead to inefficient use of resources [3,4,5,6,7,8]. Telephone triage has been implemented in many countries as a response to the challenge of non-urgent use of out-of-hours service. However, evidence regarding the effect of this measure on the use of out-of-hours service is limited [7]. Nevertheless, most European countries have implemented or improved the quality of telephone triage during the last 10 years [2].

Some countries, such as the UK, have a national telephone triage and advice system. Norway has one national telephone number for out-of-hours service; however, nurses in local out-of-hours GP cooperatives handle the calls from the local population. A review from 2017 suggested that more than 50% of calls to the telephone triage and advice system could be dealt with by providing telephone advice alone [9]. However, only 23% of incoming calls to the Norwegian out-of-hours service are managed by nurses’ telephone counselling alone [10].

High variation has been reported in measured appropriateness of advice provided by the telephone triage and advice services [9]. High quality of nurses’ telephone consultations positively affects the appropriateness of triage decisions, and a positive correlation exists between the duration of nurses’ telephone consultations and the overall communication score [11, 12]. Patient-centred communication, active listening and advising, as well as structured calls, may improve the quality of telephone triage [12].

Overall, telephone triage in out-of-hours care appears to be safe because high-risk patients are mostly identified and handled correctly [13]. However, limited evidence is available regarding the triage of callers with mild-to-moderate symptoms, such as self-limiting acute respiratory tract infections (RTIs). These conditions are non-urgent and do not necessarily require medical attention out-of-hours, even in a primary care setting. They increase the workload and busyness in the out-of-hours care, which in turn may impair clinical quality of care, as demonstrated in the case of antibiotic prescription for RTIs [14]. Telephone triage has the potential to reduce the workload of GPs [9]. Nevertheless, telephone triage nurses are reluctant to call themselves gatekeepers to the out-of-hours GP cooperatives [15]. They rather describe themselves as service providers who want to reach a consensus with the caller about the right level of care. They argue that such agreement is easier to reach if they have proper knowledge of the relevant guidelines. Consequently, a gap exists between the needs and expectations of stakeholders of primary health care as well as the needs of and reality experienced by telephone triage nurses.

Norwegian citizens are entitled to have a list-holding GP who offer planned and emergent service for their listed patients in-hour [16]. The list-holding GPs are also obliged to work sessions in the out-of-hours service. Yet, other doctors such as locums and hospital doctors manage 42% of the out-of-hours consultations [17]. The municipalities organize their out-of-hours service according to local needs and geographical variations, alone or together with neighbouring municipalities. Forty-nine % of the services cover more than one municipality [18]. The median population size per out-of-hours service was 12,823 (450–699,827) in 2022 [19]. Out-of-hours GP cooperative (with more than 15 doctors taking turns being on duty) is the most common model for organizing out-of-hours service, but rota groups (four to fifteen doctors) and individual GP practices (less than four doctors) are also common [2]. A nurse or paramedic with a bachelor’s degree is required to answer all incoming calls to the local out-of-hours service, and at least one medical doctor must be available for physical consultations 24/7 [20]. There is no record on the number of doctors on duty, but the median number of nurses employed per out-of-hours service was 17.5 in 2018, with large variation (1- > 50) [21]. In most services the nurses rotate between answering the telephone from the public and supporting the doctor in the clinical work. Primary care physicians, i.e., list holding GPs and other out-of-hours doctors, are mandatory gatekeepers to secondary health care 24/7. Patients pay a medical fee per consultation, which is 75% higher out-of-hour than in-hour. When the maximum amount of 2921 NOK (295 Euros) per year is paid, any further consultations in the current year are free of charge.

High primary care physician continuity leads to lower costs in health care, fewer hospitalizations, and lower mortality [22, 23]. Hence, one can argue that callers who can wait for the list-holding GP’s opening hours should wait, including those with RTIs. A long-lasting doctor–patient relationship is a better basis for practising a restrained antibiotic prescribing attitude than a busy out-of-hours session [14]. A transfer of 20% of patients from Norwegian out-of-hours service to day-time practice would lead to a 1.6% increase in the number of consultations for list-holding GPs [24, 25]. Despite work overload and recruitment problems in the Norwegian regular GP scheme, we believe that this increase would be manageable and pertinent [26].

Specific training may strengthen telephone triage nurses’ gatekeeping capacity. However, insufficient evidence is available to conclude the effectiveness of interventions aimed at reducing emergency department attendance [27]. Existing evidence is characterized by heterogeneous patient groups, unspecific target groups and the lack of control groups [27]. A Cochrane report from 2017 concluded that more research assessing the effect of different training interventions on clinicians’ telephone consultation skills and their effect on patient outcomes is needed [28].

The primary aim of this study was to assess the effect of an educational intervention about RTIs and communication skills for telephone triage nurses on out-of-hours attendance for RTIs. The secondary aims were to describe the distribution of RTIs between list-holding GPs and out-of-hours GP cooperatives and assess the intervention’s effect on list-holding GPs’ attendance.

Methods

Study design

This is a pragmatic randomized controlled educational intervention among Norwegian telephone triage nurses working in out-of-hours GP cooperatives. All Norwegian out-of-hours GP cooperatives that met our inclusion criteria were included in the study (Table 1).

Table 1 Inclusion criteria for out-of-hours GP cooperatives
Full size table

The out-of-hours services in the two largest cities of Norway, Bergen, and Oslo, were not included owing to their policy of direct attendance. Out-of-hours cooperatives serving less than 10,000 inhabitants were excluded as they normally will have less than 6 consultations per 24 hours and therefore are less busy [24]. Also, these cooperatives seldom have conflicts of simultaneity and are therefore less likely to use triage scales [18].

A total of 64 out-of-hours GP cooperatives serving 3.4 million inhabitants (median population size per out-of-hours GP cooperative 36.048, range 11.490–236.202) met the inclusion criteria and were randomized to intervention or control. An independent researcher performed the randomization using the RAND command in Microsoft Excel (2016) in November 2018, with stratification by the population size (small 10,000–40,000; large > 40,001).

We contacted the administration of the 32 out-of-hours cooperatives in the intervention group through e-mail or telephone and invited them to participate in a meeting during the annual leadership conference for the primary care out-of-hours service in Norway (Lederkonferansen 2019) in March 2019. In this meeting, we presented the educational intervention and explained how they could implement it locally without any external support. Nineteen persons from 16 out-of-hours GP cooperatives participated. BHL contacted the other 16 out-of-hours GP cooperatives, three by personal contact during the conference and the remaining via telephone calls after the conference. One of the 32 cooperatives refused to participate due to time constraint, while the other 31 accepted the invitation and were included in the study. Data from one out-of-hours cooperative in the intervention group, in addition to one out-of-hours cooperative and one municipality in the control group, were excluded from the study because of the merging of municipalities across the two groups during the study period. Data from two out-of-hours cooperatives (serving 180,989 inhabitants) in the control group were excluded because of data error. As a result, 59 out-of-hours GP cooperatives serving 59% of the Norwegian population, were included in the intention-to-treat analysis (Fig. 1).

Fig. 1
figure 1

Flow chart of out-of-hours cooperatives included and reasons for exclusion after randomization

Full size image

The intervention

The intervention consisted of two parts:

  1. 1.

    A 90 minute e-learning course about acute RTIs and general communication techniques for telephone triage nurses (see Appendix A for the outline). The authors prepared the course based on preliminary data from a qualitative study on telephone triage nurses’ assessment of RTIs [15].

  2. 2.

    A 90 minute group discussion for the telephone triage nurses. Dedicated local nurses led the group sessions based on a written guide (Appendix B). The main elements were to discuss gained knowledge from the e-learning course and to discuss the unique activity reports for each out-of-hours GP cooperative (Appendix C). The reports described how the population attended each out-of-hours GP cooperative and the respective GP offices during 2018, with emphasis on acute RTIs. The guide and reports were prepared by BHL and SH and were based on data from Statistics Norway and aggregated data from the open database of the Norwegian Control and Payment of Health Reimbursement registry.Footnote 1

The intervention was piloted by ten telephone triage nurses at one out-of-hours GP cooperative in May 2019 and adjusted based on the feedback from these nurses.

The unique activity reports (Appendix C) were sent to the intervention cooperatives by mail and e-mail in September 2019. The e-learning course was launched password-protected on a web platform in October 2019, exclusively for the intervention cooperatives [29]. We encouraged the local contacts to implement both parts of the intervention before December 2019. The pragmatic intention-to-treat design implied no follow-up of the implementation. However, through the log-on function of the web platform, we could see how many nurses had participated in the e-learning course.

Variables

GP offices and out-of-hours GP cooperatives send electronic compensation claims to the Norwegian Health Economics Administration. The claims contain data on date and type of contact (phone, consultation, or home visit) with GP offices and out-of-hours GP cooperatives as well as age, gender, and diagnosis of the caller/patient. We retrieved these data from the National Directory of Health [30].

We did not have access to the number of nurses working in each out-of-hours cooperative during the intervention. The number of nurses who had just started or completed the e-learning course per 1000 inhabitants in each region was therefore used as a proxy for the proportion of trained nurses in each out-of-hours GP cooperative.

The primary outcome was the change in the number of out-of-hours doctor’s consultations for RTIs per 1000 inhabitants in the intervention group versus the control group during the winter months before (December 2018–February 2019) and after (December 2019–February 2020) the intervention. The secondary outcomes per 1000 inhabitants during the same two periods were:

  • the change in the number of list-holding GPs’ consultations for all diagnoses

  • the change in the number of list-holding GPs’ consultations for RTIS

  • the change in the number of telephone contacts for all diagnoses for list-holding GPs

  • the change in the number of nurses’ telephone consultations for all diagnoses in the out-of-hours GP cooperatives

We defined RTIs according to the International Classification of Primary Care, 2nd edition, as the following groups: ‘respiratory symptoms’ (R01–05, R08–09, R21 and R23–29), acute tonsillitis (R72 and 76), acute RTIs (R74), acute sinusitis (R75), acute laryngitis (R77), acute bronchitis (R78), influenza infection (R80), pneumonia (R81–82), ‘other RTIs’ (R71, R83 and R99) and ear infections (H01, H29, H70–72 and H74). We also defined the groups urinary tract infections (U01–02, U07, U13 and U70–72), ‘other conditions’ (A03, A76–78), ‘unspecified’ (A99) and ‘all other diagnoses’ (all International Classification of Primary Care codes not included in any of the above-mentioned groups).

Statistical analysis

We used frequencies and percentages to describe the distribution of the sample population and count data relating to contacts and consultations made in the control and intervention arms, and StataSE 17 (StataCorp. 2021. Stata Statistical Software: Release 17. College Station, TX: StataCorp LLC) for the statistical analyses.

The Poisson regression is the basic model used for modelling count data. It assumes that the mean and the variance of the count variable are approximately equal. We checked this model assumption using the dispersion statistic, which should be equal to one. However, our Poisson model output gave the dispersion statistic equal to 24.2. In addition, the appropriateness of the negative binomial model over the Poisson regression model was checked using the Bayesian information criterion (BIC), which states that the model with the smaller BIC estimate is preferred over the model with a larger BIC. The negative binomial model had the smaller BIC estimate. Thirdly, we obtained a Z score test of 16.1, with a t-probability < 0.01. This test evaluates whether the data are Poisson or negative binomial. Based on this result, we rejected the null hypothesis of no overdispersion in the data. Further, we used the likelihood-ratio test of alpha = 0, to test if the dispersion parameter, was equal to zero, which would have reduced the model to a Poisson regression. Our results showed that alpha was significantly greater than zero. This implies that the data were over-dispersed, hence using the negative binomial model instead of the Poisson model was appropriate.

The negative binomial regression model was used with random effects at the level of out-of-hours GP cooperatives to check for any baseline differences between the two groups (Table 2) and for the primary (Table 3) and secondary outcomes. The model was adjusted for the number of nurses who had only started or completed the e-learning course (per 1000 inhabitants), the population size and patients’ age groups, with the out-of-hours GP cooperatives defined as clusters. We also run a three-way interaction model as a sensitivity analysis for the variable of the number of nurses who had started or completed the e-learning course (per 1000 inhabitants).

We obtained estimates of incidence rate ratios (IRRs) for the negative binomial regression model. An IRR > 1 indicates an increase in counts in one group relative to the reference group or one period relative to the reference period, whereas an IRR < 1 indicates a decrease in these parameters. An IRR = 1 indicates no difference in the number of counts between the groups or periods.

Results

Population and epidemiology at baseline

The 59 out-of-hours cooperatives served 3.12 million inhabitants (59% of the Norwegian population) in 198 (of 356) municipalities. There were 4.72 million contacts in primary health care in these municipalities in the baseline period, of which 4.37 million were list-holding GPs’ consultations and 0.35 million were GP consultations in the out-of-hours service.

Women constituted 52.5 and 57.8% of the patients who received service in out-of-hours cooperatives and GP offices, respectively. Patients were relatively younger in the out-of-hours cooperatives than in the GP offices (Fig. 2).

Fig. 2
figure 2

Proportion of age groups in GP offices and out-of-hours GP cooperatives, all diagnoses. P-values obtained from Chi-square test

Full size image

For RTIs, 34.4 and 16.5% of children under 5 years of age received a diagnosis in out-of-hours cooperatives and GP offices, respectively (Fig. 3).

Fig. 3
figure 3

Proportion of age groups in GP offices and out-of-hours GP cooperatives, RTIs. P-values obtained from Chi-square test

Full size image

Respiratory symptoms and acute upper RTI were the two most frequently diagnosed RTIs (Fig. 4), and 13.9% of all RTIs were diagnosed in out-of-hours cooperatives.

Fig. 4
figure 4

Proportion of different RTIs in GP offices and out-of-hours GP cooperatives. All the differences between GP offices and out-of-hours GP cooperatives, except for ear infections, were statistically significant (p < 0.05, Chi square test)

Full size image

The proportion of RTIs diagnosed was higher in out-of-hours cooperatives than in the GP offices (22.9% versus 14.0%). Laryngitis, other conditions (viral RTIs frequently diagnosed in children) and sore throat were the most common diagnoses in the out-of-hours GP cooperatives (Fig. 5).

Fig. 5
figure 5

Proportion of conditions diagnosed in out-of-hours GP cooperatives

Full size image

Effect of the intervention

At baseline, no significant difference was observed between the intervention and control groups with regard to how the out-of-hours GP cooperatives and the list-holding GPs were used by the population (Table 2).

Table 2 Characteristics of the groups and adjusted estimates of IRR and their 95% CIs obtained from the negative binomial regression model showing the differences in counts between the groups at baseline
Full size table

By the end of November 2019, 286 nurses (61% of registered participating nurses) had completed the e-learning course. The negative binomial regression showed that the intervention led to no significant difference in the number of consultations for RTIs between the out-of-hours GP cooperatives in the intervention and control groups (IRR 0.99, 95% CI 0.91–1.07; Table 3).

Table 3 Adjusted estimates of IRR and their 95% CIs obtained from the negative binomial regression model showing the differences in counts between the groups before and after the intervention
Full size table

The intervention did neither result in significant differences with regard to the number of telephone consultations for RTIs for out-of-hours GP cooperatives (IRR 0.83, 95% CI 0.66–1.07), the number of list-holding GPs’ consultations for all diagnoses (IRR 0.98, 95% CI 0.95–1.01) or RTIs (1.00, 95% CI 0.94–1.06) nor for telephone consultations for all diagnoses for list-holding GPs (IRR 1.02, 95% CI 0.97–1.07) or out-of-hours GP cooperatives (IRR 0.97, 95% CI 0.86–1.10).

The three-way interaction effect that could have explained how the slope of the number of RTI-consultations varies as a function of nurses per 1000 inhabitants who attended the e-learning course per study time and study group could not be estimated, as illustrated in the margins plot in Fig. 6.

Fig. 6
figure 6

Predicted number of RTI-consultations in the out-of-hours GP cooperatives as a function of the number of nurses per 1000 inhabitants who started or completed the e-learning course per study time and group

Full size image

Discussion

The patient population attending the out-of-hours service during the winter season was younger and had a higher proportion of RTIs than the population attending the regular GP offices. The educational intervention about RTIs and communication skills for telephone triage nurses did not decrease the number of consultations for RTIs in the out-of-hours GP cooperatives.

Population and epidemiology at baseline

The out-of-hours patient population is characterized by a higher proportion of children and teenagers compared with list-holding GPs’ patient population, especially for RTIs. Danish parents reported a perceived need for prompt action, unpleasant symptoms and worry as some of the main reasons for contacting the out-of-hours service [31]. This is coherent with a high rate of out-of-hours consultations for the youngest children, because they often become abruptly ill with unpleasant symptoms. Nevertheless, generally harmless conditions and insecurity concerning the assessment of respiratory symptoms of their own children may lead parents to call the out-of-hours service. Telephone triage nurses describe this group of callers as challenging, and the parents’ high degree of worry will often overrule the nurses’ medical assessment, contributing to a high consultation rate for children [15].

The proportion of sore throat diagnosed in the out-of-hours cooperatives is high. In our data, sore throat is consistent with suspected or detected streptococcal tonsillitis. There is a strong Norwegian tradition of performing rapid strep A test and initiating treatment with penicillin V if streptococci are confirmed. This tradition may contribute to high expectations in the population of a doctors’ assessment for sore throat, leading to high out-of-hours consultation rates [15].

Acute laryngitis is a hyperacute condition and tends to arise during night-time. Therefore, the high proportion of patients being diagnosed with this condition out-of-hours is unsurprising. However, the proportion of sinusitis cases was low. This finding is consistent with the low-urgency nature of sinusitis and the existing guidelines recommending a wait-and-watch attitude for this condition [32].

Effect of the intervention

Dutch GPs believe that a stricter triage and annual feedback to triage nurses will reduce the number of non-urgent visits to the out-of-hours cooperatives [8]. This viewpoint is supported by a Finnish training intervention among nurses in an emergency unit that improved the quality of telephone triage as measured using self-administered questionnaires [33]. Self-reported increase in quality as measured using questionnaires may be an indication of quality improvement, but the lacking patient outcome is a major weakness of this and other studies on educational interventions for telephone triage nurses [28]. The present study meets the need for data on patient outcome. However, the intervention would have benefited from a process analysis to evaluate whether it made the nurses more confident in assessing callers with symptoms of mild-to-moderate RTIs.

The decision to seek health care is a complex process influenced by several factors, such as personal, social, and cultural, as well as characteristics of the health care system [4, 34]. An introduction of school requirement for a sick leave certificate for teenagers led to increased list-holding GP attendance and may contribute to out-of-hours health seeking behaviour [35]. Convenience considerations and GPs’ low capacity for urgent, same-day consultations may be other reasons for seeking help out-of-hours [31, 36, 37]. Moreover, social network is also an important determinant for help seeking behaviour regarding antibiotic use [38]. Telephone triage nurses described the fee-for-service plan for GPs working in the out-of-hours cooperatives as a perverse incentive, pushing non-urgent consultations out-of-hours [15].

Hence, there may be several reasons for the lacking effect of our educational intervention on the primary and secondary outcomes. Strategies aimed at reducing out-of-hours attendance by improving the skills of individual nurses imply that they perform an active gatekeeping role, keeping low-urgency patients out of the cooperatives. From this perspective, an educational intervention similar to the one described in the present study could be expected to have a direct effect on the out-of-hours attendance, particularly because nurses report that the decision regarding the need for a doctor’s consultation is generally made in co-operation between the nurse and caller [15].

However, our e-learning course and group discussion of 90 minutes each may be too limited to oust the external factors telephone triage nurses describe as decisive and non-controllable [15]. This point of view is supported by a Swedish study that concluded that multiple organizational factors, including the triage performed and the self-care advice provided by the nurses, play an important role in primary health care centres where antibiotic prescribing is low [39]. Hence, keeping non-urgent patients from doctors’ consultations out-of-hours is not a simple decision for one particular nurse. As for the problem of crowding in emergency departments in hospitals, the challenge might be better addressed through system changes [40].

Methodological considerations

The data from the Norwegian Directory of Health are reliable and a good source of information about the study population as well as activity in primary health care [37]. Our study covers almost 60% of the Norwegian population, indicating high external validity in health care systems similar to the Norwegian health care system. We consider the data to be a reliable source of information about the epidemiology of RTIs during the two winter seasons. The first Norwegian case of coronavirus disease 2019 appeared on 26 February 2020, i.e., towards the end of the intervention period [41]. Hence, the effect of the coronavirus disease 2019 pandemic on the data is limited.

The large proportion of the population being covered by the included out-of-hours GP cooperatives, in addition to narrow confidence intervals, indicate that the present study has statistical power to answer our research question.

The e-learning course encouraged the nurses to not overrule the triage system in use in their own out-of-hours GP cooperative. Because many triage systems are less specific with regard to RTIs, generally with high sensitivity and low specificity for serious disease, this may have been an obstacle for change [42]. In this way, the e-learning course may have contributed to the nurses feeling more confident, without leading to a decrease in the number of callers being assigned a doctor’s consultation.

The pragmatic intention-to-treat design yielded limited control on the implementation of the intervention. We do not know how many nurses completed the two educational parts, and how many nurses had access to the course without attending it. This makes the assessment of the effect of the intervention more uncertain. To compensate for this weakness, we included the number of nurses who had started on or completed the e-learning course per 1000 inhabitants as a variable in the regression model. This variable did not have a significant effect (IRR 0.98, 95% CI 0.80–1.19). The sensitivity analysis of this variable was not possible to perform, since the nurses in the control group were not exposed and the proportion of nurses in this group is zero.

A longer intervention period might have increased the likelihood of a higher participation rate among nurses, but it would also increase the risk of external bias from the coronavirus disease 2019 pandemic. Hence, it is difficult to conclude about a lacking effect of the intervention based only on the primary and secondary outcomes. A pragmatic design is closer to real life and healthcare. We therefore considered it a pertinent method.

Conclusion

The described educational intervention for telephone triage nurses did not influence the out-of-hours attendance for RTIs or list-holding GPs’ attendance. Changing a population’s health service attendance is complicated, and the intervention’s pragmatic design with a lack of control on the proportion of participating nurses, and its limited scope, may be reasons for the missing effect on the primary and secondary outcomes. Future studies on educational interventions should include a process evaluation, be planned for a longer period, and involve both health planning councils and leaders at different levels, as well as GPs and nurses in out-of-hours cooperatives. Furthermore, a need exists to explore the perspective of callers with RTIs who are advised to wait-and-watch or seek their list-holding GPs to reduce non-urgent use of the out-of-hours service.

Availability of data and materials

The raw data analysed during the current study contains person sensitive information and publication of these data has not been approved by The Norwegian Data Protection Authority. However, a modified dataset is available from the corresponding author on reasonable request.

Notes

  1. Lindberg BH, Høye S. Own report. [Raw data fetched from the KUHR-database 19.05.2019 at [https://opnedata.helserefusjon.no/]. Oslo: Helsedirektoratet [The Norwegian Directorate of Health]; 2019.

Abbreviations

CI:

Confidence interval

GP:

General practitioner

IRR:

Incidence rate ratio

RTIs:

Respiratory tract infections

References

  1. Berchet C, Nader C. The organisation of out-of-hours primary care in OECD countries. OECD Health Working Papers, No. 89. Paris: OECD Publishing; 2016.

  2. Steeman L, Uijen M, Plat E, Huibers L, Smits M, Giesen P. Out-of-hours primary care in 26 European countries: an overview of organizational models. Fam Pract. 2020;37(6):744–50.

    Article  Google Scholar 

  3. Keizer E, Smits M, Peters Y, Huibers L, Giesen P, Wensing M. Contacts with out-of-hours primary care for nonurgent problems: patients' beliefs or deficiencies in healthcare? BMC Fam Pract. 2015;16:157.

    Article  Google Scholar 

  4. Keizer E, Christensen MB, Carlsen AH, Smits M, Wensing M, Senn O, et al. Factors related to out-of-hours help-seeking for acute health problems: a survey study using case scenarios. BMC Public Health. 2019;19:33.

    Article  Google Scholar 

  5. Huibers L, Keizer E, Carlsen AH, Moth G, Smits M, Senn O, et al. Help-seeking behaviour outside office hours in Denmark, the Netherlands and Switzerland: a questionnaire study exploring responses to hypothetical cases. BMJ Open. 2018;8(10):e019295.

    Article  Google Scholar 

  6. Nørøxe KB, Huibers L, Moth G, Vedsted P. Medical appropriateness of adult calls to Danish out-of-hours primary care: a questionnaire-based survey. BMC Fam Pract. 2017;18:34.

    Article  Google Scholar 

  7. Ismail SA, Gibbons DC, Gnani S. Reducing inappropriate accident and emergency department attendances: a systematic review of primary care service interventions. Br J Gen Pract. 2013;63(617):e813–20.

    Article  Google Scholar 

  8. Keizer E, Maassen I, Smits M, Wensing M, Giesen P. Reducing the use of out-of-hours primary care services: a survey among Dutch general practitioners. Eur J Gen Pract. 2016;22(3):189–95.

    Article  Google Scholar 

  9. Lake R, Georgiou A, Li J, Li L, Byrne M, Robinson M, et al. The quality, safety and governance of telephone triage and advice services - an overview of evidence from systematic reviews. BMC Health Serv Res. 2017;17(1):614.

    Article  Google Scholar 

  10. Midtbø V, Raknes G, Hunskaar S. Telephone counselling by nurses in Norwegian primary care out-of-hours services: a cross-sectional study. BMC Fam Pract. 2017;18(1):84.

    Article  Google Scholar 

  11. Huibers L, Keizer E, Giesen P, Grol R, Wensing M. Nurse telephone triage: good quality associated with appropriate decisions. Fam Pract. 2012;29(5):547–52.

    Article  CAS  Google Scholar 

  12. Derkx HP, Rethans JJ, Maiburg BH, Winkens RA, Muijtjens AM, van Rooij HG, et al. Quality of communication during telephone triage at Dutch out-of-hours centres. Patient Educ Couns. 2009;74(2):174–8.

    Article  Google Scholar 

  13. Huibers L, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: a systematic review. Scand J Prim Health Care. 2011;29(4):198–209.

    Article  Google Scholar 

  14. Lindberg BH, Gjelstad S, Foshaug M, Hoye S. Antibiotic prescribing for acute respiratory tract infections in Norwegian primary care out-of-hours service. Scand J Prim Health Care. 2017;35(2):178–85.

    Article  Google Scholar 

  15. Lindberg BH, Rebnord IK, Høye S. Phone triage nurses’ assessment of respiratory tract infections – the tightrope walk between gatekeeping and service providing. A qualitative study. Scand J Prim Health Care. 2021;39(2):139–47.

    Article  Google Scholar 

  16. Forskrift om fastlegeordning i kommunene [Regulations on the general practitioner scheme in the municipalities] FOR-2012-08-29-842 Oslo: Helse- og omsorgsdepartementet [Ministry of Health and Care Services]; 2013 [cited 2022 06.04]. Available from: https://lovdata.no/dokument/SF/forskrift/2012-08-29-842?q=fastlegeforskriften.

  17. Sandvik H, Hunskår S, Blinkenberg J. Årsstatistikk fra legevakt 2021. [Yearly statistics from primary care out-of-hours service 2021] Bergen: Nasjonalt kompetansesenter for legevaktmedisin [The National Centre for Emergency Primary Health Care], Norce Norwegian Research Centre; 2022. Report No.: 1–2022.

  18. Allertsen M, Morken T. Legevaktorganisering i Norge. Rapport fra Nasjonalt legevaktregister 2020. [Organisation of Norwegian out-of-hours care. Report from the National registry of out-of-hours primary care 2020.]. Bergen: Nasjonalt kompetansesenter for legevaktmedisin [The National Centre for Emergency Primary Health Care], Norce Norwegian Research Centre; 2021. Report No.: 3-2021.

  19. Allertsen M. Nasjonalt legevaktregister [National registry of the out-of-hours service] Bergen: Nasjonalt kompetansesenter for legevaktmedisin [The National Centre for Emergency Primary Health Care], Norce Norwegian Research Centre; 2022 [cited 2022 10.11]. Available from: https://www.norceresearch.no/prosjekter/nasjonalt-legevaktregister.

  20. Forskrift om krav til og organisering av kommunal legevaktordning, ambulansetjeneste, medisinsk nødmeldetjeneste mv. (Akuttmedisinforskriften). [Regulation on organisation of emergency services] FOR-2022-05-20-880 Oslo: Helse og omsorgsdepartementet [Ministry of Health and Care Services]; 2015 [cited 2022 10.11]. Available from: https://lovdata.no/dokument/SF/forskrift/2015-03-20-231?q=akuttmedisin.

  21. Morken T, Solberg LR, Allertsen M. Legevaktorganisering i Norge. Rapport fra Nasjonalt legevaktregister 2018 [Organisation of Norwegian out-of-hours care. Report from the National registry of out-of-hours primary care 2020]. Bergen: Nasjonalt kompetansesenter for legevaktmedisin [The National Centre for Emergency Primary Health Care], NORCE Norwegian Research Centre; 2019. Report No.: 1891-3474.

  22. Bazemore A, Petterson S, Peterson LE, Bruno R, Chung Y, Phillips RL Jr. Higher primary care physician continuity is associated with lower costs and hospitalizations. Ann Fam Med. 2018;16(6):492–7.

    Article  Google Scholar 

  23. Sandvik H, Hetlevik Ø, Blinkenberg J, Hunskaar S. Continuity in general practice as predictor of mortality, acute hospitalisation, and use of out-of-hours care: a registry-based observational study in Norway. Br J Gen Pract. 2022;72(715):e84–90.

    Article  Google Scholar 

  24. Satistics Norway. GPs and emergency primary health care. 10903: Emergency Primary Health Care consultations, by age, sex and diagnosis 2014 - 2020 Oslo: Statistics Norway; 2021 [cited 2021 03.30]. Available from: https://www.ssb.no/en/statbank/table/10903/.

  25. Statistics Norway. GPs and emergency primary health care. 10141: General practitioners consultations, by age, sex and diagnosis 2012 - 2021 Oslo: Statistcis Norway; 2022 [cited 2022 03.02]. Available from: https://www.ssb.no/en/statbank/table/10903/.

  26. Theie MG, Lind LH, Haugland LM, Skogli E. Fastlegeordningen i krise - hva sier tallene [RGP scheme in crisis - what do the numbers tell?]. Report number 83/2018. Oslo: Menon Economics; 2018.

    Google Scholar 

  27. Van den Heede K, Van de Voorde C. Interventions to reduce emergency department utilisation: a review of reviews. Health Policy. 2016;120(12):1337–49.

    Article  Google Scholar 

  28. Vaona A, Pappas Y, Grewal RS, Ajaz M, Majeed A, Car J. Training interventions for improving telephone consultation skills in clinicians. Cochrane Database Syst Rev. 2017;1(1):Cd010034.

    Google Scholar 

  29. Oppvakt - opplæring på legevakt [Bright - education in primary care out-of-hours service]. Bergen: Nasjonalt kompetansesenter for legevaktmedisin [The National Centre for Emergency Primary Health Care], Norce Norwegian Research Centre; 2019 [cited 2021 05.03]. Available from: https://kurs.uni.no/login.

  30. KUHR-databasen [KUHR database]. Oslo: Helsedirektoratet [The Norwegian Directorate of Health]; 2019

  31. Moth G, Christensen MB, Christensen HC, Carlsen AH, Riddervold IS, Huibers L. Age-related differences in motives for contacting out-of-hours primary care: a cross-sectional questionnaire study in Denmark. Scand J Prim Health Care. 2020;38(3):272–80.

    Article  Google Scholar 

  32. The Antibiotic Centre for Primary Care. Antibiotikabruk i primærhelsetjenesten [Antibiotic use in primary health care]. Oslo: Helsedirektoratet og Antibiotikasenteret for primærmedisin [The Norwegian Directorate of Health and The Antibiotic Centre for Primary Care]; 2021 [cited 2021 Aug 13]. Available from: https://www.antibiotikaiallmennpraksis.no/.

  33. Kaakinen P, Kyngas H, Tarkiainen K, Kaariainen M. The effects of intervention on quality of telephone triage at an emergency unit in Finland: Nurses' perspective. Int Emerg Nurs. 2016;26:26–31.

    Article  Google Scholar 

  34. van Loenen T, van den Berg MJ, Faber MJ, Westert GP. Propensity to seek healthcare in different healthcare systems: analysis of patient data in 34 countries. BMC Health Serv Res. 2015;15:465.

    Article  Google Scholar 

  35. Bakken IJ, Wensaas KA, Furu K, Grøneng GM, Stoltenberg C, Øverland S, et al. General practice consultations and use of prescription drugs after changes to school absence policy. Tidsskr Nor Legeforen. 2017;137:16.

  36. Huibers L, Carlsen AH, Moth G, Christensen HC, Riddervold IS, Christensen MB. Patient motives for contacting out-of-hours care in Denmark: a cross-sectional study. BMC Emerg Med. 2020;20(1):20.

    Article  Google Scholar 

  37. Sandvik H, Hunskår S, Diaz E. Use of emergency medical services by patients encompassed by the Regular GP scheme. Tidsskr Nor Laegeforen. 2012;132(20):2272–6.

    Article  Google Scholar 

  38. Ellis J, Vassilev I, Kennedy A, Moore M, Rogers A. Help seeking for antibiotics; is the influence of a personal social network relevant? BMC Fam Pract. 2019;20(1):63.

    Article  CAS  Google Scholar 

  39. Strandberg EL, Brorsson A, Andre M, Grondal H, Molstad S, Hedin K. Interacting factors associated with low antibiotic prescribing for respiratory tract infections in primary health care - a mixed methods study in Sweden. BMC Fam Pract. 2016;17:78.

    Article  Google Scholar 

  40. Panovska-Griffiths J, Ross J, Elkhodair S, Baxter-Derrington C, Laing C, Raine R. Exploring overcrowding trends in an inner city emergence department in the UK before and during COVID-19 epidemic. BMC Emerg Med. 2021;21(1):43.

    Article  CAS  Google Scholar 

  41. Seppälä E, Tønnessen R, Veneti L, Paulsen TH, Steens A, Whittaker R, et al. COVID-19 cases reported to the Norwegian Institute of Public Health in the first six weeks of the epidemic. Tidsskr Nor Laegeforen. 2020;140:18.

  42. Advanced Life Support Group. Manchester Telephone Triage and Advice, Manchester: Advanced Life Support group; 2020 [cited 2022 10.21]. Available from: https://www.triagenet.net/classroom/course/view.php?id=1203&lang=en.

  43. Helseforskningsloven. Lov om medisinsk og helsefaglig forskning. [the Norwegian Act on Medical and Health Research.] LOV-2008-06-20-44 Oslo: Helse- og omsorgsdepartementet [Ministry of Health and Care Services]; 2009. [cited 2022 04.01]. Available from: https://lovdata.no/dokument/NL/lov/2008-06-20-44.

Download references

Acknowledgements

Ibrahimu Mdala (Researcher, Department of General Practice, Institute of Health and Society, University of Oslo) provided invaluable help with the statistical analyses and descriptions. Arngeir Berge (Senior Adviser and Educationalist, National Centre for Emergency Primary Health Care, Norway) converted our texts about communication and respiratory tract infections into a vivid and interesting e-learning course. Elisabeth Holm Hansen (Professor, Faculty of Health and Social Sciences, University of South-Eastern Norway) and Arnstein Finset (Professor Emeritus, Faculty of Medicine, University of Oslo) validated the content of the e-learning course. A group of experienced out-of-hours telephone triage nurses (Siri-Linn Schmidt Fotland, Astrid Onarheim Spjeldnæs and Kjersti Stadshaug) provided valuable feedback, and the course was adjusted accordingly.

Funding

This work was supported by The Norwegian Research Fund for General Practice as a PhD grant to the first author.

Author information

Authors and Affiliations

  1. Antibiotic Centre for Primary Care, Department of General Practice, Institute of Health and Society, University of Oslo, 0315, Oslo, Norway

    Bent Håkan Lindberg & Sigurd Høye

  2. National Centre for Emergency Primary Health Care, NORCE Norwegian Research Centre, Bergen, Norway

    Ingrid Keilegavlen Rebnord

  3. Department of Global Public Health and Primary Care, University of Bergen, Bergen, Norway

    Ingrid Keilegavlen Rebnord

Authors
  1. Bent Håkan Lindberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ingrid Keilegavlen Rebnord
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sigurd Høye
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study design and the educational intervention. IKR identified the out-of-hours cooperatives that met the inclusion criteria. BHL administrated the intervention, prepared the raw data for analysis, and performed the analysis with support from SH. All the authors contributed to the interpretation of the findings. BHL wrote the initial manuscript draft. All the authors reviewed and refined the manuscript and have approved the final version.

Corresponding author

Correspondence to Bent Håkan Lindberg.

Ethics declarations

Ethics approval and consent to participate

The study protocol was assessed by The Regional Committee for Research Ethics (2018/1080/REK sør-øst C), which concluded that the study, according to the Norwegian Act on medical and health research, did not need ethical approval [43]. The Norwegian Data Protection Authority approved the use of the data in this project (542881). The leaders of each out-of-hours GP cooperative in the intervention group gave consent on behalf of the participating nurses, and they informed the local nurses about the study. The consent form was approved by The Norwegian Data Protection Authority. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lindberg, B.H., Rebnord, I.K. & Høye, S. Effect of an educational intervention for telephone triage nurses on out-of-hours attendance: a pragmatic randomized controlled study. BMC Health Serv Res 23, 4 (2023). https://doi.org/10.1186/s12913-022-08994-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-022-08994-0

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us