Table 3 Description of the four recommended indicators

Rationale and definitions

To ensure timely access for all referred patients, prioritisation of patients is needed. Priority of a referred patient is determined by a specialist based on severity of the condition and urgency because of social factors. It is defined by maximum (medically) acceptable waiting time (see footnote^a). Scarce information in referral letters implies a risk of incorrectly assigned priority. A review of the priority based on information from the first consultations can give an indication of the correctness of the first priority decision. Correct priority is defined as equal indications of acceptable waiting time in both instances, with waiting time divided into four categories.

Numerator

Number of referrals where there is a match between the priority given based on the referral letter and the priority seen as correct retrospectively based on 1–3 consultations.

Denominator

Number of all referrals.

Methodological challenges

Long waiting time implies a larger risk for changes in the patient’s mental state. It is therefore recommended to explore the impact of time between the first and second priority-settings. Guidelines for deciding acceptable waiting time must be clearly defined, and a common understanding of these among the specialists is needed.

Possible approaches for quality improvements

There may be disagreement between stakeholders (e.g., the patient, specialists and the GP) on optimal prioritisation of patients and acceptable waiting time. Data on the waiting time that is seen as acceptable by the referring GP and the patient, in addition to the specialists’ assessments, will provide a fuller picture of ‘correct prioritisation of patients’.

Rationale and definitions

If referral letters lack necessary information, initiatives to collect additional information, such as contacting the referring GP or the patient, can postpone the assessment of the referral. Then, sending a response letter to the patient and the referring GP is correspondingly delayed. High quality referral letters are expected to include enough information for priority setting without delay. Delay in the process of priority is defined as not sending the response letter after the first assessment because of insufficient referral information.

Numerator

Number of response letters sent after first assessment of the referral.

Denominator

Number of all referral letters assessed.

Methodological challenges

Contextual differences may affect the validity (e.g., the tradition for collecting additional information may vary between units).

Possible approaches for quality improvements

If the decision is made individually and not by regular assessment meeting, a more informative indicator, such as mean number of days delayed, can be used.

Rationale and definitions

To select the patients most in need, the present study recommended defining severity according to symptoms or situation rather than diagnosis. Severity can be determined by the existence of a combination of the following severity factors: severe mental illness/psychosis, risk of suicide, risk to others, in care of children, substance abuse and younger than 23 years [57]. High quality referral letters are expected to enable specialists to prioritise the patients most in need to a greater extent than low quality referral letters. ‘High priority patients’ are defined as patients suffering from three or more of the severity factors detected by a specialist after the first 1–3 consultations. Waiting time is defined as days from receipt of the referral letter by the specialised health care to the onset of (specialised) care.

Numerator

Median waiting time for patients with three or more severity factors.

Denominator

Median waiting time of all patients.

Methodological challenges

The cut-off at three severity factors currently lacks empirical support. There is a risk of false positive and false negative findings, as the presence of three severity factors does not always indicate a greater severity than the presence of two factors.

Possible improvements of the indicator

It is recommended to explore whether three is the most appropriate cut-off for the severity factors to define patients who should have less waiting time. Further, exploration of each factor’s impact can reveal whether the factors should be weighted to reduce the risk of false negative or false positive findings.

Rationale and definitions

High quality referral letters include information about necessary tests, examinations and treatment efforts that were conducted prior to the referral. The quality of referral letters is therefore expected to be positively correlated with the appropriateness of the referral. ‘Appropriate referral’ is defined as referrals assessed by the receiving specialist as appropriate on a dichotomous variable (Yes/No).

Numerator

Number of appropriate referrals.

Denominator

Number of all referrals.

Methodological challenges

The sensitivity to change is limited for dichotomous variables. There might be disagreement between primary care and specialised mental health care with regard to appropriateness. This indicator represents only the specialist health care provider’s perspective of appropriateness of referrals.

Possible improvements of the indicator

The reliability of an ordinal variable should be tested. The potential disagreement between service providers on the appropriateness of a referral can be explored.