"Mind the gap!" Evaluation of the performance gap attributable to exception reporting and target thresholds in the new GMS contract: National database analysis

Background The 2003 revision of the UK GMS contract rewards general practices for performance against clinical quality indicators. Practices can exempt patients from treatment, and can receive maximum payment for less than full coverage of eligible patients. This paper aims to estimate the gap between the percentage of maximum incentive gained and the percentage of patients receiving indicated care (the pay-performance gap), and to estimate how much of the gap is attributable respectively to thresholds and to exception reporting. Methods Analysis of Quality Outcomes Framework data in the National Primary Care Database and exception reporting data from the Information Centre from 8407 practices in England in 2005 – 6. The main outcome measures were the gap between the percentage of maximum incentive gained and the percentage of patients receiving indicated care at the practice level, both for individual indicators and a combined composite score. An additional outcome was the percentage of that gap attributable respectively to exception reporting and maximum threshold targets set at less than 100%. Results The mean pay-performance gap for the 65 aggregated clinical indicators was 13.3% (range 2.9% to 48%). 52% of this gap (6.9% of eligible patients) is attributable to thresholds being set at less than 100%, and 48% to patients being exception reported. The gap was greater than 25% in 9 indicators: beta blockers and cholesterol control in heart disease; cholesterol control in stroke; influenza immunization in asthma; blood pressure, sugar and cholesterol control in diabetes; seizures in epilepsy and treatment of hypertension. Conclusion Threshold targets and exception reporting introduce an incentive ceiling, which substantially reduces the percentage of eligible patients that UK practices need to treat in order to receive maximum incentive payments for delivering that care. There are good clinical reasons for exception reporting, but after unsuitable patients have been exempted from treatment, there is no reason why all maximum thresholds should not be 100%, whilst retaining the current lower thresholds to provide incentives for lower performing practices.


Background
Pay for performance (PfP) incentives were introduced to UK primary care in 1991. These were initially limited to vaccinations, immunizations and cervical cancer screening. The new GMS contract introduced in 2004 represented a major change to general practice funding, with a substantial increases in PfP for quality indicators, delivered through a quality and outcomes framework (QOF) based on the best available research evidence [1]. It is a large investment in UK primary care, with an estimated cost of £1.8 billion each year [2]. In the first year of contract implementation, these quality payments accounted for approximately 25% of a typical GP principal's pay [3]. Consequently delivery of care has changed and there is evidence that quality has increased in the contract conditions but not in other areas of quality care outside the contract [4].
Practices are rewarded for performance based on quality indicators, which initially covered 10 clinical domains and aspects of practice organisation, patient experience and enhanced services. The clinical indicators in the new GMS contract are listed in Additional file 1. More than half the incentive points are allocated to clinical performance (550 out of 1050). There has been a gross payment of £124.60 per point since 2005/6, based on a typical practice of 5500 patients with a typical prevalence of disease. Practices in England achieved higher results than expected by the NHS Employers negotiating team, gaining an average of 96% of the maximum points [3]. There is a mechanism for adjusting payment to each practice based on the prevalence of disease for each indicator, though this adjustment may not reflect the workload of a practice with a significantly different prevalence from the average [5]. At first glance the achievement of practices gaining almost all of the incentive payments available suggests close to maximum performance at targeting patients at risk: but maximum points and payment can be gained in every clinical indicator before all eligible patients receive indicated care.
There are two mechanisms which reduce the percentage of eligible patients that practices need to treat in order to receive maximum incentive payments. First, 'exception reporting' allows clinicians to exclude patients from the treatment indicated by a clinical indicator. Exception reporting is an important process because clinicians should not be put under pressure to prescribe inappropriately to achieve an indicator. Appropriate reasons for exception reporting have been agreed between the Department of Health and the British Medical Association and these are comprehensive (Table 1) [1]. 5.55% of patients were 'exception reported' by English practices in 2005/6 [6]. There is substantial variation between practices in exception reporting rates, with 5.4% of practices exceeding an average exception reporting rate of 10% [6].
The second mechanism which reduces the percentage of eligible patients that practices need to treat in order to receive maximum incentive payment is the presence of target thresholds set at less than 100%. For each clinical indicator there is a graduated scale of payments from a minimum to maximum threshold. A practice achieves maximum points payments at thresholds lower than 100% coverage of patients eligible for all interventions, with the lowest threshold for maximum payment being 50% (indicator CHD 10) and the highest threshold being 90% in 37 indicators ( Table 2). These thresholds were described as the "maximum practically achievable level to deliver clinical effectiveness" [1]. No definition of 'practically achievable' was given, and it was not intended to be the same as 'clinically desirable'. Importantly, once the maximum threshold is reached there is no financial incentive to treat more eligible patients. Of course doctors perform for other than financial reasons, and many (though not all) practices exceed the maximum target thresholds despite there being no additional payment for doing so.
An example of the way in which this payment scheme operates is indicator LVD 3, the percentage of patients with a diagnosis of CHD and left ventricular dysfunction A Patients who have been recorded as refusing to attend review who have been invited on at least three occasions during the preceding twelve months B Patients for whom it is not appropriate to review the chronic disease parameters due to particular circumstances e.g. terminal illness, extreme frailty C Patients newly diagnosed within the practice or who have recently registered with the practice, who should have measurements made within three months and delivery of clinical standards within nine months e.g. blood pressure or cholesterol measurements within target levels D Patients who are on maximum tolerated doses of medication whose levels remain sub-optimal E Patients for whom prescribing a medication is not clinically appropriate e.g. those who have an allergy, another contraindication or have experienced an adverse reaction F Where a patient has not tolerated medication G Where a patient does not agree to investigation or treatment (informed dissent), and this has been recorded in their medical records H Where the patient has a supervening condition which makes treatment of their condition inappropriate e.g. cholesterol reduction where the patient has liver disease I Where an investigative service or secondary care service is unavailable who are currently treated with ACE inhibitors, where the minimum threshold that triggers payment is 25%. There is a sliding scale of increasing payment up to a maximum of 10 points achieved at 70% uptake of the target population. The combination of exception reporting and low targets reduce the potential health gain from an indicator, as a practice always achieves maximum incentive points in a particular indicator before all eligible patients have received treatment. This is because all maximum target thresholds are set below 100% population coverage and this target is applied after exception reported patients have been excluded.
The objectives of this paper are: 1. To estimate the gap for each indicator between the percentage of maximum incentive gained, and the percentage of patients who receive indicated care.
2. to estimate how much of the gap is attributable respectively to thresholds for maximum payment set at less than 100%, and to exception reporting.  Table 3. Ethical approval was not needed for this study as all data was available in the public domain.

Results
The pay-performance gap for each indicator, and the amount of this gap attributable to exception reporting and target thresholds respectively are given in Table 4. At the indicator level, the mean pay:performance gap for the 65 indicators was 13.3% (s.d. 9.8). The pay:performance gap was greater than 25% in nine indicators, which included indicators with interventions with significant health gain.   Exception reporting rate (proportion), E ij /N ij = e ij Actual quality performance score before exceptions, q ij = X ij /N ij Reported quality performance score after exceptions, q R ij = X ij /N ij -E ij Rearranging gives X ij = q R ij * (N ij -E ij ) Therefore actual quality performance Pay score, p ij = (actual pay/maximum pay) for practice i and indicator j Pay-performance gap for practice i and indicator j, g ij = p ij -q ij Pay-performance gap for indicator j, g j = ∑ i p ij -q ij Pay-performance gap for practice i, g i = ∑ j p ij -q ij These pay:performance gaps were respectively; 48% for beta blockers in heart disease, 43% for glucose control in diabetes, 35% for cholesterol control after a stroke, 34% for influenza immunization in asthma, 30% for blood pressure control in diabetes, 29% in seizure control in epilepsy, 29% in cholesterol control in diabetes, 29% in cholesterol control in heart disease and 26% in blood pressure control in hypertension.
Thresholds below 100% account for 52% of the pay:performance gap, and exception reporting accounts for the remaining 48% of the pay:performance gap. Figure 1 shows the top 15 indicators with the largest pay:performance gap and the separate contribution made by exception reporting and thresholds.

Discussion
This is the first study of its kind to examine the gap between the percentage of financial incentive gained by each practice and the percentage of patients in that practice who receive indicated care. The strength of this study is that it uses data routinely collected from every practice in England for each clinical indicator which has exception reporting data. The mean gap between the percentage of maximum incentive that a practice receives and the percentage of eligible patients receiving care is 13.3%. This gap exceeds 25% in nine indicators. Five of these nine indicators with the largest pay-performance gap have an evidence base linking them to reductions in mortality. These five indicators are beta-blockers in heart disease (relative risk reduction 24%) [7], glucose control in diabetes (relative risk reduction 36%) [8], blood pressure control in diabetes (relative risk reduction 35%) [9], cholesterol control in heart disease (relative risk reduction 24%) [7] and blood pressure control in hypertension (relative risk reduction 16%) [19]. In all of these indicators the target threshold being set at less than 100% has contributed significantly to this gap between the financial incentive gained and number of patients receiving treatment.
The implication of these findings is that maximum target thresholds set at less than 100% may be contributing to reducing health gain for patients. Maximum target thresholds account for 52% and exception reporting accounts for 48% of the pay-performance gap respectively. It seems unlikely that such a large gap can be attributed to patients who are clinically unsuitable or unwilling to accept care pay: performance gap for top 15 indicators  for their condition as these patients should be accounted for by exception reporting. The combination of exception reporting and threshold targets set below 100%, while perhaps thought to be overambitious before implementation, appears now to be in danger of producing an incentive ceiling effect, where maximum payment is received for less than maximum coverage of the eligible population. This could reduce the health gain from the new GMS contract as we have observed both high exception reporting rates and low target thresholds in some indicators with significant intermediate health gain outcomes.
Exception reporting is patient specific and has the advantage of being sensitive to patients needs when it is used appropriately. The agreed criteria for exception reporting are wide ranging and will cover most circumstances where patients are not suitable for a particular indicator. Since primary care has achieved maximum points in many areas, there may be no rationale for maximum target thresholds to be set below 100% as there are comprehensive reasons for exception reporting any patient who would not theoretically benefit from the indicated care. In the 2006/7 revision to the GMS contract several of the maximum thresholds have been increased to 90%, though lower maximum thresholds still persist for several clinical indicators [11].
This work adds significant new information to previous published work on exception reporting and UK primary care. One study based in 1024 Scottish general practices found that when exception reporting is taken into account there was lower delivered quality of care in less affluent practices in 4 clinical indicators [12]. A further study based in one English primary care trust found higher exception reporting rates in diabetic populations with higher deprivation [13]. These 2 studies differ from ours in that we studied the effect of target thresholds on the percentage of patients receiving indicated care, as well as exception reporting. We used actual rather than estimated exception reporting data. We included a significantly

Conclusion
The combination of both exception reporting and target thresholds set at less than 100% lead to a misalignment of incentives in the GMS contract for UK primary care, where maximum incentive payment is reached before all eligible patients have received appropriate treatment or health care intervention. There is a high exception reporting rate in several indicators with significant intermediate health outcomes which also have low maximum target thresholds. The policy implication is that all maximum target thresholds should be set at 100%, whilst retaining the current lower thresholds to provide incentives for lower performing practices. Appropriate exception reporting is likely to be unavoidable and necessary, and should be retained for patients unsuitable for a particular indicator. Incentive payments would then be provided for 100% of eligible patients in the population.