"Mind the gap!" Evaluation of the performance gap attributable to exception reporting and target thresholds in the new GMS contract: National database analysis
BMC Health Services Research volume 8, Article number: 131 (2008)
The 2003 revision of the UK GMS contract rewards general practices for performance against clinical quality indicators. Practices can exempt patients from treatment, and can receive maximum payment for less than full coverage of eligible patients. This paper aims to estimate the gap between the percentage of maximum incentive gained and the percentage of patients receiving indicated care (the pay-performance gap), and to estimate how much of the gap is attributable respectively to thresholds and to exception reporting.
Analysis of Quality Outcomes Framework data in the National Primary Care Database and exception reporting data from the Information Centre from 8407 practices in England in 2005 – 6. The main outcome measures were the gap between the percentage of maximum incentive gained and the percentage of patients receiving indicated care at the practice level, both for individual indicators and a combined composite score. An additional outcome was the percentage of that gap attributable respectively to exception reporting and maximum threshold targets set at less than 100%.
The mean pay-performance gap for the 65 aggregated clinical indicators was 13.3% (range 2.9% to 48%). 52% of this gap (6.9% of eligible patients) is attributable to thresholds being set at less than 100%, and 48% to patients being exception reported. The gap was greater than 25% in 9 indicators: beta blockers and cholesterol control in heart disease; cholesterol control in stroke; influenza immunization in asthma; blood pressure, sugar and cholesterol control in diabetes; seizures in epilepsy and treatment of hypertension.
Threshold targets and exception reporting introduce an incentive ceiling, which substantially reduces the percentage of eligible patients that UK practices need to treat in order to receive maximum incentive payments for delivering that care. There are good clinical reasons for exception reporting, but after unsuitable patients have been exempted from treatment, there is no reason why all maximum thresholds should not be 100%, whilst retaining the current lower thresholds to provide incentives for lower performing practices.
Pay for performance (PfP) incentives were introduced to UK primary care in 1991. These were initially limited to vaccinations, immunizations and cervical cancer screening. The new GMS contract introduced in 2004 represented a major change to general practice funding, with a substantial increases in PfP for quality indicators, delivered through a quality and outcomes framework (QOF) based on the best available research evidence . It is a large investment in UK primary care, with an estimated cost of £1.8 billion each year . In the first year of contract implementation, these quality payments accounted for approximately 25% of a typical GP principal's pay . Consequently delivery of care has changed and there is evidence that quality has increased in the contract conditions but not in other areas of quality care outside the contract .
Practices are rewarded for performance based on quality indicators, which initially covered 10 clinical domains and aspects of practice organisation, patient experience and enhanced services. The clinical indicators in the new GMS contract are listed in Additional file 1. More than half the incentive points are allocated to clinical performance (550 out of 1050). There has been a gross payment of £124.60 per point since 2005/6, based on a typical practice of 5500 patients with a typical prevalence of disease. Practices in England achieved higher results than expected by the NHS Employers negotiating team, gaining an average of 96% of the maximum points . There is a mechanism for adjusting payment to each practice based on the prevalence of disease for each indicator, though this adjustment may not reflect the workload of a practice with a significantly different prevalence from the average . At first glance the achievement of practices gaining almost all of the incentive payments available suggests close to maximum performance at targeting patients at risk: but maximum points and payment can be gained in every clinical indicator before all eligible patients receive indicated care.
There are two mechanisms which reduce the percentage of eligible patients that practices need to treat in order to receive maximum incentive payments. First, 'exception reporting' allows clinicians to exclude patients from the treatment indicated by a clinical indicator. Exception reporting is an important process because clinicians should not be put under pressure to prescribe inappropriately to achieve an indicator. Appropriate reasons for exception reporting have been agreed between the Department of Health and the British Medical Association and these are comprehensive (Table 1) . 5.55% of patients were 'exception reported' by English practices in 2005/6 . There is substantial variation between practices in exception reporting rates, with 5.4% of practices exceeding an average exception reporting rate of 10% .
The second mechanism which reduces the percentage of eligible patients that practices need to treat in order to receive maximum incentive payment is the presence of target thresholds set at less than 100%. For each clinical indicator there is a graduated scale of payments from a minimum to maximum threshold. A practice achieves maximum points payments at thresholds lower than 100% coverage of patients eligible for all interventions, with the lowest threshold for maximum payment being 50% (indicator CHD 10) and the highest threshold being 90% in 37 indicators (Table 2). These thresholds were described as the "maximum practically achievable level to deliver clinical effectiveness" . No definition of 'practically achievable' was given, and it was not intended to be the same as 'clinically desirable'. Importantly, once the maximum threshold is reached there is no financial incentive to treat more eligible patients. Of course doctors perform for other than financial reasons, and many (though not all) practices exceed the maximum target thresholds despite there being no additional payment for doing so.
An example of the way in which this payment scheme operates is indicator LVD 3, the percentage of patients with a diagnosis of CHD and left ventricular dysfunction who are currently treated with ACE inhibitors, where the minimum threshold that triggers payment is 25%. There is a sliding scale of increasing payment up to a maximum of 10 points achieved at 70% uptake of the target population. The combination of exception reporting and low targets reduce the potential health gain from an indicator, as a practice always achieves maximum incentive points in a particular indicator before all eligible patients have received treatment. This is because all maximum target thresholds are set below 100% population coverage and this target is applied after exception reported patients have been excluded.
The objectives of this paper are:
To estimate the gap for each indicator between the percentage of maximum incentive gained, and the percentage of patients who receive indicated care.
to estimate how much of the gap is attributable respectively to thresholds for maximum payment set at less than 100%, and to exception reporting.
Thresholds for maximum payments were obtained from the 2003 GMS contract . Data on points gained and reported quality performance for each clinical indicator in all English practices for the year 2005/6 were obtained from the National Primary Care Database. The contract has 76 clinical indicators. Exception reporting rates for 65 clinical indicators in all English practices for the year 2005/6 were obtained from the NHS Information Centre. Exception reporting is not applicable to the remaining 11 indicators as they refer purely to setting up disease registers that are used to define the population eligible for the 65 indicators.
The 'gap' was computed for each indicator for all patients by calculating the difference between the percentage of incentive gained and the percentage of patients who received treatment. To compute the size of the gap attributable to the threshold targets the exception reporting rate was deducted from the total gap for each indicator. The computations used to determine the numbers of patients receiving care are displayed in Table 3. Ethical approval was not needed for this study as all data was available in the public domain.
The pay-performance gap for each indicator, and the amount of this gap attributable to exception reporting and target thresholds respectively are given in Table 4. At the indicator level, the mean pay:performance gap for the 65 indicators was 13.3% (s.d. 9.8). The pay:performance gap was greater than 25% in nine indicators, which included indicators with interventions with significant health gain. These pay:performance gaps were respectively; 48% for beta blockers in heart disease, 43% for glucose control in diabetes, 35% for cholesterol control after a stroke, 34% for influenza immunization in asthma, 30% for blood pressure control in diabetes, 29% in seizure control in epilepsy, 29% in cholesterol control in diabetes, 29% in cholesterol control in heart disease and 26% in blood pressure control in hypertension.
Thresholds below 100% account for 52% of the pay:performance gap, and exception reporting accounts for the remaining 48% of the pay:performance gap. Figure 1 shows the top 15 indicators with the largest pay:performance gap and the separate contribution made by exception reporting and thresholds.
This is the first study of its kind to examine the gap between the percentage of financial incentive gained by each practice and the percentage of patients in that practice who receive indicated care. The strength of this study is that it uses data routinely collected from every practice in England for each clinical indicator which has exception reporting data. The mean gap between the percentage of maximum incentive that a practice receives and the percentage of eligible patients receiving care is 13.3%. This gap exceeds 25% in nine indicators. Five of these nine indicators with the largest pay-performance gap have an evidence base linking them to reductions in mortality. These five indicators are beta-blockers in heart disease (relative risk reduction 24%) , glucose control in diabetes (relative risk reduction 36%) , blood pressure control in diabetes (relative risk reduction 35%) , cholesterol control in heart disease (relative risk reduction 24%)  and blood pressure control in hypertension (relative risk reduction 16%) . In all of these indicators the target threshold being set at less than 100% has contributed significantly to this gap between the financial incentive gained and number of patients receiving treatment.
The implication of these findings is that maximum target thresholds set at less than 100% may be contributing to reducing health gain for patients. Maximum target thresholds account for 52% and exception reporting accounts for 48% of the pay-performance gap respectively. It seems unlikely that such a large gap can be attributed to patients who are clinically unsuitable or unwilling to accept care for their condition as these patients should be accounted for by exception reporting. The combination of exception reporting and threshold targets set below 100%, while perhaps thought to be overambitious before implementation, appears now to be in danger of producing an incentive ceiling effect, where maximum payment is received for less than maximum coverage of the eligible population. This could reduce the health gain from the new GMS contract as we have observed both high exception reporting rates and low target thresholds in some indicators with significant intermediate health gain outcomes.
Exception reporting is patient specific and has the advantage of being sensitive to patients needs when it is used appropriately. The agreed criteria for exception reporting are wide ranging and will cover most circumstances where patients are not suitable for a particular indicator. Since primary care has achieved maximum points in many areas, there may be no rationale for maximum target thresholds to be set below 100% as there are comprehensive reasons for exception reporting any patient who would not theoretically benefit from the indicated care. In the 2006/7 revision to the GMS contract several of the maximum thresholds have been increased to 90%, though lower maximum thresholds still persist for several clinical indicators .
This work adds significant new information to previous published work on exception reporting and UK primary care. One study based in 1024 Scottish general practices found that when exception reporting is taken into account there was lower delivered quality of care in less affluent practices in 4 clinical indicators . A further study based in one English primary care trust found higher exception reporting rates in diabetic populations with higher deprivation . These 2 studies differ from ours in that we studied the effect of target thresholds on the percentage of patients receiving indicated care, as well as exception reporting. We used actual rather than estimated exception reporting data. We included a significantly larger number of practices (8407), and included all 65 clinical indicators for which exception reporting occurs, where as McClean et al included 33 indicators and Sigfrid et al included 15 indicators.
The combination of both exception reporting and target thresholds set at less than 100% lead to a misalignment of incentives in the GMS contract for UK primary care, where maximum incentive payment is reached before all eligible patients have received appropriate treatment or health care intervention. There is a high exception reporting rate in several indicators with significant intermediate health outcomes which also have low maximum target thresholds. The policy implication is that all maximum target thresholds should be set at 100%, whilst retaining the current lower thresholds to provide incentives for lower performing practices. Appropriate exception reporting is likely to be unavoidable and necessary, and should be retained for patients unsuitable for a particular indicator. Incentive payments would then be provided for 100% of eligible patients in the population.
General Medical Services
pay for performance
quality and outcomes framework
Department of Health: New GMS Contract 2003. Investing in general practice. NHS Confederation and the British Medical Association. London. 2003
Roland M: Linking physician pay to quality of care: a major experiment in the United Kingdom. N Eng J Med. 2004, 351: 1448-54. 10.1056/NEJMhpr041294.
Information Centre 2006. Press release "Online GP practice results database. Quality and Outcomes Framework for GP practices". Accessed 28 05 2007, [http://www.qof.ic.nhs.uk/]
Steel N, Maisey S, Clark A, Fleetcroft R, Howe A: Quality of clinical primary care and targeted incentive payments: an observational study. British Journal of General Practice. 2007, 57 (539): 449-454.
Guthrie B, McLean G, Sutton M: Workload and reward in the quality and outcomes framework of the 2004 general practice contract. British Journal of General Practice. 2006, 56: 836-841.
Information Centre 2007. Quality and Outcomes Framework exception report. Accessed 09 09 07, [http://www.ic.nhs.uk/statistics-and-data-collections/audits-and-performance/qof/quality-and-outcomes-framework-exception-report]
National Institute for Clinical Excellence: Prophylaxis for patients who have experienced a myocardial infarction: drug treatment, cardiac rehabilitation and dietary manipulation. 2001, London NICE
Adler A, Stratton I, Neil H, Yudkin J, Matthews D, Cull C, et al: UKPDS 36: Association of systolic blood pressure with macro vascular and micro vascular complications of type 2 diabetes (UKPDS 36): prospective observational study. BMJ. 2000, 321: 412-419. 10.1136/bmj.321.7258.412.
UK Prospective Diabetes Study (UKPDS) Group: Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes (UKPDS 34). Lancet. 1998, 352: 854-86. 10.1016/S0140-6736(05)61359-1.
Mulrow C, Lau J, Cornell J, Brand M: Pharmacotherapy for hypertension in the elderly. Cochrane Database of Systematic Reviews. 1998, CD000028-2
NHS Employers and the BMA: Revisions to the GMS contract 2006/7. Delivering investment in general practice. 2006, NHS Employers Company Ltd. London
McLean G, Sutton M, Guthrie B: Deprivation and quality of primary care services: evidence for persistence of the inverse care law from the UK Quality and Outcomes Framework. Journal of Epidemiology and Community Health. 2006, 60: 917-922. 10.1136/jech.2005.044628.
Sigfrid LA, Turner C, Crook D, Ray S: Using the UK primary care Quality and Outcomes framework to audit health care equity; preliminary data on diabetes. Journal of Public Health. 2006, 28 (3): 221-225. 10.1093/pubmed/fdl028.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/131/prepub
We are grateful to Andrew Wagner, the National PCT database and the Information Centre for providing data for analysis. Dr Fleetcroft is given protected time from Gt. Yarmouth and Waveney PCT for study.
All authors have received funding from the Department of Health for a study estimating the health impact gain of the new GMS contract
Contributors: RF developed the study concept and design, supervised the study, obtained and analysed the data, wrote the paper and is the guarantor. NS, RC and AH advised on the study design, and the interpretation of the results. All authors contributed to, read and approved the final manuscript.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Fleetcroft, R., Steel, N., Cookson, R. et al. "Mind the gap!" Evaluation of the performance gap attributable to exception reporting and target thresholds in the new GMS contract: National database analysis. BMC Health Serv Res 8, 131 (2008). https://doi.org/10.1186/1472-6963-8-131