A prospective analysis of false positive events in a National Colon Cancer Surveillance Program

Background The survival benefits of colon cancer surveillance programs are well delineated, but less is known about the magnitude of false positive testing. The objective of this study was to estimate the false positive rate and positive predictive value of testing as part of a surveillance program based on national guidelines, and to estimate the degree of testing and resource use needed to identify a curable recurrence. Methods Analysis of clinically significant events leading to suspicion of cancer recurrence, false positive events, true cancer recurrences, time to confirmation of diagnosis, and resource use (radiology, blood samples, colonoscopies, consultations) among patients included in a randomised colon cancer surveillance trial. Results 110 patients surgically treated for colon cancer were followed according to national guidelines for 1884 surveillance months. 1105 tests (503 blood samples, 278 chest x-rays, 209 liver ultrasounds, 115 colonoscopies) and 1186 health care consultations were performed. Of the 48 events leading to suspicion of cancer recurrence, 34 (71%) represented false positives. Thirty-one (65%) were initiated by new symptoms, and 17 (35%) were initiated by test results. Fourteen patients had true cancer recurrence; 7 resections of recurrent disease were performed, 4 of which were successful R0 metastasis Resections. 276 tests and 296 healthcare consultations were needed per R0 resection; the cost per R0 surgery was £ 103207. There was a 29% probability (positive predictive value) of recurrent cancer when a diagnostic work-up was initiated based on surveillance testing or patient complaints. Conclusion We observed a high false positive rate and low positive predictive value for significant clinical events suggestive of possible colorectal cancer relapse in the setting of a post-treatment surveillance program based on national guidelines. Providers and their patients should have an appreciation for the modest positive predictive value inherent in colorectal cancer surveillance programs in order to make informed choices, which maximize quality of life during survivorship. Better means of tailoring surveillance programs based on patient risk would likely lead to more effective and cost-effective post-treatment follow-up. Trial registration ClinicalTrials.gov identifier NCT00572143. Date of trial registration: 11th of December 2007.


Background
Colon cancer is the third most common cancer in the western world, and surgery is the only curative treatment. Approximately one-third of those resected will experience recurrent disease with an expected survival of less than two years [1]. Most patients treated with curative intent are included in some form of surveillance program involving periodic evaluations to detect asymptomatic recurrence. Reviews comparing various surveillance programs have suggested that more intensive surveillance strategies tend to increase five-year overall survival by 5-10% [2,3].
However, all preventive programs have associated costs and risks [4][5][6][7]. The survival benefits of a surveillance program for colon cancer survivors are well known, but much less is known about the potential negative impacts for patients and their loved ones [3]. False positive test results may negatively impact quality of life (QoL) by leading patients and family members to believe that recurrence has occurred. On the other hand, false negative tests may result in diagnostic delay during which a potentially curable recurrent cancer may become inoperable. In order to more rationally weigh benefits and harms of post-operative surveillance programs, patients and providers need information to help them better understand the implications of positive and negative test results [6]. In addition, it is important to understand the resource use associated with work-up of suspected recurrence and the benefit which can be expected in terms of opportunities for curative treatment of recurrence.
Previously, we have reported the cost-effectiveness and QoL results from a Norwegian randomized trial comparing general practitioner (GP) versus surgeon-organised colon cancer surveillance according to national guidelines [8,9]. In this work, we focus on the outcomes of individuals who underwent a diagnostic work-up in response to a positive surveillance test result or development of interval symptoms that raised concern for recurrence. These outcomes include whether the initial positive test results or symptoms represented a false positive or whether they served as the first sign of a true recurrence, as well as subsequent resource use for both true and false positives. Among patients with true positive tests or symptoms, we examine the extent to which curative resection of recurrent disease was possible.

Methods
Data from a randomised trial assessing a colon cancer surveillance program based on Norwegian National Guidelines (ClinicalTrials.gov identifier NCT00572143), was linked to follow-up data from the electronic medical records (EMRs) of four Norwegian hospitals [8,9]. Patients who underwent a diagnostic work-up as a result of a positive surveillance test or symptoms suggesting possible recurrence were identified, and subsequent health care resource utilization was quantified. Patients were enrolled in the surveillance program between June 1, 2007 and December 15, 2011.

Outcomes
Primary outcomes included 1) Positive predictive value and false positive rates for initial positive surveillance test results or symptoms suggestive of possible recurrence, 2) subsequent resource use for patients with both true and false positives, and 3) whether curative resection of recurrent disease was possible. Secondary outcomes included time from initial detection to confirmation of recurrence, number needed to test per curative resection of recurrence, and cost per curative resection of recurrence.

Ethics
The Regional Committee for Medical Research Ethics (P REK NORD 79/ 2006) and the Norwegian Data Inspectorate approved the research protocol; all patients provided informed written consent.

Inclusion and exclusion criteria
Inclusion criteria were age less than 75 years with recent curative surgery for Dukes' stage A, B or C colon cancer. Patients receiving postsurgical adjuvant chemotherapy (some Dukes' B and all Dukes' C) were eligible to participate. Exclusion criteria were age greater than 75 years, membership in a healthcare trust not participating in the trial, inability to provide informed consent, and Dukes' stage D cancer. Patients were followed for up to 2 years ( Table 1).

Description of the surveillance program
The GP-and surgeon-followed arms underwent the same surveillance regimens based on Norwegian Gastrointestinal Cancer Group 2007 surveillance guidelines ( Table 1). The surveillance period included in analyses included nine surveillance cycles (one month through 24 months postoperatively) with regular clinical examinations, CEA measurement, chest x-ray, contrast-enhanced liver ultrasound and colonoscopy according to the intervals described in Table 1.

Hospitals, primary and secondary care professionals
Three local hospitals and one university hospital trust participated. Approximately 100 patients with colon cancer are surgically treated annually at these four hospitals. Approximately 550 GPs work in the health care trust.

Serious clinical events
A serious clinical event (SCE) was defined as an episode leading to suspicion of cancer recurrence. An SCE could

Data collection
Data on radiology tests, colonoscopies, blood tests, surgical consultations, surgeries, pathology studies, admissions, and hospital discharges were abstracted from the EMRs of all included patients and were used to identify SCEs. In the case of missing information, the surgeons, GPs, or patients were interviewed by telephone. False positive SCEs were considered to have occurred when an SCE triggered a subsequent diagnostic work-up that ruled out recurrence. We identified successful R0 resections of recurrent local or metastatic disease by examining the surgical and postoperative pathology reports.

Statistics
Proportions were compared using 2x2 contingency tables, and Chi Squared or Fisher's exact tests with an alpha level of 0.05. Continuous values were compared using t-tests with a two-sided alpha value of 0.05. The results were expressed using the mean differences for continuous outcomes with the corresponding standard deviations (SD), 95% confidence intervals (CI), and associated p-values. Pvalues were reported to three decimal places, and p-values less than 0.001 were reported as p < 0.001. The surveillance program's positive predictive value was defined as the proportion of SCEs that were true positives (i.e., true cancer recurrences). The economic evaluation of cost per R0 resection was performed from a societal perspective. Cost elements were converted into British pounds (BP £) at a rate of BP £ = 9.39 Norwegian krone (NOK) (www.norges-bank. no). Unit costs assigned to health care resources have been previously reported [8]. To address the uncertainty aspect of the cost per R0 metastases resections, we performed a many-inputs/one-output sensitivity analyses, with results expressed in a Tornado chart. All analyses were performed using IBM SPSS Statistics v 19.0 (IBM Company SPSS 2010) and Microsoft Excel for Mac 2011.

Results
A total of 110 patients surgically treated for colon cancer met the inclusion criteria and were enrolled in the randomised trial. Fifty-five were followed by their GPs and 55 were followed in a surgical outpatient department. Overall, 85 patients (75%; GP 41 vs. surgeon 44) were followed for at least 12 months, and 58 patients (52%; GP 29 vs. surgeon 29) were followed for 24 months. The total surveillance time was 1884 person-months; median follow up time was 17 months. Overall, 1105 tests (colonoscopy n = 115, liver ultrasound n = 209, chest x-ray n = 278, carcinoembryonic antigen n = 503) were performed and 1186 consultations occurred ( Table 2).

Serious clinical events and false positive tests
A total of 48 SCEs were identified; 31 (65%) were initiated by emerging symptoms and 17 (35%) were initiated by test findings. Abdominal pain (n = 14, 29%) and blood in the stool (n = 10, 20%) were the most common presenting complaints. There were no significant differences between surgeons or GPs in terms of SCE occurrence or the time to diagnosis (Table 3). Overall, 34 patients (30% of all trial patients) experienced a false positive event. The positive predictive value (i.e., the probability of a surveillanceinitiated SCE being confirmed as true colon cancer recurrence) was 29% (surgeons 36% vs. GPs 23%).

Clinical presentation of colon cancer recurrence
Of the 48 patients with SCEs, 14 (29%) had colon cancer recurrence. Symptoms were the most common initial indication of recurrence (n = 7), followed by radiologically detected lesions (n = 4) and elevated CEA levels (n = 3) ( Table 4).

Harms and benefits of cancer surveillance
In this surveillance program, 14 recurrences were detected, seven subsequent surgeries were performed, and four R0 resections of metastases were achieved by surveilling 110 patients over a total of 1884 months. This means that 276 tests and 296 healthcare consultations were needed per R0 resection; the cost per R0 resection was £ 103207.
A 25% increase in successful R0 metastases surgeries decreased the cost per surgery to £ 82566 (sensitivity analyses Figure 1). Three patients had asymptomatic but incurable recurrences. Mean time from SCE to confirmation of recurrence was 39 days (standard deviation 35 days). The mean quality of life was equal to that of the general UK population (Table 5) [10].

Summary of findings
A representative population of patients surgically treated for colon cancer was enrolled in a colon cancer surveillance program based on Norwegian national guidelines, with an expected normal variance of demographic factors and colon cancer severity. The patients were followed for up to two years (i.e., the period during which most cancer recurrences manifest). Thirty percent (34 of 110) of all the surveillance patients experienced false positive tests, and the probability of having a cancer recurrence when a diagnostic check-up was initiated was 29%. Overall, 276 tests were needed to save one patient from recurrent colon cancer, and the total cost per successful surgery for recurrence was £ 103207.

Comparison of existing literature
The potential negative side effects (physical and psychological) and costs of cancer surveillance strategies have not been well delineated [4][5][6][7]. None of the studies included in the previous reviews of colorectal cancer surveillance have provided any specific details of the harms (mortality or morbidity) resulting from investigating or treating recurrences [2,3]. However, a potential harm from any secondary prevention program is well recognised to be over-diagnosis and false positive tests [7,11].
Some researchers have investigated the psychological effects of colorectal cancer surveillance [8,12,13]. None of these studies have found deterioration in the patient QoL with surveillance. Nevertheless, in recent meta-analyses, it was shown that anxiety, rather than depression, was a major problem among long-term cancer survivors. It is unknown, however, what impact a cancer surveillance program itself has on anxiety levels [14].
The challenge of postoperative cancer surveillance is that a vast majority of patients must undergo a large number of tests without any benefit, or even with some potential harm, to identify a few patients with curable recurrence. The high rate of false positive tests (n = 34, 30% of all surveillance patients) in this trial was more than we expected and likely negatively impacts the patient's and family's QoL. False positive and true positive tests in colorectal cancer surveillance have been addressed in previous reviews [15]. According to Kievit et al., 370 positive surveillance tests (26 true positives, 7%) and 11 surgeries were required to provide one patient with a  [5,17]. In our study, three patients (21%, Table 4) had asymptomatic but incurable colon cancer recurrence. These figures are somewhat higher than those in a previous study reporting 9% asymptomatic but incurable disease detected in a surveillance program [16].
The imperfect nature of specific surveillance tests themselves (i.e., test sensitivity and specificity) can contribute to the potential harms of surveillance. National surveillance programs are often based on serial CEA measurements, and this biomarker has several pitfalls and shortcomings. A recent study showed that the diagnostic accuracy of serial CEA measurements is low and is impacted by the cut-off value used [18]. Similarly, radiological tests have varying sensitivity and specificity, the latter of which impacts the rate of false positive tests. For example, the rate of false positive tests in CT chest scans has been reported to be as high as 30-50%; as a result, this test is not recommended by some physicians for post-treatment surveillance purposes [19]. Studies of secondary prevention practices around other cancer types have explored potential harmful effects of these programs. In a systematic review addressing screening for lung cancer using thoracic CT scans, most of the detected lung nodules (> 90%) were benign, and invasive nonsurgical procedures in patients with benign lesions were common [20]. Forty-six percent of patients reported psychological distress while awaiting the confirmation of a potential cancer diagnosis [21]. Thus, the potential harms of a preventive program must be carefully weighed against any benefits [20][21][22]. In the case of colonoscopic colorectal cancer surveillance, the potential harms of the procedure  [23].

Strengths and limitations
This study has some strength. It represents the first trial to analyse the potential side effects of colon cancer surveillance, an area of considerable uncertainty. Jeffery and Hider outlined this uncertainty around side effects of colorectal cancer surveillance in their 2007 Cochrane review [3]. This study has limitations. The trial was designed to assess whether general practice surveillance affected patient-specific QoL and cost-effectiveness compared to surgeon-led surveillance [9]. A trial assessing survival would require a larger sample size and a longer surveillance time. We acknowledge that this choice of a follow-up period might have impacted the observed frequency of SCE and thus of recurrence. Thirteen percent (n = 14) of trial patients had colon cancer recurrence; this low recurrence rate was most likely related, at least partly, to the short follow-up duration (median 17 months). Therefore, our calculation of the cost per successful R0 recurrence surgery might represent an overestimate. To address this uncertainty, we performed a sensitivity analysis in which we estimated the cost per R0 resection if a 25% increase in successful resections was observed ( Figure 1). However, in our opinion, the analysed time period gives a realistic overview of the magnitude and rate of side effects in a colon cancer surveillance program.

Conclusion
Information on a range of outcomes should be available to fully assess the net benefit or harm of colon cancer surveillance. Any survival benefit of the surveillance must be balanced against the potential harms inherent to ensure that surveillance programs are acceptable. Providers are best positioned to inform patients and their families of these benefits and potential harms. The nature of postoperative cancer surveillance is that a vast majority of patients (96% in the present study) must undergo a large number of tests without any benefit, or even with some harm to themselves and their family, to identify a few patients with curable recurrence. Patients with asymptomatic but incurable disease (21% of all recurrences in the present study) likely represent the most controversial  Mean EQ-5D Index score (CI) 0.862 (0.84-0.87) *R0 surgery is defined as positive histological resection margins. a Reported in a previous publication [8]. Equals that of the general UK population [10].
group and raise ethical and quality of life considerations [16]. The reported cost of £ 103207 per R0 resection of recurrent disease is significantly higher than that reported by others [16].
In conclusion, due to the high rate of false positive tests and low positive predictive value of the surveillance program examined, we feel that a more tailored surveillance approach based on recurrence risk and likely recurrence pattern is needed in future CRC surveillance programs. Such an approach has the potential to reduce costs and the number of false positive tests, while improving positive predictive value. There is nothing in the current evidence base suggesting that such an approach will compromise the potential survival benefit of CRC surveillance. In addition, further research is needed regarding the potential harms of colorectal cancer surveillance, including quality of life impacts due to false positive surveillance tests and to early diagnosis of incurable recurrence. The estimated cost of surveillance is considerable, and whether the identified costs are acceptable when comparing the benefits and harms is a matter of discussion not only among policy and decision makers, but also among providers, patients, and families.