A linking-records-based validation study was carried out to determine whether the HDAD might be a useful tool to identify incident cases of prostate cancer. 2286 incident prostate cancer cases, recorded in four Spanish population-based cancer registries, were used to estimate its validity. Four algorithms were developed to detect incident cases. Those using only diagnostic codes were the most sensitive strategies although sensitivity scored around 25%; those using concurrent information about procedures were the most predictive, although positive predictive value scored close to 50%.
There is no recent literature available for comparison. Some international literature showed higher values for sensitivity  in prostate cancer, although the context was extremely different (cases attended 20 years ago, a completely different prostate cancer management, different use of administrative databases and a completely different healthcare system) to allow a reasonable discussion about the consistency of our results. Unfortunately, in our context, the only published study evaluated colorectal cancer .
Nevertheless, the analysis of false positive and false negative underlying causes has provided very useful insights. Three possible sources of failure to detect incident cases could be described in our study: the HDAD itself, the algorithms to detect cancer, and the features of this specific cancer site.
Shortcomings for incident case detection using the HDAD
Firstly, although HDAD has been a compulsory information system for public institutions since the mid 90s in Spain, it was not mandatory for private hospitals. Thus, it must be expected that in those places where population is served by hospitals not required to provide hospital information, the capacity of HDAD to find cases when they exist in the registry will be limited.
In fact, this is the reason for the high number of false negative cases in the Basque Country, region in which the private-public mix is more influential in prostate cancer treatment (32% of prostate cancers registered the year under study were considered false negatives for this cause). In the remaining regions where this effect is expected to be less important, the percentage of false negative cases was lower: only 27 cases in Granada (12% of registered cancer), 25 cases in Zaragoza (5%) and 1 case in Murcia (0.3%) where information about private hospitals was also included.
Secondly, the population covered by a registry and the population attended by a hospital (from which the cases were retrieved) are different. Procedures like radical prostatectomy are usually delivered in high-volume hospitals, in which populations from different healthcare areas or regions are regularly referred to. These cases were identified as false positive cases because these patients belonged to a different area than that covered by the registry.
Although this was not an important source of false positive cases, in the most sensitive algorithm, figures ranged from 1.3% in ZD to 4.5% in BD.
Thirdly, coding processes seem to act as a potential source of variation within and among regional settings in different ways. In fact, empirical data show figures that deserve some comment. False positive diagnoses (i.e. HDAD registered prostate cancer when other type of cancer or even a benign tumour was registered in the registry) varied among regions. Taking into account the most sensitive definition, over all prostate cancer cases identified by HDAD, false positive diagnoses ranged from 4.3% in BD to 23.9% in MD (previous research found 7% of inappropriate use of codes ).
Another element that potentially could affect the number of false negative cases is the number of ICD codes registered for each patient, in each hospital, both in diagnoses and procedures. Slight differences have been found among registries: HDAD in Granada registered a mean of 4 diagnoses and 2 procedures; Basque Country recorded a mean of 3 diagnoses and 1 procedure, Murcia registered a mean of 5 diagnoses and 2.4 procedures and Zaragoza registered a mean of 4.2 diagnoses and 2.1 procedures. However, although a potential cause, the number of false negative cases for this reason was negligible.
Shortcomings related with the algorithms
As expected [11, 13] algorithms including concurrent procedures were more "restrictive", and their positive predictive values were higher than those without surgical codes. As a consequence of being so "specific" there was a significant loss of prostate cancer cases (i.e. false negative cases): 25 in GD, 125 in BD, 52 in MD and 99 in ZD. These numbers represented 11.8% of cases detected in GD, 12.6% in BD, 13.7% in MD and 26.9% in ZD and probably refers to those cases admitted for diagnostic tests that did not require surgical treatment.
On the other hand, a noticeable number of false negative cases were due to the fact that there were patients diagnosed in 2000 (i.e. admission to the hospital was in 2000) but they were discharged in 2001. This ranged from 1.4% of cases in GD to 11.7% of cases in ZD. However, flagging properly theses cases, for example by opening the timeframe two months forward, we would barely increase sensitivity in a 10%, remaining figures under 50%.
An additional source of false negative cases could be found in the exclusion of metastatic disease in the definition of incident case; it would the case for patients that initially present with metastases and the original prostatic tumour is only found once they are discharged. These patients would be flagged as false negatives if they are discharged without definitive diagnoses. None out of the 58 cases which could potentially be affected in our sample was actually misclassified.
Finally, although several strategies were used to reduce the number of prevalent cases (patients with an episode were followed backwards two years and specific exclusion criteria were used) a remarkable number of prevalent cases were retrieved by the algorithm. As expected the most specific strategy (algorithm 4) identified less prevalent cases than the most sensitive one (algorithm 2); however, 9(69.2%) prevalent cases were found in GD, 78 (82.1%) in BD, 14(66.7%) in MD and 36(81.8%) in ZD. When we reviewed cases registered in 1997 and earlier, we found a 48% of prevalent cases. This finding suggests that considering a timeframe backward of 5 years would be better in terms of gaining higher positive predictive values.
Shortcomings related with the cancer site
Long survival cancers, like prostate cancer, are supposed to have more hospital readmissions than aggressive short survival cancers. Probably, the unexpected number of prevalent cases is related with us having decided to use too short a timeframe. In fact, in a ten-year period study Brackley et al.  found a rate of false positive as small as 1%, although authors recognised a significant overlapping between HDAD and registries.
But the most important element to be considered is the different manner in which healthcare services manage prostate cancer. In fact, 947 cases (41% of the total sample of prostate cancer registered by the 4 registries in 2000) were treated in ambulatory settings, being considered false negative cases by our methodology [147 cases in GD, 343 in BD, 262 in MD and 195 in ZD].
Thus, the more hospital-based the care provided the more probability of registering positive true cases by using the HDAD. This hypothesis was tested for breast cancer in a study that compared the Medicare and SEER registries agreement: concordance was considerably higher if surgery was performed (85% vs 50% when only biopsy was provided) .
Preliminary data from colorectal cancer and female breast cancer in Spain, two sites in which surgery is needed, show a better performance of positive predictive value: our context is consistent with this hypothesis: colorectal cancer and breast cancer showed a higher positive predictive value (up to 89% and up to 86%, respectively) [24–26].
Looking at these findings, we would be able to hypothesize that a good performance of our algorithms is expected in those cancers in which surgical treatment is needed and variations in surgical population rates are small (e.g. larynx, endometrial, colorectal or breast cancer). Conversely, those cancers with high uncertainty about the effectiveness of surgical intervention (e.g. prostate, oesophagus, pancreas, lung cancer) will perform poorly.