Phase 1. Further development of the tool and cognitive debriefing interviews with cancer patients.
The aim of Phase 1 was to improve the previously reported DELAYS tool , to make it suitable for use in the trial, building on the previous work and making it more acceptable, more likely to be completed, and less likely to cause anxiety.This further development was informed by the cognitive aspects of survey methodology approach [18–20] and utilised the findings from systematic review evidence regarding survey response [21–23]. We recruited ten patients to complete the modified tool and participate in cognitive debriefing interviews. They were either allocated to self-completed or researcher-completed mode. In addition to asking patients questions about symptoms related to their own cancer, we also asked about the wording of questions relating to symptoms of other cancers. Post completion, the participants underwent a cognitive debriefing interview. Areas identified as problematic in the initial piloting  were specifically addressed.
Key interview probes were used that directly and intentionally match the steps of the standard cognitive model for survey response (comprehension, retrieval, judgement, response formulation) . These probes were:
What the respondent believed the question to be asking.
What specific words and phrases meant to the respondent.
The type of information the respondent needed to recall in order to answer the question.
What types of strategy the respondent used to retrieve information.
Whether the respondent devoted sufficient mental effort to answer accurately and thoughtfully.
Whether the respondent wanted to tell the truth, or whether the respondent wanted to say something that made him/her look better (i.e. social desirability).
Whether the respondent matched his or her internally generated answer to the response categories given by the survey question.
The data were analysed to identify dominant trends across interviews and to identify ‘discoveries’ (unexpected problems that were only present in one interview, but were important). Findings from these interviews informed further re-wording of the tool. Lastly, we worked with three patient groups within the North East Wales NHS Trust (now part of Betsi Cadwaladr University Health Board) to ensure that the tool captured the diagnostic journey in a way that was acceptable and easy to understand.
The final tool
This revised tool (C-SIM), in 11 separate versions (lung, colorectal, breast, pancreatic, gastric/oesophageal, renal/bladder, endometrial/cervical, haematological, ovarian, prostate and testicular cancers), was then tested in the RCT. It comprised questions concerning the timing of the onset and presentation of symptoms. Respondents were invited to provide either exact or estimated dates of first noticing symptoms and presenting them to primary care. These covered cancer site specific symptom questions (variable between cancers); and four general cancer symptom questions (loss of appetite, weight loss, fatigue and ‘feeling different’). One question asked whether they had been sent for any GP-initiated tests. Further questions covered employment status, educational qualifications, ethnicity, whether they lived alone, co-morbidity, smoking status, family risk of cancer. Finally, as part of the RCT a measure of anxiety, the State-Trait Anxiety Inventory (STAI) was added as the main outcome measure . The tool was produced on double sided A4 paper and was 11 pages long (see Additional file 1–the colorectal tool, and Additional file 2–the cancer specific questions for the other 10 versions). The final version of the tool built upon that previously reported  in that we:
Removed the calendar landmarking tool, because participants reported that they didn’t find it useful
Added a personal letter from the Principal. Investigator inside the front cover, to try to improve the response rate .
Improved the format of the boxes for the questions regarding symptom duration.
Added more tests (CT scan & ultrasound scan), that were missing from the original.
Made multiple changes to the wording to make it more patient-centred and easier to understand.
Phase 2. Randomised controlled trial of method of delivery of the tool.
Setting and participants
The trial took place in the three district general hospitals in North Wales (Wrexham Maelor Hospital, Ysbyty Gwynedd, and Ysbyty Glan Clywd), now all part of Betsi Cadwaladr University Health Board. Fieldwork was undertaken by research officers/nurses from the NISCHR Clinical Research Collaboration. The research officers/nurses attended MDT meetings for the following cancers: breast, colorectal, lung, gynaecological, urological, upper GI, and haematological. At these meetings all patients with new primary diagnoses were alerted to the researcher, who then liaised with the clinical nurse specialists and other members of the clinical teams in order to assess eligibility for inclusion. Patients were eligible if they had a new diagnosis of one of the 11 cancers, were aged 18 years or above (with no upper age limit), were neither too ill nor close to death at diagnosis, were not detected by screening (some cervix, breast and colorectal), and were mentally and linguistically competent to complete the tool.
Eligible patients were then sent a letter from the clinical team inviting participation, enclosing a patient information sheet, a return FREEPOST envelope and a contact form. The researchers then contacted patients who responded positively and arranged to meet them in order to obtain consent (for data collection and access to primary care records), randomisation and tool completion. Reminders letters were sent after two weeks.
Participants randomised to the self-completion arm were handed the final tool (see above) and asked to complete it. For those randomised to the researcher-completion arm, the researcher sat with the participant in a quiet place and went through the tool with them, reading out the questions and entering their answers. Identical tools were used for both groups. All of the researchers received training to ensure that the delivery of the tool was as standardised as possible. Standard prompts in response to questions for both groups were permitted.
The trial was conducted from February to December 2009. Data were entered using Cardiff TeleForm (optical character recognition system); 22 questionnaires (two for each cancer site) were manually checked.
Patients were randomised to either the researcher-completed or the self-completed mode. A computer generated random number sequence based on random permuted blocks of sizes 4, 6 and 8 was used. Allocation concealment was done by the controlled use of sequentially numbered opaque envelopes, stratified by cancer site and by hospital, except for: pancreatic, testicular and ovarian cancer where the expected prevalence was too small to stratify by hospital and a block size of 4 was used to minimise the risk of chance imbalance; and haematological, endometrial/cervix that were stratified by hospital but were also randomised using a block size of 4.
We used the six-item State-Trait Anxiety Inventory (STAI) to determine whether respondents had anxiety related to questionnaire completion . This has a low-burden, and was designed to assess short-term fluctuations in anxiety. Each item was scored 1-4, with a total score out of 24, with higher scores indicating higher anxiety.
After completion of the tool, the researcher completed a brief sheet for each patient comprising the following questions (with answer options):
How long did it take the patient to compete the questionnaire? (<5 minutes, between 5 and 10. minutes, between 11 and 15 minutes, >15 minutes)
Where did the patient complete the questionnaire? (separate room, open area, while receiving treatment).
The researcher answered one further question for the self-completed group only: ‘Did the patient ask for any help?’ (Y/N). This was to capture data relating to help sought when it had not been offered. The researcher answered two further questions for the researcher-completed group only. These were a subjective report of difficulty and anxiety observed whilst the researcher was completing the tool with each patient:
Did the patient find any of the questions difficult? (Y/N).
Was the patient made anxious? (Y/N).
Analysis of primary outcome
The primary outcome when comparing the method of delivery was anxiety, with the analysis carried out blind to treatment allocation. We aimed to transform the data to produce an approximately normal distribution and undertake parametric tests; if not possible then non-parametric testing was undertaken. Missing items were imputed (see findings).
Analysis of secondary outcomes
Completeness of questionnaire
The difference in completeness between the two methods of delivery were compared (z statistic) on questions common to all of the respondents. Hence we analysed data for:
The four generic cancer questions. If the respondent ticked the ‘yes’ box for having the particular symptom, they were then assessed as to whether the rest of the question was entered completely. If they entered dates for first noticing and first reporting (whether exact or estimated), or they reported that they didn’t tell their GP or nurse, it was regarded as ‘complete’. Any less than this was regarded as ‘not complete’.
The demographic questions.
Process and difficulties
From the post-completion sheets, we determined the process of and difficulties associated with both methods of delivery. This was descriptive other than X
2 for comparison of tool completion between modes of delivery.
Comparison of recall of date of first presentation against GP records
We sought to obtain evidence of criterion validity by comparing the dates of consultation recorded by the measure from up to 60 patients against their GP records. A protocol was developed to identify 40-50 consecutive completed questionnaires, ensuring that there were at least 10 patients from each centre and at least three from each cancer site. Patients were excluded if they had the reported symptoms for more than two years (because unlikely to represent the first presentation of cancer). With patient consent we contacted their GP, and asked them to complete a template. We validated the date of reported first presentation of each symptom, for both exact dates and estimated dates. For exact dates, we asked GPs whether there was a consultation on that particular day, or within a two-week window either side. For estimated dates, we developed a protocol for calculating a ‘pseudo-exact’ time-window from the estimated date (see Additional file 3), and provided the GP with this time window for searching. We also asked the date of the actual consultation according to the records. For simplicity, we only analysed data for one symptom per consultation (we chose the first one that the patient listed on their questionnaire).
Time intervals in the diagnostic journey
Whilst the actual time intervals in the diagnostic journey are not reported in this paper, we developed, a protocol for changing reported estimated dates into pseudo-exact dates that permit the calculation of time intervals based upon the responses obtained (Additional file 4).
The primary aim of this study was to detect a difference in anxiety between the two methods of delivery of the tool, with the completion rates as a secondary outcome. 150 patients per group (total 300), with a 95% CI (with a comparative completion rate of 75%) would allow us to detect a difference between groups on response rates assuming a difference of 10% (95% CI +/-9%). An overall completion rate of 75% would give 225 questionnaires for analysis of anxiety. This would give a 95% CI for the difference in log anxiety for the two modes of total width 0.16 (0.08 from the mean to the limit). This is based on a SD of log anxiety of 0.3 (based on previous studies).