Distributing questionnaires about smoking to patients: impact on general practitioners' recording of smoking advice
© Coleman et al; licensee BioMed Central Ltd. 2007
Received: 29 June 2007
Accepted: 24 September 2007
Published: 24 September 2007
Little is known about the impact of questionnaire-based data collection methods on the consulting behaviour of general practitioners (family physicians) who participate in research. Here data collected during a research project which involved questionnaires on smoking being distributed to patients before and after appointments with general practitioners (GPs) is analyzed to investigate the impact of this data collection method on doctors' documenting of smoking advice in medical records.
Researchers distributed questionnaires on smoking behaviour to 6775 patients who attended consultations during surgery sessions with 32 GPs based in Leicestershire, UK. We obtained the medical records for patients who had attended these surgery sessions and also for a comparator group, during which no researcher had been present. We compared the documenting of advice against smoking in patient's medical records for consultations within GPs' surgery sessions where questionnaires had been distributed with those which occurred when no questionnaires had been given out.
We obtained records for 77.9% (5276/6775) of all adult patients who attended GPs' surgery sessions, with 51.9% (2739) being from sessions during which researchers distributed questionnaires. Discussion of smoking was recorded in 8.0% (220/2739) of medical records when questionnaires were distributed versus 4.6% (116/2537) where these were not. After controlling for relevant potential confounders including patients' age, gender, the odds ratio for recording of information in the presence of questionnaire distribution (versus none) was 1.78 (95% CI, 1.36 to 2.34).
Distributing questionnaires about smoking to patients before and after they consult with doctors significantly increases GPs' recording of discussions about smoking medical records. This has implications for the design of some types of research into addictive behaviours and further research into how data collection methods may affect patients' and doctors' behaviours is warranted.
Questionnaires distributed to patients as they attend doctors are frequently used in primary care studies with behavioural outcomes. In particular, such data collection methods have been employed in trials of smoking cessation[2, 3] or health promotion interventions and in some studies investigating the impact of training physicians in smoking cessation methods on doctors' clinical behaviour and patients' smoking. However, little research has investigated the validity of such instruments for these uses. Post-consultation questionnaires are known to over-estimate rates of smoking cessation advice given by general practitioners (family physicians – GPs)[6, 7] and hospital physicians[8, 9], and false positive reports of advice increase as time elapses following patients' consultations[7, 10]. There is even less research which investigates whether giving pre- or post consultation questionnaires to patients affects doctors' consulting behaviour and after extensive literature searching, the authors could identify no studies on this topic. Consequently, we analyse data from a primary care smoking cessation study, to investigate whether or not distributing questionnaires on smoking behaviour to patients before and after they see a general practitioner influenced doctors' recording of smoking cessation advice in patients' medical records.
The Smoking Incentives Study
Data used in this analysis was collected during a study, conducted in 1997/8. This study, the 'Smoking Incentives Study (SIS) investigated the impact of a new payment made to general practitioners for their health promotion activity. Data collection for SIS took place in two distinct periods which were before and after the new payment was introduced. This data collection was undertaken by research assistants (RAs) and involved merely giving out questionnaires on smoking to patients both as they waited to see their GP and after they had done so. The only other role that the RA had within the practice was to inspect medical records of patients attending surgery sessions. The RA had no interaction with the GP, but the GP could not be blinded to and was probably aware that data collection was taking place. GPs knew that questionnaires were about smoking and they could claim the new health promotion payment whenever they identified patients who had stopped smoking for three months or more. SIS was granted ethical approval by the Leicestershire Ethics Committee and had a negative result, showing that the new payment had no impact on rates of general practitioners' brief advice against smoking, as recorded on completed post-consultation questionnaires given to patients by RAs.
Data collection for current analysis
This analysis is based on 32 general practitioners who participated in at least one of the data collection periods for SIS. Each GP had data collected (i.e. questionnaires distributed) at up to 12 randomly-selected surgery sessions over a 16 month period. All doctors used paper, rather than electronic, medical records, so to identify and obtain the records of patients who had attended surgery sessions, RAs used receptionists' lists which logged attending patients. For every surgery session during which data collection occurred, RAs identified comparator surgery sessions where no questionnaires had been distributed. To be selected as comparators, surgery sessions needed to be with the same general practitioner, on the same weekday and at the same time of day (i.e. morning or afternoon). Comparator surgery sessions were usually within one week of data collection ones, though this depended on whether or not a surgery with the same doctor was available at the appropriate time. Selecting a surgery with the same general practitioner was intended to eliminate inter-doctor variation in clinical behaviour and comparing surgery sessions on the same days at similar times was intended to minimise variation in numbers of patients being booked in different sessions.
RAs obtained the total number of patients attending each surgery session from receptionists' records and then sought the medical records of patients attending sessions. Where medical records could be obtained, RAs noted the age and sex of patients and also whether or not discussion of smoking was recorded on the date of the selected surgery session. RAs were trained to extract data on smoking cessation advice from medical records by a general practitioner and piloting of data extraction indicated that this was accurate. RAs inspected all available medical records of patients attending data collection surgeries and recorded whether or not there was any note that smoking had been discussed. From the brief notes inspected, it was not possible to differentiate whether doctors' advice to stop smoking had been given or merely that smoking status had been ascertained, so 'discussion of smoking' may reflect either occurrence. Inter-observer reliability of data extraction from medical records was tested using Cohen's kappa, comparing the data on smoking advice extracted from 100 medical records by each of two RAs with that extracted by a general practitioner.
We compared the medical records of patients attending surgeries where questionnaires were distributed with those where no data collection occurred to determine whether there was any difference in the proportions containing a note that smoking cessation advice had been given. We used a multi-level logistic regression model, allowing for variation at the practice, GP and clinic level, to estimate the effect of data collection procedures on recording of advice against smoking in records, with adjustment for age in quartiles, sex and whether or not the new health promotion payment had been available to general practitioners (via SIS) at the time of data collection.
We obtained and inspected medical records for 77.9% (5276/6775) of all adult patients who attended general practitioners' surgery sessions and the number of medical records inspected for each GP varied between 48 and 306. Overall, 6.4% (336/5276) of records contained a note that advice against smoking had been given. 60.2% (3178/5276) of records inspected were from women and the mean [SD] age of patients whose records were inspected was 40.7 [22.3], with age data being unavailable for 4 patients.
51.9% (2739) of medical records were from surgery sessions during which researchers distributed questionnaires on smoking behaviour and the remainder were from surgery sessions where no data collection occurred. The distribution of age and gender of patients was similar between the two groups, but the distribution of data collection surgeries differed between the 'after' period of SIS (when GPs could claim the new payment) and the 'before' period when this was not available; a researcher distributing questionnaires was present in 1271 out of 2767 (45.9%) patients seen during the 'after' period, and 1266 out of 2509 (50.5%) seen during the 'before' period. The level of agreement between both RAs and a general practitioner for the presence or absence of a note of discussion of smoking in the medical records was good with the Kappa values (95% CI) for agreement between a general practitioner and the two RAs being 0.79 (0.6 to 0.98) and 0.85 (0.66 to 1.0) respectively.
Recording of advice against smoking in medical records was significantly more likely when a researcher distributed questionnaires in the surgery session; when a researcher was present 8.0% (220/2739) of medical records had a note that smoking had been discussed versus 4.6% (116/2537) when no researcher was present. (Odds ratio 1.80, 95% CI 1.38 to 2.35, p < 0.001).
Recording of advice against smoking in medical records was also more likely in female than male patients (6.9% of women's notes included a record of smoking advice versus 5.6% of men's), and in the middle two quartiles of age (9.8 and 7.5%) than in the youngest (< age 24, 3.8%) or oldest (≥ age 59, 4.3%) patients. Additionally, patients' medical records from the period of SIS before the new health promotion payment became available to general practitioners were slightly more likely to have a record of advice (6.9%) than those from the period during which this could be claimed (5.8%). This suggests that the new health promotion payment had no impact on recording of smoking advice in medical records.
The effect of researcher distributing questionnaires, age, sex and availability of health promotion payment within SIS on recording of smoking advice in patient's medical records (from a multi-level logistic regression model)
Odds ratio (95% CI)
Researcher distributing questionnaires
1.78 (1.36 to 2.34)
1.16 (0.91 to 1.48)
Age (in quartiles)
0 to 24
24 to 40
2.79 (1.98 to 3.94)
40 to 59
2.12 (1.48 to 3.04)
59 to 93
1.19 (0.80 to 1.78)
Payment available in SIS study
0.90 (0.68 to 1.18)
This analysis demonstrates that when researchers distribute questionnaires about smoking to patients before and after they see their general practitioner and the doctor is not blind to questionnaire contents, he or she is almost twice as likely to record discussion of smoking in the medical records. The impact of the researcher had an independent effect which was not affected by patients' age or sex or the availability to general practitioners of a new smoking-related health promotion payment which was being evaluated.
The authors searched but could find no papers investigating the impact on general practitioners' behaviour of questionnaires distributed to patients, so we believe that this study is original. Additionally, data presented here was collected at a time when patients in the UK were not required to give consent for medical researchers to inspect their medical records for research purposes (i.e. to 'opt in' to this type of research). If the study were repeated today, researchers would need to contact patients and obtain their consent before inspecting records, introducing potential bias into this process. A weakness of the study is that researchers were only able to obtain 78% of the medical records for patients who attended selected surgery sessions and we can say nothing about the records which were not obtained. Researchers relied on clerical staff to find records in addition to performing their usual daily work tasks and it is possible that this unpaid work did not receive high priority, resulting in the relatively high rate of non-located records. It is unlikely, however, that researchers' inability to locate medical notes would be related to doctors' recording of discussing smoking, so data from records which researchers could not access would not be expected to affect study findings. Finally, it is possible that unmeasured factors may have affected doctors' documentation of the smoking advice that they delivered to patients. For example, doctors are more likely to advise smokers' who are interested in stopping or who present with smoking-related problems. It is possible, therefore, that doctors' propensity to discuss smoking when it may be relevant to patients' presenting problems, or with 'interested' smokers, results in more discussion of smoking between GPs and smokers than non-smoking patients. If we had collected data on patients' smoking status in both study groups, then we would have been able to adjust our findings for any influence that this may have had. However, our method for selecting comparison group surgeries should have ensured that these were similar and seems unlikely that smoking prevalence would differ significantly between patients attending the two groups of surgeries, such that this could explain study findings.
General practitioners do not document all of the advice against smoking which they deliver to patients, so questionnaires may have prompted more discussion of smoking by GPs, more recording of advice or both of these outcomes. One could differentiate between these possibilities by either directly observing or recording what actually occurred in consultations, but recording of consultations can also influence consulting behaviour. Researchers who plan to use pre- or post-consultation questionnaires need to consider the potential impact of these on doctors' behaviours. It is likely that questionnaires addressing other topics (e.g. alcohol or dietary intake) may have similar impacts on other aspects of clinical behaviour (e.g. discussions about alcohol or recording of these). Such impacts are likely to be most important in studies where recording of doctors' behaviours in medical records are used as outcomes. In these kinds of study, questionnaires use could result in higher overall rates of outcomes than anticipated, affecting sample sizes need to show differences between groups.
Researchers also need to be cautious when using pre-consultation questionnaires in primary care trials that involve allocating participants to "normal care" treatment groups. Some primary care smoking cessation studies have used "normal care" treatment arms[2, 16–18], but questionnaires distributed to general practitioners in 'normal care' trial arms would be expected to increase doctors' recording of smoking status and may even influence doctors' delivery of smoking advice to patients. As doctors' brief advice against smoking is an effective smoking cessation intervention, such questionnaires could potentially increase cessation rates in 'normal care' trial arms, interfering with the power of any trial to detect differences between trial groups.
This study provides direct evidence for the value of blinding study participants, including health professionals, to the contents of research questionnaires. However, in applied health services research this is not always possible because doctors will often wish to see questionnaires to be reassured that data collection instruments are not too long and will not cause offence to patients. Further research into the impact of questionnaires distributed to patients for research purposes is required and where blinding of research participants is not possible, researchers must consider the likely implications of this.
When researchers distribute questionnaires to patients before and after they consult with GPs and GPs are aware of questionnaire content, this can affect GPs' behaviour. Giving out questionnaires which asked about patients' smoking habits resulted in GPs recording more advice against smoking in patients' medical records. This increase could have been due to an increase either in the actual amount of smoking cessation advice delivered or in the recording of advice given. Study findings require consideration when designing research studies that use data recorded in medical records as outcome measures.
We wish to thank Trent NHS Executive for funding SIS during which this data was collected. We also wish to thank Miss Laura Jones and Mrs Rose Henson for secretarial assistance.
- Simons-Morton DG, Mullen PD, Mains DA, Tabak ER, Green LW: Characteristics of controlled studies of patient education and counseling for preventive health behaviors. Patient Education & Counseling. 1992, 19: 175-204. 10.1016/0738-3991(92)90196-P.View ArticleGoogle Scholar
- Russell MA, Wilson C, Taylor C, Baker CD: Effect of general practitioners' advice against smoking. British Medical Journal. 1979, 2: 231-235.View ArticlePubMedPubMed CentralGoogle Scholar
- Jackson PH, Stapleton JA, Russell MA, Merriman RJ: Predictors of outcome in a general practitioner intervention against smoking. Preventive Medicine. 1986, 15: 244-253. 10.1016/0091-7435(86)90044-7.View ArticlePubMedGoogle Scholar
- Mullen PD, Simons-Morton DG, Ramirez G, Frankowski RF, Green LW, Mains DA: A meta-analysis of trials evaluating patient education and counseling for three groups of preventive health behaviors. Patient Education & Counseling. 1997, 32: 157-173. 10.1016/S0738-3991(97)00037-2.View ArticleGoogle Scholar
- Lancaster T, Silagy C, Fowler G: Training health professionals in smoking cessation (Cochrane review). Cochrane Database Syst Rev. 2000, CD000214.Google Scholar
- Wilson A, McDonald P: Comparison of patient questionnaire, medical record, and audio tape in assessment of health promotion in general practice consultations. BMJ. 1994, 309: 1483-1485.View ArticlePubMedPubMed CentralGoogle Scholar
- Ward J, Sanson-Fisher R: Accuracy of patient recall of opportunistic smoking cessation advice in general practice. Tob Control. 1996, 5: 110-113.View ArticlePubMedPubMed CentralGoogle Scholar
- Nicholson JM, Hennrikus DJ, Lando HA, McCarty MC, Vessey J: Patient recall versus physician documentation in report of smoking cessation counselling performed in the inpatient setting. Tob Control. 2000, 9: 382-388. 10.1136/tc.9.4.382.View ArticlePubMedPubMed CentralGoogle Scholar
- Gerbert B, Hargreaves WA: Measuring physician behavior. Med Care. 1986, 24: 838-847. 10.1097/00005650-198609000-00005.View ArticlePubMedGoogle Scholar
- Solberg LI: Practical implications of recall bias. Tob Control. 1996, 5: 95-96.View ArticlePubMedPubMed CentralGoogle Scholar
- Coleman T, Wynn AT, Barrett S, Wilson A, Adams S: Intervention study to evaluate pilot health promotion payment aimed at increasing general practitioners' antismoking advice to smokers. BMJ. 2001, 323: 435-436. 10.1136/bmj.323.7310.432.View ArticlePubMedPubMed CentralGoogle Scholar
- Altman DG: Practical Statistics For Medical Research. 1991, London: Chapman and HallGoogle Scholar
- Coleman T, Wynn AT, Stevenson K, Cheater F: Qualitative study of pilot payment aimed at increasing general practitioners' antismoking advice to smokers. BMJ. 2001, 323: 432-435. 10.1136/bmj.323.7310.432.View ArticlePubMedPubMed CentralGoogle Scholar
- Wynn A, Coleman T, Barrett S, Wilson A: Factors associated with the provision of anti-smoking advice in general practice consultations. British Journal of General Practice. 2002, 52: 997-999.PubMedPubMed CentralGoogle Scholar
- Coleman T, Manku-Scott T: Comparison of video-recorded consultations with those in which patients' consent is withheld. Br J Gen Pract. 1998, 48: 971-974.PubMedPubMed CentralGoogle Scholar
- Jamrozik K, Vessey M, Fowler G, Wald N, Parker G, Van Vunakis H: Controlled trial of three different antismoking interventions in general practice. British Medical Journal Clinical Research Ed. 1984, 288: 1499-1503.View ArticlePubMedPubMed CentralGoogle Scholar
- Slama K, Redman S, Perkins J, Reid AL, Sanson-Fisher RW, Slama K, Redman S, Perkins J, Reid AL, Sanson-Fisher RW: The effectiveness of two smoking cessation programmes for use in general practice: a randomised clinical trial. BMJ. 1990, 300: 1707-1709.View ArticlePubMedPubMed CentralGoogle Scholar
- Sanders D, Fowler G, Mant D, Fuller A, Jones L, Marzillier J: Randomized controlled trial of anti-smoking advice by nurses in general practice. Journal of the Royal College of General Practitioners. 1989, 39: 273-276.PubMedPubMed CentralGoogle Scholar
- Ashenden R, Silagy C, Weller D: A systematic review of the effectiveness of promoting lifestyle change in general practice. Family Practice. 1997, 14: 160-176. 10.1093/fampra/14.2.160.View ArticlePubMedGoogle Scholar
- Hennenkens CH, Buring JE: Epidemiology in Medicine. 1987, Boston, USA: Little, Brown and CompanyGoogle Scholar
- Murphy E, Spiegal N, Kinmonth AL: 'Will you help me with my research?' Gaining access to primary care settings and subjects. British Journal of General Practice. 1992, 42: 162-165.PubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/7/153/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.