Effect of prize draw incentive on the response rate to a postal survey of obstetricians and gynaecologists: A randomised controlled trial. [ISRCTN32823119]

Abstract

Background

Response rates to postal questionnaires are falling and this threatens the external validity of survey findings. We wanted to establish whether the incentive of being entered into a prize draw to win a personal digital assistant (PDA) would increase the response rate for a national survey of consultant obstetricians and gynaecologists.

Methods

A randomised controlled trial was conducted. This involved sending a postal questionnaire to all Consultant Obstetricians and Gynaecologists in the United Kingdom. Recipients were randomised to receiving a questionnaire offering a prize draw incentive (on response) or no such incentive.

Results

The response rate for recipients offered the prize incentive was 64% (461/716) and 62% (429/694) in the no incentive group (relative rate of response 1.04, 95% CI 0.96 – 1.13)

Conclusion

The offer of a prize draw incentive to win a PDA did not significantly increase response rates to a national questionnaire survey of consultant obstetricians and gynaecologists.

Backgound

Postal surveys are commonly used in medical research as an efficient method of obtaining information about medical practitioners attributes, behaviours, attitudes and beliefs. There is evidence that response rates to surveys are declining in general practice [1] and this trend may also be reflected in hospital based medicine. This is of concern as the external validity of the findings from these surveys is dependent upon adequate response rates [1]. Therefore the identification of effective strategies to increase response rates is important to improve the quality and generalisability of such health research.

In two earlier randomised controlled trials, we reported no effect on the rate of response to a medical questionnaire of paper quality or provision of a complimentary pen [2, 3]. A recently published systematic review showed that monetary incentives doubled the odds of response in the general population. Furthermore, it demonstrated that non-monetary incentives may also be effective in improving response rates by a smaller degree, although the effect varied according to population and type of incentive [4]. In order to further test the impact of non-monetary incentives on survey response by medical professionals, we hypothesised that entry into a prize draw to win a personal digital assistant (PDA) would increase response to a survey of practising obstetricians and gynaecologists.

Methods

All current consultants identified from the Royal College of Obstetricians and Gynaecologists (RCOG) database were sent a single page questionnaire with a covering letter and prepaid response envelope. The questionnaire consisted of 10 questions aimed at determining current practise for the laparoscopic diagnosis and treatment of women with pelvic pain due to endometriosis. Recipients were randomised to receive a questionnaire containing a covering letter offering the incentive of entry to a prize draw to win a personal digital assistant (PDA) or no incentive. A prominent gynaecological device manufacturer provided the PDA and this was acknowledged in the covering letter of those randomised to receive the incentive.

The randomisation sequence was computer generated and group allocation was concealed from the participants and investigators throughout the study. One reminder was sent to non-responders three months after the original with the same incentive according to prior randomisation. Based on the response rate from a previous gynaecological questionnaire [5] we assumed that inclusion of a prize draw incentive would increase the proportion of responders by 10% from 60% to 70%. This meant that the sample size (1410) had 95 % power (alpha = 0.05, beta = 0.05). Relative response rates were calculated and statistical significance tested for a difference between proportions (independent observation) responding.

Results

Of the 1410 consultants surveyed, 716 were randomised to receive the prize draw incentive of a PDA (fig 1). The overall response rate was 890/1410 (63%). The response rate for recipients offered the prize incentive was 64% (461/716) and in the no incentive group was 62%(429/694) The relative rate of response to the questionnaire when the prize draw incentive was included compared to without was 1.04 (95% CI 0.96 – 1.13).

Figure 1

Responses to survey according to inclusion of prize draw incentive

Discussion

The addition of a prize draw incentive to win a PDA in a national questionnaire survey of medical practitioners did not significantly increase response rates. There are several explanations for this. Firstly, negative studies are often due to type II errors, that is inability to detect a difference when one actually exists, due to small sample size. However our study was adequately powered. Secondly, our assumption that a PDA is an appealing commodity to consultant gynaecologists may be wrong hence there is insufficient extra motivation for response. Also many consultants may already possess a PDA, thus reducing the desire to acquire one and hence its value as an incentive. Thirdly, any positive effect on response of the incentive may have been offset because it was explicitly stated that it was supplied by industry. The content of the questionnaire was about laparoscopic surgery in endometriosis, but the sponsoring device manufacturer produced no product directly for use in treating endometriosis. Despite this, and although only one respondent stated concern over industry sponsorship of the incentive, this view may have been more prevalent amongst non-responders.

We considered a 10% increase in response rate to be a clinically worthwhile effect to justify utilisation of non-monetary incentives when disseminating surveys. A systematic review of 45 trials where a non-monetary incentive was employed found an average odds ratio of 1.19 (95% CI 1.11 to 1.28) compatible with a 4% increase in response in the current study from 60 to 64% [4]. However, inferences regarding the role of non-monetary incentives in surveys of medical professionals are limited from this study because of significant clinical and statistical heterogeneity. Studies confined to medical professionals have shown either small increase (less than 10%) in response [6] or more rapid initial response in favour of incentive [7]. Our study did not assess the play of other factors that may be predictive of improved response to non-monetary incentives (e.g. age, gender, qualifications etc) and this could be the focus of further studies.

In view of the lack of effectiveness shown in our study and the costs associated with provision of a non-monetary incentive, investigators should be deterred from utilising this particular strategy to improve response rates from medical practitioners. If non-monetary incentives are used, then the nature of any incentive offered should be considered carefully so that it is optimally appealing to the target group.