We have shown that reduced burden (an abridged questionnaire containing fewer questions) following a postcard reminder was the most effective intervention strategy to increase GP response rates to a postal questionnaire. Whilst reduced burden increased GP response rates, this finding did not apply to dentist response rates in our study. We also found that a £20 book voucher incentive did not increase dentist response rates.
Our RCTs had a number of limitations. Our studies did not assess other factors that could predict improved response (e.g. age & gender). We made an assumption that a book voucher mailed with the first questionnaire would appeal to dentists. This incentive may have been insufficient motivation to respond. Monetary incentives have been found to be more effective than non-monetary incentives however this was not feasible due to University regulations that prevented us from mailing cash incentives. It is difficult to ascertain whether a book voucher is viewed as a monetary or non-monetary incentive. Immediate incentives given at the time of the questionnaire have been shown to be more effective than those given on condition of a response . We are unable to determine if the value of the book voucher impacted on our response rate.
Our results found that an incentive led to a reasonable (9%) increase in response rate. The sample size in RCT A was chosen to achieve a 50% response rate for the original PRIME study. If our original response rate had been higher we may not have chosen a 22% difference as the minimum clinically important difference. The fact that it did not reach conventional statistical significance might have been a power issue.
As indicated above, some caution should be used when applying the response rates in RCT B, as the analysis attributes responses to the last prompt given – even if that prompt was not necessarily pivotal in producing the response (not the number of contacts). If this is not the case, then the cost-effectiveness results may be biased against less complex strategies. The cost-effectiveness element of our results also hinges on the implicit assumption that all responses (including those who opt out) have the same value. If this is the case, then those strategies including responses from an abridged questionnaire will tend to be more cost-effective than those with longer questionnaires, given the lower material and data entry costs. Responses from long questionnaires will be more cost-effective in analyses focussing on cost per item of information, and may be more cost-effective if the quality of information can be assessed. As such, the strategy of sending out additional (long) questionnaires to a larger sample may be even more efficient to increase the absolute number of responses than suggested here. Whilst it may increase the response rate it may not provide assurance of a representative or valid response.
Our final response rates to RCT A and RCT B were not high. Following the report by Cummings et al. that up to 1995, response rates of postal questionnaires of healthcare professionals remained constant at approximately 60% , Cook et al. demonstrated that by 2005 response rates in surveys of healthcare professionals had slightly declined to an average of 57.5% . Kaner et al. reported doctors describing day to day work pressures and lack of perceived salience as reasons for not completing surveys . Our full questionnaires operationalised multiple theoretical models that resulted in long questionnaires asking seemingly repetitive questions. Additionally, our request to access radiograph and prescribing data may have deterred a larger group from completing a questionnaire.
Interventions to increase response rates may also incur negative consequences. They may lead to differential rates of response or non-response from specific subgroups. In these PRIME surveys [9, 10] we received the required pre-specified number of responses from a population sample who had a range of behavior, behavioral simulation and intention, and who reported a range of cognitions. It was not possible to explore the representativeness of responders to these interventions hence there may be a risk of bias. It is also possible that the quality of responses received may differ across the intervention groups but we did not explore quality of response. Further research is needed to explore the effectiveness and impact of other methods to maximise response rates of health care professional postal questionnaires.