International Questionnaire Postal Response Rate: An experiment comparing no return postage to provision of International Postage Vouchers – "Coupon-Réponse International"

Abstract

Background

Previous studies of methods to increase response rates on international postal surveys did not include providing return postage. We provided International Postage Vouchers – "Coupon-Réponse International" to cover this. The objective of this study was to see if these International Postage Vouchers had an effect on response rates.

Methods

Between-groups, randomized, after-only

Results

There was no difference in response rates between the group that received International Postage Vouchers and the group that did not. (p = 0.23)

Conclusion

International Postage Vouchers – "Coupon-Réponse International" have no effect on response rates for international postal surveys.

Background

Studies of interventions to increase response rates on self-administered, postal questionnaires have manipulated a variety of variables including elements of questionnaire design and presentation, sponsorship, pre-notification, postage, envelopes, incentives, rewards, and follow-up techniques. The systematic review by Edwards et al. [1] included 292 trials including 258,315 participants and 75 different strategies for increasing response to postal questionnaires. Various types of postage, on both out-going and return envelopes affected response rate. A search of the literature by Edwards et al. did not uncover any international postal questionnaire studies in which the respondents were provided with return postage in some form.

We wanted to see if providing return postage to international respondents had an effect on response rate. International Postage Vouchers – "Coupon-Réponse International", which can be exchanged at the respondents' post office for stamps, was the only form of international return postage available in Norway. Although this is a cumbersome method of providing return postage, we wanted to see if eliminating postal costs that might be a barrier for some respondents would have an effect. In addition, we hypothesized that the gesture of providing the vouchers would serve as an incentive and it would be up to the individual how to use the vouchers, either for returning the questionnaire or to redeem stamps for personal use.

Methods

Study population, allocation and sample characteristics

The study population comprised journalists in the health field, broadly defined. Of the 100 journalists to whom paper questionnaires were sent, names and postal addresses of the first 62 on the convenience sample list were obtained from the World Health Organisation's list of members of the Health Communication Network by selecting all names where the person's title was listed as "reporter" or "journalist". The remaining 38 names and addresses were obtained from the membership lists of national organisations of health and science journalists.

The 100 names on the list were randomly allocated to a control group (n = 50) and an experimental group (n = 50) by using a computer script. Three questionnaires, two in the control group and one in the experimental group, were returned by the post office as non-deliverable, resulting in a control group of 48 and an experimental group of 49. Of the 97 questionnaires assumed delivered, 73 were to journalists in 20 Western and Eastern European countries, 21 to journalists in Canada and USA, and three to one journalist in each of the following countries: Japan, Israel, and Sri Lanka. All but eleven addresses appeared to be the recipients' workplace.

Intervention

100 health journalists for whom we had no email address were sent a paper version of a questionnaire about barriers and facilitators to high quality health journalism. Five-hundred ninety four health journalists for whom we had email addresses had previously been invited to visit our website and respond to an Internet-based version of the same questionnaire [2].

All respondents received a cover letter inviting them to answer the questionnaire, and a non-stamped, pre-addressed, return envelope. Each respondent in the experimental group received two International Postage Vouchers – "Coupon-Réponse International", which could be exchanged at his or her local post office for stamps. Information on how to make use of the vouchers was included in the cover letter. Otherwise, the cover letters were identical. We reckoned that two vouchers would cover the cost of return postage from any of the countries to the study centre in Norway.

There was no follow-up of non-responders.

Results

Of the 97 questionnaires assumed to have reached the addressees, 12.4% (n = 12), 95% C.I. (5.82%, 18.92%) were returned filled in. Of these, one questionnaire from the experimental group was returned using United Parcel Service, a private postal service, and is included in the analysis as a response in the experimental group. Otherwise, it was not possible to see which respondents in the experimental group had or had not made use of the vouchers, or if stamps had been procured by other means (had on hand, affixed by employer, etc.) Two questionnaires sent to respondents in the control group were filled in and returned by eligible colleagues of the addressees and are included in the analysis.

There was a higher response rate in the control group (16%, n = 8) than in the experimental group (8.2%, n = 4), but there was no difference in the response rate estimates for the two groups in the population (Z = 1.26, p = 0.23, and 95% C.I. -4%, 21.5%).

Discussion and conclusions

Many journalists who were sent the questionnaire might have excluded themselves from the study because they did not write about health matters, resulting in an overall low response rate. Nevertheless, because of random allocation it is assumed that underlying or confounding variables would have been equally distributed in the control group and the experimental group.

As a matter of interest, there was a significant difference (p = 0.001) between the response rate for the e-mail contacted, Internet-based cohort (32%) [2] and the postal cohort (12.4%), resulting in a relative risk = 2.6. This could quite possibly be due to a selection bias between the two groups. Our result goes in the opposite direction from that of VanDenKerkhof et als study, where e-mail contacted, Internet participants were only half as likely as postal-contacted participants to respond to the questionnaire (35% vs 69%, relative risk = 0.51) [3].

The most optimistic interpretation of the 95% CI (-4, 21.5%) is that with sending vouchers, for every 25 vouchers sent the number of responses increases by one compared to not sending vouchers.

We are in danger of committing a Type II error as the study is underpowered (post-hoc calculation = 0.13) However, the results of this study are consistent with the findings of Groves and Olsson in the US that the provision of a stamped, return-addressed envelop compared with the provision of only a return-addressed label did not make a difference in the response rate [4]. When these two studies are considered together, it appears that the provision of return postage does not increase questionnaire response rates from the workplace.