Personally addressed hand-signed letters increase questionnaire response: a meta-analysis of randomised controlled trials

Mailed questionnaires are often used in health research to collect data. However, non-response reduces study sample sizes and can introduce bias [1]. Strategies that appear to increase response have been identified by a systematic review and include making questionnaires and accompanying letters more personal. In the published systematic review, all studies of personalisation were combined in one analysis [2]. The question of whether different methods of personalisation (such as hand-written signatures or including a participant's name on the cover letter) produce different effects on response was not examined. Recently, a randomised controlled trial and meta-analysis of previous trials found no advantage in hand-signed letters compared to those with photocopied or scanned signatures [3]. The study did not investigate the effect of using the participant's name, or using a hand-written signature and the participant's name together.

All randomised controlled trials that evaluated the inclusion of participants' names on the letters accompanying postal questionnaires were identified from a published systematic review [2]. There was no restriction by language, questionnaire topic, or study population. The search criteria used are described in detail in the published report which was last updated in 2003 [2]. Only studies in which all participants were known to be randomly allocated to intervention groups were included. We extracted data from each study on the year of publication, numbers of participants randomised and numbers responding. We pooled the odds ratios for response estimated by each trial in a random effects meta-analysis [4] and tested for heterogeneity in effects using the chi-squared statistic [4] and the I ² statistic [5]. Evidence for systematic differences between small and large trials (e.g. publication bias) was assessed using Egger's test [6]. We hypothesised a priori that due to increased use of these methods through electronic means, the size of the effects of personalisation on response have decreased over time as people become desensitised to the methods. We examined the evidence for this hypothesis by conducting a random-effects meta-regression of the estimated effects on year of publication. A residual maximum likelihood was used to estimate the between-study component of variance.

Fourteen randomised controlled trials including 12,102 participants were identified from the published systematic review (additional file 1: table 1) [7–20]. Further details about the numbers of records of potentially eligible studies retrieved by the search strategy are reported elsewhere [2]. Study participants were individuals from professional groups and members of the public, and included non-respondents to previous mailings of questionnaires. Thirteen trials were conducted in the US and one was conducted in New Zealand. None of the included trials reported using recorded delivery.

We found evidence for a small but statistically significant increase in response when participants were personally addressed by name and signatures were not written by hand on covering letters (pooled OR 1.18, 95% CI 1.03 to 1.34; p = 0.015; figure 1). There was no evidence for changes in this effect over time (regression coefficient = -0.006, 95% CI -0.04 to 0.028; p = 0.64). We found strong evidence for an increase in response when participants were personally addressed by name and signatures were hand-written, compared with when neither method of personalisation was used (pooled OR 1.45, 95% CI 1.27 to 1.66; p < 0.001). There was little evidence for a change in this effect on response over time (regression coefficient = 0.011, 95% CI -0.001 to 0.024; p = 0.068). Heterogeneity between trial results (unconditional on year of publication) was not statistically significant in either analysis (p = 0.753, I ² = 0.0% and p = 0.108, I ² = 35% respectively), even when the higher threshold for significance (p < 0.10) was used.[5] There was no evidence for systematic differences between small and large trials (p = 0.845 and p = 0.600 respectively). Heterogeneity conditional on year of publication remained non-significant in both analyses.

Including participants' names and hand-written signatures on letters sent with postal questionnaires appears to increase response, compared with using neither method. The use of participants' names in the absence of a hand-written signature may also improve response, but the effect appears to be smaller. The magnitude of these effects was hypothesised to have declined over recent years but we found no evidence that this was the case.

This meta-analysis of the best available evidence indicates that researchers using postal questionnaires can increase response by addressing participants by name on cover letters. The effect appears to be enhanced by including hand-written signatures.