- Research article
- Open Access
Processes for engaging and retaining women who are experiencing adversity in longitudinal health services research
BMC Health Services Research volume 19, Article number: 833 (2019)
Women and families experiencing socioeconomic and psychosocial adversity are the least likely to access health care but most likely to benefit. For health services to effectively meet the needs of individuals experiencing adversity, research involving the health services must be truly representative. However, individuals experiencing adversity are typically excluded from or underrepresented in health services research. This paper reports on the implementation of a quality improvement approach designed to support recruitment and retention of pregnant women experiencing adversity in a longitudinal, health services randomized controlled trial (“right@home”).
right@home recruited Australian women from 10 public maternity hospitals across the states of Victoria and Tasmania who were experiencing adversity (≥2 risk factors on screening survey). Regular follow-up assessments were conducted by phone or face-to-face to child age 2 years. Research processes were designed taking heed of previous research demonstrating effective strategies for recruiting and retaining minority groups (e.g. piloting the recruitment process; recruiting via the health service providing care to the subgroup; remunerating participants); however, we were concerned that important information was missing. Therefore, once recruitment began, we conducted a continuous evaluation of the research processes, testing and implementing changes to processes or new strategies to maximize recruitment and retention (e.g. using a suite of strategies to maintain contact with families, using flexible data collection methods, obtaining consent for data linkage for future health and education data).
right@home enrolled a large cohort of women (N = 722) experiencing high levels of adversity according to socioeconomic status and psychosocial risk factors, and achieved excellent retention (83% completion at 2 years). Most strategies appeared to increase recruitment and retention. All required additional time from the research team to develop and test, and some required extra funding, which ranged from minor (e.g. printing) to substantial (e.g. salaries, remuneration).
By taking a quality improvement approach, supported by sufficient resourcing and flexible research processes, it is possible to recruit and retain a large cohort of women experiencing adversity who are typically missed or lost from longitudinal research.
Recruitment and retention of participants in large-scale, longitudinal, health services research can be complex and challenging, especially when participants are selected for their experience of socioeconomic or psychosocial adversities . Individuals experiencing adversity have the greatest need for health services and supports but are the least able to access them (described as the “inverse care law”) . Reasons for this include cost and an individual’s ability to identify their own needs and seek and obtain services, and these barriers can be compounded by a mistrust of or difficulty relating to the system . Given that individuals experiencing adversity are less likely to interact with health services, researchers can struggle to make contact with eligible individuals in the first place, limiting opportunities for participation.
When it is possible to make contact and invite participation, the artificiality of research design can discourage involvement. Research activities such as providing informed consent, travel and related costs to attend assessments, and finding time to complete assessments, can all be burdensome. Committing to research over years can be difficult for individuals with less stability in their lives or less confidence in, or literacy or familiarity with, health services . However, to determine whether services can be accessed by and improve outcomes for these individuals, health services research must be truly representative of the population being studied. Otherwise, there is the potential for research funding to be wasted on evaluating services that are either inaccessible or ineffective for the individuals most in need of support .
In 2012, recognizing the substantial adversity and poor outcomes experienced by some Australian families, the Victorian and Tasmanian governments together with philanthropy funded the “right@home” partnership to develop and evaluate the largest multi-site, multi-state, Australian randomized controlled trial of nurse home visiting, offered from pregnancy to child age 2 years . Designed for delivery via the child and family (CFH) service that is freely available to all Australian families, right@home aimed to improve the learning and development of children born to women selected for their experience of adversity. We designed the research processes for right@home using the strategies identified by previous research for maximizing recruitment and retention with minority groups (such as ethnic/cultural minorities and individuals with reduced socioeconomic resources) . These strategies included identifying the venues or methods that reach the population of interest ; using researcher-led instead of clinician-led recruitment to reduce gatekeeping or selection bias ; piloting recruitment processes for acceptability and feasibility ; collecting informed consent and data in-person to help support individuals with low literacy ; and acknowledging the time and cost burdens of research participation by offering remuneration .
Although these processes appeared necessary, we were concerned that they may not be sufficient, and therefore aimed to investigate whether a quality improvement approach to implementing and testing the research processes could maximize recruitment and retention of women in the trial. The intention was not to evaluate whether specific processes worked on their own. Rather we aimed to evaluate whether taking a quality improvement approach using multiple, flexible processes could achieve (i) the target sample size (N = 714, see Protocol for calculation ) and (ii) prevent loss-to-follow-up as far as possible by attempting to treat every enrolled participant as essential to the trial’s validity.
The right@home RCT (ISRCTN89962120, https://doi.org/10.1186/ISRCTN89962120) is unique in the Australian research landscape because it prioritized the 5–7% of pregnant women experiencing greatest adversity and was delivered via the existing CFH service. Methods are previously described in the published Protocol . Briefly, researchers recruited pregnant women from the waiting rooms of antenatal clinics in 10 public maternity hospitals across Victoria and Tasmania, inviting them to complete a 10-item brief risk factor (BRF) survey . Women with 2 or more risk factors (plus additional eligibility criteria, see Additional file 1: Table S1) were invited into the trial. Interested and eligible women were visited in their homes by researchers who collected informed consent and conducted a comprehensive baseline questionnaire before random allocation.
The intervention comprised 25 home visits from pregnancy to 2 years, focusing on parent care of the child, responsivity to the child, and providing a good quality home learning environment. The standard CFH service provided the comparator (control), comprising around 6–9 predominately clinic-based appointments. Researchers followed-up with women in both groups via phone interviews (approximately 30 min) when children were 6 weeks, 6 months and 18 months old, and via home-based interviews (approximately 2 h) when children were 1 and 2 years of age. Women’s postcodes were linked with the national Socio-Economic Indexes for Areas Index of Relative Disadvantage (SEIFA); lower scores indicate greater adversity .
The research processes for recruitment and retention – the focus of this paper – were designed based on the aforementioned literature plus input from the program developers. For example, we piloted the recruitment processes,  which helped us understand how the antenatal clinics ran; build relationships with clinic staff; assess whether the BRF survey was a useful measure for identifying eligibility; gauge women’s interest in participating in the larger RCT; and decide how to remunerate women. Once recruitment began, we implemented a quality improvement approach, defined as a weekly  evaluation of how the research processes were supporting or hindering recruitment and retention. For this, informal feedback – termed anecdotal in the Table 1 – was sought from field researchers, participants and other collaborators (e.g. hospital clinic staff), on perceived barriers to recruitment and retention, and strategies to overcome them. Feedback was summarized and discussed at weekly investigator meetings. Based on consensus, the team retained processes that appeared to contribute to increasing the sample size and retention, and discarded those that did not. As the research team were blinded to randomization status until all 2 year data were collected, our processes were applied to both trial arms without differentiation.
The Table 1 lists the barriers and strategies. Those that were identified in the existing literature and implemented during the design phase are described with terms like “potential or anticipated”; otherwise, the barriers are those encountered during the trial. The Table 1 presents the findings according to 4 of the 5 research phases identified by Bonevski’s systematic review:  sampling; recruitment and gaining consent; data collection and measurement; and retention and attrition (intervention delivery is excluded as it was coordinated by a different workforce for the RCT). Where available, the number or proportion of women recruited or retained are used to evaluate the usefulness of strategies. Since assessment points (and therefore the research processes) were repeated multiple times, some processes are evaluated using data for one assessment point (most often the 2 year assessment as this was the primary outcome point and the furthest from recruitment). Data are provided for the complete sample where they were collected as discrete fields in the database and therefore easily exported and analyzed. Where information needed to be identified and extracted from the detailed descriptive participant notes, the data analysis was limited to a random sample of 100 participants selected using Stata (Intercooled Stata, v14·2 for Windows: College Station, TX, USA).
Of the 9511 pregnant women approached at clinics, 6977 were eligible and 5586/6977 (80.1%) completed the BRF survey (see Fig. 1). Of these, 1427 were eligible for the RCT, and the final enrolled cohort comprised 722 women experiencing high levels of adversity (see Additional file 2: Table S2), surpassing the necessary sample size. We maintained excellent retention and completion rates to child age 2 years (see Fig. 1): 90.4% completion and 1.5% cumulative withdrawal at 6 weeks; 91.1%/2.8% at 6 months; 88.2%/4.6% at 1 year; 85.1%/6.4% at 18 months, and 82.5%/8.9% at 2 years. The gaps between completion and cumulative withdrawal reflect the proportions of women who declined or were not contactable.
As described in the Table 1, most strategies appeared to support recruitment and retention. After recruitment started, new strategies required time from the research team to develop, obtain ethical approval (where necessary) and implement. Some changes also required additional funding, which ranged from minor (e.g. printing, postage costs) to substantial (e.g. salaries to extend the recruitment phase, funding participant remuneration). There was no evidence of differences between trial arms in the times families changed addresses or the number of contact attempts made toward completing an assessment. However, it was more common to reach out to the alternate contacts for usual care participants than for intervention participants (see Table 1). This is likely because intervention families had more frequent contact with the research team through their regular visits with intervention nurses, whereas usual care families did not. There was no evidence of differential attrition between trial arms by the 2 year follow-up assessment.
Several contextual factors supported recruitment and retention. We hired researchers who were highly skilled in building relationships with the participating women; provided rigorous researcher training in the project’s standard operating procedures; and employed a research coordinator (alongside the research manager) who was responsible for monitoring recruitment and retention by being the central point of contact for researchers, families, clinics and other contributors (e.g. Centrelink representatives). With this central role, it was possible to implement protocol changes relatively rapidly (dependent on other approvals such as HREC), obtain feedback and evaluate their usefulness.
Using a quality improvement approach to evaluate and optimize the research processes for a longitudinal, health services RCT resulted in the recruitment of a large cohort of pregnant Australian women experiencing adversity, and a high proportion of women retained to child age 2 years. Two main factors supported these successful research processes. First, the RCT had sufficient funding for the substantial research costs, which included salaries of skilled researchers employed for sufficient hours to conduct in-person research, plus researcher travel and participant remuneration. Second, we had flexibility as far as possible within a research paradigm to make continual updates and changes to the research protocol, which relied on collaborations with hospital clinic staff and ethics committees to approve and implement protocol amendments quickly.
A consequence of engaging and maintaining a large cohort is that it provides sufficient power to detect small between-group differences. This also allows for ongoing research follow-up beyond 2 years, since the sample remains large enough to meaningfully analyze long-term effects. We are confident that our efforts to obtain consent for data linkage will prove fruitful for long-term follow-up; the CFH data have already been used to conduct a cost evaluation of the intervention. Strategies in this study were implemented together (and are not mutually exclusive). This means we can only report recruitment and retention data corresponding to the relevant time period, rather than as a direct outcome of a specific strategy. For some strategies, we have no supporting data or only anecdotal reports. A formal evaluation of strategies (e.g. using a RCT design rather than a descriptive review) could provide evidence on effectiveness . However, our intention was to evaluate whether a flexible quality improvement process, that implemented multiple strategies, could achieve high recruitment and retention.
Traditional data collection processes with large cohorts, or those experiencing more advantage, use postal or online surveys and minimal reminder processes. Given the number and types of contact needed to retain participants in right@home, traditional process would have failed to adequately engage and retain the cohort. Consistent with our findings, previous studies have demonstrated strong support for person-to-person (e.g. phone or direct) data collection and flexible data collection methods, with mixed evidence for the usefulness of promotional materials and readability . Almost all other studies that have examined the effect of remuneration, including incentives, report that remuneration supports participant engagement . These studies emphasize the need to keep remuneration clear of coercion, yet, remuneration remains controversial even though the costs incurred by participants are well-established. As Swanson noted over 20 years ago,  funding agencies need to recognize both the considerable cost of research with minority groups as well as the importance of participant remuneration. Finally, we found that strategies that minimized “barriers to entry”, such as recruiting in clinic waiting rooms, and booking assessments on-the-spot with women or by text messages, as well as all strategies to maintain up-to-date contact details for participants, were important features supporting retention and data collection.
These findings are relevant for community-based research that focuses on minority groups. The findings emphasize the need for increased funding and timeframes for recruitment and retention when compared with research with cohorts experiencing less adversity. Research with individuals experiencing adversity should be considered for the required level of funding by granting bodies when competitive funding is reviewed and awarded . Future studies could evaluate strategies in a more systematic way by obtaining feedback from participants via single or simple questions asking for opinions on processes asked in-person at assessments or by text or email. In addition, it is important to increase the transparency around the true cost of the activities that make up this type of research; for example, how many researcher hours are needed, per participant, to complete enrolment or follow-up assessments.
Research that includes women and families experiencing adversity is crucial for designing health services to meet the needs of those least likely to access a service but most likely to benefit. This is only possible with sufficient funding and flexible research processes. Taking a quality improvement approach and assessing each research process in detail can support high recruitment and retention. By increasing the transparency of research processes, researchers can argue the case for adequately resourcing research to ensure robust estimates of effectiveness. This will ultimately improve the quality and reach of health services for those who need them most.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available due to ongoing analyses.
Brief risk factor survey
Child and Family Health
Human Research in Ethics Committee
International Standard Randomized Controlled Trials Number
Randomized controlled trial
Socio-Economic Indexes for Areas (SEIFA) Index of Relative Disadvantage
Bonevski B, et al. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BioMed Cent Med Research Methodol. 2014;14:42.
Tudor Hart J. The inverse care law. Lancet. 1971;297(7696):405–12.
Levesque JF, Harris MF, Russell G. Patient-centred access to health care: Conceptualising access at the interface of health systems and populations. Int J Equity Health. 2013;12(1):18.
Goldfeld S, et al. right@home: a randomised controlled trial of sustained nurse home visiting from pregnancy to child age 2 years, versus usual care, to improve parent care, parent responsivity and the home learning environment at 2 years. British Med J Open. 2017;7(3):e013307.
Swanson GM, Ward AJ. Recruiting minorities into clinical trials toward a participant-friendly system. J Natl Cancer Inst. 1995;87(23):1747–59.
UyBico SJ, Pavel S, Gross CP. Recruiting vulnerable populations into research: a systematic review of recruitment interventions. J Gen Intern Med. 2007;22(6):852–63.
Flory J, Emanuel E. Interventions to improve research participants’ understanding in informed consent for research: a systematic review. J Am Med Assoc. 2004;292(13):1593–601.
Price AM, et al. The feasibility and acceptability of a population-level antenatal risk factor survey: cross-sectional pilot study. J Paediatr Child Health. 2017;53(6):572.
Australian Bureau of Statistics. 2033.0.55.001 - Census of Population and Housing: Socio-Economic Indexes for Areas (SEIFA). Australia: Australian Bureau of Statistics; 2011.
Aldana BU, Quintero MA. A comparison of three methods for sampling hard-to-reach or hidden populations. Pensamiento Psicológico. 2008;4(10):167.
Centre for Community Child Health and Telethon Institute for Child Health Research. A Snapshot of Early Childhood Development in Australia. Canberra: Australian Early Development Index (AEDI) National Report; 2012.
Goldfeld S, D'Abaco E, Bryson H, Mensah F, Price AMH. Surveying social adversity in pregnancy: The antenatal risk burden experienced by Australian women. J Paediatr Child Health. 2018;54(7):754–60.
Goldfeld S, Snow P, Eadie P, Munro J, Gold L, Le HND, Orsini F, Shingles B, Lee K, Connell J, Watts, A. Classroom Promotion of Oral Language (CPOL): protocol for a cluster randomised controlled trial of a school-based intervention to improve children's literacy outcomes at grade 3, oral language and mental health. BMJ Open. 2017.
The “right@home” sustained nurse home visiting trial is a research collaboration between the Australian Research Alliance for Children and Youth (ARACY); the Translational Research and Social Innovation (TReSI) Group at Western Sydney University; and the Centre for Community Child Health (CCCH), which is a department of The Royal Children’s Hospital and a research group of Murdoch Children’s Research Institute. We thank all families, the researchers, nurses and social care practitioners working on the right@home trial, the antenatal clinic staff at participating hospitals who helped facilitate the research, and the Expert Reference Group for their guidance in designing the trial.
This work is supported by the state governments of Victoria and Tasmania, the Ian Potter Foundation, Sabemo Trust, Sidney Myer fund, the Vincent Fairfax Family Foundation, and the National Health and Medical Research Council (NHMRC, 1079418). The MCRI administered the research grant for the study and provided infrastructural support to its staff but played no role in the conduct or analysis of the trial. Research at the MCRI is supported by the Victorian Government’s Operational Infrastructure Support Program. SG was supported by an NHMRC Career Development Award (1082922). FM was supported by NHMRC Early Career and Career Development Fellowships (1037449 and 1111160). The funding bodies had no role in relation to the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Ethics approval and consent to participate
right@home was approved by the Human Research Ethics Committees of: The Royal Children’s Hospital (HREC 32296); Peninsula Health (HREC/13/PH/14); Ballarat Health Services (HREC/13/BHSSJOG/9); Southern Health (HREC 13084X); Northern Health (HREC P03/13) in Victoria, Australia; and The University of Tasmania (HREC H0013113), Tasmania, Australia. All participants provided informed consent before providing any data for the study. As described in the Table 1, to help support participants with potential low literacy, researchers offered to go through the recruitment materials (including information statement and consent) verbally with each woman. During recruitment, researchers asked women to reiterate what they were asking to ensure each person understood the request. Women who did not demonstrate comprehension (e.g. required an interpreter or had intellectual disability) were excluded from the trial. Those who did provided signed consent.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Price, A., Bryson, H., Smith, A. et al. Processes for engaging and retaining women who are experiencing adversity in longitudinal health services research. BMC Health Serv Res 19, 833 (2019). https://doi.org/10.1186/s12913-019-4698-5
- Quality improvement
- Health services research
- Follow-up studies