Processes for engaging and retaining women who are experiencing adversity in longitudinal health services research

Abstract

Background

Women and families experiencing socioeconomic and psychosocial adversity are the least likely to access health care but most likely to benefit. For health services to effectively meet the needs of individuals experiencing adversity, research involving the health services must be truly representative. However, individuals experiencing adversity are typically excluded from or underrepresented in health services research. This paper reports on the implementation of a quality improvement approach designed to support recruitment and retention of pregnant women experiencing adversity in a longitudinal, health services randomized controlled trial (“right@home”).

Methods

right@home recruited Australian women from 10 public maternity hospitals across the states of Victoria and Tasmania who were experiencing adversity (≥2 risk factors on screening survey). Regular follow-up assessments were conducted by phone or face-to-face to child age 2 years. Research processes were designed taking heed of previous research demonstrating effective strategies for recruiting and retaining minority groups (e.g. piloting the recruitment process; recruiting via the health service providing care to the subgroup; remunerating participants); however, we were concerned that important information was missing. Therefore, once recruitment began, we conducted a continuous evaluation of the research processes, testing and implementing changes to processes or new strategies to maximize recruitment and retention (e.g. using a suite of strategies to maintain contact with families, using flexible data collection methods, obtaining consent for data linkage for future health and education data).

Results

right@home enrolled a large cohort of women (N = 722) experiencing high levels of adversity according to socioeconomic status and psychosocial risk factors, and achieved excellent retention (83% completion at 2 years). Most strategies appeared to increase recruitment and retention. All required additional time from the research team to develop and test, and some required extra funding, which ranged from minor (e.g. printing) to substantial (e.g. salaries, remuneration).

Conclusions

By taking a quality improvement approach, supported by sufficient resourcing and flexible research processes, it is possible to recruit and retain a large cohort of women experiencing adversity who are typically missed or lost from longitudinal research.

Background

Recruitment and retention of participants in large-scale, longitudinal, health services research can be complex and challenging, especially when participants are selected for their experience of socioeconomic or psychosocial adversities [1]. Individuals experiencing adversity have the greatest need for health services and supports but are the least able to access them (described as the “inverse care law”) [2]. Reasons for this include cost and an individual’s ability to identify their own needs and seek and obtain services, and these barriers can be compounded by a mistrust of or difficulty relating to the system [3]. Given that individuals experiencing adversity are less likely to interact with health services, researchers can struggle to make contact with eligible individuals in the first place, limiting opportunities for participation.

When it is possible to make contact and invite participation, the artificiality of research design can discourage involvement. Research activities such as providing informed consent, travel and related costs to attend assessments, and finding time to complete assessments, can all be burdensome. Committing to research over years can be difficult for individuals with less stability in their lives or less confidence in, or literacy or familiarity with, health services [1]. However, to determine whether services can be accessed by and improve outcomes for these individuals, health services research must be truly representative of the population being studied. Otherwise, there is the potential for research funding to be wasted on evaluating services that are either inaccessible or ineffective for the individuals most in need of support [1].

In 2012, recognizing the substantial adversity and poor outcomes experienced by some Australian families, the Victorian and Tasmanian governments together with philanthropy funded the “right@home” partnership to develop and evaluate the largest multi-site, multi-state, Australian randomized controlled trial of nurse home visiting, offered from pregnancy to child age 2 years [4]. Designed for delivery via the child and family (CFH) service that is freely available to all Australian families, right@home aimed to improve the learning and development of children born to women selected for their experience of adversity. We designed the research processes for right@home using the strategies identified by previous research for maximizing recruitment and retention with minority groups (such as ethnic/cultural minorities and individuals with reduced socioeconomic resources) [1]. These strategies included identifying the venues or methods that reach the population of interest [5]; using researcher-led instead of clinician-led recruitment to reduce gatekeeping or selection bias [6]; piloting recruitment processes for acceptability and feasibility [1]; collecting informed consent and data in-person to help support individuals with low literacy [7]; and acknowledging the time and cost burdens of research participation by offering remuneration [1].

Although these processes appeared necessary, we were concerned that they may not be sufficient, and therefore aimed to investigate whether a quality improvement approach to implementing and testing the research processes could maximize recruitment and retention of women in the trial. The intention was not to evaluate whether specific processes worked on their own. Rather we aimed to evaluate whether taking a quality improvement approach using multiple, flexible processes could achieve (i) the target sample size (N = 714, see Protocol for calculation [4]) and (ii) prevent loss-to-follow-up as far as possible by attempting to treat every enrolled participant as essential to the trial’s validity.

Methods

The right@home RCT (ISRCTN89962120, https://doi.org/10.1186/ISRCTN89962120) is unique in the Australian research landscape because it prioritized the 5–7% of pregnant women experiencing greatest adversity and was delivered via the existing CFH service. Methods are previously described in the published Protocol [4]. Briefly, researchers recruited pregnant women from the waiting rooms of antenatal clinics in 10 public maternity hospitals across Victoria and Tasmania, inviting them to complete a 10-item brief risk factor (BRF) survey [8]. Women with 2 or more risk factors (plus additional eligibility criteria, see Additional file 1: Table S1) were invited into the trial. Interested and eligible women were visited in their homes by researchers who collected informed consent and conducted a comprehensive baseline questionnaire before random allocation.

The intervention comprised 25 home visits from pregnancy to 2 years, focusing on parent care of the child, responsivity to the child, and providing a good quality home learning environment. The standard CFH service provided the comparator (control), comprising around 6–9 predominately clinic-based appointments. Researchers followed-up with women in both groups via phone interviews (approximately 30 min) when children were 6 weeks, 6 months and 18 months old, and via home-based interviews (approximately 2 h) when children were 1 and 2 years of age. Women’s postcodes were linked with the national Socio-Economic Indexes for Areas Index of Relative Disadvantage (SEIFA); lower scores indicate greater adversity [9].

The research processes for recruitment and retention – the focus of this paper – were designed based on the aforementioned literature plus input from the program developers. For example, we piloted the recruitment processes, [8] which helped us understand how the antenatal clinics ran; build relationships with clinic staff; assess whether the BRF survey was a useful measure for identifying eligibility; gauge women’s interest in participating in the larger RCT; and decide how to remunerate women. Once recruitment began, we implemented a quality improvement approach, defined as a weekly [5] evaluation of how the research processes were supporting or hindering recruitment and retention. For this, informal feedback – termed anecdotal in the Table 1 – was sought from field researchers, participants and other collaborators (e.g. hospital clinic staff), on perceived barriers to recruitment and retention, and strategies to overcome them. Feedback was summarized and discussed at weekly investigator meetings. Based on consensus, the team retained processes that appeared to contribute to increasing the sample size and retention, and discarded those that did not. As the research team were blinded to randomization status until all 2 year data were collected, our processes were applied to both trial arms without differentiation.

Table 1 Barriers anticipated or encountered, strategies implemented, and implications of these strategies

The Table 1 lists the barriers and strategies. Those that were identified in the existing literature and implemented during the design phase are described with terms like “potential or anticipated”; otherwise, the barriers are those encountered during the trial. The Table 1 presents the findings according to 4 of the 5 research phases identified by Bonevski’s systematic review: [1] sampling; recruitment and gaining consent; data collection and measurement; and retention and attrition (intervention delivery is excluded as it was coordinated by a different workforce for the RCT). Where available, the number or proportion of women recruited or retained are used to evaluate the usefulness of strategies. Since assessment points (and therefore the research processes) were repeated multiple times, some processes are evaluated using data for one assessment point (most often the 2 year assessment as this was the primary outcome point and the furthest from recruitment). Data are provided for the complete sample where they were collected as discrete fields in the database and therefore easily exported and analyzed. Where information needed to be identified and extracted from the detailed descriptive participant notes, the data analysis was limited to a random sample of 100 participants selected using Stata (Intercooled Stata, v14·2 for Windows: College Station, TX, USA).

Results

Of the 9511 pregnant women approached at clinics, 6977 were eligible and 5586/6977 (80.1%) completed the BRF survey (see Fig. 1). Of these, 1427 were eligible for the RCT, and the final enrolled cohort comprised 722 women experiencing high levels of adversity (see Additional file 2: Table S2), surpassing the necessary sample size. We maintained excellent retention and completion rates to child age 2 years (see Fig. 1): 90.4% completion and 1.5% cumulative withdrawal at 6 weeks; 91.1%/2.8% at 6 months; 88.2%/4.6% at 1 year; 85.1%/6.4% at 18 months, and 82.5%/8.9% at 2 years. The gaps between completion and cumulative withdrawal reflect the proportions of women who declined or were not contactable.

Fig. 1

Participant flow from recruitment to the 2 year assessment

As described in the Table 1, most strategies appeared to support recruitment and retention. After recruitment started, new strategies required time from the research team to develop, obtain ethical approval (where necessary) and implement. Some changes also required additional funding, which ranged from minor (e.g. printing, postage costs) to substantial (e.g. salaries to extend the recruitment phase, funding participant remuneration). There was no evidence of differences between trial arms in the times families changed addresses or the number of contact attempts made toward completing an assessment. However, it was more common to reach out to the alternate contacts for usual care participants than for intervention participants (see Table 1). This is likely because intervention families had more frequent contact with the research team through their regular visits with intervention nurses, whereas usual care families did not. There was no evidence of differential attrition between trial arms by the 2 year follow-up assessment.

Several contextual factors supported recruitment and retention. We hired researchers who were highly skilled in building relationships with the participating women; provided rigorous researcher training in the project’s standard operating procedures; and employed a research coordinator (alongside the research manager) who was responsible for monitoring recruitment and retention by being the central point of contact for researchers, families, clinics and other contributors (e.g. Centrelink representatives). With this central role, it was possible to implement protocol changes relatively rapidly (dependent on other approvals such as HREC), obtain feedback and evaluate their usefulness.

Discussion

Using a quality improvement approach to evaluate and optimize the research processes for a longitudinal, health services RCT resulted in the recruitment of a large cohort of pregnant Australian women experiencing adversity, and a high proportion of women retained to child age 2 years. Two main factors supported these successful research processes. First, the RCT had sufficient funding for the substantial research costs, which included salaries of skilled researchers employed for sufficient hours to conduct in-person research, plus researcher travel and participant remuneration. Second, we had flexibility as far as possible within a research paradigm to make continual updates and changes to the research protocol, which relied on collaborations with hospital clinic staff and ethics committees to approve and implement protocol amendments quickly.

A consequence of engaging and maintaining a large cohort is that it provides sufficient power to detect small between-group differences. This also allows for ongoing research follow-up beyond 2 years, since the sample remains large enough to meaningfully analyze long-term effects. We are confident that our efforts to obtain consent for data linkage will prove fruitful for long-term follow-up; the CFH data have already been used to conduct a cost evaluation of the intervention. Strategies in this study were implemented together (and are not mutually exclusive). This means we can only report recruitment and retention data corresponding to the relevant time period, rather than as a direct outcome of a specific strategy. For some strategies, we have no supporting data or only anecdotal reports. A formal evaluation of strategies (e.g. using a RCT design rather than a descriptive review) could provide evidence on effectiveness [1]. However, our intention was to evaluate whether a flexible quality improvement process, that implemented multiple strategies, could achieve high recruitment and retention.

Traditional data collection processes with large cohorts, or those experiencing more advantage, use postal or online surveys and minimal reminder processes. Given the number and types of contact needed to retain participants in right@home, traditional process would have failed to adequately engage and retain the cohort. Consistent with our findings, previous studies have demonstrated strong support for person-to-person (e.g. phone or direct) data collection and flexible data collection methods, with mixed evidence for the usefulness of promotional materials and readability [7]. Almost all other studies that have examined the effect of remuneration, including incentives, report that remuneration supports participant engagement [1]. These studies emphasize the need to keep remuneration clear of coercion, yet, remuneration remains controversial even though the costs incurred by participants are well-established. As Swanson noted over 20 years ago, [5] funding agencies need to recognize both the considerable cost of research with minority groups as well as the importance of participant remuneration. Finally, we found that strategies that minimized “barriers to entry”, such as recruiting in clinic waiting rooms, and booking assessments on-the-spot with women or by text messages, as well as all strategies to maintain up-to-date contact details for participants, were important features supporting retention and data collection.

These findings are relevant for community-based research that focuses on minority groups. The findings emphasize the need for increased funding and timeframes for recruitment and retention when compared with research with cohorts experiencing less adversity. Research with individuals experiencing adversity should be considered for the required level of funding by granting bodies when competitive funding is reviewed and awarded [5]. Future studies could evaluate strategies in a more systematic way by obtaining feedback from participants via single or simple questions asking for opinions on processes asked in-person at assessments or by text or email. In addition, it is important to increase the transparency around the true cost of the activities that make up this type of research; for example, how many researcher hours are needed, per participant, to complete enrolment or follow-up assessments.

Conclusion

Research that includes women and families experiencing adversity is crucial for designing health services to meet the needs of those least likely to access a service but most likely to benefit. This is only possible with sufficient funding and flexible research processes. Taking a quality improvement approach and assessing each research process in detail can support high recruitment and retention. By increasing the transparency of research processes, researchers can argue the case for adequately resourcing research to ensure robust estimates of effectiveness. This will ultimately improve the quality and reach of health services for those who need them most.