Skip to main content
  • Research article
  • Open access
  • Published:

IMPlementation of An online Relatives’ Toolkit for psychosis or bipolar (IMPART study): iterative multiple case study to identify key factors impacting on staff uptake and use



Despite the potential of digital health interventions to improve the delivery of psychoeducation to people with mental health problems and their relatives, and substantial investment in their development, there is little evidence of successful implementation into clinical practice. We report the first implementation study of a digital health intervention: Relatives Education And Coping Toolkit (REACT), into routine mental healthcare. Our main aim was to identify critical factors affecting staff uptake and use of this online self-management tool for relatives of people with psychosis or bipolar.


A mixed-methods, theory-driven (Normalisation Process Theory), iterative multiple case study approach using qualitative analysis of interviews with staff and quantitative reporting of uptake. Carer researchers were part of the research team.


In all, 281 staff and 159 relatives from Early Intervention teams across six catchment areas (cases) in England registered on REACT; 129 staff took part in qualitative interviews. Staff were positive about REACT helping services improve support and meet clinical targets. Implementation was hindered by: high staff caseloads and difficulties prioritising carers; perception of REACT implementation as research; technical difficulties using REACT; poor interoperability with trust computer systems and care pathways; lack of access to mobile technology and training; restricted forum populations; staff fears of risk, online trolling, and replacement by technology; and uncertainty around REACT’s long-term availability.


Digital health interventions, such as REACT, should be iteratively developed, evaluated, adapted and implemented, in partnership with the services they aim to support, and as part of a long term national strategy to co-develop integrated technology-enabled mental healthcare. Implementation strategies must instil a sense of ownership for staff and ensure they have adequate IT training, appropriate governance protocols for online working, and adequate mobile technologies. Wider contextual factors including adequate funding for mental health services and prioritisation of carer support, also need to be addressed for successful implementation of carer focussed digital interventions.

Trial registration

Study registration: ISCTRN 16267685.

Peer Review reports

Contributions to the literature

  • This is the first study to examine factors affecting staff uptake and use of a digital health intervention in UK mental health services.

  • Mixed methods were used to understand staff engagement with the Relatives Education And Coping Toolkit (REACT) in Early Intervention teams across six locality based services, purposively sampled for geographical and ethnic diversity in order to maximise generalisability of findings.

  • Findings highlight the need to embed technology development within clinical services; embrace an iterative long-term model of development, testing and adaptation; and provide adequate mobile technology, IT training and digital governance infrastructures to support new ways of working.


Digital Health Interventions (DHIs) are increasingly being developed for people with severe mental health problems including psychosis and bipolar, to improve symptom monitoring [1], medication management [2], and access to information and support [3]. Whilst the evidence base for their effectiveness is less advanced than for other mental health disorders such as depression and anxiety, emerging data suggests that they can be feasible, acceptable, and usable [4,5,6], with preliminary evidence suggesting they can be as effective as more traditional, non–technological self-help interventions [7, 8].

However, evidence for successful implementation of DHIs in routine healthcare services is far more limited. Despite substantial investment, many DHIs are either not adopted by their intended users, are abandoned, fail to scale up locally or spread to other settings, or are not sustained over time [9, 10]. We urgently need to understand the key factors impacting on implementation of DHIs to improve their design, evaluation, commissioning and delivery.

A recent systematic review of 26 studies reporting factors impacting on delivery of DHIs for people with psychosis or bipolar identified the following determinants of uptake: staff and service user attitudes; complexity of the user interface; staff / peer support to use the intervention; fit with existing service IT infrastructures; and costs to development and delivery [11]. The majority of the studies reviewed were American, with only 2 in the UK, both of which were feasibility studies in a research rather than clinical context [12, 13]. Given the importance of context for implementation [14] the generalisability of these findings to routine mental health services is limited. The review found no studies of DHIs to support relatives of people with psychosis or bipolar.

Relatives of people with psychosis or bipolar provide a large amount of unpaid care [15] but at high personal cost (distress and burden) [16,17,18]. The UK National Institute for Health and Care Excellence (NICE) recommends that all relatives be given carer-focused education and support, and offered structured family intervention to enhance family coping and communication [19, 20]. However, audit data show services currently fall well below this target, with only 50% of relatives receiving a carer-focused education and support programme, and 12% a structured Family Intervention [21].

DHIs offer the potential for widespread dissemination of high quality, standardised care, made easily available, alongside a mechanism for uniting people online to share their experiences through peer support. Although DHI development costs can be substantial, ongoing delivery has the potential to be more cost effective in the long-term than face-to-face support and more accessible to those in rural areas and developing countries [22].

The Relatives’ Education And Coping Toolkit (REACT) is a supported self-management toolkit, providing accessible evidence-based information and support for relatives of people with psychosis or bipolar. Following evidence to support feasibility, acceptability, and effectiveness in reducing relatives’ distress as a paper-based tool [23], REACT was developed into an online resource, incorporating a peer support forum, direct messaging, and extended to relatives of people with bipolar [24]. The clinical and cost effectiveness of REACT when offered directly to relatives recruited from outside clinical services is being tested [25]. In this study we examined implementation of REACT online offered to relatives supported by staff working in clinical services in England. This is the first study to examine critical factors affecting staff uptake and use of a digital health intervention in UK mental health services. A full funders report will be available for further detail [26].



We used a theory-driven multiple case study design [27] integrating quantitative uptake and use assessments and qualitative exploration of mechanisms.

Normalisation Process Theory (NPT) was used to guide data collection, analysis and interpretation. NPT focuses on the work done by staff to understand the processes by which a complex healthcare intervention is implemented, embedded, and integrated (or not) into practice [28]. NPT has been extensively applied in eHealth settings [29,30,31] allowing us to compare our findings with previous studies. The protocol was published prior to the end of recruitment [24], and StaRI guidelines [32] used for reporting.


The study took place in Early Intervention (EI) teams across six National Health Service (NHS) trusts (cases) in England. Trusts provide healthcare to particular geographical areas, and within each trust there is an EI team which provides early intervention support to people with early signs of psychosis/bipolar. Cases were purposively sampled for geographical and ethnic diversity to maximise generalisability. To protect anonymity, a multi-layered taxonomy of birds and habitats was used for trusts and clinical teams (see Table 1).

Table 1 Description of features of IMPART cases relevant to study


Staff identified as having specific relevant roles were invited and consented to interviews and /or attending stakeholder groups by the research team. An IMPART lead for each trust facilitated implementation of REACT and assisted with the research.

Description of the intervention

REACT was designed with extensive involvement from relatives [33, 34] and included: 12 psychoeducation modules addressing key questions identified by relatives; peer support through a moderated group forum; a confidential direct messaging service; and a resource directory (RD). All 12 modules contained: evidence-based written information; videos of clinical experts and/or content from experts by experience to illustrate key points; and self-reflection tasks to ensure content was personalised to the user. A “Meet the Team” page ensured that relatives were fully informed about who was delivering the content of the site. ‘Mytoolbox’ offered users a confidential space to save links to any information they might want to access easily later, including specific toolkit content, self-reflection tasks, and external web links. REACT Supporters were members of the EI team, identified by the IMPART Leads, responsible for moderating the forum, responding to messages from users, updating the RD if required, and guiding users to relevant parts of the toolkit. A separate instance of REACT toolkit was set up for each case. Further details and images of the intervention can be found in the protocol paper [24].

Table 2 shows the different roles allocated within each team and their levels of site access.

Table 2 Roles and levels of access for each type of REACT user account

Implementation plan

Local stakeholder groups (SGs) of relatives and staff advised on planning how REACT was introduced in each local context. IMPART leads were encouraged to customise their REACT site by adding a logo, times that the forum/direct messaging services were actively moderated and by whom, emergency contacts, and photos and biographies of the trust’s REACT Supporters and IMPART leads. IMPART leads created REACT Supporter accounts for the staff moderating the forum and direct messaging, and clinician accounts for staff who could invite relatives to REACT by email.

All trusts received pilot versions of REACT including an online manual from 19 September 2016 to 21 November 2016, during which minor edits were made. Each trust had at least one face-to-face training session, after the site went live, providing an overview of the importance of carer information and support; an outline of key components and how to use REACT; and aims of the IMPART study.

Data were collected over 18 months, first in two trusts (wave 1) in which key factors affecting implementation were identified, and in a further two (wave 2) following work done by the research team and stakeholders to design a revised implementation plan. This was repeated for the remaining two trusts (wave 3), with revisions culminating in a final list of recommendations based on understanding of key factors impacting on implementation across all six cases. Order of trusts was determined by pragmatic reasons related to the time taken for trust approvals.

Implementation outcomes

Staff engagement with the REACT toolkit was measured by the number of accounts staff created, and number of invitations sent to relatives over the data collection period. All data were recorded on the REACT website.

Process evaluation

Factors affecting implementation outcomes were explored through individual staff interviews; observations, document analysis; researcher reflective diaries, and Stakeholder Groups (SGs). Purposive sampling of participants was used to identify: IMPART leads (ILs), REACT supporters (RSs), team managers (TMs), and frontline clinical staff. The interview guide [35] was designed to identify factors affecting implementation of REACT. Following written consent, interviews were conducted face to face or by phone, and audio-recorded. Spradley’s [36] nine dimensions of observations was used to develop an observation proforma [35], adapted to test propositions in each context. Trust and national policy documents were sampled to help inform an understanding of case context, and whether REACT fitted into existing pathways and strategies. Reflective diaries [37] were kept by the lead researchers throughout the study and used to develop the researchers’ interpretation of the data as it was collected. SGs took place in each trust prior to the revised implementation plan being delivered, providing useful information about the implementation context, and co-developing subsequent versions of the implementation plan. Attendees in each site included IMPART Leads, service managers, frontline clinical staff, and relatives.


Descriptive summaries of implementation outcomes were calculated using quantitative measures of staff activity. Qualitative data sets collected from each case were analysed using framework analysis [38] supported by NVivo software [39]. Our data were first analysed within each case before we examined similarities and differences between trusts. Data analyses were undertaken in parallel with data collection to encourage iterative testing of emerging themes and inform subsequent implementation activities.

All data were read and re-read by the Research Associates (VA, BG, PO, CM, EL), and synthesised using the four main constructs of NPT [31] with guidance from senior members of the team (FL, NRF, EM). Coding frameworks, and illustrative sections of transcripts were discussed in multi-disciplinary data clinics which included all authors. Here we report the key factors identified as impacting on implementation of REACT, in relation to the following 4 NPT core constructs: coherence; cognitive participation; collective action; and reflexive monitoring.


Evolving implementation plan

In wave 2 the implementation plan included the addition of: REACT booklets and business cards with details of how to access REACT for staff to give to relatives in face-to-face meetings; Staff office reminders with the REACT logo on including mugs and pens; automated email nudges for staff and relatives; a service planner to facilitate teams to allocate staff to key roles supporting REACT; and an auditing dashboard to show staff which relatives had been invited to use REACT. In wave 3, the plan evolved further to include a “Request Access Button” for relatives who wanted to self-refer; staff induction packs for each role; a new “REACT Champion” role to promote REACT; a revised online “how to” manual for staff including a quiz to facilitate engagement; and printable PDF versions of REACT modules to offer as “tasters” for the website.

Implementation outcomes

Table 3 provides summary statistics for implementation outcomes in each case.

Table 3 Implementation outcomes over 18 months

In all, 281 staff registered on REACT and between them, sent 355 invitations to 310 relatives, and 159 relatives registered. The highest number of clinician accounts were created at Lakes Trust (wave 3), least at Ocean (wave 2). Seashore Trust (wave 2) sent the most invitations to relatives, while Lakes Trust sent fewest. A minority of clinicians across all trusts sent invites, with the fewest being in Ocean, and most in Seashore. There was no evidence of an increase in implementation outcomes across the 3 waves, or as a result of the evolving implementation plan, though the timeframe of this analysis may have been too limited to adequately assess this. The aim of this study was to understand the factors determining these numbers, rather than to maximise them.

Process evaluation – key factors impacting on implementation

Table 4 summarises the key factors impacting on implementation, with illustrative quotes from across all six cases.

Table 4 Key factors impacting on implementation of REACT

REACT had reasonable coherence for staff, in that they clearly understood what the toolkit was for, who it was aimed at, and the potential benefits for both staff (skills development, meeting national targets [40], facilitating other family interventions) and relatives (access to standardised information at convenient time and location that empowers them to self-manage). Staff could see how REACT complemented other interventions, such as Behavioural Family Therapy (BFT), offered by the team. However, a key problem was that implementation of REACT was generally seen as research, rather than routine care and consequently may have been perceived as time-limited, and the responsibility of the research team.

…I think if REACT had been introduced as not research but just something we’re going to do from now, that by the way we might evaluate, possibly it might have been seen as more of an infrastructural thing, this is what we will do, but yeah. The kiss of death is kind of true, but people switch off as soon as you go this is a person from X university and they’re going to talk about this project. …” (Seashore, IMPART Lead)

An additional coherence problem was due to high staff turnover, disparately located teams, and preference for face-to-face training. Consequently, while staff understood the general purpose of REACT, many were not familiar with the detailed content and lacked confidence in promoting it to relatives.

I deliver it to all my patients’ families. But I’m not really sure how to deliver it still. …….What it is, you know what do we call it, what is it, is it information kind of site, or I don’t really know how to deliver it…………Possibly, yeah I haven’t had the training and no one’s really told me what it exactly is.” (Moor, Clinician).

Mitigating strategies included delegating the role of inviting relatives to a small number of individuals (as in Marsh), and appointing a REACT champion to remind clinicians about REACT (Seashore, Marsh & Lakes).

Coherence problems tracked through to cognitive participation. Staff agreed REACT could be of value to relatives, but only if delivered as part of a comprehensive care package, including face-to-face support. Opinions differed over who should have ownership over the delivery of REACT, depending on how the services were structured, and how REACT had been introduced. This impacted on buy-in from individual staff.

“It shouldn’t be psychology should it, REACT, it’s just an information sharing but because it is held in psychology people are thinking oh it’s another psychology strategy.” (Seashore, Clinician).

REACT was designed as an adjunct to a comprehensive service. However in trusts struggling to deliver services for relatives, staff feared relatives might become frustrated when they learnt from REACT about services they should have access to, but did not. Across professions, there were concerns about digital interventions replacing face-to-face therapies for psychosis, and particularly for older adults, who staff perceived as less likely to engage with DHIs.

“I suppose it is a slight break with tradition isn’t it, to use an internet-based source, because I suppose it depends how professionals are; our most valuable tool is ourselves and I don’t know whether there’s something that potentially is a bit daunting about handing that over to something else, i.e. technology” (Marsh, Clinician).

Commitment within clinical teams to integrate REACT into clinical practice was limited. REACT was rarely a regular item on any clinical meeting agendas. Tasks were usually allocated, not volunteered for, and training and supervision of key roles including the REACT Supporter role was lacking.

For all trusts, the major challenges with implementation were located in the area of collective action, and how the introduction of REACT was managed through allocation of resources, and execution of protocols, policies, and procedures. Staff in all trusts were under great pressure, with high caseloads, competing priorities and, in some trusts (Woods and Seashore) high staff absence and turnover. All staff were working at (or beyond) capacity, lacking resources to deliver REACT. Staff felt forced to prioritise within priorities, leading to concentration of efforts on measureable outcomes (service user contacts rather than carer contacts), with financial incentives attached. As the Moor Trust IMPART lead put it during one of the SG meetings, “REACT is just not the shark nearest the boat”.

“I think there’s an awful lot of new stuff happening right now, which makes it quite difficult for people to know what they should be focusing on […] you know one week we’re saying this is the absolute priority, and then the next week we’re saying, actually that’s no longer absolute priority” (Woods, Clinician).

It was unclear to researchers or clinical staff who was responsible within trusts for the strategic direction of service development and who could facilitate the shift in priorities needed for staff to embrace novel interventions such as REACT.

A further barrier to action was disjunction between the online nature of REACT (“out of sight and out of mind”), and staffs’ primarily paper-based and community-located ways of working. This was exacerbated by lack of up-to-date mobile technology, and previous negative online experiences including a staff member having been “trolled” (individually named and publically criticised) on a different online forum.

Staff who did try to use REACT, reported good interactional workability, promoting better consultations with relatives. However, some content was felt inconsistent with other aspects of the service (particularly the use of diagnostic terms), and the REACT dashboard used to create relatives’ accounts was not sufficiently user friendly. The need for staff to sign into REACT using a unique username and password also caused difficulties, as staff were required to remember multiple versions of these for different online systems across the NHS. Many staff did not have access to health-service mobile technology and could not show relatives the REACT website during home-based consultations.

“And then we can go out then, I use my own personal phone ‘cos obviously my work phone doesn’t let me go on the internet, or I’ll ask them to get their iPad out and I’ll show them that way.”(Ocean, Clinician).

Although most relatives had personal computers or tablets, staff did not feel it was appropriate to ask relatives to log in to their own desktop computers to be shown the site, particularly as these were often in the bedroom.

Finally, staff allocated to REACT Supporter roles did not feel confident or supported to moderate the online forum or respond to direct messages. Lack of clarity over who had clinical responsibility for REACT and absence of trust policies about managing risk online, resulted in poor relational integration (understanding of accountability and confidence in each other in delivering the practices required) and consequently lack of proactive engagement by REACT Supporters. Fear of being held responsible for risk was widespread across all trusts and in one (Lakes), led to some staff withdrawing support for REACT. No trusts seemed able to reassure staff or produce risk policies adapted for online communication.

“……it’s absolutely no chance I’m going to be putting my registration at risk to look over something that’s,…. but for the confidence, the security, the even if everybody.. somebody just says I need help right now doesn’t get it within 20 minutes, a stereotype saying [?] off the bridge and then well they put a cry for help knowing REACT Champion was on duty and he didn’t see it.” (Lakes, REACT Champion).

Where staff overcame these barriers and invited relatives, they were frustrated by the lack of feedback (reflexive monitoring) available from relatives about the site. Where staff sought feedback directly from relatives, this was generally positive about the module content, particularly hearing the stories of others. However, relatives were disappointed by the lack of forum activity. This was partly due the small population of relatives using the site, so no-one wanted to be the first; and partly due to lack of staff promotion of forum activity. As staff became aware of the lack of forum activity, they became less motivated to invite relatives, creating a vicious cycle.

“I think it’s just not seeing any kind of outcomes coming from registering people and it looks like there’s no activity on the site, I guess I mean people might be logging on, the few that have actually registered, but I guess there’s no way of monitoring it, no way of knowing that people are actually finding it helpful or useful” (Woods, REACT Supporter)


This is the first study that examined in detail the process of implementation of a DHI aimed at supporting relatives of people with psychosis or bipolar within national publicly funded mental health services. Staff engagement with REACT was facilitated by a good fit between the rationale and design of the toolkit, and the need for them to deliver support to carers as part of audited national clinical guidelines. The toolkit was easy to integrate alongside other services currently being offered, and staff could easily see the benefits of this approach in terms of accessibility to high quality information and support for relatives and staff. Positive feedback from relatives was a strong motivator of engagement. Barriers to implementation included high staff caseloads and difficulties prioritising supporting relatives; technical difficulties using REACT; poor interoperability with trust IT systems and care pathways; lack of access to mobile technology and IT training; restricted forum populations leading to low levels of use; staff fears of managing risk, online trolling, or replacement by technology; and uncertainty around REACT’s long-term availability.

Some of these factors are consistent with the findings of previous research into implementation of DHIs into physical healthcare settings [29, 41,42,43] and more recently in a review of studies in mental health settings [11]. We additionally identified three key factors in relation to DHIs in mental health care settings, with clear clinical implications.

The first is the significant impact of wider social context around mental health. Despite government initiatives to achieve parity of esteem for mental and physical health [44], mental health services in this study were chronically underfunded, staff morale was low, and there was high staff turnover and absences. Staff were managing many competing priorities, and supporting relatives was not activity that was systematically recorded. Successful integration of DHIs into clinical practice requires significant engagement of time and effort from staff across all levels of the organisation. This demands an adequately funded service with the capacity to flex sufficiently to accommodate changes in practice.

The second is that digital confidence, competence, governance, and access to equipment were limited, suggesting that implementation of any new DHI would have been challenging. Managing risk was a high clinical priority, but risk management policies did not include online activity, leaving staff unclear of their responsibilities. Investment is needed to ensure staff have access to the mobile hardware they need, integrated IT platforms that support single login, and digital skills training. Governance issues around digital risk and responsibility urgently need to be addressed with clear staff policies.

A third key barrier was staff perception that REACT was a research study, rather than a clinical initiative, despite extensive attempts to explain that the implementation focus of the work. This perception was compounded by the fact that the decision to adopt REACT was often made first by research leads not clinical teams. This impeded staff uptake, and was further compounded by lack of long-term funding for REACT, meaning there was no guarantee it would be available after the study. DHIs, such as REACT, need to be co-developed and iteratively evaluated, adapted and refined with extensive staff and service user input, as part of a long-term and resourced strategy to develop fully integrated technology-enabled services [45]. The current model of develop, test, implement, can lead to promising technologies being abandoned early, as staff will quickly disengage from externally delivered technology that does not immediately work for them. Integrated and embedded technology development in clinical services will also ensure that investment in DHI development is targeted at clinical need, driven by “pull” from staff and service users, rather than “push” from technology developers.

Based on this study, we have outlined generalizable recommendations for successful implementation of DHIs, using REACT as an example (see Table 5).

Table 5 Recommendations for implementing digital health interventions

Study strengths and limitations

The design was collaborative and informed by perspectives of all relevant stakeholders. Relatives and clinical staff formed part of our study team and contributed to design, data collection, and analysis. Stakeholder Groups were set up at each NHS trust, and played a key role in providing data and creating recommendations. Generalisability of findings was enhanced through use of multiple data sources, collected across multiple real world cases, sampled for diversity, and data analysis was strongly embedded in implementation theory. NPT proved a useful framework to identify important factors impacting on the day to day work done by staff to implement REACT.

The main limitations were the relatively short timeframe (18 months) of data collection, restricting focus to the early stages of implementation and precluding an understanding of embedding, integration, or the impact of the evolving implementation plan; and the dual role of the research team in developing REACT whilst also collecting data to understand the process of implementation. Further research needs to test the generalisability of the factors identified as impacting on implementation of REACT in mental health services, to other DHIs and within other healthcare systems, and the effectiveness of the recommendations. Further research could also explore how individual differences among staff impact on levels of engagement, which has not been addressed here. This study has focussed only on uptake and use by staff, but equally important are the factors impacting on uptake and use by relatives who were offered REACT, and these will be reported elsewhere. There are many alternative theoretical frameworks that could have been used and of particular interest is the more recently proposed non-adoption, abandonment, scale-up, spread, and sustainability (NASSS) framework which has been specifically developed for DHIs [46]. NASSS was not available at the start of this study but given the specificity of focus on implementation of DHIs, and the significant role that the wider context played implementing REACT, this may offer a useful framework for future work in this area.


In the first implementation study of a DHI (in this study REACT) in the UK NHS mental health services, we identified many factors across staff coherence, cognitive participation, collective action and reflexive monitoring, that impact on the success of implementation and which are likely to be relevant to other DHIs in mental health. Some of these factors could be addressed by facilitating a model of DHI development that is strongly embedded in services, prioritises long-term intervention evolution and change, and manages staff expectations around uptake and effectiveness. However, wider contextual factors including inadequate funding for mental health services; lack of prioritisation of working with carers, and dissociation between research and clinical practice also need to be addressed.

Availability of data and materials

Extensive anonymised quotes from data collected during the study have been included in the paper and more detailed summaries of each case will be published as part of the study funders report. Access to further available anonymised data may be granted following review and if appropriate agreements are in place. Please contact corresponding author.



Behavioural Family Therapy


Digital Health Interventions


Early Intervention




Implementation of an online Relatives’ Toolkit


Information Technology


Nonadoption, abandonment, scale-up, spread, and sustainability


National Health Service


National Institute for Health And Care Excellence


Normalisation Process Theory


Relatives’ Education And coping Toolkit


REACT Supporter


Stakeholder Groups


Standards for Reporting Implementation Studies


United Kingdom


  1. Ben-Zeev D, Brenner CJ, Begale M, Duffecy J, Mohr DC, Mueser KT. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophr Bull. 2014;40(6):1244–53.

    Article  Google Scholar 

  2. van der Krieke L, Wunderink L, Emerencia AC, de Jonge P, Sytema S. E-mental health self-management for psychotic disorders: state of the art and future perspectives. Psychiatr Serv. 2014;65(1):33–49.

    Article  Google Scholar 

  3. Rotondi AJ, Anderson CM, Haas GL, Eack SM, Spring MB, Ganguli R, et al. Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr Serv. 2010;61(11):1099–105.

    Article  Google Scholar 

  4. Alvarez-Jimenez M, Alcazar-Corcoles MA, Gonzalez-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res. 2014;156(1):96–106.

    Article  CAS  Google Scholar 

  5. Batra S, Baker RA, Wang T, Forma F, DiBiasi F, Peters-Strickland T. Digital health technology for use in patients with serious mental illness: a systematic review of the literature. Med Devices (Auckl). 2017;10:237–51.

    Google Scholar 

  6. Firth J, Torous J. Smartphone apps for schizophrenia: a systematic review. JMIR Mhealth Uhealth. 2015;3(4):e102.

    Article  Google Scholar 

  7. Naslund JA, Marsch LA, McHugo GJ, Bartels SJ. Emerging mHealth and eHealth interventions for serious mental illness: a review of the literature. J Ment Health. 2015;24(5):321–32.

    Article  Google Scholar 

  8. O'Hanlon P, Aref-Adib G, Fonseca A, Lloyd-Evans B, Osborn D, Johnson S. Tomorrows world: current developments in the therapeutic use of technology for psychosis. BJPscyh Advances. 2016;22(5):301–10.

    Article  Google Scholar 

  9. Bennion MR, Hardy G, Moore RK, Millings A. E-therapies in England for stress, anxiety or depression: what is being used in the NHS? A survey of mental health services. BMJ Open. 2017;7(1):e014844.

    Article  CAS  Google Scholar 

  10. So M, Yamaguchi S, Hashimoto S, Sado M, Furukawa TA, McCrone P. Is computerised CBT really helpful for adult depression?-a meta-analytic re-evaluation of CCBT for adult depression in terms of clinical implementation and methodological validity. BMC Psychiatry. 2013;13(1):113.

    Article  Google Scholar 

  11. Aref-Adib G, McCloud T, Ross J, O'Hanlon P, Appleton V, Rowe S, et al. Factors affecting implementation of digital health interventions for people with psychosis or bipolar disorder, and their family and friends: a systematic review. Lancet Psychiatry. 2019;6(3):257–66.

    Article  Google Scholar 

  12. Lobban F, Dodd AL, Sawczuk AP, Asar O, Dagnan D, Diggle PJ, et al. Assessing feasibility and acceptability of web-based enhanced relapse prevention for bipolar disorder (ERPonline): a randomized controlled trial. J Med Internet Res. 2017;19(3):e85.

    Article  Google Scholar 

  13. Poole R, Simpson SA, Smith DJ. Internet-based psychoeducation for bipolar disorder: a qualitative analysis of feasibility, acceptability and impact. BMC Psychiatry. 2012;12:139.

    Article  Google Scholar 

  14. May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(1):141.

    Article  Google Scholar 

  15. Andrew A, Knapp M, McCrone P, Parsonage M, Trachtenberg M. Effective interventions in schizophrenia- the economic case. A report prepared for the Schizophrenia Commission. London: Rethink Mental Illness; 2012.

    Google Scholar 

  16. Winefield HR, Harvey EJ. Determinants of psychological distress in relatives of people with chronic schizophrenia. Schizophr Bull. 1993;19(3):619.

    Article  CAS  Google Scholar 

  17. Barrowclough C, Tarrier N, Johnston M. Distress, expressed emotion, and attributions in relatives of schizophrenia patients. Schizophr Bull. 1996;22(4):691.

    Article  CAS  Google Scholar 

  18. Lowyck B, De Hert M, Peeters E, Wampers M, Gilis P, Peuskens J. A study of the family burden of 150 family members of schizophrenic patients. European Psychiatry. 2004;19(7):395–401.

    Article  CAS  Google Scholar 

  19. National Institute for Health and Clinical Excellence (NICE). Psychosis and schizophrenia in adults (CG178). London: National Institute for Health and Clinical Excellence; 2014 February 2014. Report No.: CG178.

  20. National Institute for Health and Care Excellence (NICE). Bipolar Disorder: the assessment and management of bipolar disorder in adults, children and young poeple in primary and secondary care. London: NICE Clinical Guideline; 2014.

  21. Healthcare Quality Improvement Partnership (HQIP). Royal College of Psychiatrists. Report of the Early Intervention in Psychosis audit. London: NHS England; 2016.

    Google Scholar 

  22. Aranda-Jan CB, Mohutsiwa-Dibe N, Loukanova S. Systematic review on what works, what does not work and why of implementation of mobile health (mHealth) projects in Africa. BMC Public Health. 2014;14:188.

    Article  Google Scholar 

  23. Lobban F, Glentworth D, Chapman L, Wainwright L, Postlethwaite A, Dunn G, et al. Feasibility of a supported self-management intervention for relatives of people with recent-onset psychosis: REACT study. Br J Psychiatry. 2013;203(5):366–72.

    Article  Google Scholar 

  24. Lobban F, Appleton V, Appelbe D, Barraclough J, Bowland J, Fisher NR, et al. IMPlementation of a Relatives' toolkit (IMPART study): an iterative case study to identify key factors impacting on the implementation of a web-based supported self-management intervention for relatives of people with psychosis or bipolar experiences in a National Health Service: a study protocol. Implement Sci. 2017;12(1):152.

    Article  Google Scholar 

  25. Lobban F, Robinson H, Appelbe D, Barraclough J, Bedson E, Collinge L, et al. Protocol for an online randomised controlled trial to evaluate the clinical and cost-effectiveness of a peer-supported self-management intervention for relatives of people with psychosis or bipolar disorder: relatives education and coping toolkit (REACT). BMJ Open. 2017;7(7):e016965.

    Article  Google Scholar 

  26. Lobban F, Appelbe D, Appleton V, Arref-Adib G, Barraclough J, Billsborough J, et al. IMPlementation of an online relatives’ toolkit for relatives of people with psychosis or bipolar experiences: the IMPART multiple case study. Health Serv Deliv Res. in press.

  27. Yin RK. Case study research : design and methods. 5th ed. Los Angeles: SAGE; 2014.

    Google Scholar 

  28. May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4:29.

    Article  Google Scholar 

  29. Mair FS, May C, O’Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ. 2012;90:357–64.

    Article  Google Scholar 

  30. Murray E, Burns J, See TS, Lai R, Nazareth I. Interactive health communication applications for people with chronic disease. Cochrane Database Syst Rev. 2005;4.

  31. Murray E, Treweek S, Pope C, MacFarlane A, Ballini L, Dowrick C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med. 2010;8.

  32. Pinnock H, Barwick M, Carpenter CR, Eldridge S, Grandes G, Griffiths CJ, et al. Standards for reporting implementation studies (StaRI) statement. Bmj. 2017;356:i6795.

    Article  Google Scholar 

  33. Lobban F, Glentworth D, Haddock G, Wainwright L, Clancy A, Bentley R. The views of relatives of young people with psychosis on how to design a relatives education and coping toolkit (REACT). J Ment Health. 2011;20(6):567–79.

    Article  Google Scholar 

  34. Honary M, Fisher NR, McNaney R, Lobban F. A web-based intervention for relatives of people experiencing psychosis or bipolar disorder: design study using a user-centered approach. JMIR mental health. 2018;5(4):e11473.

    Article  Google Scholar 

  35. IMPART - Other Resources: Lancaster University; 2019 [Available from:

  36. Spradley JP, McCurdy DW. Anthropology, the cultural perspective. New York: Wiley; 1980.

  37. Schön DA. The reflective practitioner: how professionals think in action. London: Routledge; 2017.

  38. Ritchie J, Lewis J, Nicholls CM, Ormston R. Qualitative research practice: a guide for social science students and researchers: Sage; 2013.

  39. NVivo qualitative data analysis software. Version 11 ed: QSR International Pty Ltd; 2015.

  40. NHS England. Guidance to support the introduction of access and waiting time standards for mental health services. London: NHS England; 2015.

    Google Scholar 

  41. Brewster L, Mountain G, Wessels B, Kelly C, Hawley M. Factors affecting front line staff acceptance of telehealth technologies: a mixed-method systematic review. J Adv Nurs. 2014;70(1):21–33.

    Article  Google Scholar 

  42. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016;11(1):146.

    Article  Google Scholar 

  43. Huryk LA. Factors influencing nurses’ attitudes towards healthcare information technology. J Nurs Manag. 2010;18(5):606–12.

    Article  Google Scholar 

  44. Department of Health. No health without mental health: A cross-government mental health outcomes strategy for people of all ages. London: Department of Health; 2011.

  45. Mohr DC, Lyon AR, Lattie EG, Reddy M, Schueller SM. Accelerating digital mental health research from early design and creation to successful implementation and sustainment. J Med Internet Res. 2017;19(5).

  46. Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, Hinder S, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367.

    Article  Google Scholar 

Download references


The IMPART study was supported by the Local Clinical Research Network (LCRN) including research support costs. We thank all the staff and relatives at the six participating NHS trusts and the members of the study steering committee chaired by David Kingdon, Professor of Mental Health Care Delivery within Medicine at the University of Southampton. We also thank Anthony Greenwood, for support with data collection or administrative tasks at Lancaster University.


This study was funded by the National Institute of Health Research (NIHR), Health Services and Delivery Research 14/04/16. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health. The IMPART study was supported by the Clinical Research Network (CRN) including research support costs.

Author information

Authors and Affiliations



FL was the Chief Investigator and SJ, SHJ, EM, VP, JRM, RS, JS, CJS co-investigators, and all contributed to design, and delivery of the study. NRF led on design and iterative development of the implementation plan. VA, BG, EL, BM, CM, PO’H, collected all of the data. DA and AW were responsible for IT design and management for data collection and delivery of REACT. JB, SF co-facilitated SG groups, and contributed to study design and delivery, drawing on their lived experience as carers. DG, CH, TZK, PV supported delivery of the study in the NHS. All authors contributed to analysis and write up of the data. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Fiona Lobban.

Ethics declarations

Ethics approval and consent to participate

Ethics approval for the IMPART study was granted by the Health Research Authority and the East of England–Cambridge South Research Ethics Committee (16/EE/0022). All EI staff in each of the participating trusts were made aware of the study, and organisational research and development (R&D) approval was given for the research team to monitor the use of REACT and be present in the trust. All participants (staff and relatives) who provided additional individualised data (interviews, workshops, relatives’ questionnaires) provided informed written consent prior to participating in the study.

Consent for publication

Not applicable.

Competing interests

Some of the authors (including Lobban) were also the originators of REACT and therefore are not independent researchers. Jo Rycroft-Malone is Director NIHR Health Service & Delivery Research Programme, which funded this research, and sits on the following HS&DR committees: POM Remit & Competitiveness Check Commissioned Workstream; Remit & Competitiveness Check Researcher-Led Workstream; PLG; NHS 111 Online Sub Board; Funding Committee Members.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lobban, F., Appelbe, D., Appleton, V. et al. IMPlementation of An online Relatives’ Toolkit for psychosis or bipolar (IMPART study): iterative multiple case study to identify key factors impacting on staff uptake and use. BMC Health Serv Res 20, 219 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: