E-IMR: e-health added to face-to-face delivery of Illness Management & Recovery programme for people with severe mental illness, an exploratory clustered randomized controlled trial

Background E-mental health holds promise for people with severe mental illness, but has a limited evidence base. This study explored the effect of e-health added to face-to-face delivery of the Illness Management and Recovery Programme (e-IMR). Method In this multi-centre exploratory cluster randomized controlled trial, seven clusters (n = 60; 41 in intervention group and 19 in control group) were randomly assigned to e-IMR + IMR or IMR only. Outcomes of illness management, self-management, recovery, symptoms, quality of life, and general health were measured at baseline (T0), halfway (T1), and at twelve months (T2). The data were analysed using mixed model for repeated measurements in four models: in 1) we included fixed main effects for time trend and group, in 2) we controlled for confounding effects, in 3) we controlled for interaction effects, and in 4) we performed sub-group analyses within the intervention group. Results Notwithstanding low activity on e-IMR, significant effects were present in model 1 analyses for self-management (p = .01) and recovery (p = .02) at T1, and for general health perception (p = .02) at T2, all in favour of the intervention group. In model 2, the confounding covariate gender explained the effects at T1 and T2, except for self-management. In model 3, the interacting covariate non-completer explained the effects for self-management (p = .03) at T1. In model 4, the sub-group analyses of e-IMR-users versus non-users showed no differences in effect. Conclusion Because of confounding and interaction modifications, effectiveness of e-IMR cannot be concluded. Low use of e-health precludes definite conclusions on its potential efficacy. Low use of e-IMR calls for a thorough process evaluation of the intervention. Trial registration The Dutch Trial Register (NTR4772) Electronic supplementary material The online version of this article (10.1186/s12913-018-3767-5) contains supplementary material, which is available to authorized users.


Background
In spite of the growing interest in e-mental health, evidence for the effectiveness of e-health for people with a severe or serious mental illness (SMI) is limited [1,2]. Naslund et al. [2] found that e-health interventions for people with SMI have high feasibility and acceptability. Van der Krieke et al. [3] found that people with psychotic disorders were able and willing to engage in e-health, and found larger effects for medication management [3]. However, one should be cautious about drawing conclusions regarding the effectiveness [2,3]. E-health is used in a wide range of interventions for people with SMI on (1) illness self-management and relapse prevention, (2) promoting adherence to medications and/or treatment, (3) psycho-education, supporting recovery, and promoting health and wellness, and (4) symptom monitoring [2]. E-health interventions make use of personal digital assistance, medication tracking devices, home monitoring systems, smartphone applications, SMS, and web-based interventions [2].
Also in general mental health, e-health approaches show great potential and offer the possibility of expanding access to care while being economically and socially efficient [4]. But e-health interventions in mental health have high attrition rates [5]. The addition of face-to-face contact to e-health is supposed to increase the therapeutic relation and prevent attrition [6]..In the case of people with SMI, e-health components could be added to an evidence-based face-to-face recovery-oriented intervention. Such an intervention is the Illness Management & Recovery programme (IMR) [7]. The IMR is a standardized curriculum-based approach designed to provide people with SMI the information and skills necessary for managing their illnesses effectively and working towards achieving personal recovery goals. In addition to the standard face-to-face delivery of the IMR, an e-health intervention (e-IMR) was designed which follows the IMR-curriculum, and was further developed with the end-users of the intervention [8]. The aim of this study was to explore the effect of the e-IMR for people with SMI who were referred to the Illness Management & Recovery programme.

Methods
The e-IMR was tested in an exploratory multi-centre cluster randomized controlled trial. According to the Medical Research Council guidance [9], an exploratory trial evaluated an intervention before testing it in a confirmative trial. In this study, a cluster was a subdivision of a mental health institute. The cluster randomization prevented contamination between the intervention and control group participants. Data were collected at baseline, halfway and endpoint. The inclusion period was between January and October 2015. Data collecting lasted until October 2016.
Eligible clusters delivered the IMR-programme as a whole package with an experienced trainer-couple meaning that at least one trainer completed the IMR-total-training organized by the Dutch IMR-network and executed at least the first five modules of the IMR-programme before starting the IMR-programme in the trial.

Trial monitoring
An employee of the 'Radboudumc Technology Center -Clinical Studies' monitored the process of trial administration. The administration of Trial Master Files, both paper as well as computerized files, was independently checked for completeness and accuracy.

Randomization
A statistician generated a randomization schedule using Statistical Analysis System®, version 9.4. The allocation to the intervention or control group was communicated after the participating institutional board provided their consent to participation. Because of the nature of the intervention, blinding was not possible.

Sample size
Because of the exploratory character of this study, a power calculation was considered unnecessary.

Participants
Eligible participants met the following criteria: above 18 years of age; capable of giving informed consent; and meeting the Dutch SMI criteria according to Delespaul [10] (being diagnosed with a psychiatric disorder that causes, and is due to, serious impairments in social and/ or occupational functioning which lasts longer than at least a couple of years and necessitates coordinated multidisciplinary care. Persons who were overwhelmed by disability, including dependence, denial, confusion, anger or despair, were excluded from participating.

Care as usual
All participants, in both the intervention and control group, received care consisting of extensive inpatient and/or outpatient psychiatric treatment including case management. They also received the IMR-programme, which was provided in weekly, 2-h, face-to-face group sessions according to the Dutch version of the IMR 3.0 programme [11] using the hard-copy version of 11 modules.

Intervention
On top of this care as usual, participants in the intervention group had the opportunity to use the e-IMR intervention [8]. The e-IMR intervention started with a 'welcome page' explaining the use of e-IMR and leading participants to the 11 modules. The e-IMR intervention included the same fill-in forms as in the hard-copy version of the IMR-programme. E-IMR added illustrative videos showing peer testimonials to encourage participants to talk more freely about themselves and to take steps in their recovery process. E-IMR also added problem-solving forms at the end of each module, registration of successful coping strategies, and a symptom-monitoring page.
The e-IMR was introduced to the trainers and participants of the intervention group by the first researcher in the second group session. Individuals who did not provide informed consent were allowed to join the e-IMR without participating in the research. The trainer-couples were supported in learning how to support participants in the use of e-IMR; how to install e-IMR on a computer in the session room and how to use e-IMR during the sessions.
In e-IMR, the registration forms of successful coping strategies and the symptom-monitoring page were introduced after the second module 'practical facts about mental illnesses'. Weekly emails with a link to the e-IMR platform led the participants to the symptom-monitoring page. After closing each module, one of the trainers gave feedback to the participants via the platform and guided the participants to the next module.

Data collection
Data were collected in face-to-face interviews by the researcher or a researcher assistant at three time points: at baseline, a week before starting the IMR-programme (T 0 ); halfway, after completing the 5th module (T 1 ); and endpoint, at least a week after finishing the IMRprogramme (T 2 ). The data were recorded on paper and later transferred into a LimeSurvey® [12] database. The original recorded data as well as the transferred were double-checked for accuracy and completeness.

Outcome measures
At baseline, independent demographic and clinical characteristics were recorded. At all three time points, six dependent outcome domains were gathered.
At T 0 the following participant characteristics were collected: age, gender, physical comorbidities, treatment history, cultural background, social economic status, education level, computer/Internet availability and use. At T 0 , the participant's case manager provided their diagnostic classification according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition.
The participants' self-management ability, which refers to the individual's knowledge, skill and confidence for managing his/her own health and healthcare, was measured with the Patients Activation Measure (PAM-13) [18], consisting of 13 items. The response anchors on a five-point scale, vary from not applicable (0), 'strongly disagree' (1) to 'strongly agree' (4). The term 'doctor' in the items five and six was explained as their mental health clinician, which includes a nurse and/or case manager. Raw scores were transformed into standardized activation scores ranging between 0 and 100. The PAM-13's Cronbach's alpha is .84-.88 [19][20][21][22].
The participants estimated the level of burden of symptoms they experienced using the Brief Symptom Inventory (BSI), consisting of 53 items [25]. The response anchors, on a five-point scale, vary from 'not at all' (0) to 'extremely' (4). The mean BSI scores ranged between 0 and 4. A negative time trend for the BSI means a reduced level of burden. The BSI's Cronbach's alpha is .96 [26].
The participants' subjective satisfaction with life was measured with the Manchester Short Assessment of quality of life (MANSA), consisting of 12 items [27]. The response anchors on a seven-point scale vary from 'couldn't be worse' (1) to 'couldn't be better' (7). The mean MANSA score ranged between 1 and 7. The MANSA's Cronbach's alpha is .81 [28].
The participants' general health status was measured with the Rand 36-item Health Survey (Rand-36), consisting of eight subscales: physical functioning (Rand-PF), social functioning (Rand-SF), role limitations due to a physical (Rand-RLPP) and an emotional problem (Rand-RLEP), mental health (Rand-MH), vitality (Rand-V), pain (Rand-P), and general health perception (Rand-GHP) [29]. The response anchors vary between yes/no to Likert scales with three, five, and six options. Raw scores of all the concepts were transformed into scores ranging between 0 and 100. The Cronbach's alpha of Rand-36's eight concepts are .71 and .92 [30].
The extent of participants' activity on the e-IMR platform was determined by counting the number of completed modules and number of log-ins. An e-IMR user is identified by having completed at least module one or having logged in at least five times. Users were regarded as having had the opportunity to benefit from the e-IMR.
As in other studies on IMR [31], participants who attended the face-to-face IMR programme sessions less than 50% were considered to be non-completers. In our study, this resembles stopping the IMR programme before T 1 .

Statistical methods
The Statistical Package for the Social Sciences®.23 [32] was used to carry out the analyses. Mixed model multilevel regression analyses were used to examine the main effects on the outcome measures, taking into account clustering of participants and repeated measures. This method automatically uses the 'missing at random' assumption to handle missing data. Random effects on cluster, trainer-couple, and individual participants nested within the cluster were included in the model. Model 1 included fixed main effects for time trend and group. The analyses were executed according to the intention-to-treat principle to prevent bias caused by the loss of participants [33] and to reflect the normal practice [34] of high attrition rates in treatments of people with SMI [7] and e-health [5].
Post hoc analyses of effect differences were performed to control for covariates. We considered the covariate gender to be a potential confounder because of its known differences in exposure and reactions to stress and health [35]. The covariate was included in model 2, controlling for confounding time trend effects.
In model 3, covariates were included that were expected to interact with the effect differences. Non-completion of the face-to-face IMR-programme sessions was expected to interact with the effects because being a non-completer is correlated to lower functioning; for instance, lower social functioning [36] and higher emergency room visits and hospitalization [37]. In addition, we searched for correlations in T 0 scores between the groups of completers and non-completers.
Because of the known low adherence-rate to Web-based interventions [5], additional subgroup analyses were performed within the intervention group to investigate whether actual use compared to non-use of the e-IMR leads to outcome differences. Thus in model 4, two groups of e-IMR users and non-users were included according to the aforementioned adherence measurement.

Participant flow
Nine institutions with potentially 15 clusters were screened for eligibility. Two clusters were not eligible because they did not deliver the IMR-programme as a whole. Two clusters did not start an IMR-programme group. Four clusters declined because of organizational problems. Seven clusters were included: four were allocated to the intervention group and three to the control group. In three intervention clusters, a second trainer-couple started a second IMR group. So in total, ten IMR-programme groups (seven in the intervention and three in the control group) trained 60 participants: 41 in the intervention and 19 in the control group (see Fig. 1). Table 1 shows baseline characteristics and distribution over the two groups. The characteristics 'gender' and 'inpatients/outpatients' were unequally distributed over the groups, p = .002 and p = .02 respectively.
All ten IMR-programme groups completed the trial. In the intervention group, 12 out of 41 participants were lost in the follow-up measures in the study. We lost five at T 1 and another seven at T 2 . In the control group, four out of 19 participants were lost in the follow-up at T 1 and T 2 . We have missing data at T 1 for one participant in the control group. Participants either refused to be interviewed because of being too burdened by the interviews, or they did not respond to attempts to get in touch with them. Out of the 60 participants, 51 (36 and 15) participants were interviewed at T 1 , and 45 at T 2 (29 and 16) (See Fig. 1).
Out of the total of 60 participants, eighteen (30%) were identified as a non-completer: participants who attended the face-to-face IMR programme sessions less than 50%. Eight participants (20%) in the intervention group and ten participants (58%) in the control group were non-completers, which differed significantly (p = .01). Of these non-completers, 14 participants entered the intention-to-treat analyses, eight in the intervention group and six in the control group at T 1 , and seven in both groups at T 2 .
Out of the 41 participants in the intervention group, 23 (56.1%) logged in on the e-IMR platform, twelve of whom completed the first online module and eight of whom visited the symptom-monitoring page (See Fig. 2). In total, 14 (34.1%) participants were identified as e-IMR users.

Outcomes and estimation
The mean scores and standard deviations of the outcomes in both groups are presented in the Additional file 1. Since the random effect of cluster was zero in nearly all the analyses, this factor was excluded from the analyses models. The relevant results of the mixed model analyses are shown in Table 2. In model 1, the participants in the intervention group scored significantly higher compared to the control group for the measures PAM-13 (p = .01), MHRM (p = .02), and Rand-RLEP (p = .03) at T 1 , which faded at T 2 . At T 2 , the effect on the Rand-GHP was significant (p = .02) in favour of the intervention group.

Post hoc analyses
In model 2, the analyses accounting for the covariate gender showed that the significant effects above could be explained by confounding except for the remaining effect for PAM-13 (p = .01) at T 1 . At T 0 , male participants scored significantly higher on nearly all the measures except for the PAM-13. The same exception occurred in the time trends, but contrarily in favour of female participants.
In model 3, the analyses showed that the interaction of the covariate non-completer was significant for the measures: PAM-13, (p = .03) and Rand-V (p = .03) at T 1 , which faded at T 2 . As an illustration of the interaction, the graphic in Fig. 3 shows the scores for the PAM-13,  which resembles the scores of the Rand-V. We did not find significant correlations in PAM-13 scores at T 0 between the completers and non-completers (p = .77). In model 4, the subgroup analyses within the intervention group between the groups of e-IMR users and non-users showed no significant effect differences at T 1 and T 2 .

Harm
No serious adverse events were reported during the trial.

Discussion
This study shows significant differences in main effects for the parameters self-management (PAM-13), recovery (MHRM), and role limitation due to emotional problems (Rand-RLEP) in favour of the intervention group at T 1 , which faded at T 2 . At T 2 , a significant effect for general health perception (Rand-GHP) occurred, also in favour of the intervention group.
Post hoc analyses showed that the confounder gender explained the effects for recovery and role limitation due to emotional problems at T 1, and for general health perception at T 2 . The confounding effects of gender were based on three types of differences: first, the baseline distribution showed significantly more females in the intervention group; second, at T 0 males scored significantly higher on most of the measures; and third, time trends were in favour of female participants. In general, women do differ from men in a number of ways; for instance, exposure and reactions to stress [35], needs and care [38,39], and coping styles [40]. With regard to coping styles, women could benefit more from a problem-solving-focused intervention and men from an emotion-focused one [41]. IMR, with its emphasis on learning how to manage an illness in a context of pursuing recovery goals [42], has a greater focus on problemsolving-than on emotional strategies. Therefore, women could have benefitted more from the IMR-programme than men.
Post-hoc analyses showed that the confounder gender did not explain the effects for the parameter self-management. Also in studies with people with diabetes II [43] and other chronic illnesses [44], no relations were found between gender and self-management, measured by the PAM-13.
The interaction covariate non-completer significantly modified the effect for the parameter self-management (PAM) and vitality (Rand-V) such that a large intervention effect was seen in the non-completers and a small effect in the completers. Apparently, stopping the IMR-programme was based on differences in their improvements. In this study, improvements in conditions of people who dropped out of the IRM-programme were unequally distributed over the groups, which modified the effects. The unlikeliness of the effects is confirmed by the subgroup analyses within the intervention group which showed no significant effect differences between the groups of e-IMR users and non-users.
A last issue to discuss is the low use of the e-IMR platform by the participants in the intervention group which resulted in a minor contrast in the treatments provided to the participants in the intervention and control group and further calls into question the validity of ascribing the effects observed to the e-IMR. The modest use of the e-IMR matches with 6% of consumers using e-health in general mental health in the year of this study [45]. A number of limitations should be noted. Unfortunately, the planned sample size was not achieved and a lower number of participants entered the control group. This might have caused the unequal distribution of some covariates. Due to the small sample, we could not control for more than one covariate in the mixed models without risking overfitting. Notwithstanding the small sample, a number of non-completers did not withdraw from the study. The overall non-completer rate of 30% is similar to other IMR studies [7]. Therefore, the intention-to-treat analyses resemble normal practice.

Conclusion
Finally, this study precludes definite conclusions on the potential efficacy of e-health for people with SMI. This leaves us with many questions about the barriers and facilitators of the e-IMR intervention and its implementation. Against the backdrop of the great promise of e-mental health [46], the modest use of the e-IMR platform might be an interesting outcome which needs to be further investigated. Before deciding how to continue studying the effectiveness of e-IMR, we will investigate barriers and facilitators of the e-IMR and its implementation.