Implementation strategies for telemental health: a systematic review

Background The COVID-19 pandemic resulted in a rapid shift from traditional face-to-face care provision towards delivering mental health care remotely through telecommunications, often referred to as telemental health care. However, the manner and extent of telemental health implementation have varied considerably across settings and areas, and substantial barriers are encountered. There is, therefore, a need to identify what works best for service users and staff and establish the key mechanisms for efficient integration into routine care. Objective We aimed to identify investigations of pre-planned strategies reported in the literature intended to achieve or improve effective and sustained implementation of telemental health approaches (including video calls, telephone calls, text messaging platforms or a combination of any of these approaches with face-to-face care), and to evaluate how different strategies influence implementation outcomes. Methods A systematic review was conducted, with five databases searched for any relevant literature published between January 2010 and July 2021. Studies were eligible if they took place in specialist mental health services and focused on pre-planned strategies to achieve or improve the delivery of mental health care through remote communication between mental health professionals or between mental health professionals and service users, family members, unpaid carers, or peer supporters. All included studies were quality-assessed. Data were synthesised using the Expert Recommendations for Implementing Change (ERIC) compilation of implementation strategies and the taxonomy of implementation outcomes. Results A total of 14 studies were identified as meeting the inclusion criteria from a total of 14,294 records of which 338 were assessed at full text. All ERIC implementation strategies were used by at least one study, the most commonly reported being ‘Train and educate stakeholders’. All studies reported using a combination of several implementation strategies, with the mean number of strategies used per study of 3.5 (range 2–6), many of which were reported to result in an improvement in implementation over time. Few studies specifically investigated a single implementation strategy and its associated outcomes, making conclusions regarding the most beneficial strategy difficult to draw. Conclusions Using a combination of implementation strategies appears to be a helpful method of supporting the implementation of telemental health. Further research is needed to test the impact of specific implementation strategies on implementation outcomes. Supplementary Information The online version contains supplementary material available at 10.1186/s12913-022-08993-1.

Conclusions Using a combination of implementation strategies appears to be a helpful method of supporting the implementation of telemental health. Further research is needed to test the impact of specific implementation strategies on implementation outcomes. Keywords Telemental health, Implementation, Mental health, Systematic review, Remote care, Telemedicine, Telepsychiatry

Background
Telemental health (TMH) refers to delivering mental health care remotely via telecommunications technology (as opposed to face-to-face) [1]. Adoption of TMH has expanded during the COVID-19 pandemic to allow services to continue to offer mental health support despite social distancing restrictions. This shift in care delivery was often conducted rapidly as part of the emergency response to the pandemic, in which services had to adapt their existing face-to-face models of treatment to include remote forms of care [2].
Some benefits to delivering mental health support in this way have been identified, for example, increasing access for service users who live remotely, have difficulty travelling, or find mental health care settings stigmatising or intimidating, and greater convenience for some service users [3,4]. However, there are also some challenges associated with this approach, for example, some service users may not have access to technology, internet connectivity, or a private space to use during TMH care, whilst others have identified challenges in developing and maintaining a therapeutic relationship [5,6]. A recent systematic review also identified that TMH may not be suitable for all types of therapy, for example, exposure therapy or when treating trauma [2]. This review also identified challenges in delivering TMH care to certain populations, for example, children and service users with learning difficulties or severe anxiety [2].
The rapid switch to TMH during the pandemic has resulted in great variations in how and to what extent TMH has been adopted and sustained, across different geographical locations and services [7]. Due to the rapid nature of the implementation of TMH, staff have raised concerns around a lack of appropriate training to be able to conduct remote mental health care effectively and safely [6,8]. Nonetheless, both staff and service users express interest in incorporating TMH in routine care beyond the pandemic, increasing service user and staff choice and convenience. To move beyond the piecemeal pandemic implementation of TMH to strategies for incorporating it in routine care in the future, we need a greater understanding of the best approaches to introducing and sustaining it in contexts where it is potentially helpful. Implementation research can be defined as "the scientific inquiry into questions concerning implementationthe act of carrying an intention into effect, which in health research can be policies, programmes, or individual practices" [9]. This review will focus on implementation strategies, which represent "the 'how to' component of changing healthcare practice" [10] and are key in determining the success of an intervention [10]. Implementation strategies are the methods by which changes to existing healthcare practice are introduced and sustained in practice. There is a need to establish what works for whom in TMH and identify the key mechanisms for acceptable, effective, and efficient integration of this intervention into routine care. Studies using implementation science methods are especially focused on meeting this need, and therefore efforts to synthesise available information on implementation efforts form a key part of bridging the gap between what is known about effective treatment and what can feasibly be utilised within mental health settings [10].
The current review aimed to synthesise evidence on how best to implement and sustain TMH during the recovery from the pandemic and beyond, integrating it across the mental health system in a flexible and sustainable way that both maximises its potential in everyday practice and allows a response to be rapidly mobilised to any future emergency.
This review specifically aimed to: 1) identify and describe strategies that have been used to improve the implementation of TMH approaches; 2) synthesise evidence on how these strategies influence implementation outcomes.

Methods
This systematic review is reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidance [11]. The review was prospectively registered on PROSPERO (CRD42021266245).

Inclusion criteria
We included studies meeting the following criteria: Participants: Staff who worked within any specialist mental health service, including inpatient, outpatient, community and crisis care settings; people of any age who received organised mental health care in specialist mental health services, or their family members or carers.
Studies conducted in primary care or standalone psychotherapy service settings, or that involved service users with substance misuse, neuropsychiatry/neurology or dementia diagnoses were excluded.
Interventions: Pre-planned strategies only (strategies which comprise deliberate and purposeful efforts, planned in advance, to support the effective and sustained implementation of TMH [10]). We included various modalities of TMH, including video calls, telephone calls, text messaging platforms and hybrid approaches combining different platforms, or a combination of remote and face-to-face care. TMH care must have included spoken or written communication carried out remotely between mental health professionals or between mental health staff and patients, service users, family members, unpaid carers, or peer supporters.
Studies where the intervention was only delivered to selected participants recruited for the purpose of the study, as opposed to being rolled out across an existing service, were excluded.
Outcomes: At least one of the outcomes from Proctor and colleagues' [12] taxonomy of implementation outcomes, defined for the purposes of this review as the effects of deliberate and purposive actions to implement TMH (see Evidence synthesis for more detail), had to be reported: • Acceptability (to service users or staff ) • Adoption (including any individual differences in those reached or not reached) • Appropriateness • Feasibility, e.g. actual fit, suitability for use • Fidelity • Cost and cost effectiveness (of implementation support intervention or strategy) • Penetration, e.g. spread, level of institutionalisation • Sustainability Studies with or without a comparator were included. Studies were excluded which reported findings about the extent of implementation of a TMH programme, or described barriers to or facilitators of the implementation of TMH, but did not describe and evaluate an explicit pre-planned strategy designed to achieve more widespread, effective and/or sustained implementation of TMH, or did not report a relevant outcome according to Proctor's taxonomy.
Study designs: There were no restrictions based on study design or language of papers.

Further exclusion criteria
We also excluded conference abstracts, review articles, editorials and opinion pieces. Papers were excluded if they were published before January 2010 as earlier studies may be less relevant due to changes in both the availability of and familiarity with technology that can be used to support telehealth.

Search strategy
The search strategy included the following: The full search strategy is provided in Additional file 1: Appendix 1.

Screening
All references were de-duplicated in Endnote X9 [13] and then imported into Rayyan [14] for title and abstract screening. Title and abstract screening was conducted by five reviewers (RA, PB, SS, ER, MT), with 100% included references and 25% of excluded references checked by another member of the research team (PB, MY, MW) to ensure inclusion criteria had been applied correctly. Full text screening was conducted by four reviewers (PB, RA, ET, NL), with 100% included references and 25% of excluded references checked by another member of the research team to assess if they agreed with the original decision. All disagreements were resolved by discussion with a third reviewer.

Data extraction
All included references were imported into EPPI-Reviewer 4.0 [15] for data extraction. A data extraction form was created and piloted on a small number of included studies by three reviewers (RA, NSJ, PB), before data for the remaining studies was extracted by three reviewers (NL, ER, ET). All data extraction was checked by another member of the research team (PB, NSJ, RA, JP).
Details on the service setting, study design, characteristics of the clinical population, characteristics of the staff and TMH modalities used were extracted from each study. Details of the implementation strategy used, categorised according to the ERIC compilation of implementation strategies [16] (outlined in more detail below) and implementation outcomes (categorised according to Proctor's taxonomy [12]) were also extracted.

Quality appraisal
Quality appraisal was conducted using the Mixed Methods Appraisal Tool (MMAT) [17] for all primary research studies which aimed to answer a research question. MMAT recommendations are to report the percentage of criteria met for each study (e.g. 20% represents 1/5 criteria met, while 80% represents 4/5 criteria met). Studies were considered of low quality if they met 20% or less, medium quality if they met 40-60% of the criteria, and high quality if they met at least 80% of the criteria. The AACODS (Authority, accuracy, coverage, objectivity, date, significance) tool for descriptive studies [18] was used to appraise the quality of studies which did not meet the MMAT criteria of having a clear research question. We describe these studies as "descriptive studies". Quality assessment was carried out by one member of the research team (NL, ER, ET, or RA) and checked by another reviewer (RA, PB, or NVSJ). Due to the relatively small number of papers included in this review, the results of quality assessment were not used to determine eligibility for inclusion, although quality assessment along with study design were considered during interpretation of findings.

Evidence synthesis
We conducted a framework synthesis [19] to consolidate findings from the included studies, a method which "allows themes or concepts identified a priori to be specified as coding categories from the outset" (p1). We used two pre-established implementation science frameworks as the basis for our framework synthesis, and mapped data from included papers according to headings from these two frameworks. Firstly, we used the overarching categories from the ERIC compilation of implementation strategies [16], to identify and record strategies used to implement TMH (see Table 1 for further details). Secondly, outcomes of studies were categorised according to the Proctor taxonomy of implementation outcomes [12], which consists of the following: acceptability (stakeholders' perception that the intervention is agreeable), adoption (uptake of the intervention), appropriateness (the perceived compatibility of the intervention), feasibility (the extent a new intervention can be used in a particular setting), fidelity (whether an intervention was implemented as originally prescribed), implementation cost (the cost effect of implementing the intervention), penetration (the integration of the intervention within a service) and sustainability (the extent to which a new intervention is maintained). Details of outcomes and interventions from the framework were used to create summaries of the strategies employed and resulting outcomes in each study. As per recommendations for the use of these outcomes [12], we recorded within the taxonomy both those outcomes reported as resulting from the preplanned strategies intended to optimise TMH, and outcomes relating to the TMH interventions themselves (as a result of implementation strategies) where reported by the included studies. For example, we recorded acceptability of training reported by clinicians as well as acceptability of the TMH interventions reported by service users, which may have been impacted by the implementation strategy of 'train clinicians' .

Study selection
Database searches identified 20,858 papers, of which 14,294 were screened by their title and abstract once duplicates had been removed. A total of 338 papers were screened at full text, resulting in 14 studies identified for inclusion in the review. No additional papers were identified from preprint servers or from forward or backward citation searching. The study selection and screening process is summarised in Fig. 1.

Study characteristics
Of the 14 included studies, 11 were conducted in the USA, with one each in the UK, Canada, and Australia. Six studies focused only on service users, seven only on staff, and one involved both staff and service users as participants. Four studies used quantitative methods, four were mixed methods studies, two were qualitative studies and four studies were descriptive in nature (did not aim to address a specific research question and instead provided a description of actions and their outcomes). No trials or studies with a comparison group were identified.
The majority (n = 11) of studies explored implementation in community mental health services, and five studies used an established implementation framework to inform their analysis. Further details of included studies can be found in Table 2.

Quality of included studies
MMAT [17] quality appraisal was conducted for the 10 primary studies which aimed to answer a specific research question (as opposed to purely describing the implementation of TMH). Of these, two studies met 20% of criteria (low quality), three studies met 40% of criteria (medium quality), three studies met 60% or criteria (medium quality), and one study each met 80% and 100% of criteria (high quality). Four descriptive studies were appraised using the AACODS checklist [18], and as all studies met all (3/4) or all but one (1/4) criteria, we considered these studies as appropriate descriptive studies but considered primary studies as more important. A full breakdown of the results of the quality assessment is provided in Additional file 2: Appendix 2.

Evidence synthesis
Each type of implementation strategy in ERIC's taxonomy was reported in at least one study. The most commonly used strategy was 'Train and educate stakeholders' , which was identified in nine studies, whilst the least used was 'Utilise financial strategies' , which was only reported in one study [24]. The mean number of strategies used per study was 3.5, while the most common numbers of strategies used per study was 2 or 3. The implementation strategies reported by each study can be found in Table 3.
Most of the implementation strategies were reported as being associated with good outcomes for TMH implementation, such as improved knowledge for clinicians regarding how to conduct care via TMH, increased acceptability or adoption. However, some barriers to TMH implementation remained, including a lack of staff time, higher administrative burden, or a preference for face-to-face appointments (amongst either staff or service users).
While most studies reported implementation outcomes only in relation to the TMH intervention itself, five studies reported implementation outcomes resulting specifically from the strategies used [23,25,28,31,33]. Felker et al. [23] used three strategies: 'provide interactive assistance' , 'adapt and tailor to the context' , and 'train and educate stakeholders' , the latter strategy resulted in high ratings of the acceptability, adoption, and appropriateness of the training provided. Lindsay et al. [25] also reported using the same three strategies and found that Table 1 Details of all categories covered by the ERIC compilation of implementation strategies -from Powell et al. [16] Use of evaluative and iterative strategies Examples include conducting a local needs assessment, provision of consumer feedback on the implementation, assessment for readiness, identification of barriers, quality monitoring tools, audit and feedback

Provision of interactive assistance
Examples include processes of enabling and supporting individuals, groups, or organisations to adopt or incorporate effective practice, local technical assistance, ongoing supervision, and centralisation of technical assistance for implementation issues such as help-desks and online "frequently asked questions"

Adaptation and tailoring to context
Examples include adapting interventions to address previously identified barriers, and identifying which aspects of the intervention can be adapted to suit need

Development of stakeholder interrelations
Examples include identification of champions or leaders to support and drive implementation and overcome resistance, development of multidisciplinary support teams with protected time to reflect on practice and share lessons, recruitment and cultivation of relationships with partners or community resources, such as charities, and identification of early adopters who others can learn from

Training and educating stakeholders
Examples include ongoing training throughout implementation for clinicians, support staff and facilitators, ongoing consultation with experts, development of manuals and toolkits and training designated people to train others

Supporting clinicians
Examples include facilitating the relay of information to clinicians, resource sharing agreements with organisations that have relevant required resources, revision of professional roles and changes to clinical teams to ensure the necessary skills are available

Engagement of consumers
Examples include involving service users in the implementation effort, encouraging adherence, problem solving and spreading the word about the intervention

Utilising financial strategies
Examples include funding to encourage uptake or incentivising adoption

Changes to infrastructure
Examples include encouraging leadership to declare the intervention a priority, adaptation of physical structures such as room layout and changing accreditation and certification requirements therapists reported a high degree of satisfaction with the external facilitation provided as part of training. Myers et al. [28] used three strategies: 'provide interactive assistance' , 'develop stakeholder interrelationships' and 'train and educate stakeholders' during the implementation of a TMH platform, and found that whilst adoption of TMH increased, there were additional implementation costs in terms of time associated with unpaid site champion roles developed to provide interactive assistance. Puspitasari et al. [31] used five implementation strategies in the rapid adoption of TMH due to the COVID-19 pandemic: 'use evaluative and iterative strategies' , 'provide interactive assistance' , 'adapt and tailor to the context' , 'develop stakeholder interrelationships' , and 'train and educate stakeholders' . They found education, training, and ongoing supervision were of particular importance at the start of teletherapy implementation to support clinicians' successful engagement with the technology. Taylor et al. [33] used two strategies-'use evaluative and iterative strategies' and 'develop stakeholder interrelationships' to investigate the importance of clinical facilitation for the implementation and sustainability of TMH in perinatal and infant mental health services and concluded that ongoing clinical facilitation is necessary for the sustainability of TMH services.
We had originally aimed to make specific links between strategies and general implementation of TMH outcomes, but as all but one study reported several implementation strategies in varying combinations, this was not possible. Instead, strategies and reported outcomes are presented in Table 4 and outlined for each study in Additional file 3: Appendix 3. Outcomes for each study are categorised according to the taxonomy of implementation outcomes [12].
There was no discernible difference in the implementation strategies used by studies which did or did not use implementation frameworks to guide their approach, as all studies used a variety of different strategies. In addition, we did not observe any commonalities across the outcomes of these studies.
We also compared the use of telemental health strategies between the studies which implemented telemental health before and after the start of the COVID-19 pandemic, but as all studies used a variety of different     Use evaluative and iterative strategies  The model was designed to facilitate communication between primary care and mental health services using a mental health advisor

Acceptability (Service user and carer views)
97% of mothers reported overall satisfaction with the intervention and 100% rated the quality of the mental health advisor as high Fidelity Mental health advisors were trained using standardized procedures and followed a written treatment manual and study protocol. Fidelity ratings were 83% Chen et al (2021) b [

22] Use Evaluative and Iterative Strategies
Quality improvement data was gathered to allow rapid identification of problems and adjustments to be made Adapt and Tailor to the Context Services were developed for TMH delivery based on a review of the literature and consultation with clinicians with previous experience of TMH

Train and Educate Stakeholders
TMH was integrated into the existing Psychology training programmes, with the goal of offering TMH training to all existing Psychology training programmes within the next three years Support Clinicians Five new psychologists were hired for the main hub, weekly calls were set up between spoke sites and hub staff to establish the services. One staff member served as the primary point of contact for each spoke

Adoption
Within five months the service reached its preestablished productivity goals of 80 veteran encounters per month, per provider for the first year Penetration From March 2017 to January 2018, 377 consults were received for TMH psychology services and 252 veterans engaged in TMH services. However, 32% did not receive treatment due to a variety of reasons, such as disengagement or discharge prior to services being offered

Felker et al. (2021) [23]
Provide Interactive Assistance Training courses and workshops to address the specific practical aspects of providing TMH. Clinicians were encouraged to engage in TMH with at least 2 patients and attend at least 10, 1-h consultation calls to ask questions related to TMH (clinical or implementation issues) Adapt and Tailor to the Context Internal facilitators from each team provided consultation to external facilitators regarding the unique clinical and cultural aspects of their team (e.g. patients served, types of services provided, administrative needs, technological needs). External and internal facilitators tailored the TMH training programme to address clinic specific culture and barriers and meet unique clinic goals

Acceptability (Clinician views) a
Following the training, 95% of providers agreed (n = 42) or strongly agreed (n = 35) that they were satisfied with the training provided Adoption a Providers reported increased knowledge, skills and interest in TMH after training Appropriateness a 95% of providers agreed (n = 50) or strongly agreed (n = 28) that the amount of information covered was sufficient to begin using TMH. 76% of participants agreed (n = 45) or strongly agreed (n = 17) that they felt confident using TMH after receiving training Feasibility Barriers identified included: lack of patient interest (45%), administrative burden (20%), preference for in-person appointments (18%), concern about increased workload (11%), not completed all of the training components (6%), lack of supervisor support (4%), lack of provider interest (4%), and other reasons (4%)

Hensel et al. (2020) [24] Adapt and Tailor to the Context
Initial survey of barriers allowed implementation to be tailored to the specific challenges identified by staff Develop Stakeholder Interrelationships Worked with emergency departments to establish support staff available to assist with referral. Implementation leads were appointed at site and leadership at all levels were engaged in the programme. Clinical champions with TMH experience encouraged staff engagement

Train and Educate Stakeholders
Education, anecdotes and evidence review from experienced providers. Initial training of a core group to develop expertise was conducted to build group confidence before engaging a larger cohort of providers. Training was offered to inexperienced providers

Engage Consumers
Clear explanations were given to patients and families regarding the TMH programme Utilize Financial Strategies Existing fee schedules were reviewed to support physicians and psychiatrists were salaried to avoid renumeration challenges. They also worked with regional authorities and hospitals to secure funding when needed Change Infrastructure They worked with participating emergency departments to install dedicated equipment where possible or make arrangements regarding existing equipment

Adoption
In the first year of operation, 243 assessments were completed Workload increased by 42% between the 6 months pre-programme and the second 6 months of programme operation. There was a 2% increase in presentations at the hub, and some increase in workload from the spokes which saw declines in on-site support and an 8% increase in total mental health and addiction presentations. The percentage of transfers avoided increased from 0% pre-programme to 65% in December 2018 Lindsay et al. (2015) b [25] Provide Interactive Assistance Technical support was provided through weekly consultation calls with a facilitator to discuss technical and logistical issues specific to the delivery of TMH Adapt and Tailor to the Context Site-specific implementation plans were tailored to unique needs of the site including needs of stakeholders

Train and Educate Stakeholders
Intensive training in evidence-based practice for PTSD was provided to providers including an experientially orientated 2-3-day workshop and weekly consultations with experts

Acceptability (Clinician views) a
Therapists reported a high degree of satisfaction and rated the external facilitation model as very helpful in their efforts to implement video telehealth (6.67 out of 7), viewing the regular facilitation calls as very important to establishing video telehealth services Penetration Compared to baseline, participating sites averaged a 6.5-fold increase in psychotherapy sessions conducted via TMH, whereas non-participating sites averaged a 1.7-fold increase

Lynch et al. (2020) [26] Use Evaluative and Iterative Strategies
In response to reports of problems with maintaining attention in virtual sessions, clinicians problem solved with clients to minimise distractions, used screen sharing features and interactive activities, and provided additional brief breaks when needed Provide Interactive assistance Virtual training on the features and functionality of telehealth platforms were provided to staff Support Clinicians Factors to support and capture work from home productivity were considered for staff

Engage Consumers
Individualized instruction about telehealth platforms were provided to service users as needed

Adoption
TMH acceptance rates indicated that 90% (n = 18) of the 20 patients enrolled at the time of conversion agreed to TMH sessions within ten days of the service transition and maintained their specific treatment plans virtually. An additional five service users began using TMH after the start of the study. There were no significant differences in attendance rates before conversation to TMH, and no differences in acceptance between the TMH and non-TMH group Feasibility Following conversion to TMH, participants and clinicians sought to maintain individualized treatment plans and group schedules whenever possible, which may have contributed to the high acceptance rates and unchanged service utilization  Author (year) Implementation strategies used (ERIC Categories)

Lynch et al. (2021) [27] Use Evaluative and Iterative Strategies
The service responded to challenges identified by staff with new implementation strategies Adapt and Tailor to the Context Group session material was adapted to be engaging on virtual platforms

Develop stakeholder interrelations
In addition to formal systems that were put in place to ensure consistent communication (e.g., end-of-day email debriefs), staff had increased support from supervisors to facilitate both client care coordination and opportunities for staff to "support each other as individuals. "

Engage Consumers
Through a collaborative approach some service users who were challenged by TMH helped the team to come up with web etiquette guidelines for other service users Change Infrastructure The proactive culture at the clinic helped rapid transition to TMH and aided continuity of care. Resources, workflows and infrastructure were developed in anticipation of regulatory change, rather than in response

Acceptability (Clinician views)
Though staff perceived the shift to TMH as slightly more challenging for themselves than for clients, they reported learning to navigate the technology and virtual interaction fairly quickly However, TMH negatively impacted staff's ability to communicate with each other, due to the lack of informal contacts. 'Zoom fatigue' and exhaustion were also reported by staff Acceptability (Service user and carer views) All respondents who completed the questionnaire (n = 18) provided a score > 23, suggesting satisfaction with the TMH services. However, 78% of respondents stated that they would still prefer in-person sessions if there were no health risks. Only 50% reported feeling that TMH was as good as in-person sessions Adoption 93% of service users enrolled at the time of conversion agreed to maintain their specific treatment plans virtually. 7% opted out. Session attendance did not significantly differ over time or between in-person and TMH formats. The mean no show/cancellation rate was 37% less at 13-18 weeks after implementation of TMH compared to in-person (B = -.47, p < 0.05)

Appropriateness
TMH was deemed appropriate because of its increased flexibility to adapt scheduling to client capacity for engagement, e.g. offering shorter, more frequent breaks, or reducing session duration but increasing frequency. However, staff raised concerns that for some service users, long-term TMH utilization may hinder recovery, as the routine and engagement associated with traveling to a clinic may enhance treatment investment and pro-health behaviours Feasibility  Site champions (with previous experience or trained for leadership roles) were utilised to support implementation

Train and Educate Stakeholders
The TMH champions assisted with mandatory training of policy and procedures, and with selection criteria for determining appropriateness of treatment via TMH

Adoption
The site failed to address lack of internet or phone access for service users, which affected implementation. However, use of TMH was increased by 42% Appropriateness TMH was considered appropriate other than for suicidal or psychotic individuals. Lack of appropriateness for these service users, however, limited the ability to provide crisis support Feasibility Providers reported concerns about the feasibility of TMH: 1) it reduced their ability to respond to emergencies (e.g., responding to suicidal patients); 2) it may not be feasible for some veterans considered "too high risk" or unstable; 3) some veterans were not respecting therapeutic boundaries (e.g., trying to engage in treatment sessions while driving); 4) too much time was lost attending to technical issues; and 5) difficulty in delivering measurement-based care Implementation Cost a The main cost was time-related (the role of site champion was unpaid) Sustainability Sustainability of TMH may vary by site, depending on organisational constraints (administration, other role commitments which may inhibit implementation and ongoing support) It is unclear if all providers should be "telehealth generalists" or if TMH should be a speciality Owens & Charles (2016) [29] Use Evaluative and Iterative Strategies Clinicians and service users worked closely with the research team and software developers through a series of three iterations or feedback loops to optimise the intervention and assess whether it was sufficiently likely to normalise to be worth evaluating in a full trial Develop stakeholder interrelations Three clinicians in each team supported and mentored each other for the duration of the study and cascaded knowledge through the team, influencing others to adopt the intervention

Acceptability (Clinician views)
Clinicians saw it as a potentially valuable tool to help young people manage their self-harming behaviour Adoption The most significant barrier to adoption was the need for buy-in at management levels and the time it took to obtain this Feasibility CAMHS teams reported being under very high pressure which negatively affected their ability to be involved in new projects Appropriateness In the context of very heavy caseloads, high stress levels and exhaustion, the effort involved in mastering a new technology and incorporating it into everyday practice was perceived to be too much by clinicians. Although some reported that they were using apps of various kinds with their clients, others appeared to be resistant to technological interventions. Nearly all informants believed that CAMHS was not the ideal delivery setting as clinicians see only the most acute and complex cases and duration of contact with CAMHS is typically short

Puspitasari et al. (2021a) [30] Train and educate stakeholders
Counsellors attended weekly consultation meetings facilitated by a clinical psychologist to ensure treatment adherence and fidelity. All disciplines attended daily meetings to discuss safety management and patient progress Engage Consumers Service users who were accepted into the programme received assistance from programme staff and information technology support staff to prepare for the first TMH session. Each group was led by two counsellors: one as the primary facilitator leading the presentation and group discussion, the other assisting patients with any technological issues

Adoption
The completion rate of the programme was 70/76. This completion rate was higher than typical completion rates for psychiatric Intensive outpatient or partial hospitalization programmes Feasibility Zoom features (including chat, whiteboard, shared screen and waiting room) improved feasibility. It was also feasible to conduct psychotherapy experiential exercises via videoconferencing, e.g., performing guided group mindfulness exercises, completing psychotherapy forms, and watching psychotherapy skills videos

Puspitasari et al. (2021b) b [31] Use Evaluative and Iterative Strategies
A staged implementation strategy was used where the TMH group intervention was first piloted in one site, which indicated readiness for TMH implementation, openness among clinicians and availability of resources. Challenges faced during the TMH rollout were informally assessed and communicated to team members for efficient problem solving Provide Interactive Assistance A multi-disciplinary TMH committee coordinated the change to TMH and ensured clinicians had access to necessary technology. All clinicians had 24/7 access to the IT help desk for additional support. An operations manager coordinated the preparation, adoption, and implementation phase. This individual was responsible for managing the workflow and engaging other stakeholders within and outside of the department to ensure a smooth transition to teletherapy. Quick reference guides were also created for clinicians to help them adapt to TMH Adapt and Tailor to the Context The committee met twice weekly for the first month during the most rapid phase of implementation to review, update and expand upon existing training resources, guidelines, and policies. Due to the closing of many behavioural health services in the surrounding area and increased need for intensive outpatient care, the service expanded capacity and added an additional intervention for patients suffering with mood and anxiety disorders Develop Stakeholder Interrelationships TMH champions were identified (including directors, an operations manager, committee members, IT specialists, and several clinicians with TMH experience or enthusiasm). These champions were fully integrated into the team to provide adequate support for its other members. Daily virtual meetings attended by all staff allowed discussion of patient progress and issues Train and Educate Stakeholders Education, training, and ongoing supervision were integral implementation strategies prior to TMH adoption Adoption a Education, training, and ongoing supervision were of particular importance at the start of teletherapy implementation to support clinicians' successful engagement with the technology, as well as to establish an effective practice for virtual therapy Feasibility Data on patient attrition indicated that TMH was feasible to assure patient retention, since many service users completed the programme and the average number of sessions attended was high Penetration A plan was established by the pilot site to initiate full implementation following the pilot This study demonstrates the feasibility of rapidly building upon an existing telemedicine infrastructure to train a large group of multidisciplinary providers to deliver urgent home-based TMH services. However, the key message is that even with a well-established telemedicine infrastructure, programmes must expect to encounter serious challenges during crises. Planning for the next crisis should start now

Implementation Cost
Funding fell dramatically after transitioning patients from clinic to home. Interim phone appointments while awaiting full implementation of TMH services yielded less revenue per appointment than in-clinic or TMH appointments, although required the same amount of time and almost the same level of documentation by the faculty Penetration After 1 month, TMH was offered to all established outpatients for individual visits and the clinic started a trial process for enrolling new patients. Continued work on expanding TeleGroups occurred. Only the crisis clinic continued a regular in-clinic presence Sustainability The faculty's relatively rapid but complex development of clinic-wide home-based TMH and TeleGroups was reported to help to advance and increase access to psychiatric care. Authors argued that in the future, home-based TMH may help overcome barriers to treatment such as distance, transportation and scheduling Taylor et al. (2019) [33] Use evaluative and iterative strategies A pilot project established the efficacy of the intervention in improving the skills and knowledge of local health service providers but identified a need for additional clinical support in specialist areas. This was therefore integrated into the model

Develop Stakeholder Interrelationships
As a result of the pilot project, General practitioners (GPs), mental health professionals and other service providers were offered access to secondary consultations with perinatal and infant psychiatrists. The service also employed a clinical facilitator who was responsible for service promotion, site visits, staff education and training, co-ordinating case conferences and video consultations

Acceptability (Clinician views)
Mental health workers who had used TMH were unanimously complimentary about the service, reporting that it allowed expert input into care planning, reduced professional isolation, upskilled remote workers and provided a sense of security for remote care providers Appropriateness The study showed that TMH can help address unmet need for specialist mental health services in regional, rural and remote areas Sustainability a Clinical facilitation is likely to be more important in intermittent compared with high-volume services where regular clinics can make TMH more visible. Ongoing facilitation is necessary for the sustainability of TMH services due to intermittent demand and local impediments, such as fragmentation of service providers and transiency of the workforce a Outcomes of the strategy to improve implementation of the TMH intervention (vs outcomes of the intervention itself ) b Low quality or descriptive study strategies there were no obvious differences in the types of strategy used or reported outcomes. However, Felker et al. had implemented telemental health prior to the pandemic and concluded that "earlier TMH training efforts likely helped foster staff interest and experience using TMH, and in doing so contributed to the relatively strong TMH adoption rate observed in this health care system during the early months of the COVID-19 pandemic. "

Summary of findings
In this study, we have reviewed literature on strategies used to improve implementation of TMH and their associated implementation outcomes. We identified as meeting our inclusion criteria fourteen studies, conducted across five countries. Both staff and service user views and outcomes were represented in these studies.
Results indicated that using a combination of different planned implementation strategies could be associated with successful implementation of TMH, although the methodologies of most studies were such that firm causal conclusions were difficult to draw. Whilst we had originally planned to explore links between individual types of implementation strategy and implementation outcomes directly, we were unable to isolate the effects of specific mechanisms as all studies reported outcomes of initiatives that combined multiple implementation strategies and none had a comparison group. We are, however, able to propose some tentative conclusions based on the synthesis of findings from studies which reported outcomes of these strategies. Ongoing support and facilitation, for example, through either technical assistance or ongoing consultation, was common and tended to be strongly linked to successful implementation. Providing initial training and the use of 'digital champions' to model best practice, also benefited implementation of TMH.
Other recent studies further support and supplement these conclusions. In our recent rapid realist review of TMH [34], we found that providing staff with training on the use of technology to deliver mental health services, a strategy reported by several studies in the current review, was reported to increase confidence in and uptake of TMH. The rapid realist review also found that adapting the use of TMH to take into account service user preferences was beneficial in removing barriers to accessing TMH [34] (see Table 5 for further discussion). For example, a commonly cited barrier to service user engagement with telemental health has been a lack of choice of modality [5,35], so that offering a choice of platforms to service users may remove some barriers to telementation implementation.
Research indicates that ERIC strategies are considered suitable to influence different implementation outcomes [36], but there is currently little consensus or evidence regarding which strategies affect specific outcomes. Furthermore, Powell and colleagues [37] argue that implementation strategies should be tailored to the circumstances and context of each change project, as they may be more likely to result in changes to practice. They suggest four different methods to identify appropriate strategies: concept mapping, group model building, conjoint analysis, and intervention mapping. Essentially, this means that the appropriateness and effectiveness of an implementation strategy or implementation support 'bundle' may well depend on the context in which a clinical intervention or service delivery mechanism, such as TMH, are introduced to. It logically follows that the same This systematic review reveals the lack of knowledge on how best to implement telemental health (TMH). The included studies refer to a range of settings, dates, TMH applications and patient groups making it difficult to draw a single conclusion which might work across them. Participants' age likely affects affinity for technology (not recorded in 8 studies), and other demographic and cultural factors may impact on access. Implementation strategies might vary across healthcare systems internationally, and the pandemic impacted on available resources and funding streams which may have influenced procurement choices The underlying assumption is that TMH is beneficial. New technology is generally viewed as progress, but in health and social care the most important consideration should be human relationships, both with patients and between staff. Technologies may disturb these relationships and the full implications upon both workforce and healthcare are yet to be considered. For example, service users emphasise that choices are essential, including the option of whether to use TMH or not. However, anecdotal evidence suggests that some clinicians view choosing TMH over face-to-face appointments as a reluctance to fully engage The conclusions of this study seem rather obvious for any new development: that offering training and ongoing support would help. Training needs will differ significantly from familiar SMS messages to bespoke software, but in any case this assumes the intervention is known to be comparatively effective and desirable for all parties Future studies should pay specific attention to what is useful for which groups of people, including adaptations to context, before progressing to investigate implementation. Inevitably, any singular approach will leave some communities excluded Fundamentally, we would argue that consumers provide the ultimate litmus test of acceptability and effectiveness for any TMH modality. Co-production, from design stage to evaluation, is surely key to the success of any implementation implementation strategy may be very well suited to one context, but redundant in another if it fails to address specific barriers to implementation. Hence selection of implementation strategies should be tailored to the local context of application. This can be illustrated by studies which have reported various staff barriers or challenges to using TMH. These have included: feeling isolated from colleagues, tasks taking more time when working from home, lack of support, concerns around privacy or confidentiality, and concerns around developing and maintaining a therapeutic relationship [2,38,39], indicating the importance of tailoring implementation strategies to overcome local barriers. Whilst some studies included in this review evaluated the barriers and facilitators to TMH before using implementation strategies to address these, not all papers reported taking this approach in a systematic manner.

Implications for future research
As all studies reported using multiple implementation strategies, we were unable to compare the effectiveness of specific strategies. This could be addressed by future research, for example, cluster randomised controlled trials to formally compare the effectiveness of different implementation strategies linked to specific outcomes at a team or organisational level, although it would also be helpful to evaluate a theoretically informed approach to selecting a bundle of implementation strategies to fit a particular context, using more robust study designs. Inclusion of a control group in future evaluations of the implementation of TMH is critical: without this we will remain unable to establish causal links between the presence or absence of a strategy to support implementation, or bundle thereof, and success of implementation offers an illustration of a clustered randomised evaluation of different interventions to support the implementation of four evidenced-based psychosis treatments. Within this study, what was randomised was not the clinical therapies, but rather the level of implementation support, which was limited in the control arm (provision of treatment manuals) and substantially enhanced in the intervention arm to include toolkits, training, implementation facilitation, and data-based feedback. The primary endpoint was fidelity of treatment delivery, and the trial concluded that the implementation 'bundle' was successful in enhancing fidelity across all four studied treatments for psychosis. Similarly designed studies in the context of TMH provision will significantly expand our knowledge regarding how best to deliver it sustainably and effectively.
Beyond controlled evaluations, our understanding of the relative effectiveness and suitability of strategies to support implementation of TMH across different settings can be further enhanced. Future observational studies should offer a detailed description of local (or wider, as appropriate) settings in which TMH is offered, a wellarticulated rationale for the selection of strategies (such as those we summarised above) to support implementation, and clear details of what each strategy involves -such that subsequent evidence syntheses can offer a better articulation of which strategies may be better suited to which contexts. We further propose that selection of implementation support interventions should be based on a description and mapping of the barriers and drivers an implementation effort is likely to face, for example carried out using one of the methods proposed by Powell et al. [38].
It is also important to note that most studies identified in this review were conducted in the United States. There is therefore a need to replicate these findings and conduct further research in other countries with different healthcare structures and funding models to generalise the findings from this research.

Implications for practice
Our review identified a range of potential implementation strategies to be deployed to improve TMH implementation in routine settings, with evaluations spanning a full range of types of implementation strategy. Although we were unable to identify causal links between implementation strategies and outcomes, findings across the included studies suggest that when implementing TMH, service planners should consider a multi-component implementation strategy. This strategy should be tailored to the local context and designed to address any pre-identified barriers. It is likely that staff training and facilitation support are key factors in the success of implementing TMH.

Strengths and limitations
A strength of our review is that it focused only on studies which implemented TMH as part of their routine service (i.e., not just in a trial), which means findings are more likely to generalise to 'real world' settings. This review also integrated data from studies conducted before the COVID-19 pandemic with those conducted during the pandemic, which enables findings to be used to inform future models of service development.
Limitations of this research should be acknowledged. Firstly, as noted earlier, the high heterogeneity of strategies and outcomes reported across studies makes it hard to reach firm judgements about which strategies are linked with effective implementation of TMH. As all studies reported the inclusion of multiple strategies, we were unable to draw conclusions regarding the active ingredients of specific strategies. Secondly, researchers were not blinded to the results of screening and quality assessment during double screening. This was due to the short timeframe in which the review was conducted in order to make its results relevant to the current service context. We limited the scope of our review to the implementation of TMH in specialist mental health services, which typically serve mainly people with severe and enduring mental illness. Our review is therefore unable to provide evidence about TMH implementation in primary care settings, or for other client groups including people with substance misuse (in the absence of a mental illness) or people with dementia or other organic disorders. Finally, included studies were mostly of moderate to low quality, and only five interventions were informed by an established implementation framework.

Conclusion
Using a combination of implementation strategies appeared associated with successful implementation of TMH, but it was not possible to infer conclusively causal relationships between specific types of implementation strategy and outcomes. Potentially valuable strategies to improve the implementation of TMH include providing initial training for clinicians, as well as ongoing support and consultation. Further research utilising more robust study designs to evaluate individual implementation strategies is needed to explore which specific factors can influence implementation of TMH.