Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Building the capacity of policy-makers and planners to strengthen mental health systems in low- and middle-income countries: a systematic review

  • Roxanne Keynejad1,
  • Maya Semrau2,
  • Mark Toynbee3,
  • Sara Evans-Lacko4, 5,
  • Crick Lund6, 2,
  • Oye Gureje7,
  • Sheila Ndyanabangi8,
  • Emilie Courtin9,
  • Jibril O. Abdulmalik7,
  • Atalay Alem10,
  • Abebaw Fekadu11, 12,
  • Graham Thornicroft13 and
  • Charlotte Hanlon14Email authorView ORCID ID profile
BMC Health Services ResearchBMC series – open, inclusive and trusted201616:601

https://doi.org/10.1186/s12913-016-1853-0

Received: 9 July 2016

Accepted: 15 October 2016

Published: 21 October 2016

Abstract

Background

Little is known about the interventions required to build the capacity of mental health policy-makers and planners in low- and middle-income countries (LMICs). We conducted a systematic review with the primary aim of identifying and synthesizing the evidence base for building the capacity of policy-makers and planners to strengthen mental health systems in LMICs.

Methods

We searched MEDLINE, Embase, PsycINFO, Web of Knowledge, Web of Science, Scopus, CINAHL, LILACS, ScieELO, Google Scholar and Cochrane databases for studies reporting evidence, experience or evaluation of capacity-building of policy-makers, service planners or managers in mental health system strengthening in LMICs. Reports in English, Spanish, Portuguese, French or German were included. Additional papers were identified by hand-searching references and contacting experts and key informants. Database searches yielded 2922 abstracts and 28 additional papers were identified. Following screening, 409 full papers were reviewed, of which 14 fulfilled inclusion criteria for the review. Data were extracted from all included papers and synthesized into a narrative review.

Results

Only a small number of mental health system-related capacity-building interventions for policy-makers and planners in LMICs were described. Most models of capacity-building combined brief training with longer term mentorship, dialogue and/or the establishment of networks of support. However, rigorous research and evaluation methods were largely absent, with studies being of low quality, limiting the potential to separate mental health system strengthening outcomes from the effects of associated contextual factors.

Conclusions

This review demonstrates the need for partnership approaches to building the capacity of mental health policy-makers and planners in LMICs, assessed rigorously against pre-specified conceptual frameworks and hypotheses, utilising longitudinal evaluation and mixed quantitative and qualitative approaches.

Keywords

Mental health Capacity-building Policy-makers Health system strengthening LMICs Systematic review Psychiatry Global mental health

Background

The global burden of mental, neurological and substance use (MNS) disorders is high, resulting from chronic disability combined with premature mortality [1, 2]. Untreated MNS disorders also impact negatively on global health priorities [3], and may be associated with human rights abuses [4]. The majority of people with MNS disorders in LMICs are unable to access effective mental health care, with the treatment gap higher than 90 % in some low-income countries [5].

Health system constraints are potent threats to the scale-up of evidence-based mental health care for people affected by MNS disorders in LMICs [6]. Policy-makers and planners play a critical role in the successful strengthening of mental health systems, but despite historical attempts at capacity-building [7], may not be appropriately equipped for the task. The limited evidence and evaluation of capacity-building interventions for policy-makers in LMICs has been recognized in physical health care [8] and is likely to be even more significant in mental health care. In a mixed-methods report of implementing a national mental health policy in South Africa, barriers included the relatively low priority given to mental health care by planners, provincial bureaucracy around service coordination, insufficient staff for policy-making and service planning, and disinclination by some local authorities to lead mental health policy implementation [9].

In a qualitative study involving national and regional stakeholders in Ghana, South Africa, Uganda and Zambia, low perceived legitimacy of the problem of scaling up mental health services and inadequate government support were identified as factors perpetuating the low priority accorded to mental health care [10]. A qualitative survey of leaders and specialists in international mental health specifically identified the need for a broader and more holistic view of mental health care, adopting a more public health-level perspective among mental health policy-makers [11]. In particular, the lack of training and experience of clinicians to fulfil leadership roles was emphasized as a barrier to high quality mental health policy-making.

There is international consensus on the need for mental health system strengthening and for a specific focus on building the capacity of key stakeholders, including policy-makers and planners, and service users [12]. ‘Systems thinking’ casts health services as a type of complex adaptive system, characterized by large numbers of individuals occupying a range of roles, acting in the context of and adapting to constant internal and external changes [13]. It advocates a ‘systems perspective’ which unifies health care with education, research and policy-making, through “collaboration across disciplines, sectors and organizations; ongoing, iterative learning; and transformational leadership” [14]. This approach in part represents a reaction to the former emphasis on targeted, often disease-specific health investments, which have not demonstrated benefits for the wider health system [15]. However, the specific means by which the capacity of mental health policy-makers and planners should be built, to facilitate such systems-wide scaling up, are less well-documented. A recent systematic review of the involvement of mental health service users and their caregivers in mental health system strengthening, identified a lack of high quality research [16].

Given the widespread support for building the capacity of policy-makers and planners in LMICs to achieve mental health system strengthening, this systematic review aimed to identify the best evidence and experience for specific capacity-building intervention models. A secondary aim was to identify methods of evaluating models of capacity-building for mental health policy-makers and planners in LMICs.

Methods

This systematic review formed part of the ‘Emerging mental health systems in LMICs’ (Emerald) program, which focuses on the health system inputs, processes and outputs required for mental health service scale-up in LMICs [17]. A systematic investigation of existing evidence-based strategies was an important precursor of the Emerald project, since one of its aims is to build the capacity of policy-makers and planners to strengthen the mental health system. This work was registered on the PROSPERO international prospective register of systematic reviews (Registration Number: CRD42016032798).

Eligibility criteria

In view of the anticipated scarcity and heterogeneity of relevant literature, the criteria used for selection of studies into this review were intentionally broad and inclusive. The review set out to include any type of study design, review or report on the evidence, experience or evaluation of capacity-building of policy-makers, service planners or managers in mental health system strengthening in LMICs. Studies were included whenever the recipients of system-focused capacity-building were either currently involved in policy-making, planning or co-ordination of services, or when recipients had the potential to take on such a role, regardless of their professional background. Mental health systems were interpreted to include services addressing the priority MNS disorders outlined in the World Health Organization’s mental health Gap Action Program [18]. Papers written in the following languages were included: English, Spanish, Portuguese, French or German. Studies were excluded where system level interventions to support implementation and expansion of mental health care were implemented by external agencies and did not report specifically on the building of local capacity. A common reason for exclusion was that studies described mental health training of clinical staff without discussing training needs of mental health policy-makers, or commented on the need for capacity-building without presenting any interventions to build policy-makers’ capacity. Studies that reported solely from high-income countries were excluded.

Search strategy

The following databases were searched on 12 November 2013: MEDLINE (from 1946 until December 2013), Embase (1974 to December 2013), PsycINFO (1806 to December 2013), Web of Knowledge, Web of Science, CINAHL, LILACS, ScieELO, Google Scholar and Cochrane and on 27 November 2013, Scopus (all from the start date of the database to November 2013). The search was updated on 25 November 2015 to look for any papers published in the intervening period of two years. Details of the search strategy used are given in Additional file 1. In addition to database searches, the reference lists of included papers were hand-searched for relevant studies and experts in the field were contacted to identify any further studies. Grey literature including reports and web-based resources were identified via the Google Scholar search and experts in the field.

Study selection and data extraction

Figure 1 shows the flow of papers from identification to selection and data extraction. The titles and abstracts of all 2950 papers identified in the search were assessed by two independent reviewers. This number comprises 2040 papers identified in the first database search up to November 2013, a further 882 papers identified in the second database search up to November 2015 and 28 papers identified by contacting experts or key informants, hand-searching references, and searching the internet. In the first search, the 1704 papers excluded at the stage of abstract review were either not eligible for inclusion due to their subject matter (1581), because they were duplicates (66), based in high income countries (51) or due to language (6). In the second search, 837 papers were excluded at the stage of abstract review because of ineligible subject matter (827), based in high income countries (8) or due to language (2). Papers were considered to pertain to LMICs if the work described took place in a year in which the country was classified by the World Bank [19] as a low or middle-income country.
Fig. 1

PRISMA flow diagram for selection of peer-reviewed articles

This process yielded a total of 409 papers which were reviewed as full text articles. For the screening of full text papers identified by the database search, all articles were reviewed independently by at least two reviewers (RK, SE-L, JA or MT). Additional papers identified via experts and informants and hand-searching references and the internet were screened independently by SE-L, JA, MT or RK. Where there was disagreement between independent reviewers of full text articles regarding inclusion, the paper was reviewed by two senior independent reviewers (CH and MS), who made a final decision. In several papers, the language used to describe participant groups and interventions was vague, creating ambiguities about whether the work constituted clinical training alone or clinical training combined with capacity-building. In such cases, papers were reviewed again and where necessary, input from a third independent reviewer was obtained. MS reviewed 7 full texts where there was disagreement, leading to inclusion of 4 studies. CH reviewed 5 full texts where there was disagreement and included 3 studies. This was sufficient in all cases and contacting the authors of such papers was not required.

Of the 409 full text papers screened, 395 were excluded due to not addressing the focus of this review: capacity-building interventions for mental health policy-makers and planners.

Data from included papers were extracted by independent reviewers (RK, MT and JA) and inputted into a data extraction form (Additional file 2). The data extracted included: authorship (noting whether or not this included authors from LMICs), type of publication, countries involved, study design, capacity-building participants, sample size, nature of capacity-building and any guiding framework, type of evaluation, type of evaluation data and a summary of findings.

Quality evaluation

Given the mixed methods employed by included studies, their quality was evaluated using two methods: one to assess quantitative aspects (Additional file 3) and one to assess qualitative aspects (Additional file 4). Included papers were assessed by MT and RK using the Effective Public Health Practice Project (EPHPP) Quality Assessment Tool for Quantitative Studies [20], which rates studies from 1 (‘strong’) to 3 (‘weak’) for selection bias, study design, confounders, blinding, data collection method, withdrawals and dropouts. These scores are used to obtain a global rating of strong (no ‘weak’ ratings), moderate (one ‘weak’ rating) or weak (two or more ‘weak’ ratings). Included papers were also evaluated by MT or RK against 12 criteria for high quality qualitative research, derived from the literature and previously employed to distinguish papers of low, medium and high quality [21, 22]. The specific criteria are included in a footnote to Additional file 4 and pertain to quality of reporting, use of strategies to increase reliability, validity and rigor. Studies were assessed as of low quality if they met seven criteria or fewer, medium quality if they met between seven and nine criteria and high quality if they met ten or more criteria.

Results

Following the process of systematic searching, 14 papers fulfilled the inclusion criteria and were included in this review. The data extracted from included papers were not amenable to statistical synthesis through meta-analysis, being heterogeneous and largely non-numerical. The study findings were therefore reviewed in a narrative manner instead.

Included papers: capacity-building interventions for mental health policy-makers and planners

The 14 papers included in the review described nine capacity-building interventions. One intervention aimed to equip policy-makers and planners with “knowledge, skills and attitudes required to make the paradigm shift and the translation into the effective provision of health care” for people with substance use disorders [23]. Another implemented a range of interventions to scale up a harm reduction program for people who inject opioids [24]. Three interventions of the nine were orientated towards developing skills in planning and co-ordination of mental health care [2530] and a fourth did so with a specific focus on de-institutionalization of children’s services [31]. Two programs focused specifically on enhancing mental health leadership [3235] and a further intervention aimed to build regional capacity to prevent and control mental illness and other non-communicable diseases (NCDs) across 24 countries [36].

Each capacity-building intervention combined short training courses or workshops with a system of continued mentorship or dialogue. In most cases the interventions were not specific to policy-makers and planners but to any person involved in mental health service co-ordination, planning or development, including mental health professionals. The nine capacity-building interventions described by the papers included in this review are summarized in Table 1.
Table 1

Summary of capacity-building interventions

Capacity-building intervention

Summary

Capacity-building for substance misuse system change in Russia [22]

As part of a larger program of mental health service enhancement, a project was implemented in St Petersburg, to support a shift from a universally medical model of drug and alcohol services to one which incorporates psychosocial aspects of care. Through a program of exchange between Russia and England, staff from the University of Portsmouth and St Petersburg paid learning visits to each country’s services and shadowed peers, supplemented by seminars and workshops on different therapies and clinical skills, such as patient assessment and motivational interviewing, followed by supervision sessions. Training participants included staff with responsibilities for policy-making and implementation.

Scaling up harm reduction for opioid injecting in India [23]

Project ORCHID (organized response for comprehensive HIV interventions in selected high-prevalence districts) aimed to establish and scale-up a harm reduction package for people injection opioid drugs in two states of north-eastern India, through capacity-building of community participation and a network of 24 local non-government organizations (NGOs). They created a program management team comprising injecting opioid users, which gradually took responsibility for supporting local advocacy work. Through its emphasis on project management and support, ORCHID fostered data-driven decision-making, development of guidelines and training manuals to standardize care and widespread, robust monitoring procedures. NGO-led strategies such as task-shifting, harm reduction methods, opioid agonist treatment, mobile outreach and peer education were introduced, which later influenced national policy.

Mental health system strengthening in Russia [24]

In a partnership between the Institute of Psychiatry, London and the Russian Federal Government, WHO, local municipalities and universities led the implementation of a multifaceted intervention in urban, semi-urban and rural areas of Sverdlovsk oblast. Following a context-specific literature review and situational assessment, interventions implemented over two years incorporated local and government-level policy dialogues, creation of Inter-sectoral Steering Committees (ISCs) to facilitate patients’ access to health care and other services, mental health training for social workers and general medical doctors. In addition, training and technical support were provided to build the capacity of non-government organizations in the fields of advocacy, service provision, and management.

Mental health system strengthening in Kenya and Tanzania [2528]

Two diverse mental health system strengthening programs were implemented in Kenya and Tanzania, each comprising initial country-level situation analysis, local and national multidisciplinary policy dialogue, a system to coordinate mental health services and mental health training and supervision, establishment of clinical guidelines, inter-sectoral partnerships from local to national levels, awareness-raising activities and work to incorporate mental health into general health systems. In Kenya, alongside training of primary care workers, capacity-building courses were delivered to provincial, district, and sub-district general hospitals, to promote the incorporation of mental health into local annual operational plans (which feed into the national annual operational plan), and to foster supervision and coordination of primary mental health care by secondary care psychiatric and public health nurses.

Mental health capacity-building in Nicaragua [29]

In a four year collaboration between the National Autonomous University of Nicaragua in León and the Centre for Addiction and Mental Health in Canada, a needs assessment was conducted for mental health capacity-building. Two international workshops on mental health and addictions focusing on primary care were delivered to clinical staff, academics, and NGO members, after which diploma and masters-level courses were developed, to enhance inter-professional leadership, knowledge exchange, networking, and education.

De-institutionalization of children’s services in Russia [30]

From 2009 to 2012, a two-tier educational program was implemented in the Nizhny Novgorod region of Russia, to establish a model of children’s community services to reduce institutionalization. In the first tier, five week-long seminars delivered a course introducing principles of managing children’s community services with a focus on early intervention and family support to experienced clinical staff motivated to take on roles in service planning. In the second tier, a mentorship and training program was delivered over two years to staff at eight pilot sites and additional staff from five ‘participant sites’, covering the knowledge and skills to lead family support and early intervention programs in children’s services. During this period, four week-long supervisory seminars created opportunities for staff to assess progress, receive service user feedback, and discuss challenges faced.

The International Mental Health Leadership Program (iMHLP) [3133]

Founded in 2001, the capacity-building intervention of iMHLP is embedded within a broader program of mental health system strengthening housed at the University of Melbourne’s Centre for International Mental Health. A collaboration with the University’s Department of Global Health and Social Medicine and Harvard Medical School, iMHLP’s goal is “to contribute to the development of effective mental health systems by providing training and mentoring in leadership”.

The program focuses on training future leaders of mental health systems in LMICs. It comprises a four week intensive workshop, complemented with subsequent provision of mentoring and supervision to participants at home. The program encompasses skills in leadership, policy, mental health budgeting, community service design, human resources, advocacy, human rights, and research aspects. Course attendees undertake a research or service development project, with support from staff at the University of Melbourne and seniors in their home country.

Examples of previous participants include key stakeholders in the development of community mental health systems in both Sri Lanka and the Indonesian province of Aceh, whose health and social care services were severely affected by the 2004 Indian Ocean earthquake and tsunami.

The mental health Leadership and Advocacy Program (mhLAP) [34]

mhLAP represents a collaboration between the Department of Psychiatry, University of Ibadan, Nigeria, the international development organization, CBM International, and the Center for International Health, University of Melbourne. Commencing in 2010, the program’s objectives are “capacity-building for mental health leadership and advocacy” and “development of stakeholder groups with the ability to identify and pursue country-specific mental health service development needs and targets”.

The program comprises an annual two week interactive training course, whose curriculum was influenced by that of the iMHLAP, including research developments in public health and health system development, and stakeholder needs in LMICs. The course covers the “burden of mental, neurological, and substance use disorders, organization of mental health services, evaluation of services, mental health financing, mental health policy and legislation, social determinants of mental health, principles and practice of health promotion, the art of communication, stigma of mental illness, and mental health system reform and strengthening. The course also aims to inform the trainees about the strong reciprocal links that exist, at both personal and national levels, between poverty, social determinants, and mental health.”

In addition, a country facilitator is appointed in each of the participating countries in West Africa. Their role includes performing a baseline assessment of local mental health services, to inform later program evaluations. The role additionally entails building connections between mental health stakeholders and supporting activities identified as their priorities. This proceeds through creation of a National Stakeholders Council (NSC) or support for existing local networks. An annual three day workshop is held in Nigeria for NSC representatives, to facilitate regular monitoring and review, and sharing of experiences from each country’s activities.

The Consortium for NCD prevention and control in sub-Saharan African (CNCD-Africa) [35]

At an international meeting of stakeholders, including representatives from the United States Centers for Disease Control and Prevention (CDC), the Department of Health of England, the International Union for Health Promotion and Education, and World Health Organization, CNCD-Africa was established. Its objectives were to provide a framework for dialogue about NCDs in sub-Saharan Africa, to bring together regional activities in NCD prevention and control, to develop policies, standards, guidelines, and protocols, and mobilize regional resources, to foster networking, partnership, and evaluation of NCD interventions, and identify causative factors and communicate results worldwide. Mental health was included among NCDs and regional capacity-building took place through convening of symposia and meetings, representation of CNCD-Africa at external meetings, creation of two open-access documents on NCD policies in the region, local and international promotion, and awareness raising and networking through social media.

Methods of evaluating capacity-building interventions for mental health policy-makers and planners

The strength of evidence to support any particular capacity-building approach for policy-makers and planners was low. One study used a conceptual framework for evaluation, which included the collection of primary evaluation data [23]. In that case, external and internal evaluation processes were employed to collect complementary data on the success of the program. The external evaluator collected data from documentary analysis of subsequent funding applications, project manager reports, telephone and face-to-face interviews with UK partners and Russian participants. This was combined with an ‘insider’ evaluation influenced by ‘utilization-focused’ approaches [37]. The authors applied a systematic framework used in previous substance misuse research [38] because it was considered to account for the complex nature of cross-cultural education and training of health care staff. The mental health leadership and advocacy (mhLAP) intervention embraced a systematic approach to evaluation, with consideration of numerical process indicators, such as numbers of policy-makers and planners participating in the national stakeholder council, output indicators, such as numbers of advocacy activities by type and outcome indicators, such as preparation of a country-specific situational analysis report [35].

For the other capacity-building interventions, a descriptive account of how the mental health system had been strengthened was given, but with no clear methodology to guide the evaluation. In none of the reports did the evaluation methodology attempt to isolate the impact of the capacity-building intervention from other contextual factors that might also have led to the reported outcomes. For example, when multi-faceted interventions were implemented which targeted both clinical service delivery and capacity-building of policy-makers and planners, efforts were not made to independently assess the impact of capacity-building activities, in isolation.

Impact of capacity-building interventions for mental health policy-makers and planners

The 14 papers included in this narrative review presented evidence of success, including process indicators (e.g. number of people trained within the program), output indicators (e.g. participation of trained people in mental health system strengthening activities) and outcome activities (e.g. indications of increased commitment to mental health care scale-up, through preparation of implementation plans, committing more resources and efforts to support reduction of stigma, discrimination and abuse). Details of the evaluations are presented in Additional file 5. Information about outcome indicators was generally not collected but when presented, was not robustly evaluated. For example, the outcome indicator “evidence of heightened awareness about the salience of improving mental health services and/or reduction in stigmatization” following mhLAP varied from “efforts to improve” one mental health centre in Liberia and “rehabilitation of homeless mentally ill citizens in some states” in Nigeria to “plans to review the mental health legislation are on-going” in Gambia [35]. More general ‘lessons learned’ were more frequently reported, which focused on the importance of building sustainable, high quality relationships and the need for continued mentoring and joint activities to support mental health systems strengthening.

Quality assessment

Quality assessment of quantitative aspects using the EPHPP tool found all included studies to be ‘weak,’ with between 3 and 6 ‘weak’ ratings across 8 domains. Assessment of qualitative aspects found 12 included papers to be of low quality (scoring less than 7 out of 12), and two more recent papers to be of medium quality (scoring between 7 and 9 out of 12: [30, 31]).

Excluded papers

Many papers were excluded because they referred to service level capacity-building, for example, building capacity in health workers to deliver mental health care. In several excluded papers, policy-making and planning activities to strengthen mental health systems were described, for example, raising awareness and advocating for change, establishing a Ministry of Health mental health co-ordination desk or implementing guidelines for reliable drug supplies, but no specific interventions to build the capacity of policy-makers and planners to implement these activities were described [3943]. Similarly, four of the 28 papers identified outside the database search described innovative mental health development work in Latin America and the Caribbean [4447]. Despite detailed documentation of the work’s historical context, political shifts and clinical detail, the lack of description of capacity-building of policy-makers and planners prohibited their inclusion.

In some cases, an outside agency carried out the system strengthening activity [40, 41] and, in other cases, a ‘sustained policy dialogue’ was undertaken [39, 43]. These studies were excluded because the policy dialogues were not described or evaluated. Similarly, WHO developed training materials (including PowerPoint presentations, workshop facilitation guidelines and case materials) to build capacity for mental health policy and service development for LMICs, linked to the 13 modules of its Mental Health Policy and Service Guidance Package [7]. These resources provide practical, step-by-step guidance for developing national mental health policies, information systems and strengthening human resources and can be adapted for training purposes. However, since no formal evaluation of their impact on policy-makers and planners was published, these interventions were not included in this review.

Discussion

This systematic review was prompted by the need for evidence on the most effective approaches to building capacity of mental health policy-makers and planners in LMICs. Health policy and systems research methods as they pertain to LMICs have been a neglected field [48]. Challenges identified include under-funding, fear that health policy and systems research does not yield generalizable findings, poor quality and a lack of alignment between research foci and policy-makers’ needs. The negative impact of a weak evidence base in health systems research on clinical outcomes, such as those specified in the Millennium Development Goals, is not a new observation [49].

Our study demonstrates that a restricted number of capacity-building interventions for policy-makers and planners in LMICs to strengthen mental health systems have been developed, implemented, evaluated and described in published literature. However, the quality of evidence evaluating the effectiveness of the different capacity-building approaches trialled is generally low and little attempt has been made to isolate the impact of the intervention, distinct from other contextual factors. Nevertheless, several features of effective capacity-building (Table 2) and evaluation of such interventions (Table 3) are suggested by this review.
Table 2

Features of effective capacity-building suggested by this review

• Meaningful partnership, in some cases between better and less well-resourced settings.

• Establishing sustainable long-term support networks both between regions of mental health care delivery within a country and between nations, which develop over periods of years.

• Technical and advisory support materials founded on regional and national experience and expertise, to assist with local mental health policy-making and planning.

• Local statutory co-operation with interventions, with opportunities to expand successful models.

• Emphasis on training, education and empowerment of leadership, including through formal qualifications.

• Multidisciplinary investment targeting a range of mental health professionals and stakeholders, including service users, carers and staff in allied sectors such as social services.

Table 3

Recommendations for evaluation of capacity-building interventions suggested by this review

• It would be preferable to implement and evaluate specific interventions in isolation wherever possible, in order to ensure that the impact of individual components of more multifaceted programs can be assessed.

• The use of a systematic evaluation framework incorporating defined process, output and outcome indicators and external reviewers where possible, is recommended.

• It is recommended that intervention programs devise instruments to assess the mental health system strengthening competencies of stakeholders before and after participation.

• Identification of the optimum duration of training for the target group of mental health policy-makers and planners in different LMIC settings is crucial, given the tension between delivering adequate content and feasible roll-out to busy professionals.

• The impact of staff turnover in course leadership positions on the outcome of capacity-building interventions requires further investigation.

• In-depth qualitative evaluation, including use of formal case study methodology, interviews and focus groups with staff who did and did not participate in the intervention, service users, carers and community members may provide valuable insights into mechanisms of impact, but may be more locally relevant.

• Recent proposals that capacity-building interventions should be assessed according to their effect on access to, interaction with and receptivity to research evidence, at the level of the individual, the organization and the institution should be explored with specific reference to mental health policy-makers and planners.

Management and leadership skills for general health systems strengthening, targeting decision-makers in low-income countries, are one area which has begun to be explored, with evidence of effective programs in post-conflict Liberia [50]. The importance of tailoring leadership capacity-building interventions to the stakeholder context was identified by a qualitative study involving health care leaders from Ethiopia, Ghana, Liberia and Rwanda [51]. The authors criticized the largely Western-centric literature on leadership capacity-building for neglecting the historical, political, and socio-cultural aspects of leadership in different countries, with health care leadership in sub-Saharan Africa still less well-studied. It is, perhaps, unsurprising, that capacity-building for mental health leadership in LMICs has been even less thoroughly researched.

This review identified two courses building mental health leadership capacity for participants from nine African [35] and 18 Asian countries [3234]. Process and output indicators supported the favorability and perceived relevance of both leadership courses and reported that delegates went on to engage in mental health strengthening activities in their countries of origin.

Disease-specific health investment can distort national priorities and allocation of staff [15]. As a result, the need for renewed focus on wider health systems strengthening has been increasingly supported [14]. In each capacity-building intervention identified by this review, specific training for policy-makers and planners was embedded within a larger program of activities to support mental health system strengthening. A range of inter-related achievements were reported by each multi-faceted intervention, including staff attitude change [23, 31], increased funding [24, 25] policy change [2629], education and qualifications for stakeholders [30], pursuit of doctoral and other research by participants [3234] and creation of national [35] and international stakeholder councils or consortia [36]. These findings support a ‘systems thinking’ approach, whereby unified interventions collaboratively target health care, education, research and policy-making, acknowledging the dynamic nature of health services as complex adaptive systems [13]. The need for longer-term mentoring and partnership, often between low and high income countries [23, 25, 30, 3235], underpinning shorter-term interventions, built on high quality relationships, emerged as being particularly important for success. Many of the partnerships reported developed over years, usually for time periods dictated by the duration of external funding. While the optimal duration of partnership could not be inferred by this review, the importance of long-term partnership working was evident across the range of published studies.

However, despite yielding demonstrable benefits, the complex systems-wide approach to capacity-building reported by each study rendered isolated evaluation of specific components difficult, especially given uncertainty about which ‘active ingredients’ of the intervention contribute to outcomes. The International Mental Health Leadership Programme (iMHLAP) model prioritized leadership skills, but also provided training in a broad range of topics relevant to mental health system strengthening [32, 33, 35]. Training courses developed for provincial and district-level planners in Kenya and Tanzania [2629] were not well-described, but focused on practical skills in mental health service planning, budgeting and co-ordination. The difficulty associated with presenting evidence for an individual capacity-building intervention for mental health policy-makers and planners epitomizes the neglect of health policy and systems research methods in LMICs [48].

No study presented evaluation findings on the impact of specific capacity-building initiatives upon the mental health system strengthening competencies of participants. Furthermore, the optimal duration of training workshops for this target group remains unclear, having ranged from several days in Kenya and Tanzania to two weeks in West Africa and four weeks in Asia; a balance is required between adequate content and duration and feasible roll-out to busy professionals. Neither was evidence presented for the impact of course accessibility and staff turnover in course leadership positions on the outcomes of such interventions. As has been previously argued, the weakness of health policy and systems research methods in LMICs may compromise the efficacy and longer-term impact of capacity-building programs [49].

The evaluation of capacity-building interventions embedded in larger programatic interventions is inevitably complex. The indicators identified by this review provide potential starting points for more in-depth evaluation. The use of a systematic evaluation framework, as used in the capacity-building intervention targeting substance misuse in Russia, may be less susceptible to bias than more descriptive accounts of program outcomes [23]. Qualitative exploration, incorporating semi-structured interviews with all individuals responsible for policy-making and planning of mental health services, including those who did not participate in specific interventions, may yield insights into the mechanisms by which capacity-building activities work [10].

Training evaluation offers an important process indicator that may help to isolate the immediate impact of capacity-building interventions for mental health policy-makers and planners. Formal case study methodology may also facilitate a systematic, in-depth but contextualised evaluation of impact [352]. A recent study of capacity-building activities to apply evidence from general health research to policy-making in four LMICs proposed a framework for measurement and evaluation [8]. Advocating a more systematic approach to evaluation, the authors propose that capacity-building interventions should be assessed in terms of the extent that they increase access to, interaction with and receptivity to research evidence, with respect to the three levels of the individual, the organization and the institution. For example, increased access to research papers works at the individual level, whereas evidence-based policies work at the institutional level. A similar framework could provide a more systematic approach to evaluating interventions specifically focused on capacity-building of mental health policy-makers and planners.

A potential limitation of our review could be the tendency for reports on capacity-building for this target group of policy-makers and planners to be located within the grey literature, so that we may have missed relevant studies.

Conclusions

Despite international consensus regarding the need for in-depth, system-wide capacity-building interventions for mental health policy-makers and service planners in LMICs, there is a paucity of published research studies and service evaluations in this field. This systematic review identified just fourteen studies describing nine different approaches, focused largely on systems-wide interventions to enhance mental health services, or on leadership courses. All studies shared a broad focus on partnership and high quality relationships, in keeping with the shift in focus from disease-specific investment to wider health system strengthening to build mental health capacity in LMICs. The preliminary evaluations presented here support a positive impact on process and output indicators. However, systematic research and evaluation methods were absent from all studies, compromising the potential to draw conclusions regarding outcomes on mental health system strengthening. This review demonstrates the need for high-quality capacity-building interventions for mental health policy-makers and planners, assessed using rigorous mixed methods to capture both quantitative and qualitative indices, as part of longitudinal evaluation of multi-faceted partnership approaches to mental health system strengthening in LMICs.

Abbreviations

CDC: 

Centers for Disease Control and Prevention

CNCD-Africa: 

Centre for Non-Communicable Disease, Africa

EPHPP: 

Effective public health practice project

iMHLAP: 

International Mental Health Leadership Progam

ISC: 

Inter-sectoral Steering Committee

LMICs: 

Low- and middle-income countries

mhLAP: 

Mental health leadership and advocacy

MNS: 

Mental, neurological and substance abuse

NCD: 

Non-communicable disease

NGO: 

Non-government organization

NSC: 

National Stakeholders Council

WHO: 

World Health Organization

Declarations

Acknowledgements

None.

Funding

The Emerald program is funded by the European Community’s Seventh Framework Program (FP7/2007-2013) under grant agreement no 305968. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

The partner organizations involved in the Emerald program are Addis Ababa University (AAU), Ethiopia; Butabika National Mental Hospital (BNH), Uganda; GABO:mi Gesellschaft für Ablauforganisation :milliarium GmBH & Co. KG (GABO:mi), Germany; HealthNet TPO, Netherlands; King’s College London (KCL), United Kingdom (UK); Public Health Foundation of India (PHFI), India; Transcultural Psychosocial Organization Nepal (TPO Nepal), Nepal; Universidad Autonoma de Madrid (UAM), Spain; University of Cape Town (UCT), South Africa; University of Ibadan (UI), Nigeria; University of KwaZulu-Natal (UKZN), South Africa; and World Health Organization (WHO), Switzerland.

The Emerald program is led by Prof Graham Thornicroft at KCL. The project coordination group consists of Prof Atalay Alem (AAU), Prof José Luis Ayuso-Mateos (UAM), Dr Dan Chisholm (WHO), Dr Stefanie Fülöp (GABO:mi), Prof Oye Gureje (UI), Dr Charlotte Hanlon (AAU), Dr Mark Jordans (HealthNet TPO; TPO Nepal; KCL), Dr Fred Kigozi (BNH), Prof Crick Lund (UCT), Prof Inge Petersen (UKZN), Dr Rahul Shidhaye (PHFI), and Prof Graham Thornicroft (KCL).

Parts of the program are also coordinated by Ms Shalini Ahuja (PHFI), Dr Jibril Omuya Abdulmalik (UI), Ms Kelly Davies (KCL), Ms Sumaiyah Docrat (UCT), Dr Catherine Egbe (UKZN), Dr Sara Evans-Lacko (KCL), Dr Margaret Heslin (KCL), Dr Dorothy Kizza (BNH), Ms Lola Kola (UI), Dr Heidi Lempp (KCL), Dr Pilar López (UAM), Ms Debra Marais (UKZN), Ms Blanca Mellor (UAM), Mr Durgadas Menon (PHFI), Dr James Mugisha (BNH), Ms Sharmishtha Nanda (PHFI), Dr Anita Patel (KCL), Ms Shoba Raja (BasicNeeds, India; KCL), Dr Maya Semrau (KCL), Mr Joshua Ssebunya (BNH), Mr Yomi Taiwo (UI), and Mr Nawaraj Upadhaya (TPO Nepal).

The Emerald program’s scientific advisory board includes A/Prof Susan Cleary (UCT), Prof Derege Kebede (WHO, Regional Office for Africa), Prof Harry Minas (University of Melbourne, Australia), Mr Patrick Onyango (TPO Uganda), Prof Jose Luis Salvador Carulla (University of Sydney, Australia), and Dr R Thara (Schizophrenia Research Foundation (SCARF), India).

The following individuals are members of the Emerald consortium: Dr Kazeem Adebayo (UI), Ms Jennifer Agha (KCL), Ms Ainali Aikaterini (WHO), Dr Gunilla Backman (London School of Hygiene and Tropical Medicine; KCL), Mr Piet Barnard (UCT), Dr Harriet Birabwa (BNH), Ms Erica Breuer (UCT), Mr Shveta Budhraja (PHFI), Amit Chaturvedi (PHFI), Mr Daniel Chekol (AAU), Mr Naadir Daniels (UCT), Mr Bishwa Dunghana (TPO Nepal), Ms Gillian Hanslo (UCT), Ms Edith Kasinga (UCT), Ms Tasneem Kathree (UKZN), Mr Suraj Koirala (TPO Nepal), Prof Ivan Komproe (HealthNet TPO), Dr Mirja Koschorke (KCL), Mr Domenico Lalli (European Commission), Mr Nagendra Luitel (TPO Nepal), Dr David McDaid (KCL), Ms Immaculate Nantongo (BNH), Dr Sheila Ndyanabangi (BNH), Dr Bibilola Oladeji (UI), Prof Vikram Patel (KCL), Ms Louise Pratt (KCL), Prof Martin Prince (KCL), Ms M Miret (UAM), Ms Warda Sablay (UCT), Mr Bunmi Salako (UI), Dr Tatiana Taylor Salisbury (KCL), Dr Shekhar Saxena (WHO), Ms One Selohilwe (UKZN), Dr Ursula Stangel (GABO:mi), Prof Mark Tomlinson (UCT), Dr Abebaw Fekadu (AAU), and Ms Elaine Webb (KCL).

GT is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care South London at King’s College London and King’s Health Partners. The views expressed are those of the author(s) and not necessarily those of the NHS, NIHR or the Department of Health. GT acknowledges financial support from the Department of Health via the National Institute for Health Research (NIHR) Biomedical Research Centre and Dementia Unit awarded to South London and Maudsley NHS Foundation Trust in partnership with King’s College London and King’s College Hospital NHS Foundation Trust. SEL has received consulting fees from Lundbeck.

Availability of data and materials

The datasets supporting the conclusions of this article are included within the article and the Additional files 1, 2, 3, 4 and 5.

Authors’ contributions

CH, MS, SEL, CL and GT conceptualized the review. MS ran the searches. CH, MS, SEL, RK, MT, EC and JA screened abstracts and full texts of papers. RK and CH drafted the paper. RK, CH, MS, MT, SEL, CL, OG, EC, SN, JA, AF, AA and GT contributed to the interpretation of findings. All authors read and approved the final paper.

Competing interests

None.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
South London and Maudsley NHS Foundation Trust, Maudsley Hospital
(2)
Centre for Global Mental Health, King’s College London, Institute of Psychiatry, Psychology and Neuroscience
(3)
Department of Psychiatry, University of Oxford, Warneford Hospital
(4)
Health Service and Population Research Department, Institute of Psychiatry, Psychology & Neuroscience, King’s College London
(5)
PSSRU, London School of Economics and Political Science
(6)
Department of Psychiatry and Mental Health, Alan J Flisher Centre for Public Mental Health
(7)
Department of Psychiatry, College of Medicine, University of Ibadan
(8)
Ministry of Health
(9)
London School of Economics and Political Science, Social Policy Department – LSE Health and Social Care
(10)
College of Health Sciences, School of Medicine, Department of Psychiatry, Sixth Floor, College of Health Sciences Building, Addis Ababa University, Tikur Anbessa Hospital
(11)
College of Health Sciences, School of Medicine, Department of Psychiatry, Addis Ababa University
(12)
Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, Centre for Affective Disorders
(13)
Centre for Global Mental Health and Centre for Implementation Science, King’s College London, Institute of Psychiatry, Psychology and Neuroscience
(14)
Addis Ababa University, College of Health Sciences, School of Medicine, Department of Psychiatry & Centre for Global Mental Health, King’s College London, Institute of Psychiatry, Psychology and Neuroscience

References

  1. Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the global burden of disease study. Lancet. 2013;382:1575–86.View ArticlePubMedGoogle Scholar
  2. Fekadu A, Medhin G, Kebede D, Alem A, Cleare A, Prine M, et al. Excess mortality in severe mental disorders: a 10-year population-based cohort study in rural Ethiopia. Br J Psychiatry. 2015;206:289–96.View ArticlePubMedGoogle Scholar
  3. Prince M, Patel V, Saxena S, Maj M, Maselko J, Phillips MR, et al. No health without mental health. Lancet. 2007;370:859–77.View ArticlePubMedGoogle Scholar
  4. Drew N, Funk M, Tang S, Lamichhane J, Chavez E, Katonka S, et al. Human rights violations of people with mental and psychosocial disabilities: an unresolved global crisis. Lancet. 2011;378:1664–75.View ArticlePubMedGoogle Scholar
  5. Alem A, Kebede D, Fekadu A, Shibre T, Fekadu D, Beyero T, et al. Clinical course and outcome of schizophrenia in a predominantly treatment-naive cohort in rural Ethiopia. Schizophrenia Bull. 2009;35:646–54.View ArticleGoogle Scholar
  6. Jacob KS, Sharan P, Mirza I, Garrido-Cumbrera M, Seedat S, Mari JJ, et al. Mental health systems in countries: where are we now? Lancet. 2007;370:1061–77.View ArticlePubMedGoogle Scholar
  7. WHO Mental Health Policy and Service Guidance Package. http://www.who.int/mental_health/policy/essentialpackage1/en. Accessed 14 June 2016.
  8. Hawkes S, Aulakh BK, Jadeja N, Jimenez M, Buse K, Anwar I, et al. Strengthening capacity to apply health research evidence in policy making: experience from four countries. Health Policy Plann. 2007;31:161–70.View ArticleGoogle Scholar
  9. Draper CE, Lund C, Kleintjes S, Funk M, Omar M, Flisher AJ. Mental health policy in South Africa: development process and content. Health Policy Plann. 2009;24:342–56.View ArticleGoogle Scholar
  10. Bird P, Omar M, Doku V, Lund C, Rogers Nsereko J, Mwanza J, et al. Increasing the priority of mental health in Africa: findings from qualitative research in Ghana, South Africa, Uganda and Zambia. Health Policy Plann. 2011;26:357–65.View ArticleGoogle Scholar
  11. Saraceno B, van Ommeren M, Batniji R, Cohen A, Gureje O, Mahoney J, et al. Barriers to improvement of mental health services in low-income and middle-income countries. Lancet. 2007;370:1164–74.View ArticlePubMedGoogle Scholar
  12. Petersen I, Lund C, Stein DJ. Optimizing mental health services in low-income and middle-income countries. Curr Opin Psychiatr. 2011;24:318–23.View ArticleGoogle Scholar
  13. Paina L, Peters DH. Understanding pathways for scaling up health services through the lens of complex adaptive systems. Health Policy Plann. 2011;27:1–9.Google Scholar
  14. Swanson RC, Cattaneo A, Bradley E, Chunharas S, Atun R, Abbas KM, et al. Rethinking health systems strengthening: key systems thinking tools and strategies for transformational change. Health Policy Plann. 2012;27:iv54–61.View ArticleGoogle Scholar
  15. Sundewall J, Swanson RC, Betigeri A, Sanders D, Collins TE, Shakarishvilli G, et al. Health-systems strengthening: current and future activities. Lancet. 2011;377:1222–3.View ArticlePubMedGoogle Scholar
  16. Semrau M, Lempp H, Keynejad R, Evans-Lacko S, Mugisha J, Raja S, et al. Service user and caregiver involvement in mental health system strengthening in low- and middle-income countries: systematic review. BMC Health Serv Res. 2016;16:79.View ArticlePubMedPubMed CentralGoogle Scholar
  17. Semrau M, Evans-Lacko S, Alem A, Ayuso-Mateos JL, Chisholm D, Gureje O, et al. Strengthening mental health systems in low- and middle-income countries: the Emerald programme. BMC Med. 2015;79.Google Scholar
  18. mhGAP Intervention Guide. http://www.who.int/mental_health/publications/mhGAP_intervention_guide/en. Accessed 14 June 2016.
  19. New Country Classifications. http://blogs.worldbank.org/opendata/new-country-classifications-2016. Accessed 3 Apr 2016.
  20. Thomas BH, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldv Evid-Based Nu. 2004;1:176–84.View ArticleGoogle Scholar
  21. Harden A, Brunton G, Fletcher A, Oakley A. Teenage pregnancy and social disadvantage: systematic review integrating controlled trials and qualitative studies. BMJ. 2009;339.Google Scholar
  22. Mays N, Pope C. Qualitative research in health care: assessing quality in qualitative research. BMJ. 2000;320:50–2.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Green AJ, Holloway DG, Fleming PM. Substance misuse in Russia: A partnership for policy change and service development. Int J Drug Policy. 2000;11:393–405.View ArticlePubMedGoogle Scholar
  24. Lalmuanpuii M, Biangtung L, Mishra RK, Reeve MJ, Tzudier S, Singh AL, et al. Scale-up of a comprehensive harm reduction programme for people injecting opioids: lessons from north-eastern India. B World Health Organ. 2013;91:306–12.View ArticleGoogle Scholar
  25. Jenkins R, Lancashire S, McDaid D, Samyshkin Y, Green S, Watkins J, et al. Mental health reform in the Russian Federation: an integrated approach to achieve social inclusion and recovery. B World Health Organ. 2007;85:858–66.Google Scholar
  26. Jenkins R, Kiima D, Njenga F, Okonji M, Kingora J, Kathuku D, et al. Integration of mental health into primary care in Kenya. World Psychiatry. 2010;9:118–20.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Jenkins R, Kiima D, Okonji M, Njenga F, Kingora J, Lock S. Integration of mental health into primary care and community health working in Kenya: context, rationale, coverage and sustainability. Mental Health Fam Med. 2010;7:37–47.Google Scholar
  28. Kiima D, Jenkins R. Mental health policy in Kenya -an integrated approach to scaling up equitable care for poor populations. Int J Mental Health Syst. 2010;4:19.View ArticleGoogle Scholar
  29. Mbatia J, Jenkins R. Development of a mental health policy and system in Tanzania: an integrated approach to achieve equity. Psychiat Serv. 2010;61:1028–31.View ArticleGoogle Scholar
  30. Sapag JC, Herrera A, Trainor R, Caldera T, Khenti A. Global mental health: transformative capacity building in Nicaragua. Glob Health Action. 2013;6:21328.View ArticleGoogle Scholar
  31. Johnson DE, Dovbyna SV, Morozova TU, Richards MA, Bogdanova JG. From institutional care to family support: Development of an effective early intervention netowrk in the Nizhny Novgorod region, Russian Federation, to support family care for children at risk of institutionalization. Inf Mental Health J. 2014;35:172–84.View ArticleGoogle Scholar
  32. Beinecke RH, Minas H, Goldsack S, Peters J. Global mental health leadership training programmes. Int J Leadership Pub Serv. 2010;6:63–72.View ArticleGoogle Scholar
  33. Kakuma R, Minas H, van Ginneken N, Dal Poz MR, Desiraju K, Morris JE, et al. Human resources for mental health care: current situation and strategies for action. Lancet. 2011;378:1654–63.View ArticlePubMedGoogle Scholar
  34. Minas H. The centre for international mental health approach to mental health system development. Harvard Rev Psychiatry. 2012;20:37–46.View ArticleGoogle Scholar
  35. Abdulmalik J, Fadahunsi W, Kola L, Nwefoh E, Minas H, Eaton J, et al. The Mental Health Leadership and Advocacy Program (mhLAP): a pioneering response to the neglect of mental health in Anglophone West Africa. Int J Mental Health Syst. 2014;8:5.View ArticleGoogle Scholar
  36. Amuyunzu-Nyamongo M, Owuor JO, Blanchard C. The Consortium for NCD Prevention and Control in Sub-Saharan Africa (CNCD-Africa): from concept to practice. Glob Health Promot. 2013;20:97–103.View ArticlePubMedGoogle Scholar
  37. Patton MQ. Utilisation-focussed evaluation. 3rd ed. London: Sage; 1997.Google Scholar
  38. Cranfield S, Stoneman P. Guidelines for good practice in education and training for nurses, midwives and health visitors. 1st ed. London: ENB; 1996.Google Scholar
  39. Jenkins R, Heshmat A, Loza N, Siekkonen I, Sorour E. Mental health policy and development in Egypt. Integrating mental health into health sector reforms 2001 to 9. Int J Mental Health Syst. 2010;4:17.View ArticleGoogle Scholar
  40. Saxena S, van Ommeren M, Saraceno B. Mental health assistance to populations affected by disasters: World Health Organization’s role. Int Rev Psychiatry. 2006;8:199–204.View ArticleGoogle Scholar
  41. Hawkins Z, Tilman T. The family is the clinic, the community is the hospital: Community mental health in Timor-Leste. Australas Psychiatry. 2011;19:S95–7.View ArticlePubMedGoogle Scholar
  42. Ventevogel P, van de Put W, Faiz H, van Mierlo B, Siddiqui M, Komproe IH, et al. Improving access to mental health care and psychosocial support within a fragile context: A case study from Afghanistan. PLoS Med. 2012;9:e1001225.View ArticlePubMedPubMed CentralGoogle Scholar
  43. Jenkins R, Mussa M, Haji SA, Haji MS, Salim A, Suleiman S, et al. Developing and implementing mental health policy in Zanzibar, a low income country off the coast of East Africa. Int J Mental Health Syst. 2011;5:6.View ArticleGoogle Scholar
  44. Caldas de Almeida JM. Estrategias de cooperación técnica de la Organización Panamericana de la Salud en la nueva fase de la reforma de los servicios de salud mental en América Latina y el Caribe. Rev Panam Salud Publ. 2005;18:314–26.View ArticleGoogle Scholar
  45. Caldas de Almeida JM, Cohen A. Innovative mental health programs in Latin America & the Caribbean. Washington, DC: Pan American Health Organization; 2008.Google Scholar
  46. Caldas de Almeida JM, Horvitz-Lennon M. An overview of mental health care reforms in Latin America and the Caribbean. Psychiat Serv. 2010;61:218–21.View ArticleGoogle Scholar
  47. Levav I, Restrepo H, Guerra de Macedo C. The restructuring of psychiatric care in Latin America: A new policy for mental health services. J Public Health Pol. 1994;15:71–85.View ArticleGoogle Scholar
  48. Mills A. Health policy and systems research: defining the terrain; identifying the methods. Health Policy Plann. 2012;27:1–7.View ArticleGoogle Scholar
  49. Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, et al. Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet. 2004;364:900–6.View ArticlePubMedGoogle Scholar
  50. Rowe LA, Brillant B, Cleveland E, Dahn BT, Ramanadhan S, Podesta M, et al. Building capacity in health facility management: guiding principles for skills transfer in Liberia. Hum Resour Health. 2010;8:5.View ArticlePubMedPubMed CentralGoogle Scholar
  51. Curry L, Taylor L, Chen PG, Bradley E. Experiences of leadership in health care in sub-Saharan Africa. Hum Resour Health. 2012;10:33.View ArticlePubMedPubMed CentralGoogle Scholar
  52. Neale P, Thapa S, Boyce C. Preparing a case study: Guide for designing and conducting a case study for evaluation input. Watertown: Pathfinder International; 2006.Google Scholar

Copyright

© The Author(s). 2016