Skip to main content
Download PDF

Research funding impact and priority setting – advancing universal access and quality healthcare research in Malaysia

BMC Health Services Research volume 19, Article number: 248 (2019) Cite this article

Abstract

Background

Health Research Priority Setting (HRPS) in the Ministry of Health (MOH) Malaysia was initiated more than a decade ago to drive effort toward research for informed decision and policy-making. This study assessed the impact of funded prioritised research and identified research gaps to inform future priority setting initiatives for universal access and quality healthcare in Malaysia.

Methods

Research impact of universal access and quality healthcare projects funded by the National Institutes of Health Malaysia were assessed based on the modified Payback Framework, addressing categories of informing policy, knowledge production, and benefits to health and health sector. For the HRPS process, the Child Health and Nutrition Research Initiative methodology was adapted and adopted, with the incorporation of stakeholder values using weights and monetary allocation survey. Workshop discussions and interviews with stakeholders and research groups were conducted to identify research gaps, with the use of conceptual frameworks to guide the search.

Results

Seventeen ongoing and 50 completed projects were identified for research funding impact analysis. Overall, research fund allocation differed from stakeholders’ expectation. For research impact, 48 out of 50 completed projects (96.0%) contributed to some form of policy-making efforts. Almost all completed projects resulted in outputs that contributed to knowledge production and were expected to lead to health and health sector benefits. The HRPS process led to the identification of research priority areas that stemmed from ongoing and new issues identified for universal access and quality healthcare.

Conclusion

The concerted efforts of evaluation of research funding impact, prioritisation, dissemination and policy-maker involvement were valuable for optimal health research resource utilisation in a resource constrained developing country. Embedding impact evaluation into a priority setting process and funding research based on national needs could facilitate health research investment to reach its potential.

Peer Review reports

Background

Health research priority setting (HRPS) is essential to optimise the impact of health systems research investment [1], as the process of setting health research priorities can improve the efficiency of research fund utilisation and reduce duplication [2]. With this approach, a country could identify the health research needed in a transparent and systematic way [3], as well as align research initiatives to current needs [4].

Considerable investment goes into health research yearly, yet the resources required for health research funding far exceed availability. On top of this, Malaysia’s health system continues to face challenges in rising healthcare demands, shift in disease burden from infectious to non-communicable diseases, and in demographic transition [5]. Ultimately, the national health goal is to improve the health of Malaysia’s population by ensuring universal access and quality healthcare (UAQH), as highlighted in Malaysia’s strategic plans for national development (termed as Malaysia Plan (MP)) [6].

In Malaysia, the National Institutes of Health (NIH), a government agency with a network of research institutes under the Ministry of Health (MOH) Malaysia, funds health research initiatives that are aligned with national health priorities. Within the NIH and MOH, there have been ongoing efforts to address the gap between research, action and policy [7].

One of the key efforts was the establishment of HRPS in MOH. In 2011, the HRPS for the tenth MP had identified and prioritised health research areas in the following clusters: health systems, healthy lifestyles, empowerment, burden of disease, health technology and sustainable environment. Under health systems research, priorities were categorised into the domains of health financing and economics, governance, health information, human resources for health and service delivery [7]. These research domains were retained in the HRPS for the eleventh MP in 2017 and were collectively identified as the UAQH cluster [8].

Despite the establishment of HRPS processes in MOH, the impact and implications of research in supporting informed decision and policy-making are minimally known; only research output data such as the number of reports, publications and presentations produced were collected.

It is recognised that research impact assessment is challenging due to the lack of systematic approaches to evaluate research impact, particularly impact on health policy and practice [9]. Various frameworks have been developed to overcome this issue [10], such as the Payback Framework [11], UK Research Excellence Framework (REF 2014) [12], and the Becker Medical Library Model [13]. Among these models, the Payback Framework is one of the most commonly used frameworks [14]. Developed by the Health Economics Research Group (HERG), the framework defines research impacts into five categories [11], that they and others subsequently presented as: knowledge production, benefits to future research and research use, informing policy and product development, health and health sector benefits as well as broader economic benefits [15, 16]. The Payback Framework addresses conceptual issues and is applied to collect, analyse and report data consistently to capture research impacts and outputs, serving as a tool to assist funders and stakeholders to evaluate possible impact from research [17].

This study aimed to assess the impact of funded prioritised research to improve future HRPS and research fund allocation. We used the Payback Framework [11] to assess the impact of research projects, adopted the Child Health and Nutrition Research Initiative (CHNRI) methodology [1] for the identification and determination of research priority areas and used relevant conceptual frameworks for UAQH research gap identification.

Methods

We assessed NIH-funded research projects in the 10th MP, from 2011 to 2015 [18], to inform priority setting for the 11th MP. Projects were included if the research objectives fulfilled research areas under the cluster of UAQH research [7]. A total of 71 projects were identified, and verified using administrative data from NIH and the National Medical Research Register database [19]. From these 71 projects, 67 were included and 4 excluded, as these did not receive direct NIH funding (Fig. 1).

Fig. 1
figure 1

Flow chart of projects funded under Universal Access and Quality Healthcare Research, 10th Malaysia Plan

Full size image

Evaluation of research funding and impact

Projects identified under the cluster of UAQH in the 10th MP were categorised according to the domains of the World Health Organization (WHO) framework [20]: Governance, Health Economics (HE), Human Resources for Health (HRH), Information & Technology (IT) and Service Delivery (SD). Distribution of research funds were analysed based on these domains.

Stakeholder survey

Stakeholders consisted of researchers, decision makers, healthcare providers, research funders and academicians, from government and private sectors. Stakeholders’ perspectives and expectations on each of the five domains’ importance were elicited in the form of percentages. These values were considered as the intended fund allocation. If all domains had equal importance, each would be allocated 20%. However, if some domains were more important than others, the percentage allocated to each domain would differ. The feedback scores from stakeholders were used to compare against the 10th MP NIH research fund allocation. In November 2016, approximately 200 questionnaires were distributed to all participants of the NIH Research Week and a research priority setting workshop, as well as emailed to 85 stakeholders who were invited to the workshop but could not attend. To maintain anonymity, participants were requested to drop off completed questionnaires into collection boxes available at the venue or to respond online.

Key informant interview

We conducted document review and multiple semi-structured interviews with the principal investigator and/or project team member(s) in a workshop setting for project impact assessment (Additional file 1). Out of the 67 projects that were reviewed, 50 were included for impact analysis as the remaining 17 were ongoing projects. The analysis focused on three impact categories, which were informing policy, knowledge production and benefits to health and health sector, adopting the modified Payback Framework by Kwan et al. [11, 21] and the Canadian Institutes of Health Research [9, 16, 22]. The impact category of benefits to health and health sector was set out to show benefits that could or were expected to result in improved service delivery, cost savings, improved health or increased equity [21].

Data on funds awarded to each project were obtained from NIH administrative records, and divided into tertiles [21], namely low (<RM32,600 (€6936)), moderate (RM32,601–87,000 (€6936-18,510)) and high (>RM87,000 (€18,510)), using the exchange rate in April 2017 of €1 = RM4.70 [23].

Health research priority setting process

Identification of research gaps for the 11th Malaysia Plan

Health research priority areas of the 10th MP [7], national and global health plans [6, 18, 24, 25] were reviewed to identify national level research needs. We reviewed literature for conceptual frameworks to guide the identification of research gaps, and to increase comprehensiveness and inclusivity for each domain. Research gaps were grouped according to the domains of the WHO framework [20].

Research priority setting

We adopted and adapted the CHNRI methodology [1], a frequently used method to gain consensus on research areas. This involved setting criteria for prioritisation, incorporating stakeholders’ values through the application of weights, scoring research gaps, and calculating final scores for initial prioritisation. Stakeholders deliberated on this and identified the revised prioritised areas for the 11th MP.

Setting criteria for prioritisation

In setting criteria for prioritisation, we reviewed literature [1, 3, 26,27,28] and selected three (i.e. answerability/feasibility, importance/potential impact and magnitude/severity), in which each consisted of three questions (Additional file 2) chosen based on applicability, prior use in previous HRPS [7] and consensus from the project team. These criteria were applied in the stakeholders’ values instrument and research gaps assessment.

Incorporating stakeholders’ values

The stakeholder survey instrument contained a section on stakeholders’ values regarding prioritisation (Additional file 3). We incorporated these values as weights for the criteria, adopting the method of allocating monetary value [1, 3]. Stakeholders were asked to rate the importance of each criterion by allocating RM100 across the three criteria. For example, if all criteria had equal importance, each criterion should be allocated RM33.33. However, if one was more important, the allocation could be different. For example, the allocation could be as follows: answerability/feasibility criterion = RM65, importance/potential impact criterion = RM15 and magnitude/severity criterion = RM20. Regardless of research area, the total amount allocated to all criteria should add up to RM100.

Scoring of research gaps

In scoring each research area/gap, we asked the project team, predominantly researchers, to assess each area/gap by answering the questions for the criteria (see Additional file 2). We awarded 1 point to answers indicating agreement, 0.5 points for neither agreement nor disagreement, and 0 points for disagreement.

Calculation of scores for prioritisation

Summation of scores from the three criteria resulted in an intermediate score for each research area/gap. The stakeholders’ values in the form of monetary allocation for each criterion were incorporated as weighted mean score, representing the values of non-research expert groups. Thus, each research area’s final score consisted of the collective inputs of both researchers and stakeholders, similar to the method described by Rudan et al. [1] and was used for ranking. Further information on the calculation of weights to reflect stakeholders’ values can be found in Additional file 4.

Statistical analysis

All statistical analyses were conducted using Statistical Package for the Social Science Version 21. The One-Sample Wilcoxon Signed Rank Test was conducted to test the association between stakeholders’ values for the three criteria and hypothetical values.

Results

Research funding and impact

Among the five research domains, differences exist in the distribution of research funds between the actual fund allocated and stakeholders’ expectations, with the lowest allocation for IT (4.6%) and the highest for SD (33.5%), likely due to the number of research projects in the respective domains (Table 1).

Table 1 Distribution of research fund allocation versus stakeholders’ expectations
Full size table

Response rate was 100% for key informant interview. Of the 50 completed projects, 36.0% (n = 18) supported decision and policy-making. Another 60.0% (n = 30) were reported to have led to policy-maker engagement and/or future agenda setting. Two projects in the lowest tertile reported no policy-maker engagement (Table 2).

Table 2 Fund awarded for projects by policy impact level
Full size table

As an indicator of knowledge production impact, there were 148 presentations, 33 reports, 22 research highlights and 8 publications produced. Evaluation of impact under the category of benefits to health and health sector showed that 36 projects expected to see an impact related to quality of care, while 17 expected health system delivery benefits (Table 3).

Table 3 Knowledge production and expected benefits to health and health sector
Full size table

Health research priority setting process

Out of 150 respondents in the stakeholder survey, 63.3% were researchers/academicians, 16.0% were decision makers and the remaining 20.7% were healthcare providers and others. Majority (92.0%) were from MOH, while the remaining were from public/private universities and other ministries. Most had more than 3 years of experience in their designated positions. Stakeholders placed higher values on answerability and importance than magnitude (Table 4).

Table 4 Stakeholders’ values on criteria for priority setting
Full size table

Research gaps and priorities

The outcome of the HRPS process was a comprehensive list of research gaps for UAQH with topics ranked and prioritised. An example of this is in Additional file 5, and the complete prioritised areas for UAQH health research document is available elsewhere [8]. In general, the list of priority areas for each domain comprised of new and ongoing UAQH issues, research scope, research sub-domain, gaps and needs with the associated rationale. New research areas include transparency in health planning and delivery process, readiness/utilisation of IT as well as transparency and sustainability in health financing.

Discussion

This study shed light into the impact of UAQH research funded in 2011–2015 in Malaysia. The concentration of funding in HE and SD domains was evident, likely due to the large number of priority areas identified in the HRPS for 10th MP in those domains. Almost all completed projects achieved some measure of policy impact and had outputs for knowledge production. For fund allocation, stakeholders deemed the criteria of answerability and importance to be more valuable than magnitude and affirmed that SD should have a larger share of the pie. The use of multipronged approach of conceptual frameworks, literature search, stakeholder engagement and research gaps review from previous research efforts led to a more comprehensive list of research areas for future funding.

Research funding is driven by many factors, such as the interests of research funders and stakeholders. Often, it is unguided by research priorities, causing most funded research to have little contribution to health systems and policy [29]. For example, in Mexico, most health research did not contribute to health policies as many projects were funded without priority setting [30]. The need to set priorities for health research for effective fund utilisation is crucial, in view of the potential for health systems research to contribute towards health system strengthening for universal health coverage and quality healthcare [31].

The funded amount was not a prominent factor in determining research evidence uptake for MOH policy-making, consistent with the results from Wooding et al. [15]. Most funded projects achieved some level of policy impact, suggesting that disseminated research on prioritised areas with policy-maker engagement could increase likelihood of research evidence integration into policy. Literature shows that despite significant lag time for translation into policy, policy-maker engagement, needs-led research and dissemination could contribute towards evidence uptake [9, 32,33,34,35]. However, we did not compare research impact of prioritised with non-prioritised research.

The involvement of a wide range of stakeholders in the HRPS process provided insight into the value stakeholders placed on UAQH research. Additionally, it fostered process legitimacy [3]. Research priorities that correspond to the needs of funders and those who could benefit from research outputs improves the overall credibility and potential health impact [36]. The HRPS process employed in MOH Malaysia provided a means for effective stakeholder engagement in research priority setting.

The fund allocation survey revealed stakeholders’ preference on research domains, although these differed from the actual fund awarded to each domain. This is in line with the notion that research funding practices are not only influenced by perceptions and research interests, but by other factors such as evidence needs, research gaps and prioritised areas [29, 37]. The crux is to achieve a balance between research domain importance and equitable research funding.

Research gaps and priority areas identified in the HRPS process were based upon issues in developing and sustaining efforts to improve UAQH in Malaysia. From 2011 to 2015, there was a heavier emphasis on research under SD and it continues to be of high importance, likely due to demand, number of research gaps, and conversely, reflective of capacity strengthening needs in other domains. For example, substantial research gaps continue to exist under the domain of IT as research initiatives are constrained by inadequate IT knowledge, skill, and capacity [38], as well as budget.

Although priority setting is extremely complex and challenging with the large number of competing research ideas for limited available funding [39], this initiative shows promising evidence that HRPS undertaken for national health research priorities in a developing country could foster better health system outcomes. The stepwise approach in our methodology was beneficial in generating priorities for distinct health research areas, compared with other priority setting processes in low or middle-income countries that addressed broad areas or specified populations/domains such as child health and mental health [4, 40]. It is hoped that this experience could prove beneficial for other developing countries.

However, research projects assessed included only those funded by NIH. Excluded were projects with grants from other sources, either local or international, or funded through operational budgets. Additionally, this study specifically assessed projects grouped under the UAQH research cluster, and hence findings may not be applicable for other MOH research funding initiatives such as funding for environmental health, non-communicable diseases or burden of disease [7]. We did not attempt to identify the economic or capacity building benefits of NIH-funded health research due to data unavailability, but it could form part of a future research for a more holistic measurement of research impact. Policy-maker engagement availability was studied; future exploration into degree of engagement and its benefits would be valuable for funders and applicants to increase impact.

Although response rate was 100% with key project leader involvement, the inherent subjectivity of self-reporting and timing of impact assessment could still affect results. However, Hanney et al. [32] noted that on average, self-reported data did not seem to over-emphasise impact. This study considered all projects equally, irrespective of methodological quality, project size and impact significance, which limits and complicates efforts to draw lessons from the overall impact assessment [32]. Despite this, research impact assessment is still essential to build evidence to demonstrate the return on investment of research funds.

We strived for stakeholder inclusivity, similar to other HRPS efforts [3, 36], but it proved to be a challenge. Resource intensive, time sensitive, contextual and cyclical in nature, research priority setting is difficult yet beneficial for a developing country. Future research could assess the degree of knowledge translation strategies employed, extent of incorporation of national priorities into research agenda and fund allocation. Additionally, evaluation of HRPS process and the extent of achievement of desired outcomes could inform future HRPS initiatives [36, 41].

Conclusion

The concerted efforts of evaluation of research funding impact, prioritisation, dissemination and policy-maker involvement were valuable for optimal health research resource utilisation in a resource constrained developing country. The assessment of research impact provided evidence to inform future priority setting, and the combination of priority setting approach with stakeholder engagement likely contributed to research impact. Despite modest funding and early phase of post-research completion, a fairly good level of benefit was noted for decision and policy making, with health and health sector gains expected, especially in quality of care and health system delivery. It is undeniable that embedding impact evaluation into a priority setting process and funding research based on national needs would facilitate health research investment to reach its potential.

Abbreviations

CHNRI:

Child Health and Nutrition Research Initiative

HE:

Health Economics

HRH:

Human Resources for Health

HRPS:

Health Research Priority Setting

IT:

Information and Technology

MOH:

Ministry of Health Malaysia

MP:

Malaysia Plan

NIH:

National Institutes of Health

SD:

Service Delivery

UAQH:

Universal Access and Quality Healthcare

WHO:

World Health Organization

References

  1. Rudan I, Gibson JL, Ameratunga S, El Arifeen S, Bhutta ZA, Black M, Black RE, Brown KH, Campbell H, Carneiro I, et al. Setting priorities in global child health research investments: guidelines for implementation of the CHNRI method. Croat Med J. 2008;49(6):720–33.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Fleurence RL, Togerson DJ. Setting priorities for research. Health Policy. 2004;69(1):1–10.

    Article  PubMed  Google Scholar 

  3. Kapiriri L, Tomlinson M, Chopra M, El AS, Black RE, Rudan I. Setting priorities in global child health research investments: addressing values of stakeholders. Croat Med J. 2007;48(5):618–27.

    PubMed  PubMed Central  Google Scholar 

  4. McGregor S, Henderson KJ, Kaldor JM. How are health research priorities set in low and middle income countries? A systematic review of published reports. PLoS One. 2014;9(10):e108787.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Non-Communicable Disease (NCD) Section. National Strategic Plan for Non-Communicable Disease (NSPNCD) 2016–2025. Putrajaya: Ministry of Health Malaysia; 2016.

    Google Scholar 

  6. Economic Planning Unit. Eleventh Malaysia Plan, 2016–2020: Anchoring growth on people. Putrajaya: Economic Planning Unit; 2015.

    Google Scholar 

  7. Haniza MA, Sararaks S, Asmaliza I, Roslinda AS. Health Research Priorities in Malaysia for the 10th Malaysia Plan (2011 – 2015). Kuala Lumpur: National Institutes of Health; 2011. http://nih.gov.my/web/wp-content/uploads/2014/11/HRPS10_20062012.pdf. Accessed 30 June 2017.

  8. Sararaks S, Fun WH, Tan EH, Chong DWQ, Siti Haniza M, editors. Advancing Universal Access & Quality Healthcare as a Health Research priority under the 11th Malaysian Plan (2018-2020). Shah Alam: Institute for Health Systems Research and Institute for Health Management; 2017. http://www.ihsr.moh.gov.my/images/publication_material/HRPS_Document.pdf. Accessed 10 May 2018.

  9. Banzi R, Moja L, Pistotti V, Facchini A, Liberati A. Conceptual frameworks and empirical approaches used to assess the impact of health research: an overview of reviews. Health Res Policy Syst. 2011;9(26).

  10. Rivera SC, Kyte DG, Aiyegbusi OL, Keeley TJ, Calvert MJ. Assessing the impact of healthcare research: a systematic review of methodological frameworks. PLoS Med. 2017;14(8):e1002370.

    Article  Google Scholar 

  11. Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996;1(1):35–43.

    Article  CAS  PubMed  Google Scholar 

  12. Higher Education Funding Council for England. REF 2014: assessment framework and guidance on submissions. Bristol: HEFCE; 2011. http://www.fapesp.br/avaliacao/manuais/ref_guidelines.pdf. Accessed 3 June 2018.

  13. Sarli CC, Dubinsky EK, Holmes KL. Beyond citation analysis: a model for assessment of research impact. J Med Libr Assoc. 2010;98(1):17–23.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Greenhalgh T, Raftery J, Hanney S, Glover M. Research impact: a narrative review. BMC Med. 2016;14(78).

  15. Wooding S, Hanney S, Buxton M, Grant J. Payback arising from research funding: evaluation of the arthritis research campaign. Rheumatology. 2005;44(9):1145–56.

    Article  CAS  PubMed  Google Scholar 

  16. Panel on Return on Investment in Health Research. Making an impact: a preferred framework and indicators to measure returns on investment in health research. Ottawa: Canadian Academy of Health Sciences; 2009.

  17. Hanney SR, Watt A, Jones TH, Metcalf L. Conducting retrospective impact analysis to inform a medical research charity’s funding strategies: the case of Asthma UK. Allergy Asthma Clin Immunol. 2013;9(1):17.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Economic Planning Unit. Tenth Malaysia Plan (2011–2015). Putrajaya: Economic Planning Unit; 2010.

    Google Scholar 

  19. National Medical Research Register. Malaysia: National Institute of Health Malaysia; 2006. https://www.nmrr.gov.my/fwbLoginPage.jsp. Accessed 30 June 2017.

  20. de Savigny D, Adam T. Systems thinking for health systems strengthening. Geneva: Alliance for Health Policy and Systems Research, World Health Organization; 2009. https://apps.who.int/iris/bitstream/handle/10665/44204/9789241563895_eng.pdf;jsessionid=1BADF474A09A6A9BB2ED18F73F82EB36?sequence=1. Accessed 30 June 2017.

  21. Kwan P, Johnston J, Fung AY, Chong DS, Collins RA, Lo SV. A systematic evaluation of payback of publicly funded health and health services research in Hong Kong. BMC Health Serv Res. 2007;7(1):121.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Canadian Institutes of Health Research. Developing a CIHR framework to measure the impact of health research Ottawa: Canadian Institutes of Health Research; 2005. http://publications.gc.ca/collections/Collection/MR21-65-2005E.pdf. Accessed 4 June 2018.

  23. Central Bank of Malaysia. Exchange Rates: Central Bank of Malaysia; 2017. http://www.bnm.gov.my/index.php?ch=statistic&pg=stats_exchangerates&lang=en&StartMth=1&StartYr=2017&EndMth=12&EndYr=2017&sess_time=1700&pricetype=Mid&unit=rm. Accessed 20 April 2017.

  24. Ministry of Health Malaysia. MOH Strategic Plan (2016–2020). Putrajaya: Health Plan and Policy Planning Unit, Planning Division, Ministry of Health Malaysia; 2016.

    Google Scholar 

  25. United Nations. Transforming our world: the 2030 Agenda for Sustainable Development: United Nations; 2015. https://sustainabledevelopment.un.org/post2015/transformingourworld/publication. Accessed 5 June 2018.

  26. Wazny K, Sadruddin S, Zipursky A, Hamer DH, Jacobs T, Kallander K, Pagnoni F, Peterson S, Qazi S, Raharison S, et al. Setting global research priorities for integrated community case management (iCCM): results from a CHNRI (child health and nutrition research initiative) exercise. J Glob Health. 2014;4(2):020413.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Rudan I, El Arifeen S, Bhutta ZA, Black RE, Brooks A, Chan KY, Chopra M, Duke T, Marsh D, Pio A. Setting research priorities to reduce global mortality from childhood pneumonia by 2015. PLoS Med. 2011;8(9):e1001099.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Rudan I, Chopra M, Kapiriri L, Gibson J, Ann LM, Carneiro I, Ameratunga S, Tsai AC, Chan KY, Tomlinson M, et al. Setting priorities in global child health research investments: universal challenges and conceptual framework. Croat Med J. 2008;49(3):307–17.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Ali N, Hill C, Kennedy A, IJsselmuiden C. COHRED Record paper 5: what factors influence national health research agendas in low and middle income countries?: Council on Health Research for Development (COHRED) 2006. http://www.cohred.org/downloads/cohred_publications/rp5.pdf. Accessed 30 June 2018.

  30. Martínez-Martínez E, Zaragoza ML, Solano E, Figueroa B, Zúñiga P, Laclette JP. Health Research funding in Mexico: the need for a long-term agenda. PLoS One. 2012;7(12):e51195.

    Article  PubMed  PubMed Central  Google Scholar 

  31. World Health Organization. The world health report 2013: research for universal coverage. Geneva: World Health Organization; 2013. https://apps.who.int/iris/bitstream/handle/10665/85761/9789240690837_eng.pdf;jsessionid=EC056580318B33FF78C3BC9A83B69DAB?sequence=2. Accessed 15 June 2018.

  32. Hanney S, Greenhalgh T, Blatch-Jones A, Glover M, Raftery J. The impact on healthcare, policy and practice from 36 multi-project research programmes: findings from two reviews. Health Res Policy Syst. 2017;15(1):26.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Canadian Institutes of Health Research. Guide to knowledge translation planning at CIHR: Integrated and end-of-grant approaches. Ottawa: Canadian Institutes of Health Research; 2012. http://www.cihr-irsc.gc.ca/e/documents/kt_lm_ktplan-en.pdf. Accessed 15 June 2018

    Google Scholar 

  34. Henderson J, Sword W, Niccols A, Dobbins M. Implementing stakeholder-informed research in the substance abuse treatment sector: strategies used by connections, a Canadian knowledge translation and exchange project. Subst Abuse Treat Prev Policy. 2014;9(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Camden C, Shikako-Thomas K, Nguyen T, Graham E, Thomas A, Sprung J, Morris C, Russell DJ. Engaging stakeholders in rehabilitation research: a scoping review of strategies used in partnerships and evaluation of impacts. Disabil Rehabil. 2015;37(15):1390–400.

    Article  PubMed  Google Scholar 

  36. Viergever RF, Olifson S, Ghaffar A, Terry RF. A checklist for health research priority setting: nine common themes of good practice. Health Res Policy Syst. 2010;8(1):36.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Anderson S, Allen P, Peckham S, Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Res Policy Syst. 2008;6(1):7.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Atun R, Berman P, Hsiao W, Myers E, Yap W. Malaysia Health Systems Research Volume 1: Contextual Analysis of the Malaysian Health Systems, March 2016. Putrajaya: Ministry of Health Malaysia and Harvard School of Public Health; 2016. https://www.researchgate.net/publication/320799211_Malaysia_Health_System_Research_MHSR_Volume_1. Accessed 30 June 2017.

  39. Yoshida S. Approaches, tools and methods used for setting priorities in health research in the 21(st) century. J Glob Health. 2016;6(1):010507.

    PubMed  Google Scholar 

  40. Mihalopoulos C, Carter ROB, Pirkis J, Vos T. Priority-setting for mental health services. J Ment Health. 2013;22(2):122–34.

    Article  PubMed  Google Scholar 

  41. Mador RL, Kornas K, Simard A, Haroun V. Using the nine common themes of good practice checklist as a tool for evaluating the research priority setting process of a provincial research and program evaluation program. Health Res Policy Syst. 2016;14(1):22.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Bermudez LG, Williamson K, Stark L. Setting global research priorities for child protection in humanitarian action: results from an adapted CHNRI exercise. PLoS One. 2018;13(8):e0202570.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the Director-General of Health, Malaysia for permission to publish this paper. The authors also thank all who were involved in the workshops and round-table discussions. We would like to acknowledge the contribution from the following team members of the Health Research Priority Setting for the 11th Malaysia Plan, from the NIH: Ainul Nadziha Mohd Hanafiah, Fathullah Iqbal Ab. Rahim, Mohd Fairuz bin Abdul Razak, Mohd Idris Omar, Mohd. Ridzwan Shahari, Mohd. Shaiful Jefri bin Mohd. Nor Sham, Mohd. Lutfi Fadil Lokman, Mohd. Najib Baharuddin, Nor Haniza Zakaria, Noraziani Khamis, Norazlin Muharam, Noriah Bidin, Nur Amalina Zaimi, Nurul Salwana, Nurul Zawani Zaini, Shakirah Md. Sharif, Sudharshana Mahaletchumy, Suhana Jawahir, Sunita Shanmugam, Zalilah Abdullah, and Zulkarnain Abdul Karim.

Funding

No direct funding was received for this study.

Availability of data and materials

The data that supports the findings of this study are available from Ministry of Health Malaysia, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data is available from the authors upon reasonable request and with permission from Ministry of Health Malaysia.

Author information

Authors and Affiliations

  1. Institute for Health Systems Research, Ministry of Health Malaysia, Shah Alam, Selangor, Malaysia

    Weng Hong Fun, Sondi Sararaks, Ee Hong Tan, Kar Foong Tang, Diane Woei Quan Chong, Lee Lan Low & Siti Haniza Mahmud

  2. National Institutes of Health Secretariat, Ministry of Health Malaysia, Shah Alam, Selangor, Malaysia

    Roslinda Abu Sapian & S. Asmaliza Ismail

  3. Department of Parasitology, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia

    Suresh Kumar Govind

  4. Office of Deputy Director General of Health, Research and Technical Support, Ministry of Health Malaysia, Putrajaya, Malaysia

    Shahnaz Murad

Authors
  1. Weng Hong Fun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sondi Sararaks
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ee Hong Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kar Foong Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Diane Woei Quan Chong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lee Lan Low
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Roslinda Abu Sapian
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. S. Asmaliza Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Suresh Kumar Govind
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Siti Haniza Mahmud
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Shahnaz Murad
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SS, WHF, EHT, LLL, DWQC, RAS, SAI, SKPG, SHM and SM contributed to the conceptualisation and design of the study. WHF, SS, EHT, LLL, DWQC and KFT analysed and interpreted data from project interviews and the research priority setting exercise. WHF conducted the statistical analyses. SS and WHF wrote the first draft of the manuscript with contributions from EHT and KFT. All authors contributed to the revision of manuscript, have read and approved the final manuscript.

Corresponding author

Correspondence to Weng Hong Fun.

Ethics declarations

Ethics approval and consent to participate

This was a Monitoring and Evaluation (M&E) exercise conducted by the fund-governing agency to assess organisational performance and funding impact within Ministry of Health Malaysia. In this exercise, the funder required detailed feedback from recipients of the research fund. Stakeholders were solicited as part of the engagement process to address their research needs. The undertaking of CHNRI exercises do not usually involve the request for formal ethics review as it does not involve individual sensitive data [42] and this was practised in Malaysia. This M&E was registered in the National Medical Research Register of Malaysia [19] (NMRR-18-1078-42046).

Consent for publication

Not applicable.

Competing interests

SS, RAS, SAI, SHM and SM were involved in NIH Malaysia’s fund governance. All remaining authors confirm they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Key informant interview form. This form was used in the key informant interview. (DOCX 14 kb)

Additional file 2:

Items within each criterion used for priority setting. This table describes the three criteria and their associated questions chosen to evaluate health research areas identified in the HRPS process. (DOCX 21 kb)

Additional file 3:

Survey on Health Research Priority Setting: Values & Importance. This survey form was developed to elicit stakeholders’ values on research criteria and importance of each research domain. (DOCX 230 kb)

Additional file 4:

Example of weight assigned for each criterion. This table describes the method used to calculate the average weight for each criterion, that takes into consideration the scores allocated by all stakeholders involved. (DOCX 18 kb)

Additional file 5:

Examples of research priority areas and research gaps for UAQH in Malaysia. This table shows examples of UAQH research priority areas across five WHO domains and the associated research gaps and expected outcomes that were identified in the HRPS for 11th MP. (DOCX 17 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fun, W.H., Sararaks, S., Tan, E.H. et al. Research funding impact and priority setting – advancing universal access and quality healthcare research in Malaysia. BMC Health Serv Res 19, 248 (2019). https://doi.org/10.1186/s12913-019-4072-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-019-4072-7

Keywords

  • Universal access
  • Quality healthcare
  • Payback framework
  • Health research priority setting
  • Research priority areas
  • Research impact
  • Stakeholder values
  • Stakeholder engagement
  • Research funding

BMC Health Services Research

ISSN: 1472-6963

Contact us