Skip to main content
Download PDF

Identifying competencies for integrated knowledge translation: a Delphi study

BMC Health Services Research volume 21, Article number: 1181 (2021) Cite this article

Abstract

Background

Considerable progress has been made to advance the field of knowledge translation to address the knowledge-to-action gap in health care; however, there remains a growing concern that misalignments persist between research being conducted and the issues faced by knowledge users, such as clinicians and health policy makers, who make decisions in the health care context. Integrated knowledge translation (IKT) is a collaborative research model that has shown promise in addressing these concerns. It takes advantage of the unique and shared competencies amongst researchers and knowledge users to ensure relevance of the research process and its outcomes. To date, core competencies have already been identified to facilitate training in knowledge translation more generally but they have yet to be prioritized for IKT more specifically. The primary aim of this study was to recruit a group of researchers and knowledge users to identify and prioritize core competencies for researchers and knowledge users to engage with IKT.

Methods

We recruited health care knowledge users (KUs) and researchers with experience and knowledge of IKT for a quantitative, cross-sectional study. We employed a modified Delphi approach consisting of three e-survey rounds to establish consensus on competencies important to IKT for KUs and researchers based on mean rating of importance and agreement between participants.

Results

Nineteen (73%) of the initial 26 participants were researchers (response rate = 41% in the first round; retention in subsequent rounds > 80%). Participants identified a total of 46 competencies important for IKT (18 competencies for KUs, 28 competencies for researchers) under 3 broad domains. Technical research skills were deemed extremely important for researchers, while both groups require teamwork and knowledge translation skills.

Conclusions

This study provides important insight into distinct and overlapping IKT competencies for KUs and researchers. Future work could focus on how these can be further negotiated and contextualized for a wide range of IKT contexts, projects and teams. Greater attention could also be paid to establishing competencies of the entire team to support the research co-production process.

Peer Review reports

Background

Over the last few decades, considerable progress has been made to advance the field of knowledge translation to address the knowledge-to-action gap in health care. Among these advances is the increasing recognition that challenges with the uptake of evidence into health care policy and practice are not only attributed to underutilization of research evidence, but may also stem from failures in the knowledge production process. Scholars have voiced a growing concern that misalignments may exist between the research conducted and the issues faced by knowledge users, such as clinicians and policy makers, who make decisions in the health care context [1,2,3,4]. These incongruences have led to views that research results are inaccessible and have limited applicability to everyday clinical practice.

Integrated knowledge translation (IKT) is a collaborative research model that has shown promise in addressing the aforementioned views [3, 4]. This approach involves researchers and knowledge users (clinicians, managers, policy makers, etc.) “engaging in a mutually beneficial research project or program of research” wherein knowledge users identify a problem and have the authority to implement the research recommendations [5, 6]. Thus, the IKT approach considers researchers and knowledge users as co-producers of knowledge [7,8,9,10,11].

Research co-production has been described as “a collaborative endeavour of academic and non-academic actors” [12], characterized by “a space where science intersects with non-science” [13]. Academic and non-academic participants in the research endeavour interact throughout the research process to co-produce knowledge rather than only interacting during dissemination of the research results. The nature of such collaborations is sustained for a period of time, and roles and responsibilities between researcher and knowledge user are at times necessarily indistinct. As such, IKT approaches take advantage of the unique as well as shared competencies amongst researchers and knowledge users [14, 15]. Consequently, the knowledge that is co-produced through these collaborative efforts is more likely to be contextually-relevant and therefore applied in health care practice and policy.

Although the ideals of research co-production are not new, conceptualization of the researcher and knowledge user roles within the research process requires greater attention, especially with respect to how capacity can be built to improve and sustain the practice of IKT. For example, since ongoing collaborative relationships between researcher and knowledge user is a key feature of the IKT process [16], it follows that both groups may need to develop competence in forming, contributing, and sustaining collaborative working relationships for the purpose of co-producing research.

A range of enabling conditions have been identified to support organizations to succeed in IKT. Categorized as organizational, professional and individual-level conditions, these constitute a framework by which organizations can develop and monitor their capacity for IKT [17]. Of the individual-level conditions identified, acquisition of adequate IKT-specific knowledge and skills, or competence, were deemed as essential to the IKT process. To date, core competencies have already been identified to facilitate training in knowledge translation more generally [18]; however competencies have yet to be prioritized for research co-production more specifically. What remains to be clarified is the extent to which the competencies identified by Mallidou et al. [18] pertain to either or both researchers and knowledge users.

The primary aim of this study was to recruit a group of researchers and knowledge users to identify and prioritize core competencies for researchers and knowledge users to engage with IKT.

Methods

We used a modified Delphi approach [19, 20] to identify and prioritize a set of IKT competencies for the two key groups of stakeholders involved in the research process: knowledge users (KU) and researchers. The Delphi technique is an established and widely used methodology in healthcare research that enables consensus building among individuals with pertinent experience and potentially diverse views [21]. Results from Delphi studies are considered data generated at a single point in time for a particular group of individuals, and are therefore intended for further testing and verification to ultimately inform theory and practice [19].

Participants and recruitment

We considered two stakeholder groups to be relevant in this study: knowledge users (KUs) and researchers who are familiar and experienced with partnered research using an IKT approach. We define a KU as “an individual who is likely to be able to use the knowledge generated through research to make informed decisions about health policies, programs and/or practices” [22]. Knowledge users in the health care system, such as organizational or system-level decision-makers in health care settings (clinical managers, health care managers, policy-makers), and point of care health care providers, were eligible to participate. Individuals who were a KU or researcher and who had worked in the health sector for at least 2 years were considered eligible. Potential study participants were identified through the Integrated Knowledge Translation Research Network (IKTRN), and through the research team’s professional network. The IKTRN is a network of “knowledge users and researchers committed to studying, teaching and practicing integrated knowledge translation” in Canada [23] or are known to the research team to have experience in the field. Given the uniqueness of the patient perspectives with different health conditions and healthcare settings, and the growing focus specifically on patient engagement in health research, we opted to focus this study exclusively on clinician and health system knowledge users and therefore did not include patients as study participants. Since sample sizes for Delphi panels remain variable [19, 24], we prioritized the need for participants to reflect the full range of KU and researcher stakeholders to optimize the credibility and acceptability of the Delphi results [24], with an ultimate pragmatic target sample size of 20–25 panelists.

Following a modified Dillman approach for recruitment [25], we sent a pre-notice email prior to the first invitation to participate in round one of the survey, followed by two thank you/reminder emails and a final reminder email for each survey round. Sending multiple email reminders is an appropriate method of increasing response rates in Delphi studies [26].

Ethical considerations

Participants demonstrated implied consent by responding to the online questionnaire and provided their email address if they agreed to be contacted in subsequent rounds. The University of Toronto Research Ethics Board approved the study protocol.

Modified Delphi methodology

Our modified Delphi methodology was established a priori and consisted of three rounds, which is considered to be the optimal number of rounds [21]. We asked participants to rate the importance of competencies needed by KUs and researchers who are engaging in IKT on a five-point ordinal scale (Not Important, Slightly Important, Important, Very Important, Extremely Important) through an online questionnaire developed using SurveyMonkey™. In addition, participants were asked to identify new competencies not already included in the questionnaire. Participant feedback was provided between rounds as this is an important aspect of developing group consensus [27, 28]. In rounds two and three, we provided participant feedback in the form of the overall median rating and the percent of the participants who rated the competency as either ‘Very Important’ or ‘Extremely Important’ from the previous round for each competency being rated. Data were collected over a course of 2 months (April – June, 2019), and participants had 2 weeks to complete each round with 1 week separating the rounds.

Identifying the initial list of competencies

The initial list of competencies for KUs and researchers was developed based on the ‘Knowledge Translation Pathways (KTP)’ tool that arose from a scoping review of competencies for knowledge translation [18]. This is in keeping with other Delphi studies that utilized pre-existing information for the initial Delphi round [29]. Two members of the research team (EY, SS) reviewed and identified competencies that were most relevant to IKT. EY is an Integrated Knowledge Translation Expert member of the IKTRN and SS is a healthcare professional and research trainee with graduate level education in knowledge translation. They synthesized the list into 29 competencies for KUs and 30 competencies for researchers through discussion. In keeping with the original ‘KTP’ tool, the competencies were divided into domains of ‘Evidence’, ‘Teamwork’, and ‘Knowledge Translation (KT) Activities’ rather than discrete knowledge, skills and attitudes. We believe this categorization accounts for the integrated nature of competencies, and allows us to better visualize the competencies relevant to the key processes critical to IKT.

Defining consensus for core competencies

Our a priori definition of a highly rated competency consisted of an overall median rating of ‘Very Important’ or ‘Extremely Important’ in addition to at least 70% of respondents agreeing with either of those ratings within a round. In combination, median score and percent agreement is the most commonly used criteria for consensus based on a systematic review of health care Delphi studies [24]. We considered there to be consensus for a core competency if that competency was highly rated in two rounds. Competencies that did not meet the highly rated criteria for two rounds were excluded from the final list of core competencies. Competencies that were rated twice and received inconsistent ratings between rounds (e.g., highly rated in Round 2 and not highly rated in Round 3) were resolved through group discussion by members of the research team (EY, SS and NMS).

Round 1

The aims of Round 1 were to (1) characterize the participants in terms of stakeholder group (KU or researcher) as well as work role and setting, (2) rate the importance of all initially identified competencies for KUs and researchers to engage in IKT, and (3) generate new competencies based on participants’ comments. The Round 1 questionnaire was reviewed by a member of the research team (NMS), and minor adjustments were made to improve content and clarity prior to administering the survey.

Round 2

The aims of Round 2 were to (1) provide feedback (median rating and percent of participants who rated the competency as either ‘Very Important’ or ‘Extremely Important’) on the competencies rated in Round 1, (2) re-rate the importance of all competencies from the initial list to assess for consistency between rounds, (3) rate the importance of newly added competencies, and (4) generate new competencies based on participants’ comments if they arise.

Round 3

The aims of Round 3 were to (1) provide feedback (median rating and percent of participants who rated the competency as either ‘Very Important’ or ‘Extremely Important’) on the competencies rated in Round 2 that were being rated again, (2) re-rate the importance of initial competencies that received inconsistent ratings between Round 1 and Round 2, and (3) re-rate the importance of competencies that were added after Round 1 based on participants’ comments.

Data analysis

After each round, we calculated the overall median rating and the percent of the participants who rated each competency as ‘Very Important’ or ‘Extremely Important’ (percent agreement). We also conducted a post-hoc analysis of the median rating and percent agreement for the individual KU and researcher groups since there was an unplanned difference between the number of KUs and researchers in the sample. To determine whether one group influenced the overall ratings of competencies, we followed the steps outlined by Elwyn et al. [30] to calculate the equimedian for competencies where there was discordance between the median rating for the KU and researcher groups (i.e., one group median was “Very Important” or above, and the other group median was below “Very Important”). The equimedian is the median of a cumulative distribution function that gives equal weight to the two groups’ responses. We used Microsoft Excel 2016 to perform the data analysis.

Results

Participants

Sixty-three potential participants were invited to participate in this study, 26 of whom responded to the first round of the questionnaire, representing a 41.3% response rate (Table 1). One participant did not provide an email address and therefore could not be contacted in subsequent rounds. Twenty-two out of the 26 participants from round 1 completed the questionnaire for the second round (84.6% retention rate) and 18 of 22 participants responded to the third round of the survey, representing an 81.8% retention rate.

Table 1 Characteristics of participants (Rounds 1, 2, and 3)
Full size table

Participants were asked to identify the predominant role that they fill in the IKT process, either KU or researcher. Researchers represented approximately 68 to 73% of the participants in each round (Table 1). In an open-ended comment section in Round 1, two participants reported that they consider themselves to fill the role of KU and researcher, although both participants selected researcher as their predominant role.

In round 1, 7 KUs and 19 researchers responded. The most common type of work was program development/evaluation (n = 2, 29%) and research support (n = 2, 29%) for KUs, with research (n = 16, 84%) being the most common type of work for researchers. KUs worked primarily in government (n = 2, 29%) and ‘Other’ settings (n = 4, 57%), and researchers primarily worked in academic institutions (n = 11, 58%) or hospital (n = 5, 26%) settings.

Round 1

Of the 59 initial competencies (KU competencies =29, researcher competencies = 30), 43 (72.9%) met the highly rated criteria (KU competencies = 16, researcher competencies = 27). The majority of the KU competencies that did not meet the highly rated criteria (9/16, 56.3%) were from the ‘Evidence’ domain. Based on the participants’ comments, we identified seven new competencies (KU competencies = 5, researcher competencies = 2) to include in subsequent rounds (Table 2).

Table 2 New competencies identified by participants in Round 1
Full size table

Round 2

Of the 43 competencies that met the highly rated criteria from Round 1, nine (20.9%) (KU competencies = 2, researcher competencies = 7) fell below the threshold after Round 2 and were subsequently eliminated. All 16 (100%) competencies that did not meet the highly rated criteria in Round 1 also did not meet these criteria in Round 2, demonstrating consistency between rounds. One (14.3%) of the seven new competencies that were added after Round 1 was rated highly. Participants did not identify new competencies after Round 2.

Round 3

The nine (100%) competencies that showed inconsistency between Round 1 and Round 2 (i.e., met the highly rated criteria in Round 1 but not in Round 2), were rated highly in Round 3, and were therefore retained in the final list of core competencies (Table 3). Of the 7 new competencies that were added in Round 2, one (14.3%) remained consistently highly rated, four (57.1%) remained consistently below the highly rated threshold, and two (28.6%) met the highly rated criteria in Round 3 despite missing the threshold in Round 2. Members of the research team discussed the two competencies that demonstrated inconsistency between Rounds 2 and 3 and resolved to include them in the final list of prioritized competencies for IKT.

Table 3 List of core competencies for knowledge users and researchers to engage in IKT after 3 survey rounds
Full size table

Core competencies for IKT

A total of 46 competencies were identified as highly important for IKT in our study (Table 3). Eighteen competencies were deemed core competencies for KUs (Evidence = 3, Teamwork = 6, Knowledge Translation Activities = 9), and there were 28 core competencies for researchers (Evidence = 8, Teamwork = 8, Knowledge Translation Activities = 12).

Competencies eliminated during the Delphi process

A total of 20 competencies from the initial list were eliminated (Table 4). Sixteen of these were KU competencies (Evidence = 11, Teamwork = 3, Knowledge Translation Activities = 2) and 4 of these were researcher competencies (Evidence = 1, Teamwork = 1, Knowledge Translation Activities = 2).

Table 4 List of eliminated competencies for knowledge users and researchers to engage in IKT after 3 survey rounds
Full size table

Discordance between KU and researcher groups

To determine if the unequal number of KUs and researchers had an impact on the outcome of the Delphi study, we conducted a post-hoc analysis for competencies that received discordant ratings between KU and researcher groups by calculating the equimedian, or weighted median, for each competency rating [30]. Equimedian values are presented in Supplemental Table 1 (core competencies) and Supplemental Table 2 (eliminated competencies). Based on this analysis, we concluded that use of the equimedian rather than the overall median as an a priori criterion for a high rating, would have resulted in very few (3%) of the competencies receiving a higher median rating. Therefore, the group influence on the overall ratings of the competencies was very small. Additionally, because a high rating was also dependent on the percent agreement with ‘Very Important’ or ‘Extremely Important’, using the equimedian would not have changed the outcomes of this study.

Discussion

This study provides important insights regarding KU and researcher competencies that are perceived by study participants as important to engaging in IKT. Both KU and researcher participants agreed on a total of 46 competencies, with 18 competencies deemed important for KUs in the domains of Evidence (n = 3); Teamwork (n = 6), and Knowledge Translation Activities (n = 9); and 28 competencies rated as important for researchers under the same domains (Evidence (n = 8), Teamwork (n = 8), and Knowledge Translation Activities (n = 12)).

‘Evidence’ domain

The competencies prioritized for KUs and researchers in the ‘Evidence’ domain were complementary and support the sentiment that the partnerships formed between KUs and researchers are greater than the sum of its parts in the IKT context [31, 32].

Knowledge user competencies

The competencies from the ‘Evidence’ domain that were rated as highly valuable skills for KUs pertain to having a working familiarity with the local health care system and the use of different forms of knowledge in decision-making. This is consistent with assertions that a deep understanding of the context for KT activities and an appreciation for a range of evidence resources are critical for successful knowledge translation [18]. This is also in keeping with the conceptualization of evidence-based practice in which the application of different types of knowledge, including tacit knowledge, has been previously recognized as central to decision-making within a range of health care contexts [31, 33,34,35]. For IKT in particular, KUs are particularly well-positioned to bring unique context-related knowledge and perspectives to the development of research priorities, and the co-production and application of research evidence [31, 36].

Competencies in the ‘Evidence’ domain that were not prioritized for KUs included the ability to critically appraise research findings, create and execute an efficient search strategy within relevant electronic databases, examine and interpret selected evidence from a literature search, and keep up to date on relevant literature. This was a notable yet surprising finding given the importance attributed to developing research knowledge and skills for knowledge users such as health care professionals in the context of evidence-based practice [37]. Due to the low number of KUs in this study and their diverse roles, some participants may not have perceived these competencies as important for their day-to-day work, and competency requirements may vary across contexts. For example, while KUs serving in clinical roles may be expected to learn continuously about new clinical approaches, the same expectations may not be applied to KUs working in managerial roles with respect to the explicit use of research evidence. Issues of feasibility may also prevent all stakeholders from engaging with the entire IKT process in every circumstance [18]. Moreover, the competencies not prioritized for KUs may have been perceived as more relevant to the researcher role; suggesting that the knowledge-to-action gap may still be perceived as a knowledge transfer issue [36] rather than one of knowledge co-production involving KUs and researchers [38, 39]. Greater attention will need to be paid to interventions and curricula that facilitate attitudinal shifts toward beliefs and values that support the entire IKT process rather than simply focusing on knowledge transfer between distinct groups of researchers and knowledge users [18, 37].

Researcher competencies

The competencies in the ‘Evidence’ domain that were prioritized for researchers relate directly to the research process. This is not surprising since researchers are generally academic researchers who are directly involved in and skilled at carrying out the research process [32, 39]. Notably, “Identify and set priorities regarding policy needs and research options” was not a highly rated competency for researchers. In keeping with IKT theory and practice [17, 32], this may be because this competency was considered by study participants to be a role of the KUs who may be better suited to set priorities given their deep understanding of the context.

‘Teamwork’ domain

Our study findings elaborate on the competencies suggested by Mallidou et al. [18] who described collaborative skills in managing and leading teams as central to knowledge translation. Competencies within the ‘teamwork’ domain in our study were not only highly rated in all 3 Delphi rounds in this study, but were considered important for both KUs and researchers. They pertain to navigating the ‘boundary organizations’ occurring at the intersection between science and practice [13] and promoting ongoing partnerships between KUs and researchers. Consistent with other participatory knowledge co-production approaches, many of the highly rated ‘Teamwork’ competencies support KU and researchers’ roles in knowledge sharing and in forming and maintaining healthy relationships between KUs and researchers [14, 15, 40,41,42]. The knowledge co-production process is particularly vulnerable to the impact of power relationships amongst researchers and KUs, especially for issues of sustainability, and the integration of diverse perspectives and ideas [12]. Data from the present study support the development of specific ‘teamwork’ competencies to address these shortcomings and to minimize these common vulnerabilities, particularly when marginalized populations are involved. Among these are competencies related to the formation and maintenance of healthy relationships and networks throughout the research cycle to optimize team functioning; these include recognizing and valuing the expertise of all team members, and for KUs to advocate for their involvement in the IKT process. These competencies are well-aligned with community-based participatory approaches in which efforts aimed at fostering the team’s capacity for shared ownership of research results afforded opportunities for all team members to engage in meaningful dialogue [43]. However, the full potential of ‘teamwork’ competencies may only be optimally realized if the underlying values and commitment of team members are sufficiently oriented toward supporting IKT [18].

‘Knowledge translation activities’ domain

Knowledge user competencies

The KU competencies prioritized under the ‘KT Activities’ domain support activities typically executed towards the end of the research process that promote the application of research evidence to the local context. Notably, the two KU competencies that were eliminated from the ‘KT Activities’ domain were 1) utilizing appropriate KT framework(s) for the context being considered and 2) leading the team in synthesizing research evidence to improve healthcare services. Until more recently, both of these competencies have been traditionally associated with the researcher role [3]. Owing to their familiarity with stakeholder needs and the local environment in which knowledge is to be applied, some have argued for KUs, and specifically managers, to be at the helm for planning and executing KT interventions [38, 44]. Indeed, KUs may not view themselves as agents of change in a formal manner and may not recognize the implicit ways in which they are already applying KT frameworks in policy and practice. However, there is growing evidence to suggest that KUs can be effective leaders and change agents for implementing innovations, particularly in contexts where KUs are highly committed and proactively forming working relationships with community partners [45]. Thus, while team leadership in synthesizing research evidence was not prioritized as a KU competency in this study, it may still be worthwhile to examine the need to build capacity among individual KUs as well as organizations, noting specific conditions under which KUs may best serve as change agents and leaders over knowledge translation activities [17].

Researcher competencies

There is emerging evidence to suggest that there are potential benefits to blurring the lines between research, quality improvement (QI) and evaluation which has important implications for partnership formation between academic and health organizations [36]. Thus, it is surprising that recognizing the difference between research and evaluation in a particular context was not prioritized as a researcher competency. This is perhaps due to a lack of understanding of the potential opportunities afforded by overlapping research with QI and evaluation efforts. When research, QI and evaluation are conceptualized as distinct activities within an organization, it can inadvertently lead to an artificial separation of roles and responsibilities to the detriment of the IKT process. Thus, future work is warranted to explore how traditional conceptions of research could be reimagined [36] and the competencies required of KUs and researchers for this reconceptualization.

Recommendations

Our study highlights the criticality of teamwork competencies to enable KUs and researchers from both sides of the ‘boundary organizations’ to interact effectively and efficiently to co-produce knowledge [12]. In the face of long-standing hierarchical traditions and power differentials within the research context, one possible way forward is to build team members’ capacity to create and/or contribute to research environments in which teamwork competencies and practices are promoted, normalized and made explicit. Creating a safe space for exchange of dialogue by setting ground rules, ensuring participant interactivity, and encouraging shared ownership of study results are examples of strategies to assist in managing power relations [43]. Central to these strategies is KUs and researchers’ capacity to be reflexive about one’s own values and assumptions, as well as the ability to communicate clearly and effectively [43, 46]. Notably, competency in reflexivity was not included in the initial Delphi survey, nor did study participants recommend this as a new competency to include in subsequent rounds. Thus, a potential focus of competency development to support IKT could be to build reflexive skills among KUs and researchers.

Evidence is also accumulating from studies of health care teams to suggest that training efforts should additionally attend to the collective competence of the entire team or of the organization [18, 46, 47]. Building individual competence may be necessary but insufficient to address the multitude of contextual factors facing each unique IKT scenario [46, 47]. Similarly, teams working in the IKT context must simultaneously address the expertise relevant to the task at hand as well as the processes unique to that context that allow for optimal collective functioning. Thus, further research is warranted to determine if and to what extent the notion of collective competence plays a role in the IKT context, and how team-based learning approaches may benefit the development of expertise in IKT.

Since a key ingredient for successful IKT is establishing a shared understanding of language, roles and responsibilities, and ultimately how decisions are made, it stands to reason that KUs and researchers should also be equipped to negotiate the IKT process. While negotiation was not an explicit competency in this study, we believe it is implicit in the KU competencies prioritized in this study, such as to ‘address inconsistencies between research findings and expert or patient preferences’, and ‘advocate for inclusion of appropriate KUs in the IKT process’. Similarly, strong negotiation skills are implicit in researcher competencies including setting research priorities and identifying the roles of partners in enhancing user engagement. Moreover, negotiation of roles and responsibilities to enable successful teamwork is important across diverse organizational and non-organizational contexts, as well as teams that represent multiple stakeholders. In a study of how nurse practitioners were integrated into primary health care teams, role negotiation and clarification was a critical determinant in the performance of health care teams [48]. In this case, role clarification was most effective when individuals mobilized their negotiation skills within the constraints of their respective organizational processes. Therefore, it may be prudent to draw on organizational development and transdisciplinary literatures, and health care team practices to identify ideal approaches to build capacity among KUs and researchers to negotiate effectively.

This study led to the exclusion of ‘leadership’ over KT activities as a KU competency in despite of literature pointing to the important role of KUs, such as managers, in implementation leadership [44, 45]. This could be reflective of persistent traditional paradigms of research in which leadership over KT activities is held by researchers. While the rewards of more collaborative models of leadership have been purported in the health care practice context [46, 47], wider adoption of similar models in IKT must be considered. As we strive to understand ways to improve implementation processes, a way forward may be to explore the extent to which domains of the Implementation Leadership Scale (proactive, knowledgeable, supportive and perseverant leadership) may be relevant to and fostered amongst KUs [49, 50]. Pursuing this line of research can potentially yield opportunities to invest in alternative means for KUs to contribute meaningfully to the IKT process.

Limitations

We acknowledge that of the use of previously identified general KT competencies may have inadvertently de-emphasized or excluded competencies relevant to IKT, including those that underpin social processes important to IKT. Hence, future work could explore specific competencies, such as negotiation skills, that are critical to managing the evolving roles of KU and researcher throughout the research cycle. We also recognize that while some competencies did not meet the criteria in this study, they may still hold relevance for researchers and knowledge users in some contexts, and hence should not be ignored altogether. While we sought appropriate representation from researchers and knowledge user groups, participants were recruited from a single international IKT network that is dominated by Canadian knowledge users who might share a particular worldview about IKT. We also acknowledge the smaller number of KU participants in this study and the absence of data regarding gender identities. Future studies could examine the external validity of the competencies prioritized in this study for a wide range of research contexts, research partnerships and research co-production team roles assumed by KUs. This should include establishing competencies to support the formation of patient partners for projects that involve patient oriented research.

Conclusion

Development of highly skilled KUs and researchers is critical if IKT teams are to effectively and efficiently co-produce research to address gaps in health care policy and practice. While core competencies have previously been identified to facilitate training in knowledge translation more generally [15], this study begins to clarify the competencies that are most relevant to KUs and researchers to support successful IKT. The results from this Delphi study specify competencies in three domains (‘evidence’, ‘knowledge ‘teamwork’ and ‘knowledge translation activities’) that are of particular importance for KUs and researchers to attend to when engaging in IKT. The competencies identified reflect the dynamic and complex nature of IKT wherein both KUs and researchers must be skilled in managing team dynamics and functions throughout all phases of the research process. Although competencies have been specified separately for KU and researchers in this study, future work could focus on how these can be further negotiated and contextualized due to the wide variations in IKT context, project and team

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

IKT:

Integrated knowledge translation

IKTRN:

Integrated Knowledge Translation Research Network

KTP:

Knowledge Translation Pathway

KT:

Knowledge translation

KU:

Knowledge user

References

  1. Nelson ML, Grudniewicz A, Albadry S. Applying clinical practice guidelines to the complex patient: insights for practice and policy from stroke rehabilitation. Healthc Q. 2016;19(2):38–43. https://doi.org/10.12927/hcq.2016.24697.

    Article  PubMed  Google Scholar 

  2. Fillion B, Rochette A, Girard A. Challenges of being a scholarly clinician as perceived by stroke rehabilitation professionals. Disabil Rehabil. 2014;36(6):521–8. https://doi.org/10.3109/09638288.2013.797516.

    Article  PubMed  Google Scholar 

  3. Bowen S, Graham ID. Integrated knowledge translation. In: Straus SE, Tetroe J, Graham ID, editors. Knowledge translation in health care: moving from evidence to practice. 2nd ed. UK: Wiley; 2013. p. 14–23. https://doi.org/10.1002/9781118413555.ch02.

    Chapter  Google Scholar 

  4. Kothari A, McCutcheon C, Graham ID. Exploring the frontiers of research co-production: the Integrated Translation Research Network concept papers. Collection published in Research Involvement and Engagement, Health Resarch Policy and Systems, Implementation Science, Implementation Science Communications, BMC Medical Research Methodlogy, BMC Health Services Research, and Systematic Reviews. https://www.biomedcentral.com/collections/IKT. Accessed 19 July 2021.

  5. Kothari A, McCutcheon C, Graham ID. Defining integrated knowledge translation and moving forward: a response to recent commentaries. Int J Health Policy Manag. 2017;6(5):299–300. https://doi.org/10.15171/ijhpm.2017.15.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated Knowledge Translation (IKT) in health care: a scoping review. Implement Sci. 2016;11(38). https://doi.org/10.1186/s13012-016-0399-1.

  7. Elliot H, Popay J. How are policy makers using evidence? Models of research utilization and local NHS policy making. J Epidemiol Community Health. 2000;54(6):461–8. https://doi.org/10.1136/jech.54.6.461.

    Article  Google Scholar 

  8. Kothari A, MacLean L, Edwards N, Hobbs A. Indicators at the interface: managing policymaker-researcher collaboration. Knowl Manag Res Pract. 2011;9(3):203–14. https://doi.org/10.1057/kmrp.2011.16.

    Article  Google Scholar 

  9. Ross S, Lavis J, Rodriguez C, Woodside J, Denis JL. Partnership experiences: involving decision-makers in the research process. J Health Serv Res Policy. 2003;8(2):26–30. https://doi.org/10.1258/135581903322405144.

    Article  PubMed  Google Scholar 

  10. Salbach NM, Guilcher SJT, Jaglal SB, Davis DA. Determinants of research use in clinical decision making among physical therapists providing services post-stroke: a cross-sectional study. Implement Sci. 2010;5(1):77. https://doi.org/10.1186/1748-5908-5-77.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Walter I, Davies H, Nutley S. Increasing research impact through partnerships: evidence from outside health care. J Health Serv Res Policy. 2003;8(S2):58–60. https://doi.org/10.1258/135581903322405180.

    Article  PubMed  Google Scholar 

  12. Pohl C, Rist S, Zimmerman A, Fry P, Gurung GS, Schneider F, et al. Researchers’ roles in knowledge co-production: experience from sustainability research in Kenya, Switzerland, Bolivia and Nepal. Sci Public Policy. 2010;37(4):267–81. https://doi.org/10.3152/030234210X496628.

    Article  Google Scholar 

  13. Guston DH. Boundary organizations in environmental policy and science: an introduction. Sci Technol Hum Values. 2001;26(4):399–408. https://doi.org/10.1177/016224390102600401.

    Article  Google Scholar 

  14. Jull J, Giles A, Graham ID. Community-based participatory research and integrated knowledge translation: advancing the co-creating of knowledge. Implement Sci. 2017;12(1):150. https://doi.org/10.1186/s13012-017-0696-3.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Nguyen T, Graham ID, Mrklas KJ, Bowen S, Cargo M, Estabrooks CA, et al. How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field. Health Res Policy Syst. 2020;18(1):35. https://doi.org/10.1186/s12961-020-0539-6.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Kothari A, McCutcheon C, Graham ID. Defining integrated knowledge translation and moving forward: a response to recent commentaries. Int J Health Policy Manag. 2017;6(5):2–1. https://doi.org/10.15171/ijhpm.2017.15.

    Article  Google Scholar 

  17. Gagliardi AR, Dobrow MJ. Identifying the conditions needed for integrated knowledge translation (IKT) in health care organizations: qualitative interviews with researchers and research-users. BMC Health Serv Res. 2016;16(1):256. https://doi.org/10.1186/s12913-016-1533-0.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Mallidou AA, Atherton P, Chan L, Frisch N, Glegg S, Scarrow G. Core knowledge translation competencies: a scoping review. BMC Health Serv Res. 2018;18(1):502. https://doi.org/10.1186/s12913-018-3314-4.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Hasson F, Keeney S. Enhancing rigour in the Delphi technique research. Technol Forecasting Soc Change. 2011;78(9):1695–704. https://doi.org/10.1016/j.techfore.2011.04.005.

    Article  Google Scholar 

  20. Junger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on conducting and reporting Delphi studies (CREDES) in palliative are: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706. https://doi.org/10.1177/0269216317690685.

    Article  PubMed  Google Scholar 

  21. Trevelyan EG, Robinson PN. Delphi methodology in health research: how to do it? Eur J Integr Med. 2015;7(4):423–8. https://doi.org/10.1016/j.eujim.2015.07.002.

    Article  Google Scholar 

  22. Canadian Institutes of Health Research. Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches. vol. Cat. No. MR4–11/2012E-PDF. 2012. Available from: https://cihr-irsc.gc.ca/e/45321.html.

  23. IKT Research Network. Who we are. Available from: https://iktrn.ohri.ca/aboutus/who-we-are/. Accessed 14 Sept 2021.

  24. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476. https://doi.org/10.1371/journal.pone.0020476.

  25. Dillman DA. Mail and internet surveys: the tailored design method. 2nd ed. Hoboken: Wiley; 2007.

    Google Scholar 

  26. Turnball AE, Dinglas VD, Aronson Friedman L, Chessare CM, Sepulveda KA, et al. A survey of Delphi panelists after core outcome set development revealed positive feedback and methods to facilitate panel member participation. J Clin Epidemiol. 2018;102:99–106. https://doi.org/10.1016/j.jclinepi.2018.06.007.

    Article  Google Scholar 

  27. Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CFB, Askham J, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2(3):i–iv, 1–88.

    Article  CAS  Google Scholar 

  28. Powell C. The Delphi technique: myths and realities. J Adv Nurs. 2003;41(4):376–82. https://doi.org/10.1046/j.1365-2648.2003.02537.x.

    Article  PubMed  Google Scholar 

  29. Dunfield C. The Delphi technique: a comparison of results obtained using two expert panels. Int J Nurs Stud. 1993;30(3):227–37. https://doi.org/10.1016/0020-7489(93)90033-Q.

    Article  Google Scholar 

  30. Elwyn G, O’Connor A, Stacey D, Volk R, Edwards A, Coulter A. Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ. 2006;333(7565):417.

    Article  Google Scholar 

  31. Banner D, Bains M, Carroll S, Kandola DK, Rolfe DE, Wong D, et al. Patient and public engagement in integrated knowledge translation research: are we there yet? Res Involv Engagem. 2019;5:8.

    Article  Google Scholar 

  32. Kothari A, Wathen CN. A critical second look at integrated knowledge translation. Health Policy. 2013;109(2):187–91. https://doi.org/10.1016/j.healthpol.2012.11.004.

    Article  PubMed  Google Scholar 

  33. Albarqouni L, Hoffman T, Straus S, Rydland Olsen N, Young T, Ilic D, et al. Core competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and Delphi survey. JAMA Netw Open. 2018;1(2):e180281. https://doi.org/10.1001/jamanetworkopen.2018.0281.

    Article  PubMed  Google Scholar 

  34. Kothari AR, Bickford JJ, Edwards N, Dobbins MJ, Meyer M. Uncovering tacit knowledge: a pilot study to broaden the concept of knowledge in knowledge translation. BMC Health Serv Res. 2011;11(1):198. https://doi.org/10.1186/1472-6963-11-198.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71–2. https://doi.org/10.1136/bmj.312.7023.71.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Bowen S, Botting I, Graham ID, MacLeod M, de Moissac D, Harlos K, et al. Experience of health leadership in partnering with university-based researchers in Canada – a call to “re-imagine” research. Int J Health Policy Manag. 2019;8(12):684–99. https://doi.org/10.15171/ijhpm.2019.66.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Lehane E, Leahy-Warren P, O’Riordan C, Savage E, Drennan J, O’Tuathaigh C, et al. Evidence-based practice education for health care professions: an expert view. BMJ Evid Based Med. 2019;24(3):103–8. https://doi.org/10.1136/bmjebm-2018-111019.

    Article  PubMed  Google Scholar 

  38. Cornelissen E, Mitton C, Sheps S. Knowledge translation in the discourse of professional practice. Int J Evid Based Healthc. 2011;9(2):184–8. https://doi.org/10.1111/j.1744-1609.2011.00215.x.

    Article  PubMed  Google Scholar 

  39. McIsaac JLD, Penney TL, Storey KE, Sigfridson L, Cunningham J, Kuhle S, et al. Integrated knowledge translation in population health intervention research: a case study of implementation and outcomes from a school-based project. Health Res Policy Syst. 2018;16(1):72. https://doi.org/10.1186/s12961-018-0351-8.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Dari T, Laux JM, Liu Y, Reynolds J. Development of community-based participatory research competencies: a Delphi study identifying best practices in the collaborative process. Prof Counsellor. 2019;9(1):1–19. https://doi.org/10.15241/td.9.1.1.

    Article  Google Scholar 

  41. Horowitz CR, Robinson M, Seifer S. Community-based participatory research from the margin to the mainstream. Are researchers prepared? Circulation. 2009;119(19):2633–42. https://doi.org/10.1161/CIRCULATIONAHA.107.729863.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Rahman R, Kirkbride G, Bauta BH, Jabbour J, Maranaga G. Using community-based participatory research to develop a series of core competency training within a developmental disability program. J Soc Serv Res. 2018;44(4):448–58. https://doi.org/10.1080/01488376.2018.1476295.

    Article  Google Scholar 

  43. Nixon SA, Casale M, Flicker S, Rogan M. Applying the principles of knowledge translation and exchange to inform dissemination of HIV survey results to adolescent participants in South Africa. Health Promot Int. 2013;28(2):233–43. https://doi.org/10.1093/heapro/dar094.

    Article  PubMed  Google Scholar 

  44. Birken SA, Lee SD, Weiner BJ. Uncovering middle managers’ role in healthcare innovation implementation. Implement Sci. 2012;7(28). https://doi.org/10.1186/1748-5908-7-28.

  45. Birken SA, Lee SD, Weiner BJ, Chin MH, Schaefer CT. Improving the effectiveness of health care innovation implementation: middle managers as change agents. Med Care Res Rev. 2013;70(1):29–45. https://doi.org/10.1177/1077558712457427.

    Article  PubMed  Google Scholar 

  46. Lingard L, Espin S, Evans C, Hawryluck L. The rules of the game: interprofessional collaboration on the intensive care unit team. Crit Care. 2004;8(6):403–8. https://doi.org/10.1186/cc2958.

    Article  Google Scholar 

  47. Leggat SG. Effective healthcare teams require effective team members: defining teamwork competencies. BMC Health Serv Res. 2007;7(1):17. https://doi.org/10.1186/1472-6963-7-17.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Brault I, Kilpatrick K, D’Amour D, Contandriopoulos D, Chouinard V, Dubois C, et al. Role clarification processes for better integration of nurse practitioners into primary healthcare teams: a multiple-case study. Nurs Res Pract. 2014;2014:1–9. https://doi.org/10.1155/2014/170514.

    Article  Google Scholar 

  49. Aarons GA, Ehrhart MG, Farahnak LR. The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership. Implement Sci. 2014;9(1):45. https://doi.org/10.1186/1748-5908-9-45.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Gifford W, Graham ID, Ehrhart MG, Davies BL, Aarons GA. Ottawa model of implementation leadership and implementation leadership scale: mapping concepts for developing and evaluating theory-based leadership interventions. J Healthc Leadersh. 2017;9:15–23. https://doi.org/10.2147/JHL.S125558.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We would like to thank Chris McCutcheon for his insights and assistance during development of the manuscript.

Funding

This study was supported by a subgrant from a Canadian Institutes of Health Research (CIHR) Foundation Grant to IDG (FDN# 143237). The CIHR had no role in the project.

Author information

Authors and Affiliations

  1. University of Toronto, 160-500 University Avenue, Toronto, Ontario, M5G 1V7, Canada

    Euson Yeung, Stephanie Scodras & Nancy M. Salbach

  2. Western University, Health Sciences Building, Rm 222, 1151 Richmond St, London, Ontario, N6A 5B9, Canada

    Anita Kothari

  3. School of Epidemiology and Public Health, University of Ottawa, 600 Peter Morand Cresent, Ottawa, Ontario, K1G 5Z3, Canada

    Ian D. Graham

  4. Centre for Practice-Changing Research, The Ottawa Hospital Research Institute, 501 Smyth Road, Box 711, Ottawa, Ontario, K1H 8L6, Canada

    Ian D. Graham

Authors
  1. Euson Yeung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephanie Scodras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nancy M. Salbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anita Kothari
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ian D. Graham
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EY, AK and IG contributed to the conceptualization of the study. EY designed and led the research project. SS collected the data and was a major contributor in writing the manuscript. EY, SS and NS analyzed and interpreted the survey data. All authors contributed to data analysis, and read and approved the final manuscript.

Corresponding author

Correspondence to Euson Yeung.

Ethics declarations

Ethics approval and consent to participate

The University of Toronto Research Ethics Board approved the study protocol. Protocol ID: 37448. All methods were performed in accordance with the guidelines and regulations of this research ethics board. Informed consent was obtained from all participants in this research study.

Consent for publication

not applicable

Competing interests

The authors declare that they have no competing interests” in this section.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yeung, E., Scodras, S., Salbach, N.M. et al. Identifying competencies for integrated knowledge translation: a Delphi study. BMC Health Serv Res 21, 1181 (2021). https://doi.org/10.1186/s12913-021-07107-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-021-07107-7

Keywords

BMC Health Services Research

ISSN: 1472-6963

Contact us