This article has Open Peer Review reports available.
How does context influence collaborative decision-making for health services planning, delivery and evaluation?
© Gagliardi et al.; licensee BioMed Central Ltd. 2014
Received: 13 February 2014
Accepted: 21 October 2014
Published: 19 November 2014
Collaboration among researchers (clinician, non-clinician) and decision makers (managers, policy-makers, clinicians), referred to as integrated knowledge translation (IKT), enhances the relevance and use of research, leading to improved decision-making, policies, practice, and health care outcomes. However IKT is not widely practiced due to numerous challenges. This research explored how context influenced IKT as a means of identifying how IKT could be strengthened.
This research investigated IKT in three health services programs for colon cancer screening, prostate cancer diagnosis, and the treatment of pancreatic cancer. Qualitative methods were used to explore contextual factors that influenced how IKT occurred, and its impact. Data were collected between September 1, 2012 and May 15, 2013 from relevant documents, observation of meetings, and interviews with researchers and decision-makers, analyzed using qualitative methods, and integrated.
Data were analyzed from 39 documents, observation of 6 meetings, and 36 interviews. IKT included interaction at meetings, joint undertaking of research, and development of guidelines. IKT was most prevalent in one program with leadership, clear goals, dedicated funding and other infrastructural resources, and an embedded researcher responsible for, and actively engaged in IKT. This program achieved a variety of social, research and health service outcomes despite mixed individual views about the value of IKT and the absence of a programmatic culture of IKT. Participants noted numerous challenges including lack of time and incentives, and recommendations to support IKT. A conceptual framework of factors that influence IKT and associated outcomes was generated, and can be used by others to plan or evaluate IKT.
The findings can be applied by researchers, clinicians, managers or policy-makers to plan or improve collaborative decision-making for health services planning, delivery, evaluation or quality improvement. Further research is needed to explore whether these findings are widespread, and further understand how IKT can be optimized.
KeywordsIntegrated knowledge translation Health services research Collaboration Qualitative methods
For some time it has been known that interaction between knowledge producers and users is an influential means by which to generate practice-relevant knowledge and enable evidence-informed practice -. The imperative to improve evidence-informed decision making for health care planning, delivery, evaluation and improvement has prompted broader recognition of the need for such partnerships, and the conduct of research on how to optimize researcher-research user collaboration. Now more commonly referred to as integrated knowledge translation (IKT), this involves the development of a relationship between researchers and decision-makers (clinicians, managers, policy-makers, etc.) for the purpose of engaging in a mutually beneficial project or program of research . It is distinguished from knowledge translation due to its emphasis on partnership or collaboration . Decision-making research shows that complex problems require input from individuals with different but relevant expertise and perspectives to formulate, execute, and evaluate solutions . Collaboration - like that promoted through IKT approaches - involves ongoing, dynamic interactions among researchers and decision-makers and represents an ideal means by which to address complex problems . Proposed benefits of IKT include research questions that are more practice and policy relevant, and feasible to address; adaptable findings that are received by a primed and receptive audience; and an increase in mutual understanding of roles and values among researchers and decision-makers .
Empirical research has demonstrated concrete benefits of IKT. Interviews with researcher and National Health Services decision-maker partners of nine initiatives in the United Kingdom revealed that all achieved improved clinical care through a variety of IKT approaches . An exploratory study of a university partnership with professionals in six Scottish health authorities revealed that communication improved, training was offered, and professionals reported enhanced skill and confidence, and a number of changes in their practice . Early evaluation of an Australian obesity prevention network involving researchers at three universities and professionals at 78 community organizations found that the initiative resulted in the joint generation of best practice guidance and delivery of numerous professional development sessions . Qualitative evaluation of an academic-primary healthcare collaborative initiative in the United Kingdom resulted in improved communication between agencies, identification of competency and knowledge deficits, and development and implementation of a tailored training program .
Despite this evidence of its positive impact on use of research, and improved health care planning, delivery and outcomes, we lack a thorough and shared understanding of its value and how to achieve it. In particular, IKT is challenged by the time and resources required for interaction; differential timelines between researchers and decision-makers; few professional and academic incentives tied to performance; and lack of interest, knowledge and skill to engage in IKT ,,,. The need to foster IKT was recognized in both nursing and primary care sectors in Australia and the United Kingdom; among emergency medicine professionals from 16 countries; and by representatives of 33 research funding agencies in Canada, Australia, France, the Netherlands, Scandinavia, the United Kingdom, and United States ,,,. A survey of health policy experts representing 30 European countries found that there were no explicit IKT mechanisms in most respondent countries other than a few examples where researchers were embedded in government research institutes or members of advisory committees . The Canadian Academy of Health Sciences held a series of discussions with national leaders in 2004 and 2005 to address the lack of IKT for health services research and issued several recommendations, including the need for an audit of current and required capacity for IKT .
Existing research offers limited insight on approaches or strategies for operationalizing IKT. Reflection by those involved in a researcher-primary care partnership resulted in several recommendations to facilitate IKT including identifying partners with pre-established links to ease and expedite interaction; establishing clear expectations about role, scope and contribution to foster trust and avoid role confusion and misconceptions; fostering dialogue; and assessing progress to implement changes as needed . A case study based on three health service delivery programs found that IKT was dynamic and highly influenced by the complex context within which decisions were being made including social and political norms . Therefore, rather than imposing external or rigid IKT structures and processes, the investigators suggested that IKT could be optimized by first examining naturalistic interaction that takes place in a given program or organization to identify how IKT could be supported. They proposed that, by considering contextual influencing factors, both barriers and facilitators, existing mechanisms could be enhanced, and IKT may be more likely to be accepted and applied.
Context is widely recognized as a multi-dimensional factor that must be accommodated when tailoring, adapting, implementing, scaling-up or spreading programs or interventions to optimize impact and enhance sustainability ,. Context has been referred to as anything that cannot be described as an intervention or its outcome . A recent review and amalgamation of determinants of practice identified 57 unique factors that were grouped in seven domains: guideline/innovation, individual health professional, patient, professional interaction, incentives and resources, capacity for organizational change, and social, political and legal factors . The Promoting Action on Research Implementation in Health Services (PARIHS) framework captures several of these factors, and has been used as the basis for planning or evaluating interventions, and the factors that influence their success . The PARIHS framework suggests that the nature of evidence, the context in which it is applied, and facilitation of the implementation or adoption process together influence research use. Research in clinical settings highlighted the importance of context relative to evidence and facilitation, and that context includes micro (individual) and meso (organizational) level factors ,. According to PARIHS context is comprised of an organizational culture that is receptive to change, leadership that supports the involvement of individual staff, and evaluation and feedback mechanisms -.
Empirical evidence shows that IKT improves healthcare planning, delivery, evaluation and quality improvement but due to a variety of challenges is not widely practiced. Further research is needed to reveal how IKT can be promoted and supported. Context refers to a broad array of factors that can influence the practice or success of interventions, but the contextual factors that influence IKT have not been examined. The purpose of this research has therefore been to identify contextual factors influencing IKT in different healthcare planning or improvement programs and, based on the findings, generate a conceptual framework by which others could plan, promote, strengthen or evaluate IKT.
This research took place in the setting of Ontario, Canada and employed a qualitative approach to explore the use and impact of IKT, including how researchers interacted with decision-makers and how their research or knowledge about research was used by decision-makers. This was examined in three health services programs that sought to improve underuse of an organized colon cancer screening program, overuse of prostate cancer screening, and high mortality of pancreatic cancer. Qualitative research is useful when there is a need to develop a rich and thorough appreciation of the contextual factors that influence views and behaviour when there has been little prior research . Rigour was optimized by integrating data (“triangulation) collected in variety of ways (interviews, observation, documents) from different sources (decision-makers, researchers); sampling participants with various characteristics that could influence their views; fully analyzing and interpreting data including deviant cases; checking findings with participants; demonstrating responses from an array of participants by including an anonymous identification code with exemplary quotes; and comparison of independently-derived thematic coding across two individuals . Rigour was further ensured by complying with Relevance, Appropriateness, Transparency and Soundness (RATS) principles for the reporting of qualitative research  as the citation in Additional file 1. Ethics approval for this study was acquired from the University Health Network which required that participants provide written informed consent prior to being interviewed. Those observed at meetings provided unanimous oral consent before the start of the meeting. In this study IKT was defined as interaction among researchers and decision-makers for program planning or evaluation, or research.
There is no single or comprehensive theory or model that encompasses the influencing contextual factors, processes and outcomes of IKT so the goal of this research was to generate, rather than prove existing theory. Naturalistic mechanisms of IKT in the involved programs, associated outcomes and, in particular, contextual factors that emerged from the data were compiled in a conceptual framework of contextual factors that influence IKT practice and impact. To analyze data and identify unique findings, emerging contextual factors were examined according to the components of the PARIHS framework including culture (normative views and behaviour), leadership (visibly involved in activities and actively engage staff), and evaluation (feedback is provided to staff on performance and improvement) .
Sampling and recruitment
Three programs were chosen as they had been identified as priorities by the provincial cancer agency, and were thought to differ in a number of ways that could influence IKT including type and prevalence of cancer, health service issue, type of stakeholders, and quantity and quality of evidence underlying the health services issue. For each program, a lead key informant was identified. They suggested and brokered links with individuals for interviews, some of whom recommended additional individuals for interviews. Lead key informants also identified relevant meetings for observation, and documents for content analysis. Purposive sampling was used to recruit interview participants. The goal was to interview five researchers and five decision-makers in each of the three programs for a minimum of 30 interviews. Researchers included clinician and non-clinician/PhD researchers. Decision makers included policy-makers, managers, or clinicians who were affiliated with the chosen programs. Detailed information from representative informants, rather than a large number of participants is needed in qualitative research. Recruitment was concurrent with data collection and analysis, and continued until saturation was achieved, meaning no further unique themes emerged from successive interviews as determined by discussion between two independent reviewers .
Data collection and analysis
Data was collected between September 1, 2012 and May 15, 2013. Document analysis: Content analysis is used to describe phenomena in written, verbal or visual communication . The content of strategic plans, reports, articles, meeting minutes, web sites and other documents referred to, or provided by participants were examined using directed content analysis techniques . This means that the explicit content was coded for direct or indirect instances of collaboration. Data were extracted by a trained research assistant from relevant documents using a structured guide that included document name, location, type, purpose and date of publication, authors and roles, and any direct or indirect evidence of researcher-decision-maker collaboration or influence of researchers on decisions. The principal investigator re-examined all documents to confirm collected data. Field Notes: Field work included the observation of meetings to identify whether and how researchers interacted with decision-makers, and were involved in influencing decisions. Coordinators of relevant committees were contacted to request access to meetings. Data were collected by a trained research assistant who attended all meetings and used a structured guide to document meeting purpose, location, duration, number and role of participants, nature of interaction, and researcher involvement in or influence on decisions. Field notes were summarized to describe observed interaction or mention of interaction between researchers and decision-makers during meetings. Interviews: Grounded theory was used to collect and analyze interviews data . This means that ideas emerged inductively from the data and were then compared with existing theory, rather than using theory to structure data collection which can restrict exploration and miss relevant themes. Interview candidates were contacted by email with an invitation and consent form. Semi-structured interviews were conducted by a trained research assistant via telephone, audio-recorded and transcribed verbatim. Participants were asked four questions - to describe their understanding of IKT, IKT activities in which they were involved, challenges of IKT, and recommendations for how it could be promoted and enabled. Mean length of interviews was 29.47 minutes (median 29.10, range 16.01 to 57.11). Interview transcripts were analyzed iteratively using constant comparison to identify, code and organize themes ,. Initially, open coding was performed to get a general feel for the content of the data, then more selective coding was performed after core concepts began to emerge. Three members of the research team independently read and coded the first three interview transcripts, then met to compare their analyses and developed a codebook that guided subsequent analysis of remaining transcripts. Data Integration: Data from interviews, field work and document analysis were integrated using relational analysis . Key findings from each mechanism of data collection were listed (program description by lead key informants, document analysis, observation, interviews). All unique components were categorized as IKT processes, outcomes or contextual factors which were further organized as individual or organizational level contextual factors, and then as sub-categories. The integrated findings were summarized both textually and visually. Contextual factors influencing IKT that emerged from the study were compared with the components of the PARIHS framework -. A summary of the analysis was prepared, shared with the study team (“member checking”), and refined based on their feedback.
Overview of clinical programs
The three clinical programs are described in Additional file 2, and key characteristics are highlighted here. The colorectal cancer screening program was described by the lead key informant as “established”. It was conceptualized from 1999 to 2002, a pilot program was completed in 2006, and a population-based program was implemented in 2008. Its explicit goals are to reduce deaths from colorectal cancer by inviting and reminding individuals for screening, tracking individuals throughout screening and diagnosis, and providing support for screening to health care providers. Evaluation reports released in 2010 and 2012 both highlighted increased participation in screening, though rates were lower than targeted. The program is funded by the provincial Ministry of Health and cancer agency, operates a secretariat with leadership and staff at the cancer agency. Its activities are guided by several different working groups or committees that include health professionals and researchers from across the province. The prostate cancer program was described by the lead key informant as in the “beginning stages”. It was not a formally recognized program with clearly stated goals or infrastructure. Activities including several meetings during which leaders from the cancer agency gathered health professionals and researchers from across the province to identify quality improvement issues and generate recommendations for ongoing activity. Similar to the prostate cancer program, the pancreatic cancer program did not have formal status with particular goals or infrastructure. While considerable activity had taken place as early as 2006 and over several subsequent year to evaluate quality of surgical care and associated outcomes, and issue recommendations for hospitals delivering such services. The program was in a period of dormancy as research priorities were shifting from treatment to identifying the molecular mechanism of disease which might lead to prevention or early identification of disease. It was described by the lead key informant as an “upcoming priority”.
Document analysis - interaction through meetings, research, guideline development
Documents examined (27 colorectal, 8 prostate, 4 pancreatic) included strategic plans or proposals, meeting minutes, meeting or workshop summary reports, program evaluations, clinical practice guidelines and published research papers. There was no explicit description of interaction between researchers and decision-makers. However, physicians and physician-researchers were named as co-authors on a few documents relevant to prostate cancer diagnosis and pancreatic cancer treatment; and managers, physicians, physician researchers and PhD researchers were named as co-authors on several documents relevant to colorectal cancer screening. This implies that IKT took place, more so for the colorectal program, in the form of interaction at formal meetings, collaboration for research, and joint guideline development. Details of timing, frequency, level of interaction and impact (other than publications or reports) were lacking.
Meeting observation - researchers supported and influenced decision-making
No meetings were observed relevant to prostate or pancreatic cancer because none were scheduled during the study period. Six meetings relevant to the colorectal context were observed. Physician researchers who were embedded as staff in, or affiliated with or appointed to programs or committees appeared to be involved in decision-making and in guiding decision-making in a variety of ways. They did so by having summarized available evidence for review at the meeting, describing and commenting on the quantity and quality of the evidence, interpreting the evidence and whether and how it was relevant or could be applied, and raising issues or implications, or offering suggestions to guide further decision-making. They also offered examples based on their own clinical experience, recommendations from published guidelines, and known risk factors from published research. While the impact of these efforts could not be discerned, formal meetings at which researchers were present appeared to offer opportunities for IKT to occur.
Interviews - numerous contextual factors challenge IKT
Professional role of study participants
Awareness or knowledge of IKT
Unaware or unclear
It’s not a term that I commonly use [012CO/R]
Equated with dissemination
It’s the process whereby information that has been generated scientifically is transmitted to others who have interest in the knowledge [017CO/RU]
Understanding of collaborative nature
A relationship between knowledge users and producers that can be on-going [010CO/U]
Researchers initiated/used IKT
I have to make sure that what I’m finding gets out there [002CO/R]
Need for tailoring to context/audience
Depending on who your audience is, you try and construct the activity to the audience [007CO/U]
Benefit of IKT
Improved the relevance and quality of research
Without that dialogue the likelihood of producing something that’s relevant in the healthcare context that you wish to impact on is lower [004CO/R]
Inform new research questions
Stakeholders can also inform the researchers in what are research priorities etc. [011GE/RU]
Increase efficiency of research
Ultimately save time, money and other resources [032PR/R]
Facilitated learning relevant to professional tasks
There’s mutual learning and assistance to help the various participants move ahead in whatever their agenda is [022PR/RU]
Enhanced understanding of different perspectives
Useful for researchers to understand what the research priorities are from the perspective of the policy- makers [014GE/R]
Promote use of research
You’re creating a ready-made receptor for the knowledge along the way [022PR/RU]
Improved decision-making, policy, practice and outcomes
Bring knowledge to practice in order to improve outcomes [030PR/R]
Accelerates production and use of research
The best way to accelerate the process of integration of best evidence in practice, policy, etc. [014GE/R]
Engagement in IKT
Embedded researchers interacted with managers
Because I’m embedded-that allows that exchange of knowledge to happen [004CO/R]
Focus on interaction with clinicians
Much of my work in terms of presentations has been to clinical people [030PR/R]
Managers unclear about benefit
I don’t know where there would be a benefit of direct interaction with researchers [020GE/U]
Researchers engaged in program evaluation
The only time that we have [involved researchers] is after the fact [006GE/U]
Managers prefer learning about research through publications
I really like systematic reviews of evidence so that I don’t have to sift through all of the stuff that’s out there [009CO/U]
IKT took place in regularly scheduled meetings
It’s generally a meeting over video-conference or a face-to-face meeting [018PR/U]
Challenges and enablers of IKT
Willingness to collaborate
They’re responsive and willing to learn, and collaborate with us [008CO/R]
Innate ability to collaborate
Some people are good at committees, some are not. Stipend or formal position won’t change that [003PA/R]
Motivation and incentives
Researchers often are involved in research as a pure academic pursuit so they don’t think through the need to share that information quickly with policy-makers. They’re focused on peer-reviewed publication as their reward [019PR/U]
Need for formal linkages
You have to be linked to a program or service to really know what’s going on [002CO/R]
It’s me who is initiating the request [to interact with policy makers] [008CO/R]
Awareness of, or access to opportunities
We don’t have the means to know about all the other forums for exchange that are going on and often they’re invitation only for people that are clinicians [029PR/U]
Resistance to change
Users are not necessarily so open to embracing change [023PR/RU]
Unintended transformation or use of research findings
It transforms the product so much that you’ve lost what was the principle research finding [004CO/R]
It’s just synchronizing everyone towards the same goal at the same time [028PR/RU]
There’s got to be buy-in from the institution [033PR/R]
The manager is the interface [013GE/RU]
Mentors and champions
Having a local champion has been helpful [025PA/RU]
The biggest challenge would just simply that everyone’s very, very busy [003PA/R]
Resources and incentives
It’s impossible to get very busy people to do this kind of work without (compensation) [002CO/R]
Difficult to evaluate and show impact
It’s difficult to measure and also difficult to find an impact [017GE/RU]
Recommendations to support IKT
Challenge expressed by participants
Solutions recommended or inferred
Awareness of/knowledge about IKT
• Organizational culture
Funding, resources, time for IKT
• Organizational culture
• IKT-specific strategic plan
• Demonstrate impact that can be derived
Attitude about IKT
• Organizational culture
• Demonstrate impact that can be derived
Willingness to collaborate, resistance to change
• Professional and academic incentives
• Demonstrate impact that can be derived
Skill for collaboration
Access to opportunities
• IKT-specific strategic plan
• Knowledge brokers/facilitators
• Formal program affiliations for researchers
• Inventory of initiatives/research
• Use of technology-enabled communication tools
Mismatch timing/goals between researchers and decision-makers
• Knowledge brokers/facilitators
Coordination/integration across programs
• IKT-specific strategic plan
• Knowledge brokers/facilitators
Variety of forums for IKT
• IKT-specific strategic plan
• Use of technology-enabled communication
Responsibility for initiating IKT
• Organizational culture
• Knowledge brokers/facilitators
• IKT -specific strategic plan
• Professional and academic incentives
Summary of integrated findings
Conceptual analysis of integrated findings
Study findings confirmed several components of the PARIHS framework. The PARIHS framework suggests that research use is influenced by context, where context is defined as a culture receptive to change, leadership support, and feedback to staff. In this research both culture and leadership appeared to be associated with IKT and associated outcomes, though feedback to staff was not evident as either an influencing factor, IKT process or an outcome. Cultural factors that were relevant to this research included reliance on published evidence; resistance to change on the part of both researchers and decision-makers; mismatch between the goals and timing of researchers and decision-makers; and lack of clarity around IKT roles and responsibilities. While leadership was apparent in the colorectal program, in part accounting for IKT activity, participants of this and other programs recommended other forms of leadership or leadership activities including high level recognition for IKT as an organizational priority described in strategic plans, incentives for IKT, and greater opportunities for interaction. Study findings revealed several additional contextual factors that influenced IKT beyond the components of PARIHS. These included organizational capacity, which could be further categorized according to socio-political (clear goals, consensus on goals), economic (dedicated funding, resources and infrastructure such as space, administrative support) and social (formal and informal interaction, affiliated or embedded roles, diversity of engaged disciplines) factors that either enabled or challenged IKT. Findings also confirm the relevance of contextual factors at the individual (knowledge, beliefs, motivation) and organizational (culture, leadership, capacity) levels. It is notable that in this study leadership and capacity (infrastructure, funding, clear goals), in the absence of a strong IKT culture, were sufficient to achieve tangible outcomes including a variety of social, research and health service outcomes, which also elaborated the PARIHS description of outcomes beyond the general term of research use.
This study was conducted to examine how context influences IKT and, in so doing, generate a conceptual framework of factors that influence IKT practice and outcomes that could be used to strengthen IKT and guide ongoing research. Regular meetings of researchers, decision-makers and end-users were the most common forum for IKT. Meetings were most prevalent in the program that featured some contextual factors identified by PARIHS -, in particular leadership and capacity (infrastructure, dedicated funding, clear goals) and several additional contextual factors and associated outcomes identified by this research, and which achieved a variety of social, research and health service outcomes. The additional contextual factors unique to this research have been captured by sociological theories including social, political, and economic factors that may influence individual beliefs and practices, or programmatic support for IKT . The outcomes achieved by the more established and IKT-active program included use of research considered as conceptual (awareness, acceptance) and instrumental (decision-making) . Findings were captured in a conceptual framework of factors that influence IKT practice and outcomes. Recommendations were issued for a variety of strategies that could support and optimize IKT. Recommendations were similar to factors that enabled collaboration in literature on management networks, interprofessional health services research, teamwork, knowledge translation, communities of practice, and quality improvement collaboratives .
The interpretation and application of study findings may be limited by several issues. Few documents and meetings were available for two of three programs, however, that data was retained to compare the lack of development of those programs and in associated IKT given the limited nature of certain contextual factors such as program funding, leadership, infrastructure and joint goals. We also may not have interviewed all relevant stakeholders associated with each program, partly due to non-response. However we did sample for interviews according to program and professional role within each program so findings likely captured a variety of views that may have been influenced by these factors, and interview data was supplemented with information from content analysis of documents and meeting observation. The chosen programs may not have revealed a variety of IKT activity or possible factors influencing IKT given that two contexts were not well established or active. Further research of a similar nature but in additional programs would extend these findings and further examine the influence of context on the prevalence, nature and outcomes of IKT.
Study findings confirm those of a case study that evaluated three health service delivery programs which found that IKT was highly influenced by the complex context within which decisions were made including social and political issues . It is recommended that these contextual factors be considered when tailoring or implementing programs or interventions . Other research that evaluated the influence of context focused on the implementation or adoption of clinical practices in primary, acute and long-term care settings ,. The conceptual framework generated here provides organizations with a means of planning or evaluating contextual issues specific to the use and impact of IKT. Existing research offers limited insight on the approaches or strategies by which to operationalize IKT. Suggestions generated by other researchers included identifying partners with pre-established links to ease and expedite interaction; establishing clear expectations about role, scope and contribution to foster trust and avoid role confusion and misconceptions; fostering dialogue; and assessing progress to implement changes as needed . These findings add to that body of knowledge by offering a detailed framework of individual and organizational contextual factors to consider.
A key role appeared to be that of an embedded scientist. This was identified as one of the few mechanisms for enabling IKT in a survey of health policy experts representing 30 European countries . This individual was an active participant in planning committees, generated and contributed to research, and offered information and insight that influenced decisions about program planning and evaluation. Further research should explore the role of embedded scientists to elaborate on the scope of such roles, how they can be supported and their impact. While this research did identify contextual factors that influenced IKT along with challenges and suggested enablers, it identified few naturalistic IKT activities upon which to build, an approach theorized to result in greater acceptance and sustained use of IKT . When it occurred, most interaction took place at committee meetings, and through joint development of guidelines and collaborative research. Team meetings and teleconferences were also the main IKT activity in an international network of over 60 researchers, trainees and decision-maker partners from Canada, the United States, the United Kingdom, Asia, Europe and Australia after its first three years of operation . Views expressed by middle and high level managers suggest they prefer published research over interaction with researchers, and non-embedded researchers noted that it was difficult to access opportunities for collaboration. Overall, most individuals had a limited understanding of IKT and how it could be achieved, and disagreed on whether more meetings were desirable. Other research has identified these and other challenges, including a lack of incentives for undertaking IKT ,,,. That was certainly evident here. Research is needed to further understand this tension between lack of time and incentives for interaction, and the need for interaction to support IKT. Such research may reveal options other than meetings by which to achieve IKT.
Several studies have identified tangible improvements in health services or clinical outcomes as a result of IKT -. Several positive outcomes were achieved in the participating programs but not health service outcomes. While that may be due to the limited infrastructure for, and challenges of IKT identified in the contexts studied here, longitudinal evaluation may be needed since such impact may not be immediate. A study that examined self-reported research use by pediatric nurses found that cultural norms and formal interaction were negatively associated with conceptual research use (cognitive awareness and acceptance), which is thought to be a precursor to instrumental research use (actual application in decision-making or behavior) . This emphasizes the need to longitudinally examine a variety of outcomes associated with IKT including cognitive impact. Kothari et al. generated indicators reflecting the capacity for, and impact of collaboration by interviewing researchers and policy decision-makers affiliated with a ten-year Ministry of Health and Long Term Care initiative . The indicators included measures of partnership and measures reflecting the impact of partnership that differed early in the relationship and longitudinally after partnerships matured.
Regular meetings of researchers, decision-makers and end-users were the most common forum for IKT across all programs. Meetings were most prevalent in the program with leadership, clear goals, dedicated funding and other infrastructural resources, and an embedded researcher responsible for, and actively engaged in IKT. This program achieved a variety of social, research and health service outcomes despite mixed individual views about the value of IKT and the absence of a programmatic culture of IKT. Together with challenges and enablers identified by participants, this study generated a conceptual framework of factors that influence IKT practice and outcomes which can be used by others to plan or evaluate IKT. Further research is needed to explore whether these findings are widespread, and understand how IKT can be optimized. However, these findings can be applied by researchers, clinicians, managers or policy-makers to plan or improve collaborative decision-making for health services planning, delivery, evaluation or quality improvement.
ARG conceived the study, acquired funding, coordinated all aspects of its conduct including training and oversight of the research assistant who collected data and independent review of all data analyses, and prepared this manuscript. All authors contributed to study planning and data interpretation, and reviewed and approved the final version of this manuscript. In particular, FW provided guidance on qualitative study design and analysis; NNB, AF and SG identified relevant documents, meetings and stakeholders as key informants for three contexts; and MCB also suggested documents, meetings and stakeholders as sources of data.
This research was conducted with funding from the Cancer Care Ontario - Ontario Institute of Cancer Research Health Services Research Program - Knowledge Translation Research Network, which took no role in the study.
- Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy. 2002, 7: 239-244. 10.1258/135581902320432778.View ArticlePubMedGoogle Scholar
- Ouimet M, Landry R, Amara N, Belkhodja O: What factors induce health care decision-makers to use clinical guidelines? Evidence from provincial health ministries, regional health authorities and hospitals in Canada. Soc Sci Med. 2006, 62: 964-976. 10.1016/j.socscimed.2005.06.040.View ArticlePubMedGoogle Scholar
- Lomas J: Using `linkage and exchange’ to move research into policy at a Canadian foundation. Health Aff. 2000, 19: 236-240. 10.1377/hlthaff.19.3.236.View ArticleGoogle Scholar
- Kothari A, Wathen CN: A critical second look at integrated knowledge translation. Health Policy. 2013, 109 (2): 187-191. 10.1016/j.healthpol.2012.11.004.View ArticlePubMedGoogle Scholar
- Bowen SJ, Graham ID: From knowledge translation to engaged scholarship: Promoting research relevance and utilization. Arch Phys Med Rehabil. 2013, 94 (1 Suppl): S3-S8. 10.1016/j.apmr.2012.04.037.View ArticlePubMedGoogle Scholar
- Denis JL, Hebert Y, Langley A, Lozeau D, Trottier LH: Explaining diffusion patterns for complex health care innovations. Health Care Manage Rev. 2002, 27 (3): 60-73. 10.1097/00004010-200207000-00007.View ArticlePubMedGoogle Scholar
- Dooley KL: A complex adaptive systems model of organization change. Nonlinear Dynamics Psychol Life Sci. 1997, 1 (1): 69-97. 10.1023/A:1022375910940.View ArticleGoogle Scholar
- Oborn E, Barrett M, Prince K, Racko G: Balancing exploration and exploitation in transferring research into practice: a comparison of five knowledge translation entity archetypes. Implement Sci. 2013, 8: 104-10.1186/1748-5908-8-104.View ArticlePubMedPubMed CentralGoogle Scholar
- Wilkinson H, Gallagher M, Smith M: A collaborative approach to defining the usefulness of impact: lessons from a knowledge exchange project involving academics and social work practitioners. Evid Policy. 2012, 8 (3): 311-327. 10.1332/174426412X654040.View ArticleGoogle Scholar
- Allender S, Nichols M, Foulkes C, Reynolds R, Waters E, King L, Gill T, Armstrong R, Swinburn B: The development of a network for community-based obesity prevention: the CO-OPS Collaboration. BMC Public Health. 2011, 11: 132-10.1186/1471-2458-11-132.View ArticlePubMedPubMed CentralGoogle Scholar
- Tee S, Bockle E: Closing the gap - A partnership approach to community care education for long term conditions. Nurse Edu Today. 2012, 32: 822-828. 10.1016/j.nedt.2012.05.018.View ArticleGoogle Scholar
- King G, Currie M, Smith L, Servais M, McDougall J: A framework of operating models for interdisciplinary research programs in clinical service organizations. Eval Program Plan. 2008, 31: 160-173. 10.1016/j.evalprogplan.2008.01.003.View ArticleGoogle Scholar
- Hall JG, Buchan A, Cribb A, Drummond J, Gyles C, Hicks TP, McWilliam C, Paterson B, Ratner PA, Skarakis-Doyle E, Solomon P: A meeting of the minds: interdisciplinary research in the health sciences in Canada. CMAJ. 2006, 175 (7): 763-771. 10.1503/cmaj.060783.View ArticlePubMedPubMed CentralGoogle Scholar
- Dayan PS, Osmond M, Kuppermann N, Lang E, Klassen T, Johnson D, Strauss S, Hess E, Schneider S, Afilalo M, Pusic M: Development of the capacity necessary to perform and promote knowledge translation research in emergency medicine. Acad Emerg Med. 2004, 14 (11): 978-983. 10.1111/j.1553-2712.2007.tb02376.x.View ArticleGoogle Scholar
- Tetroe JM, Graham ID, Foy R, Robinson N, Eccles MP, Wensing M, Durieux P, Legare F, Nielson CP, Adily A, Ward JE, Porter C, Shea B, Grimshaw JM: Health research funding agencies’ support and promotion of knowledge translation: an international study. Milbank Q. 2008, 86 (1): 125-155. 10.1111/j.1468-0009.2007.00515.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Ettelt S, Mays N: Health service research in Europe and its use for informing policy. J Health Serv Res Policy. 2011, 16 (suppl 2): 48-60. 10.1258/jhsrp.2011.011004.View ArticlePubMedGoogle Scholar
- Hofmeyer A, Scott C, Lagendyk L: Researcher-decision-maker partnerships in health services research: Practical challenges, guiding principles. Health Serv Res. 2012, 12: 280-10.1186/1472-6963-12-280.View ArticleGoogle Scholar
- Ward V, Smith S, House A, Hamer S: Exploring knowledge exchange: a useful framework for practice and policy. Soc Sci Med. 2012, 74 (3): 297-304. 10.1016/j.socscimed.2011.09.021.View ArticlePubMedGoogle Scholar
- Ilott I, Gerrish K, Pownall S, Eltringham S, Booth A: Exploring scale-up, spread and sustainability: an instrumental case study tracing an innovation to enhance dysphagia care. Implement Sci. 2013, 8: 128-10.1186/1748-5908-8-128.View ArticlePubMedPubMed CentralGoogle Scholar
- McDonald KM: Considering context in quality improvement interventions and implementation: concepts, frameworks and application. Acad Pediat. 2013, 13 (6S): S45-S53. 10.1016/j.acap.2013.04.013.View ArticleGoogle Scholar
- Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M, Baker R, Eccles MP: A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci. 2013, 8: 35-10.1186/1748-5908-8-35.View ArticlePubMedPubMed CentralGoogle Scholar
- Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, Estabrooks C: Ingredients for change: revisiting a conceptual framework. Qual Saf Health Care. 2002, 11: 174-180. 10.1136/qhc.11.2.174.View ArticlePubMedPubMed CentralGoogle Scholar
- Cummings GG, Estabrooks CA, Midodzi WK, Wallin L, Hayduk L: Influence of organizational characteristics and context on research utilization. Nurs Res. 2007, 56 (4S): S24-S39. 10.1097/01.NNR.0000280629.63654.95.View ArticlePubMedGoogle Scholar
- Rycroft-Malone J, Seers K, Chandler J, Hawkes CA, Crichton N, Allen C, Bullock I, Strunin L: The role of evidence, context and facilitation in an implementation trial: implications for the development of the PARIHS framework. Implement Sci. 2013, 8: 28-10.1186/1748-5908-8-28.View ArticlePubMedPubMed CentralGoogle Scholar
- Auerbach CF, Silverstein LB: Qualitative Data: An Introduction to Coding and Analysis. 2003, New York University Press, New YorkGoogle Scholar
- Barbour RS: Checklists for improving rigour in qualitative research: a case of the tail wagging the dog?. BMJ. 2011, 322 (7294): 1115-1117. 10.1136/bmj.322.7294.1115.View ArticleGoogle Scholar
- Clark JP: How to peer review a qualitative manuscript. Peer Review in Health Sciences. Edited by: Godlee F, Jefferson T. 2003, BMJ Books, London, 2Google Scholar
- Elo S, Kyngas H: The qualitative content analysis process. J Adv Nurs. 2008, 62 (1): 107-115. 10.1111/j.1365-2648.2007.04569.x.View ArticlePubMedGoogle Scholar
- Hsieh HF, Shannon SE: Three approaches to qualitative content analysis. Qual Health Res. 2005, 15 (9): 1277-1288. 10.1177/1049732305276687.View ArticlePubMedGoogle Scholar
- Strauss A, Corbin J: Basics of Qualitative Research: Grounded Theory Procedures and Techniques. 1990, Sage, Newbury Park, CAGoogle Scholar
- Strauss AL: Qualitative Analysis for Social Scientists. 2003, Cambridge University Press, CambridgeGoogle Scholar
- Pope C, Ziebland S, Mays N: Analysing qualitative data. Br Med J. 2000, 320: 114-116. 10.1136/bmj.320.7227.114.View ArticleGoogle Scholar
- Robinson OC: Relational analysis: An add-on technique for aiding data integration in qualitative research. Qual Res Psychol. 2011, 8 (2): 197-209. 10.1080/14780887.2011.572745.View ArticleGoogle Scholar
- Macionis J, Gerber L: Sociology, Seventh Canadian Edition. 2010, Pearson Education Canada, Don MillsGoogle Scholar
- Squires JE, Estabrooks CA, Scott SD, Cummings GG, Hayduk L, Kang SH, Stevens B: The influence of organizational context on the use of research by nurses in Canadian pediatric hospitals. BMC Health Serv Res. 2013, 13: 351-10.1186/1472-6963-13-351.View ArticlePubMedPubMed CentralGoogle Scholar
- Gagliardi AR, Dobrow MJ, Wright FC: How can we improve cancer care? A review of interprofessional collaboration models and their use in clinical management. Surg Oncol. 2011, 20 (3): 146-154. 10.1016/j.suronc.2011.06.004.View ArticlePubMedGoogle Scholar
- Sharp ND, Pineros SL, Hsu C, Starks H, Sales AE: A qualitative study to identify barriers and facilitators to implementation of pilot interventions in the Veterans Health Administration (VHA) Northwest Network. Worldviews Evid Based Nurs. 2004, 1 (2): 129-139. 10.1111/j.1741-6787.2004.04023.x.View ArticlePubMedGoogle Scholar
- Cammer A, Morgan D, Stewart N, McGilton K, Rycroft-Malone J, Dopson S, Estabrooks C: The hidden complexity of long-term care: how context mediates knowledge translation and use of best practices. Gerontologist 2013, doi: 10.1093/geront/gnt068.Google Scholar
- Kothari A, Sibbald S, Wathen CN: PreVAiL. Partnership Evaluation Report: Phase1. Preventing Violence Across the Lifespan Research Network. 2013, University of Western Ontario, London ONGoogle Scholar
- Kothari A, MacLean L, Edwards N, Hobbs A: Indicators at the interface: managing policymaker-researcher collaboration. Knowledge Manage Res Pract. 2011, 9: 203-214. 10.1057/kmrp.2011.16.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.