The use of co-production, co-design and co-creation to mobilise knowledge in the management of health conditions: a systematic review
BMC Health Services Research volume 22, Article number: 877 (2022)
Knowledge mobilisation is a term used in healthcare research to describe the process of generating, sharing and using evidence. ‘Co’approaches, such as co-production, co-design and co-creation, have been proposed as a way of overcoming the knowledge to practice gap. There is a need to understand why researchers choose to adopt these approaches, how they achieve knowledge mobilisation in the management of health conditions, and the extent to which knowledge mobilisation is accomplished.
Studies that explicitly used the terms co-production, co-design or co-creation to mobilise knowledge in the management of health conditions were included. Web of Science, EMBASE via OvidSP, MEDLINE via OvidSP and CINHAL via EBSCO databases were searched up to April 2021. Quality assessment was carried out using the Joanna Briggs Institute qualitative quality assessment checklist. Pluye and Hong’s seven steps for mixed studies reviews were followed. Data were synthesised using thematic synthesis.
Twenty four international studies were included. These were qualitative studies, case studies and study protocols. Key aspects of ‘co’approaches were bringing people together as active and equal partners, valuing all types of knowledge, using creative approaches to understand and solve problems, and using iterative prototyping techniques. Authors articulated mechanisms of action that included developing a shared understanding, identifying and meeting needs, giving everyone a voice and sense of ownership, and creating trust and confidence. They believed these mechanisms could produce interventions that were relevant and acceptable to stakeholders, more useable and more likely to be implemented in healthcare. Varied activities were used to promote these mechanisms such as interviews and creative workshops. There appeared to be a lack of robust evaluation of the interventions produced so little evidence in this review that ‘co’approaches improved the management of health conditions.
Those using ‘co’approaches believed that they could achieve knowledge mobilisation through a number of mechanisms, but there was no evidence that these led to improved health. The framework of key aspects and mechanisms of ‘co’approaches developed here may help researchers to meet the principles of these approaches. There is a need for robust evaluation to identify whether ‘co’approaches produce improved health outcomes.
The term ‘knowledge mobilisation’ is used in the healthcare literature to describe the active, iterative and collaborative process of creating, sharing and using research evidence [1, 2]. Ideally all forms of knowledge, such as experience, values and beliefs are considered in this process—not just scientific factual knowledge [3, 4]. This is in contrast to the term ‘evidence’ where patients’ voices are considered bottom of the evidence hierarchy . Research and healthcare practice inhabit very different worlds, with contrasting goals and using different languages . A shift from hierarchical models of evidence, that favour scientific/medical knowledge, to other forms where patient voice is more at the forefront has been recommended . This has led to a change from linear, rational approaches to knowledge mobilisation to more disordered, relational, context driven ones [4, 5]. Knowledge mobilisation as a concept remains confusing and is often considered an umbrella term for other forms of knowledge sharing and use such as knowledge translation, exchange and dissemination [3, 5, 6]. These terms are frequently used interchangeably within the literature.
Involving patients and clinicians in the generation of new knowledge is considered important to ensure research findings are impactful and to reduce research waste [7, 8]. The need to make public services evidence-based remains of high importance  in order to improve the management of health conditions such as cardiovascular disease, osteoarthritis and cancer. Many of these health conditions require long term management that place high burden on healthcare services . Sharing and generating knowledge between patients and clinicians can help improve understanding of living with and treating these conditions. This can positively impact disease progression, burden of care and health outcomes . However involving patients and clinicians in research or service improvement is challenging and sometimes tokenistic . Social hierarchies exist which means not all knowledge is valued and considered equally . Co-creative approaches to knowledge production have been advocated to bridge the knowledge-to-practice gap [5, 8]. There are many different collaborative and participatory methods in the health research and service improvement literature , with a multitude of approaches being used. Co-production, co-design and co-creation are common terms; these terms have been summarised as ‘co’approaches . The fundamentals of ‘co’approaches have been described in the literature, for example the UK’s National Institute for Health Research (NIHR) principles for co-production . Despite this, there is little consensus about the type of approaches the three terms describe [11, 13]. Common uses of these terms are: 1) co-production of a research project where researchers, practitioners and the public work together throughout the course of the project ; 2) co-creation of new knowledge by academics working alongside other stakeholders  and; 3) co-design when developing complex interventions . In practice, the three terms are often used interchangeably and adopted and described inadequately and ambiguously [11, 15]. Many ‘co’approaches do not address the egalitarian and utilitarian values of what is considered ‘genuine’ co-production leading to a crowded landscape of terms and approaches beginning with the word ‘co’ that Williams et al. (2019) have described as ‘cobiquities’ .
There is currently a lot of interest in knowledge mobilisation and ‘co’approaches in health, with multiple publications about their use. Several reviews have explored the use of specific co-production, co-design or co-creation processes. A recent review undertook content analysis of the co-creation of knowledge for health interventions aiming to reduce the term’s ambiguity and provide a clear definition . The authors developed a new evidence-based definition of knowledge co-creation but included a number of other ‘co’ terms within this, still leaving the reader to address a confusing landscape of ‘cobiquities’. A rapid review of research co-design in health settings had a specific focus on the planning stages of a research project only . Another review sought to understand the outcomes associated with developing and implementing co-produced interventions in acute healthcare settings . The latter reported findings related to understanding the processes of co-designing a service rather than evaluating outcomes themselves. They found different forms of co-production were reported, often uncritically, with a lack of consistent use of terminology to support this diverse range of participatory approaches [16, 17].
To the authors’ knowledge there has yet to be a systematic review that has specifically explored the use of ‘co’approaches in knowledge mobilisation in the management of health conditions. This systematic review aimed to explore why researchers use ‘co’approaches, how researchers think ‘co’approaches can achieve health improvement, the activities they use, and whether they achieve knowledge mobilisation in the management of health conditions (actual or perceived).
This is a mixed studies systematic review, that is, a comprehensive review and synthesis of a wide range of literature of diverse designs . Mixed studies reviews are useful for understanding complex phenomena such as ‘co’approaches for knowledge mobilisation. Seven standard systematic review steps for mixed studies reviews have been followed : 1. Writing a review question. 2. Defining eligibility criteria. 3. Applying an extensive search strategy in multiple information sources. 4. Identifying potentially relevant studies (by two independent researchers screening titles and abstracts). 5. Selecting relevant studies (based on full text). 6. Appraising the quality of included studies using an appropriate tool. 7. Synthesising included studies.
Conduct and reporting of the review followed the Preferred Reporting Items for Systematic reviews and Meta Analysis checklist and flow chart to ensure transparency and complete reporting of the findings . The review was registered with PROSPERO (registration number CRD42020187463 September 2020).
What is the rationale for using ‘co’approaches to mobilise knowledge in the management of health conditions?
What mechanisms of ‘co’approaches achieve knowledge mobilisation (actual or perceived) in the management of health conditions?
What type of activities are used within ‘co’approaches to mobilise knowledge in the management of health conditions?
To what extent do ‘co’approaches achieve knowledge mobilisation (actual or perceived) to help manage health conditions?
Defining eligibility criteria
Specific inclusion and exclusion criteria were defined using the PICOS framework, Population, Intervention, Context, Outcome and Study type . See Table 1. One of three common terms, that is co-production, co-design and co-creation, had to be explicitly used in a paper for inclusion in this review.
Applying an extensive search strategy in multiple information sources
Systematic search of academic literature
Searches were conducted of four electronic databases: Web of Science (all databases) 1970—April 2021, EMBASE via OvidSP 1988 – April 2021, MEDLINE via OvidSP 1946 – April 2021, CINHAL via EBSCO 1981—April 2021. Initial full database searches were carried out up to 26th May 2020. Search alerts were used from this point on for all four databases up until the end of April 2021. The University of York’s Centre for Reviews and Dissemination database, the Cochrane Library (CENTRAL) and Trip medical database were also searched. Bibliographic searches of selected articles reference lists were browsed for any additional relevant studies .
Structured search of the grey literature
Grey literature (unpublished) searches were also conducted to identify any literature from non-traditional sources and to minimise publication bias . Grey literature sources such as Open Grey and Google were conducted as well as websites of professional networks in the field, for example the Canadian Integrated Knowledge Translation (IKT) Network. It is acknowledged that a google search may produce many pages of potentially relevant literature. In this case the first eight pages of the google search were screened. At which point the number of relevant literature significantly diminished. Publications situated on the university profile pages of academic experts in co-production and or knowledge mobilisation were also searched. These were identified through a UK Knowledge Mobilisation Alliance and through recommendations of academic peers. Citation searching from the reference lists of included studies was also carried out.
A comprehensive search strategy was developed in conjunction with an information specialist and was performed by the primary reviewer (CG). A wide variety of key search terms, based on terms in the review question, were used. They included free text and subject headings (such as MeSH) where appropriate. Truncationfor certain key words was used for completeness. Boolean logic operators AND / OR were then utilised to combine terms . For example:
Co-production OR co-prod* OR coproduction OR coproduc* OR co production OR co produc*OR codesign OR co-design OR co design OR co-creat* OR cocreat* OR co creat*
Knowledge mobil* OR Knowledge transl*OR knowledge utili*OR knowledge exchange OR knowledge uptake OR Knowledge to action OR Knowledge to practice OR Evidence based practice.
Search terms were purposely limited to try and provide some focus on what is a very crowded and complex landscape. Multiple terms are often used in the literature for co-productive activities which can be confusing. This systematic review purposely sought to provide some clarity on the use of the three common ‘co’ terms, co-production, co-design and co-creation rather than, for example patient and public involvement and engagement. The same can be said for knowledge mobilisation. Therefore this study limited the use of knowledge mobilisation terms to those frequently seen in the healthcare literature and which encompassed a more interactional, two way flow of knowledge. Implementation was specifically not used, even though it could be argued it is the final stage of knowledge mobilisation, so not to cause confusion between these two different but similar terms and their meanings.
See supplementary material 1 (word document) for detailed search terms used.
Identifying relevant studies
All database search results were imported and organised in Endnote X8 and exported to an Excel spreadsheet. Duplicate references were removed. This selection process allowed for transparency and reproducibility . Documents were screened by title and then by abstract using the pre-determined eligibility criteria. Any articles that appeared to fulfil the inclusion criteria were obtained in full [20, 22, 23]. One reviewer (CG) screened all citations by title and abstract and a second reviewer (EC) independently screened 50. A high level of agreement was achieved between CG and EC on initial screening (90%). The remaining 10% were uncertainties mainly on CG’s part, who was an early career researcher. These uncertainties were resolved through discussion with EC, a more experienced researcher. It was therefore agreed, due to the high level of initial agreement and lessons learnt through the discussions, that the process was robust enough for CG to review the remaining titles and abstracts.CG then assessed the full text of all potentially eligible studies and EC reviewed 20% of the full text articles. EC provided a second opinion for papers CG was unclear about. CG and EC discussed any uncertainties and disagreements and reached a consensus on which studies to include.
Data extraction and management
A standardised data extraction form was developed and tested on a small number of selected studies and then refined [20, 23]. The type of data extracted included: study characteristics such as type of study, setting, participant characteristics, rationale given by researcher for using a ‘co’approach, proposed mechanisms of ‘co’approach, type of activities used and outcomes of ‘co’approach (measured or perceived impact on knowledge mobilisation). The first reviewer (CG) extracted the data from all the included studies and a second reviewer (EC) double extracted 20% of papers to ensure consistency.
Appraising the quality of included studies
There was a mixture of study types in this review including qualitative studies, co-design case studies and study protocols. Five of the 24 papers were mixed methods with qualitative research dominance, that is, they collected survey data alongside the main qualitative findings. The Joanna Briggs Institute (JBI) quality assessment checklists were chosen as they cover a variety of study designs . Due to the nature of the included studies, the JBI qualitative quality assessment check list was used for all studies as a ‘best fit’. This was because there are no specific checklistsfor study protocols and case studies. Studies were not excluded based on quality as long as they addressed the focus of the review. This was to ensure no rich and meaningful insights from the data were lost . CG appraised all selected studies and EC double appraised 20% of the selected studies. Any disagreements were resolved through discussion.
Synthesising included studies
A thematic synthesis approach was used based on the principles of Thomas and Harden (2008) . This has three stages: line by line coding of text, development of descriptive themes, and generation of analytical themes . Analytical themes were not relevant for all the research questions so descriptive themes are presented. NVivo QSR (2020) was used to store and organise the extracted data. There was a small amount of quantitative data extracted in this review in the form of descriptive statistics. A convergent integrated approach was used [23, 26]. The quantitative data was ‘qualitized’ and turned into textual descriptions and then combined with the qualitative data [23, 26]. This allowed for a narrative interpretation of the quantitative results .
Characteristics of studies
The searches identified 1171 studies. After deduplication 782 were screened by title and abstract. This was a challenging task due to the broad and varied use of the terms co-production, co-design, co-creation and knowledge mobilisation in the literature. The remaining 286 articles were reviewed in full text to assess their eligibility, resulting in 24 included in the review. See Fig. 1.
Study characteristics are shown in Table 2. The included studies were conducted internationally: in the UK (n = 9) [27,28,29,30,31,32,33,34,35], Australia (n = 7) [36,37,38,39,40,41,42], Canada (n = 5) [43,44,45,46,47], Sweden (n = 2) [48, 49] and Italy/UK (n = 1) . The majority of the studies were qualitative case studies [27,28,29, 32, 33, 35, 36, 38,39,40,41, 50]. Five of these studies collected and presented survey data alongside the narrative data [30, 42, 43, 48, 49]. Three papers were qualitative study protocols [31, 37, 47]. One was a patient-led (co-designed) qualitative study , and there were three case study collections [34, 44, 45]. Numbers of participants varied across studies from 7- 156. All three terms co-production [28, 29, 32,33,34,35, 50], co-design [28, 30, 31, 34, 37,38,39,40,41, 43, 44, 48, 49] and co-creation [36, 45, 47], were used to define their knowledge mobilisation approach.
Quality of studies
Eighteen out of the 24 papers were assessed as moderate to high quality. Three papers—two non-peer reviewed casebooks and a study protocol, were assessed as low quality. Another three papers were deemed low-moderate quality and consisted of another casebook, a study protocol and a qualitative case study. The latter was assessed as low quality due to unclear reporting. It is possible that the casebooks and study protocols scored poorly due to the lack of appropriate assessment tools for these types of publications. (see Table 3).
Overview of Themes
Overall four themes were identified: 1. Key aspects of ‘co’approaches for knowledge mobilisation. 2. Mechanisms of action. 3. Activities used. 4. Outcomes of ‘co’approaches for knowledge mobilisation. The themes and their sub-themes, along with the relationships between them, are illustrated in Fig. 2.
Key aspects of ‘co’ approaches for knowledge mobilisation
The aspects of ‘co’approaches that authors proposed as important to mobilise knowledge to improve the management of health conditions included: bringing people together as active and equal partners, valuing all knowledge, using a creative approach, and iterative prototyping techniques.
Bringing diverse people together as active and equal partners
Forming collaborations between different stakeholders was considered critical [29, 32, 36, 38, 42, 47]. Authors believed that partnership working led to the sharing of goals , responsibilities and decision making throughout the process [27, 30, 31, 44, 47, 48]. Involving the right people in the ‘co’approach was considered to be central to knowledge mobilisation. For example, one study recognized that:
‘involving all stakeholders can provide richer insights than involving patients or professionals alone’ .
Another proposed that by promoting inclusivity:
‘meaningful egalitarian partnerships are formed between participants’ .
‘active agents not merely passive subjects or recipients of services’ .
Valuing all knowledge
Authors acknowledged the existence of disparate types of knowledge in terms of research evidence, experience and opinions. They highlighted the need to include, recognise and understand all knowledge [27, 31, 32, 41, 44, 49] and place equal importance  on evidence-based research knowledge, clinical knowledge and experiential knowledge [27, 28, 36, 40, 47, 50]. Some authors suggested that ‘co’approaches offered an opportunity to generate, share and gain locally generated knowledge and experience from different sources [28, 30, 36, 48].
‘Our approach is potentially efficient in making use of all available knowledge (scientific and ‘practical’); and potentially effective in being grounded in the reality and complexity of applied practice’ .
Using a creative approach
‘the researchers and clinicians in some of the projects found that their experience of working in collaboration on the projects was different to how they had carried out research before (‘game changers’) and opened up new possibilities and capacity’ .
Design and creative practice were recognised as a means to successfully bring the knowledge, skills, expectations and beliefs of heterogeneous groups of people together [28, 32, 34, 50]. Encouraging those involved to think and behave in different ways [29, 30] enhancing idea generation [39, 41].
Maintaining engagement of stakeholders was recognised as difficult. One study found that despite regular project meetings and media awareness campaigns they did not maintain engagement of key stakeholders through to implementation . In contrast other studies [38, 41, 44] that favoured creative activities, felt that their design and participatory methods helped to engage diverse groups of people with varying goals, feelings and abilities. They perceived that their ‘co’approach helped retain engagement even within those groups who do not traditionally get involved in research [34, 35, 39, 50]:
‘The research and development cycle that we employed in this study is an optimal methodology to engage, retain, and work more efficiently with hard-to-reach populations’ .
Innovative, iterative and prototyping techniques
Many of the study authors proposed to use a flexible, iterative process to achieve successful knowledge mobilisation [27, 28, 30, 33, 35, 44, 46]. For example, the iterative PaCER process in one study allowed learning from participants in each phase to inform the next . Another felt that flexibility was essential to adapt knowledge to context in a complex dynamic system such as healthcare .
Iterative prototyping, often used in design practice, was adopted in a number of studies [28, 30, 31, 34, 35, 39,40,41]. Prototyping was considered useful for turning knowledge into practical, tangible objects [28, 34, 35]. For example, one study used quick, easy and cheap, low fidelity prototypes to generate iterative cycles of feedback and development . In other studies, visual design artefacts such as videos, drawings and sketches were used [28, 31, 34, 39,40,41, 50]. Authors felt that ideas could be quickly communicated in this way in simple, understandable forms making knowledge more accessible [28, 30, 34, 50].
Expert facilitation of these varied activities was considered to be crucial to their success. The use of independent facilitators was found to be successful [34, 35, 43]. They appeared to reduce anxieties regarding participation and encourage open and honest contributions [34, 43]:
‘Having a design facilitator enabled visualisation of thoughts and ideas as they arose. This allowed real time synthesis of occurring knowledge, for example through drawings, which was presented in a form that was easy to understand and which accurately represented participant’s views’ .
Alternatively training could be given to enable researchers to facilitate these activities successfully .
Mechanisms of action
‘Co’approaches were considered to achieve knowledge mobilisation through a number of mechanisms of action directly related to the key aspects described. Study authors considered that bringing people together as active partners, valuing all forms of knowledge, using a creative approach and iterative prototyping techniques, could facilitate a shared understanding of the problem and identify important needs and how to meet them, thereby balancing power differentials, offering a sense of ownership, and engendering trust and confidence in solutions.
Authors reported engaging multiple stakeholders in the process could identify wider perspectives and contexts and contribute to a shared understanding of the problems and potential solutions [27, 28, 30, 32, 33, 38, 40, 43, 46].
Using design artefacts to communicate participants’ thoughts and feelings could facilitate the generation of knowledge and develop a mutual understanding of what was important to stakeholders [28, 30, 34, 50]. The use of personas [28, 30, 34, 35] and scenarios [30, 34] were thought to help distance participants from their own positions and prevent a ‘them and us’ dynamic developing .
‘The persona seemed to be particularly powerful for the professional group and prompted a focus on considering the “whole person” experience that the attendees said they may not have considered otherwise’ .
This meant that outputs were a consensus between participants, considering all perspectives, rather than the product of situated assumptions, such as what health care professionals think patients want or need .
Identify and meet needs
Authors described that by bringing diverse groups of people together, pooling their ‘creative assets’ , and considering and valuing their different types of knowledge, expertise and perspectives, they could produce outputs that were tailored to everyone’s needs [29, 32, 38, 41, 42, 46, 47, 50]. They felt that by including people with lived experience of a health condition in the process they were able to contribute their unique perspectives and ideas [29, 32, 35, 48] and the research addressed the areas that patients felt were most important [36, 43]. This challenged the traditional medical model which assumes the clinician knows best [27, 43].
‘because clinical guidelines are often developed using the medical model where clinicians are considered to possess knowledge and expertise over what is best for the patient’ .
By valuing diverse evidence and knowledge, authors perceived that complex systems and services, such as those in healthcare, could be better understood as no one individual could understand them completely [33, 35]. In this way ‘co’approach outputs could attend and align to context [28, 29, 34, 38] including wider organisational factors . Authors felt that using creative and iterative prototyping techniques allowed them to challenge and refine ideas into practical concepts that were fit for purpose and more likely to meet stakeholder needs [30, 38].
Balancing power and voice
Authors felt that balancing power and voice of those involved aided knowledge mobilisation. Authors felt this was achieved in various ways. Two studies suggested that giving clinicians, patients and the public a more active role in the whole research process meant that they felt valued and had a more equal role [29, 45]. In other studies, involving people with lived experience meant their voices were listened to and valued [45, 46]. One study used research based theatre to achieve this :
‘Theatre makers on the panel were able to explain the process of developing research based Theatre and by doing so revealed how the voices of research participants were respected and heard’ .
‘The discussion was not led by power players such as scientists or surgeons that could have used their status to lead the discussion’ .
Several studies [28, 34, 35, 39, 41, 50] found that the use of creative activities had a positive influence on group dynamics. For example one study felt that their design-led activities enabled participants to:
‘share and express themselves in an inclusive environment using a common language.’ .
Another author felt that power hierarchies could be flattened and more voices heard by making ideas tangible . Creative activities were found to be helpful in engaging people ‘who might otherwise have struggled to participate’  and contribute to the process, such as people with verbal communication problems or lower literacy levels [34, 35]. Skilled facilitation was recognised as important in order to manage the power asymmetries found in heterogenous groups of people .
Sense of ownership
Authors anticipated that knowledge could be shared and generated by bringing people together to form collaborative partnerships, creating a sense of ownership and common purpose [28, 44] that would help reduce the research to practice gap . Ownership was reinforced by considering context, implementation and by valuing all stakeholder knowledge [28, 29, 34]:
‘These include developing strong cross-sector partnerships with stakeholders to co- create and share emerging knowledge, integrating and utilizing all stakeholders’ relevant expertise and experience and promoting a sense of ownership and common purpose’ .
Trust and confidence
Authors identified that stakeholders would have more trust and confidence in the final outputs because their needs were identified, a shared understanding was gained, power and voice was attended to and a sense of ownership was achieved [28, 46]. A number of authors deemed their outputs to be more credible, relevant, practical, realistic, and trustworthy, because of their ‘co’approach [28, 29, 33, 34, 39, 40, 42, 43, 46, 48].
‘This experience only confirmed their view that it was important to include representatives of all the relevant professionals in the process of building a model, to make it sufficiently realistic and trustworthy, and to increase the chances of the results being accepted by them and acted upon’ .
Activities used in ‘co’approaches
Authors used a range of activities, regardless of the term used for their ‘co’approach, in order to achieve the mechanisms of action discussed. It is useful to document these because often researchers rely on research methods when other activities can help to achieve these mechanisms (see Table 4). For example a number of studies included creative activities drawn from design, such as drawing and sketching, personas, journey maps and prototyping [27, 28, 30, 31, 34, 35, 38,39,40,41, 48,49,50]. Some used the amalgamation of interview and focus group data to inform their ‘co’approach process [42, 44, 46]. Others were co-production or co-design of a whole research project [29, 44,45,46]. Prioritisation and consensus techniques were common, including nominal group and Delphi techniques [27, 31, 34, 37, 43, 47,48,49]. One study used a writing committee  and others used meetings and discussion groups [27, 32, 36, 37, 44, 49]. Generally some form of workshop was common.
Few of the included studies measured outcomes. Authors tended to describe the outcomes they believed they were more likely to achieve. These included more relevant research products, more usable knowledge, outputs more likely to be implemented in practice, and improved health.
More accessible, relevant and acceptable knowledge mobilisation products
Two authors perceived that their ‘co’approach helped overcome the problem of research and research findings seeming inaccessible and irrelevant to non-academic audiences [28, 35]. Other authors felt their use of visualisations and design artefacts improved the accessibility of knowledge by simplifying complex concepts [28, 30, 35, 39, 50]. Making research and its findings more accessible and relevant was considered an important outcome [35, 43, 47].
‘The participation of end users in the design process ensured that the prototype was accessible to individuals of varying literacy levels with a range of cultural differences’ .
Authors indicated that by using collaborative approaches they could produce more engaging, functional, practical and acceptable products [28, 37, 39,40,41,42]. Findings from user testing of prototype functionalities for an e-mental health management system supported this view [39,40,41]. Authors felt that their participatory ‘co’approach could: ‘help ensure the end product meets everyone’s needs; improve usability; and increase engagement of users’  and ‘could result in better products that are more functional in real-life settings’ .
More usable knowledge products
A number of authors felt their ‘co’approach produced outputs with potential to be useful and useable in practice [28,29,30, 33, 34, 39, 42, 43]. Several felt that their outputs were more likely to be accepted and therefore more likely to be acted upon and used, leading to successful changes in practice [28, 29, 33, 34, 44, 45, 47, 48]. Authors felt that outputs would be fit for purpose in the real world because their ‘co’approach ensured cultural and contextual factors were captured and used to inform their generation [28, 33, 34, 40, 43, 48].
‘Including people with lived experience in guideline development can aid improved understanding of treatment options, greater involvement in health care decision making, and increased satisfaction in primary and secondary health care. This model can be used to to ultimately produce a product that has real‐world utility for patients and their families’ .
Implementation in practice
Authors proposed that because their research was more relevant, acceptable and usable it was more likely to be implemented in practice. A number of studies provided insights into how their outputs had been implemented and impacted on clinical practice both locally and nationally [29, 33, 35, 36, 44, 46].
“because of our adoption of the Toolbox, our implementing clinicians have assessed chronic pain in over 70% of their pediatric patients who may not have otherwise discussed their chronic pain’’ .
Two casebooks used the IKT approach to ensure research outputs were more implementable [44, 45]. Other studies found that prototypes incorporating culturally and contextually specific information had the potential to aid implementation [28, 34, 39,40,41]. most of the studies in this review produced outputs that required further refinement before being ready to be implemented .
It was acknowledged that implementation and sustained engagement with outputs was challenging. In order to achieve sustainability and long term impact after research teams departed local champions were required to continue to drive implementation forward .
None of the included studies in this systematic review undertook an in depth post implementation evaluation nor did they measure or report on specific health outcomes. Many of the authors aspired to, and in some cases reported, the goal of improving healthcare outcomes and quality of care [28, 30, 34, 37, 43, 46, 50]. However, these claims were not based on robust evaluation data and evaluation methods were not clearly reported. A number of authors felt improving the relevance [40, 41, 43, 46], acceptability [37, 40] and usability [40, 41] of outputs would improve outcomes or quality of care.
‘the development of a codesigned conservative model of care involving patients, clinical staff, members of the public and other stakeholders is more likely to be accepted by both providers and users, resulting in a higher rate of stakeholder satisfaction, continuous improvement and a reduced failure risk’ .
Other studies demonstrated actual changes in practice as a result of introducing the co-designed outputs. These included improved consistency in clinician assessment and identification of patient problems that were previously missed , changes to clinical pathways , fewer hospital visits and admissions  and a reduction in the number of patients who failed to attend appointments . Additional positive outcomes such as, patient satisfaction were either shown or perceived to be possible [33, 43].
From the 24 included studies authors’ main reasons for choosing a ‘co’approach were: 1. Bringing people together. 2. valuing all knowledge. 3. To produce more relevant research products. 4. To improve health outcomes. These were achieved through several mechanisms, such as identifying and meeting all stakeholders’ needs and enabling trust and confidence in the outputs. However, there was little evidence that these approaches improved health because of the lack of robust evaluation of the interventions produced. Despite this, the findings provide useful insights into how ‘co’approaches might mobilise knowledge in health condition management and they are aligned with the five principles for co-production described by a leading research funder in the UK . The NIHR  propose the principles of: 1. Sharing power. 2. Including all perspectives and skills. 3. Respecting and valuing all knowledge. 4. Reciprocity and 5. Building and maintaining relationships. Our review builds on these principles by highlighting activities researchers use to achieve them, further key aspects and mechanisms of action, and the relationships between them. For example, sharing of power may be facilitated if the ‘co’approach brings people together as active partners and uses creative activities. Building and maintaining relationships may be promoted by using iterative prototyping techniques. The findings from this review suggest that the process of developing adaptable, visible and tangible outputs helps participants see that their knowledge and ideas have been heard and valued. Participants may have more trust in the process and reciprocity achieved by producing relevant and acceptable outputs that meet everyone’s needs.
Langley et al.’s 2018 ‘collective making’ knowledge mobilisation model  specifically considers the influences of creative practices. The authors propose that their ‘collective making’ ‘co’approach influences the participants involved, the knowledge being mobilised and implementation in a number of ways  similar to the findings in this review. For example, influencing participants through balancing power and voice and enabling articulation of complex concepts; influencing knowledge through accessing, sharing and valuing different types of knowledge; influencing implementation through creating a sense of ownership and trust in the co-created outputs. Our review complements this model and highlights that some researchers believe similar benefits can be gained without the use of creative activities. This review demonstrates that there is no ‘one size fits all’ approach. All three ‘co’approaches, that is co-production, co-design and co-creation, were used in the studies in this review utilising a variety of activities, from research methods such as interviews and focus groups to workshops using creative activities drawn from design.
Strengths and limitations
This is the first systematic review of ‘co’approaches for knowledge mobilisation for the management of health conditions and included a large number of studies. There were however some limitations. First, there was a lack of studies that had formally evaluated the outputs of their ‘co’approach. A review focused explicitly on the effectiveness of interventions for knowledge mobilisation might have identified more relevant literature than our review. Second, the inclusion/exclusion criteria may have excluded some studies. For example, some collaborative and participatory research that could be deemed to sit under the co-production umbrella, such as studies using an IKT approach, were not included because they did not explicitly describe their approach as co-production, co-design or co-creation. The focus of this systematic review was on these three commonly used terms specifically and knowledge mobilsation. Therefore on reflection, we think that this exclusion criterion was necessary in order to make some sense of this diverse and complex field. Third, the elasticity of the term knowledge mobilisation in the healthcare literature meant the inclusion criteria for this term was broader and encompassed other terms such as knowledge exchange and evidence into practice. This meant that there was room for interpretation by the reviewers which may have led to reviewer bias. Fourth, the lack of use of MeSH terms may have reduced the number of search results meaning some potentially relevant papers may have been missed. Finally, the lead reviewer conducted the majority of the screening process and was the author or co-author of some of the included papers. The bias of the first author was minimised to some degree by working closely with a second reviewer and discussions with other authors of the review.
Conclusions and Implications for future research
This systematic review suggests that ‘co’approaches show promise in achieving successful knowledge mobilisation to improve the way health conditions are managed. However, the findings relied heavily on authors’ beliefs, with only some supporting evidence for short term outcomes such as producing acceptable outputs. There is a need for robust evaluation to ascertain the extent to which ‘co’approaches can produce improved health outcomes. A systematic review that evaluates outputs from ‘co’approaches versus those produced using alternative approaches in a diverse range of settings is recommended to assess whether the former are more likely to achieve knowledge mobilisation and improved outcomes.
Finally, undertaking research using ‘co’approaches is no easy task and it is a common criticism within the literature that authors rarely report their activities in detail nor the steps they have taken to adapt their methods to align with the key principles of ‘co’approaches . The themes diagram in this review is a form of logic model  displaying the pathways through which ‘co’approaches might achieve desired outcomes. This could be used as a framework to help people using ‘co’approaches align their chosen activities to the key aspects and mechanisms, as identified within this review, and the principles of ‘co’approaches articulated elsewhere [12, 70]. This will aid transparency in reporting and potentially improve an intervention’s chance of achieving successful knowledge mobilisation.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Melville-Richards L, Rycroft-Malone J, Burton C, Wilkinson J. Making authentic: exploring boundary objects and bricolage in knowledge mobilisation through National Health Service-university partnerships. Evidence and Policy. 2019;2:1–23.
Powell A, Davies H, Nutley S. Missing in action? The role of the knowledge mobilisation literature in developing knowledge mobilisation practices. Evidence and Policy. 2017;13(2):201–23.
Ward V. Why, whose, what and how? A framework for knowledge mobilisers. Evidence and Policy. 2017;13(3):477–97.
Ferlie E, Crilly T, Jashapara A, Peckham A. Knowledge mobilisation in healthcare: A critical review of health sector and generic management literature. Soc Sci Med. 2012;74:1297–304.
Powell A, Davies HTO, Nutley SM. Facing the challenges of research-informed knowledge mobilization: ‘Practising what we preach’? Public Adm. 2018;96(1):36–52.
National Institute of Health Research. https://www.nihr.ac.uk/about-us/glossary.htm?letter=K&postcategory=-1. Accessed 2020 Mar 31.
Locock L, Boaz A. Drawing straight lines along blurred boundaries: qualitative research, patient and public involvement in medical research, co-production and co-design. Evidence and Policy. 2019;15(3):409–21.
Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving Research Impact Through Co-creation in Community-Based Health Services: Literature Review and Case Study. Milbank Q. 2016;94(2):392–429.
Brand S, Timmons S. Knowledge sharing to support long-term condition self-management—Patient and health-care professional perspectives. Health Expect. 2021;24:628–37.
Farr M. Critical Social Policy Power dynamics and collaborative mechanisms in co-production and co-design processes. Critical Soc ialPolicy. 2018;38(4):623–44.
Cowdell F, Dyson J, Sykes M, Dam A, Pendleton R. How and how well have older people been engaged in healthcare intervention design, development or delivery using co-methodologies: A scoping review with narrative summary. Health Soc Care Community. 2020;00:1–23.
NIHR INVOLVE. Guidance on co-producing a research project. 2019.
Williams O, Sarre S, Papoulias SC, Knowles S, Robert G, Beresford P, et al. Lost in the shadows: reflections on the dark side of co-production. Health Research Policy and Systems. 2020;18(1):43.
O’Cathain A, Croot L, Sworn K, Duncan E, Rousseau N, Turner K, et al. Taxonomy of approaches to developing interventions to improve health: a systematic methods overview. Pilot Feasibility Studies. 2019;5(1):41.
Pearce T, Maple M, Shakeshaft A, Wayland S, McKay K. What is the co-creation of new knowledge? A content analysis and proposed definition for health interventions. International Jourmal of Environmental Research and Public Health. 2020;17(7):1–18.
Slattery P, Saeri AK, Bragge P. Research co-design in health: a rapid overview of reviews. Health Res Policy Syst. 2020;18(17). https://doi.org/10.1186/s12961-020-0528-9.
Clarke D, Jones F, Harris R, Robert G. What outcomes are associated with developing and implementing co-produced interventions in acute healthcare settings? A rapid evidence synthesis. BMJ Open. 2017;7:e014650. https://doi.org/10.1136/bmjopen-2016-014650.
Pluye P, Nha HQ. Combining the Power of Stories and the Power of Numbers: Mixed Methods Research and Mixed Studies Reviews. Annu Rev Public Heal. 2014;35:29–45.
Page MJ, Mckenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n160.
Higgins JPT., Thomas J., Chandler J., Cumpston M., Li T., Page MJ. Welch VA, editor. Cochrane handbook for systematic reviews of interventions [Internet]. 2nd ed. Chichester, UK: Wiley Blackwell; 2019. Accessed 2013 Mar 27. Available from: http://www.cochrane.org/handbook/how-cite-version-handbook.
Booth A, Sutton A, Papaioanna D. Systematic Approaches to a successful literaure review. London: SAGE Publications Ltd; 2012.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. J Clin Epidemiol. 2009;62(10):1006–12.
Aromataris E, Munn Z, editor. Joanna Briggs Institute Reviewer ’ s Manual. The Joanna Briggs Institute. 2017. Available from:https://reviewersmanual.joannabriggs.org/
Swaithes L, Paskins Z, Dziedzic K, Finney A. Factors influencing the implementation of evidence-based guidelines for osteoarthritis in primary care: a systematic review and thematic synthesis. Musculoskeletal Care. 2020;18(2):101–10. https://doi.org/10.1002/msc.1452. Epub 2020 Jan 30. PMID: 31997576.
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(45). https://doi.org/10.1186/1471-2288-8-45.
Hong Q.N, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Rev. 2017;6(61). https://doi.org/10.1186/s13643-017-0454-2.
Cowdell F, Ahmed T, Layfield C. Knowledge mobilisation: a UK co-creation study to devise strategies to amend lay and practitioner atopic eczema mindlines to improve consultation experiences and self-management practices in primary care. BMJ Open. 2020;10:36520.
Grindell C, Tod A, Bec R, Wolstenholme D, Bhatnagar R, Sivakumar P, et al. Using creative co-design to develop a decision support tool for people with malignant pleural effusion. BMC Med Inform Decis Mak. 2020;20(1)179. https://doi.org/10.1186/s12911-020-01200-3.
Heaton J, Day J, Britten N, h J, D J, Heaton J, et al. Collaborative research and the co-production of knowledge for practice: an illustrative case study. Implementation Science. 2016;11(1):20.
Knowles S, Hays R, Senra H, Bower P, Locock L, Protheroe J, et al. Empowering people to help speak up about safety in primary care: Using codesign to involve patients and professionals in developing new interventions for patients with multimorbidity. Health Expect. 2018;21:539–48.
Law RJ, Williams L, Langley J, Burton C, Hall B, Hiscock J, Morrison V, Lemmey A, Partridge R, Lovell-Smith C, Gallanders J, Williams N. ‘Function First - Be Active, Stay Independent’ - Promoting physical activity and physical function in people with long-term conditions by primary care: A protocol for a realist synthesis with embedded co-production and co-design. BMJ Open. 2020;10(2): e035686.
Lewando Hundt G, Stuttaford MC, Bryanston C, Harrison C. “Research Usually Sits on Shelves, Through the Play It Was Shared.” Co-producing Knowledge Through Post-show Discussions of Research-Based Theatre. Front Sociol. 2019;4. https://doi.org/10.3389/fsoc.2019.00048.
Reeve J, Cooper L, Harrington S, Rosbottom P, Watkins J. Developing, delivering and evaluating primary mental health care: The co-production of a new complex intervention. BMC Health Serv Res. 2016;16(1):470.
Wolstenholme D Grindell C, Tod A Bec R. Translating Knowledge into Action Report 2019. Using creative methods to co-design better healthcare experiences. Available from: https://elements.shu.ac.uk/viewobject.html?id=150861&cid=1
Wolstenholme D, Poll R, Tod A. Innovating access to the nurse-led hepatitis C clinic using co-production. J Res Nurs. 2020;25(3):211–24.
Dent E, Hoon E, Kitson A, Newburry J, Harvey G, Gill TK, Belby J. Translating a health service intervention into a rural setting: lessons learned. BMC Health Services Research. 2016;16:62.
Livings R, Naylor JM, Gibson K, Dennis S, Thom J, Mills K, Schabrun SM. Implementation of a community-based, physiotherapy-led, multidisciplinary model of care for the management of knee osteoarthritis: Protocol for a feasibility study. BMJ Open. 2020;10(7):1–8.
Milton cA, Hambleton A, Dowling M, Roberts AE, Davenport T, Hickie I. Technology-enabled reform in a nontraditional mental health service for eating disorders: participatory design study. J Med Internet Res. 2021;23(2):e19532.
Ospina-Pinillos L, Davenport T, Diaz AM, Navarro-Mancilla A, Scott EM, Hickie IB. Using participatory design methodologies to co-design and culturally adapt the Spanish version of the mental health eClinic: Qualitative study. J Med Internet Res. 2019;21(8):1–20.
Ospina-Pinillos L, Davenport TA, Navarro-Mancilla AA, Cheng VWS, Alarcón ACC, Rangel AM, et al. Involving End Users in Adapting a Spanish Version of a Web-Based Mental Health Clinic for Young People in Colombia: Exploratory Study Using Participatory Design Methodologies. JMIR Mental Health. 2020;7(2): e15914.
Ospina-Pinillos L, Davenport TA, Ricci CS, Milton AC, Scott EM, Hickie IB, et al. Developing a Mental Health eClinic to Improve Access to and Quality of Mental Health Care for Young People: Using Participatory Design as Research Methodologies. J Med Internet Res. 2018;20(5): e188.
Yeganeh L, Johnston-Ataata K, Vincent AJ, Flore J, Kokanović R, Teede H, et al. Co-designing an Early Menopause Digital Resource: Model for Interdisciplinary Knowledge Translation. Seminars in Reproductive Medcine. 2021. Available from: http://www.thieme-connect.de/DOI/DOI?10.1055/s-0041-1726273.
Fonseka TM, Pong JT, Kcomt BScPhm A, Kennedy SH, Parikh S V., Sagar Parikh C V, et al. Collaborating with individuals with lived experience to adapt CANMAT clinical depression guidelines into a patient treatment guide: The CHOICE-D co-design process. J Eval Clin Pract. 2019;26(4):1–11.
Kothari A, McCutcheon C, Graham I, editors. How We Work Together: The Integrated Knowledge Translation Casebook. Volume 1. Vol. 1. Ottawa; 2019. Available from: https://ktpathways.ca/resources/how-we-work-together-integrated-knowledge-translation-research-network-casebook
Kothari A, Mccutcheon C, Boland L, Graham, editors. How We Work Together. The Integrated Knowledge Translation Network Casebook. No.3. 2020. Available from: https://ktpathways.ca/resources/how-we-work-together-integrated-knowledge-translation-research-network-casebook
Miller JL, Teare SR, Marlett N, Shklarov S, Marshall DA. Support for Living a Meaningful Life with Osteoarthritis: A Patient-to-Patient Research Study. Patient. 2016;9(5):457–64.
Thompson AP, MacDonald SE, Wine E, Scott SD. An Evaluation of Parents’ Experiences of Patient Engagement in Research to Develop a Digital Knowledge Translation Tool: Protocol for a Multi-Method Study. JMIR Research Protocols. 2020;9(8): e19108.
Revenäs Å, Hvitfeldt Forsberg H, Granström E, Wannheden C. Co-Designing an eHealth Service for the Co-Care of Parkinson Disease: Explorative Study of Values and Challenges. JMIR Research Protocols. 2018;7(10): e11278.
Wannheden C, Revenäs Å. How people with parkinson’s disease and health care professionals wish to partner in care using ehealth: Co-design study. J Med Internet Res. 2020;22(9):e19195.
Dal Mas F, Biancuzzi H, Massaro M, Miceli L. Adopting a knowledge translation approach in healthcare co-production. A case study. Management Decision. 2020. Available from: https://www.emerald.com/insight/0025-1747.htm
Bryman A. Social Research Methods. 3rd ed. New York: Oxford University Press; 2008.
Liamputtong P. Qualitative Inquiry. In: Handbook of Research Methods in Health Social Sciences. Singapore: Springer; 2019.
Maggio LA, Sewell JL, Artino AR. The Literature Review: A Foundation for High-Quality Medical Education Research. J Grad Med Educ. 2016;8(3):297–303.
Jones J, Hunter D. Qualitative Research: Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–80.
Waggoner J, Carline JD, Durning SJ. Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Academic Medcine. 2016;91(5):663–8.
Gallagher M, Hares T, Spencer J, Bradshaw C, Webb I. The nominal group technique: A research tool for general practice? Fam Pract. 1993;10(1):76–81.
Kothari A, McCutcheon C, Graham ID, Alvarez G, Beaupre B, Botting I, et al. Defining Integrated Knowledge Translation and Moving Forward: A Response to Recent Commentaries. International Journal of Health Policy Management. 2017;6(5):299.
Sanders EBN, Stappers PJ. Probes, toolkits and prototypes: Three approaches to making in codesigning. CoDesign. 2014;10(1):5–14.
LEGO® Serious Play. [accessed 2021 Nov 16]. Available from: https://www.lego.com/en-us/seriousplay
Wengel Y, McIntosh A, Cockburn-Wootten C. A critical consideration of LEGO SERIOUS PLAY methodology for tourism studies. Tourism Geography. 2019;23(2):1–23.
Wikipedia. [accessed 2021 Nov 16]. Available from: https://en.wikipedia.org/wiki/Sketch_(drawing)
Mitchell C, Theron L, Stuart J, Smith A, Campbell Z. Drawings as research method. In: Theron L, Mitchell C, Smith A, Stuart J, editors. Picturing research. Drawings as visual methodology. Rotterdam; 2011. p. 19–36. https://doi.org/10.1007/978-94-6091-596-3_2.
Sanders EBN, Stappers PJ. Convivial toolbox. Generative research for the front end of design. Amsterdam: BIS Publishers; 2012.
Stickdorn M, Sneider J, editors. This Is Service Design Thinking. Basic- Tools- Cases. Amsterdam: BIS Publishers; 2010.
Hundt GL, Bryanston C, Lowe P, Cross S, Sandall J, Spencer K. Inside, “Inside View”: Reflections on stimulating debate and engagement through a multimedia live theatre production on the dilemmas and issues of pre-natal screening policy and practice. Health Expect. 2011;14(1):1–9.
Collins Dictionary. Available from: https://www.collinsdictionary.com/dictionary/english/blue-sky-thinking
Locock L, Kirkpatrick S, Brading L, Sturmey G, Cornwell J, Churchill N, et al. Involving service users in the qualitative analysis of patient narratives to support healthcare quality improvement. Res Involv Engagem. 2019;5(1). https://doi.org/10.1186/s40900-018-0133-z.
Jungk R, Mullert N. Future workshops : how to create desirable futures. London: Institute for Social Inventions; 1987. p. 126.
Dipex International. [acessed 2021 Aug 1]. Available from: https://dipexinternational.org/
Langley J, Wolstenholme D, Cooke J. “Collective making” as knowledge mobilisation: the contribution of participatory design in the co-creation of knowledge in healthcare. BMC Health Serv Res. 2018;18(1):585.
Mills T, Lawton R, Sheard L. Advancing complexity science in healthcare research: the logic of logic models. BMC Med Res Methodol. 2019;19:55.
The authors would like to thank Angie Rees information specialist at The University of Sheffield for her assistance in developing the search strategy and conducting the literature searches for this systematic review. Also Chris Redford design researcher at Lab4Living, Sheffield Hallam University for his help with figure 2. Rights of retention statement: For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising"
This systematic review was completed as part of a University of Sheffield faculty of Medicine, Dentistry & Health, University Post Graduate Research Committee (UPGRC) Scholarship and publication funded by the University of Sheffield Institutional Open Access Fund.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Grindell, C., Coates, E., Croot, L. et al. The use of co-production, co-design and co-creation to mobilise knowledge in the management of health conditions: a systematic review. BMC Health Serv Res 22, 877 (2022). https://doi.org/10.1186/s12913-022-08079-y