A systematic mixed studies review on Organizational Participatory Research: towards operational guidance
BMC Health Services Research volume 18, Article number: 992 (2018)
Organizational Participatory Research (OPR) seeks organizational learning and/or practice improvement. Previous systematic literature reviews described some OPR processes and outcomes, but the link between these processes and outcomes is unknown. We sought to identify and sequence the key processes of OPR taking place with and within healthcare organizations and the main outcomes to which they contribute, and to define ideal-types of OPR.
This article reports a participatory systematic mixed studies review with qualitative synthesis A specialized health librarian searched MEDLINE, CINAHL, Embase Classic + Embase, PsycINFO, the Cochrane Library, Social Work Abstracts and Business Source Complete, together with grey literature data bases were searched from inception to November 29, 2012. This search was updated using forward citation tracking up to June 2014. Reporting quality was appraised and unclear articles were excluded. Included studies clearly reported OPR where the main research related decisions were co-constructed among the academic and healthcare organization partners. Included studies were distilled into summaries of their OPR processes and outcomes, which were subsequently analysed using deductive and inductive thematic analysis. All summaries were analysed; that is, data analysis continued beyond saturation.
Eighty-three studies were included from the 8873 records retrieved. Eight key OPR processes were identified. Four follow the phases of research: 1) form a work group and hold meetings, 2) collectively determine research objectives, 3) collectively analyse data, and 4) collectively interpret results and decide how to use them. Four are present throughout OPR: 1) communication, 2) relationships; 3) commitment; 4) collective reflection. These processes contribute to extra benefits at the individual and organizational levels. Four ideal-types of OPR were defined. Basic OPR consists of OPR processes leading to achieving the study objectives. This ideal-type and may be combined with any of the following three ideal-types: OPR resulting in random additional benefits for the individuals or organization involved, OPR spreading to other sectors of the organization and beyond, or OPR leading to subsequent initiatives. These results are illustrated with a novel conceptual model.
The model provides operational guidance to help OPR stakeholders collaboratively address organizational issues and achieve desired outcomes and more.
As per PROSPERO inclusion criteria, this review is not registered.
Collaborative approaches to research involve academic researchers partnering with those who have a stake in the research, but do not necessarily have any formal research training. Stakeholders may include patients, policy makers, communities, organizations or any other individual or group who may benefit from or use the results [1,2,3]. The premise is that the non-academic stakeholders have insight into problems and their potential solutions . The non-academic stakeholders may participate in identifying the problem and formulating the research questions, selecting the research methods, collecting the data, analysing the data, interpreting the results, and applying and disseminating results. Throughout these stages of the research process, non-academic stakeholders may participate to varying degrees. Typologies of this participation have been put forth illustrating continua from the non-academics participating passively (e.g., providing input, but not actively engaging in decisions) to taking full control of the study [3,4,5,6]. Further, participatory research can be distinguished by the three main drivers of the various research approaches, namely knowledge translation, social and environmental justice, and self-determination . Herein, we focus on organizational participatory research (OPR) which is most closely aligned with the knowledge translation driver.
An organization is a “context of action in which relationships of cooperation, exchange, and conflict between actors with divergent interests are being established and managed”  and which fluctuates in response to changes in the environment. OPR takes place with and within organizations—in this article, organizations offering healthcare-related services—for the purpose of organizational learning and/or practice improvement [8,9,10,11]. This research approach is rooted in the works of Kurt Lewin on action research  and of Chris Argyris and Donald Schön on action science . It is a capacity building endeavour where capacity building is a process or an approach used to develop, enhance, or leverage a collection of characteristics (e.g., organization stakeholders’ insider knowledge, academic stakeholders’ research methods expertise) for a purpose (e.g., effecting sustainable organizational change) that is context dependent .
Extant reviews of participatory forms of research taking place within healthcare organizations and with their stakeholders (practitioners, staff, management, or service users) report positive outcomes, such as stakeholders gaining confidence, skills, and knowledge; addressing challenges and implementing innovations; improved job and patient satisfaction [3, 14,15,16]. However, attributing the outcomes to the participatory processes has been difficult. A minimum level of non-academic partner participation needed to guarantee success has not been determined, either . Furthermore, these reviews examined studies that were heterogeneous regarding the type and timing of organization stakeholders’ participation. Finally, some reported difficulty applying detailed frameworks of participation as the reviewed studies were often lacking detail [3, 15]. Therefore, to gain clarity regarding how participatory processes contribute to outcomes, we used a framework of two distinct forms of organization stakeholder participation, at opposite ends of the continuum: (1) providing input when consulted by academics (no research co-governance) for at least three research decisions: (a) identification of the research question(s); (b) determination of the methodology, and/or data collection and/or analysis, and/or interpretation of results; (c) implementation or dissemination of results; or (2) co-constructing at least these three research decisions with academic partners (research co-governance) . Regarding OPR processes, we focussed on any academic-organization collaborative activity throughout all phases of a study. We conceived of an OPR outcome as any consequence of an OPR process.
We used a sequential mixed methods review design where the quantitative synthesis of phase-1 informed the qualitative synthesis of phase-2 [17,18,19,20]. Phase-1, reported elsewhere , measured the likelihood of an OPR study yielding ‘extra benefits’. We define ‘extra benefits’ as positive outcomesof an OPR study that clearly do not meet the specific participatory research project objective(s) for change (for a complete definition, see PL Bush, P Pluye, C Loignon, V Granikov, MT Wright, J-F Pelletier, G Bartlett-Esquilant, AC Macaulay, J Haggerty, S Parry, et al. ). We compared OPR studies that clearly represented either Co-construction OPR (n = 83) or consultation OPR (n = 24). We found no significant association between the form of participation and the likelihood of at least one extra benefit of OPR. Yet, when the OPR was initiated by the organization (as opposed to the academic stakeholders or the organization and academic stakeholders together), the likelihood of the study resulting in at least one extra benefit was quadrupled (OR = 4.11, CI = 1.21–14.01) .
This paper reports phase-2 of the review with the following three objectives: 1) to identify OPR processes and the outcomes to which they contribute (descriptive objective); 2) to understand the sequence of OPR processes and outcomes, in particular extra benefits (analytical objective); 3) to identify ideal-types of OPR process-outcome sequences (explanatory objective). OPR is an inherently chaotic or ‘messy’  endeavour. Identifying characteristics of OPR in the empirical literature and assembling them into ideal types  may help to provide some order to our understanding of this messy research approach. The ultimate goal of this mixed studies review is to provide recommendations for OPR practice to help academic and organization research teams achieve their agreed upon objectives and experience extra benefits; thereby, helping to ensure the added value of the participatory approach to their research.
A participatory research approach was used to conduct this systematic review which followed a process as outlined in QN Hong, P Pluye, M Bujold and M Wassef . Specifically, academic researchers and health organization stakeholders on our team co-constructed the research questions, the interpretation of results and the resulting publications. Moreover, some academic and organization stakeholders contributed to various other stages of the review including developing the bibliographic data base search strategy, examining raw data, guiding the analysis, and discussing preliminary results. In this second phase of the review, the subset of studies incorporating a co-construction form of health organization stakeholder participation from Phase-1 was included. The rationale for this purposeful sample was that these studies provide a richer description of participatory process and their consequences. As per PROSPERO inclusion criteria, this review is not registered. The Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ) framework guided the reporting of the synthesis .
Peer reviewed and grey literature databases were searched from inception through 2012 using MEDLINE, CINAHL, Embase Classic + Embase, PsycINFO, the Cochrane Library, Social Work Abstracts, Business Source Complete, ProQuest Dissertations & Theses database, the New York Academy of Medicine – Grey Literature Report, OpenGrey, and Google. The search was updated in 2014 using forward citation tracking. See Additional file 1 for search strategies. Full details regarding search methods are reported in PL Bush, P Pluye, C Loignon, V Granikov, MT Wright, J-F Pelletier, G Bartlett-Esquilant, AC Macaulay, J Haggerty, S Parry, et al. .
The database search identified 13,837 records which were exported to EndNote where duplicates were removed. Forward citation tracking led to an additional 150 records for a total of 8873 unique records. As described elsewhere , nine inclusion criteria were iteratively developed with all research team members. Two independent reviewers screened the records for relevance and selected 992 full texts for further assessment. In all, 140 studies met the inclusion criteria. (See PL Bush, P Pluye, C Loignon, V Granikov, MT Wright, J-F Pelletier, G Bartlett-Esquilant, AC Macaulay, J Haggerty, S Parry, et al.  for full details and flow chart.)
The quality appraisal focused on identifying studies reporting an explicit link between OPR processes and outcomes; 33 studies were excluded because there was no such statement. Among the 107 studies included in phase-1, 83 represent the co-construction type of participation and were thus selected for phase-2.
For each included study, the lead author and two research assistants extracted descriptive data, study objectives, and text passages describing OPR processes, OPR outcomes, and OPR processes explicitly linked with OPR outcomes (Additional file 2). For the purposes of this review, OPR processes that are explicitly linked to outcomes are ‘key processes’ of OPR. Moreover, OPR outcomes are not the study results, but rather anything that occurs following an OPR process. For instance, an OPR outcome that is explicitly linked to an OPR process could be a statement such as: “working together on the OPR project helped to develop the synergy of the partnership”. In this example, the OPR process is ‘working together’ and the partnership synergy is the OPR outcome. Thus, we reviewed the authors’ descriptions of the OPR processes they used and the outcomes they said occurred as a result (in the above example, working together helped to develop the synergy of the partnership). Consequently, many data were extracted from the methods and discussion sections. For studies described across multiple publications, all papers were used, and data were extracted in order, beginning with the earliest publication. Data were entered into one Excel workbook per study with the descriptive and process-outcome data on separate pages. The process-outcome page had one column for processes and one for outcomes. OPR process and outcome passages were pasted in order, one per row. Text passages that made an explicit link between an OPR process and outcome were pasted on the same row, and the process and outcome cells were merged. The top row was reserved for the study objective. The resulting process-outcome pages of each Excel workbook presented a clear story of the study objective, the OPR processes used, and the OPR outcomes observed. The lead author reviewed all Excel workbooks for accuracy of extracted data and of the categorisation of text excerpts as OPR process and/or outcome.
The co-authors of this study are academic and organization stakeholders with experience in OPR. As per a participatory research approach, co-authors participated in decisions throughout this review. Regarding this qualitative synthesis, co-authors commented on the Excel documents prepared in the preceding phase. Then, the lead author copied the process-outcome pages of each Excel workbook into MS Word, removed redundancies, and re-arranged the excerpts into OPR summaries with: (1) objective(s), people and place; (2) OPR processes; (3) linked OPR processes and outcomes. This reflexive step was necessary because much of the data were not the product of empirical investigation, but rather the authors’ reflections on their practical experience with OPR. Yet, all included studies were peer reviewed and most were co-authored by a team of investigators. For the most part, the OPR summaries (raw data) are the study authors’ words; although, some excerpts were rewritten to ensure proper grammar and syntax or paraphrased for the sake of parsimony.
To identify processes linked with outcomes and to understand their sequences throughout an OPR (objectives 1 and 2), the lead author used hybrid (inductive/deductive) thematic analysis . She coded all of the OPR summaries (assigned extracts of text to themes) according to OPR process, OPR outcome, and linked OPR process-outcome using qualitative data analysis software (NVivo 10). Coding the data pertaining only to an OPR process or only to an OPR outcome was important to gain an understanding of the data corpus. However, given the specific objectives of this review, the final synthesis is based solely on the linked process-outcome codes. This coding was guided by partnership synergy theory [25, 26] capacity building [27, 28] and benefits of participatory research  frameworks (deductive coding). Themes and codes outside of these frameworks were also used where the data suggested them (inductive coding).
In line with traditional guidance for consistency and rigour in qualitative thematic data analysis  and its application as ‘qualitative thematic synthesis’ in literature reviews , our synthesis is based on an interpretative method and research meetings, where coding processes were shared and discussed. Through iterations of analysis and discussion of preliminary results with team members, the analysis became increasingly inductive. All 83 summaries were coded, rather than stopping the analysis once saturation had been reached. All results reported herein were found in multiple summaries. The lead author aggregated the codes into OPR process-outcome themes and wrote an overall OPR process-outcome story depicting themes that occur at the beginning of an OPR and others that occur during and toward the end of an OPR. Co-authors revised this story commenting, in particular, on its credibility. The lead author delved back into the data and refined the analysis to address co-authors’ comments. The continued interaction among the university and organization co-authors was critical to recognize, discuss and deal with our biases. Additionally, preliminary findings were presented at academic conferences over the course of the analysis process, and discussions with academic researcher and health practitioner delegates provided further perspectives that were taken into account.
To identify ideal types of OPR (objective 3), the lead author examined the process-outcome sequences that occur toward the end of an OPR and proposed types based on ultimate OPR outcomes. These OPR ideal types were verified and refined during a grouping exercise . Two research assistants with expertise in OPR (RS & JH) and two co-authors (PLB & PP) met to read and discuss the 83 summaries in relation to the proposed types. They grouped the summaries according to the dominant OPR type and defined ideal-types  of OPR.
Characteristics of included studies
In total, 83 studies described across 145 publications, met our inclusion criteria. The complete results of our search and selection processes are published elsewhere . The 83 studies were each reduced to an OPR summary of 450 words, on average (Additional file 3). The coding trees are in Additional file 4. To respond to our first two objectives, below we detail the eight key OPR processes that were identified (objective 1). Four of these occur during subsequent research phases, and four are present throughout an OPR. We then describe and exemplify the extra benefits for the organization stakeholders and the organization as a whole that the OPR processes contribute to (objective 1). These OPR processes and OPR outcomes are presented in our results following the sequence in which they occur in OPR (objective 2). Finally, we present four ideal-types  of OPR (objective 3). Each is illustrated with an OPR summary. To lend credibility to our findings and to help readers relate to them and determine their applicability to their contexts, excerpts from the OPR summaries (raw data) are included throughout the results section. When the excerpts are direct quotations from the original publications, they are indicated in italic text.
Eight key OPR processes
Four phases of OPR
We begin by illustrating the four key processes of OPR that describe the sequence of activities that should be carried out to help achieve extra benefits from the OPR approach.
Form a working group (WG) and hold meetings
Typically, a WG of university and organization stakeholders is formed to carry out the OPR. Organization stakeholders vary depending on the needs of the project, but a multidisciplinary WG is common (e.g., nurse, physician, manager, social worker, etc.). Many conclude that diverse membership is crucial and can help ensure the relevance and uptake of the OPR. For instance, some assert “the diverse blend of experts from the [organization] and the academic investigators has created projects and programs that are better suited to and accepted by the community and with greater chance for sustainability than could have been produced by either group independently” .
Regarding administration’s and management’s participation in OPR, L Hamelin Brabant, M Lavoie-Tremblay, C Viens and L Lefrancois  write that administrators’ involvement helped to “instil the motivation needed to implement new practices.” Others suggest roles management can assume, such as supporting the OPR by reducing red tape, acknowledging progress reports by letter, implementing some recommendations, and encouraging staff to become actively involved in the process.
The reviewed studies suggest that WGs meet regularly, though the frequency of meetings varies (once per week to once every month or two, or less often). Meetings must be facilitated effectively and often an academic partner assumes this role. In one study, WG members “believed the project was well-managed and that this was a factor in its success” [36, 37]. This may suggest the importance of WG meetings being structured and focussed on specific issues or tasks, as described by some author teams.
WG meetings provide opportunities for the interaction, discussion, debate, reflection, and consensus required to enhance the OPR and achieve more nuanced results and products. During meetings, members complete OPR tasks, make decisions, learn from the experience of others, become more familiar with one another, and understand each other more in their various professional contexts, roles, and contributions--ultimately changing and improving practice. Furthermore, the OPR processes contribute to improved professional relationships and collaborations and increased job satisfaction. S Andrews, E Lea, T Haines, J Nitz, B Haralambous, K Moore, K Hill and A Robinson  explain that WG members’ “collaborative working challenged the traditional hierarchical relationships between staff and opened up an opportunity to work in partnership, thus building capacity and empowering staff.”
WG members typically value and benefit from the meetings. In one study, WG members “felt that the [OPR] process was worthwhile. All staff agreed that ‘the meetings were valuable because we could discuss different ideas’. Generally, staff welcomed the opportunity of raising issues and discussing ways that these could be dealt with” .
Collectively determine research objectives
The WG begins by determining objectives that all members consider valuable. This process helps members to develop relationships and contributes to the success of the OPR by, for example, helping WG members to “feel great enthusiasm about the project” . Some suggest that establishing project goals quickly “helps promote unity between the participants working towards this chosen end” . M Wallis and S Tyson  discuss how they determined their OPR objective:
The OPR process began with a number of meetings between members of the team, in which the nature and scope of the problems with practice were explored and critically examined. A multidisciplinary team decided to develop a chemotherapy protocol manual (based on analysis of the data related to timing of the administration protocols and reflection on these results) that would explain all the requirements of each protocol and provide more accurate estimations of the time required to administer the protocol and care for the patient.
While it can be challenging to reconcile differing perspectives and priorities of WG members, addressing organisation stakeholders’ needs can lead to more relevant and longer lasting changes, as underscored by H Waterman, R Harker, H MacDonald, R McLaughlan and C Waterman : “We would argue that, once the project had reconfigured its interests to those of patients, more detailed and lasting suggestions were implemented.” Furthermore, R Khresheh and L Barclay  write: “shared goals guided the researcher and practitioners in their joint work and created commitment for the considerable effort needed for the research to succeed.”
Collectively analyse data
Collective, iterative data analysis provides opportunities for WG members to exchange thoughts, understand each other’s perspectives, and reflect as a group. This helps to further develop relationships. For instance, EM Eisenberg, J Baglia and JE Pynes  note that “working through the narratives with the ER staff gave them a role as research partners and enabled a dialogue that encouraged the ER staff to acknowledge and understand each other’s ways of viewing the world.” Two studies describe what collective data analysis may entail. B Taylor  writes: “Descriptions of participant observation were analysed individually using a reflective analysis method and collectively by group discussion.” For their part, A Gregorowski, E Brennan, S Chapman, F Gibson, K Khair, L May and A Lindsay-Waters  indicate:
The nurse consultants met with the research fellow on a regular basis to analyse the data together. In this way, a number of themes were collaboratively identified and divided into subthemes through paired and group work. Once themes were identified, the nurse consultants agreed to work in pairs on the individual themes. Each took a theme to work on as the primary researcher and another to work on as the secondary researcher. The group then came together to organise the themes and subthemes into a framework.
Collectively interpret results and decide how to use them
Discussing and interpreting results during WG meetings can be validating for organization stakeholders when the results confirm their perceptions. Additionally, WG members can learn how to use study results to make evidence informed decisions about practice, and their motivation to effect change may increase given the awareness they gain regarding the issues under study. One author team writes: “by creating a structured and supportive environment for data interpretation, the study reduced their fear. As they became more familiar with the charts and graphs, they began to look at data as a basis for decision making” . KT Ngwerume and M Themessl-Huber  provide another example: “Discussions about challenges in utilizing this knowledge also made them aware of the difficulties in applying this new knowledge in practice and at the same time provided them with options about how to implement changes.”
Four continuous OPR processes
Our analyses suggest that an additional four key processes are present throughout a successful OPR: communication, relationships development, commitment, and collective reflection. First, effective communication is open, two-way, transparent (e.g., making findings accessible), ongoing, and requires language all OPR team members understand. To exemplify, L Olsen and L Wagner  write: “Using broadly defined terms (e.g., for ‘discharge’ and ‘prevention’) helped participants to find some common ground, despite their different backgrounds and mandates.”
The quality of the communication within the WG can enhance stakeholder involvement and influence the OPR, help pursue issues to stakeholders’ satisfaction, improve team work and team spirit, and help to bring about change, as described by J Bothe and J Donoghue :
As a result of the opportunity to communicate openly with others, in addition to the team’s ability to think and discuss their work critically, their practice became more effective, safer for patients and patient centred. These changes were apparent to others, and provided a model of enablement that is now used elsewhere in the organization.
The meeting facilitator plays an important role by fostering an environment of trust and respect to enhance the sharing of diverse viewpoints. K Galvin, C Andrewes, D Jackson, S Cheesman, T Fudge, R Ferris and I Graham  note that sharing views during meetings “facilitated collaborative working and provided an opportunity to clarify any confusion, air any tensions, and to agree a way forward.” Producing and circulating meeting notes or other documents is one communication method some have used. This can help “to reflect on discussions and ignite new thoughts and deliberations” , and “correct any misunderstandings” . In one study, a public website was used to post information about the OPR:
Interestingly, even when a member could not attend a session, reading the archive allowed them to feel that they had participated and resume without disruption. There was no evidence that the group regressed throughout the year or lost productivity when new members joined, which was unexpected .
Communication between the WG and the rest of the organization regarding the OPR is important and can increase interdepartmental understanding, help to prevent resistance to change and enhance buy-in. Moreover, reaching out to the whole organization increases the number of voices that are heard and taken into account. H Waterman and J Grabham  provide an example:
The action research group was not insular in that it integrated closely with hospital management systems, for example, senior nurse and general management meetings. Some people who attended the action research meetings, also attended these other meetings, and so ideas and actions from one influenced the other. This ‘spreading and enveloping’ of understanding of issues from all perspectives became vital in the acceptance and inauguration of change.
Second, relationships characterised by mutual trust, respect and support, develop over time, and are important to the success of the OPR. AJ Beringer and ME Fletcher  write: “The single most important indicator of full achievement of outcomes was that the work group members developed mutually supportive and trusting relationships between themselves and with the facilitator. Where these relationships did not develop, this impeded achievement.” Strategies for nurturing relationships include recognizing partners’ contributions and providing positive reinforcement. One author team suggests: “the value of commending the partnership and acknowledging the productivity of the collaboration cannot be underestimated as a means reinvigorating the relationship and sustaining the collaboration” [34, 57].
Third, WG members’ commitment can lead to more active involvement and the sustainability of changes as illustrated by the following:
The Admiral Nurses’ commitment to stay with the project, organizational commitment to embed the framework in their practice development strategy and support from service managers contributed to the success of the development and implementation of the Admiral Nurses’ Competency Framework. The project commissioners are continuing their support for the framework by explicitly linking it with work on standards of care, new job descriptions and, importantly, through the role of the Consultant Admiral Nurse .
Fourth, the collective reflection that occurs during WG meetings seems particularly important. Specifically, collectively reflecting on the OPR and about professional practices stimulates personal reflection and objectivity and supports increased confidence, skills, and insights among WG members. This leads to modifications or adjustments to the OPR and the identification of new or additional issues to pursue via OPR.
The opportunity for the [Falls Action Research Group] members to meet with their colleagues, from within their own facility and those from another [Residential Aged Care Facility], fostered the development of new understandings about their respective workplaces and the conditions that shaped their practice. As such, the [Falls Action Research Group] members became more familiar with the contributions their colleagues, from other occupational groups, made to resident care .
Participation in the inquiry by health professionals was important as it fostered cooperation among clinicians and shared decision-making at different levels of interaction. (….) Doctors, pharmacists, and nurses had an opportunity to work closely together on the common goal of implementing change to medication management. In the process, team members also gained a deeper understanding of effective and safe prescribing practices .
The process-outcome sequences above lead to extra benefits for individual stakeholders and the organization as a whole. Regarding stakeholders, the awareness, understanding, and general knowledge they gain about research, their work, and their colleagues’ work has many benefits. For example, they experience a sense of achievement, their clinical confidence, skills, and care practices improve, and they develop a drive to do research. In one study, “data had a direct impact on staff serving to broaden understanding of patients’ views on equipment and preoperative education, and it helped monitor the changes that were being put into place” .
Attitude changes also occur through the OPR process and lead to practice improvements and other benefits.
During the data interpretation activities, the physicians moved from a classical medical orientation of the individual as the unit of analysis to examining disease patterns in the population. The physicians also began to move from an exclusively curative orientation to disease to a more preventive one .
These authors discuss that when partners “saw study results in the form of charts and tables, their level of enthusiasm rose markedly, and they began to participate actively in data interpretation, and to understand how the study could be helpful to them in operating the clinical or preventive sectors of their facility.” Others report that the confidence and sense of ownership organization stakeholders gain from the process increase their autonomy, enthusiasm and responsibility for the work, and empowers them to effect change.
The new behaviours, practices, and skills developed through the OPR also contribute to extra benefits. As S Lauri  writes:
In the opinions of the public health nurses, the conscious implementation of the action model also produced beneficial effects on their work as a whole. Most nurses began to pay more attention to the various areas of child development, and to explore the needs of the child more extensively as the basis of guidance. The objectives and programmes, in the opinion of the public health nurses, gave a direction to and a foundation for the guidance and counselling.
Regarding the organization, OPR that takes place in a part of a health organisation (e.g., hospital ward) can ultimately affect other parts of the organisation, or beyond. Some studies lead to additional OPR or initiatives as the following OPR summary excerpt illustrates.
As a result of the hospital study, the health education staff already plan to focus on alcohol abuse in those communities which seem to have an elevated problem. Tuberculosis has emerged as a second area of outreach program development. The Foundation’s board of directors has used the hospital data in a fundraising effort for a community education and control program. The hospital’s medical director planned to use the data to identify the priority villages for the tuberculosis program .
The OPR process may also expose or raise awareness for additional issues the organization may subsequently address. Two excerpts provide examples:
Using the suggestions for change that had been made by patients, carers and healthcare professionals, eight changes to practice were identified and it was agreed who would be responsible for their implementation. The changes are now being re-evaluated providing the opportunity for healthcare professionals within the colorectal unit, and a new group of patients and carers, to comment on the effects and effectiveness of the changes made thus far .
After the manual and new forms had been used for some time, the nurses thought that there were still inefficiencies in the system. Consequently, further phases of the action research project were designed to improve the patient appointment booking and staff allocation systems .
Four ideal-types of OPR
The third and final objective of this synthesis was to identify ideal types of OPR. We found that the 83 studies reviewed can be grouped into four such ideal-types, as presented in Table 1. Additional file 3 presents the 83 summaries organised by the four ideal-types. Select OPR summaries illustrating the ideal types are presented in Table 2 as follows. The most basic type consists of OPR processes that contribute to achieving intended outcomes. This is depicted in a study by Lucas et al. [36, 37]. This first OPR ideal type can lead to random benefits unrelated to the study objectives (OPR ideal type 2). The summary of an OPR by Barker & Barker  illustrates multiple such benefits. The third ideal type consists of the OPR (or part of it) being replicated elsewhere in the healthcare organization or even outside of it. A study by Boniface et al.  illustrates an OPR in one unit spreading county-wide. In the fourth ideal type, OPR processes contribute to achieving intended outcomes and also to generating, or initiating, new activities or new OPR. The OPR summary of a study by P O’Connor, RR Franklin and CH Behrhorst  mentions a variety of new activities that were initiated as a result of the OPR. Any combination of these four ideal types is possible. Some studies in our sample illustrate two, three or all four of the ideal types. For example, a combination of ideal types 2 and 4 is represented in the summary of the dissertation by Heyns  (see additional file 3, p. 111). The summary of the study by Sorensen and Haugbolle  illustrates a combination of ideal types 1, 2, and 3. (see additional file 3, p. 106). The summary of the study by MS Fagermoen, GA Hamilton, B Svendsen and H Hjellup  illustrates an OPR representing all 4 ideal types (Table 2).
The qualitative synthesis of 83 co-construction OPR studies suggested eight key OPR processes, four that follow the sequence of the research endeavour (form a working group and hold meetings, collectively determine research objectives, collectively analyse data, and collectively interpret results and decide how to use them) and four that can be observed throughout an OPR (communication, relationships development, commitment, and collective reflection). Extra benefits resulting from these key OPR processes occur at the individual and organizational levels.
Together, these key processes and outcomes of OPR can be interpreted through a model for how to conduct an OPR to achieve extra benefits (Fig. 1). This conceptual model suggests that through regular, structured WG meetings stakeholders can voice their varied experiences, ideas for change, fears and other feelings. The WG should assemble a broad variety of stakeholders (practitioners, patients, managers, etc.) and provide a supportive environment with the promise of confidentiality. WG meeting processes include identifying needs and formulating study objectives, collectively analysing data, and discussing results and how to act on them. Circulating meeting summaries between meetings is important for deliberations, to correct misunderstandings, and to help engage WG members who are unable to attend some meetings. During meetings, WG members learn from one another, gain awareness of constraints to addressing the OPR objectives, gain confidence (which in turn increases commitment to continue the research and to take responsibility for decisions and actions) through systematic reflection on the OPR. This is in line with H Waterman, D Tillen, R Dickson and K de Koning  who found that sharing ideas is part of learning that takes place through the process, which in turn increases participation in the research. Discussing OPR results within the WG is valuable in that results often validate perceptions and raise awareness. Discussing results also increases WG members’ understanding of how to use research findings to inform changes, enables joint problem solving, increases members’ motivation to make change, and helps them identify additional needs. In their review, G Munten, J Van Den Bogaard, K Cox, H Garretsen and I Bongers  identified communication and feedback of results strategies as common. Our work goes a step farther in illustrating the potential consequences of such strategies.
The OPR processes in which the WG engages during meetings contribute to improved communications and coordination both within the WG and between the WG and their healthcare organization, which in turn increases the organization members’ acceptance of change. Ultimately, WG members improve or develop new skills (supports empowerment), and teamwork, mutual understanding and job satisfaction are increased or improved. Readiness for change increases and improved care and sustainable changes ensue. Importantly, the changes pave the way for subsequent changes. While H Waterman, D Tillen, R Dickson and K de Koning  cite many of these results as benefits of participation, our synthesis clearly suggests it is the WG meetings that contribute to these outcomes because they provide invaluable time and space for WG members to present, discuss, debate, and reflect on various identified needs. They are also a space to reach consensus or to accept compromise. Similarly, in their review, G Munten, J Van Den Bogaard, K Cox, H Garretsen and I Bongers  found that meetings in small or large groups was the strategy most often cited in included studies. However, they found the nature of the interactions during these meetings was not sufficiently described. They underscore the need for authors to provide detailed descriptions to help understand the ‘black box’ of this research approach in order to close the research-practice gap. We have delved into the ‘mess’  of OPR combining the experiential, propositional and practical OPR knowledge of our diverse team. Examining authors’ reports of their OPR activities and consequences (process-outcome linkages) in their OPR studies helped us to begin to unpack this black box.
Applicability and practical implications of the review findings
Previous reviews, whether about OPR or other types of participatory research, have not described how to conduct this type of research to the same degree, nor have they illustrated how participatory processes contribute to outcomes. In phase-1 of this systematic review , we found five types of extra benefits related to a framework of capacity building: leadership, general workforce development, group benefits, broad systemic developments or changes, academic researchers’ capacity. With this second phase of the review, we have been able to go beyond this description of extra benefits to explain how to achieve them. The conceptual model (Fig. 1), can be applied widely to guide the conduct and assessment of OPR. While based on OPR with healthcare organizations, the processes and outcomes illustrated in the model may be applicable to any organization. Since we focussed on OPR processes linked to OPR outcomes, the data analysed had an inherent chronology (narrative causation) : processes lead to outcomes. This is visible in the final coding trees which consist of process-outcome sequences, each with inductively derived codes that depict passage of time (additional file 4). It is important to note the iterative nature of the processes and outcomes. In practice, OPR is not as linear as the model suggests. Throughout an OPR project, the process-outcome sequences repeat and overlap, and there are feedback loops between them. However, in keeping with the principle of parsimony, our model provides a simple illustration of the overall flow of theses sequences and explains how to conduct OPR to achieve extra benefits. The parsimonious representation is also in line with the philosophy of OPR to produce useful results. Finally, this simplified conceptualisation is an actionable message, which some would argue is necessary for its uptake and use [68, 69]. The model is thus a significant contribution of our work.
The four ideal types illustrate features of learning organisations such as open systems thinking, building individuals’ capacities, sharing knowledge, and learning collectively . Our synthesis illustrates that while OPR is a means to achieve study objectives (basic OPR), it may also lead to any combination of (a) outcomes unrelated to the study, or ‘random sparks’ as described by H Atlan  (OPR ideal type 2), (b) replication of outcomes (OPR ideal-type 3), or (c) initiation of new OPR or activities (OPR ideal type 4) . Replication was also a finding of one OPR review that reported effects beyond the location in 13% of included studies (n = 4) . Moreover, this is inline with the capacity building and sustainability literature [73, 74] which suggests change is more likely to endure if it spreads or multiplies. Thus, OPR as we have operationalised it in our model, can be a means for academic and organization stakeholder to co-create lasting change. Our findings are in line with a CBPR review  found that community-based participatory research leads to new unanticipated projects and activity, can be conceived of as initiation, as per JC Greene, VJ Caracelli and WF Graham . While previous works provide typologies of non-academic stakeholder participation [3, 5, 14], going forward, it may be relevant to categorise studies as per the four ideal-types of OPR.
Limitations and strengths and transferability of the evidence
This qualitative synthesis is not entirely based on empirical results of included studies. In many cases, the data were from the discussion section of the reviewed studies and are, thus, the authors’ reflections on their practical experience with OPR. Given that our analysis is therefore a meta-reflection, it is possible that our results underscore the assumptions and beliefs of the authors of the publications included in this review. Future research should test our conceptual model and ideal types of OPR. However, our team consists of eleven OPR practitioners and scholars with varied frames of reference, and the critical input of each throughout our review helped us to challenge our own assumptions about the data and our analyses; thereby, adding to the trustworthiness of our results. Moreover, the large number of studies included in this review, the multiple descriptions of OPR processes-outcome sequences in these studies, and the data saturation lends credibility to our results.
Although predominantly influenced by nursing studies in hospital settings (given this is the most common type of OPR to date), this review is more comprehensive than previous ones which focussed on OPR in UK health settings  in nursing , on implementation of evidence-based practice in nursing  or in adult intensive care units . We included a variety of types of health OPR in diverse types of healthcare organizations. Moreover, this review overcomes issues previously reported regarding attributing organization members’ participation in research to particular outcomes [3, 14, 15] since we selected studies based on a precise definition of OPR and excluded those that did not make an explicit link between OPR processes and outcomes [3, 14, 15].
Similar to all literature reviews, our work is limited by publication bias. As found in previous reviews [15, 16], the challenges of OPR (i.e., negative outcomes), and means to mitigate with them (processes), were rarely reported in the included studies and are, thus, not covered in this article. Our practical experience, however, suggests that OPR can be quite challenging. A primary study to identify challenges OPR stakeholders face and describe how they deal with them would be a valuable contribution to the literature.
With this review, we have broadened the understanding of OPR and the value of this research approach by identifying and illustrating sequences of OPR process-outcome sequences. Specifically, our results suggest that OPR stakeholders form a WG and hold meetings where they collectively determine the research objectives, analyse the data and interpret the results and decide how to use them. Throughout these research phases, communication, relationships development, commitment, and collective reflection should be maintained. These processes contribute to knowledge, attitude and behaviour changes in the stakeholders and the healthcare organization. Since our analysis is based only on OPR processes that were explicitly linked with OPR outcomes, we assert that these are the key processes to follow when conducting OPR. Moreover, as per the four ideal types of OPR, we submit that when these processes are followed, OPR teams will achieve their objectives, and may also achieve one or more extra benefits in the form of sparks, replication or initiation. Overall, this review provides operational guidance to help OPR stakeholders collaboratively address organizational issues and achieve desired outcomes and more.
Organizational Participatory Research
Cargo M, Mercer SL. The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health. 2008;29.
Green LW, George MA, Daniel M, Frankish CJ, Herbert CJ, Bowie WR, O'Neill M. Study of participatory research in health promotion: review and recommendations for the development of participatory research in health promotion in Canada. Ottawa: The Royal Society of Canada; 1995.
Waterman H, Tillen D, Dickson R, de Koning K. Action research: a systematic review and guidance for assessment. Health Technol Assess. 2001;5(23).
Cornwall A, Jewkes R. What is participatory research? Soc Sci Med. 1995;41(12):1667–76.
Hart EO, Bond M. Action research for health and social care : a guide to practice. Buckingham: Open University Press; 1995.
Holter IM, Schwartz-Barcott D. Action research: what is it? How has it been used and how can it be used in nursing? Journal of Advanced Aging. 1993;18:298–304.
Friedberg E: Local orders : dynamics of organized action. Greenwich, Conn: jai Press; 1997.
Argyris C, Putnam R, Smith DM. Action science: concepts, methods, and skills for research and intervention. 1st ed. San Francisco, CA: Jossey-Bass; 1985.
Bush PL, Pluye P, Loignon C, Granikov V, Wright MT, Pelletier J-F, Bartlett-Esquilant G, Macaulay AC, Haggerty J, Parry S, et al. Organizational participatory research: a systematic mixed studies review exposing its extra benefits and the key factors associated with them. Implement Sci. 2017;12(1):119.
Pluye P, Nadeau N, Lehoux P. Comment favoriser la recherche clinique en pédopsychiatrie? Une expérience de recherche-action collaborative. Santé Mentale au Québec. 2001;16:245–66.
Pluye P, Nadeau N, Nadeau L. Les contraintes imposées par le terrain sur la rigueur. Recherches Qualitatives [wwwrecherche-qualitativeqcca]. 2000;21:125–44.
Lewin K. Action research and minority problems. J Soc Issues. 1946;2:34–46.
Simmons A, Reynolds RC, Swinburn B. Defining community capacity building: is it possible? Prev Med. 2011;52(3–4):193–9.
Munn-Giddings C, McVicar A, Smith L. Systematic review of the uptake and design of action research in published nursing research, 2000-2005. J Res Nurs. 2008;13(6):465–77.
Munten G, Van Den Bogaard J, Cox K, Garretsen H, Bongers I. Implementation of evidence-based practice in nursing using action research: a review. Worldviews Evid-Based Nurs. 2010;7(3):135–57.
Soh KL, Davidson PM, Leslie G, Rahman ABA. Action research studies in the intensive care setting: a systematic review. Int J Nurs Stud. 2011;48(2):258–68.
Heyvaert M, Hannes K, Onghena P. Using mixed methods research synthesis for literature reviews: the mixed methods research synthesis approach. Thousand Oaks, CA: SAGE Publications; 2016.
Pluye P, Hong QN. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu Rev Public Health. 2014;35.
Hong QN, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Systematic Reviews. 2017;6(1):61.
Pluye P, Hong QN, Bush PL, Vedel I. Opening-up the definition of systematic literature review: the plurality of worldviews, methodologies and methods for reviews and syntheses. J Clin Epidemiol. 2016;73:2–5.
Cook T. The purpose of mess in action research: building rigour though a messy turn. Educational Action Research. 2009;17(2):277–91.
Weber M. Economy and Scociety. Los Angeles: CA: University of California Press; 1978.
Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12(1):181.
Fereday J, Muir-Cochrane E. Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods. 2006;5(1):80–92.
Jones J, Barry MM. Developing a scale to measure synergy in health promotion partnerships. Glob Health Promot. 2011;18(2):36–44.
Lasker RD, Weiss ES, Miller R. Partnership synergy: a practical framework for studying and strengthening the collaborative advantage. Milbank Q. 2001;79(2):179–204.
Goodman RM. A construct for building the capacity of community-based initiatives in racial and ethnic communities: a qualitative cross-case analysis. J Public Health Manag Pract. 2008:S18–25.
NSW Health Department. A framework for building capacity to improve health. In: Department NH. Sydney, editor. In. Australia: NSW Health Department; 2001.
Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J, Sirett E, Wong G, Cargo M, Herbert CP, et al. Uncovering the benefits of participatory research: implications of a realist review for Health Research and practice. The Milbank Quarterly. 2012;90(2):311–46.
Boyatzis RE. Transforming qualitative information: thematic analysis and code development. Thousand Oaks, CA: Sage; 1998.
Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008;8(1):e45.
Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M: Guidance on the conduct of narrative synthesis in systematic reviews. Lancaster, UK: Lancaster University; 2006.
Weber M. ‘Objectivity’ in social science and social policy. New York: Free Press; 1904.
Dobransky-Fasiska D, Nowalk MP, Pincus HA, Castillo E, Lee BE, Walnoha AL, Reynolds CF, 3rd, Brown C: Public-academic partnerships: improving depression care for disadvantaged adults by partnering with non-mental health agencies. Psychiatric services (WashingtonDC) 2010, 61(2):110–112.
Hamelin Brabant L, Lavoie-Tremblay M, Viens C, Lefrancois L. Engaging health care workers in improving their work environment. J Nurs Manag. 2007;15(3):313–20.
Lucas B, Cox C, Perry L, Bridges J. Pre-operative preparation of patients for total knee replacement: an action research study. Int J Orthop Trauma Nurs. 2013;17(2):79–90.
Lucas B, Cox C, Perry L, Bridges J. Changing clinical team practices in preparation of patients for Total knee replacement: using social cognitive theory to examine outcomes of an action research study. Int J Orthop Trauma Nurs. 2013;17(3):140–50.
Andrews S, Lea E, Haines T, Nitz J, Haralambous B, Moore K, Hill K, Robinson A. Reducing staff isolation and developing evidence-informed practice in the aged care environment through an action research approach to falls prevention. Adv Nurs Sci. 2012;35(1):3–13.
Lindeman M, Smith R, Vrantsidis F, Gough J. Action research in aged care: a model for practice change and development. Geriaction. 2002;20(1):10–4.
Khresheh R, Barclay L. Practice-research engagement (PRE): Jordanian experience in three Ministry of Health hospitals. Action Res. 2007;5(2):123–38.
Whitehead D, Keast J, Montgomery V, Hayman S. A preventative health education programme for osteoporosis. J Adv Nurs. 2004;47(1):15–24.
Wallis M, Tyson S. Improving the nursing management of patients in a hematology/oncology day unit: an action research project. Cancer Nurs. 2003;26(1):75–83.
Waterman H, Harker R, MacDonald H, McLaughlan R, Waterman C. Advancing ophthalmic nursing practice through action research. J Adv Nurs. 2005;52(3):281–90.
Eisenberg EM, Baglia J, Pynes JE. Transforming emergency medicine through narrative: qualitative action research at a community hospital. Health Commun. 2006;19(3):197–208.
Taylor B. Identifying and transforming dysfunctional nurse-nurse relationships through reflective practice and action research. Int J Nurs Pract. 2001;7(6):406–13.
Gregorowski A, Brennan E, Chapman S, Gibson F, Khair K, May L, Lindsay-Waters A. An action research study to explore the nature of the nurse consultant role in the care of children and young people. J Clin Nurs. 2012.
O'Connor P, Franklin RR, Behrhorst CH. Hospital record studies as a tool for staff education: a participatory research project in Guatemala. J Community Health. 1987;12(2–3):92–107.
Ngwerume KT, Themessl-Huber M. Using action research to develop a research aware community pharmacy team. Action Res. 2010;8(4):387–406.
Olsen L, Wagner L. From vision to reality: how to actualize the vision of discharging patients from a hospital, with an increased focus on prevention. Int Nurs Rev. 2000;47(3):142–56.
Bothe J, Donoghue J. Using action research to develop a model of patient-centred day care. Practice Development in Health Care. 2009;8(3):152–60.
Galvin K, Andrewes C, Jackson D, Cheesman S, Fudge T, Ferris R, Graham I. Investigating and implementing change within the primary health care nursing team. J Adv Nurs. 1999;30(1):238–47.
Day J, Higgins I, Koch T. Delirium and older people: what are the constraints to best practice in acute care? Int J Older People Nursing. 2008;3(3):170–7.
Mitchell EA, Conlon AM, Armstrong M, Ryan AA. Towards rehabilitative handling in caring for patients following stroke: a participatory action research project. J Clin Nurs. 2005;14(3 A):3–12.
Tolson D, Irene S, Booth J, Kelly TB, James L. Constructing a new approach to developing evidence-based practice with nurses and older people. Worldviews Evid-Based Nurs. 2006;3(2):62–72.
Waterman H, Grabham J. Ophthalmic theatre nursing. Part 2: frameworks for practice. Br J Theatre Nurs. 1995;5(3):5.
Beringer AJ, Fletcher ME. Developing practice and staff: enabling improvement in care delivery through participatory action research. J Child Health Care. 2011;15(1):59–70.
Dobransky-Fasiska D, Nowalk MP, Cruz M, McMurray ML, Castillo E, Begley AE, Pyle P, RNDC-community partners, Pincus HA, Reynolds 3rd CF et al: A community-academic partnership develops a more responsive model to providing depression care to disadvantaged adults in the US. Int J Soc Psychiatry 2012, 58(3):295–305.
Dewing J, Traynor V. Admiral nursing competency project: practice development and action research. J Clin Nurs. 2005;14(6):695–703.
Ko A, Leontini R, Ngian V, Hughes I, Clemson L, Chan D. Quality and safety of prescribing practices in aged care and rehabilitation units in an Australian hospital. Asian J Gerontol Geriatr. 2013;8(2):68–77.
Lauri S. The public health nurse as a guide in infant child-care and education. J Adv Nurs. 1981;6(4):297–303.
Gregory S, Poland F, Spalding NJ, Sargen K, McCulloch J, Vicary P. Multidimensional collaboration: reflections on action research in a clinical context. Educational Action Research. 2011;19(3):363–78.
Barker SB, Barker RT. Managing change in an interdisciplinary inpatient unit: an action research approach. J Ment Health Adm. 1994;21(1):80–91.
Boniface G, Fedden T, Hurst H, Mason M, Phelps C, Reagon C, Waygood S. Using theory to underpin an integrated occupational therapy service through the Canadian model of occupational performance. Br J Occup Ther. 2008;71(12):531–9.
Heyns T: A journey towards emancipatory practice development. University of South Africa (South Africa); 2008.
Sorensen EW, Haugbolle LS. Using an action research process in pharmacy practice research-a cooperative project between university and internship pharmacies. Res Soc Adm Pharm. 2008;4(4):384–401.
Fagermoen MS, Hamilton GA, Svendsen B, Hjellup H. Partners in change: action research in action in clinical practice. Nordic Journal of Nursing Research and Clinical Studies / Vård i Norden. 2002;22(3):45–7.
Abbott A. The causal devolution. Sociol Methods Res. 1998;27.
Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J. Knowledge transfer study G: how can research organizations more effectively transfer research knowledge to decision makers? The Milbank Quarterly. 2003;81(2):221–48.
Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implementation Science : IS. 2012;7:50.
JE WILKINSO, RK RUSHMER, HTO DAVIES. Clinical governance and the learning organization. J Nurs Manag. 2004;12(2):105–13.
Atlan H: Les étincelles du hasard. Tome 1 : Connaissance spermatique. Seuil, Paris: La Librairie du XXe siècle; 1999.
Greene JC, Caracelli VJ, Graham WF. Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis. 1989;11:255–74.
Hawe P, Noort M, King L, Jordens C. Multiplying health gains: the critical role of capacity-building within health promotion programs. Health policy (Amsterdam, Netherlands). 1997;39(1):29–42.
Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998;13(1):87–108.
We would like to thank Joseph LeMaster, MD, MPH and Anne MacFarlane, PhD for their comments on preliminary analyses; Reem El Sherif, MSc. MBBCh and Joshua Hamzeh, BSc for their contribution to grouping the 83 summaries according to four ideal types of OPR; Soultana Macridis PhD and Rafaela Queiroga Souto PhD for their help with data extraction.
This systematic mixed studies review was supported by the Canadian Institutes of Health Research (CIHR reference number 136889) and the Quebec-SPOR SUPPORT Unit (http://unitesoutiensrapqc.ca/).
Avalability of data and materials
Ethics approval and consent to participate
Consent for publication
All co-authors declare they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Search strategies. This documents provides the detailed search strategies for each bibliographic data base we searched. (PDF 195 kb)
Data extraction form. This Excel form used for all 83 studies included in this review. (XLSX 12 kb)
Eighty-three Organizational Participatory Research Summaries. This document includes the 83 OPR summaries organized according to four ideal-types of OPR. (PDF 873 kb)
Coding trees. This document includes screenshots of the coding trees with the number of OPR summaries (sources) per code and number of codes per summary (references). (PDF 199 kb)
About this article
Cite this article
Bush, P.L., Pluye, P., Loignon, C. et al. A systematic mixed studies review on Organizational Participatory Research: towards operational guidance. BMC Health Serv Res 18, 992 (2018). https://doi.org/10.1186/s12913-018-3775-5