- Research article
- Open Access
A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in pediatric healthcare
BMC Health Services Research volume 19, Article number: 912 (2019)
In 2012, the Saskatchewan Ministry for Health mandated a system-wide Lean transformation. Research has been conducted on the implementation processes of this system-wide Lean implementation. However, no research has been done on the sustainability of these Lean efforts. We conducted a realist evaluation on the sustainability of Lean in pediatric healthcare. We used the context (C) + mechanism (M) = outcome (O) configurations (CMOcs) heuristic to explain under what contexts, for whom, how and why Lean efforts are sustained or not sustained in pediatric healthcare.
We employed a case study research design. Guided by a realist evaluation framework, we conducted qualitative realist interviews with various stakeholder groups across four pediatric hospital units ‘cases’ at one acute hospital. Interview data was analyzed using an integrated approach of CMOc categorization coding, CMOc connecting and pattern matching.
We conducted thirty-two interviews across the four cases. Five CMOcs emerged from our realist interview data. These configurations illustrated a ‘ripple-effect’ from implementation outcomes to contexts for sustainability. Sense-making and staff engagement were prominent mechanisms to the sustainment of Lean efforts. Failure to trigger these mechanisms resulted in resistance. The implementation approach used influenced mechanisms and outcomes for sustainability, more so than Lean itself. Specifically, the language, messaging and training approaches used triggered mechanisms of innovation fatigue, poor ‘sense-making’ and a lack of engagement for frontline staff. The mandated, top-down, externally led nature of implementation and lack of customization to context served as potential pitfalls. Overall, there was variation between leadership and frontline staff’s perceptions on how embedded Lean was in their contexts, and the degree to which participants supported Lean sustainability.
This research illuminates important contextual factors and mechanisms to the process of Lean sustainment that can be applicable to those implementing systems changes. Future work is needed to continue to develop the science on the sustainability of interventions for healthcare improvement.
Lean is a quality management system aimed at maximising value for customers by reducing waste (e.g., overproduction, wait times, unnecessary inventory and motion) and reconfiguring organizational processes [1,2,3,4]. Lean is increasingly recognized as a potential approach to improve problems in healthcare . Lean has been implemented in a variety of healthcare settings (e.g., emergency departments, outpatient clinics, pediatric care) [6,7,8,9], through a variety of implementation approaches (e.g., mandated at macro level, driven by frontline staff at meso level of a system), targeting various levels of healthcare systems (macro, meso or micro). It has also been conceptualized in multiple ways (e.g., a philosophy or management system versus quality improvement (QI) tools) [10,11,12,13].
The majority of previous research on Lean implementation in healthcare has not addressed the contextual factors and mechanisms that influence the sustainability of Lean efforts [14,15,16,17,18]. Understanding these factors that contribute to its lasting effect are as important as understanding how to implement Lean in the first place . Sustainability is an important yet understudied area of implementation research [20, 21]. This paper presents the final phase of a multi-phase realist investigation on the sustainability of Lean efforts in pediatric healthcare. Phase 1 consisted of initial program theory development. Phase 2 consisted of a realist review to further develop and refine our initial program theory . Phase 3, reported in this paper, was a realist evaluation to test and refine our program theory and context (C) + mechanism (M) = outcome (O) configurations (CMOcs) developed during phase 1 and 2 of this research.
Our research aim was to generate, test and refine a program theory on the sustainability of Lean efforts in pediatric healthcare using a realist approach. We drew from the conceptualization that sustainability is the continuation or the integration of new practice within an organization whereby it has become a routine part of care delivery and continues to deliver desired outcomes, whereby the ways of thinking and attitudes behind processes and outcomes have changed and the new practice has become the new way of working . For our research we undertook a realist approach to identify the contextual factors and mechanisms that influenced whether Lean became a routine part of care delivery, continued to deliver desired outcomes and became the sustained, normalized way of working. We aimed to identify participants’ perceptions of Lean implementation processes, contextual factors, resources and mechanisms that enabled or hindered the sustainment of Lean efforts.
Study design and setting
We followed a realist evaluation methodology framework  with an explanatory case study research design . We defined a ‘case’ as a pediatric unit involved in Lean implementation from one acute hospital setting in the Saskatchewan health system. One central tenet of realist methodology is that programs work differently in different contexts ; hence, we chose to conduct interviews across various cases (units) where Lean implementation had occurred in the research context of the Saskatchewan health system.
In 2012, the Saskatchewan Ministry of Health committed a multi-million dollar investment to implement a broader system-wide Lean transformation, led by an external consulting group . This has been titled “the largest Lean transformation in the world” . The overarching aim was to create “better health, better value, better care, and better teams” . Early-stages of this implementation focused on leadership training, and the creation of Kaizen Promotion Offices (KPOs) to provide supportive infrastructure for the Lean transformation . This was intended to build internal capacity and capability for continuous quality improvement . The Saskatchewan Lean management system was used in combination with Hoshin Kanri, and daily visual management  and used a variety of Lean tools and activities (e.g., Rapid Process Improvement Workshops (RPIW’s), Mistake Proofing, Kanban, and Value Stream Map). The Saskatchewan Ministry of Health  proposed that, “Lean empowers employees to find ways to improve. It focuses on identifying and reducing waste. In healthcare, that would include things like excess inventory, time spent waiting for services, and inefficient processes that reduce time spent on direct patient care.”
This large-scale Lean transformation created a novel opportunity for our research on the sustainability of Lean across multiple pediatric healthcare settings. There is no prior evaluation on Lean sustainability in Saskatchewan and none specific to pediatric healthcare. Pediatric healthcare was an important context for us to study under the Saskatchewan Lean management system due to the development of the provinces first children’s and maternal hospital based on Lean.
Ethics approval for this study was granted by the University of Alberta Health Research Ethics Board and the University of Saskatchewan Behavioural Research Ethics Board. Institutional approval was provided by the Saskatoon Health Region, Inter-professional Practice, Education and Research office.
Recruitment and data collection
To develop an understanding of how Lean efforts were embedded in practice, we purposefully selected four pediatric units (a pediatric inpatient unit, outpatient unit, pediatric intensive care unit (PICU) and neonatal intensive care unit (NICU) involved in the effort. All staff from each included case were invited to participate in the interviews to gain broad and diverse perspectives. Staff was invited to participate through communication by the unit managers at each case and the Director of children’s services. A range of perspectives via professional roles were purposefully sought to contribute to refinement of the program theory.
We conducted qualitative realist interviews  using an interview guide (Additional file 1.) to test and further refine our initial program theory (developed in phase 1), the CMOcs substantiated in our realist review (phase 2) and to explore new emerging relevant CMOcs. The interviews explored participants’ perceptions of Lean, implementation processes, contextual factors, resources and mechanisms that enabled or hindered the sustainment of Lean efforts. Interviews were conducted using semi-structured interview guides. All interviews were conducted in person or by telephone by the lead author (RF), audio recorded and transcribed. Realist interviews explicitly discuss the program theory with the participants, giving them the opportunity to confirm, refute or refine the theory, this is described as the teacher-learner cycle [24, 32].
Methodological approach: realist evaluation
A realist approach offers ways to address how, when, why and where the intervention works or not through the generation of an explanatory program theory . Realist evaluations are driven by the question: what works, how, for whom, in what circumstances and to what extent? . A realist evaluation is underpinned by the context (C) + mechanism (M) = outcome (O) configurations (CMOcs) heuristic . A CMOc is a hypothesis that the program works (O) because of the action of some underlying mechanism (M), which only comes into operation in particular contexts (C) . The development, testing and refinement of CMOcs in a realist evaluation provides causal explanation of how and why a program works . The realist terms used for this research are detailed in Table 1.
We drew upon the National Health Sustainability Model (NHS SM) and Normalization Process Theory (NPT) to help us to understand the contextual factors and mechanisms that trigger the sustainability or otherwise of Lean efforts. The NHS SM identifies ten key contextual factors that increase the likelihood of sustainability and continuous improvement. These factors are grouped into three domains: Process, staff and organization [23, 39]. The NPT is a middle range theory used to understand the implementation, embedding and integration of evidence-based innovations into healthcare settings as a result of people working individually and collectively to enact them [40, 41]. This middle-range theory is concerned with the social organization of the work (implementation), of making practices routine to everyday (embedding), and of sustaining embedded practices in their social contexts (integration) . The key theoretical constructs to NPT are: coherence, cognitive participation, collective action and reflexive monitoring . These constructs of NPT offer potential mechanisms that promote or inhibit the embedding of complex interventions into routine everyday practice and the likelihood of sustainability.
Interviews were analysed using CMOc heuristic . We followed Maxwell’s  categorising and connecting strategies for data analysis. Firstly we analysed each case separately as a ‘whole study’ and then analysed and summarized similar and/or opposing evidence across the four included cases through data triangulation and pattern matching . During categorization, C, M, O coding for each case was done by a process of data extraction using a bespoke form and coding using NVivo 11 software. Cross case comparisons were made to determine how the same causal mechanisms played out in different contexts and produced the same or different outcomes. Connecting data involved pattern matching across cases and CMOcs and higher abstraction. This was done using Microsoft Word 13 and Microsoft Excel 13. During the process of analysis it became clear that significant CMOcs were not confined to the cases which we had identified as contexts but rather by contextual elements (e.g., work areas that experience constant change) that triggered mechanisms (e.g., staff felt overwhelmed from the constant change) that lead to outcomes (e.g., negative perceptions of Lean, resistance and lack of support by frontline staff). Thus we analyzed these CMOc patterns within and across cases.
A visual model was developed to show CMOcs patterns across cases (Additional file 2.). These patterns denote the causal pathways leading to program outcomes. Building upon our previous realist review we showed how CMOcs can be linked to each other - with some outcomes of early implementation becoming an aspect of context for sustainability, this is known as the ‘ripple effect’ .
Analysis was iterative, going ‘back-and-forth’ between the program theory and the CMOcs from phase 2 and the data gathered in phase 3. The intent was to deliberately attempt to refine and specify the program theory on the sustainability of Lean in pediatric healthcare. We also drew upon our middle-range theories to explain contextual factors and causal mechanisms for the sustainability of Lean in the research context under study. The NHS SM was used to explain process, staff and organization factors that influenced the sustainability of Lean from the experiences of participants. NPT was used to identify any mechanisms related to coherence, cognitive participation, collective action and reflexive monitoring that either hindered or facilitated the embedding of Lean from the perceptions of the participants.
Under a realist lens, the focus of validity is the judgement of the degree to which the researcher has encapsulated the multiple perspectives pertaining to a given situation (, p82). We followed Pawson et al.  Transparency, Accuracy, Purposivity, Utility, Propriety, Accessibility and Specificity (TAPUPAS) criteria to enhance the trustworthiness of data collection and documentation. The steps we took to meet the TAPUPAS criteria are outlined in Table 2.
Participant demographics and lean training
Thirty-two realist interviews were conducted; nine participants from case 1, three participants from case 2, seven participants from case 3 and four participants from case 4, and nine participants that were across cases. The total sample consisted of registered nurses (n = 9), unit managers (n = 5), physicians (n = 4), allied health professionals (n = 4), clinical nurse educators (n = 2), care assistants (n = 2), senior leaders (n = 4), and parents (n = 2). Of the thirty-two participants, 30 were female and two were male. The majority of participants had been working in their profession from 1 to 5 years (n = 9), 31–35 years (n = 6), 6–10 years (n = 4) or 16–20 years (n = 4). The remaining participants had been working in the profession between 11 and 15 years (n = 3), 21–25 years (n = 2), 26–30 years (n = 2) or 36–40 years (n = 2). The majority of participants had been working in their current role 1–5 years (n = 14), or 6–10 years (n = 9).
Seventy-five per cent of the participants had received the Kaizen basic Lean training offered through the organization. The most prevalent Lean activities were visual daily management huddles (n = 22), visibility walls/wall walks (n = 21) and 5 s events (n = 19). Lean involvement responses ranged from one to 6 years ago, the most common response was approximately 3 years ago (n = 10). Twenty-eight participants (88%) were aware of Lean events currently taking place on their unit.
Five CMOcs were evident through the realist interviews across each case (Table 3). We have arranged our findings according to these five configurations. We present the most prominent quotes from participants to illustrate each CMOc.
‘Ripple effect’: the funded, mandated, top-down, externally led nature of lean implementation
The majority of the frontline staff participants viewed the top-down, mandated, and externally led Lean implementation negatively. Some staff felt that Lean was a cost cutting measure, a “fashion fad”, something that was pushed on them, where implementation was too quick and did not have a clear purpose. Most of the unit managers also viewed the use of an external consultancy company negatively. Participants were conscious of the estimated costs of the consultancy company’s fees and felt that this money could have been used more appropriately. The consultancy company was viewed as an outsider pushing a message that didn’t connect with healthcare. In contrast, some unit managers and senior leaders did value the top-down, mandated approach used, stating that changes would not have occurred to the same degree without such an approach.
Media had a powerful influence over participant’s perceptions and attitudes towards Lean. The portrayal of Lean in the media was primarily negative, this triggered negative perceptions of Lean by frontline staff. Lean training by the consultancy company did not make staff feel involved in Lean changes. The Lean language used by the consultancy company did not make sense for many participants and initial implementation efforts failed to connect Lean to the context of pediatric healthcare. These factors triggered outcomes of resistance from early-stages of implementation, these implementation outcomes in-turn served as hindering contexts for sustainability. Quotes to support this CMOc are presented in Additional file 3.
Lack of fit between lean and healthcare and a lack of customization to context
In addition to the externally led, mandated implementation of Lean, there was a lack of fit between Lean principles and the healthcare context (e.g., cars versus patient care). There was also a lack of customization to context during early stages of implementation. This resulted in some negative effects, particularly for frontline staff and their support of Lean continuation. The lack of customization to local context triggered mechanisms of disconnect, lack of coherence and negative perceptions about Lean.
Pediatric healthcare was discussed as a complex field which requires a family-centred and flexible approach to care, which some participants believed did not align with Lean. Despite these contextual issues, there were evident shared values between Lean principles and participants’ professional values, such as patient safety, efficiency and waste reduction. However, Lean principles were primarily viewed as incongruent with healthcare. The training provided failed to translate Lean concepts, principles and their meanings from a manufacturing perspective to a healthcare perspective. This hindered sense-making processes. These contextual issues and subsequent mechanisms influenced the degree of support for Lean continuation. Quotes to support this CMOc are presented in Additional file 3.
Rapidly evolving healthcare contexts overtime – ‘innovation fatigue’
The constant changes occurring in the work environment led to feelings of confusion and uncertainty about what changes were as a result of Lean implementation or something else, such as the changes occurring in relation to the new children’s hospital development within this context. The degree of constant change also triggered feelings that Lean would not “stick”, and that it was another “make-work” project. Unit managers expressed that staff were overwhelmed and staff engagement in Lean was a challenge. These challenges were also coupled with a lack of follow up regarding the Lean changes implemented. These contextual issues and mechanisms produced negative perceptions and an unawareness of what changes were due to Lean efforts. Quotes to support this CMO configuration are presented in Additional file 3.
Process of lean customization to context- positive and negative effects
In 2014, Lean implementation by the consultancy company ended and the continuation of Lean was placed on internal senior leaders and unit managers. This led to the process of Lean customization to local context. This process involved removal of the Lean Japanese language and a less stringent implementation of Lean activities and principles. There was a shift to better involve and engage frontline staff to lead Lean changes. Unit managers recognized staff involvement as an important factor for the normalization of Lean in everyday practice. This customization process was viewed as a positive transition by unit managers. It removed Lean elements that did not resonate with staff. Unit managers believed that this would improve staff involvement, engagement and buy-in. Although it was recognized that the word Lean had negative connotations for frontline staff buy-in, the customization of Lean to the local context did not make a difference to how Lean was perceived and supported by frontline staff. In contrast, customization led some participants to believe that Lean was no longer used or embedded in practice. There was a clear discrepancy between stakeholder groups across various levels of the system, in how much they felt Lean had become embedded in their everyday work and the degree to which they supported the continuation of Lean efforts. Quotes to support this CMOc are presented in Additional file 3.
Shared values and sense-making processes for normalization
The continuation of Lean efforts and the normalization of Lean in every day practice relied on how staff ‘made sense’ of Lean and whether the values of Lean aligned with their own personal and/or professional values. These were core mechanisms to the sustainability of Lean that were important from early stages of Lean implementation. Lean values of efficiency, patient safety, and waste reduction were congruent with participants’ professional values as healthcare providers. However, Lean training failed to translate how the principles of Lean aligned with the context of healthcare. Sense-making by frontline staff was hindered by a) the implementation approach used (top-down, use of an external consultancy firm), b) the Lean training received by the consultancy company and c) the type of Lean messaging by media and the consultancy company. It is important to note that this was not the case for the unit managers, who supported the continuation of Lean. Quotes to support this CMO configuration are presented in Additional file 3.
The NHS SM served as an appropriate middle-range theory to identify and explain contextual factors that influence the likelihood of Lean sustainability. Factors such as staff involvement and training to sustain the process, staff attitudes towards sustaining the change, credibility and adaptability. NPT served as an appropriate middle-range theory to identify and explain mechanisms of change and provided an explanatory model of the normalization of Lean in everyday practice. As identified processes of individual and communal sense-making (coherence), degree of cognitive participation and collective action influence the degree to which Lean efforts are embedded. A realist stance helped to address the complexity of translating Lean to healthcare or provide explanations of what works, for whom, in what respects, to what extent, in what contexts, and how?
The ‘ripple-effect’ is based on the idea that a program (Lean) is a series of “events in the history of a system, leading to the evolution of new structures of interaction and new shared meanings (, p., 267). The ‘ripple-effect’ in our research shows the causal relationship between Lean implementation and sustainability, and how implementation processes and outcomes shape sustainability. Our realist interviews primarily illustrate how implementation outcomes (e.g., resistance, lack of customization to context and negative perceptions), nature of implementation (e.g., training that did not connect the meaning of Lean to healthcare, external Lean consultants that were not from healthcare), and the implementation approach (e.g., mandated top-down approach) shape the contexts (resistance, lack of customization and negative perceptions and variation in Lean training and exposure); mechanisms (e.g., degree of sense-making, staff engagement, awareness); and outcomes (e.g., degree of support, continuation and normalization) for the sustainability of Lean efforts.
Our findings also highlight incongruence between leadership (i.e., senior leaders and unit managers) versus frontline healthcare providers in relation to the degree of normalization and continued support of Lean. Similar to recent research findings by Goodridge et al. , our research revealed that major gaps remain in the normalization and sustainment of Lean efforts into everyday practice, particularly among frontline staff. For the purposes of this discussion, we would like to focus on four key points that have influenced the normalization process of Lean in our research findings:
The use, approach and effect of an external consultancy company to lead early-stages of implementation.
The importance of customization to context.
The importance of shared values, sense- making and engagement for normalization.
The interface of Lean along the hierarchical structures of healthcare and the resulting incongruence between leadership and frontline staff.
The use, approach and effect of an external consultancy to lead early-stages of implementation
In our research, the use of an external consultancy company to lead implementation was primarily perceived negatively, as an outsider that did not understand healthcare. Concern about the cost of the consultancy company was also raised. An average of over $19 million Canadian dollars (CAD) in consulting fees were paid for a 2-year term  with an average cost of over $46 million CAD for Lean implementation in Saskatchewan between 2012 and 2014 . There is variation about whether top-down large-scale transformations or bottom-up, small-scale incremental improvements are more effective [7, 48,49,50]. Braithwaite  argues that complex systems, such as healthcare, will not change because one mandates a solution. Instead, complex systems adapt overtime to suit their own norms, values, practices and contexts . Our research confirms an adaptation over time but begs the question: how much adaption is acceptable in order to determine if Lean efforts are sustained?
Training and messaging by the consultancy company, as an implementation approach, had negative effects for some participants. The early-stages of implementation focused on senior leadership capacity building, through Lean leadership training. The focus on senior leadership resulted in an unintended negative consequence, that frontline staff did not feel involved and instead felt pressured to adopt Lean. Yet staff engagement is critical to the success of adoption [7, 48]. A recent study on the implementation process of Lean in Saskatchewan  found that those with Lean leadership training, were more likely to see potential in the value of Lean and support the use of Lean for their work.
The nature and type of Lean training and participation in Lean activities has implications for the extent of normalization. Though training and resources are important to any implementation of organizational change , simply receiving training is not sufficient. Our findings show that the nature and approach of the training and resources used are critical to change. Training needs to involve and engage participants and closely emulate the local environment . The most efficacious training is tailored to context, the target audience and based on evidence and feedback [52,53,54,55]. Our findings highlighted that initial training failed to demonstrate a connection between Lean and healthcare which triggered negative perceptions and resistance to Lean. This shows that perhaps it is not the mode of delivery that needs consideration, but the messaging used during training.
In contrast to the above findings, senior leadership noted that without the use of a consultancy company and a mandated top-down implementation approach, changes may not have occurred or occurred at a much slower pace. Contrary to our findings, Fine  suggested that Lean engages frontline staff, in the sense that staff develop and make the changes. This poses the question of whether a top-down implementation approach and use of a consultancy company contributed to the lack of staff engagement in our research context. As discussed by Braithwaite  people resist change that is imposed by others and that mandated change is never given the same weight as clinically driven change.
The importance of customization to context
Similar to our realist review findings , the degree to which mechanisms occurred was influenced by external pressures to use Lean , the complexity of care processes , the fit between Lean and local context [7, 57]; and other competing needs or demands , such as the constant change in healthcare environments. Early stages of implementation led by the consultancy company failed to customize Lean to local contexts, this triggered some pitfalls to the normalization of Lean in practice (e.g., feelings of disconnect, negative perceptions, resistance to Lean and a lack of support for Lean).
The constant change and “innovation fatigue” experienced by participants was one critical contextual factor. Similar to other findings, this can result in Lean being considered another “fashion fad” or “flavor of the month,”  that can lead to potential negative effects on adoption. Complexity was also raised as an important contextual factor, which can affect adoption and normalization [51, 55]. A failure to understand how and why the complexity of context influences the process of normalization will impact the use and sustainability of Lean in healthcare [59,60,61,62]. Our findings supplement the existing argument that it cannot be assumed that the translation of Lean from manufacturing to healthcare without consideration of context will offer the same benefits as achieved by Toyota [15, 57].
It is well supported that context is critical to the degree of success in the implementation of large-scale interventions [63,64,65,66]. Contextual factors can have a direct effect on the uptake and outcomes of interventions [64, 65]. Complex interventions that struggle to integrate into existing contexts are unlikely to be normalized . It is also important to note differences in terms of macro level (system) contexts. Examples of successful Lean implementations in health systems across America (e.g., Virginia Mason, Seattle Children’s Hospital) may prove different in the context of Canadian healthcare where funding models, insurance models, and governance are different.
Waring and Bishop  suggest that Lean is likely to be adapted to ensure it fits with the contexts for clinical practice. The process of customization to existing contexts may facilitate the normalization of interventions , such as Lean. In the context of our research study, when the consultancy company contract ended this led to a process of customization to the context. However, despite this shift from overt Lean implementation to implicit implementation, there was still variation to the degree to which people supported the continuation of Lean. There was clear discrepancy between leadership and frontline staff perceptions on how much they supported the continuation of Lean. This poses questions around the process and timing of customization to context, the degree of influence of early-stages of implementation on sustainability and the influence of organizational hierarchical structures on sustainability.
The importance of shared values, sense- making and engagement for normalization
In addition to a receptive context, Greenhalgh et al.  argued there also needs to be a good fit between the program being implemented and the needs and values of the potential adopters. The degree that staff values an intervention or program from early-stages of implementation is associated with the degree of effective adoption [69,70,71]. In our findings, the nature of the Lean training, poor knowledge translation strategies (e.g., education, training, audit and feedback) and external Lean consultants hindered frontline staff engagement and sense-making. Our realist review found that the more people value the change being implemented the more likely they will engage in the implementation efforts . However, our realist interviews showed that despite shared values with Lean (e.g., patient safety, efficiency, waste reduction), normalization did not occur due to failed sense-making processes from early-stages of implementation. These issues make Lean implementation a highly contested process [18, 73].
To facilitate normalization, it is necessary to appeal to the values and reasoning of potential adopters . Fine et al.  argue that those who truly make sense of Lean will see its value for their work and subsequently begin to apply it. This study describes the idea of the “tipping point” where leaders no longer had to “push” Lean ideas out to staff. Instead, staff “pull” Lean and demand it for themselves (, p34). It appears this was the intention in Saskatchewan, when there was the shift in the implementation approach. However, reflecting on our findings, it seems the “tipping-point” has not come to fruition yet. Sense-making about Lean may occur during early stages of implementation but is equally as important to maintain for the normalization and sustainability of Lean efforts. Another pitfall in our findings that affected sense-making processes was the ways in which Lean was messaged, the lack of “stickiness” to the Lean messaging used, in other words the lack of natural appeal for frontline staff [74, 75]. The concept of “stickiness” is required for effective messaging and uptake.
Our findings also demonstrated that frontline staff engagement was hindered by poor messaging, lack of sense-making processes and the implementation approach used. Engagement of nurses has already been found to be an issue with regards to Lean implementation in Saskatchewan. In 2014, a survey conducted by the Saskatchewan Union of Nurses  found a statistically significant negative effect of Lean on nurse engagement. Physician involvement is also widely addressed as a critical factor to implementation and QI success [76, 77]. Our study had limited physician participation, the reasoning for poor participation is unknown yet mirrors previous work on Lean implementation in Saskatchewan . Future research that solely focuses on physician perspectives on the Saskatchewan Lean management system would be valuable.
Misunderstandings of Lean also creates staff disengagement . Misunderstandings may be triggered from the overuse of ‘Japanese’ Lean language that does not resonate with all health professionals. Several studies have reported that the conceptualization of Lean in healthcare is unclear and varied [1, 79,80,81] and may be conceptually challenging for staff [16, 65]. Another issue is the blending of several QI methodologies with Lean, without clear definitions. This makes it difficult to differentiate Lean from other approaches and thus it is hard to evaluate what successes or failures are attributed to Lean or not. There needs to be more consistent and standardized conceptualizations of Lean and clearer differentiations between QI approaches in order to distinguish Lean from other QI approaches. This duty should be a collaborative role of research and leaders in healthcare improvement.
The interface of lean along the hierarchical structures of healthcare
There were ample differences in interview responses between leadership and frontline staff. Frontline staff portrayed more negative perceptions of Lean in comparison to their unit managers or senior leaders. Similarly, a recent survey on Lean implementation processes in Saskatchewan using NPT  found that respondents in leadership positions were much more likely to view Lean implementation and outcomes positively. The results of this survey also found wide variation between the perspectives of leaders and frontline staff regarding the NPT constructs of coherence, cognitive participation and reflexive monitoring. This survey illustrated issues around staff familiarity with Lean principles and activities and perspectives that Lean is not currently a part of their work. It appears that part of these issues are a result of the silo and hierarchical nature of healthcare [82, 83].
This idea of the interface of Lean along the hierarchical structures of healthcare and the impact of professional role status along that hierarchy on the success of Lean implementation requires further exploration. These structures and roles in healthcare may impede the ability to achieve alignment from senior leadership to frontline staff . Alignment is the consistency of plans, visions, resources, actions and results to support system-wide goals . Clear accountability structures and integration are needed for system-wide alignment. Previous work has recognized the hierarchical nature of healthcare and professional silos as a barrier to Lean success [84, 85]. However, it remains unclear how to achieve such changes in highly entrenched hierarchical systems.
Our research demonstrates a ‘ripple-effect’, that is a causal link between implementation and sustainability. Sustainability is hinged on the degree of success at early-stages of implementation. We identified sense-making and engagement as critical mechanisms to sustainability. Sense-making is facilitated or hindered by certain messaging, training and language used during initial stages of implementation. The degree of sense-making and engagement by staff at early-stages of implementation had a ripple-effect on sustainability. The interface of Lean with the hierarchical structures and professional silos of healthcare also play a role to the degree of normalization of Lean. The traditional hierarchical structures and silos in healthcare may impede the ability to achieve alignment from senior leadership to frontline staff and thus hinder the likelihood of embedding Lean in everyday practice. The customization of Lean to context was also critical to the degree of sustainability. Context is known to have a direct effect on the uptake and outcomes of interventions. However there remains knowledge gaps and questions about the timing of and approach to customization and requires further exploration. This research provides practical guiding principles that healthcare leaders may incorporate into planned Lean implementation.
Our research also identified challenges to evaluating sustainability of complex interventions. There is variation in the literature on the conceptualization of sustainability, measurements and outcomes of sustainability. We recognize like others that there is a need for the development and pilot testing of theoretical frameworks and tools to evaluate the sustainability of complex interventions in healthcare. Without such guidance, it is difficult to develop a science on the sustainability of QI efforts and complex interventions in healthcare. Such developments need to make sense and be applicable to those people using them in health systems. Further work using other methods is needed to examine and further test the mechanisms identified in our realist evaluation in other contexts for theory development and to identify predictors of sustainability.
Availability of data and materials
The qualitative data supporting this study is not available as participants did not consent to having their data publicly available. As a result, we are not authorized to share the dataset.
Context + Mechanism = Outcome configuration
Kaizen Promotion Offices
- NHS SM:
National Health Services Sustainability Model
Normalization Process Theory
Rotter T, Plishka C, Lawal A, et al. What is Lean management in health care? Development of an operational definition for a Cochrane systematic review. Eval Health Prof. 2018:1–25. https://doi.org/10.1177/0163278718756992 PMID:29635950.
Womack JP, Jones DT, Roos D. The machine that changed the world: the story of lean production. New York: Harper Collins; 1990.
Womack JP, Jones DT. Lean thinking: banish the waste and create wealth in your corporation. London: Simon and Schulster; 2003.
Mann D. Creating a lean culture: tools to sustain lean conversions. 2nd ed. New York: Productivity Press; 2010. https://doi.org/10.1201/EBK1439811412.
Goodridge D, Westhorp G, Rotter T, Dobson R, Bath B. Lean and leadership practices: development of an initial realist program theory. BMC Health Serv Res. 2015;15(1):362–77. https://doi.org/10.1186/s12913-015-1030-x.
Mazzocato P, Savage C, Brommels M, Aronsson H, Thor J. Lean thinking in healthcare: a realist review of the literature. Qual Saf Health Care. 2010;19(5):376–82 PMID:20724397.
Mazzocato P, Thor J, Bäckman U, et al. Complexity complicates lean: lessons from seven emergency services. J Health Organ Manag. 2014;28(2):266–88. https://doi.org/10.1108/JHOM-03-2013-0060.
Holden RJ. Lean thinking in emergency departments: a critical review. Ann Emerg Med. 2011;57(3):265–78. https://doi.org/10.1016/j.annemergmed.2010.08.001.
Grove AL, Meredith JO, Macintyre M, Angelis J, Neailey K. Lean implementation in primary care health visiting services in National Health Service UK. Qual Saf Health Care. 2010;19(5):e43 PMID:20511600.
McCann L, Hassard J, Granter E, Hyde P. Casting the lean spell: the promotion, dilution and erosion of lean management in the NHS. Hum Relat. 2015;68(10):1557–77. https://doi.org/10.1177/0018726714561697.
Bhasin S, Burcher P. Lean viewed as a philosophy. J Manuf Tech Manag. 2006;17(1):56–72. https://doi.org/10.1108/17410380610639506.
Burgess N, Radnor Z. Evaluating lean in healthcare. Int J Health Care Qual Assur. 2013;26(3):220–35. https://doi.org/10.1108/09526861311311418.
Smith G, Poteat-Godwin A, Harrison LM, Randolph GD. Applying lean principles and kaizen rapid improvement events in public health practice. J Public Health Manag Pract. 2012;18(1):52–4. https://doi.org/10.1097/PHH.0b013e31823f57c0.
Young TP, McClean SI. A critical look at lean thinking in healthcare. Qual Saf Health Care. 2008;17(5):382–6. https://doi.org/10.1136/qshc.2006.020131.
Radnor Z, Osborne SP. Lean: a failed theory for public services. Public Manage Rev. 2013;15(2):265–87. https://doi.org/10.1080/14719037.2012.748820.
Andersen H, Røvik KA, Ingebrigtsen T. Lean thinking in hospitals: is there a cure for the absence of evidence? A systematic review of reviews. BMJ Open. 2014;4(1):e003873. https://doi.org/10.1136/bmjopen-2013-003873.
DelliFraine JL, Langabeer JR II, Nembhard IM. Assessing the evidence of six sigma and lean in the health care industry. Qual Manag Health Care. 2010;19(3):211–25. https://doi.org/10.1097/QMH.0b013e3181eb140e.
Waring JJ, Bishop S. Lean healthcare: rhetoric, ritual and resistance. Soc Sci Med. 2010;71(7):1332–40. https://doi.org/10.1016/j.socscimed.2010.06.028.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):117–28. https://doi.org/10.1186/1748-5908-8-117.
Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7(1):17–36. https://doi.org/10.1186/1748-5908-7-17.
Proctor E, Luke D, Calhoun A, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88–101. https://doi.org/10.1186/s13012-015-0274-5.
Flynn R, Newton AS, Rotter T, Hartfield D, Walton S, Fiander M, Scott SD. The sustainability of Lean in pediatric healthcare: a realist review. BMC Syst Rev. 2018;7:137. https://doi.org/10.1186/s13643-018-0800-z.
Maher L, Gustafson D, Evans A. NHS Sustainability Model. NHS Institute for Innovation and Improvement; 2010. http://webarchive.nationalarchives.gov.uk/20160805122935/http://www.nhsiq.nhs.uk/media/2757778/nhs_sustainability_model_-_february_2010_1_.pdf. Accessed 31 May 2018.
Pawson R, Tilley N. Realistic evaluation. Thousand Oaks: SAGE Publications Ltd; 1997.
Yin RK. Case study research: design and methods. 3rd ed. Thousand Oaks: Sage Publications; 2003.
Mackenzie J, Hall W. “Lean” in Canadian health care: doing less while achieving more, The Conference Board of Canada. Ottawa: Publication 6262; 2014. Accessed 18 Jan 2018
Kinsman L, Rotter T, Stevenson K, et al. “The largest lean transformation in the world”: the implementation and evaluation of lean in Saskatchewan healthcare. Healthc Q. 2014;17(2):29–32. https://doi.org/10.12927/hcq.2014.23880.
Government of Saskatchewan. Ministry of Health: Plan for 2013–14; 2013. http://publications.gov.sk.ca/documents/13/81400-ministry_of_health_ar_2013-2014-web.pdf. Accessed 3 May 2018.
Sari N, Rotter T, Goodridge D, Harrison L, Kinsman L. An economic analysis of a system wide lean approach: cost estimations for the implementation of lean in the Saskatchewan healthcare system for 2012-2014. BMC Health Serv Res. 2017;17(1):523. https://doi.org/10.1186/s12913-017-2477-8.
Saskatchewan-Health-Quality-Council. Lean Reform: Saskatchewan Healthcare Adopts Lean Management for Big Benefits; 2014. http://blog.hqc.sk.ca/2014/2008/2028/lean-reform-saskatchewan-healthcare-adopts-lean-management-for-big-benefits/. Accessed 3 May 2018.
Rotter T, Kinsman L, Bath B, Goodridge D, Harrison L, Dobson R. et al. A first phase evaluation of Saskatchewan’s lean health care transformation: final report; 2014. http://research-groups.usask.ca/rotter/documents/select-publications/Lean%20Report%20-%20Full.pdf. Accessed 6 Nov 2017.
Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016;22(3):342–60. https://doi.org/10.1177/1356389016638615.
Salter KL, Kothari A. Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review. Implement Sci. 2014;9(1):115–40. https://doi.org/10.1186/s13012-014-0115-y.
Pawson R, Manzano-Santaella A. A realist diagnostic workshop. Evaluation. 2012;18(2):176–91. https://doi.org/10.1177/1356389012440912.
Punton M, Vogel I, Lloyd R. Reflections from a realist evaluation in progress: scaling ladders and stitching theory. CDI Pract Pap. 2016;18:18–29.
Jagosh J, Bush PL, Salsberg J, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health. 2015;15(1):725–36. https://doi.org/10.1186/s12889-015-1949-1.
Dopson S, Fitzgerald LA. The active role of context. In: Dopson S, Fitzgerald LA, editors. Knowledge to action? Evidence-based health care in context. Oxford: Oxford University Press; 2005. p. 79–103. https://doi.org/10.1093/acprof:oso/9780199259014.003.0005.
Astbury B, Leeuw F. Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. 2010;31(3):363–81. https://doi.org/10.1177/1098214010371972.
Doyle C, Howe C, Woodcock T, et al. Making change last: applying the NHS institute for innovation and improvement sustainability model to healthcare improvement. Implement Sci. 2013;8(1):127–37. https://doi.org/10.1186/1748-5908-8-127.
May CR, Mair F, Finch T, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4(1):29. https://doi.org/10.1186/1748-5908-4-29.
May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology. 2009;43(3):535–54. https://doi.org/10.1177/0038038509103208.
Maxwell JA. A realist approach for qualitative research. Thousand Oaks: Sage Publications; 2012.
Porter S. Validity, trustworthiness and rigour: reasserting realism in qualitative research. J Adv Nurs. 2007;60(1):79–86. https://doi.org/10.1111/j.1365-2648.2007.04360.x.
Pawson R, Boaz A, Grayson L, Long A, Barnes C. Types and quality of knowledge in social care. London: Social Care Institute for Excellence; 2003.
Wong G, Westhorp G, Manzano A, Greenhalgh J, Jagosh J, Greenhalgh T. RAMESES II reporting standards for realist evaluations. BMC Med. 2016;14:1–18. https://doi.org/10.1186/s12916-016-0643-1.
Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43(3–4):267–76. https://doi.org/10.1007/s10464-009-9229-9.
Goodridge D, Rana M, Harrison EL, et al. Assessing the implementation processes of a large-scale, multi-year quality improvement initiative: survey of health care providers. BMC Health Serv Res. 2018;18(1):237–59. https://doi.org/10.1186/s12913-018-3045-6.
Wong AM, During D, Hartman M, Lappan-Gracon S, Hicks M, Bajwa S. Lean transformation of the eye clinic at the hospital for sick children: challenging an implicit mental model and lessons learned. Healthc Q. 2016;19(1):36–41. https://doi.org/10.12927/hcq.2016.24611.
Hung DY. Spreading lean: taking efficiency interventions in health services delivery to scale. In: Agency for Healthcare Research and Quality; 2016. https://www.ahrq.gov/sites/default/files/publications/files/execsumm-lean-redesign.pdf. Accessed 6 Nov 2017.
Braithwaite J. Changing how we think about healthcare improvement. BMJ. 2018;361:k2014. https://doi.org/10.1136/bmj.k2014.
Braithwaite J, Churruca K, Long JC, Ellis LA, Herkes J. When complexity science meets implementation science: a theoretical and empirical analysis of systems change. BMC Med. 2018;16(1):63–77. https://doi.org/10.1186/s12916-018-1057-z.
Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc. 2013;20(e1):e9–e13. https://doi.org/10.1136/amiajnl-2013-001684.
Paré G, Sicotte C, Jaana M, Girouard D. Prioritizing the risk factors influencing the success of clinical information system projects. A Delphi study in Canada. Methods Inf Med. 2008;47(3):251–9 PMID:18473092.
Ash JS, Stavri PZ, Kuperman GJ. A consensus statement on considerations for a successful CPOE implementation. J Am Med Inform Assoc. 2003;10(3):229–34. https://doi.org/10.1197/jamia.M1204.
Cummings A, Lund S, Campling N, May CR, Richardson A, Myall M. Implementing communication and decision-making interventions directed at goals of care: a theory-led scoping review. BMJ Open. 2017;7(10):e017056. https://doi.org/10.1136/bmjopen-2017-017056.
Fine BA, Golden B, Hannam R, Morra D. Leading lean: a Canadian healthcare leader’s guide. Healthc Q. 2009;12(3):32–41. https://doi.org/10.12927/hcq.2013.20877.
Mazzocato P, Holden RJ, Brommels M, et al. How does lean work in emergency care? A case study of a lean-inspired intervention at the Astrid Lindgren Children’s hospital, Stockholm, Sweden. BMC Health Serv Res. 2012;12(1):28–41. https://doi.org/10.1186/1472-6963-12-28.
Northway T, Krahn G, Thibault K, et al. Surgical suite to pediatric intensive care unit handover protocol: implementation process and long-term sustainability. J Nurs Care Qual. 2015;30(2):113–20. https://doi.org/10.1097/NCQ.0000000000000093 PMID:25426649.
Øvretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. 2011;20(suppl 1):i18–23. https://doi.org/10.1136/bmjqs.2010.045955.
Robinson S, Radnor ZJ, Burgess N, Worthington C. SimLean: utilising simulation in the implementation of lean in healthcare. Eur J Oper Res. 2012;219(1):188–97. https://doi.org/10.1016/j.ejor.2011.12.029.
Plsek PE, Greenhalgh T. Complexity science: the challenge of complexity in health care. BMJ. 2001;323(7313):625–8. https://doi.org/10.1136/bmj.323.7313.625.
May C, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11:1–12. https://doi.org/10.1186/s13012-016-0506-3.
Radnor ZJ, Holweg M, Waring J. Lean in healthcare: the unfilled promise? Soc Sci Med. 2012;74(3):364–71. https://doi.org/10.1016/j.socscimed.2011.02.011.
McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Titchen A, Seers K. Getting evidence into practice: the meaning of ‘context’. J Adv Nurs. 2002;38(1):94–104. https://doi.org/10.1046/j.1365-2648.2002.02150.x.
Curran JA, Grimshaw JM, Hayden JA, Campbell B. Knowledge translation research: the science of moving research into policy and practice. J Contin Educ Heal Prof. 2011;31(3):174–80. https://doi.org/10.1002/chp.20124.
Walshe K, Freeman T. Effectiveness of quality improvement: learning from evaluations. Qual Saf Health Care. 2002;11(1):85–7. https://doi.org/10.1136/qhc.11.1.85.
Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11(1):40. https://doi.org/10.1186/s13012-016-0396-4.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. https://doi.org/10.1111/j.0887-378X.2004.00325.x.
Fritz Z, Fuld JP. Development of the universal form of treatment options (UFTO) as an alternative to do not attempt cardiopulmonary resuscitation (DNACPR) orders: a cross-disciplinary approach. J Eval Clin Pract. 2015;21(1):109–17. https://doi.org/10.1111/jep.12256.
Brimblecombe C, Crosbie D, Lim WK, Hayes B. The goals of patient care project: implementing a proactive approach to patient-centred decision-making. Intern Med J. 2014;44(10):961–6. https://doi.org/10.1111/imj.12511.
Weiner BJ. A theory of organizational readiness for change. Implement Sci. 2009;4(1):67–76. https://doi.org/10.1186/1748-5908-4-67.
Kim CS, Spahlinger DA, Kin JM, Billi JE. Lean health care: what can hospitals learn from a world-class automaker? J Hosp Med. 2006;1(3):191–9. https://doi.org/10.1002/jhm.68.
Braithwaite J, Runciman WB, Merry AF. Towards safer, better healthcare: harnessing the natural properties of complex sociotechnical systems. Qual Saf Health Care. 2009;18(1):37–41. https://doi.org/10.1136/qshc.2007.023317.
Zander U, Kogut B. Knowledge and the speed of the transfer and imitation of organizational capabilities: an empirical test. Organ Sci. 1995;6(1):76–92. https://doi.org/10.1287/orsc.6.1.76.
Saskatchewan Union of Nurses. Lean: A Safe and Effective Tool for Registered Nursing? 2014. http://sun-nurses.sk.ca/+pub/document/Communications/SUNSpots/2014/SUNSpots%2040-3%20w%20Feature%20(July%2015-14).pdf. Accessed 3 May 2018.
Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress. Milbank Q. 1998;76(4):593–624. https://doi.org/10.1111/1468-0009.00107.
Atkinson P. Creating and implementing lean strategies. Manag Serv. 2004;48(2):18–33.
Van Vliet EJ, Bredenhoff E, Sermeus W, Kop LM, Sol JC, Van Harten WH. Exploring the relation between process design and efficiency in high-volume cataract pathways from a lean thinking perspective. Int J Qual Health Care. 2011;23(1):83–93. https://doi.org/10.1093/intqhc/mzq071.
Radnor Z. Implementing lean in health care: making the link between the approach, readiness and sustainability. Int J Ind Eng Manag. 2011;2(1):1–12.
Maijala R, Eloranta S, Reunanen T, Ikonen TS. Successful implementation of lean as a managerial principle in health care: a conceptual analysis from systematic literature review. Int J Technol Assess Health Care. 2018;34(2):134–46. https://doi.org/10.1017/S0266462318000193 PMID:29642955.
Radnor Z, Walley P, Stephens A, Bucci G. Evaluation of the Lean Approach to Business Management and its Use in the Public Sector (full report). Edinburgh: Office of Chief Researcher, Scottish Executive; 2006. http://www.gov.scot/Resource/Doc/129627/0030899.pdf . Accessed 3 May 2018.
Brandao de Souza L, Pidd M. Exploring the barriers to lean health care implementation. Public Money Manag. 2011;31(1):59–66. https://doi.org/10.1080/09540962.2011.545548.
Lukas CV, Holmes SK, Cohen AB, et al. Transformational change in health care systems: an organizational model. Health Care Manag Rev. 2007;32(4):309–20. https://doi.org/10.1097/01.HMR.0000296785.29718.5d.
Ben-Tovim DI, Bassham JE, Bolch D, Martin MA, Dougherty M, Szwarcbord M. Lean thinking across a hospital: redesigning care at the flinders medical Centre. Aust Health Rev. 2007;31(1):10–5. https://doi.org/10.1071/AH070010.
Brandao de Souza L. Trends and approaches in lean healthcare. Leadersh Health Serv. 2009;22(2):121–39. https://doi.org/10.1108/17511870910953788.
The authors would like to thank the participants that took part in this study.
Alberta Innovates, the Women & Children’s Health Research Institute provided salary support for RF’s PhD research. Funding from Alberta Innovates supported the costs incurred for data collection, analysis, and interpretation of data. SDS holds a Canada Research Chair (Tier II) for knowledge translation in child health. ASN is supported by the Canadian Institutes of Health Research (CIHR) as a CIHR New Investigator.
Ethics approval and consent to participate
Ethical Approval for this study was granted by the University of Alberta Health Research Ethics Board and the University of Saskatchewan Behavioural Research Ethics Board. Institutional approval was provided by the Saskatoon Health Region, Inter-professional Practice, Education and Research office. Written informed consent was required and obtained from all participants in this study.
Consent for publication
Informed consent was obtained from participants, for the publication of quotes in this manuscript.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Flynn, R., Rotter, T., Hartfield, D. et al. A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in pediatric healthcare. BMC Health Serv Res 19, 912 (2019). https://doi.org/10.1186/s12913-019-4744-3
- Quality improvement
- Realist evaluation