Influences of post-implementation factors on the sustainability, sustainment, and intra-organizational spread of complex interventions
BMC Health Services Research volume 22, Article number: 666 (2022)
Complex interventions are increasingly applied to healthcare problems. Understanding of post-implementation sustainment, sustainability, and spread of interventions is limited. We examine these phenomena for a complex quality improvement initiative led by care aides in 7 care homes (long-term care homes) in Manitoba, Canada. We report on factors influencing these phenomena two years after implementation.
Data were collected in 2019 via small group interviews with unit- and care home-level managers (n = 11) from 6 of the 7 homes using the intervention. Interview participants discussed post-implementation factors that influenced continuing or abandoning core intervention elements (processes, behaviors) and key intervention benefits (outcomes, impact). Interviews were audio-recorded, transcribed verbatim, and analyzed with thematic analysis.
Sustainment of core elements and sustainability of key benefits were observed in 5 of the 6 participating care homes. Intra-unit intervention spread occurred in 3 of 6 homes. Factors influencing sustainment, sustainability, and spread related to intervention teams, unit and care home, and the long-term care system.
Our findings contribute understanding on the importance of micro-, meso-, and macro-level factors to sustainability of key benefits and sustainment of some core processes. Inter-unit spread relates exclusively to meso-level factors of observability and practice change institutionalization. Interventions should be developed with post-implementation sustainability in mind and measures taken to protect against influences such as workforce instability and competing internal and external demands. Design should anticipate need to adapt interventions to strengthen post-implementation traction.
Healthcare systems are increasingly conceptualized as complex adaptive systems that warrant complex solutions to improve care processes and practices . Solutions are themselves frequently complex evidence-based interventions to introduce and support practice changes, requiring considerable effort and resources to implement. Researchers underscore the importance and challenges of sustaining practices, benefits, and outcomes introduced through complex interventions . Most changes introduced through interventions subside over time [2, 3] – even highly implementable changes like prescribing practices after cardiac events .
Intervention sustainability is among “the most significant translational research problems of our time” , but sustainability-associated phenomena are persistently under-researched . Implications of failing to sustain processes and outcomes of practice change initiatives are numerous. Failure (1) wastes financial and human resources ; (2) makes improvements short-lived, detracting from care quality and increasing variation ; (3) leads to inequities in health services access and quality [6, 7]; (4) diminishes probability of spreading beneficial solutions; and (5) reduces morale and belief in improvement initiatives .
In understanding how to implement to sustain, most work addresses influences of peri-implementation factors (salient during the implementation phase) on sustainability. These factors include supports from implementers and features of pre-implementation context [2, 9,10,11]. Our premise is that influences of post-implementation factors may be just as, or more, important to longer-term sustainability – particularly with well-established challenges to sustaining behavior change in notoriously intractable social settings such as healthcare . We argue that post-implementation factors are likely to prevail and gain salience after implementation supports are removed. While some post-implementation factors may not directly affect continued application of new practices, others may have profound influence. Factors may support new processes and practices, erode changes during implementation, or reinforce reversion to old ways of doing . Withdrawal or redirection of management support, or loss of staff with knowledge of a practice change, may lead to knowledge loss and organizational forgetting.
Our study was further motivated by two other major literature gaps. First, the related phenomena of sustainability, sustainment, and spread (defined below) have been studied separately, or sustainment and sustainability are conflated. Little is known about whether factors that influence one phenomenon influence others. Second, how sustainability, sustainment, and spread are related is understudied and important to advancing sustainability science [5, 13].
Defining sustainment, sustainability, and spread
The terms sustainment, sustainability, and spread lack universal definitions . In the implementation science literature, sustainment and sustainability are considered post-implementation phenomena of concern once (peri)implementation supports are withdrawn or exhausted. Common supports are research funding for coaching and facilitation, access to clinical and other expertise, training, and financial incentives. Chambers et al. (2013)  usefully distinguish sustainment and sustainability as related but distinct aspects of continued knowledge use. Sustainment occurs when stakeholders continue to enact core processes and related behaviors originally conveyed through an intervention. Sustainability is continuation of key benefits or outcomes from the intervention. We refer to spread as uptake or diffusion of an intervention (or core components) from its original context to intra- or inter-organizational contexts . While spread is often considered conceptually separate from sustainment and sustainability, we contend that it is a logically linked related phenomenon. If change processes are not sustained, or benefits cease to be realized, a change initiative is unlikely to spread . We refer to sustainment, sustainability, and spread collectively as sustainability phenomena.
What we know about sustainability phenomena and complex interventions in healthcare
Stirman and colleagues’ (2012)  critical review of the literature on sustainability of evidence-based programs and practices in healthcare settings highlighted four categories of factors that influence continuation of procedures and outcomes: (1) organizational contexts for practice change interventions (e.g., culture, leadership, structure); (2) internal and external capacity (e.g., resources, support, champions); (3) processes (e.g., training, engagement); and (4) aspects of new programs or practices themselves (e.g., fit, modifiability). These factors are influential peri-implementation and influence sustainability phenomena post-implementation. Stirman et al. (2012)  noted lack of consideration of sustainability dynamics. Most studies they reviewed considered sustainability at one time point post-implementation, for static presence/absence/extent of peri-implementation factors. Stirman et al. (2012)  concluded that sustainability research needs careful critical development, including enhanced conceptual clarity on sustainability and concerted examination of dynamics of contexts where sustainability is a concern.
More recent findings align with the categories of Stirman et al. (2012) . Important peri-implementation factors that influence sustainability are intervention design and delivery, intervention process, people involved, resources, organizational setting, and external environment . Proctor and colleagues (2015)  highlighted the need for definitional and conceptual consistency within the sustainability literature. They sought perspectives of sustainability researchers and stakeholders to identify important issues for sustainability research. They highlighted two priority areas: testing how intervention properties and adaptations affect sustainability, and identifying contextual factors that influence sustainability. They noted the importance of distinguishing between influences of factors on implementation and post-implementation.
Lennox and colleagues (2020; 2018) [8, 13] reviewed approaches to planning, assessing, or evaluating continued intervention activities and benefits. Their consolidated summary of sustainability determinants in these approaches is highly consistent with Stirman et al. (2012) . They usefully identified recent empirical exploration of sustainability dynamism, noting a detectable “shift in [researchers’] perspectives from sustainability as an outcome to sustainability as an ongoing process” (, p. 12). About one third of identified approaches draw on theories that view sustainability as an ongoing process [8, 13]. Those relying on complexity theory emphasize interactions among intervention, organizational context, and broader context. Those relying on ecological theory propose ongoing intervention adaptation as important to maintaining fit with constantly changing inner and outer contexts. Those referring to open systems theory posit that broader environment influences organizational context of the intervention, requiring constant organizational adaptations to sustain the intervention. In all theoretical perspectives, the dynamic view of sustainability calls for continuous intervention improvement and adaptation and (where feasible) its inner and outer contexts . These perspectives depart markedly from traditional linear views that treat sustainability as a static outcome.
While emerging dynamic perspectives on sustainability are promising and much more likely to reflect the complex reality of sustaining practice change over time, none of this work adequately considers changing intervention contexts. Assuming that factors active during implementation are equally active and effective post-implementation is counter-intuitive (and possibly naïve). The literature is silent on specific influences of post-implementation factors on sustainability.
Sustainability phenomena of the SCOPE (Safer Care for Older People in Residential Environments) intervention
Our study is part of the Sustainment, Sustainability, and Spread Study (SSaSSy)  of a complex quality improvement (QI) initiative, SCOPE (ClinicalTrials.gov NCT03426072) . Supplementary Fig. 1 shows the different stages of SCOPE, data collection for the current study, and SSaSSy. Our focus is sustainability phenomena in CHs in the province of Manitoba, Canada, where SCOPE was implemented during 2016–2017. We report more details of the SCOPE implementation in Manitoba elsewhere .
In Canadian CHs, care aides (unregulated direct care workers) account for the largest workforce and provide up to 90% of direct care for older adults . SCOPE is a complex intervention to empower care aides to lead a QI initiative on their CH unit, with facilitated support. SCOPE has four components (Supplementary Fig. 2) [20, 22]. (1) Learning Congresses led by experienced quality advisors convey key QI skills and processes identified in the Institute for Health Improvement’s collaborative model (Institute for Healthcare Improvement, 2003) . Interim action periods of Plan-Do-Study-Act (PDSA) cycles provide experiential QI learning. Small PDSA change cycles, enacted by collaborative teams and emphasizing pre-post measurement, gauge effects of small-scale practice change/QI initiatives. (2) SCOPE introduces a team QI approach. Care aides lead teams of 3–4 CH staff. (3) SCOPE formalizes internal team support, with Team Sponsors (unit managers) and Senior Sponsors (CH managers) mentoring and supporting QI activities of care aides. (4) Quality advisors provide external team supports, with ongoing coaching and guidance to Sponsors and QI teams over PDSA cycles between Learning Congresses.
SCOPE’s key components convey its core elements (processes, behaviors) designed to deliver key benefits (Table 1). Core elements are essential to sustain, as “most closely associated with desired health benefits” (, p. 10). Each care aide-led QI team selected one of three clinical areas for their project focus: responsive behaviors, pain, or mobility. SCOPE’s intended key benefits (Table 1) are care aides’ QI capacity and leadership, use of best evidence in their practices, improved quality of work life, and enhanced work engagement. Research in work psychology  suggests that SCOPE’s influence on outcomes for work autonomy, empowerment, and engagement is likely to influence downstream work performance, including improved quality of resident care.
A SCOPE pilot was conducted in Alberta and British Columbia CHs (2010–2011) . We examined sustainability and spread of SCOPE-related QI activities in 7 participating homes at 6 and 12 months post-implementation . We did not differentiate peri- and post-implementation factors or sustainment and sustainability, but we identified factors influencing continued processes (sustainment) and sustained benefits (sustainability) up to 12 months post-implementation: turnover of key team members (care aides), facilitative team dynamics, observable improvements from SCOPE, strong management support of QI teams, buy-in from other CH staff, and (in)adequate resources . Factors contributed to absence of continued processes (sustainment), sustained benefits (sustainability), or spread: lack of resources (financial, human) in one CH, leadership change in another, and competing government priorities in the third. These findings on short-term sustainability suggested factors warranting further exploration in this study.
Our overarching goal is to understand post-implementation factors that influence dynamic longer-term sustainment, sustainability, and spread of complex care interventions. In this exploratory study, we aimed to explore post-implementation factors that influenced the sustainment, sustainability, and spread of the SCOPE intervention in the complex environments of Canadian care homes (CHs; long-term care homes). Our research question was: What were unit and care home managers’ perspectives of multi-level post-implementation factors that influenced the continuation of core SCOPE elements and key benefits/impacts? The SCOPE intervention has been evolving as this research team developed , piloted , and implemented  the intervention. Therefore, our research team has the unique opportunity to contribute to knowledge about the dynamism of sustainability phenomena.
We used secondary data that were originally collected in the preparation stage of SSaSSy to assess the post-implementation continuation of SCOPE-related QI activities in care homes and to inform the refinement of SSaSSy (Supplementary Fig. 1). We focused on post-implementation factors that unit and CH managers (Team and Senior Sponsors) identified as instrumental to continued or discontinued use of SCOPE’s core elements and benefits (Table 1).
We employed a qualitative, descriptive research design  to explore factors influencing SCOPE sustainability phenomena two years post-implementation. This design allowed us to stay close to the data with low levels of researcher interpretation or inference . We collected data through small group interviews by telephone (May to July 2019), two years after SCOPE’s implementation in Manitoba (2016–2017).
Setting and participants
SCOPE was implemented in 7 CHs in Winnipeg (Manitoba’s largest urban center). While multiple groups of staff (care aides, regulated staff, managers) participated in the SCOPE implementation, in this study we focused on managers’ perspectives of sustainability phenomena post-implementation. We invited all unit and CH managers to participate via email; 11 managers from 6 of 7 CHs agreed. We undertook 6 small group interviews (1 to 3 managers per interview), one for each CH participating.
Guide for small group interviews
We based our semi-structured interview guide (Supplementary File 1) on literature on QI program sustainability in healthcare settings [2, 8, 13, 17] and our prior work . We elicited participants’ descriptions of their experiences during SCOPE, continued QI activities post-implementation, post-implementation factors influencing sustainment (dis/continuation of behaviors) and sustainability (dis/continuation of outcomes) in the QI unit, and inter-unit spread.
Data collection and preparation
LM sought written informed consent from potential participants prior to each interview. LM and WB conducted telephone interviews of up to 45 min, each with manager participants from the same care home. Interviews were audio-recorded, professionally transcribed, and de-identified before analysis.
We used inductive thematic analysis to identify data patterns and themes [27, 28] and Microsoft Word to support the analysis. Table 2 details the analysis steps, including familiarizing with the data, generating, and applying coding scheme, generating themes, and refining themes. When developing and refining the coding scheme, we organized observed factors into three levels: (1) micro-level factors relating to individuals, teams, and their interactions; (2) meso-level factors relating to unit and CH; and (3) macro-level factors relating to CH operating environment or systems.
For qualitative rigor , multiple researchers worked closely in analysis and recorded major decisions and findings in extensive memos. A three-member team (YS, LM, WB) led the data analysis; this team included a health services research PhD student, a nursing postdoctoral fellow, and a health services researcher. Several other authors (LG, MD, AW, LC) contributed to the identification and differentiation of themes and to the refinement of them.
General observations on post-implementation sustainment, sustainability, and spread
Sustainment of core elements
Two years after SCOPE, Sponsors (managers) from 5 of 6 teams (CH1, CH3-6) reported continuation of several SCOPE-related QI activities (Supplementary Table 1) for core processes and behaviors. Only CH2 reported complete cessation. Sustained processes and behaviors included: QI processes like PDSA cycles to identify and trial change ideas (CH3); team-based approaches to QI (e.g., regular staff huddles [CH1]); QI ideas identified during SCOPE implementation (e.g., individualized mobility reminder cards [CH1], music to reduce responsive behaviors [CH4,5]); and QI tools to continually assess progress (e.g., safety crosses, aggressive behavior assessment tool [CH4]). Some units had well-preserved SCOPE team membership and core elements (Table 3 Quote 1).
Sustainability of key benefits
Sponsors reported sustained SCOPE benefits over the two years post-implementation (Supplementary Table 1). Improved quality of work life across sites CH1 and CH3–6 manifested as enduring care aide empowerment and increased self-efficacy. Sponsors remarked on care aides’ empowerment, demonstrated through behaviors outside traditional roles (e.g., initiating QI discussions, continuing to suggest specific change ideas). The CH3 Senior Sponsor described one care aide continuing to lead QI initiatives post-implementation (Table 3 Quote 2). Several Sponsors described shifts in their perceptions of care aides’ QI capacity, which persisted post SCOPE. Sponsors involved care aides in more QI initiatives, likely influencing sustainability of key benefits including work engagement. The CH6 Senior Sponsor saw value of care aides’ SCOPE work and planned to involve them in another QI project (Table 3 Quote 3).
Sponsors from three CHs (CH1,4,5) reported that SCOPE-related activities spread to additional units (Supplementary Table 1). After a SCOPE initiative to reduce responsive behaviors by reducing over-stimulation during meals, the CH5 Team Sponsor noted: “The other three units on the other side of the building are now being more aware of content on the television, music during mealtimes.” A SCOPE initiative to improve resident mobility spread to all of CH1 (Table 3 Quote 4).
Post-implementation factors influencing sustainability phenomena
For the post-implementation factors at micro-, meso-, and macro-levels, many relate to sustainment of SCOPE core elements. Some are differentially associated with sustainability and inter-unit spread. Figure 1 summarizes the relationships among post-implementation factors and sustainability phenomena.
Critical QI capacity of care aides and Sponsors is associated with sustainment and sustainability
QI skills and processes conveyed to care aide teams are core intervention elements (Table 1). Interview participants saw opportunities to apply skills and processes post-implementation as important to sustainment and sustainability of benefits. Opportunities arose through QI leadership by care aides (a sustained key benefit) and continued internal support by Sponsors (a sustained core process). The CH3 Senior Sponsor illustrated the importance of combined opportunities and support. One care aide persistently engaged colleagues in applying the PDSA cycle. This engagement was reinforced by continued Sponsor support through open communication and active facilitation.
Adaptive capacity of care aides and adaptability of the intervention facilitate sustainment
Sponsors from three CHs (CH1,3,5) observed that SCOPE team members continued to use core QI skills and processes (e.g., PDSA cycles, pre/post measurement, team-based approaches to QI opportunities) post-implementation. These were, however, adapted from the original intervention. Sponsors described adaptations as critical to sustainment. For example, a PDSA cycle approach developed post-implementation in one CH relied on less formal means of identifying problems, trialing small-scale solutions, and assessing results (Table 3 Quote 5).
Sponsors asserted that adapted approaches fit better with care aides’ typical reliance on verbal exchanges with co-workers about residents’ needs and health. This underscores concurrent importance of care aides’ adaptive capacity and adaptability of the SCOPE intervention itself.
Facilitative formal leadership of Sponsors is important to sustainment and sustainability
Most Sponsors perceived their continued support for care aides as key to SCOPE continuation. To continue supporting care aide engagement in and leadership of QI initiatives, Sponsors adapted their leadership behaviors. Post-implementation, Sponsors described new initiatives to formally recognize care aides for their QI work (CH1,5), facilitate team activities and encourage open communication (CH3), and highlight CH-wide the importance of care aide efforts (CH4). In CH2, the only home in which sustainment and sustainability were not observed, Sponsors’ post-implementation support for care aides diminished with competing demands. This presented sustainment challenges for care aides, who required Sponsor endorsement to continue work outside their usual role and pursue QI.
Sponsors keenly felt significant changes to ways they led: adopting new behaviors (facilitating – not leading – QI initiatives) and embracing uncertainties (with some discomfort) in trial-and-error learning (Table 3 Quote 6). Sponsors saw their role as encouraging care aides to take initiative. All these behaviors were acknowledged as departing from top-down leadership norms typical of long-term care.
Effective support also required Sponsors to adjust their behaviors to care aides’ fluctuating needs. For example, the CH4 Team Sponsor took a less directive approach to encourage care aide leadership but was intermittently more directive when QI team momentum faltered (Table 3 Quote 7).
While Sponsors acknowledged that care aides’ QI skills were essential to sustain core SCOPE elements, a few remarked on organizational skills and care aides’ ability to look beyond task-driven work. CH2 Team and Senior Sponsors attributed additional reasons for failure to sustain SCOPE to care aides’ apparent lack of skills in QI team organizing and inability to think beyond tasks to bigger picture QI initiatives. This signals real tension from competing demands on care aides and infeasibility of addressing higher order demands if care aides lack overt management support to step outside usual roles.
Social dynamics among Sponsors and care aides are important to sustainment
Sponsors shared examples of positive social dynamics peri-implementation that were important to continuing core elements and key benefits post-implementation. Dynamics facilitated sustainment through interactions between Sponsors and care aides and interactions among care aides. The CH3 Senior Sponsor specifically referenced the importance of trust to sustaining proactive QI communication and activity (Table 3 Quote 8).
Post-SCOPE, the CH4 Team Sponsor and the remaining 3 SCOPE care aides worked on different units but exchanged ideas during breaks. The Team Sponsor cited this as the source of sustained momentum and post-implementation sustainment of SCOPE-related activities. Indirect interactions between CH visitors and care aides motivated project continuation (Table 3 Quote 9).
Culture of innovation and change is associated with sustainment
Participants associated unit cultures open to innovation and change with post-implementation sustainment of QI processes. Some described the culture of innovation and change as prevailing before SCOPE and reflecting attitudes toward knowledge use for positive practice change (Table 3 Quote 10).
Less experience with and openness to change negatively affected ability of QI teams to sustain core processes post-implementation. CH2 Sponsors described teams struggling to engage others on their unit. Their QI initiative was seen as unnecessary extra work. Challenges continued post-implementation. Culture and staff attitudes to change were described as “making or breaking” post-implementation sustainment. Infrequently but notably, Sponsors described staff outside a QI team resisting activities initiated by team member peers, seemingly because activities were counter-cultural and challenged established routines.
Structural mechanisms are associated with sustainment and sustainability
Some Sponsors associated structural mechanisms within the unit and their CH with sustainment and sustainability. Sponsors intentionally introduced these mechanisms to routinize processes and behaviors, including team-based approaches to care and QI approaches to problems. In team, unit, and CH forums, staff could ask QI teams about initiatives, which sustained interest and momentum. One Senior Sponsor described additional benefits to teams of staff recognition and sustained engagement, with QI as a standing agenda item for staff meetings (Table 3 Quote 11).
Institutionalized practice changes contribute to sustainment and spread
Formally embedding evidence-based practice changes in care aides’ daily work routines was one means to sustain changes. Participants offered examples of “fixed” changes to mealtime environment and resident care processes that were still in place two years post-implementation. The CH1 Senior Sponsor observed impact of embedding practice changes on both sustainment and spread of a mobility initiative to other units (Table 3 Quote 12).
Workforce stability influences sustainment
Both team member and formal leadership stability were identified as important. Sponsors from CH1 and CH3–6 noted at least one original SCOPE care aide team member who remained with each original participating unit. Sponsors described them as keepers of institutional knowledge on core SCOPE elements who were essential to sustainment. For example, 2 CH1 care aides from the original team still relied on team-based approaches, organizing regular huddles to develop and track success of QI solutions to identified problems on their unit.
Care aide persistence was helpful but insufficient to guarantee sustainment of QI activities. Stability of Sponsors was seen as crucial to sustaining overall momentum in using core processes. The CH4 Team Sponsor described turnover of their original Senior Sponsor as a serious challenge to continuing activities post-implementation. They were unable to sustain financial support for their QI ideas once the Senior Sponsor left the CH.
Competing internal demands detract from sustainment
Some Sponsors described competing demands for care aides’ time and energy, hindering ability to sustain post-implementation QI processes. Care aides often worked multiple jobs, limiting full engagement in “extra” QI work and challenging their ability to gain peer support.
Sponsors also experienced numerous demands. The CH2 Senior Sponsor cited these when reflecting on why their SCOPE QI project was not sustained (Table 3 Quotes 13 & 14).
Observable benefits are associated with sustainment, sustainability, and spread
Most Sponsors described noticeable benefits at unit or CH levels – including reduced staff workload and improved resident quality of life – from practice changes through SCOPE. In some instances, observable benefits stimulated buy-in of care aides outside the QI team, contributing to overall post-implementation QI capacity in the unit or CH and continued practice changes (Table 3 Quote 15).
Seeing benefits was important to post-implementation inter-unit spread of core processes and change ideas originating from QI teams. CH4 Sponsors and original QI team members proactively presented and highlighted observable improvements in resident care and in work outcomes like reduced workload. This opened conversations for spreading initiatives to other staff and units (Table 3 Quote 16).
Alignment with system imperatives affects sustainment and sustainability
CH4 and CH6 Sponsors commented that resource constraints in long-term care created competing demands in CHs that reduced their ability to support staff QI initiatives post-implementation (Table 3 Quote 17).
Participants saw QI as one of many initiatives competing for resources. One means to reconcile competing internal and external imperatives was to align CH operational and provincial priorities. Where this was possible, related processes and benefits were sustained (Table 3 Quote 18).
Discussion and implications
This study focused on understanding unit and facility managers’ perspectives of relationships between post-implementation factors and the phenomena of sustainment, sustainability, and spread of complex interventions. Sustainment of core elements and sustainability of key benefits was observed across most CHs in our study. Spread of the SCOPE intervention was noted in half of the sites observing sustainability and sustainment. Relationships emerged through our analysis of participant interview data.
Core elements, key benefits, and precedence of influential factors
Factors influencing sustainability phenomena are not well understood [2, 10], for reasons including persistent lack of conceptual consistency, under-specification of complex interventions’ core elements, reductionism leading to limited insight on interrelations among broad influence categories (context, intervention characteristics, processes relating to sustainability, capacity to sustain), and failure to consider factors acting dynamically on sustainability phenomena at different levels within and outside the post-implementation context [2, 8, 18, 30].
Distinctions between core elements, processes, and associated behaviors that are sustained (sustainment) and ensuing benefits (sustainability) are helpful to conceptualize and study sustainability phenomena. Our deep history in developing , piloting , and implementing  SCOPE helped us make distinctions and consider influences of factors active at different levels on sustainment of core elements and sustainability of key benefits. Distinguishing sustainment and sustainability had clear value to our study and may be helpful in other settings and studies of sustainability phenomena.
While we differentiate sustainment and sustainability in analysis levels, the identified post-implementation factors align well with the four broad categories of influence for peri-implementation factors : organizational context (culture, leadership), internal and external capacity, knowledge-conveying and supportive processes, and intervention adaptability. In the broadest sense (discussed further below), our findings contribute understanding of the particular importance of factors. Several micro-level factors and the macro-level factor of system alignment contribute to sustainment of core process elements. Meso-level factors contribute to sustainability of key benefits/outcomes and sustainment of some core processes. Critically, the meso-level factors of observability and practice change institutionalization related exclusively to inter-unit spread.
Our findings suggest multiple influential factors that merit further research. As observed across studies in Stirman and colleagues’ review (2012) , and as described by our study participants, changes to outer context (e.g., policies, legislation) can alter internal priorities and shift limited resources away from QI initiatives. Capacity to sustain, including funding, resources, and workforce stability, is similarly susceptible to external pressures . Even smaller changes to local or regional contexts that influenced management turnover or care aide workforce instability profoundly influenced sustainment of core processes and sustainability of QI capacity and leadership.
Distinguishing influences of implementation and post-implementation factors
Birken and colleagues (2020)  note that research rarely distinguishes between determinants of sustainment and implementation. Our findings suggest that these differ. Specifically, we found that ability to sustain core processes is susceptible to several micro-level factors and vulnerable to vagaries of extra-organizational change. Sponsors perceived competing system priorities as barriers to sustaining and spreading SCOPE-related activities. This was not identified as a factor influencing SCOPE’s implementation in our earlier work . Unlike post-implementation, peri-implementation supports (e.g., additional resources, expertise, and focused attention) likely protect the implementation micro-context temporarily from external pressures that discourage internal change and innovation.
The importance of adaptation
Adaptation is widely acknowledged as an important, but understudied, factor influencing sustainment and sustainability [17, 32]. Here, two aspects of adaptation emerged as important: adaptability of the intervention itself and staff’s adaptive capacity to change the intervention. Sponsors described care aides adapting features of SCOPE elements post-implementation, making features easier to sustain. One unit adapted application of the PDSA cycle, sharing cause-and-effect relationships orally. This viable, reliable alternative to formalized pen-and-paper repeated measures aligned the intervention with context. Reliance on oral communications is characteristic of long-term care and care settings with non-standardized work, complex work environments, and learning by doing . Our findings corroborate observations by other sustainability scholars that “interplay between contextual factors and the innovation itself is to be expected given the dynamic nature of the complex systems into which innovations are introduced” (, p. 10).
Such observations also relate to fidelity. Partial sustainment is more common than complete sustainment . Adaptation sustains intervention elements that fit with context and discards others. Elements most closely related to desired benefits are more feasibly executed. Prevalence of partial sustainment suggests that fidelity is rarely absolute and raises questions about whether departures from fidelity that achieve partial sustainment – and improve fit – jeopardize intervention integrity. We must distinguish core intervention elements and less critical peripheral ones. Fidelity sufficiency is essential for pragmatic implementation .
Causal pathways to sustainment, sustainability, and spread
Lack of knowledge about causal pathways to sustainment and sustainability is a major acknowledged gap in sustainment and sustainability research . Our study offers insights into these pathways, including evidence of differentiable influences of some factors on particular phenomena and on interrelationships among factors. Research that extends examination of interrelationships among sustainability phenomena and influence factors strikes us and others as promising [7, 25, 32]. Stirman et al. (2012)  points to potential for this research to illuminate compensatory factors if elements like funding are missing. We observed the importance of strong social dynamics to sustainment and sustainability, even with diminished dedicated QI support.
Only two factors affecting sustainment and sustainability influenced intra-CH spread of SCOPE core processes: institutionalized practice changes and observability of benefits from QI initiatives. Spread occurred in only half of study CHs, suggesting that these factors have more nuanced differences in units where spread occurred (e.g., how practices were embedded, how observable benefits were captured or communicated) that warrant further study.
Dynamism of sustainability phenomena
With this study and earlier work on factors influencing sustainability phenomena , we can compare and contrast factors influencing those phenomena at one year and two years post-implementation. This addresses a critique that sustainability is generally examined at a single time point, overlooking its dynamism [2, 18]. All factors influencing sustainability phenomena that were noted at one year post-implementation  were also noted in this study two years post-implementation: turnover of key team members, facilitative team dynamics, observable improvements from SCOPE, strong management support of QI teams, buy-in from other CH staff, and (in)adequate resources. However, several additional factors emerged at two years: adaptive capacity of SCOPE teams, adaptability of SCOPE original elements, structural mechanisms for routinization, and institutionalized practice changes. Adaptive capacity of team and intervention adaptability sustained SCOPE core elements post-implementation. Lack of influence of these factors at one year suggests that peri-implementation research supports (e.g., coaching) may compensate. With lingering effects of external research support removed by two years, intervention fit with organizational context became more important to continuing SCOPE processes.
Structural mechanism for routinization and institutionalized practice changes both sustained momentum two years post-implementation. During peri-implementation, several SCOPE elements boosted momentum (e.g., exchanging experiences during Learning Congresses, regular coaching) and may have remained high one year post-intervention . Other factors sustaining momentum at two years may have been deemed unimportant at one year.
Strengths and limitations
By defining sustainment and sustainability as distinct concepts, we considered both SCOPE’s continued practice and achieved benefits. Previous research suggests that continued practices do not guarantee intended benefits, requiring us to distinguish these concepts  and examine both. We interviewed most Sponsors who participated in SCOPE, keeping our findings relevant to local contexts and reflecting CHs with and without sustainment activities.
First, all sites were from the same region and regulated by the same health authority. They may have more similarities than differences, limiting generalizability of findings. Second, this study used secondary data that were originally collected to inform the SSaSSy study. Secondary data analysis limited our ability to ask follow-up questions or probe deeper on issues related to this study. Also, data from care aides were unavailable. Care aide experiences and perspectives may differ from those of Sponsors, and Sponsors’ perceptions and reports may be positively skewed. However, Sponsors gave insightful administrative overviews. In next phases of the SSaSSy project, we will seek perspectives of both Sponsors and care aides. Third, this retrospective study was susceptible to recall bias. Fourth, we used telephone interviews to collect data. Lacking face-to-face interactions may have made it challenging to establish trust with the interviewees. However, this challenge was mitigated as the managers knew the interviewer (WB) and the research team during their prior participation of SCOPE.
Contributions to theory and practice
Our foremost practical contribution is demonstrating that post-implementation factors are important influencers of sustainment, sustainability, and spread of complex interventions. This suggests that similar interventions in the long-term care sector should be developed with post-implementation sustainability in mind, with measures to protect against influences such as workforce instability and competing internal and external demands. The need to adapt intervention aspects must be anticipated, to enhance post-implementation traction. Participants offered many ways to anticipate vagaries of post-implementation contexts: institutionalizing practice changes, “hardwiring” structural changes that supported process and behavioral changes, highlighting observable benefits, and – whether intentionally or not – building positive social dynamics to keep vital communication channels open and engender staff trust.
Our findings corroborate relationships suggested in extant sustainability theories, models, and frameworks. Many factors identified as influencing sustainment and sustainability related to social interactions among care workers, aligning with well-established implementation science theories that highlight the crucial role of social dynamics among individuals and organizations in implementing and sustaining change interventions [34,35,36,37]. Our findings also align with learning theories that underscore importance of institutionalizing changes to practice routines . We identify several unit- and CH-level factors that facilitated integration of SCOPE elements into institutional routines. We observed system-level imperatives that compete with sustaining core elements and key benefits, aligning with open systems theories of organizations as systems profoundly influenced by micro- and macro-environments .
Our empirical evidence supports the dynamic view of sustainability [8, 11, 18]. Our findings support the importance for sustainability of constant adaptations and improvement of intervention and organizational context. Facilitative factors for sustainability at two years post-implementation emerged in response to decreasing organization resources for intervention delivery. This corroborates the dynamic sustainability framework, which “embraces change as a central influence on sustainability” (, p. 7), including intervention–organization fit. That framework also posits the critical role of an organization’s ability to continuously learn and adapt as “a core value of the implementation setting” (, p. 6). Facilitative factors for sustainability that emerged only by two years post-implementation indicate that CHs can adapt to external environment changes by spontaneously improving organizational context and intervention.
Our study responds directly to calls for research that attends to conceptual clarity of sustainability phenomena, considers influences of factors at multiple levels simultaneously, and considers these influences in light of sustainment of core elements and sustainability of key benefits of complex interventions .
We identify post-implementation factors active at micro, meso, and macro analysis levels that influence sustainability phenomena. This work complements existing research on how peri-implementation factors influence sustainability.
Most sustainability research focuses on interventions with clinicians in public health and community care [6, 11, 30, 38]. Our study contributes understanding of sustainability phenomena in long-term care, which is inarguably complex, under-resourced relative to most care sectors, and reliant predominantly on unregulated healthcare workers with less formal exposure to evidence-informed complex interventions. The fact that complex QI skills and processes were sustained for two years after implementation spotlights under-acknowledged capacity for learning and improvement in this sector.
Availability of data and materials
The datasets generated during and analysed during the current study are not publicly available due to limitations of ethical approval involving the patient data and anonymity but are available from WB (email@example.com) on reasonable request.
The Sustainment, Sustainability, and Spread Study
Safer Care for Older People in Residential Environments
Khan S, Vandermorris A, Shepherd J, Begun JW, Lanham HJ, Uhl-Bien M, et al. Embracing uncertainty, managing complexity: applying complexity thinking principles to transformation efforts in healthcare systems. BMC Health Serv Res. 2018;18(1):192.
Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7(1):17.
Herlitz L, MacIntyre H, Osborn T, Bonell C. The sustainability of public health interventions in schools: a systematic review. Implement Sci. 2020;15(1):1–28.
Olomu AB, Stommel M, Holmes-Rovner MM, Prieto AR, Corser WD, Gourineni V, et al. Is quality improvement sustainable? Findings of the American college of cardiology’s guidelines applied in practice. Int J Qual Health Care. 2014;26(3):215–22.
Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.
Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. 2011;101(11):2059–67.
Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability science: an integrated approach for health-programme planning. Lancet. 2008;372(9649):1579–89.
Lennox L. Sustainability. Handbook on Implementation Science: Edward Elgar Publishing; 2020.
Flynn R, Rotter T, Hartfield D, Newton AS, Scott SD. A realist evaluation to identify contexts and mechanisms that enabled and hindered implementation and had an effect on sustainability of a lean intervention in pediatric healthcare. BMC Health Serv Res. 2019;19(1):912.
Flynn R, Mrklas K, Campbell A, Wasylak T, Scott SD. Contextual factors and mechanisms that influence sustainability: a realist evaluation of two scaled, multi-component interventions. BMC Health Serv Res. 2021;21(1):1–7.
Ghag K, Bahl R, Winter C, Lynch M, Bautista N, Ilagan R, Ellis M, de Salis I, Draycott TJ. Key components influencing the sustainability of a multi-professional obstetric emergencies training programme in a middle-income setting: a qualitative study. BMC Health Serv Res. 2021;21(1):1–1.
Berta W, Cranley L, Dearing JW, Dogherty EJ, Squires JE, Estabrooks CA. Why (we think) facilitation works: insights from organizational learning theory. Implement Sci. 2015;10(1):1–13.
Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. 2018;13(1):27.
Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(1):117.
Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, Hinder S, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19(11):e367.
Berta WB, Wagg A, Cranley L, Doupe MB, Ginsburg L, Hoben M, et al. Sustainment, Sustainability, and Spread Study (SSaSSy): protocol for a study of factors that contribute to the sustainment, sustainability, and spread of practice changes introduced through an evidence-based quality-improvement intervention in Canadian nursing homes. Implement Sci. 2019;14(1):109.
Proctor E, Luke D, Calhoun A, McMillen C, Brownson R, McCrary S, et al. Sustainability of evidence-based healthcare: research agenda, methodological advances, and infrastructure support. Implement Sci. 2015;10(1):88.
Shelton RC, Lee M. Sustaining evidence-based interventions and policies: recent innovations and future directions in implementation science. Am J Public Health. 2019;109(S2):S132–4.
Ginsburg L, Easterbrook A, Berta W, Norton P, Doupe M, Knopp-Sihota J, et al. Implementing frontline worker–led quality improvement in nursing homes: getting to “how.” Jt Comm J Qual Patient Saf. 2018;44(9):526–35.
Doupe M, Brunkert T, Wagg A, Ginsburg L, Norton P, Berta W, Knopp-Sihota J, Estabrooks C. SCOPE: safer care for older persons (in residential) environments—a pilot study to enhance care aide-led quality improvement in nursing homes. Pilot Feasibility Stud. 2022;8(1):1-3. 21.
Hewko SJ, Cooper SL, Huynh H, Spiwek TL, Carleton HL, Reid S, et al. Invisible no more: a scoping review of the health care aide workforce literature. BMC Nurs. 2015;14(1):38.
Cranley LA, Norton PG, Cummings GG, Barnard D, Estabrooks CA. SCOPE: safer care for older persons (in residential) environments: a study protocol. Implement Sci. 2011;6(1):71.
Institute for Healthcare Improvement. The breakthrough series: IHI’s collaborative model for achieving breakthrough improvement. Boston, MA: Institute for Healthcare Improvement; 2003.
Norton P, Cranley L, Cummings G, Estabrooks C. Report of a pilot study of quality improvement in nursing homes led by healthcare aides. Eur J Pers Cent Healthc. 2013;1(1):255–64.
Cranley LA, Hoben M, Yeung J, Estabrooks CA, Norton PG, Wagg A. SCOPEOUT: sustainability and spread of quality improvement activities in long-term care- a mixed methods approach. BMC Health Serv Res. 2018;18(1):174.
Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40.
Boyatzis RE. Transforming qualitative information: Thematic analysis and code development. Thousand Oaks: Sage; 1998.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101.
Krefting L. Rigor in qualitative research: the assessment of trustworthiness. Am J Occup Ther. 1991;45(3):214–22.
Klaic M, Kapp S, Hudson P, Chapman W, Denehy L, Story D, Francis JJ. Implementability of healthcare interventions: an overview of reviews and development of a conceptual framework. Implement Sci. 2022;17(1):1–20.
Birken SA, Haines ER, Hwang S, Chambers DA, Bunger AC, Nilsen P. Advancing understanding and identifying strategies for sustaining evidence-based practices: a review of reviews. Implement Sci. 2020;15(1):88.
Penno LN, Davies B, Graham ID, Backman C, MacDonald I, Bain J, et al. Identifying relevant concepts and factors for the sustainability of evidence-based practices within acute care contexts: a systematic review and theory analysis of selected sustainability frameworks. Implement Sci. 2019;14(1):108.
Nonaka l, Takeuchi H, Umemoto K. A theory of organizational knowledge creation. Int J Technol Manag. 1996;11(7–8):833–45.
Flynn R, Scott SD. Understanding determinants of sustainability through a realist investigation of a large-scale quality improvement initiative (lean): a refined program theory. J Nurs Scholarsh. 2020;52(1):65–74.
Goodman RM, Steckler A. A model for the institutionalization of health promotion programs. Fam Community Health. 1989;11(4):63–78.
May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization Process Theory. Implement Sci. 2009;4(1):1–9.
Scott WR, Davis GF. Organizations and organizing: Rational, natural and open systems perspectives. Oxfordshire: Routledge; 2015.
Greve M, Brendel AB, van Osten N, Kolbe LM. Overcoming the barriers of mobile health that hamper sustainability in low-resource environments. J Public Health. 2022;30(1):49–62.
Our sincere thanks to Ms. Jennifer Pietracci for assistance in connecting us with study participants, and Mr. Don McLeod for inputs in preparing the interview questions based upon his industry experience. We thank Cathy McPhalen, PhD, of thINK Editing Inc, Edmonton, Canada, for providing editorial support in accordance with Good Publication Practice (GPP3) guidelines (http://www.ismpp.org/gpp3).
Funding for this study has been provided by the Canadian Institutes for Health Research, Grant # 201803PJT-400653-KTR-CEAA-107836. The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Ethics approval and consent to participate
Ethics approval was obtained from the University of Toronto, University of Manitoba, and Winnipeg Regional Health Authority. All methods were performed in accordance with relevant guidelines and regulations. Written informed consent was obtained from each participant.
Consent for publication
The authors declare that they have no completing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Song, Y., MacEachern, L., Doupe, M.B. et al. Influences of post-implementation factors on the sustainability, sustainment, and intra-organizational spread of complex interventions. BMC Health Serv Res 22, 666 (2022). https://doi.org/10.1186/s12913-022-08026-x