Skip to main content
  • Research article
  • Open access
  • Published:

Healthcare Quality Improvement and ‘work engagement’; concluding results from a national, longitudinal, cross-sectional study of the ‘Productive Ward-Releasing Time to Care’ Programme

Abstract

Background

Concerns about patient safety and reducing harm have led to a particular focus on initiatives that improve healthcare quality. However Quality Improvement (QI) initiatives have in the past typically faltered because they fail to fully engage healthcare professionals, resulting in apathy and resistance amongst this group of key stakeholders. Productive Ward: Releasing Time to Care (PW) is a ward-based QI programme created to help ward-based teams redesign and streamline the way that they work; leaving more time to care for patients. PW is designed to engage and empower ward-based teams to improve the safety, quality and delivery of care.

Methods

The main objective of this study was to explore whether PW sustains the ‘engagement’ of ward-based teams by examining the longitudinal effect that the national QI programme had on the ‘work-engagement’ of ward-based teams in Ireland. Utilising the Utrecht Work Engagement Scale questionnaire (UWES-17), we surveyed nine PW (intervention) sites from typical acute Medical/Surgical, Rehabilitation and Elderly services (representing the entire cohort of a national phase of PW implementation in Ireland) and a cohort of matched control sites. The numbers surveyed from the PW group at T1 (up to 3 months after commencing the programme) totalled 253 ward-team members and 249 from the control group. At T2 (12 months later), the survey was repeated with 233 ward-team members from the PW sites and 236 from the control group.

Results

Overall findings demonstrated that those involved in the QI initiative had higher ‘engagement’ scores at T1 and T2 in comparison to the control group. Total ‘engagement’ score (TES), and its 3 dimensions, were all significantly higher in the PW group at T1, but only the Vigour dimension remained significantly higher at T2 (p = 0.006).

Conclusion

Our results lend some support to the assertions of the PW initiative itself and suggest that when compared to a control group, ward-based teams involved in the QI programme are more likely to be ‘engaged’ by it and its associated improvement activities and that this is maintained over time. However, only the Vigour dimension of ‘engagement’ remained significantly higher in the PW over time.

Peer Review reports

Background

There are many drivers for improving quality in healthcare. The pressures associated with growing populations, changing healthcare needs, increasing healthcare costs, in addition to concerns about patient safety and reducing harm are amongst some of the most compelling. In addition, poor perception and experiences of healthcare from both patients and the media have collectively combined to provide many of the ingredients for a ‘perfect storm’ in healthcare crisis terms. This has led to an international call-to-action to ‘rescue’ the provision of healthcare with a particular renewed focus on initiatives that improve the quality of healthcare [1, 2].

Although the requirement to improve healthcare whilst trying to master what works well and why is very well established; there is in fact a limited understanding of the exact impacts, outcomes and outputs from many of the interventions that are designed to improve healthcare quality. This lack of understanding has created an interest amongst researchers and clinicians to articulate what constitutes a successful healthcare QI intervention or initiative [3].

Some of the current views on implementing healthcare quality innovations have their foundational basis in Pressman and Wildavsky’s study of policy implementation, [4] and Havelock’s change agent studies in education [5]. There appears to be broad agreement in the literature that the implementation of healthcare QI is decidedly more complex and fraught with many more variables than had been previously assumed [6, 7].

In this regard, what is termed ‘Improvement Science’ has evolved to look beyond the descriptive theories of innovation, implementation and change and to focus on other important components that are required for effective implementation of QI interventions and strategies. These include the many contextual variables such as circumstances, behaviours and interactions that result in improved quality.

QI initiatives have in the past typically failed to ‘engage’ healthcare professionals and faltered as a result. Many studies, for example, report apathy and resistance from clinicians when such initiatives are introduced [8, 9]. In this regard, getting healthcare professionals to think and behave in different ways is not all that straightforward and many need to be convinced of the value and merits of improvement methods tools or programmes [10]. For some healthcare professionals QI can be perceived as a ‘job-demand’ that detracts from clinical care and not a ‘resource’ that enables them to improve quality [11]. It is now widely acknowledged that engaging clinicians (regardless of setting or discipline) is a precondition for the success of QI initiatives [12, 13].

In trying to find a panacea for what ‘engages’ healthcare professionals effectively, many healthcare organisations have attempted to replicate the improvement and ‘engagement’ successes lauded in the manufacturing sector. QI programmes adopted from industry and remodelled or adapted for the health care setting currently take a variety of formats, including ‘Lean Healthcare’, ‘Six Sigma’, ‘Clinical Microsystems’ and the ‘Model for Improvement’.

It could be argued that many of these programmatic approaches to healthcare QI have also failed to fully ‘engage’ healthcare employees and that this may in part, be due to the bifurcation of intent between the significant stakeholders upon whom healthcare QI success depends [10]. Healthcare professionals’ views and understanding of QI have been shown to be divergent from that of managers [14]. Indeed, clinicians can often perceive these initiatives as a threat to their professional practice, identities, status and power [15].

This dichotomy of world views has been previously described in terms of competing logics - professional, managerial and institutional [16, 17], and provides some explanation as to why many front-line clinical staff (including nurses) generally steer away from the financial aspects of care and direct their attention and efforts towards care provision and the patient experience [18]. It also provides some understanding as to why non-clinical stakeholders generally tend to focus on just the one dimension; reducing the per capita cost of health care.

The challenge for contemporary healthcare QI initiatives are how they ‘engage’ the many competing logics, interests and intentions to deliver improvements in the key areas of population health, patient experience and cost [1, 18]. Productive Ward: Releasing Time to Care™ (PW) is one such QI programme that was designed to synergise the multiple aspects of QI and directly impact care provision, patient experience and costs [19, 20].

PW, designed and developed by the UK’s National Health Service Institute for Innovation and Improvement (NHSI) in 2005, may be best described as a ward-based QI programme which helps ward-based teams redesign and streamline the way they work in order to leave more time to care for patients. Using Lean improvement techniques, the intrinsic motivators of social movement theory and the front-line ‘engagement’ theories of large-scale change, the PW programme is designed to ‘engage’ and ‘empower’ teams to improve the safety, quality and delivery of care. The PW programme, which has been adopted in a number of countries, has been the focus of multiple studies and evaluations [21] and is reported to ‘engage’ front-line teams and clinicans in QI and QI activities [19, 20, 22].

In December 2010 Ireland made PW one of its national clinical care programmes and priorities. The recruitment of appropriate sites for implementation began in September 2011 and organisations/hospitals who had expressed an interest in participating were interviewed. A readiness audit tool (based on a previous NHSI site-selection tool) was developed to ensure that only well-prepared, well-supported sites were chosen for the initial phase. Fifty-four sites expressed an interest and in November 2011, 17 pilot sites (involving 24 wards) throughout Ireland were selected for the first phase of a national implementation.

In November 2012, a second cohort, consisting of nine wards (seven sites), were chosen for the next phase of implementation in early 2013. Evaluation of the initiative was carried out with this second implementation phase and focused on examining the ‘engagement’ claims of the QI initiative and the impact that the PW programme and its associated QI activities had on the ‘engagement’ (and the sustaining of ‘engagement’) of participants. Initial ‘engagement’ scores were obtained from this cohort in early 2013 and have been previously reported [22].

Methods

Objectives

The main objective of this study was to explore whether PW sustains the ‘engagement’ of ward-based teams by examining the longitudinal effect that a national QI programme (PW) had on ‘engaging’ ward teams by:

  • Measuring and comparing the preliminary ‘engagement’ scores from the ward-based teams involved in the national pilot phase of the PW and a control group (of similar size, from similar clinical specialty areas, who were not involved in a quality improvement programme, initiative or improvement activity) [22], as a baseline measure (T1).

  • Measuring and comparing ‘engagement’ scores within both the intervention and control group again approximately 12 months later (T2).

  • Comparing changes in ‘engagement’ scores (T2-T1) in the intervention and control groups, controlling for other variables.

Study design

This study uses a longitudinal cohort survey design to examine ward team ‘engagement’ (nursing and non-nursing) in the national ‘PW’ QI initiative in Ireland across two time periods. Ethics approval was provided by Waterford Institute of Technology and the Health Service Executive.

Setting

Nine PW (intervention) sites (representing the entire cohort of a national phase of PW implementation in Ireland) and a cohort of matched control sites participated in this study. The nine PW sites represent the typical acute Medical/Surgical, Rehabilitation and Elderly services that participated in the entire national PW initiative (see Table 1). All resident ward team members (nursing and non-nursing) were invited to participate in this study.

Table 1 Research setting and response rates

Participants

A stratified sample of 253 ward-team members from the nine wards/units involved in the initiative (the total eligible population of a national phase of PW implementation) were identified through the project lead in each setting and surveyed in early 2013 (T1). Data was collected up to 12 weeks into the implementation of the QI programme and compared to data from a matched (approximate fit) control group collected at the same time.

The matching control quota sample (using approximate matching criteria) were recruited by local Quality leads and used as a comparator for the PW sites (see Table 1). The stratification characteristics of the control group - a purposive sample - were:

  • number of wards/units (n = 9)

  • similar clinical specialty/ward environment

  • similar ward size and sample size

  • consent to participate in the study

  • non-participation in a QI initiative

Table 2 below, shows the grade and age distributions of the respondent sample (T1) and highlights the strong relationship between age and employment grade. For example, it is seen that the Staff Nurse grade comprises 100% of the 18–24 age group, 72.2% of the 25–44 age group, but only 53.1% of the 45–65 age group. The vast majority of respondents were in either the 25–44 age group (57.7%) or the 45–65 age group (38.7%).

Table 2 Age and grade distribution of respondents

Because of this strong relationship between employment grade and age, we did not control for age in subsequent analyses. Neither did we control for gender, as the sample was overwhelmingly female. The variables analysed in this study were, therefore, ‘engagement’ scores, changes in these ‘engagement’ scores over time, employment grade, specialty and group (PW/control).

Description sample T2

This phase of the research design was a repeat of the T1 survey (same intervention, PW sites, control group, instrument and procedure) approximately 12 months later. At T2, 233 ward-team members from the PW sites and 236 from the control group were again surveyed. A descriptive breakdown of T2 participants is included in Table 1 and Table 3. Staff Nurse grades again represented the largest group of respondents (68.2%) in the sample, followed by Healthcare Support workers (16.1%) and Nurse Managers (8.6%).

Table 3 Descriptive breakdown of participants who completed both T1 & T2

Data sources/measurement

The 17-item Utrecht Work Engagement Scale questionnaire (UWES-17) -a three-dimensional model of vigour, dedication and absorption-, [23] was used to measure the total levels of ‘engagement’ (TES). The measure conceptualises ‘engagement’ as a positive, fulfilling, work-related state of mind and is characterised by positive combinations of all three dimensions.

  • Vigour refers to high levels of energy and mental resilience while working; the willingness to invest effort in one’s work and persistence even in the face of difficulties

  • Dedication refers to being strongly involved in one’s work and experiencing a sense of significance, enthusiasm, inspiration, pride, and challenge

  • Absorption refers to being fully concentrated and happily engrossed in one’s work whereby time passes quickly and one has difficulties with detaching oneself from work [24].

The concept and measure is grounded in the job-demands domain of the job demands/job-resources conceptual framework where job-resources are positively valued physical, social or organisational aspects of the job that are integral to achieving work goals or reduce job-demands (negatively valued physical, social, or organisational aspects of the job) [25].

Each item is scored on a seven-point rating scale from 0 (never) to 7 (every day). The mean scale score (TES) of the three UWES subscales is computed by adding the scores on the particular scale and dividing the sum by the number of items of the subscale involved. A similar procedure is followed for the total score. Hence, the UWES, yields three subscale scores and/or a total score (TES) that range between 0 and 6.

The UWES is the most established and widely accepted measure of employee ‘engagement’ in the academic literature, [26, 27] and has repeatedly been shown to be psychometrically sound and reliable in a variety of international healthcare settings [28,29,30].

Statistical methods

Data were analysed using the commercial software SPSS (version 21). Frequency and descriptive statistics were generated for each of the variables contained in the questionnaire. Statistical analyses which we had performed included:

  1. a.

    Standard reliability analysis of the questionnaire items, in order to confirm suitability of the UWES-17 scales in both a QI and Irish setting;

  2. b.

    Comparison of UWES scores (total ‘engagement’ scores and the individual constructs) in PW and control groups, using independent sample t-tests;

  3. c.

    Investigation of relationships between ‘engagement’ scores and other variables, using t-tests or contingency table analysis, as appropriate;

  4. d.

    Analysis (using general linear models) of ‘engagement’ scores in PW and control groups, controlling for confounding variables identified in (c);

Non-parametric analogues of t-tests were also employed, as appropriate, e.g. when ‘engagement’ scores were found to be not normally distributed. Five percent level of statistical significance was adopted throughout this study, without adjustment for multiple testing.

For the current study we changed the general linear model analysis to a linear mixed model analysis, following a journal reviewer’s recommendation that the analysis should reflect that study participants were clustered in Wards. We repeated the earlier T1 analysis of ‘engagement’ scores, using the new models, and continued to use the new models for the analysis of T2 ‘engagement’ score’s, and for the analysis of change in ‘engagement’ scores from T1 to T2. In all cases, the models chosen had Ward as a Level 2 variable, a random intercept term, and explanatory variables Group (PW/control), employment grade and specialty. Only main effect models were analysed.

Results

T1

The initial results obtained at T1 have been reported previously, the main finding being that, controlling for employment grade and specialty, all ‘engagement’ scores at T1 were significantly higher in the PW group compared to the control group [22]. However, that conclusion was based on general linear model analyses, and on a larger sample than the current study. In the current study, we repeated the T1 analyses of ‘engagement scores’, based on subjects who also completed study questionnaires at T2, and on this occasion (as suggested by a journal reviewer) we used linear mixed model analysis (with Ward as a Level 2 variable) instead of the general linear models used in the earlier analysis.

We now report that the main conclusions from the earlier study still stand. Based on the linear mixed model analyses, and the smaller sample size, TES at T1, and its separate components, are all significantly higher in the PW group. Table e (Additional file 1) has detailed output from the mixed models; Table 4 (upper panel) below, summarises the main T1 findings from these models, as they relate to ‘engagement’ scores – we present effect sizes (average amount by which PW ‘engagement’ scores exceed control ‘engagement’ scores), and 95% confidence intervals (CI’s) for each effect. For all ‘engagement’ measures, the effects are positive, substantial and statistically significant i.e. all ‘engagement’ scores are significantly higher, on average, in the PW group compared to the control group.

Table 4 T1 & T2 effect sizes and p-values

Also Included in table e (Additional file 1), but not in Table 4 (which focuses on ‘engagement’), are some interesting details about the other explanatory variables, employment grade and specialty. There are statistically significant differences among employment grades, in TES, vigour and absorption at T1, with Nurse Managers consistently showing higher ‘engagement’ scores than other categories e.g. from the mixed model output, it is seen that the TES at T1 for Nurse Managers is 0.36 higher than for support staff and 0.67 higher than for staff nurses. Elderly specialties had consistently higher T1 ‘engagement’ scores than other specialties, but these were not statistically significant.

T2

In total, 192 participants completed and returned surveys in both T1 and T2 phases, representing a total response rate of 56.8% of the original 338 participants who returned surveys in T1, but only 38.2% of the 502 originally surveyed 12 months previously (Table 3). At T2, 101 responded from the PW group (52.6%) and 91 responded from the control group (47.4%). The total 192 completed questionnaires were included in the T1/T2 analysis reported below, but in analyses involving employment grades, only 187 subjects were included, because we removed the two employment grades with small numbers (Clerical and Household).

'Engagement' scores in PW and control groups at T2 T2

We used the same statistical approach (linear mixed models) as described above, with T2 ‘engagement’ scores replacing the T1 scores as dependent variable in each analysis. Table’s m and n (Additional file 1) have the detailed output from these models. Table 4 (lower panel) and Fig. 1 below highlights the main findings as they relate to comparison of PW and control groups. In general, following these analyses, T1 results were replicated at T2 and the overall conclusions remain the same: the effect sizes in Table 4 (lower panel, T2) above, are all still positive (after controlling for clinical specialty and employment grade), meaning that T2 ‘engagement’ scores were still uniformly higher in the PW group when compared with the control group; however, most of these PW scores were no longer significantly higher (at the 5% level) at T2.

Fig. 1
figure 1

T1 & T2 ‘Engagement’ Scores by clinical specialty; highlighting the higher engagement scores that were generally maintained in the PW group over the two time periods

Table n (Additional file 1) also has information (from the T2 mixed models) on the relationship of T2 ‘engagement’ scores to specialty and employment grade. It is seen that the superiority of ‘engagement’ scores in the Nurse Manager grade continues at T2 (although not always significantly higher). The main difference is that, at T2, statistically significant differences among specialties becomes evident, with the Rehab specialty now showing substantially higher ‘engagement’ scores (for vigour and dedication) than the Elderly specialty (which had uniformly higher scores at T1).

Changes in ‘engagement’ scores from T1 to T2

It is evident, from the T1 and T2 findings reported above, that statistically significant results at T1 were not always replicated at T2. For the remainder of this results section, we provide information on some of the changes that occurred between T1 and T2, which may have contributed to the observed discrepancies in the two sets of results.

Table 5 presents the average ‘engagement’ scores (total and individual dimensions) at both of T1 and T2, separately within each group, and (final column) the independent t-test p values for between-group comparison of changes in these scores over the study period. As seen in Table 5, absorption score grew significantly more (over the 12-month period) in the control group, explaining why there was no longer a statistically significant difference in absorption between groups at T2. Changes in the other three ‘engagement’ scores were not significantly different between groups, but in all three cases the PW TES declined from T1 to T2, whereas the control group TES and dedication scores increased. Table 4, also highlights that only vigour remained significantly higher in the PW group at T2.

Table 5 T1 & T2 means, standard deviations and p values for change between T1 and T2

We also used a linear mixed model, with T2-T1 difference as dependent variable, specialty, grade and group (PW, Control) as factors, and Ward as a Level II variable (see Additional file 1: Tables y,x,u). These analyses revealed (i) that the T2-T1 change in absorption score was significantly different between PW and control groups (p = 0.024 in linear mixed model, with control group showing the greater increase Additional file 1: Table y) – confirming the finding from the simpler analysis in Table 5 and (ii) that T2-T1 change in TES (p = 0.013) and vigour (p = 0.011) were significantly different by specialty (Additional file 1: Tables x,u) – with Rehab showing the greatest improvement on both scores. Apart from these, T2 – T1 changes in ‘engagement’ scores were not significantly different from changes in the control group ‘engagement’ scores.

Discussion

Key results

The primary aim of this study was to examine the longitudinal impact of the nationally introduced QI initiative PW in Ireland on the levels of ‘engagement’ in the ward teams implementing it and to establish whether they were more ‘engaged’ than ward teams not involved in PW or any other QI intervention. The findings of this study only partially confirm the previous findings [22] at T1, in that ‘engagement’ scores, controlling for employment grade and clinical specialty, remained higher at T2 in the group implementing the PW initiative compared to a control group. However, only the Vigour score was significantly higher in the PW group at T2 (p = 0.006).

At a time when it is becoming widely acknowledged that engaging clinicians from all settings and disciplines is a precondition for the success of QI initiatives, [12, 13, 31] the findings from this study provide mildly encouraging evidence that PW, as a programmatic QI approach, is quite likely to ‘engage’ its participants and sustain moderate levels of interest, energy (vigour) and ‘engagement’. This may in part be due to the initial testing, piloting, marketing and investment that went into its development. It has previously been reported that the ‘releasing time to care’ strapline was added pre-launch, following initial piloting, to further ‘engage’ and ‘entice’ nurses and ward teams who had declared that the QI programme represented a hard industrial methods approach [32]. The use of marketing and straplines that connect healthcare QI implementers to a ‘purpose’ has been previously described within the ‘social movement theory’ context and outlined as a critical enabler for QI ‘engagement’ and success [33].

The ‘connection to a purpose’ approach adopted by PW has however received some criticism, for creating a state of ‘desirability’ amongst nurses, [34] where it has been argued that nurses are potentially drawn to ‘engage’ with the programme on a professional ‘principle’ basis (releasing time to care) and not because improving all aspects of healthcare quality is the right thing to do. Sustaining ‘engagement’ in PW on this professional principle-basis would be a most difficult endeavour. Mostly because the purpose ‘releasing time to care’ has been difficult to definitively measure and improve [32]. The modestly positive ‘engagement’ findings (compared to a control group) across two time periods, would however suggest that to some extent, the PW programme does not just ‘engage’ ward-based teams on an initial principle-based or social movement basis, it is quite possible that it can continue to ‘engage’ ward team members (in the the various QI activities) over a prolonged (12-month) period.

Previous reports of PW, [20, 35,36,37] have indicated how clinical workload; bed shortages; sick leave; increased winter activity and shortage of temporary/relief staff have affected the morale and participation of staff (hampering ‘engagement’ and progress with some QI activities). In the current study, there was some decline in the PW group ‘engagement’ scores over time, and one could argue that this may be partly have been due to the implementation of this large-scale QI programme (and study) being undertaken during the most austere of economic times in Ireland [38]. However, whilst participants involved in this national PW maintained their higher ‘engagement’ scores there were increases observed in the control group.

We believe that there are a number of possible contextual explanations for the PW group generally maintaining their higher scores including:

  • the phased modular content and associated ease of learning QI methods,

  • the local supports associated with the national and structured implementation in Ireland,

  • the perception that this initiative was one of the few development opportunities being offered to staff during an austere-induced drought of development in healthcare in Ireland,

  • the positive impact that the QI tools/methods had on participants levels of professional self-efficacy; which have been recently been identified as a predictor of ‘engagement’ [39].

It could be argued that the austerity cuts to healthcare in Ireland did have some effect or impact on the control group. A moderate rise in ‘absorption’ scores (the only statistically significant group effect detected when changes in scores from T1 to T2 were analysed) was observed. It is noteworthy though that no increases were observed in the control group in relation to the other dimensions ‘vigour’ or ‘dedication’. It could be postulated that the increased activity, short staffing, reduced promotional opportunities/training and cost-cutting that took place during the economic crisis may inadvertently have encouraged ward-based teams in the control group to just focus their attention on and be more ‘absorbed’ by their clinical work; in-line with the expression of ‘keeping your head at work’ and a possible defensive coping mechanism to combat additional ‘stress’ [40].

The moderately higher TES’s for the non-nursing Clerical/Administration and Household (indirect care) team members that were observed at T1 in both the PW and control groups, could not be explored further as a result of reduced participants at T2. The findings from T1 however should not be ignored. Higher levels of stress and emotional demands are regularly reported in many healthcare occupations, like nursing (the predominant employment grade in this study), who provide direct patient care. This has been shown to make this group particularly susceptible to burnout, [40,41,42,43,44] which is well recognised as an antithesis to vigour (high levels of energy and mental resilience while working) and ‘engagement’ [45,46,47]. This offers some explanation as to why the nursing employment grades recorded slightly lower ‘engagement’ scores than ward-team members not involved in direct patient care and this should be considered in further measures of ‘engagement’ within the QI arena.

The significantly higher TES’s at T1 for the Elderly sites were much more moderate at T2. However the elevated TES’s that were observed in the other non-acute setting Rehab, remained elevated in both the PW and the control group. Indeed this non-acute setting represented the only clinical site to significantly improve their ‘engagement’ scores over the two time periods. Whilst some studies have shown different levels of ‘engagement’ between the acute and non-acute sectors, [48] this study is amongst the first to demonstrate higher levels of ‘engagement’ in non-acute settings involved in a programmatic QI intervention and merits further investigation.

This may in part be due to the very contrasting contexts, competing commitments to QI systems and processes experienced between the acute and non-acute healthcare settings [49].

It also may be due to lower levels of burnout reported in many non-acute settings that are a result of contrasting work-related factors [42, 50], (reduced patient turnover; a reduced focus on budgets and targets; higher levels of organisational slack and more of a focus on person-centeredness) that are not observed in the busy acute healthcare sector.

The findings from this study provide some evidence that ‘off the shelf’ programmatic QI initiatives like PW are most likely to be perceived as a ‘job-resource’ and have the capacity to ‘engage’ and maintain that ‘engagement’ for at least 12 months. It also adds to the growing body of knowledge of ‘engagement’ and the job-demands/job-resources conceptual framework within a QI context and further validates the use of the ‘work engagement’ construct and measure as being suitable for use with healthcare QI.

Finally this study provides some response to the critics of lean-based QI programmes like PW, [51, 52] who have raised doubts about the impact and sustainability of such initiatives and what they can meaningfully deliver. If ‘engagement’ is one of the key challenges to improving quality, [9, 13] the modestly positive, sustained results from the PW initiative in Ireland (particularly around the vigour dimension) are encouraging.

Bias

Potential biases can be identified in the national implementation strategy employed, roll-out scheduling, as well as the selection of clinical units involved in both phases of PW. We sought to avoid an additional source of bias by the use of anonymous subject ID’s when collecting and recording responses from individual study participants.

Limitations

Firstly, it is important to highlight that the PW (intervention group) were selected as part of a national pilot phase of the initiative. It is plausible that many of the organisations and wards who put themselves forward for the pilot have some degree of highly motivated individuals who applied or have a track record as a high performing ward or department.

A second limitation of this study is the use of a non-probability quota sampling strategy for recruiting the control group. Whilst the characteristics of size and clinical context of the control group generally reflect that of the PW group, the sample is in essence a purposive sample; the matching exercise, no matter how rigorous, can never be truly representative. Access to a randomised control group would of course be ‘gold standard’ for a QI study of this nature, but realistically would be extremely challenging from a number of perspectives. This study did control (using linear mixed models) for variables such as specialty and employment grade which differed between the intervention and control groups, and which were also related to the ‘engagement’ outcome measures.

All study variables were extracted from the resident ward teams’ reported perspectives. Other objective variables and perspectives from extended ward-team members (visiting medical colleagues and allied healthcare professionals) could extend confirmation of these study results.

Finally, although this study examines the ‘engagement’ scores across two time points, it could be considerably strengthened with pre-intervention measures and additional data from further phases which would allow more complex analysis of relationships between employment grade, specialty site and the control group. It would also go some way to help address the growing concerns in regard to understanding the many influences involved in sustaining healthcare QI over time [1, 53, 54]. However, it is worth noting that QI programmes, interventions and activities by their very nature are action-orientated occurrences and do not naturally lend themselves to the criteria and standards of experimental design. Pre-intervention measures were hampered in this study by long delays in ethics approval and the overriding requirements by the services to start the QI initiative and associated activities.

Generalisability

This study was performed across a variety of acute and non-acute hospital settings in one national health system and generalisation to other settings should be approached with caution. All findings in this study can only be viewed through the lens in which they were studied, i.e. teams involved in implementing the PW in Ireland. However, the generalisability and transferability of learning from all QI initiatives deserves consideration when trying to broaden, spread or replicate QI efforts as there are many organisational, contextual and social challenges to take into account [55, 56].

Conclusion

Overall these results suggest that when compared to a control group (of similar size, from similar clinical specialty areas, who were not involved in a quality improvement programme, initiative or improvement activity) ward-based teams who participate in the QI programme PW are more likely to be ‘engaged’ by it and its associated improvement activities and it is possible to maintain these elevated levels of ‘engagement’ over a 12-month period. Further longitudinal studies with pre-intervention measures are required to examine how sustained this effect might be. We hypothesise that the relatively stable positive ‘engagement’ measures observed in this study indicate that PW and its QI activities are most likely viewed as a job-resource (a positive, valued aspect of one’s job which may also possibly negate the job-demands of one’s job). However, using the job demands/job resources theoretical framework within a healthcare QI context requires further exploration and comparison with and against other QI initiatives. This study demonstrates the complexity of QI implementation and that ‘one size does not fit all’, by reporting the variances in ‘engagement’ scores across different clinical settings and various employment grades. It raises the question that there may be certain components of the PW programme which are more effective or have different impacts depending on each individual context, environment and implementation strategy and these merit further exploration. Practically these findings add value for healthcare QI researchers and practitioners by endorsing (to some degree) the ‘engagement’ claims of PW and its developers. They also provide support for the concept, content and implementation approach of PW as a lean-based, programmatic approach to engaging ward-based teams and improving quality.

Abbreviations

PW:

Productive ward – releasing time to care

QI:

Quality Improvement

T1:

Time one

T2:

Time two

TES::

Total engagement score

UWES 17:

17-item Utrecht work engagement survey

References

  1. Ovretveit J. Contemporary quality improvement. Cadernos De Saúde Pública. 2013;29:424–6.

    Article  PubMed  Google Scholar 

  2. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Q. 2001;79:281–315.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Gill D, Mountford J, Arasaratnam R. What is a quality improvement project? Br J Hosp Med. 2012;73:252–6.

    Article  Google Scholar 

  4. Pressman JL, Wildavsky AB. Implementation: how great expectation in Washington are dashed in Oakland; or, why It’s amazing that Federal Programs Work at all, this being a Saga of the economical Developemnt Admistration as told by two Sypathetic observers who seek to build morals on a foundation. Berkeley: University California Press; 1973.

  5. Havelock RG, Educational Technology Publications. The change Agent’s guide to innovation in education. Englewood Cliffs N. J: Educational Technology Publications; 1973.

    Google Scholar 

  6. Fixsen DL, Naoom SF, Blase KA, Fiedman RM, Wallace F. Implementation research: a synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute; 2005.

    Google Scholar 

  7. Ovretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. 2011;20(Supplement 1):18–23.

    Article  Google Scholar 

  8. Davies H, Powell A, Rushmer R. Why don’t clinicians engage with quality improvement? J Health Serv Res Policy. 2007;12:129–30.

    Article  PubMed  Google Scholar 

  9. Taitz JM, Lee TH, Sequist TD. A framework for engaging physicians in quality and safety. BMJ Qual Saf. 2012;21:722–8.

    Article  PubMed  Google Scholar 

  10. Gollop R, Whitby E, Buchanan D, Ketley D. Influencing sceptical staff to become supporters of service improvement: a qualitative study of doctors’ and managers’ views. Qual Saf Health Care. 2004;13:108–14.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Giesbers AP, Schouteten RL, Poutsma E, van der Heijden BI, van Achterberg T. Feedback provision, nurses’ well-being and quality improvement: towards a conceptual framework. J Nurs Manag. 2015;23:682–91.

    Article  PubMed  Google Scholar 

  12. Siriwardena AN. Engaging clinicians in quality improvement initiatives: art or science? Qual Prim Care. 2009;17:303–5.

    PubMed  Google Scholar 

  13. Dixon-Woods M, McNicol S, Martin G. Ten challenges in improving quality in healthcare: lessons from the Health Foundation's programme evaluations and relevant literature. BMJ Qual Saf. 2012;21:876–84.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Price M, Fitzgerald L, Kinsman L. Quality improvement: the divergent views of managers and clinicians. J Nurs Manag. 2007;15:43–50.

    Article  PubMed  Google Scholar 

  15. van Os A, de Gilder D, van Dyck C, Groenewegen P. Responses to professional identity threat. J Health Org Manage. 2015;29:1011–28.

    Article  Google Scholar 

  16. Reay T, Hinings CR. Managing the rivalry of competing institutional logics. Organ Stud. 2009;30:629–52.

    Article  Google Scholar 

  17. Glouberman S, Mintzberg H. Managing the care of health and the cure of disease--part I: differentiation. Health Care Manag Rev. 2001;26:56–69.

    Article  CAS  Google Scholar 

  18. Bisognano M, Kenney C. Pursuing the triple aim. Jossey Bass Ltd: US; 2012.

    Google Scholar 

  19. NHS Institute. Rapid impact assessment of the productive ward: releasing time to care. Warwick: NHS Institute: NHS Institute. 2011;

  20. Avis K. Releasing time to Care in Saskatchewan:promising signs that programme engages clinicians. Health Quality Council: Saskatoon SK; 2009.

    Google Scholar 

  21. White M, Wells JS, Butterworth T. The transition of a large-scale quality improvement initiative: a bibliometric analysis of the productive Ward: releasing time to care programme. J Clin Nurs. 2014;23:2414–23.

    Article  PubMed  Google Scholar 

  22. White M, Wells JSG, Butterworth T. The impact of a large-scale quality improvement programme on work engagement: preliminary results from a national cross-sectional-survey of the ‘productive Ward’. Int J Nurs Stud. 2014;51:1634–43.

    Article  PubMed  Google Scholar 

  23. Schaufeli WB, Salanova M, González-romá V, Bakker AB. The measurement of engagement and burnout: a two sample confirmatory factor analytic approach. J Happiness Stud. 2002;3:71–92.

    Article  Google Scholar 

  24. Schaufeli WB. What is engagement? In: Truss C, Alfes K, Delbridge R, Shantz A, Soane E, editors. Employee engagement in theory and practice. Oxford: Routledge; 2014. p. 15–36.

    Google Scholar 

  25. Schaufeli WB, Taris TW. A critical review of the job demands-resources model: implications for improving work and health. In: Bauer G, Hämmig O, editors. Bridging occupational, organizational and public health: a Transdisciplinary approach. Dordrecht: Springer Netherlands; 2014. p. 43–68.

    Chapter  Google Scholar 

  26. Simpson M. Engagement at work: a review of the literature. Int J Nurs Stud. 2009;46:1012–24.

    Article  PubMed  Google Scholar 

  27. Freeney YM, Tiernan J. Exploration of the facilitators of and barriers to work engagement in nursing. Int J Nurs Stud. 2009;46:1557–65.

    Article  PubMed  Google Scholar 

  28. Storm K, Rothmann I. A psychometric analysis of the Utrecht work engagement scale in the south African police service. South African J Ind Psychol. 2003;29(4):62–80.

    Google Scholar 

  29. Schaufeli WB, Bakker AB, Salanova M. The measurement of work engagement with a short questionnaire - a cross-national study. Educ Psychol Meas. 2006;66:701–16.

    Article  Google Scholar 

  30. Wefald AJ, Downey RG. Construct dimensionality of engagement and its relation with satisfaction. J Psychol. 2009;143:91–111.

    Article  PubMed  Google Scholar 

  31. Daugherty Biddison EL, Paine L, Murakami P, Herzke C, Weaver SJ. Associations between safety culture and employee engagement over time: a retrospective analysis. BMJ Qual Saf. 2016;25:31–7.

    Article  PubMed  Google Scholar 

  32. NHS Institute, NNRU. Improving healthcare quality at scale and pace, lessons for the productive Ward: releasing time to care programme. NHS Institute: Warwick; 2010.

    Google Scholar 

  33. Bate P, Robert G, Bevan H. The next phase of healthcare improvement: what can we learn from social movements? Qual Saf Health Care. 2004;13:62–6.

    Article  CAS  PubMed  Google Scholar 

  34. Rudge T. Desiring productivity: nary a wasted moment, never a missed step. Nurs Philos. 2013;14:201–11.

    Article  PubMed  Google Scholar 

  35. NHS Institute, NNRU. The productive Ward: releasing time to care learning and impact review. NHS Institute: Warwick; 2010.

    Google Scholar 

  36. Robert G, Morrow E, Maben J, Griffiths P, Callard L. The adoption, local implementation and assimilation into routine nursing practice of a national quality improvement programme: the productive Ward in England. J Clin Nurs. 2011;20:1196–207.

    Article  PubMed  Google Scholar 

  37. Morrow E, Robert G, Maben J, Griffiths P. Implementing large-scale quality improvement: lessons from the productive Ward: releasing time to care. Int J Health Care Qual Assur. 2012;25:237–53.

    Article  PubMed  Google Scholar 

  38. Wells J, White M. The impact of the economic crisis and austerity on the nursing and midwifery professions in the Republic of Ireland – ‘boom’, ‘bust’ and retrenchment. J Res Nurs. 2014;19:562–77.

    Article  Google Scholar 

  39. Ventura M, Salanova M, LLorens S. Professional self-efficacy as a predictor of burnout and engagement: the role of challenge and hindrance demands. J Psychol. 2014;149:1–26.

    Google Scholar 

  40. Aiken L, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288:1987–93.

    Article  Google Scholar 

  41. Aiken LH, Sloane DM, Bruyneel L, Van den Heede K, Griffiths P, Busse R, et al. Nurse staffing and education and hospital mortality in nine European countries: a retrospective observational study. Lancet. 2014;383:1824–30.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Schaufeli WB, Janczur B. Burnout among nurses - a polish-Dutch comparison. J Cross-Cult Psychol. 1994;25:95–113.

    Article  Google Scholar 

  43. Adriaenssens J, De Gucht V, Van Der Doef M, Maes S. Exploring the burden of emergency care: predictors of stress-health outcomes in emergency nurses. J Adv Nurs. 2011;67:1317–28.

    Article  PubMed  Google Scholar 

  44. Dall'Ora C, Griffiths P, Ball J, Simon M, Aiken LH. Association of 12 h shifts and nurses’ job satisfaction, burnout and intention to leave: findings from a cross-sectional study of 12 European countries. BMJ Open. 2015;5:1–7.

    Article  Google Scholar 

  45. Maslach C, Schaufeli WB, Leiter MP. Job burnout. Annu Rev Psychol. 2001;52:397–422.

    Article  CAS  PubMed  Google Scholar 

  46. Shirom A. Vigor as a positive affect at work: conceptualizing vigor, its relations with related constructs, and its antecedents and consequences. Rev Gen Psychol. 2011;15:50–64.

    Article  Google Scholar 

  47. Shuck B. Four emerging perspectives of employee engagement: an integrative literature review. Hum Resour Dev Rev. 2011;10:304–28.

    Article  Google Scholar 

  48. Laschinger HKS, Finegan J. Empowering nurses for work engagement and health in hospital settings. J Nurs Adm. 2005;35:439–49.

    Article  PubMed  Google Scholar 

  49. Alexander JA, Weiner BJ, Shortell SM, Baker LC. Does quality improvement implementation affect hospital quality of care? Hosp Top. 2007;85:3–12.

    Article  PubMed  Google Scholar 

  50. Lee JSY, Akhtar S. Effects of the workplace social context and job content on nurse burnout. Hum Resour Manag. 2011;50:227–45.

    Article  Google Scholar 

  51. Wright S, McSherry W. A systematic literature review of releasing time to care: the productive Ward. J Clin Nurs. 2013;22:1361–71.

    Article  PubMed  Google Scholar 

  52. Bradley DKF, Griffin M. Staff perceptions of a productive community services implementation: a qualitative interview study. Int J Nurs Stud. 2015;52:1052–63.

    Article  PubMed  Google Scholar 

  53. Radnor Z. Implementing lean in health care: making the link between the approach, readiness and sustainability. Int J Ind Eng Manage. 2011;2:1–12.

    Google Scholar 

  54. Glasgow JM, Davies ML, Kaboli PJ. Findings from a national improvement collaborative: are improvements sustained? BMJ Qual Saf. 2012;21:663–9.

    Article  PubMed  Google Scholar 

  55. Ovretveit J, Gustafson D. Evaluation of quality improvement programmes. Qual Saf health Care. 2002;11:270–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Langley A, Denis JL. Beyond evidence: the micropolitics of improvement. BMJ Qual Safety. 2011;20(Suppl 1):43–6.

    Article  Google Scholar 

Download references

Acknowledgements

There has been no institutional, private or corporate financial support associated with this Paper. We thank the ward-based teams in both the PW and control sites for their participation. We would like to acknowledge the statistical advice and guidance provided by Mr. Jim Stack, Macular Pigment Research Group, School of Health Sciences, Waterford Institute of Technology, Waterford, Ireland.

Funding

There has been no institutional, private or corporate financial support associated with this paper.

Availability of data and materials

General Linear and Mixed Linear Model outputs are attached as an additional file.

Author information

Authors and Affiliations

Authors

Contributions

MW conducted all data collection and analyses, wrote the first draft of the manuscript and rewrote new drafts following feedback and input from JW and TB. JW conceptualised and designed the paper, planned the initial level of analyses and gave input on manuscript drafts. TB planned and reviewed the analyses and gave input on all manuscript drafts. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mark White.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained for all phases of this study in the first instance from Waterford Institute of Technology and then for each participating PW and control site, via their overseeing ethics committee (HSE North-East, South-East, West, Mid-West Regional ethics committees and at the National Rehabilitation Hospital, Dublin and the Mercy University Hospital, Cork). All staff members in the intervention and control sites were invited to participate and provided with: a detailed information sheet about the study, a formal invitation to participate and an information guide how to engage and consent with the study, decline or withdraw from the study. Completed and returned questionnaires were accepted as implied consent to participate.

Consent for publication

N/A.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

SPSS output for Healthcare QI and 'Work Engagement'. Linear mixed model analysis outputs. Frequency & Descriptive statistics: Tables a-d, T1 Results: Tables e-l, T2 Results: Tables m-t, T2-T2 Results: Tables u-z2.This file contains the SPSS outputs for the linear mixed model analysis performed on Total Engagement Scores, Vigour Scores, Absorption Scores and Dedication Scores at T1, T2 and T2-T1.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

White, M., Butterworth, T. & Wells, J.S. Healthcare Quality Improvement and ‘work engagement’; concluding results from a national, longitudinal, cross-sectional study of the ‘Productive Ward-Releasing Time to Care’ Programme. BMC Health Serv Res 17, 510 (2017). https://doi.org/10.1186/s12913-017-2446-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-017-2446-2

Keywords