Skip to content

Advertisement

You're viewing the new version of our site. Please leave us feedback.

Learn more

BMC Health Services Research

Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Implementation fidelity of a nurse-led falls prevention program in acute hospitals during the 6-PACK trial

  • Renata T. Morello1Email author,
  • Anna L. Barker1,
  • Darshini R. Ayton1,
  • Fiona Landgren2,
  • Jeannette Kamar3,
  • Keith D. Hill4,
  • Caroline A. Brand1,
  • Catherine Sherrington5,
  • Rory Wolfe1,
  • Sheral Rifat1 and
  • Johannes Stoelwinder1
BMC Health Services ResearchBMC series – open, inclusive and trusted201717:383

https://doi.org/10.1186/s12913-017-2315-z

Received: 29 March 2016

Accepted: 17 May 2017

Published: 2 June 2017

Abstract

Background

When tested in a randomized controlled trial (RCT) of 31,411 patients, the nurse-led 6-PACK falls prevention program did not reduce falls. Poor implementation fidelity (i.e., program not implemented as intended) may explain this result. Despite repeated calls for the examination of implementation fidelity as an essential component of evaluating interventions designed to improve the delivery of care, it has been neglected in prior falls prevention studies. This study examined implementation fidelity of the 6-PACK program during a large multi-site RCT.

Methods

Based on the 6-PACK implementation framework and intervention description, implementation fidelity was examined by quantifying adherence to program components and organizational support. Adherence indicators were: 1) falls-risk tool completion; and for patients classified as high-risk, provision of 2) a ‘Falls alert’ sign; and 3) at least one additional 6-PACK intervention. Organizational support indicators were: 1) provision of resources (executive sponsorship, site clinical leaders and equipment); 2) implementation activities (modification of patient care plans; training; implementation tailoring; audits, reminders and feedback; and provision of data); and 3) program acceptability. Data were collected from daily bedside observation, medical records, resource utilization diaries and nurse surveys.

Results

All seven intervention components were delivered on the 12 intervention wards. Program adherence data were collected from 103,398 observations and medical record audits. The falls-risk tool was completed each day for 75% of patients. Of the 38% of patients classified as high-risk, 79% had a ‘Falls alert’ sign and 63% were provided with at least one additional 6-PACK intervention, as recommended. All hospitals provided the recommended resources and undertook the nine outlined program implementation activities. Most of the nurses surveyed considered program components important for falls prevention.

Conclusions

While implementation fidelity was variable across wards, overall it was found to be acceptable during the RCT. Implementation failure is unlikely to be a key factor for the observed lack of program effectiveness in the 6-PACK trial.

Trial registration

The 6-PACK cluster RCT is registered with the Australian New Zealand Clinical Trials Registry, number ACTRN12611000332921 (29 March 2011).

Keywords

Falls preventionInjury preventionHospitalsProgram evaluationImplementation fidelityQuality improvementProcess evaluationComplex health intervention

Background

In-hospital falls continue to be a major clinical and economic problem for hospitals [1]. They are the most common adverse event in the acute hospital setting [2, 3] and are a source of personal harm [4], increased length of stay (LOS) and increased hospitalisation costs [1]. Consequently, falls prevention is a priority for patient safety activity internationally, with the development of falls prevention best practice guidelines [57], the adoption of in-hospital falls as a quality indicator for hospital performance [810], and the implementation of a variety of in-hospital falls prevention programs. Despite these efforts, there remains limited evidence to support the effectiveness of such initiatives [11, 12]. Poor implementation fidelity (i.e., the program not implemented as intended) may explain the failure of programs to reduce falls in previous trials.

Falls are a complex problem [13], occurring as a result of complex interactions between physiological, behavioral and environmental factors [14]. This complexity is compounded in the inpatient setting due to the acutely ill nature of many patients, their short LOS and the dynamic nature of the acute hospital ward environment [15]. Therefore, prevention programs are commonly complex, dynamic and patient-orientated, to ensure adequate targeting to the needs of the individual [11]. However, the challenge with testing the effectiveness of such programs is that they are often reliant on the extent to which they have been implemented in practice [16], often referred to in the literature as implementation of intervention fidelity.

Despite repeated calls for the examination of implementation fidelity as an essential component of undertaking trials to evaluate interventions designed to improve the delivery of care [17], it has been infrequently examined in prior falls prevention research [18]. Implementation strategies and levels of implementation fidelity are seldom reported [15, 18]. Only a small number of studies have examined program adherence [19]. The majority of these report less than ideal implementation; with low levels of program adherence [2022]. Other studies questioned whether limited practice change hindered the program’s effectiveness [23, 24], however they did not report adherence. Therefore the persistent problem of falls in acute hospitals may be due to sub-optimal adoption of falls prevention practices.

Examining implementation fidelity is particularly relevant for complex interventions in multi-center trials, where the same intervention may be implemented and received in different ways across sites [17, 25]. It enables researchers to make valid conclusions about an intervention’s effectiveness or ineffectiveness, ensuring unsuccessful outcomes reflect failure of the intervention and not failure to implement it as intended. A number of frameworks have been developed to examine implementation fidelity [2630]. The framework proposed by Carroll et al. conceptualizes implementation fidelity as being multifaceted, encompassing both the intervention and its delivery [27]. This framework proposes two key elements: 1) program adherence; and 2) the degree to which adherence was influenced by factors that may have influenced the delivery process [27]. Program adherence, “the bottom-line measurement of implementation fidelity” [27] incorporates program content, dose and coverage. Factors that influence program delivery include intervention complexity, implementation strategies, participant responsiveness and quality of delivery.

Case study: the 6-PACK program

This study concerns the implementation fidelity of the 6-PACK program, a targeted nurse-led multi-component falls prevention program designed specifically for acute hospital wards [29]. It incorporates a nine-item falls-risk tool (the TNH-STRATIFY) [28] and delivery of up to six interventions to patients classified as high-risk based on the tool. However, when tested in a multi-center cluster randomized controlled trial (RCT) (n = 31,411 patients), the 6-PACK program did not reduce falls (IRR = 1 · 04, 95% CI, 0 · 78–1 · 37; P = 0 · 796) or fall injuries (IRR, 0 · 96; 95% CI, 0 · 72–1 · 27; P = 0 · 766) [31]. The primary trial explored reasons for no effect and provided evidence that contamination and confounding were highly unlikely [31]. Given the complexity of falls prevention in the acute ward setting, trial results may have been influenced by the degree to which the program was implemented into practice. The aim of this study was to examine the implementation fidelity— program adherence and organizational support—of the 6-PACK falls prevention program during a cluster RCT [31] to assist with the interpretation of trial results.

Methods

Study setting and participants

The methodology and findings of the RCT, conducted between January 2012 and April 2013, are described elsewhere [29, 31]. Intervention wards that participated in the RCT were included in this study—12 acute wards (medical or surgical) from six public hospitals (both metropolitan and regional teaching hospitals) in two states in Australia (Victoria and New South Wales). Participating wards comprised of general medical (n = 3), general surgical (n = 3), general medical short stay (n = 1), specialist medical (n = 4) and specialist surgical (n = 1). All patients admitted to participating wards were included in this study.

The RCT was a pragmatic trial with the 6-PACK program implemented by ward nursing staff as part of usual care practice. Data from a sample of intervention ward nurses and senior managers from each hospital were used in this evaluation. Sampling procedures are described in the data collection section below.

The 6-PACK falls prevention program

The 6-PACK program (Fig. 1) was implemented onto the 12 participating intervention wards during the cluster RCT. Nurses were required to complete the falls-risk tool for each patient on their daily care plan. For patients classified as high falls risk, nurses were to select, document and apply a ‘Falls alert’ sign above the patient’s bed and one or more of the remaining 6-PACK interventions (supervision while in the bathroom; use of a low-low bed, lowered as close as possible to the floor; ensuring their walking aid is within reach; establishment of a toileting regime; and use of a bed/chair alarm when they are in the bed/chair). The selection of 6-PACK interventions were based on the nurse’s clinical judgement of the patient’s needs.
Fig. 1

The 6-PACK falls prevention program

The implementation of the 6-PACK program was managed at a central level (by the research team) and tailored locally by participating hospitals. It was informed by two mixed-methods studies: 1) perceived acceptability of the 6-PACK program (suitability, practicality and benefits) [32] and 2) perceived barriers to, and enablers of, implementation of the 6-PACK program [33]. It incorporated a number of implementation strategies (Table 1), that focused on the training and support of a hospital appointed part-time site clinical leader to facilitate program implementation on intervention wards.
Table 1

6-PACK implementation protocol (strategies for implementation of the 6-PACK program on intervention wards)

 

Description

Resources

Hospital

Appointment of an executive sponsor (type not specified)

Provision of 6-PACK falls prevention program equipmenta:

• ‘Falls alert’ signsb. One for each inpatient ward bed.

• Low-low beds (to be able to be lowered 250 mm from the floor level or lower). A minimum of 1 low-low bed to 3 standard beds on medical wards and 1 low-low bed to 10 standard beds on surgical wards.

• Bed/chair alarms. Three on medical wards and one on surgical wards.

Appointment of a part-time falls prevention site clinical leaderc for the 12 month study period. Recommended 0.1 Full Time Equivalent (FTE) for each intervention ward.

Ward

Appointment of two nurses from current permanent staff to act as ward champions to support the site clinical leader and the local assimilation of the 6-PACK program.

Activities

Hospital

Integration of the 6-PACK program documentation (risk tool and interventions) into the daily care-plan

Site clinical leader

Attend small interactive face-face group training sessions provided by the research team which included training on the use of the 6-PACK program, leadership, education and change management and provision of a implementation and training guide (two one day sessions, one prior to program implementation and one refresher 6-months post implementation).

Develop a ‘ReadySetGo’ 6-PACK implementation plan tailored to the hospital and participating wards.

Deliver small interactive group training sessions to nurses on intervention wards on the use of the 6-PACK program and documentation. Training sessions were based on material provided by the research team but tailored to the needs of the ward by the site clinical leader. A review of site specific case studies were also encouraged. Minimum of two training sessions to be delivered per ward.

Attend monthly site clinical leader network teleconference meetings with the research team to discuss implementation progress and trouble-shoot implementation challenges (approximately 1 h in length).

Communicate to ward staff (ward nurses, ward champions and Nurse Unit managers) monthly to provide data on fall event outcomes, risk assessment completion and program adherence.

Undertake 15-min ward ‘walk rounds’ with ward staff and champions that utilize bedside audit, reminders and feedback:

• Weekly for the first month;

• Fortnightly for the next 5-months; and

• Monthly for the final 6-months.

Wards

Provide monthly data extract reports from the hospital incident reporting database for all participating wards as part of the feedback loop.

Ward nurses attend 6-PACK program education sessions run by the site clinical leader.

Ward champions/Nurse Unit managers undertake monthly compliance audits on the use of 6-PACK documentation and nurse’s adherence to the 6-PACK program.

aDue to local hospital policies and equipment purchase procedures the make and model of the falls prevention equipment was at the discretion of the hospital. Recommendations were provided to hospitals based on the successful program at The Northern Hospital

b Sign holders for the ‘Falls alert’ signs were recommended but not required

cSite clinical leaders were appointed by the hospital, at the discretion of the hospital. Recommendations were provided to hospitals regarding site clinical leader FTE and staff experience, knowledge and skills; however, these were not absolute

Site clinical leaders were provided with a comprehensive 6-PACK implementation, training and monitoring guide. Throughout the trial, support was also provided by the research team (program designer [JK], change management facilitator [FL], project manager [RM] and chief investigator [AB]) on a needs basis via telephone, email and site visits. Need was determined by the site clinical leader or the research team and based primarily on ward program adherence. This included review of complex fall cases, additional training of site clinical leaders on the program and implementation strategies such as audit and feedback and trouble-shooting problems arising with equipment use. Further detail on the 6-PACK program and its implementation can be found elsewhere [29, 31].

Outcome indicators

An adapted version of the framework developed by Carroll and colleagues was used to assess implementation fidelity. Outcome indicators and data collection sources are summarized in Table 2.
Table 2

Outcome indicators and data collection sources for examination of implementation fidelity

Program adherence: the adherence of nursing staff to the use of the 6-PACK program

Factor

Research question

Data source

Content

Did nurses use the individual 6-PACK interventions as designed by the program designers?

Daily structured observation of patient’s bedside

Daily audit of patient medical records

Frequency and duration (dose)

Did nurses deliver the 6-PACK program as often and for as long as planned, based on targets outlined in Box 1?

Daily structured observation of patient’s bedside

Daily audit of patient medical records

Coverage

Was the program delivered to all appropriate participants (participant selection)?

As this was a ward based intervention and all patients admitted to participating wards were recruited as part of this study, program coverage was not specifically examined

Organizational support: factors influencing program delivery by nursing staff

Factor

Research question

Data source

Hospital and ward resources

Did hospitals provide the recommended 6-PACK program resources (outlined in Table 1) to allow the intervention wards to carry out the program successfully?

Daily structured observation of patient’s bedside

Daily audit of patient medical records

Weekly site clinical leader resource utilization diaries

Resource and implementation activity compliance log

Implementation activities

What implementation strategies (outlined in Table 1) were used by participating hospitals to support, optimize and facilitate the delivery of the 6-PACK program into routine clinical practice?

Daily structured observation of patient’s bedside

Daily audit of patient medical records

Weekly site clinical leader resource utilization diaries

Resource and implementation activity compliance log

Attendance at training sessions and network meetings

Staff perceptions

How accepting were staff to the implementation of a new falls prevention program? Including staff perceptions on:

a) The need for the program

b) The alignment of the program with the existing care models and the needs of the patients

Nurse surveys

Nursing practice adherence indicators (program dose) were: 1) falls-risk tool completion each day of admission; and for patients classified as high-risk provision of 2) a ‘Falls alert’ sign; and 3) a sign and at least one additional 6-PACK intervention. As this was a ward-based intervention and all patients admitted to participating wards were recruited as part of this study, program coverage was not specifically examined. Organizational support indicators were: 1) provision of hospital and ward resources (executive sponsorship, site clinical leaders and equipment); 2) implementation activities (modification of patient care plans; training; implementation tailoring; audits, reminders and feedback; and provision of data); and 3) program acceptability among nursing staff.

Data collection

Data were prospectively collected. Data on program adherence were collected through daily audit of patient care plans and structured bedside observation. Data collection was undertaken by members of the research team (site data collectors) using a standardized tool. Budget constraints meant these data were only able to be collected from the commencement of the cluster RCT until December 2013. Weekly site clinical leader resource utilization diaries detailed time spent on program implementation activities. A member of the research team recorded attendance to site clinical leader training sessions and meetings and kept a log on the compliance of hospitals and wards to resource commitment, care-plan integration and data provision. Nurse perceptions were explored as part of a staff knowledge, attitudes and practices survey, developed by the research team. Nineteen items in the staff survey related to program acceptability scored on a 5-point Likert scale (1 = strongly disagree; 5 = strongly agree). The survey was administered prior to program implementation. The researcher attended handover sessions or designated ward meetings to distribute the survey. Nurses that had worked on an intervention ward for at least 7.5 h per week in the 2 months prior were invited to complete the survey. Completed surveys were placed in sealed boxes collected by the researcher at the end of the 2-week dissemination period. To avoid hierarchical coercion, Nurse Unit Managers were not involved in the distribution of the surveys. The number of surveys distributed to, and returned from, each ward was recorded.

Data analysis

Descriptive statistics were used to profile the study sample, program adherence and staff perceptions. Adherence to program content was categorized into three levels: adherent, partially adherent and non-adherent. Program dose was defined as a proportion. Data on hospital and ward resources and implementation activities were measured against expectations and/or recommendations and categorized into three level of adherence: adherent, partially adherent and non-adherent. Items were only coded as adherent if the recommendation was completely met. Analyses were undertaken using Stata MP v13 statistical software. Data visualizations of staff survey item responses were created using Tableau Desktop v9.0.

Results

During the RCT there were 22,670 admissions to the intervention wards representing 17,698 individual participants, as some patients were admitted more than once.

6-PACK program adherence

Data on program use were collected from 103,398 daily observations and medical record audits (Table 3). All participating wards were adherent to program content. Overall, 75% of patients had their falls-risk tool completed each day of their admission (range: 52–90%). Of patients classified as high-risk, 79% had a ‘Falls alert’ sign placed above the bed (range: 61–90%) and 63% were provided with a sign and at least one additional 6-PACK intervention (range: 48–74%). Use of 6-PACK program components increased during the RCT period. By the 10th month of data collection 79% of high-risk patients had a ‘Falls alert’ sign plus at least one other intervention (range: 61–95%). Substantial fluctuations in program adherence over the study period and between hospitals were noted (Additional file 1).
Table 3

6-PACK program adherence

 

Total

(n = 103, 398)

Hospital 1: Medical

(n = 8846)

Hospital 1: Surgical

(n = 9045)

Hospital 2: Medical

(n = 8915)

Hospital 2: Surgical

(n = 8076)

Hospital 3: Medical 1

(n = 9231)

Hospital 3: Medical 2

(n = 9933)

Hospital 3: Surgical

(n = 7630)

Hospital 4: Medical

(n = 10,617)

Hospital 4: Surgical

(n = 8358)

Hospital 5: Medical

(n = 9671)

Hospital 6: Medical 1

(n = 6657)

Hospital 6: Medical 2

(n = 6419)

Program content

Program content

Program dosage

Falls-risk score

 Daily completion of falls-risk score, n (%)

77,592

(75)

7842

(89)

8161

(90)

7123

(80)

5672

(70)

7625

(83)

7050

(71)

5795

(76)

7897

(74)

4321

(52)

7508

(78)

3748

(56)

4850

(76)

Patient identified as high risk, n (%)

29,330

(38)

3998

(51)

1818

(22)

2651

(37)

2344

(41)

4044

(53)

3221

(46)

669

(12)

2112

(27)

1063

(25)

4309

(57)

1181

(32)

1920

(40)

‘Falls alert’ sign

 ‘Falls alert’ sign observed to be in place, n (%)

23,136

(79)

3425

(86)

1542

(85)

2385

(90)

2049

(87)

3354

(83)

2456

(76)

508

(76)

1626

(77)

737

(69)

2934

(68)

718

(61)

1402

(73)

‘Falls alert’ sign + 1

 ‘Falls alert’ sign and at least one additional 6-PACK strategy, n (%)

18,445

(63)

2907

(73)

1327

(73)

1972

(74)

1308

(56)

2782

(69)

2096

(65)

407

(61)

1383

(65)

555

(52)

2067

(48)

615

(52)

1026

(53)

Adherent

Program adherence: the adherence of staff to the use of the 6-PACK program. Base on program content and program dosage

Program content: Did nurses use the individual 6-PACK interventions as designed by the program designers?

Program dosage (frequency and duration of program use): Did nurses deliver the 6-PACK program as often and for as long as planned, based on three adherence targets: 1) falls-risk tool completion each day of admission; and for patients classified as high risk provision of 2) a ‘Falls alert’ sign; and 3) a sign and at least one additional 6-PACK intervention strategy

Organizational support for program delivery

Table 4 summarizes data on the organizational support provided by hospitals during the trial period.
Table 4

Organizational support for the implementation of the 6-PACK program: resources and implementation activities

 

Hospital 1

Hospital 2

Hospital 3

Hospital 4

Hospital 5

Hospital 6

Hospital and ward resources

      

Appointment of an executive sponsor

Provision of 6-PACK equipment as recommendeda

 ‘Falls alert’ sign

 Low-low beds

 Bed/chair alarms

Site Clinical Leader, FTE per hospital

0.2

0.2

0.1

0.4

0.1

0.1

Appointment of 2 ward champions

Implementation activities

Hospital activities

 6-PACK program integrated into daily care-plan

Site clinical leader activities

 Attended at SCL training sessions

 Developed of ‘ReadySetGo’ plan

 Delivered ward staff education and training

 Attended monthly SCL network meetings

ϕ

ϕ

ϕ

ϕ

 Monthly communication to wards

ϕ

Undertake 15-min ward ‘walk rounds’

  Weekly for the first month

ϕ

ϕ

ϕ

ϕ

ϕ

ϕ

  Fortnightly for the next 5-months

ϕ

ϕ

ϕ

ϕ

ϕ

ϕ

  Monthly for the final 6-months

ϕ

ϕ

ϕ

ϕ

Ward activities

 Monthly data extract provided

 Staff attended education sessions

 Monthly compliance audits completed

Φ

ϕ

ϕ

ϕ

ϕ

Adherent

ϕ Partially adherent

aRecommended allocation of equipment: ‘Falls alert’ signs: 1 per patient hospital bed; Low-low beds: a minimum of 1 low-low to 3 standard beds on medical wards and 1 low-low to 10 standard beds on surgical wards; Bed/chair alarms: three on medical wards and one on surgical wards

FTE Full time equivalent

Hospital and ward resources

All hospitals provided ‘Falls alert’ signs, bed/chair alarms and low-low beds as recommended. Five of the six hospitals implemented the recommended alarm (Hospital 3 opted for an alternative model). All wards implemented low-low beds that lowered to 250 mm from floor level (as per trial criteria). However, only two of the six hospitals implemented beds that lowered to 100 mm from the floor as recommended.

All hospitals appointed an executive sponsor to support program implementation during the RCT. There was general compliance with the appointment of a site clinical leader. However, three of the six hospitals had less than the recommended 0.1 full time equivalent (FTE) per intervention ward. Hospital 4 provided the greatest site clinical leader resources (0.2 FTE per intervention ward) and Hospital 3 supported the least (0.03 FTE per intervention ward). Hospital 4 experienced a change in their site clinical leader 4 months into program implementation. All hospital wards nominated the recommended number of ward champions, although Hospital 2 experienced a number of staff changes that meant the role was vacant for many months.

Implementation activities

All hospitals integrated 6-PACK documentation into the patient care-plan. Each site clinical leader attended two formal face-face training sessions and were provided with the 6-PACK implementation guide. All site clinical leaders were adherent with implementation activities (Table 4), with variability noted in the time allocated to each activity (Additional file 2). Some site clinical leaders spent substantial time undertaking activities not originally envisaged as part of the program implementation, such as administration, email communication and equipment review (range 18–46% of allocated time over the 12-month study period).

Staff perceptions

A total of 208 intervention ward nurses (66% response rate) completed the survey. The majority of respondents were qualified registered nurses (85%) and had worked on the participating intervention wards for more than 12 months (68%) with at least five shifts per week (51%). Results have been summarized in Fig. 2. Nurses considered the program components to be important for falls prevention. Falls risk assessment tools were considered useful (74%, aggregate of ‘strongly agree’ and ‘agree’), as were ‘Falls alert’ signs above the bed (87%, aggregate of ‘strongly agree’ and ‘agree’) and low-low beds (83%, aggregate of ‘strongly agree’ and ‘agree’). Survey data revealed that nurses’ believed it was their responsibility to assess the falls risk each shift (87%, aggregate of ‘strongly agree’ and ‘agree’) and implement prevention strategies for patients identified as high falls risk (92%, aggregate of ‘strongly agree’ and ‘agree’).
Fig. 2

Factors influencing implementation of the 6-PACK program: Staff acceptability of the program

Nurse responses toward the prevention of falls were less positive. Almost one third agreed that falls cannot be avoided in older people (30%, aggregate of ‘strongly agree’ and ‘agree’). A degree of uncertainty amongst nurses for some items were noted, for example, more than one third neither agreed nor disagreed that there was strong leadership for falls prevention on their ward (38%, ‘neutral’). Moderate variation in staff responsiveness on some survey items between hospitals were observed (Additional file 3).

Discussion

Overall, an acceptable level of implementation fidelity was observed, though variation was noted between wards and over time. This affirms previously published conclusions, that implementation failure was unlikely to be a key factor for the finding of no-effect in the 6-PACK trial [31].

Program adherence by hospital ward varied across and within hospitals, with adherence levels ranging from 48 to 90%. Substantial fluctuations over time were also noted, with many wards experiencing an increase in adherence as the program continued beyond the initial months. It is likely that these variations are due to a myriad of factors including observed differences in site clinical leaders, their resources and the time allocated to individual implementation activities. The heterogeneity of included wards and patient risk profiles make it difficult to explore these in detail. Nurses have a key role in the implementation of falls prevention programs such as 6-PACK. They care for patients in hospital 24 h a day, 7 days a week, 52 weeks a year. Generally high levels of acceptability of program components was observed among nursing staff, although some concepts demonstrated a proportion of staff with knowledge or attitude limitations that may have limited implementation success.

The completion of the falls-risk tool and provision of falls prevention interventions observed in this study compares favorably with routine practice (e.g. 71% of patients had their risk score updated daily during the RCT, compared with only 13% of patients having their risk score updated at least once during their ward admission in an audit of usual falls prevention practices) [34]. Nearly two-thirds of all high-risk patients received the 6-PACK program as defined in the protocol. While, there may be scope for improvement in program adherence on some participating wards, the characteristics of patients in acute wards (short LOS, complex medical conditions, acute medical instability), mean it may not be realistic to expect 100% adherence. Few previous studies have evaluated adherence to falls prevention interventions in acute hospitals. Therefore, the ability to benchmark the adherence levels observed in this study to prior literature is restricted, and it is difficult to determine what levels of program adherence are actually required to elicit meaningful changes in outcomes.

Methodological discussion

When interpreting these results, the following methodological limitations should be considered. The provision of 6-PACK interventions was reliant on the daily completion of the falls-risk tool and accurate classification of patients at high-risk. In addition, it does not necessarily reflect optimum practice (the provision of a ‘Falls alert’ sign and all relevant 6-PACK strategies for those identified as high-risk) or allow for the examination of the appropriate use of strategies. The authors acknowledge that qualitative data would provide additional insights into the factors influencing implementation fidelity, yet it was beyond the scope of this study. Considerable but different barriers to implementation appeared to be present in some hospitals/wards during the 6-PACK trial. Further exploration of organizational characteristics, existing infrastructure and surrounding social structures, ward location, the availability, skills and leadership qualities of the local implementation team and other competing priorities experienced by staff should be considered.

Conclusions

In-hospital falls represent a complex problem that occurs within the complex environment of a hospital ward, often to individuals with complex care needs. Therefore, it is inevitable that examining implementation fidelity is also challenging. Currently there is limited consensus on how best to define and measure implementation fidelity [35]. The framework used in this study provided systematic guidance to measurement, however, it does not provide any direction on how to explore the relationships between its key elements [36].

In this study, while program adherence was not optimal at some participating hospitals, we considered overall implementation fidelity sufficient in the 6-PACK trial. Therefore, implementation failure is unlikely to be a key factor for the observed lack of program effectiveness on fall and fall injuries. This study provides valuable learnings for the development and implementation of future falls prevention programs or other nurse-led safety programs. Given the challenges and costs involved with the implementation of complex health interventions, such as falls prevention programs, examination of implementation fidelity should be an integral part of any evaluation of an intervention’s effectiveness.

Abbreviations

FTE: 

Full time equivalent

LOS: 

Length of stay

RCT: 

Randomized controlled trial

Declarations

Acknowledgements

We acknowledge the Injury Prevention Unit, The Northern Hospital, Northern Health, Melbourne, Australia who developed the 6-PACK program, the late Damien Jolley who provided substantial input to the design of the 6-PACK project, and all investigators involved in the design and application for funding of the project. The study could not have been completed without the collaboration and support from the participating hospitals, site clinical leaders and nurses.

Funding

This project was funded by the National Health and Medical Research Council (NHMRC), Australia (APP1007627). A Barker’s salary was funded by a Career Development Fellowship from the NHMRC (APP1067236). R Morello’s salary was supported by a postgraduate scholarship from the NHMRC (APP1055604). C Sherrington’s salary was funded by a Senior Research Fellowship from the NHMRC (APP632929).

Availability of data and materials

Informed consent was not obtained from participants for the publication of the datasets generated and analyzed during the current study. Therefore to ensure the participants’ rights to privacy and to protect their identity, the authors would prefer the data not be made publicly available but available from the corresponding author on reasonable request.

Authors’ contributions

RM led the drafting of all sections of this manuscript in conjunction with AB. RW provided advice on the statistical analysis and interpretation of data. SR assisted with the data management and cleaning. RM, AB, DR, FL, JK, KH, CB, CS, RW, SR and JS allreviewed and revised the manuscript critically for important intellectual content and approved the final version of the manuscript to be published.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

This study received multicenter ethics approval from the Monash University Human Research Ethics Committee (Project Number: CF11/0229–2011000072). Site-specific ethics and research governance approval were also obtained from local ethics committees at all participating hospitals. As this was part of a cluster randomized controlled trial, wards not patients were recruited to this study. A waiver of individual patient consent was obtained and the presented data are anonymized so that risk of identification is low.

Transparency statement

The lead author (RM) affirms that the manuscript is an honest, accurate and transparent account of the study being reported; no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Health Services Research Unit, Department of Epidemiology and Preventive Medicine, Monash University
(2)
Project Health
(3)
The Northern Hospital, Northern Health
(4)
School of Physiotherapy and Exercise Science, Curtin University
(5)
The George Institute for Global Health, Sydney Medical School, The University of Sydney

References

  1. Morello RT, Barker AL, Watts JJ, Haines T, Zavarsek SS, Hill KD, et al. The extra resource burden of in-hospital falls: a cost of falls study. Med J Aust. 2015;203(9):367.View ArticlePubMedGoogle Scholar
  2. Briggs K, Steel K. Falls in a hospital setting. J Am Geriatr Soc. 2007;55(10):1676.View ArticlePubMedGoogle Scholar
  3. Hitcho EB, Krauss MJ, Birge S, Claiborne Dunagan W, Fischer I, Johnson S, et al. Characteristics and circumstances of falls in a hospital setting: a prospective analysis. J Gen Intern Med. 2004;19(7):732–9. Pubmed Central PMCID: 1492485.View ArticlePubMedPubMed CentralGoogle Scholar
  4. Hill K, Vu M, Walsh W. Falls in the acute hospital setting – impact on resource utilisation. Aust Health Rev. 2007;31(3):471–7.View ArticlePubMedGoogle Scholar
  5. Australian Commission on Safety and Quality in Health Care. Preventing falls and harm from falls in older people: best practice guidelines for Australian hospitals. Sydney: Australian Commission on Safety and Quality in Healthcare; 2009.Google Scholar
  6. National Institute for Health and Care Excellence. Falls: assessment and prevention of falls in older people. 2013. Available from: https://www.nice.org.uk/guidance/cg161. 15th Apr 2014.
  7. American Geriatrics Society/British Geriatrics Society. AGS/BGS clinical practice guideline: prevention of falls in older persons. American Geriatrics Society; 2010. http://www.americangeriatrics.org/files/documents/health_care_pros/JAGS.Falls.Guidelines.pdf.
  8. Center for Performance Sciences. Acute care implementation manual. Maryland: Centre for Performance Sciences; 2006.Google Scholar
  9. The Australian Council on Healthcare Standards. National safety and quality health service standards program. NSW: The Australian Council on Healthcare Standards; 2012.Google Scholar
  10. Canadian Institute for Health Information. Canadian hospital reporting project technical notes – clinical indicators. Ottawa: CIHI; 2013.Google Scholar
  11. Cameron ID, Murray GR, Gillespie LD, Robertson MC, Hill KD, Cumming RG, et al. Interventions for preventing falls in older people in nursing care facilities and hospitals. Cochrane Database Syst Rev. 2010;1:CD005465.Google Scholar
  12. DiBardino D, Cohen ER, Didwania A. Meta-analysis: multidisciplinary fall prevention strategies in the acute care inpatient population. J Hosp Med. 2012;7(6):497–503.View ArticlePubMedGoogle Scholar
  13. Evans D, Hodgkinson B, Lambert L, Wood J. Falls risk factors in the hospital setting: a systematic review. Int J Nurs Pract. 2001;7(1):38–45. Epub 2002/01/29. Eng.View ArticlePubMedGoogle Scholar
  14. Coussement J, De Paepe L, Schwendimann R, Denhaerynck K, Dejaeger E, Milisen K. Interventions for preventing falls in acute- and chronic-care hospitals: a systematic review and meta-analysis. J Am Geriatr Soc. 2008;56(1):29–36.View ArticlePubMedGoogle Scholar
  15. Miake-Lye IM, Hempel S, Ganz DA, Shekelle PG. Inpatient fall prevention programs as a patient safety strategy: a systematic review. Ann Intern Med. 2013;158(5 Pt 2):390–6.View ArticlePubMedGoogle Scholar
  16. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.View ArticlePubMedGoogle Scholar
  17. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, et al. Developing and evaluating complex interventions: the new medical research council guidance. BMJ. 2008;337:a1655. Pubmed Central PMCID: 2769032.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Hempel S, Newberry S, Wang Z, Booth M, Shanman R, Johnsen B, et al. Hospital fall prevention: a systematic review of implementation, components, adherence, and effectiveness. J Am Geriatr Soc. 2013;61(4):483–94. Pubmed Central PMCID: 3670303.View ArticlePubMedPubMed CentralGoogle Scholar
  19. Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, et al. Designing and evaluating complex interventions to improve health care. BMJ. 2007;334(7591):455–9. Pubmed Central PMCID: 1808182.View ArticlePubMedPubMed CentralGoogle Scholar
  20. Dempsey J. Falls prevention revisited: a call for a new approach. J Clin Nurs. 2004;13(4):479–85.View ArticlePubMedGoogle Scholar
  21. Bakarich A, McMillan V, Prosser R. The effect of a nursing intervention on the incidence of older patient falls. Aust J Adv Nurs. 1997;15(1):26–31.PubMedGoogle Scholar
  22. Schwendimann R, Buhler H, De Geest S, Milisen K. Falls and consequent injuries in hospitalized patients: effects of an interdisciplinary falls prevention program. BMC Health Serv Res. 2006;6:69. Pubmed Central PMCID: 1534028.View ArticlePubMedPubMed CentralGoogle Scholar
  23. Williams TA, King G, Hill AM, Rajagopal M, Barnes T, Basu A, et al. Evaluation of a falls prevention programme in an acute tertiary care hospital. J Clin Nurs. 2007;16(2):316–24.View ArticlePubMedGoogle Scholar
  24. Cumming RG, Sherrington C, Lord SR, Simpson JM, Vogler C, Cameron ID, et al. Cluster randomised trial of a targeted multifactorial intervention to prevent falls among older people in hospital. BMJ. 2008;336(7647):758–60. Pubmed Central PMCID: 2287238.View ArticlePubMedPubMed CentralGoogle Scholar
  25. Oakley A, Strange V, Bonell C, Allen E, Stephenson J, Team RS. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6. Pubmed Central PMCID: 1370978.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Mihalic SF, Fagan AA, Argamaso S. Implementing the LifeSkills Training drug prevention program: factors related to implementation fidelity. Implement Sci. 2008;3:5. Pubmed Central PMCID: 2265741.View ArticlePubMedPubMed CentralGoogle Scholar
  27. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40. Pubmed Central PMCID: 2213686.View ArticlePubMedPubMed CentralGoogle Scholar
  28. Barker A, Kamar J, Graco M, Lawlor V, Hill K. Adding value to the STRATIFY falls risk assessment in acute hospitals. J Adv Nurs. 2011;67(2):450–7.View ArticlePubMedGoogle Scholar
  29. Barker A, Brand C, Haines T, Hill K, Brauer S, Jolley D, et al. The 6-PACK programme to decrease fall-related injuries in acute hospitals: protocol for a cluster randomised controlled trial. Inj Prev. 2011;17(4):e5.View ArticlePubMedGoogle Scholar
  30. Hendrich A, Chow MP, Skierczynski BA, Lu Z. A 36-hospital time and motion study: how do medical-surgical nurses spend their time? Perm J. 2008;12(3):25–34. Pubmed Central PMCID: 3037121.View ArticlePubMedPubMed CentralGoogle Scholar
  31. Barker AL, Morello RT, Wolfe R, Brand CA, Haines TP, Hill KD, et al. 6-PACK programme to decrease fall injuries in acute hospitals: cluster randomised controlled trial. BMJ. 2016;352:h6781. Pubmed Central PMCID: 4727091.View ArticlePubMedPubMed CentralGoogle Scholar
  32. Barker AL. Morello RT. Ayton DR. Hill KD. Brand CA, Livingston PM, Botti M. Acceptability of the 6-PACK falls prevention program: a pre-implementation study in hospitals participating in a cluster randomised controlled trial. PLOSone. 2017. https://doi.org/10.1371/journal.pone.0172005.
  33. Ayton DR. Barker AL. Morello RT. Brand CA. Talevski J. Landgren FS. et al. Barriers and enablers to implementation of the 6-PACK falls prevention program: a pre-implmentation study in acute hospitals participating in a cluster randomised controlled trial. PLOS one. 2017. https://doi.org/10.1371/journal.pone.0171932.
  34. Centre of Research Excellence in Patient Safety. An evaluation of the preventing falls and harm from falls in older people best practice guidelines for Australian hospitals. Melbourne: Monash University; 2012.Google Scholar
  35. Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health. 2014;68(2):101–2. Pubmed Central PMCID: 3892708.View ArticlePubMedGoogle Scholar
  36. Hasson H, Gilbert-Ouimet M, Baril-Gingras G, Brisson C, Vezina M, Bourbonnais R, et al. Implementation of an organizational-level intervention on the psychosocial environment of work: comparison of managers’ and employees’ views. J Occup Environ Med. 2012;54(1):85–91.View ArticlePubMedGoogle Scholar

Copyright

© The Author(s). 2017

Advertisement