- Research article
- Open Access
- Open Peer Review
Evaluation of paediatric nursing-sensitive outcomes in an Australian population using linked administrative hospital data
BMC Health Services Researchvolume 13, Article number: 396 (2013)
Research into nursing-sensitive outcomes using administrative health data has focussed on hospitalised adults. However, we developed algorithms for the identification of 13 paediatric nursing-sensitive outcomes, which we seek to examine for clinical utility. The aims were to determine the rates of paediatric nursing-sensitive outcomes in a Western Australian hospital and ascertain sociodemographic and clinical characteristics associated with a greater risk of developing nursing-sensitive outcomes in hospitalised children.
A retrospective cohort study used linked administrative data of all Western Australian children ≤18 years admitted to the only tertiary paediatric hospital in Perth between 1999 and 2009. Rates per 1,000 hospital separations and per 10,000 patient days were calculated for the following nursing-sensitive outcomes: lower respiratory tract infection (LRTI), gastrointestinal (GI) infection, pneumonia, sepsis, arrest/shock/respiratory failure, central nervous system complication, central venous line infection, infectious disease, pressure ulcer, failure to rescue, surgical wound infection, physiologic/metabolic derangement, and postoperative cardiopulmonary complications. Poisson multiple regression models were fitted to estimate rate ratios (RR) and 95% confidence intervals (CI) for suspected risk factors.
Linked records of 129,719 hospital separations were analysed. Rates ranged from 0.5/1,000 for pressure ulcer to 14.0/1,000 hospital separations for GI infections. Age was significantly associated with the risk of a nursing-sensitive outcome: compared with adolescents, toddlers had greater risk of GI infection (RR 9.89; 95% CI 6.24, 15.69); infants had 7.74 times greater risk of LRTI (95% CI 5.11, 11.75), while neonates had lower risks for sepsis (RR 0.26; 95% CI 0.08, 0.90) and physiologic/metabolic derangement (RR 0.12; 95% CI 0.04, 0.35). The risk of surgical wound infection was 7.78 times greater (95% CI 5.10, 11.86) for emergency admissions than elective admissions.
Seven of the 13 defined nursing-sensitive outcomes occurred with sufficient frequency (>100 events over the 10 year study period) to be potentially useful for monitoring the quality of nursing care. These nursing-sensitive outcomes are: LRTI, GI infection, pneumonia, surgical wound infection, physiologic/metabolic derangement, sepsis and postoperative cardiopulmonary complications. When used for quality improvement or to benchmark with other agencies, data need to be adjusted for, or stratified by age and admission type, to ensure equitable comparisons.
There is growing interest in Australia in developing clinical indicators to measure the quality of health care [1, 2]. As nurses make up more than 50% of the Australian health workforce,  it is not surprising that so-called 'nursing-sensitive indicators’ have been proposed. These indicators measure the structures, processes and outcomes of nursing care,  and the latter of which are defined as “changes in health status upon which nursing care has had a direct influence” [5, 6]. Algorithms for detecting nursing-sensitive outcomes in administrative hospital records, have been developed, [7, 8] and used to measure the quality of nursing care in North America [9–11], Europe,  New Zealand  and Australia [14–16].
Most of the work in nursing-sensitive outcomes has involved adult patients. Some researchers have included paediatric patients within predominantly adult populations when measuring the quality of nursing care [17–19], however these nursing-sensitive outcomes have only been validated in adult acute care [7, 8], so may not be applicable in the paediatric context. Although paediatric nursing-sensitive indicators [4, 20] and generic paediatric outcome indicators have been developed, our previous paper and was one of the first to report algorithms for defining paediatric nursing-sensitive outcomes using linked administrative hospital data .
Hospital discharge summaries are useful for research in this field as they commonly include demographic information about the patients as well as standardised coding of their principal diagnosis, comorbid diagnoses, procedures performed and complications that arise during hospitalisation . However, there are many specific challenges in identifying nursing-sensitive outcomes in paediatrics. Children are generally less likely to be admitted to hospital: they are a healthier group with fewer chronic illnesses, and have diverse physiological, psychosocial and developmental stages . Furthermore, unlike the Charlson or Elixhauser Comorbidity scores [24, 25] for adults, there is no established paediatric comorbidity classification system.
There are also generic challenges related to the use of administrative hospital records when determining nursing-sensitive outcomes. Most importantly is the challenge of differentiating outcomes due to the patient’s premorbid or comorbid state, or other social or environmental factors, rather than nursing care. This challenge has been addressed through the development of complex algorithms that exclude an adverse patient outcome from being considered a nursing-sensitive outcome in the presence of certain comorbid conditions or patient characteristics. These algorithms are usually applied to a single hospital record, so accuracy of nursing-sensitive identification is contingent on the comorbid condition also being coded on that record. However, in our previous work using linked data , we found that recording of comorbid conditions was not consistent. For example, we found that 43% of children with pressure ulcers had a form of paralysis recorded on a previous hospital admission, but not on the hospital admission which identified the supposed nursing-sensitive outcome of pressure ulcer . Without the ability to 'look back’ at prior hospitalisation records (using linked data), we could have incorrectly identified these adverse events as nursing-sensitive and thus over-estimated the rate of nursing-sensitive pressure ulcers. Building on previous work with nursing-sensitive outcomes in adult patients,  generic paediatric quality outcomes [26, 27], and incorporating the benefits of linked hospital data, we have proposed 13 possible paediatric nursing-sensitive outcomes and their corresponding inclusion and exclusion algorithms for use with linked administrative hospital data (Table 1).
The aims of our current study were to determine the rates of paediatric nursing-sensitive outcomes in children admitted to the paediatric tertiary hospital in Perth, Western Australia over a 10 year period, and determine the potential clinical utility of measuring nursing-sensitive outcomes in paediatric hospitals using linked hospital data. In addition, we were interested in determining factors associated with children experiencing any of the adverse events.
Study design and patients
A retrospective cohort study included all hospitalisation records for Western Australian (WA) children (≤ 18 years), who were admitted to the tertiary paediatric hospital in Perth (WA) for at least one night, during the 10 year period from July 1999 to June 2009. Each child’s hospital record was recorded electronically and was linked to all previous and subsequent electronic hospitalisation records at any public or private hospital in WA (back to 1989 and forward until March 2010) and to the Australian census data, by the Western Australian Data Linkage Branch (WADLB) .
Our earlier work used the linked data to develop algorithms for 13 paediatric nursing-sensitive outcomes [21, 30]. These algorithms were used to evaluate numerators and denominators for the following outcomes: hospital acquired lower respiratory tract infection other than pneumonia (LRTI); gastrointestinal (GI) infection; pneumonia; sepsis; cardiac/respiratory arrest, shock or respiratory failure; central nervous system (CNS) complication; central venous line (CVL) infection; infectious disease; pressure ulcer; failure to rescue (FTR); surgical wound infection; physiologic/metabolic derangement; and postoperative cardiopulmonary complications. Table 1 provides the criteria for individual paediatric nursing-sensitive outcomes and these algorithms were applied to identify the nursing-sensitive outcomes in this study.
The WA Hospital Morbidity Data (HMD) provided abstracts of demographic and clinical information on all hospital separations from every hospital within WA . Up to 22 diagnoses and 12 procedures are coded for each hospitalisation. Diagnoses were coded using the International Classification of Diseases, Version 10, Australian Modification (ICD-10-AM)  and procedures were coded according to the Australian Classification of Health Interventions (ACHI) . Variables for major diagnostic categories (MDC) and Australian Refined Diagnostic Related Groups (AR-DRG) were also provided. The data also included the date, time and ward of every patient’s ward movement within the tertiary hospital. The information from the Australian Census data linked to each HMD record included the accessibility and remoteness index (ARIA+)  and the socioeconomic status associated with the residential area of the patient (SEIFA) .
The quality of data linkage has been assessed by sampling and the estimated percentage of invalid links (false positives) and missed links (false negatives) was 0.11% . Validation studies for the quality of data in administrative health databases have reported the following percentages: <1% missing demographic information; 87-90% primary diagnoses recorded accurately; > 90% procedures recorded accurately and the accuracy of records of secondary diagnoses varies [36, 37]. Data were received in anonymised files and then merged and analysed in SPSS for Windows (Version 22.214.171.124; 2010 IBM SPSS Chicago, Il, USA). Logical checks, such as checking for duplicate records, that hospital separations occurred after hospital admissions, and that each child’s sex, date of birth and ethnicity were the same across databases, were undertaken by the principal researcher during data clean-up. Seventy five records were deleted as they were for adults.
Month and year of birth were provided for each child. Age on admission was calculated and categorised into groups that describe growth, cognitive and psychosocial development (1–28 days = neonate, 29–365 days = infant, > 1-3 years = toddler, > 3-6 years = pre-schooler, > 6-13 years = school-age, > 13- ≤ 18 years = adolescent) . The risk pool for surgical wound infections, postoperative cardiopulmonary complications and physiologic/metabolic derangement was surgical patients only. This subset contained patients classified as 'surgical’ based on AR-DRG codes supplied by WADLB. The remainder of the cohort was classified as 'non-surgical’ and contained children coded as 'medical’ or 'other’. The number of times each patient moved wards during a hospital admission was calculated.
To determine any associations with socioeconomic status, the Index of Relative Socio-Economic Disadvantage (IRSD) was chosen from the Australian Census data as it best represents residential areas of low income, high unemployment and low levels of education . A lower score indicates the family resides where many households in the area were disadvantaged and a higher score indicates the family resides in an area with the least number of households who were disadvantaged. The geographic measure of remoteness (ARIA+) based on accessibility to service centres along road networks was also used. IRSD and ARIA + are constructed from census data collected every five years, therefore scores from the most recent census preceding the hospital admission were used. The IRSD and ARIA + indices are based on geographic levels with the census Collection District (CD) being the smallest spatial unit in the classification. As the IRSD reflect the area in which the family live and not the individual family, CD is most likely to reflect the actual socioeconomic status of the family. Where a CD was not available for an individual, the index for the Statistical Local Area (SLA) was assigned, which is the next smallest spatial unit in the classification . Once assigned, IRSD and ARIA + were categorised into dichotomous variables; least disadvantaged and most disadvantaged, and city and regional or remote WA resident.
Linked data was used to identify the nursing-sensitive outcomes as described in the background. However, when calculating proportions and rates of the outcomes each hospital separation record was assumed to be an independent event; therefore calculations were based on hospital separations or records rather than patients, as in other studies [10, 14, 15, 40].
Rates per 10,000 patient days and number per 1,000 hospital separations were calculated for each of the 13 paediatric nursing-sensitive outcomes. The risk pool for surgical wound infection, physiologic/metabolic derangement and postoperative cardiopulmonary complications included surgical records only whereas the risk pool for the other outcomes was medical and surgical records.
Differences between characteristics of those that did and did not acquire a nursing-sensitive outcome were analysed using Chi-square tests for categorical data and independent t - tests for continuous data. Poisson regression models were used to determine whether there was a significant association between each nursing-sensitive outcome and each of the potential risk factors: age, sex, type of admission, case mix, number of ward movements, season, year of admission, IRSD and ARIA+. Risk factors which had a statistically significant relationship with one or more nursing-sensitive outcomes were subsequently included in multiple regression models.
For each nursing-sensitive outcome, a Poisson multiple regression model was fitted to estimate the rate ratios (RR) and 95% confidence intervals (CI) for age, sex, admission type, case mix, season, year of admission, IRSD and ARIA+. Except for the nursing-sensitive outcome, surgical wound infection, all Poisson regression models included length of stay (LOS) as an offset variable. As the date of surgery was not supplied in the HMD we were unable to calculate a length of stay post-surgery for surgical wound infection. Significance was set at p < .05.
The project received approval from the Human Research Ethics Committees of the study hospital and the WA Department of Health. Waiver of consent was granted as the study adhered to the National Health and Medical Research Council’s National Statement on Ethical Conduct in Human Research. Permission to use the data for research was granted by the data custodian for each database. Data was de-identified and all personal identifiers were removed before being provided to the researchers. Confidentiality statements were signed by members of the research team. Data files were encrypted when transported and kept securely on a password protected server which was not accessible via the internet consistent with the Practice Code for the use of Personal Health Information . Destruction of the data will adhere to instructions provided in the Australian Government Information Security Manual .
Of 129,719 hospital separations for 79,016 children, 79% were for emergency admissions and 83% were for medical admissions. The number of hospital separations each year ranged from 12,025 in 2003–04 to 14,476 in 2000–01 with a mean number of 12,972/year. The leading major diagnostic category was respiratory disorders, which accounted for 16% of records. Once admitted, 80% of children remained on the same ward throughout their hospital stay. The median LOS (excluding patients who remained less than 1 night) was 2 days, inter-quartile range (IQR) = 1-4 days.
A total of 1,919 (1.5%) hospital separations recorded one or more nursing-sensitive outcomes over the 10 year period. They were experienced by 1,740 children and represented 2,037 occurrences of nursing-sensitive outcomes. Of the separation records that had a nursing-sensitive outcome, 77% were for emergency admissions, 67% were for medical patients and 18% were for the major diagnostic category of gastrointestinal disorder. Once admitted, 62% remained on the same ward throughout their hospital stay and 20% had one within hospital transfer. The median LOS was 6 days and IQR = 4-11 days. There were statistically significant differences between the proportions with each characteristic of those with and without a nursing-sensitive outcome as shown in Table 2.
Rates of nursing-sensitive outcomes
There were 113 separation records of the total 129,719 in this study that had more than one nursing-sensitive outcome recorded (0.09%) and there were separation records for 179 children that recorded a nursing-sensitive outcome on more than one separation (0.14%). Table 3 displays the crude (unadjusted) rate of each paediatric nursing-sensitive outcome per 10,000 patient days and the proportion per 1,000 hospital separations. The rate of physiologic and metabolic derangement was greatest at 15.15/10,000 patient days. This was followed by the postoperative cardiopulmonary complications rate of 11.12/10,000 patient days and LRTI and gastrointestinal infections rates of 10.71 and 10.70/10,000 patient days respectively. As proportions of hospital separations, GI infection was 14.01/1,000, surgical wound infection was 10.42/1,000, LRTI was 9.24/1,000 and physiologic and metabolic derangement was 9.07/1,000. Pressure ulcer had the lowest rate at 0.61/10,000 patient days and 0.51/1,000 hospital separations.
Risk factors for nursing-sensitive outcomes
Table 4 presents the association between patient sociodemographic characteristics and the risk for each paediatric nursing-sensitive outcome. The risk of developing a LRTI was greater with each decreasing age group. Compared to adolescents, the risk for infants was nearly 8 times greater (RR = 7.74, 95% CI 5.11, 11.75). Winter and spring were also associated with greater risk of developing a LRTI compared to summer (RR = 2.02, 95% CI 1.51, 2.72; and RR = 1.54, 95% CI 1.13, 2.10) respectively.
Similarly, the risk of developing pneumonia was greater in younger children, with toddlers at greatest risk. The risk of developing pneumonia was 4 times greater in toddlers than adolescents (RR = 4.19, 95% CI 2.78, 6.30). Compared to summer, there was greater risk in winter (RR = 1.68, 95% CI 1.21, 2.34) and spring (RR = 1.58, 95% CI 1.13, 2.21). Medical patients and emergency admissions had lower risks than surgical and elective admissions of developing pneumonia (RR = 0.70 95% CI 0.55, 0.88 and RR = 0.36, 95% CI 0.29, 0.46) respectively.
The risk of developing GI infection was nearly 10 times greater in toddlers than adolescents (RR = 9.89, 95% CI 6.24, 10.32). The risk was also greater in each age group except neonates. Boys and medical patients were at greater risk (RR = 1.35, 95% CI 1.08, 1.70 and RR = 3.40, 95% CI 2.26, 5.11).
Age was the only factor that was significantly associated with a risk of sepsis. Compared to adolescents, neonates were at a lower risk of developing sepsis (RR = 0.26, 95% CI 0.08, 0.90) and the risks were greatest in toddlers (RR = 3.62, 95% CI 1.97, 6.64). The risk of arrest, shock or respiratory failure, was nearly five and a half times greater in toddlers than adolescents (RR = 5.48, 95% CI 2.57, 11.66). There was a greater risk in each age group except for neonates where there was no association found. Compared to surgical patients, medical patients were twice as likely to arrest, develop shock or respiratory failure (RR = 2.08, 95% CI 1.11, 3.88).
Compared to adolescents, there was a lower risk in occurrence of CNS complications in infants, toddlers and pre-schoolers by approximately one third in each category. The risk in medical patients was 3.69 times greater than surgical patients (95% CI 1.76, 7.72). The only factor to be associated with CVL infections was admission type and the risk in emergency admissions was decreased by 20% (RR = 0.23, 95% CI 0.14, 0.40) compared to children who were elective admissions.
There were no statistically significant risk factors identified for outcomes of failure to rescue, pressure ulcer or infectious diseases. Although significant in one or more univariate models, there were no statistically significant associations between IRSD and ARIA + indices and any of the nursing-sensitive outcomes in multiple regression models.
Table 5 presents the RR and 95% CI for three paediatric nursing-sensitive outcomes that used a risk pool restricted to surgical patients. The risk of developing a surgical wound infection was nearly 8 times greater for emergency admissions compared with elective admissions (RR = 7.78, 95% CI 5.10, 11.86) and lower by over 10% in neonates compared with adolescents (RR = 0.87, 95% CI 0.47, 1.62).
Compared to adolescents, the risk of developing physiologic and metabolic derangement was 4.04 times greater in infants (95% CI 2.32, 7.05) and it was also lower in toddlers (RR = 2.45, 95% CI 1.22, 4.91) and school-age children (RR = 1.95, 95% CI 1.11, 3.43). The risk was almost 90% lower in neonates (RR = 0.12, 95% CI 0.04, 0.35). The risk was 1.65 times greater in boys (95% CI 1.17, 2.33) and 3.46 times in children who were emergency admissions (95% CI 2.32, 5.18).
The risk of postoperative cardiopulmonary complications was nearly twice as great in toddlers as adolescents (RR = 1.81, 95% CI 1.01, 3.26), but risks in other age groups did not differ significantly. The risk associated with emergency admission was nearly 60% less than that of elective admission (RR = 0.38, 95% CI 0.26, 0.55). Sex, season, IRSD and ARIA + indices were not statistically significantly associated with any of the surgical nursing-sensitive outcomes.
Application of our 13 'paediatric’ nursing-sensitive outcome algorithms  identified a limited number of nursing-sensitive outcomes that had more than 100 events over the 10 year period. They were: hospital acquired infections of LRT, GI tract, surgical wounds, sepsis and pneumonia, and physiologic/metabolic derangement and postoperative cardiopulmonary complications. Incidences ranging from 3-14/1,000 hospital separations suggest that they may occur frequently enough to be useful as measures of quality nursing care. The minimum number of events for clinical utility has not been defined to date, however the Canadian Institute for Health Information  recommends that a 'cut off’ or a minimum number of events for each outcome be discussed with consideration of its value for a meaningful and feasible analysis. Further consideration regarding how to report information when numbers of events are small is also warranted [43, 44].
The low incidence rates for arrest/shock/respiratory failure, CNS complications, CVL infection, infectious disease, pressure ulcer and failure to rescue suggest that either the algorithms are not sensitive enough to accurately identify the number of nursing-sensitive outcomes, that there are insufficient numbers of outcomes recorded, or insufficient numbers of outcomes occurred, suggesting the usefulness of these outcomes as quality indicators in paediatric nursing may be questionable. The rates found in this study for pressure ulcer, FTR and infectious disease are lower than those reported in other studies [40, 45–49]. The administrative hospital data used to identify rates of pressure ulcer and FTR in these other studies was not linked when identifying records with nursing-sensitive outcomes of pressure ulcer and FTR which may account for their higher rates; however adjustments were made for comorbidities [40, 45–48]. Since the reporting of rates of hospital acquired infectious diseases in an Australian point prevalence study , the national immunisation schedule has increased the range of vaccines freely available to Australians. Consequently, the increased level of immunity has decreased the chances of acquiring infectious diseases both in the community and in hospital. Although a 'cut off’ for a minimum number of events has not been stipulated, these studies consistently conclude that there are currently insufficient events in paediatric populations to be clinically useful for measuring quality [40, 50].
Five of the seven nursing-sensitive outcomes that could be considered for use, include hospital acquired infections (LRTI, GI infection, pneumonia, sepsis, surgical wound infection). Age was significantly associated with risks of developing these infections except for surgical wound infections. Neonates and infants have immature immune systems and are more prone to infections. However, compared with adolescents, neonates had similar risks of nursing-sensitive infections, but risks were high for infants, and even higher for toddlers. The study hospital accommodates neonates and infants in areas specific to their age, which may increase awareness of their vulnerability and specific needs for infection control precautions. Parents may also be more aware of the vulnerability of young children to infection and do not visit or allow visitors if they have an infection. Infants and neonates are not independently mobile so are confined to cots and their rooms whereas older children are mobile and their movement around ward areas and interaction with other patients and visitors could contribute to their greater risk. In addition, the airway of a child under approximately eight years of age differs anatomically to that of an adolescent or adult. These anatomical differences put neonates, infants, toddlers and pre-schoolers at greater risk of developing LRTI, pneumonia and arrest/shock/respiratory failure as shown in our results. Therefore it is important to consider age when comparing these nursing-sensitive outcomes between agencies or between units. Age was stratified to represent the accepted developmental groups  and our results, which report comparisons between groups, are considered to be more useful than comparisons based on annual changes.
Seasonal differences from summer occurred in winter and spring for rates of LRTI and pneumonia. The viruses causing these conditions tend to be more prevalent in the community and increase the chances of infection during these seasons. Accordingly, children’s hospital inpatient numbers increase during winter and spring. Further research would be useful to determine if levels of nurse staffing, such as nurse patient ratio or nursing hours per patient day, alter during the busy seasons and whether there is any association with the nursing-sensitive outcomes. Stratton  measured five nursing-sensitive outcomes for each quarter and found a significant association between winter and parent/family complaints. Twigg  adjusted for season when seeking associations between nurse staffing and nursing-sensitive outcomes in an adult population based on conceptual grounds but did not report whether there were any associations.
An important reason for measuring the quality of nursing care is to identify areas where the care can be improved by implementing evidence informed practices. Where there is evidence of lowering rates of nursing-sensitive outcomes by initiating the same 'bundle of care’ or changes in nurses’ behaviour related to the bundle of care, consideration can be given to combining results and reporting the rates as a composite indicator. For example, LRTI, GI infection and infectious disease are related to adherence of infection control procedures, a bundle of care, such as hand hygiene, assessment of patient history, patient isolation and family education . Therefore, 'grouping’ the results of these nursing-sensitive outcomes may make implementation of practice changes easier and monitoring more straightforward during quality improvement activities. Similarly, rates of sepsis, surgical wound infection, and CVL infection are related to hand hygiene and patient/family education; consequently, these outcomes could be grouped and reported for surgical patients.
As well as providing evidence informed care that is safe and effective, nurses are increasingly accountable to the public for the expenditure of their care [52, 53]. Therefore it is necessary to consider the significant difference in the hospital length of stay (LOS) between children who did and did not develop a nursing-sensitive outcome. It is unclear whether the increased LOS occurred as a result of the nursing-sensitive outcome or whether the children developed the nursing-sensitive outcome as a result of being in hospital longer. There was also a significant difference in number of ward movements. Finally, the increased number of ward movements in those who developed a nursing-sensitive outcome may be a result of the nursing-sensitive outcome rather than the ward movements or 'churn’ contributing to the outcome . Many of the outcomes measured were infectious so children would need to be moved to an area where they could be isolated from non-infected children or to an intensive care area if the child’s airway and respiratory system were compromised. This data was not available.
Using administrative data to measure nursing-sensitive outcomes has both advantages and limitations. Advantages are that the data collection follows national standards and it provides the groundwork for benchmarking and comparisons of rates between the paediatric hospitals in Australia, the data is collected routinely reducing the need to collect information from case records or regular audits which is time consuming and often takes nurses away from providing nursing care . Limitations are that the outcomes measured are not necessarily those that best reflect the quality of nursing care and there may be other patient outcomes that provide this;  and the inability to determine whether secondary diagnoses are pre-existing comorbid conditions or complications that occurred during hospitalisation, which increases the likelihood of under or over reporting of outcomes. Under reporting tends to occur when the outcomes have no financial implication for the health service provider [56, 57]. The advantages of feasibility, cost saving and having complete longitudinal population data when using administrative health data offsets the limitations and can provide reliable population-based estimates of nursing-sensitive outcomes. These implications of using administrative data have been discussed in our previous work .
Firstly, patient’s conditions present on admission are not all routinely recorded in the separation abstract of WA HMD. Therefore, the algorithms to identify the nursing-sensitive outcomes remove some children that are at high risk of developing the outcome. As a consequence, application of these algorithms may not provide a true indication of the quality of nursing care. However, if the purpose of measuring the outcomes is for quality improvement or to provide a benchmark, and the same algorithms are used, rates could be comparable.
Secondly, an illness severity score or comorbidity index has not been used in this study. As risk adjustment must be context specific  and no paediatric comorbidity index has been validated, identification of comorbid conditions, not identified in algorithms, that increase the risk of a child developing a nursing-sensitive outcome was not undertaken.
Thirdly, although patient hospital records were linked to identify the nursing-sensitive outcomes, each hospital record with any nursing-sensitive outcomes was considered an individual event. There is a chance that the same nursing-sensitive outcome has been counted twice. However, it could be argued that if a child is readmitted with a nursing-sensitive outcome, it should still be counted as a separate outcome as it could be related to inappropriate discharge planning and suboptimal child and family education which are fundamental nursing roles.
Fourthly, we were only able to consider risk factors that are available in the data collected. There may be other factors associated with the quality of care that influence nursing-sensitive outcomes such as processes of care. Structures that influence care such as levels of nurse staffing and experience and education of nurses need to be considered.
Finally, neither ethnicity nor race was included in any models. Although both have been demonstrated to be associated with greater rates of inpatient safety indicators in studies in adult populations , associations with poorer health outcomes in paediatric studies have not been demonstrated . If hospital care is equitable, the incidence of nursing-sensitive outcomes should not vary between racial or ethnic groups and this warrants further research. Unfortunately, indigenous status is not a reliable variable in HMD [61–63] precluding meaningful analysis.
Implications for policy and practice
There are sufficient numbers of events of some paediatric nursing-sensitive outcomes to make them useful measures of quality of nursing care for quality improvement and to benchmark with other Australian tertiary paediatric hospitals. This information can lead to evidence informed practice as nurses’ share their practices and strive to minimise the numbers of adverse patient events. Future research can focus on comparisons with other paediatric hospitals to understand variability in outcome rates. Comparisons with adult hospitals can be undertaken to determine variability with children’s hospitals.
To assist in the accurate identification of hospital acquired complications, it is recommended that a 'present on admission’ variable be recorded in the separation abstract, as used in California and Canada . Such a variable would eliminate the need for complex algorithms which have the potential of removing children who are at high risk of getting the outcome from the risk pool. Children who are at high risk of getting the outcome are the children we need to be including in the quality improvement exercise.
Risks of developing the nursing-sensitive outcomes investigated tend to decrease as the child gets older. Reporting of nursing-sensitive outcomes should be stratified by developmental age groups to better reflect the areas to target for quality improvement. As well as comorbidities or severity of illness, factors that have not been considered are the levels of nurse staffing and the education levels of nurses. Many studies have shown relationships between rates of nursing-sensitive outcomes and levels of nurse staffing predominantly in adult hospitals [8, 9, 11–15] with minimal paediatric studies [40, 50]. As paediatric nursing-sensitive outcomes are available, further study of their association with levels of nurse staffing can now be undertaken.
In addition, if nursing-sensitive outcomes are to be used for benchmarking between hospitals, adjusting for hospital characteristics such as bed number, teaching status, number of intensive care beds and case mix needs to be considered. Such adjustments were not required for this study as it was carried out in one children’s hospital.
In conclusion, there are adequate numbers of events and incidence for LRTI, GI infection, pneumonia, surgical wound infection, physiologic/metabolic derangement, sepsis and postoperative cardiopulmonary complications for them to be considered useful nursing-sensitive outcomes to contribute to the measurement of quality paediatric nursing care. When used for quality improvement or to benchmark with other agencies, data needs to be adjusted for age or stratified by age to ensure equitable comparisons. Other characteristics that may alter the risks for children developing a nursing–sensitive outcome and require consideration include sex, admission type, whether the admission type is medical or surgical, and season. Ultimately the goal of implementing evidence informed nursing practice should be targeted to groups most at risk of developing a nursing-sensitive outcome.
Australian Institute of Health and Welfare: Australia’s health 2010. 2010, Canberra ACT (Australia): AIHW
Council of Australian Governments: National healthcare agreement. 2008, Canberra: COAG, 1-33.
Labour force-health. http://www.aihw.gov.au/labourforce/health.cfm.
Nursing-sensitive indicators. http://www.nursingworld.org/MainMenuCategories/ThePracticeofProfessionalNursing/PatientSafetyQuality/Research-Measurement/The-National-Database/Nursing-Sensitive-Indicators_1.aspx.
Nursing matters fact sheet: nursing sensitive outcome indicators. http://www.icn.ch/images/stories/documents/publications/fact_sheets/15c_FS-Nursing_Sensitive_Outcome_Indicators.pdf.
Yang KPA, Simms LM, Yin JCT: Factors influencing nursing-sensitive outcomes in Taiwanese nursing homes. Online J Issues Nurs. 1999, 4: 2.
AHRQuality indicators: introduction. http://www.qualityindicators.ahrq.gov/.
Needleman J, Buerhaus PI, Potter V, Mattke S, Stewart M, Zelevinsky K: Nurse staffing and patient outcomes in hospital. 2001, Boston: Harvard School of Public Health, i-487.
Aiken LH, Clarke SP, Sloane DM, Lake ET, Cheney T: Effects of hospital care environment on patient mortality and nurse outcomes. J Nurs Adm. 2008, 38: 5.
Needleman J, Buerhaus P, Pankratz VS, Leibson CL, Stevens SR, Harris M: Nurse staffing and inpatient hospital mortality. N Engl J Med. 2011, 364: 11-10.1056/NEJMoa1009492.
Tourangeau AE, Tu JV: Developing risk-adjusted 30-day hospital mortality rates. Res Nurs Health. 2003, 26: 6.
Van den Heede K, Sermeus W, Diya L, Clarke SP, Lesaffre E, Vleugels A, Aiken LH: Nurse staffing and patient outcomes in Belgian acute hospitals: cross-sectional analysis of administrative data. Int J Nurs Stud. 2009, 46 (7): 928-939. 10.1016/j.ijnurstu.2008.05.007.
McCloskey BA, Diers DK: Effects of New Zealand’s health reengineering on nursing and patient outcomes. Med Care. 2005, 43: 11.
Duffield C, Diers D, O’Brien-Pallas L, Aisbett C, Roche M, King M, Aisbett K: Nursing staffing, nursing workloa21d, the work environment and patient outcomes. Appl Nurs Res. 2011, 24: 244-255. 10.1016/j.apnr.2009.12.004.
Twigg D, Duffield C, Bremner A, Rapley P, Finn J: The impact of the nursing hours per patient day (NHPPD) staffing method on patient outcomes: a retrospective analysis of patient and staffing data. Int J Nurs Stud. 2011, 48: 5.
Twigg D, Duffield C, Thompson PL, Rapley P: The impact of nurses on patient morbidity and mortality: the need for a policy change in response to the nursing shortage. Aust Health Rev. 2010, 34: 3-10.1071/AH09655.
Blegen M, Goode C, Reed L: Nurse staffing and patient outcomes. Nurs Res. 1998, 47: 1-10.1097/00006199-199801000-00001.
Shuldham C, Parkin C, Firouzi A, Roughton M, Lau-Walker M: The relationship between nurse staffing and patient outcomes: a case study. Int J Nurs Stud. 2009, 46: 7-10.1177/002088171004600203.
Sochalski J: Is more better? The relationship between nurse staffing and the quality of nursing care in hospitals. Med Care. 2004, 42 (Suppl 2): II-67-II-73.
Schwalenstocker E, Bisarya H, Lau S, Adebimpe O: Nursing-sensitive indicators for children’s hospital care quality: a white paper prepared by the Pediatric Data Quality Systems (Pedi-QS) Collaborative. 2007, : US Pediatric Data Quality Systems (Pedi-QS) Collaborative, 1-198.
Wilson S, Bremner AP, Hauck Y, Finn J: Identifying paediatric nursing-sensitive outcomes in linked administrative health data. BMC Health Serv Res. 2012, 12: 209-10.1186/1472-6963-12-209.
National Centre for Classification in Health: Australian coding standards. 2008, Sydney: University of Sydney, 6
Beal AC, Co JPT, Dougherty D, Jorsling T, Kam J, Perrin J, Palmer RH: Quality measures for children’s health care. Pediatrics. 2004, 113: 1-10.1542/peds.113.1.1.
Charlson ME, Pompei P, Ales KL, MacKenzie CR: A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987, 40: 5.
Elixhauser A, Steiner C, Harris DR, Coffey RM: Comorbidity measures for use with administrative data. Med Care. 1998, 36: 1-10.1097/00005650-199801000-00001.
McDonald KM, Davies SM, Haberland CA, Geppert JJ, Ku A, Romano PS: Preliminary assessment of pediatric health care quality and patient safety in the United States using readily available administrative data. Pediatrics. 2008, 122: 2.
Pediatric quality indicators technical specifications - version 4.4. 2012, http://www.qualityindicators.ahrq.gov/Modules/PDI_TechSpec.aspx.
McCloskey BP: The effects of health policy on nursing and patient outcomes in New Zealand [dissertation]. 2003, School of Nursing: Yale University
Data linkage Western Australia. http://www.datalinkage-wa.org/.
Wilson S, Hauck Y, Bremner A, Finn J: Quality nursing care in Australian paediatric hospitals: a Delphi approach to identifying indicators. J Clin Nurs. 2012, 21: 11-12. 10.1111/j.1365-2702.2011.03739.x.
Government of Western Australia Department of Health: Hospital morbidity data system reference manual. 2010, Perth: Government of Western Australia
National Centre for Classification in Health: The international statistical classification of diseases and related health problems, tenth revision, Australian modification (ICD-10-AM). 2008, Sydney: University of Sydney, 6
National Centre for Classification in Health: Australian classification of health interventions. 2008, Sydney: University of Sydney, 6
About ARIA + (accessibility/remoteness index of Australia). http://www.adelaide.edu.au/apmrc/research/projects/category/about_aria.html.
Adhikari P: Analytical Services. Socio-economic indexes for areas: introduction, use and future directions. 2006, Canberra, ACT (Australia): Australian Bureau of Statistics, 1-48.
Holman CDAJ, Bass AJ, Rouse IL, Hobbs MST: Population-based linkage of health records in Western Australia: development of a health services research linked database. Aust N Z J Public Health. 1999, 23: 5.
Henderson T, Shepheard J, Sundararajan V: Quality of diagnosis and procedure coding in ICD-10 administrative data. Med Care. 2006, 44: 11.
Child development. http://www.cdc.gov/ncbddd/childdevelopment/positiveparenting/index.html.
Australian Bureau of Statistics: Socio-economic indexes for areas (SEIFA) - technical paper 2006. 2008, Canberra, ACT (Australia): Australian Bureau of Statistics, 1-76.
Mark BA, Harless DW, Berman WF: Nurse staffing and adverse events in hospitalized children. Policy Polit Nurs Pract. 2007, 8: 2.
Department of Health Western Australia: Practice code for the use of personal health information. 2009, Perth WA (Australia): Government of Western Australia, 1-18.
Department of Defence Intelligence and Security: Australian government information security manual. 2011, Canberra, ACT (Australia): Commonwealth of Australia, 1-306.
Canadian Institute for Health Information: Hospital standardised mortality ratio (HSMR) technical notes Ottawa. 2011, Ontario: CIHI, 1-17.
Australian Commission on Safety and Quality in Health Care: National core, hospital-based outcome indicator specification (2012). version 1.1. Consultation draft. 2012, Sydney: ACSQHC, 1-97.
Miller MR, Zhan C: Pediatric patient safety in hospitals: a national picture in 2000. Pediatrics. 2004, 113: 6.
Sedman A, Harris JM, Schulz K, Schwalenstocker E, Remus D, Scanlon M, Bahl V: Relevance of the agency for healthcare research and quality patient safety indicators for children’s hospitals. Pediatrics. 2005, 115: 1.
Kronman MP, Hall M, Slonim AD, Shah SS: Charges and lengths of stay attributable to adverse patient-care events using pediatric-specific quality indicators: a multicenter study of freestanding children’s hospitals. Pediatrics. 2008, 121: 6.
Scanlon MC, Harris JM, Levy F, Sedman A: Evaluation of the agency for healthcare research and quality pediatric quality indicators. Pediatrics. 2008, 121: 6.
Burgner D, Dalton D, Hanlon M, Wong M, Kakakios A, Isaacs D: Repeated prevalence surveys of paediatric hospital-acquired infection. J Hosp Infect. 1996, 34: 3.
Stratton KM: Pediatric nurse staffing and quality of care in the hospital setting. J Nurs Care Qual. 2008, 23: 2.
Cruickshank M, Ferguson J: Reducing harm to patients from health care associated infection: the role of surveillance. Australian Commission on Safety and Quality in Health Care. Edited by: Cruickshank M, Ferguson J. 2008, Sydney: , 374.
Alexander GR: Nursing sensitive databases: their existence, challenges, and importance. Med Care Res Rev. 2007, 64 (2 suppl): 44S-63S. 10.1177/1077558707299244.
Health reform implementation taskforce. http://www.health.wa.gov.au/hrit/docs/publications/clinicalframework.pdf.
Duffield C, Diers D, Aisbett C, Roche M: Churn: patient turnover and case mix. Nurs Econ. 2009, 27: 3.
National Health Service (UK): From the impact of nursing on patient clinical outcomes: developing quality indicators to improve care [internet]. 2005, Glasgow: NHS
Lawthers AG, McCarthy EP, Davis RB, Peterson LE, Palmer RH, Iezzoni LI: Identification of in-hospital complications from claims data: is it valid?. Med Care. 2000, 38: 8.
Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K: Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002, 346: 22.
Iezzoni LI: Risk adjustment for measuring health care outcomes. 2003, Washington DC: AcademyHealth, 3
Coffey RM, Andrews RM, Moy E: Racial, ethnic, and socioeconomic disparities in estimates of AHRQ patient safety indicators. Med Care. 2005, 43 (Suppl 3): I-48-I-57.
Jutte DP, Roos LL, Brownell MD: Administrative record linkage as a tool for public health research. Annu Rev Public Health. 2011, 32: 92-108.
Fremantle E, Zurynski YA, Mahajan D, D’Antoine H, Elliott EJ: Indigenous child health: urgent need for improved data to underpin better health outcomes. Med J Aust. 2008, 188: 10.
Australian Institute of Health and Welfare: Indigenous identification in hospital separations data–quality report. 2010, Canberra, ACT (Australia): AIHW, 1-68.
Australian Institute of Health and Welfare: Indigenous identification in hospital separations data. Volume 25. 2005, Canberra, ACT (Australia): AIHW, 1-135.
Hughes JS, Averill RF, Goldfield NI, Gay JC, Muldoon J, McCullough E, Xiang J: Identifying potentially preventable complications using a present on admission indicator. Health Care Financ Rev. 2006, 27: 3.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/13/396/prepub
The authors thank the staff at the Western Australian data linkage branch and the data custodian of the hospital morbidity data system.
The corresponding author was a recipient of a National Health and Medical Research Council (NHMRC) Public Health Postgraduate Research Scholarship which has enabled this study. She was also the recipient of a Lady Court Scholarship funded jointly by the Australian College of Children and Young People’s Nurses (ACCYPN) and Princess Margaret Hospital (PMH) Foundation to pay for the services of WA Data Linkage Branch. The NHMRC, ACCYPN and PMH Foundation had no input into the study design or any aspects of the study.
The authors declare that they have no competing interests.
SW conceived the study, secured funding, carried out the study design, data analyses, drafted the manuscript. AB assisted with data analyses, revised manuscript. YH revised manuscript. JF conceived the study, secured funding, assisted with data analyses, revised manuscript. All authors read and approved the final manuscript.