- Research article
- Open Access
Implementation of a pediatric early warning score tool in a pediatric oncology Ward in Palestine
BMC Health Services Research volume 21, Article number: 1159 (2021)
Pediatric Early Warning Scores (PEWS) are nurse-administered clinical assessment tools utilizing vital signs and patient signs and symptoms to screen for patients at risk for clinical deterioration.1–3 When utilizing a PEWS system, which consists of an escalation algorithm to alert physicians of high risk patients requiring a bedside evaluation and assessment, studies have demonstrated that PEWS systems can decrease pediatric intensive care (PICU) utilization, in-hospital cardiac arrests, and overall decreased mortality in high income settings. Yet, many hospital based settings in low and lower middle income countries (LMIC) lack systems in place for early identification of patients at risk for clinical deterioration.
A contextually adapted 16-h pediatric resuscitation program included training of a PEWS tool followed by implementation and integration of a PEWS system in a pediatric hematology/oncology ward in Beit Jala, Palestine. Four PDSA cycles were implemented post-implementation to improve uptake and scoring of PEWS which included PEWS tool integration into an existing electronic medical record (EMR), escalation algorithm and job aid implementation, data audits and ward feedback.
Frequency of complete PEWS vital sign documentation reached a mean of 89.9%. The frequency and accuracy of PEWS scores steadily increased during the post-implementation period, consistently above 89% in both categories starting from data audit four and continuing thereafter. Accuracy of PEWS scoring was unable to be assessed during week 1 and 2 of data audits due to challenges with PEWS integration into the existing EMR (PDSA cycle 1) which were resolved by the 3rd week of data auditing (PDSA cycle 2).
Implementation of a PEWS scoring tool in an LMIC pediatric oncology inpatient unit is feasible and can improve frequency of vital sign collection and generate accurate PEWS scores.
Contribution to the literature
This study demonstrates how to effectively implement a PEWS scoring tool into an LMIC clinical setting.
This study demonstrates how to utilize a robust feedback mechanism to ensure a quality program uptake.
This study demonstrates an effective international partnership model that other institutions may utilize for implementation science.
More than 6 million children around the world die each year, with the majority of these deaths from preventable disease . Almost one-third of deaths under the age of five are due to reversible critical illness such as respiratory failure and sepsis, with children in low- and lower-middle income countries (LMIC) disproportionately affected [2,3,4,5]. In hospital-based care, inadequate initial assessment, lack of ongoing monitoring, and inappropriate treatment contribute to poor outcomes, in part due to lack of systems that identify patients at risk for clinical deterioration [6,7,8].
Pediatric Early Warning Scores (PEWS) are nurse-administered clinical assessment tools utilizing vital signs and patient signs and symptoms to accurately identify patients at risk for clinical deterioration [9,10,11]. When utilized in a PEWS system which includes an escalation algorithm to alert physicians of high risk patients, studies have demonstrated PEWS systems can decrease PICU utilization, in-hospital cardiac arrests, and overall mortality in high income settings [12, 13]. Yet, many hospital based settings in LMICs lack systems in place for early identification of patients at risk for clinical deterioration.
In 2018, Boston Children’s Global Health Program (BCH GHP) partnered with a Ministry of Health (MOH)-Non-Governmental Organization (NGO) run pediatric hematology/oncology department in the occupied Palestinian territory (oPt)—an LMIC economy —to implement a formal PEWS tool as a component of a pediatric resuscitation training initiative (Fig. 1). There was no previous department-wide training in pediatric resuscitation or indentification of ill appearing patients. We utilized a quality improvement approach to assess the feasibility of implementation of a PEWS tool in a resource limited setting. Our aims were to:
Evaluate baseline vital sign documentation with a goal to increase compliance above 80%.
Implement a PEWS tool with greater than 80% compliance in frequency and accuracy of PEWS scoring over a three month post-intervention surveillance period.
Setting and context
This study was undertaken in a pediatric hematology and oncology (h/o) ward housed within a general adult and pediatric MOH hospital in Beit Jala, oPt. Patients in the oPt rely on a fragmented health care system defined by severe resource limitations including health care workforce shortages . The ward benefits from an MOH-NGO partnership for technical and operational support. The 14-bed ward admits approximately 100–120 patients per month. There are approximately 75 new cancer diagnoses per year presenting or referred to the department from within the oPt. There is no pediatric intensive care (PICU) on-site, thus any patients requiring critical care, including non-invasive ventilation, mechanical ventilation, or vasoactive therapies require transfer to outside hospitals within the West Bank, including Jerusalem, or hospitals in Israel for further evaluation and management.
The ward team is comprised of an interdisciplinary team that includes one pediatric oncologist, two general practicioners, rotating resident physicians, nurses, one pharmacist, and a ward social worker. Physician coverage includes two physicians during day shifts and coverage by a rotating resident physician during the evening and overnight shift. Rotating resident physicians are also responsible for overnight coverage of the neonatal intensive care unit, general pediatrics ward, and pediatric emergency department. There are 2–3 nurses staffing the ward during the day and one nurse caring for the entire ward during the overnight shift.
On-site needs assessment
In April 2019, prior to the PEWS intervention, an on-site needs assessment was undertaken by a physician and nurse team from the BCH GHP. The purpose of this assessment was to inform the development of a contextually adapted resuscitation and PEWS program. The needs assessment consisted of nursing, physician, and pharmacy leadership and staff individual interviews and focus groups to identify current ward care practices, resource availability, and expectations. All interviewees were directly involved in clinical care or clinical operations on the ward. The needs assessment discussions were primarily driven by input of participants to identify ward priorities for patient care improvements guided by a list of open-ended questions (Supplemental Document 1).
Needs assessment data were reviewed to identify common themes related to limitations, opportunities, and ward priorities. Common themes during interviews and focus groups (Table 1) included limitations such as high patient volumes, limited resources, staff shortages, and concern for outdated resuscitation protocols. Opportunities for improving care in the ward that were identified included the desire for further training in resuscitation and identification of the ill patient, desire for further basic nursing education including basic vital sign collection, and updating of protocols, policies, code medications, resources, and education. A summary of the thematic content was compiled and presented to the MOH-NGO ward leadership. Feedback from leadership on the identified needs, opportunities, and priorities was incorporated into the development of the PEWS implementation.
The intervention consisted of the following components: A pediatric resuscitation course for nurses and phsyicians, PEWS tool didactic and bedside education, integration of the PEWS tool into nursing workflow, and development and integration of an escalation algorithm into the ward workflow.
Ward priorities and requests for resuscitation curriculum content were elucidated prior to and during the needs assessment. Priorities included training for physicians and nurses in contextually adapted pediatric resuscitation and identification of the child at risk for clinical deterioration. In discussion with the nursing leadership, refreshers in basic nursing education, an orientation to the medical code cart, medication mixing, medication administration, and weight based dosing guidelines were all identified or requested. Furthermore, refreshers on pediatric physiology and vital signs collection were core components of training and served as a proxy for ensuring accurate physiologic observations in vital signs collection. The needs assessment and aforementioned priorities informed the development of a contextually adapted 16-h pediatric resuscitation program that included training of PEWS and consisted of a didactic course, hands-on resuscitation skills workshops, and low fidelity simulation for physicians and nurses. Each provider was provided with a resuscitation manual translated to English and Arabic. Instruction was provided by two pediatric emergency medicine physicians and one pediatric emergency nurse with expertise and experience in teaching in pediatric resuscitation. Parallel nursing and physician tracks were developed to tailor content to the respective needs of each discipline (Table 2). Two cohorts of physicians (total n = 20), which included physicians from other departments in the hospital, and two cohorts of nurses (total n = 12) successfully completed the resuscitation/PEWS course with over 95% attendance from September 1st to 15th 2019.
Pediatric early warning score tool and escalation algorithm
A literature review of pediatric early warning scoring tools was undertaken after review of the needs assessment was completed. PEWS-Resource Limited (PEWS-RL), a validated PEWS tool in a resource limited setting , was chosen as the PEWS tool to be implemented as the scoring parameters closely match bedside assessment capabilities of the ward compared to other PEWS tools which varied in number and complexity of assessment of scoring components. Six vital signs (heart rate, respiratory rate, respiratory distress, oxygen use, temperature, mental status) of which two are age-adjusted (heart rate and respiratory rate) are required to score PEWS-RL (Supplemental Document 2). An escalation algorithm was developed based on the ward staffing model which outlines the appropriate physician to contact if an elevated PEWS score is obtained.
Study population, measures and analysis
Inclusion criteria for patients in the study included the following: In the pre-intervention period, all patients admitted to the inpatient ward were included in the study. In the post-intervention period, inclusion criteria for data audits were based on a random interval of data collection every 7–10 days of all patients admitted to the ward on the given day of data collection. Six months of pre-intervention vital signs data (March 1st 2019 to September 1st 2019) were collected to establish the baseline vital signs documentation prior to the intervention. Pre-intervention data included all vital signs (heart rate, respiratory rate, oxygen use, respiratory distress, temperature, mental status) collected during vital sign occurences for all admissions to the ward during the given time interval. A vital sign occurrence is defined as the event where nursing collects vital signs at the bedside, which occurs four times daily. Vital signs are documented in patient charts located in an electronic medical record (EMR). Pre-intervention data was collected using a case report form (CRF) based in KoBoToolbox, an open-source field data collection tool . Three months of post-intevention data collection (September 15th to December 31st 2019) was collected. Random data audits were completed as a feasible real time PEWS feedback mechanism. Data collected included individual vitals signs and ‘complete’ vital sign occurrences. A ‘complete’ vital sign occurrence was defined as all six PEWS-RL vital signs documented during a vital sign collection. Complete vital sign collection should occur three times daily. A fourth vital sign check, which is comprised of a temperature check, is noted as ‘complete’ if the temperature is documented. Outcomes were defined as the change in vital signs frequency between the pre-intervention period and post-intervention data audit period and the PEWS frequency and accuracy in the post-intervention period. PEWS frequency was defined as the percentage of time that PEWS was scored in a given data audit, and accuracy was defined as an accurate PEWS score as verified by a study research assistant calculating PEWS from raw data audit.
Data compliance and IRB
All data collection was completed on-site in Beit Jala, oPt. Institutional review board approval was obtained through Boston Children’s Hospital and the Palestinian MOH.
PDSA #1: PEWS-RL scoring tool EMR integration
The PEWS-RL scoring tool was integrated into the EMR. Initial location for PEWS-RL scoring documentation within the EMR was identified and launched in real time after completion of the resuscitation course. The initial EMR documentation section (within the vital signs tab) that was identified for nurses to document the PEWS-RL score was experiencing technical issues, thus an alternative site in the vital signs documentation location within the EMR was chosen for scoring documentation.
PDSA #2: PEWS-RL and job aid implementation
Real-time, shoulder-to-shoulder nursing implementation support was provided by the BCH GHP team comprised of one pediatric emergency nurse and two pediatric emergency medicine physicains.
Job aids including PEWS tool scoring charts, normal vital signs for age, and a ward escalation algorithm were placed in strategic charting locations throughout the ward to facilitate use and documentation of the score in the identified EMR field and activation of the escalation algorithm when indicated.
PDSA #3: Data audits to identify compliance in PEWS-RL scoring frequency and accuracy (Fig. 2)
During the post-implementation period (September 16th to December 31th 2019), random data audits completed every 7–10 days were done to assess for vital sign and PEWS scoring frequency as well as PEWS scoring accuracy. These audits were undertaken by a local, Arabic language fluent research assistant who retrospectively collected vital signs and PEWS data on select admitted patients. All data was collected after discharge on the same day via the KoBoToolbox CRF . Review of nursing PEWS scoring accuracy was completed remotely by the BCH GHP implementation team to identify inaccurate PEWS scoring (Table 3). Vital sign completeness and PEWS accuracy and frequency were reported back to the ward physician and nursing leadership. Specific examples of common errors in scoring were noted for potential areas of score improvement.
PDSA #4: Post-implementation encouragement through data audits (Fig. 2)
Results from data audits were disseminated to ward leadership. Updates were provided to and discussed with nursing and physician staff at morning ward meetings. Ward nursing and physician feedback was solicited, which informed modifications to improve the ease of PEWS scoring, documentation and job aid utilization. Ward QI champion roles were defined and carried out by a physician and nursing leader who demonstrated interest in continuing the QI implementation work.
We used a pre-post cohort design to assess differences in frequency of complete vital signs documentation per vital sign occurrence utilizing statistical process control methodology. For post-intervention data audits, we approximated a 10% data audit to minimize the chance for data sampling bias.
Demographics, admission characteristics, and individual vital sign occurrence were assessed for the 6-month pre-intervention period. A total of 4136 vital sign occurrences were collected in the pre-intervention period. 184 unique patients accounted for 843 total admissions. The majority of encounters resulted in discharge (n = 832, 98%), with the remainder transferred for higher level of care (n = 8, < 1%) or missing disposition (n = 3, < 1%). In the post-intervention period, a total of 8 data audits were completed over the course of 14 weeks. Each data audit reviewed between 35 to 78 vital sign occurrences with the number dependent upon the ward volume on the day of data collection (Table 3) for a total of 438 vital sign occurrences over the course of the post-intervention period (Fig. 2). Fig. 3 demonstrates the frequency of individual vital sign documentation per vital sign occurrence in both the pre-intervention post-intervention data audit periods. During the post-implementation period, there was a substantial improvement in documentation of individual vital signs, with all vital signs being collected with over 95% compliance at the end of the three month post-intervention period. A statistical process control chart (SPC) (Fig. 4) evaluated the frequency of completed PEWS vital signs documentation per vital sign occurrence. In the post-intervention period, frequency of complete PEWS vital sign documentation reached a mean of 89.9% (Fig. 4). The frequency and accuracy of PEWS scores steadily increased during the post-implementation period (Table 3), consistently above 89% in both categories from data audit four onward. Accuracy of PEWS scoring was unable to be assessed during week 1 and 2 of data audits due to challenges with PEWS integration into the existing EMR (PDSA cycle 1) which were resolved by the 3rd week of data auditing (PDSA cycle 2).
Utilizing a partnership approach between physicians and nurses, we collectively developed and implemented a contextually adapted resuscitation program and adapted and operationalized an early warning score system into the ward nursing and physician workflow. Using standard QI methodology including a needs assessment, PDSA cycles for implementation, and SPC charts to plot measurement, we dramatically improved vital sign collection for all vital sign occurences, significantly improved complete vital sign collections (all six vital signs collected during a vital sign occurrence), and sustained frequent and accurate PEWS scoring throughout the post-implementation data auditing period. Previous studies have shown that early warning score tools can accurately identify patients at risk for clinical deterioration in low-resource settings , thus providing nurses with objective data to identify patients requiring physician or resident bedside assessments. Our initiative demonstrates a successful approach to implementing a pediatric early warning tool in a low-resource setting, using easily replicatable interventions.
In low- and lower-middle income countries and economies, limitations on resource availability predispose patients to higher morbidity and mortality and contributes to substandard quality of care. In the occupied Palestinian territory, structural determinants of health have led to a fragmented health care system defined by severe resource limitations including health care workforce shortages. Systems-based partnership initiatives, such as our resuscitation and early warning score initiative, support capacity building of the health system to address these limitations. Effective global health partnership interventions should start with effective planning through robust needs assessments. This includes aligning requests and solutions with local health partners, adaptation of external programs to the local context, and assessing for feasibility and sustainability. Incorporating contextual nuances into the education and implementation, such as medication or equipement availability, shift coverage and timing when developing escalation algorithms, PEWS data documentation location, among others, help to ensure uptake and sustainability of programming. From our experience, on-the-ground implementation support and multiple early, rapid PDSA cycles were able to effectively recognize and respond in real time to challenges identified during the period of shoulder-to-shoulder support. Examples of this included ideal location of PEWS documentation within the EMR, which required multiple, rapid iterations as well as a flexible care model. Delays in these cycles could have resulted in delayed or failed ward uptake of the PEWS system.
Our interventions maintained the longitudinal compliance of a PEWS tool by engaging successfully in a partnership consisting of three health-focused organizations. Nursing staff were able to consistently collect all vital signs and score PEWS accurately during the majority of vital sign occurrences. Explanations for the encouraging results are likely due to the tailoring of ward needs to the delivered intervention, the prioritization of physician and nursing education by the ward leadership, buy in from all stakeholders, and supportive ward leadership throughout the implementation process.
The process measures in this study, including the evaluation of the implement of the PEWS tool and improving baseline vital signs collection, are a foundational first step to evaluating the effects of PEWS systems—the combination of PEWS tool, escalation algorithms, and physician and nursing clinical training—on patient level clinical outcomes. The distinction between PEWS tools and systems is important as PEWS scores are integral in the ability to identify patients at risk for clinical deterioration but cannot function without the ability to alert skilled providers to those patients via escalation algorithms. Previous studies have demonstrated that a modified PEWS tool—a score which retained key elements of traditional PEWS but adjusted for nursing knowledge-base, vital sign limits per hospital standards, and practice variations—decreased clinical deterioration events and PICU utilization in an LMIC oncology inpatient setting when integrated into a PEWS system . Furthermore, given that pediatric cancer patients are at high risk for severe infection during treatments such as chemotherapy, PEWS systems may be of specific importance to identifying, triaging, and responding to this higher risk population. In the pediatric ward setting, an ongoing study to evaluate the PEWS system implementation on clinical outcomes, including time to antibiotics, time to fluid resuscitation, and mortality in patients with elevated PEWS scores, is currently ongoing.
Utilizing PEWS systems for risk stratification of clinically deteriorating patients may also help address patient volume burdens placed on health settings that suffer staff shortages. In the ward, the morning/early afternoon shift relies on two oncologists providing direct patient care to inpatients, outpatients, and the infusion center. During the afternoon/overnight shift, one resident covers all pediatric wards (floor, neonatal intensive care unit, emergency department, and the ward). Given the potential burden of volume of patients a physician or resident may be responsible for, PEWS may provide nurses with an objective triaging tool to efficiently risk stratify and prioritize patients at risk for clinical deterioration, leading to more efficient nursing and physician workforce utilization. Yet, it should be noted that a PEWS system that alerts providers to evaluate patients at the bedside that ultimately do not require interventions could potentially place an unintended strain on ward workforce, which is an important balancing measure in a PEWS system implementation. In our clinical setting, further evaluation of PEWS escalation ‘triggers’ and related clinical interventions should be longitudinally assessed to evaluate for unintended negative consequences of the PEWS intervention.
There are several limitations to our study. Post intervention vital signs and PEWS data relied on random sampling of limited vital signs occurrences, thus potentially overestimating the compliance and accuracy of vital signs and PEWS scores, although the randomized selection of charts to be audited and the variation in days between audits should have minimized the likelihood of this phenomenon. Selection bias in data audits might also have resulted in the assessment of vital sign collection from a limited number of nurses and not reflect departmental practices as a whole. Yet, the random selection of data audit days should limit the potential for this bias. There may also be limitations to generalizability of the findings given the implementation was limited to a pediatric ward.
Implementation of a PEWS scoring tool into practice in an LMIC pediatric hematology/oncology ward is feasible and can foster effective practice change by improving the frequency of nursing vital sign collection and generate an accurate PEWS scores, which are key process measures for successful implementation of an early warning system. PEWS education, job aids, and frequent QI audits provide robust support and feedback to nurses and physicians to improve and maintain practices in real-time.
Availability of data and materials
The datasets used and/or analyzed during the current study may be available from the corresponding author on reasonable request.
- BCH GHP:
Boston Children’s Hospital Global Health Program
Pediatric early warning scores
- MOH :
Ministry of Health
occupied Palestinian territories
Low or Lower Middle Income Country
Pediatric intensive care unit
Case report form
United Nations Inter-agency Group for Child Mortality Estimation (UN IGME), ‘Levels & Trends in Child Mortality: Report .
You D, Hug L, Ejdemyr S, Idele P, Hogan D, Mathers C, et al. Global, regional, and national levels and trends in under-5 mortality between 1990 and 2015, with scenario-based projections to 2030: a systematic analysis by the UN inter-agency Group for Child Mortality Estimation. Lancet. 2015;386(10010):2275–86. https://doi.org/10.1016/S0140-6736(15)00120-8.
Weltgesundheitsorganisation, ed. State of Inequality: Reproductive, Maternal, Newborn and Child Health: Interactive Visualization of Health Data. Buch: ... WHO; 2015.
Forum of International Respiratory Societies, European Respiratory Society. The Global Impact of Respiratory Disease.; 2017.
Wardlaw TM, Johansson EW, Hodge MJ, UNICEF, Division of communication, World Health Organization. Pneumonia: the forgotten killer of children. UNICEF : Geneva, Switzerland : world health Organization; 2006.
Nolan T, Angos P, Cunha AJ, et al. Quality of hospital care for seriously ill children in less-developed countries. Lancet. 2001;357(9250):106–10. https://doi.org/10.1016/S0140-6736(00)03542-X.
Ralston ME, Day LT, Slusher TM, Musa NL, Doss HS. Global paediatric advanced life support: improving child survival in limited-resource settings. Lancet. 2013;381(9862):256–65. https://doi.org/10.1016/S0140-6736(12)61191-X.
Molyneux E, Ahmad S, Robertson A. Improved triage and emergency care for children reduces inpatient mortality in a resource-constrained setting. Bulletin of the World Health Organization. Published online 2006:6.
Duncan H, Hutchison J, Parshuram CS. The pediatric early warning system score: a severity of illness score to predict urgent medical need in hospitalized children. J Crit Care. 2006;21(3):271–8. https://doi.org/10.1016/j.jcrc.2006.06.007.
Haines C, Perrott M, Weir P. Promoting care for acutely ill children—development and evaluation of a Paediatric early warning tool. Intensive and Critical Care Nursing. 2006;22(2):73–81. https://doi.org/10.1016/j.iccn.2005.09.003.
Parshuram CS, Hutchison J, Middaugh K. Development and initial validation of the bedside Paediatric early warning system score. Crit Care. 2009;13(4):R135. https://doi.org/10.1186/cc7998.
Sefton G, McGrath C, Tume L, Lane S, Lisboa PJG, Carrol ED. What impact did a Paediatric early warning system have on emergency admissions to the paediatric intensive care unit? An observational cohort study. Intensive Crit Care Nurs. 2015;31(2):91–9. https://doi.org/10.1016/j.iccn.2014.01.001.
Bonafide CP, Localio AR, Roberts KE, Nadkarni VM, Weirich CM, Keren R. Impact of rapid response system implementation on critical deterioration events in children. JAMA Pediatr. 2014;168(1):25–33. https://doi.org/10.1001/jamapediatrics.2013.3266.
Country Cooperation Strategy for WHO and the Occupied Palestinian Territory 2017–2020. Cairo: WHO Regional Office for the Eastern Mediterranean; 2017. Licence: CC BY-NCSA 3.0 IGO.
Right to Health in the Occupied Palestinian Territory: 2018. Cairo: WHO Regional Office for the Eastern Mediterranean; 2018. Licence: CC BY-NC-SA 3.0 IGO.
Rosman SL, Karangwa V, Law M, Monuteaux MC, Briscoe CD, McCall N. Provisional validation of a pediatric early warning score for resource-limited settings. Pediatrics. 2019;143(5):e20183657. https://doi.org/10.1542/peds.2018-3657.
KoBo Toolbox: Kobo Toolbox; 2012. Http://Www.Kobotoolbox.Org.
Rosman S. Validation of a pediatric early warning score for resource-limited settings (PEWS-RL). Unpublished.
Agulnik A, Mora Robles LN, Forbes PW, Soberanis Vasquez DJ, Mack R, Antillon-Klussmann F, et al. Improved outcomes after successful implementation of a pediatric early warning system (PEWS) in a resource-limited pediatric oncology hospital: improved outcomes using PEWS in Guatemala. Cancer. 2017;123(15):2965–74. https://doi.org/10.1002/cncr.30664.
We would like to acknowledge the Palestine Children’s Relief Fund for their support in undertaking this project.
No funding sources.
Ethics approval and consent to participate
The study was approved by Boston Children’s IRB and Palestine MOH IRB. This was a retrospective study in nature thus no direct patient consent (written or verbal) was required.
Consent for publication
No known conflicts of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Mills, D., Schmid, A., Najajreh, M. et al. Implementation of a pediatric early warning score tool in a pediatric oncology Ward in Palestine. BMC Health Serv Res 21, 1159 (2021). https://doi.org/10.1186/s12913-021-07157-x
- Quality improvement
- Workforce and workload
- Developing countries