- Study protocol
- Open Access
A cluster randomized controlled trial for the Evaluation of routinely Measured PATient reported outcomes in HemodialYsis care (EMPATHY): a study protocol
BMC Health Services Research volume 20, Article number: 731 (2020)
Kidney failure requiring dialysis is associated with poor health outcomes and health-related quality of life (HRQL). Patient-reported outcome measures (PROMs) capture symptom burden, level of functioning and other outcomes from a patient perspective, and can support clinicians to monitor disease progression, address symptoms, and facilitate patient-centered care. While evidence suggests the use of PROMs in clinical practice can lead to improved patient experience in some settings, the impact on patients’ health outcomes and experiences is not fully understood, and their cost-effectiveness in clinical settings is unknown. This study aims to fill these gaps by evaluating the effectiveness and cost-effectiveness of routinely measuring PROMs on patient-reported experience, clinical outcomes, HRQL, and healthcare utilization.
The EMPATHY trial is a pragmatic multi-centre cluster randomized controlled trial that will implement and evaluate the use of disease-specific and generic PROMs in three kidney care programs in Canada. In-centre hemodialysis units will be randomized into four groups, whereby patients: 1) complete a disease-specific PROM; 2) complete a generic PROM; 3) complete both types of PROMs; 4) receive usual care and do not complete any PROMs. While clinical care pathways are available to all hemodialysis units in the study, for the three active intervention groups, the results of the PROMs will be linked to treatment aids for clinicians and patients. The primary outcome of this study is patient-provider communication, assessed by the Communication Assessment Tool (CAT). Secondary outcomes include patient management and symptoms, use of healthcare services, and the costs of implementing this intervention will also be estimated. The present protocol fulfilled the Standard Protocol Items: Recommendations for Intervention Trials (SPIRIT) checklist.
While using PROMs in clinical practice is supported by theory and rationale, and may engage patients and enhance their role in decisions regarding their care and outcomes, the best approach of their use is still uncertain. It is important to rigorously evaluate such interventions and investments to ensure they provide value for patients and health systems.
At the end of 2018 there were over 40,000 Canadians with kidney failure requiring dialysis; a 35% increase from the previous decade . Kidney failure requiring dialysis is associated with high symptom burden and poor health-related quality of life (HRQL)    . Patients can experience a wide variety of symptoms ranging from physical to psychological in nature . These symptoms are often under-recognized by kidney care teams, which may be related to breakdown in communication . In a recent priority setting exercise, the top research priority for patients with kidney failure requiring dialysis in Canada was to enhance communication between patients and clinicians to maximize patient participation in decision making and facilitate self-management . Furthermore, five of the top 10 research priorities for patients with kidney failure requiring dialysis focus on reducing symptoms and improving HRQL .
The priorities of enhancing communication and improving symptoms and HRQL are potentially well aligned. In other chronic diseases, the incorporation of routinely measuring and reporting symptoms and HRQL improved communication [9,10,11]. Without routine symptom monitoring, this situation is unlikely to change. Further, the presence and severity of symptoms can change considerably over a short period of time in dialysis patients, emphasizing the need for regular surveillance of symptoms. Incorporation of patient-reported outcome measures (PROMs) into dialysis care could address this problem.
What are PROMs?
PROMs are reports coming directly from patients about how they function or feel in relation to a health condition and its therapy, without interpretation of the patient’s responses by a clinician or anyone else . PROMs present an opportunity to explore aspects of patients’ health that may otherwise be overlooked. They capture patients’ experiences of symptoms and impact of disease on functioning and can support clinicians to monitor disease progression and facilitate patient-centered care. It is proposed that PROMs could act to improve the quality of care in the same way as any other benchmarking tool , and some suggest that PROMs have the potential to transform healthcare . There has been an increased use of PROMs in health systems around the world, a movement initiated and led by the National Health System (NHS) in the UK . In Canada, the Canadian Institute for Health Information (CIHI) and the Alberta PROMs and EQ-5D Research and Support Unit  are leading similar efforts to establish provincial and national PROMs strategies and support the use of PROMs in the Canadian healthcare system [15, 16].
PROMs in clinical practice
Although PROMs have become widely used in clinical effectiveness research, their usefulness in clinical settings is still unclear. In the clinical context, ideally PROMs results would be fed back to clinicians who would use that information to improve patient care, much like routine clinical laboratory measures . However, since PROMs were generally not developed or applied for this purpose, little is known about how they perform in this context. A limited number of randomized controlled trials have examined the impact of using PROMs in clinical practice, particularly in oncology, and found that the routine measurement of PROMs, followed by reporting of results to clinicians and patients, had a positive impact on patient-provider communication, patient satisfaction with care, and HRQL [9, 18,19,20,21,22,23,24]. Similarly, in adolescents with type 1 diabetes, the periodic monitoring and discussion of PROMs led to improvement in psychosocial wellbeing, however, this benefit was not sustained after the completion of the intervention [25, 26]. In patients with gastrointestinal disorders, the one-time use of a disease-specific PROM had no impact on patient satisfaction, provider interpersonal skills, or shared decision-making .
Systematic reviews of these clinical trials and others suggest that feedback of PROMs to clinicians have an impact on processes of care, may enhance patient-provider relationship, improve communication, and support shared decision-making [27,28,29,30]; however, their impact on health outcomes is less apparent [27, 31,32,33]. Aside from these limited controlled studies examining the clinical effectiveness of PROMs themselves as an intervention, few studies have examined their value and cost-effectiveness to justify their use in clinical practice .
Additionally, qualitative studies that explored practical issues surrounding the implementation of PROMs in clinical practice found that demonstrating clinical utility to clinicians is a key factor for successful and effective implementation of PROMs [35, 36]. Other factors identified include the use of technology, having the appropriate infrastructure in place, embedding PROMs into routine workflow, improving the interpretability of PROMs data, and engaging clinicians in the planning phase of the intervention . Finally, a key element identified by clinicians is the linkage of PROMs with tools to support the clinical management of symptoms or problems identified by assessments . For example, if a patient identifies problems with anxiety or depression, clinicians should have readily available treatment protocols to follow to alleviate those symptoms [35, 36]. Although the current evidence on the impact of PROMs on patient outcomes is relatively limited and has several methodological limitations, it generally suggests that the effectiveness of PROMs depends on the function of the PROM and the type of feedback provided.
PROMs can be generic or disease-specific, and each has several advantages and disadvantages . Generic measures are used to compare outcomes across different populations and interventions, particularly for cost-effectiveness studies and health system decision-makers. Disease-specific measures assess the health status of a specific patient population and may be more sensitive for the detection and quantification of changes that are important to clinicians or patients. A common recommendation in clinical research with PROMs is to combine both a generic and a disease-specific measure for use in a particular patient population, especially when PROMs are used to monitor health and how it changes over time. The presumed benefit of this combination is to broaden the scope of measurement and allow for different uses of the measures (e.g., comparison within versus across disease populations). However, evidence to support this recommendation is limited. In fact, in a study examining the usefulness of generic and disease-specific measures of health status in patients with Alzheimer’s, the EQ-5D (a generic measure) was found to be a suitable alternative to a disease-specific measure of health . Another study reported that generic and disease-specific measures performed similarly in assessing changes in HRQL in patients with kidney failure requiring hemodialysis . There is usually some level of overlap between generic and disease-specific measures, and the need for combining both types of measures depends on the target patient population and the constructs assessed by each measure. Given the workload associated with collecting and using PROM data in clinical settings, it is imperative to explore whether one single PROM is sufficient to improve patient-provider communication, processes of care, and ultimately patient outcomes, before recommending the use of a combination of generic and disease-specific measures.
Justification for PROMs in dialysis care
One of the key areas where PROMs have been implemented is in patients with kidney failure requiring dialysis. The prevalence of chronic kidney disease (CKD) is relatively high in Canada (12.5% or 3 million Canadian adults) and its advanced stages can lead to kidney failure . Kidney failure requiring dialysis is expensive and is associated with poor health outcomes and HRQL. To report symptom burden and HRQL alongside clinical and laboratory measures, it is imperative to measure and report what is important to patients. PROMs could be used to evaluate and monitor patients’ health, inform care planning, and facilitate the introduction of treatments for patients with kidney failure requiring dialysis .
Moreover, the implementation of PROMs assessments and integration into the workflow of a busy clinical setting such as dialysis clinics requires a feasible system to capture and report PROMs. This would be facilitated by the development of PROM report cards to present results to clinicians and patients in a manner that is easily understood and interpretable, analogous to clinical laboratory measures used in the hemodialysis setting. We believe that routine hemodialysis care provides the ideal setting for implementation and rigorous evaluation of a PROM intervention, and as such, we are conducting a pragmatic, cluster randomized controlled trial (RCT) of a PROMs intervention. Our comprehensive evaluation will assess the effectiveness of this intervention, the adoption and implementation in the busy clinical setting of hemodialysis units, as well as the cost-effectiveness of PROMs in routine hemodialysis care.
Aims of the research
The overall aim of this study is to explore the usefulness of integrating PROM assessments in the clinical management of hemodialysis patients. Specifically, the primary objective is to determine the effects of routine measurement and reporting to clinicians of PROMs, namely, the Edmonton Symptom Assessment System- revised: Renal (ESAS-r: Renal)  , or the Integrated Palliative care Outcome Scale – Renal (IPOS-Renal)  and/or the EQ-5D-5L  on patient-reported experience, particularly patient-clinician communication, measured by the Communication Assessment Tool (CAT), for patients with kidney failure requiring hemodialysis. The secondary objectives are as follows:
To determine if there is a difference in patient-reported experience and other outcomes induced by a condition specific (ESAS-r: Renal /IPOS-Renal) versus a generic (EQ-5D-5L) PROM;
To determine the effects of routinely measuring and reporting the ESAS-r: Renal/IPOS-Renal and/or the EQ-5D-5L on the clinical management of symptoms;
To determine the effects of routinely measuring and reporting the ESAS-r: Renal /IPOS-Renal and/or the EQ-5D-5L on overall HRQL;
To determine the effects of routinely measuring and reporting the ESAS-r: Renal/IPOS-Renal and/or the EQ-5D-5L on symptom burden, mental health outcomes and healthcare utilization;
To determine the cost effectiveness of routinely measuring the ESAS-r: Renal/IPOS-Renal and/or the EQ-5D-5L for patients with kidney failure requiring hemodialysis;
To explore the perspectives and experience of patients and clinicians with routine measurement and reporting of PROMs in clinical practice.
To date, the effect of PROMs on patient outcomes were generally assessed in conventional parallel-group RCTs of individuals. Few convincingly demonstrated benefits to patient-important outcomes, which might be in part due to contamination effects and small sample sizes . This study uses a cluster RCT design, with clustering occurring at the level of individual hemodialysis units, to reduce potential contamination between the study groups (Fig. 1). EMPATHY is an open-label RCT, since blinding is not feasible . Consequently, research staff, clinicians, and all patients are aware of group allocation. Dialysis units, stratified by three regions (Northern Alberta, Southern Alberta, and Ontario), were randomly allocated using random number sequences with all units from a region randomized on the same day. Dialysis units were allocated in a 1:1:1:1 ratio into four groups:
Group 1: Patients complete ESAS-r: Renal or IPOS-Renal
Group 2: Patients complete EQ-5D-5L
Group 3: Patients complete both ESAS-r: Renal or IPOS-Renal and EQ-5D-5L
Group 4: Usual care (i.e. the control group). To assess study outcomes, patients complete the PROMs at baseline, 6 and 12 months, but these data are not reported back to clinicians or patients.
Groups 1–3 will collect their allocated PROM(s) from all eligible patients every 2 months, for 12 months (a total of 7 assessments). PROMs are administered by pen and paper or through an electronic platform on a tablet in accordance with hemodialysis unit resources. In Alberta, the electronic medical records (EMRs) generate a PROM report card with a symbol scheme for nurses to follow up with clinical management where applicable. The symbol scheme comprises of a check mark for no problems/symptoms (ESAS-r: Renal = 0, IPOS-Renal = 0, EQ-5D-5L = 1) a caution sign for mild problems/symptoms (ESAS-r: Renal = 1–3, IPOS-Renal = 1, EQ-5D-5L = 2) and a stop sign for moderate-severe symptoms/problems (ESAS-r: Renal = 4–10, IPOS-Renal = 2–4, EQ-5D-5L = 3–5). The PROM report card is printed and added to the patient’s medical chart for review by clinicians (i.e., nurses, nephrologists, and allied health professionals). The report displays each patient’s most recent PROM scores in comparison with their previous scores, much like lab test results. A sample PROM report card is shown in Fig. 2. In Ontario, EMRs do not have report card capabilities and units store the paper-completed PROM questionnaires in the patient’s medical chart, along with previously completed questionnaires for review by clinicians and the patient. In all locations, PROMs are accompanied by treatment aids for some symptoms assessed (pain, pruritus, restless legs, sleeping problems, tiredness, nausea, shortness of breath, anxiety, and depression). Patients are also offered a copy of their PROM report card (if possible) and patient-facing self-management guides for symptom management. This trial is intended to generate evidence on the implementation within the health systems. While we will aim to maintain the study design and key elements of the intervention as consistently as possible, we will allow for the interventions to be tailored to local health systems, accommodating considerations such as the clinical work flow, information systems and capabilities, and internet/WiFi access (Table 1).
We hypothesize that the measurement and reporting of the ESAS-r: Renal or IPOS-Renal, EQ-5D-5L or a combination every 2 months will lead to improved patient-clinician communication in dialysis units. We also hypothesize that this will lead to reduced symptoms and improved mental health and HRQL. Furthermore, we anticipate that this intervention will be cost-effective and will be acceptable and feasible in the hemodialysis units’ environment.
Setting and population
Dialysis programs frequently provide administrative functions for separate hemodialysis units; however, the unit of randomization is the dialysis unit, not dialysis programs or individual patients. In total, 44 units from three renal programs were randomly allocated to one of the four study groups. We enrolled all eligible hemodialysis units in Alberta, including 17 hemodialysis units from the Alberta Kidney Care - North (AKC-N) program and 12 units from the Alberta Kidney Care - South (AKC-S) program. Of Ontario’s 92 hemodialysis units, 15 enrolled in the study, based on voluntary acceptance of invitation to participate.
Dialysis unit eligibility criteria
Hemodialysis units meeting the following inclusion criteria were included in this study:
The unit provides chronic in-center hemodialysis;
Hemodialysis clinicians are willing to review individual patients’ routinely collected ESAS-r: Renal or IPOS-Renal and/or EQ-5D-5L as part of routine patient assessment;
Hemodialysis units are excluded if they meet any of the following criteria:
The unit is unable to administer the ESAS-r: Renal or IPOS-Renal and EQ-5D-5L to in-centre hemodialysis patients as part of clinical workflow to all possible patients;
The unit is part of a long-term care facility
There are 5 or fewer patients treated at that unit
Lack of WiFi connectivity (Alberta only)
Patient eligibility criteria
All patients undergoing chronic hemodialysis who are 18 years or older at the start of the study and are willing and able to complete the PROMs as part of the trial are eligible to participate in this study. Patients with cognitive impairment, undergoing acute dialysis, or transiently dialyzing in the unit are not included. Simplified Chinese, Punjabi, and Vietnamese PROMs and study surveys were provided. Patients that had other language barriers had family help them complete their PROMs assessments and/or study surveys or the health system interpretive services were used. For patients with eyesight issues, their family members or nurses helped them complete the PROMs and study surveys.
Description of PROMs
ESAS-r: Renal: The Edmonton Symptom Assessment System, or ESAS, is a clinically validated and reliable symptom measurement tool, extensively used for a number of chronic diseases including cancer, heart disease, and kidney disease (Table 2). The original tool was developed by the Regional Palliative Care Program, Capital Health in Edmonton, Alberta . The ESAS-r: Renal was modified from the original ESAS and validated for use with CKD patients in Canada to assess symptom burden . Despite its widespread use in dialysis units in Canada, the use of ESAS-r: Renal has largely been limited to measurement of symptoms without systematic reporting of results to clinicians.
The Palliative care Outcome Scale, or POS, was developed in 1999  for use with patients with advanced disease, and to improve outcome measurement by evaluating important outcomes in palliative care (Table 2). The Integrated Palliative care Outcome Scale, or IPOS, integrates questions from three different POS measures, incorporating symptoms, information needs, practical concerns, family anxieties, and overall feeling of being at peace. The IPOS-Renal contains all the elements of the IPOS, and the most common symptoms renal patients experience. The IPOS-Renal has also demonstrated validity and reliability and has been studied in dialysis and non-dialysis CKD patients in Australia .
Generic PROM: EQ-5D-5L
The EQ-5D-5L is a generic preference-based measure of HRQL and has been selected as the PROM of choice in many clinical areas and settings (Table 2). The EQ-5D-5L includes a health status classification system with five dimensions (mobility, self-care, usual activities, pain/discomfort, anxiety/depression), each with five levels of problems (1 = none, 2 = mild, 3 = moderate, 4 = severe, 5 = extreme), describing 3125 distinct health states  . An index score for each health state can be calculated using population preferences , which can then be used to estimate quality-adjusted life years (QALYs) in economic evaluations of health interventions or innovations . Empirical evidence on the clinical application of the EQ-5D-5L with regards to individual patient care is limited, but a recent review of PROMs in CKD recommended the EQ-5D-5L for use in this patient population .
Treatment aids are assessment and treatment resources developed by expert clinicians for the management of certain symptoms and disorders. Each renal program developed treatment aids specific to symptoms assessed by the ESAS-r: Renal or IPOS-Renal (e.g., itchiness, restless leg syndrome, nausea) and the EQ-5D-5L (e.g., pain, anxiety/depression). Treatment aids are intended to support clinicians in the assessment and management of symptoms identified by these PROMs and will be made available for all clinicians at all study sites regardless of the group to which they are randomized.
Patient-facing materials are also provided to patients, to help them better understand the reasons why their symptom/problem may have developed, treatment options that might be considered, as well as self-care activities they can undertake to alleviate the symptom or problem.
Nurses were responsible for administering PROMs to patients. They received training on the use of the ESAS-r: Renal or IPOS-Renal and/or EQ-5D-5L, depending on their study group allocation, which included an overview of the instruments, interpretation of patients’ scores from one measurement and changes in scores over time, the use of PROMs data during clinical visits (i.e. referring to treatment aids or clinical guidelines), as well as data entry in the EMR, if applicable. A toolkit for using PROMs in the dialysis unit was developed and shared with all clinicians in all participating dialysis units. A train-the-trainer model was adopted, where clinical nurse educators will receive training and subsequently provide on-site training to other nurses. Training was administered in a series of webinars and workshops. Figure 3 demonstrates the workflow whereby nurses were trained. Nephrologists received training through in-person presentations and print materials.
Primary and secondary outcome measures
The primary outcome in this study is patient-provider communication, which is assessed using a modified version of the Communication Assessment Tool (CAT) .
Secondary outcomes include:
HRQL, assessed using the EQ-5D-5L. .
Patient experience with their disease management, assessed using the Patient Assessment of Chronic Illness Care 11-items questionnaire (PACIC-11) .
Healthcare utilization (e.g., physician, emergency department and hospitalization), assessed through linkage with provincial administrative health care databases at the individual patient-level.
Demographic characteristics and the outcome measures are collected at baseline, 6-months, and 12-months. Data on medical history, dialysis history, and clinical measures are collected from patient charts.
Sample size and power
The EMPATHY trial is a pragmatic clinical trial with a sample size determined based on eligibility of hemodialysis units and its encompassing patients. Our intention was to enroll all eligible dialysis units within the province of Alberta, and a sample of eligible units in Ontario. Therefore, we did not calculate an optimal sample size for adequate power to detect statistical significance. From 44 dialysis units, we will be utilizing data from approximately 3000 patients. Based on an estimated intraclass correlation of 0.1 and a standard deviation for the CAT (primary outcome measure) of 0.74 (based on pilot data from HD patients in Alberta), this will provide 80% power to detect an effect size of 0.23 with a two-tailed test at typical level of significance, α = 0.05 . While this is a small effect size for the primary outcome, we are nonetheless confident in the design, given that this is a pragmatic quality improvement study, involving 44 dialysis units from three renal programs with several secondary objectives related to patient experience and outcomes.
Quantitative trial data analysis
The primary analysis will assess change in the overall CAT score, between study groups from baseline to 12 months using a linear mixed effects model, with the mean CAT for the dialysis unit at baseline as a fixed effect covariate, and dialysis unit (cluster) as a random intercept. Ideally, analysis should use individual patients’ baseline CAT and follow-up CAT rather than the unit mean baseline CAT to compute change over time. However, due to concerns by the regional renal programs to maintain anonymity of experience data as well as the natural drop-in and drop-out of patients in the dialysis units due to patient deaths and initiation of new patients onto dialysis, individual CAT surveys were made completely anonymous. As such, our primary analytic plan will be based on changes in mean CAT score at the unit level, from baseline to 12 months. The primary analysis will be based on all of the PROMs intervention groups combined compared to the control (no PROMs intervention) group as fixed effects. Any units that drop-out after randomization but prior to initial data collection will be excluded. We will employ last observation carried forward methods to impute missing data where appropriate. As one of our secondary objectives, assessing differences between specific and generic PROMs, we will compare the individual interventions groups (i.e., ESAS-r: Renal /IPOS alone, EQ-5D-5D alone, or the combination) to the control group.
As part of our secondary analyses, we will also compare the proportion of patients reporting symptoms/problems on the ESAS-r: Renal or IPOS-Renal and EQ-5D-5L and the management of such symptoms/problems. Through clinical records, we will link PROMs scores with the use of specific symptomatic treatments as per the treatment aids. We will obtain medication lists, allied health referrals, and chart notes to capture how symptoms are being treated. Because the PROMs data will be collected as part of the clinical record, and linked with individual patient-level data, we will be able to analyze these data using logistic mixed-effects regression models adjusting for individual patient-level covariates. Each renal program (i.e., AKC-N, AKC-S and ORN) will be analyzed separately then we will use meta-analyses to analyze the data from each renal program.
A cost-effectiveness analysis will be employed of the different study groups, compared to the usual care group as the reference case. The incremental costs and incremental QALYs of the different interventions relative to the usual care group will be compared to calculate incremental cost per QALYs gained ratios [60, 61]. Costs will be estimated for patients in the different study groups, including treatments received by patients, taking the perspective of the provincial health care system, personnel costs for training and delivery of the PROMs assessments, and subsequent health care utilization for patients in each of the study groups. Health care utilization will be estimated through linkage with provincial administrative health care databases at the individual patient-level. Costs associated with provision of the intervention, including training and nursing time, will be estimated through observations for time and valued using standard staffing wage scales for the respective geographic regions. QALYs will be calculated based on changes in the EQ-5D-5L index score, using the Canadian preference weighting . Discounting or inflationary factors will not be applied as the analysis will be limited to the period of the study itself (i.e., 12 months). Self-reported survey and clinical data will be linked to administrative data on healthcare utilization (e.g., number of hospitalizations, number of emergency department visits, number of family physician and specialist visits, cost), one-year prior and one-year after the intervention starts. Linkage to lab and medication datasets will also be performed where available.
Separate qualitative assessments will be undertaken within the regional dialysis programs. Within AKC-N, we will employ an interpretative description approach , ascertaining the experiences of patients and clinicians that will participate in the trial, in the routine measurement and reporting of PROMs and the use of the treatment aids, and in relation to communication and clinical management. Interpretative description is an inductive approach used to understand clinical phenomenon with the end goal of informing practice. Of particular interest will be an exploration of how the PROMs intervention is implemented within the workflow of the different clinical settings amongst the dialysis units. This qualitative study will enhance the larger trial by examining patient and clinician experiences regarding the study’s primary outcome of patient-provider communication. A variety of data collection methods will be employed, including clinic observations, interviews, and open-ended comments from patients on the study survey. These data will be managed using Atlas.ti and thematically analyzed.
A distinct qualitative study will also be undertaken within AKC-S. Using a qualitative descriptive approach [64, 65], we will explore the PROM implementation process among hemodialysis units randomized to an intervention group of the trial. In this study, our aim is to characterize perceived barriers and facilitators to effective and sustainable use of the PROM intervention in routine hemodialysis care. Interviews and focus groups with hemodialysis patients and clinicians will be guided by the Consolidated Framework for Implementation Research (CFIR), whose domains (i.e., intervention characteristics, outer/inner setting, individual characteristics, and implementation process) can be used to inform and/or evaluate implementation strategies . Directed observations in hemodialysis units of PROMs assessments and feedback will provide additional contextual information about the setting, workflow, and PROMs integration in care. Data will be managed using NVivo 11 and thematically analyzed .
Patient engagement in this study was guided by Can-SOLVE CKD Network strategies and guidelines. Development of the research question, outcome measures, and study design were informed by a series of meetings held with the EMPATHY research team, which included people with lived experience of kidney failure on hemodialysis. These patient partners helped develop and review all the symptom information handouts for patients. They also evaluated the overall burden of study participation during the design process and helped determine the type and frequency of PROM administration. Additionally, patient partners prioritized focusing on mental health in our intervention and our study outcomes. Mental health treatment aids were developed and mental health outcome measures were added to the study survey as a result. The scope of this study was significantly impacted by the addition of patient partners on the study team.
The three renal programs implemented EMPATHY on different timelines. AKC-N implemented EMPATHY in September 2018 with all units starting at once. AKC-N completed the 1-year trial period in October 2019. AKC-S and ORN implemented EMPATHY with a phased approach, meaning groups of units started the trial at different points. AKC-S started phasing units into the trial in January 2019 and ORN in April 2019. ORN and AKC-S will complete the trial by the end of 2020.
Data storage and security
Any hard copies of PROMs reports are kept in patients’ charts. Study data is stored in the EMR (Alberta) or an Excel file (Ontario) and transferred to the research team using secure mechanisms, with appropriate encryption. Electronic data is password protected, saved on a secure university server and accessed only by members of the research team. All research team members follow the relevant codes of practice concerning confidentiality, information security management and records management.
Outcome measure data was collected by paper surveys or by iPads and directly entered into REDCap, a web-based application for electronic data collection. The REDCap database is saved on a secure Alberta Health Services (health authority) server. Using REDCap limits the amount of paper-based data, further ensuring data integrity and safety. Paper surveys are sent to the research team by courier mail, fax with cover-page, encrypted email, or in-person and are manually entered into the REDCap (Alberta) or Excel (Ontario) database by a member of the research team. A 25-year data retention policy will be adopted for hard-copy data and electronic records, as per Health Canada regulations.
The present protocol fulfilled the Standard Protocol Items: Recommendations for Intervention Trials (SPIRIT) checklist.
The evaluation of PROM interventions such as those described in this protocol can be challenging given the nature of clinical practice in busy hemodialysis units, the variations in organization and clinical workflow across units, as well as regional programs. We will study the implementation of PROM interventions in real-world health settings. Thus, we will adopt a pragmatic approach and accept the necessary variations in implementation, while attempting to retain as many of the core elements of the interventions across the programs. Furthermore, given the desire to maintain anonymity of patient-reported experience, our primary analytic strategy will be limited to comparison of the outcome measures at the group level. These pragmatic approaches to the design of this implementation study limit the rigour of the evaluation of the efficacy of PROM interventions, but on the other hand, enhance our evaluation of real-world effectiveness. Along with the planned qualitative assessments, the information gained from this implementation trial will provide valuable information for the regional dialysis programs in enhancing and sustaining the PROM interventions in the most appropriate manner.
A question that is often raised with the implementation of PROM programs is the choice of the measures. We are choosing to implement a combination of specific and generic measures. While a secondary objective is to compare these types of measures, it is possible that the differences (e.g., breadth or scope) between measures matter less than either approach simply serving as a mechanism for engaging patients and clinicians in a more focused discussion on symptoms or problems. This is the intention with our primary analytic strategy of combining the intervention groups to compare against the control group with no PROM intervention. There may be other reasons, beyond the patient-clinician interaction, however, that guide decisions on the choice of measures. For example, programs may be interested in evaluating the overall quality and efficiency of services, and may therefore be interested in a measure that provide data to support economic evaluation. In this case a generic preference-based measure such as the EQ-5D provides an overall index score compatible with that purpose, which may not be available with measures assessing symptom burden.
A further question faced by clinical programs is the choice of any particular measure for either generic or specific health status. Indeed, within our trial protocol, we will have two different instruments assessing symptom burden. The ESAS-r: Renal will be used in AKC-North and the ORN, based on previous use of this measure in dialysis and other programs locally and across Canada; the IPOS will be implemented in AKC-South, where there was no previous PROM collection, and as a choice informed by regional patient engagement. Similarly, we chose to use the EQ-5D-5L as a generic measure, which was recommended for this patient population in a review paper that was available at the time we planned our interventions . A more recent review of measurement properties for PROMs for adults with chronic kidney disease recommends the KDQOL-SF or KDQOL-36 . The authors note, however, that the available evidence for the reliability and validity of these measures is limited, calling for further research to close this information gap . Nonetheless, as we noted earlier, it may be that subtle differences between measures matter less than implementing any measure and linking these with actionable clinical pathways to alleviate bothersome symptoms and improve patients’ well-being.
Incorporating PROMs into clinical practice seems like an entirely appropriate strategy to engage patients and enhance their role in decisions regarding their care and outcomes. However, how this is best done is uncertain and requires a substantial allocation of healthcare resources. It is important to rigorously evaluate such interventions and investments to ensure they provide value for patients and health systems. The results of this trial will guide the use of PROMs in Alberta and Ontario dialysis units. These results will also inform the use of PROMs in routine dialysis care nationally and internationally and potentially for the care of other chronic disease patient populations. Dissemination of findings will be undertaken through meetings, reports, academic papers, and conference presentations.
Availability of data and materials
The data that supports the findings of this study are available from the University of Alberta, University of Calgary and McMaster University but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the corresponding author upon reasonable request and with permission of the aforementioned universities.
Alberta Kidney Care – North
Alberta Kidney Care – South
Communication Assessment Tool
Consolidated Framework for Implementation Research
Canadian Institute for Health Information
Chronic kidney disease
Electronic medical record
Renal: Edmonton Symptom Assessment System-revised: Renal
Generalized Anxiety Disorder 2-item
Health-related quality of life
Integrated Palliative care Outcome Scale – Renal
Ontario Renal Network
Patient Assessment of Chronic Illness Care
Patient Health Questionnaire 9-item
Patient-reported outcome measures
Quality-adjusted life year
Randomized controlled trial
Canadian Institute for Health Information. Organ replacement in Canada: CORR annual statistics, 2019. 2019 June 1, 2020]; Available from: https://www.cihi.ca/en/organ-replacement-in-canada-corr-annual-statistics-2019.
Murtagh FE, Addington-Hall J, Higginson IJ. The prevalence of symptoms in end-stage renal disease: a systematic review. Adv Chronic Kidney Dis. 2007;14(1):82–99.
Jablonski A. The multidimensional characteristics of symptoms reported by patients on hemodialysis. Nephrol Nurs J. 2007;34(1):29–37 quiz 38.
Abdel-Kader K, et al. Individual quality of life in chronic kidney disease: influence of age and dialysis modality. Clin J Am Soc Nephrol. 2009;4(4):711–8.
Lowney AC, et al. Understanding what influences the health-related quality of life of hemodialysis patients: a collaborative study in England and Ireland. J Pain Symptom Manag. 2015;50(6):778–85.
Manns B, et al. Setting research priorities for patients on or nearing dialysis. Clin J Am Soc Nephrol. 2014;9(10):1813–21.
Weisbord SD, et al. Renal provider recognition of symptoms in patients on maintenance hemodialysis. Clin J Am Soc Nephrol. 2007;2(5):960–7.
Moskovitch JT, Mount PF, Davies MRP. Changes in Symptom Burden in Dialysis Patients Assessed Using a Symptom-Reporting Questionnaire in Clinic. J Palliat Care. 2019:825859719827315.
Velikova G, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol. 2004;22(4):714–24.
Hilarius DL, et al. Use of health-related quality-of-life assessments in daily clinical oncology nursing practice: a community hospital-based intervention study. Cancer. 2008;113(3):628–37.
Dy SM, et al. Tell us™: a web-based tool for improving communication among patients, families, and providers in hospice and palliative care through systematic data specification, collection, and use. J Pain Symptom Manag. 2011;42(4):526–34.
Food and Drug Administration. Patient-reported outcomes measures: use in medical product development to support labeling claims, R.N. 74, Editor. 2009.
Devlin NJ, Appleby J. Getting the most out of PROMs: putting health outcomes at the heart of NHS decision-making. London: The King’s Fund and The Office of Health Economics; 2010.
Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
APERSU. Alberta PROMs and EQ-5D Research and Support Unit (APERSU). 2015 30, 2016]; Available from: www.apersu.ca.
Canadian Institute for Health Information. Patient Reported Outcome Measures. 2015 30, 2016]; Available from: https://www.cihi.ca/en/health-system-performance/quality-of-care-and-outcomes/outcomes/patient-reported-outcome-measures.
Greenhalgh J. The applications of PROs in clinical practice: what are they, do they work, and why? Qual Life Res. 2009;18(1):115–23.
Detmar SB, et al. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. JAMA. 2002;288(23):3027–34.
Berry DL, et al. Enhancing patient-provider communication with the electronic self-report assessment for cancer: a randomized trial. J Clin Oncol. 2011;29(8):1029–35.
Berry DL, et al. Electronic self-report assessment for cancer and self-care support: results of a multicenter randomized trial. J Clin Oncol. 2014;32(3):199–205.
Berry DL, et al. Exposure to a patient-centered, web-based intervention for managing cancer symptom and quality of life issues: impact on symptom distress. J Med Internet Res. 2015;17(6):e136.
Rosenbloom SK, et al. Assessment is not enough: a randomized controlled trial of the effects of HRQL assessment on quality of life and satisfaction in oncology clinical practice. Psychooncology. 2007;16(12):1069–79.
Velikova G, et al. Patients report improvements in continuity of care when quality of life assessments are used routinely in oncology practice: secondary outcomes of a randomised controlled trial. Eur J Cancer. 2010;46(13):2381–8.
Almario CV, et al. Impact of National Institutes of Health gastrointestinal PROMIS measures in clinical practice: results of a multicenter controlled trial. Am J Gastroenterol. 2016;111(11):1546–56.
de Wit M, et al. Monitoring and discussing health-related quality of life in adolescents with type 1 diabetes improve psychosocial well-being: a randomized controlled trial. Diabetes Care. 2008;31(8):1521–6.
de Wit M, et al. Follow-up results on monitoring and discussing health-related quality of life in adolescent diabetes care: benefits do not sustain in routine practice. Pediatr Diabetes. 2010;11(3):175–81.
Boyce MB, Browne JP. Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review. Qual Life Res. 2013;22(9):2265–78.
Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13:211.
Marshall S, Haywood K, Fitzpatrick R. Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Pract. 2006;12(5):559–68.
King S, et al. The use and impact of quality of life assessment tools in clinical care settings for cancer patients, with a particular emphasis on brain cancer: insights from a systematic review and stakeholder consultations. Qual Life Res. 2016;25(9):2245–56.
Valderas JM, et al. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res. 2008;17(2):179–93.
Gilbody SM, House AO, Sheldon TA. Routinely administered questionnaires for depression and anxiety: systematic review. Bmj. 2001;322(7283):406–9.
Gilbody SM, House AO, Sheldon T. Routine administration of health related quality of life (HRQoL) and needs assessment instruments to improve psychological outcome--a systematic review. Psychol Med. 2002;32(8):1345–56.
Price A, et al. The Arthroplasty candidacy help engine tool to select candidates for hip and knee replacement surgery: development and economic modelling. Health Technol Assess. 2019;23(32):1–216.
Boyce MB, Browne JP, Greenhalgh J. The experiences of professionals with using information from patient-reported outcome measures to improve the quality of healthcare: a systematic review of qualitative research. BMJ Qual Saf. 2014;23(6):508–18.
Antunes B, Harding R, Higginson IJ. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med. 2014;28(2):158–75.
Krawczyk M, et al. Micro-Meso-macro practice tensions in using patient-reported outcome and experience measures in hospital palliative care. Qual Health Res. 2019;29(4):510–21.
Patrick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Med Care. 1989;27(3 Suppl):S217–32.
Bhattacharya S, et al. Generic and disease-specific measures of quality of life in patients with mild Alzheimer's disease. Dement Geriatr Cogn Disord. 2010;30(4):327–33.
Laupacis A, Wong C, Churchill D. The use of generic and specific quality-of-life measures in hemodialysis patients treated with erythropoietin. The Canadian Erythropoietin Study Group. Control Clin Trials. 1991;12(4 Suppl):168s–79s.
Arora P, et al. Prevalence estimates of chronic kidney disease in Canada: results of a nationally representative survey. Cmaj. 2013;185(9):E417–23.
Perrone RD, et al. Patient-reported outcomes in clinical trials of CKD-related therapies: report of a symposium sponsored by the national kidney foundation and the U.S. Food and Drug Administration. Am J Kidney Dis. 2013;62(6):1046–57.
Davison SN, Jhangri GS, Johnson JA. Cross-sectional validity of a modified Edmonton symptom assessment system in dialysis patients: a simple assessment of symptom burden. Kidney Int. 2006;69(9):1621–5.
Davison SN, Jhangri GS, Johnson JA. Longitudinal validation of a modified Edmonton symptom assessment system (ESAS) in haemodialysis patients. Nephrol Dial Transplant. 2006;21(11):3189–95.
Raj R, et al. Validation of the IPOS-renal symptom survey in advanced kidney disease: a cross-sectional study. J Pain Symptom Manag. 2018;56(2):281–7.
Herdman M, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res. 2011;20(10):1727–36.
Fayers PM. Evaluating the effectiveness of using PROs in clinical practice: a role for cluster-randomised trials. Qual Life Res. 2008;17(10):1315–21.
Sedgwick P. What is an open label trial? BMJ. 2014;348:g3434.
Bruera E, et al. The Edmonton symptom assessment system (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care. 1991;7(2):6–9.
Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative care Core audit project advisory group. Qual Health Care. 1999;8(4):219–27.
The EuroQol Group. About EQ-5D-5L [Internet]. 2014 26, 2017]; Available from: www.euroqol.org.
Devlin NJ, Krabbe PF. The development of new research methods for the valuation of EQ-5D-5L. Eur J Health Econ. 2013;14(Suppl 1):S1–3.
Dolan P. Modeling valuations for EuroQol health states. Med Care. 1997;35(11):1095–108.
Gibbons, E., Casañas I Comabella C, and Fitzpatrick R, A structured review of patient-reported outcome measures for patients with skin cancer, 2013. Br J Dermatol,2013 168(6): p. 1176–1186.
Makoul G, Krupat E, Chang CH. Measuring patient views of physician communication skills: development and testing of the communication assessment tool. Patient Educ Couns. 2007;67(3):333–42.
Skapinakis P. The 2-item generalized anxiety disorder scale had high sensitivity and specificity for detecting GAD in primary care. Evid Based Med. 2007;12(5):149.
Kroenke K, Spitzer RL, Williams JB. The patient health Questionnaire-2: validity of a two-item depression screener. Med Care. 2003;41(11):1284–92.
Glasgow RE, et al. Use of the patient assessment of chronic illness care (PACIC) with diabetic patients: relationship to patient characteristics, receipt of care, and self-management. Diabetes Care. 2005;28(11):2655–61.
van Breukelen GJ, Candel MJ. Calculating sample sizes for cluster randomized trials: we can keep it simple and efficient. J Clin Epidemiol. 2012;65(11):1212–8.
Wozniak L, et al. Contextualizing the effectiveness of a collaborative care model for primary care patients with diabetes and depression (Teamcare): a qualitative assessment using RE-AIM. Can J Diabetes. 2015;39(Suppl 3):S83–91.
Johnson JA, et al. Cost-effectiveness evaluation of collaborative Care for Diabetes and Depression in primary care. Am J Prev Med. 2016;51(1):e13–20.
Xie F, et al. A time trade-off-derived value set of the EQ-5D-5L for Canada. Med Care. 2016;54(1):98–105.
Thorne, S., Interpretive Description: Qualitative Research for Applied Practice, ed. 2. 2016, New York: Routledge.
Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40.
Sandelowski M. What’s in a name? Qualitative description revisited. Res Nurs Health. 2010;33(1):77–84.
Damschroder LJ, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):50.
Braun V, Clarke V. Chapter 7: Thematic Analysis. In: Lyons AC, editor. Qualitative research in clinical and Health Psychology. UK: Palgrave Macmillan; 2015.
Aiyegbusi OL, et al. Measurement properties of patient-reported outcome measures (PROMs) used in adult patients with chronic kidney disease: a systematic review. PLoS One. 2017;12(6):e0179733.
We would like to acknowledge contributions of the following individuals for planning and implementation of the EMPATHY Trial: Alberta Kidney Care: Duperron P (deceased), Ilkiw N, Nhan J, Wasylynuk BA; Ontario Renal Network: Blake P, Cooper R, Glazer A, Heale E, Lum R, Mackinnon M, Rust J.
EMPATHY was independently peer-reviewed and funded by Can-SOLVE CKD Network, supported by the Canadian Institutes of Health Research under Canada’s Strategy for Patient-Oriented Research (reference #SCA-145103). This large network supports a large number of research projects, including the EMPATHY Trial and provides basic infrastructure and resources. The EuroQol Research Foundation provided funding for the qualitative sub-study undertaken in AKC-N through the University of Alberta; the Kidney Foundation of Canada provided funding for the qualitative sub-study undertaken in AKC-S through the University of Calgary.
Ethics approval and consent to participate
Ethics approvals for the study were granted from all required institutions, including the University of Alberta (Health Research Ethics Board Pro00077850) and the University of Calgary (Conjoint Health Research Ethics Board REB18–0672). The ORN received ethics approvals from various regional authorities including Timmins and District Hospital Ethics and Research Committee, Bluewater Health Research Ethics Committee (2019-RES-01), Halton Healthcare Research Ethics Board, Joint Group Health Centre/Sault Area Hospital Research Ethics Board (2019-12-02), and Hamilton Integrated Research Ethics Board (1717). All ethics review boards approved the study to be conducted under a waiver of consent. This was considered appropriate because PROM collection was occurring already, or was planned to be implemented, as determined by the renal programs; the intervention (linking the PROM to patient/provider discussion) is of minimal risk to patients, and all treatments are ascribed based on the provider judgment, not by study protocol; seeking informed consent would not be feasible in the framework of making this part of routine clinical care; seeking informed consent would likely bias participation resulting in inaccurate estimations of effect which would render the results of the trial uninformative for the use of these measures as part of routine care. The University of Alberta’s Health Research Ethics Board and the University of Calgary’s Conjoint Health Research Ethics Board separately approved the qualitative sub-studies (HREB #Pro00085021, CHREB #REB18–1786, respectively).
Consent for publication
Fatima Al Sayah and Jeffrey Johnson are members of the EuroQol Group. J Johnson is a member of the Board of Directors for the EuroQol Research Foundation. Braden Manns is co-Principal Investigator of the Can-SOLVE CKD Network. All authors have no other conflicts of interest to declare.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Johnson, J.A., Al Sayah, F., Buzinski, R. et al. A cluster randomized controlled trial for the Evaluation of routinely Measured PATient reported outcomes in HemodialYsis care (EMPATHY): a study protocol. BMC Health Serv Res 20, 731 (2020). https://doi.org/10.1186/s12913-020-05557-z
- Kidney failure
- Patient-reported outcome measures
- Symptom burden
- Quality improvement
- Controlled trial