- Research article
- Open Access
A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting
BMC Health Services Research volume 13, Article number: 211 (2013)
Despite growing interest and urges by leading experts for the routine collection of patient reported outcome (PRO) measures in all general care patients, and in particular cancer patients, there has not been an updated comprehensive review of the evidence regarding the impact of adopting such a strategy on patients, service providers and organisations in an oncologic setting.
Based on a critical analysis of the three most recent systematic reviews, the current systematic review developed a six-method strategy in searching and reviewing the most relevant quantitative studies between January 2000 and October 2011 using a set of pre-determined inclusion criteria and theory-based outcome indicators. The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system was used to rate the quality and importance of the identified publications, and the synthesis of the evidence was conducted.
The 27 identified studies showed strong evidence that the well-implemented PROs improved patient-provider communication and patient satisfaction. There was also growing evidence that it improved the monitoring of treatment response and the detection of unrecognised problems. However, there was a weak or non-existent evidence-base regarding the impact on changes to patient management and improved health outcomes, changes to patient health behaviour, the effectiveness of quality improvement of organisations, and on transparency, accountability, public reporting activities, and performance of the health care system.
Despite the existence of significant gaps in the evidence-base, there is growing evidence in support of routine PRO collection in enabling better and patient-centred care in cancer settings.
Patient reported outcome (PRO) measures include health status assessments and measures for health-related quality-of-life (HRQOL), symptom reporting, satisfaction with care, treatment satisfaction, economic impact, and specific dimensions of patient experience such as depression and anxiety . The USA Food and Drug Agency (FDA) adopts a much broader definition  as “A PRO is any report coming directly from patients about a health condition and its treatment”, meaning that PROs capture patients’ perspectives about how illness or new therapies impact on their general well-being. There is a growing interest from clinicians, researchers, industry and policy-makers in routinely collecting PROs to facilitate timely, patient-centred and evidence-based care. For example, the National Health Service (NHS) of the UK has been implementing a world-leading initiative for the routine collection of PROs that firstly included a few selected elective surgeries (e.g. unilateral hip replacements, unilateral knee replacements, groin hernia surgery or varicose vein surgery)  but are soon expanding to many other conditions such as mastectomy and breast cancer, among others. In the USA, the Patient-Reported Outcomes Measurement Information System (PROMIS), a National Institutes of Health funded initiative starting in 2004, is providing a publicly available web-based resource that can be used to measure key health symptoms and HRQOL . The traditional paper-based PROs instruments are limited by its lack of flexibility, language and literacy requirement, [5, 6] possible inappropriateness towards minority groups, [7, 8] lack of timeliness (in generating instantaneous clinical meaningful interpretations)  and inability to adopt state-of-the-art measurement science such as Item Response Theory (IRT) and Computer Adapted Test (CAT) technique . To overcome the difficulty of integrating the administration and analysis of PRO instruments into clinical practice, researchers are developing and validating alternatives to traditional paper-based instruments such as office-based touch-screen computers, [11–13] telephone-based interactive voice-response (IVR) systems, [14–16] hand-held computers, [17, 18] mobile phones, [19–21] and more recently, the Internet [22–24]. Some rationales [25–28] put forward for measuring PROs in a cancer setting include, but not limited to: 1) better communication and shared decision making by patients and providers; 2) assessing the health status of patients entering therapy and identifying treatable problems; 3) determining the degree and sources of the patient’s decreased ability to function; 4) distinguishing between types of problems, including physical, emotional, and social; 5) detecting adverse effects of therapy; 6) monitoring the effects of disease progression and response to therapy; 7) informing decisions about changing treatment plans, and 8) predicting the course of disease and outcomes of care.
However, despite growing interest and urges by the leading experts for applying routinely collected PROs for all cancer patients, there has not been an updated comprehensive review of the evidence regarding the impact of adopting such a strategy on patients, services providers and organisations. The most recent review focused only on clinical trial design  studies of cancer patients, and only assessed a limited number of outcomes. The current project aims to provide the much needed comprehensive review update, including all relevant quantitative studies investigating the effectiveness of routine PRO collection in cancer patients. The review research questions were:
What are the impacts of composite measures of PROs collected on cancer patients during treatment with regards to:
Provider behaviour for improving care delivered;
Organisational changes within health care settings for improving processes and models of care (e.g. targeting and tailoring care);
Improving clinical outcomes for patients; and
Improving patient experience of care (e.g. self-care).
What mechanisms are involved in the link between PROs and the impacts identified in 1(a)?
What factors moderate the extent of the impacts identified in 1(a)?
Existing systematic reviews and rationale for the current review
In order to develop an efficient search and review strategy, over 200 existing reviews on the same or similar topics were firstly systematically examined (identified in a broad search covering PROs and quality of life measures between January 2000 and October 2011). Three reviews [26–28] were identified as the baseline reviews for this project and their review strategies were carefully examined in aspects such as the aim and scope, time span, search strategy and search terms used, articles included in each review, and conclusions drawn. A table summarising the three systematic reviews is presented in Table 1.
Review search strategy
Analysing the results of above three systematic reviews demonstrates the importance of search strategies in determining what literature will be included in the study, which in turn, may influence what conclusions will be derived. Valderas et al.’s (2008)  review excluded three out of the five clinical trials on cancer patients that were included in Marshall et al.’s (2006)  review. Lucket et al.’s (2009) review  excluded one article (Taenzer et al. (2000),  a before-after study) from Marshall et al.’s review . A mixed methodology search was developed in order to maximise the identification of recent literature in a short period of time. The search was conducted in six different ways as follows:
A text-based search strategy was developed based on previous reviews. To elicit previous reviews, a search was conducted for the text terms ‘patient reported outcome*’, ‘self-reported’, ‘self-assessed’ anywhere in title, abstract and key words, combined with ‘quality of life’, ‘symptom’, ‘functional status’, ‘health status’, ‘patient satisfaction’, ‘unmet need*’ anywhere in title, abstract and key words. For original articles, a search was conducted using the same strategy as above but restricted to those with ‘neoplasm’ or ‘cancer’ in the key words. The search results were restricted to between January 2000 and October 2011 (full search strategy is listed in Additional file 1: Appendix 1).
All reviews were evaluated (over 200 in total on various topics but not limited to only cancer patients) with three baseline reviews used as the starting point for our top-down and bottom-up search strategy. We chose the three baseline reviews because that: 1) they are all systematic reviews that could be helpful in forming the structure or strategy of the current review (but not necessarily restricted to cancer patients); and 2) they were published after 2005.
All articles were examined if they cited the 7 key randomised controlled trials [33–39] listed in the above reviews (bottom-up approach). References were also sought from the most recently published trials, editorials, and commentaries (a top-down approach). The powerful citation tracking feature of Scopus™ made this strategy feasible.
Simplified text terms (i.e. patient reported outcome, PRO, PROM, Quality of life, QOL) were used to conduct a web search for identifying grey literature.
Leading researchers and experts in the field (elicited through the advice of Cancer Institute NSW (CINSW), editorials, review articles and most cited articles) were purposefully searched in order to analyse the references and citations in their publications.
Some key cancer centres’ websites were also searched in order to get more detailed information.
The search was limited to the Scopus™ database as it is the largest abstract and citation database of peer-reviewed literature and quality web sources including 100% coverage of Medline titles and EBASE. It also tracks, analyses and visualises publication results, which is well suited to our top-down and bottom-up search strategy.
Aim, study selection and endpoints of the review
In this review, the aim was to synthesize the evidence in relation to the impact of routinely collected PROs on patients, providers, and health organisations. The frameworks proposed by Greenhalgh and colleagues  and by Abernethy and colleagues  were adopted to guide our evaluation of the existing literature. Greenhalgh et al.  proposed a framework (Figure 1) that depicts mechanisms between the routine collection of PROs and changes in patient outcomes. The authors suggest that the multilayer mediators (i.e. changes to doctor-patient communication, monitoring treatment responses, detecting unrecognised problems, changes to patient health behaviour, changes to clinicians’ management plans, and improved patient satisfaction) have complex relationships among them. The studies that revealed these complex relationships may assist in understanding whether and how the underlying mechanisms of routinely collected PROs work to improve the intended outcomes.
Recently, Abernethy and colleagues  have argued that the routine collection of PROs has the capacity to impact not only at the patient-level, but by addressing the logistics of data linkage, and could ensure that the system will grow to accommodate other clinical- and health system-level issues; for example, evaluating comparative effectiveness of treatments, monitoring quality of care, and translating basic science findings into clinical practice (Figure 2). The integration of data systems will fuel rapid learning cancer care at the national and societal levels (see Figure 2a and b), making many types of research and system learning possible across institutions and health sectors. The benefits and implications of such a rapid learning health care system may include, but is not limited to, strong and effective quality improvement (QI), increased transparency, accountability, public reporting, better health system performance (monitoring, planning, financing, evaluating, responding) and better quality of care.
Combining both frameworks, a list of outcome indicators was developed (Table 2) against which each eligible study was assessed. To include not only the doctors’ experience with patients after collecting PROs, but also the experience of other health services providers (i.e. nurses, allied health workers), the term ‘Patient-provider communication’ was used instead of ‘doctor patient communication’ as proposed by Greenhalgh et al. . In order to answer review questions 2 & 3 for the studies included, all possible explicit mediation effects were reviewed through examining if a path-analysis or a mediation-analysis by multiple, staged regression approach was presented in the paper. To examine potential moderating effect, each study was examined to determine if it explicitly tested the interaction effect/moderating effect, or inexplicitly conducted subgroup analysis. Significant possible mediating or moderating effect results were indicated as part of review endpoints in Table 3. Inferences made and the discussion were based on these results.
Inclusion and exclusion criteria
The inclusion criteria were: 1) substantial content in presenting empirical evidence on the impact of routinely collected PROs on at least one of the outcomes listed in Table 2; 2) adult cancer patients; 3) conducted in an oncologic setting including inpatient, outpatient and outreach services; and 4) studies using a composite PRO. We defined a composite PRO as those PROs are often based on a well-developed instrument and with an aim for measuring a substantial aspect of patient conditions (or treatment) with at least 4 items. To reflect the demanding and complex nature of evaluating the impact of routine collected PROs, eligible studies included a variety of designs including, but not limited to, randomised controlled trials (RCTs), controlled before-after trials (CBA) and interrupted time series (ITS). ITS designs have a longitudinal character, with repeated measurements and at least three data points before and after the intervention point. Surveys and clinical audits were also included if the studies provided quantitative results relevant to the listed outcomes.
Studies were excluded if they were non-English language articles, opinion and theoretical articles, historical descriptions, review articles, feasibility studies of some PROs collection devices, studies investigating child cancer patients or qualitative studies with no substantial quantitative results on the review endpoints.
Data extraction and quality assessment
Electronic search results were downloaded into EndNote bibliographic software. Two reviewers independently (JC, LO) screened all titles and abstracts of citations identified by the electronic search, applied the selection criteria to potentially relevant papers, and extracted data from included studies using a standardised form. Any disagreements concerning studies to be included were resolved by consensus.
All studies were classified into two domains. Domain 1 correlated sample characteristics with population wide characteristics, and Domain 2 focused on study design. The data extraction form was adapted from other review studies using the outcome measures discussed above (see Table 4). For each eligible study, a list was made including the leading author, country and jurisdiction, design, sample, outcome measures, the PROs used, times of feedback and intervention, members of medical teams given feedback, management plans offered to teams, and training (see Table 5). All qualifying studies were listed chronologically with the outcome indicators (see Table 3).
In Domain 1, the routinely collected PROs in particular participants or samples was classified as rated on a 4-point scale representing how closely the participants or samples overlapped with the characteristics and needs of the intended study populations(1 star=very weakly related to 4 stars =very strong related). For example, for a study conducted in the US on a sample of lung cancer patients, the degree of overlap of the study sample with the characteristics of lung cancer patients in the US overall was assessed by considering the study setting, sample size and sampling frame, response rate, loss-to-follow-up, and characteristics of the study sample. In Domain 2, study design was classified and rated on 4 categories with 1 star indicating the weakest design and 4 stars indicating the strongest design. Four stars indicated a randomised trial or experimental study; 3 stars indicated a controlled trial, pre–post trial with control (controlled before–after trial), time series, or observational cohort with multivariable adjustment; 2 stars indicated a pre–post trial without control, observational cohort study without multivariable adjustment, cross-sectional study without multivariable adjustment, analysis of time trends without control, or well-designed qualitative study; and 1 star indicated a case series, other qualitative study, or survey (descriptive) study.
Revised appraisal criteria were adapted from the guidelines on the assessment of quality improvement interventions [58, 59]. A global rating was also created using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system . The British Medical Journal has recommended the GRADE system since 2006 for grading evidence when submitting a clinical guidelines article. It has multiple advantages and is useful for systematic reviews and health technology assessments, as well as for evaluating research on clinical guidelines. The global rating created in the current study was based on the integration of the Domain 1 and Domain 2 ratings, as well as the intervention fidelity (the degree of success of the interventional strategy, the patients’ and providers’ adherence to the intervention strategy), dose–response gradient, precision and validity of outcomes (potential confounding factors and biases), and uncertainty of the direction of results. The global rating was divided into three categories; indication that the study should carry great (3 checks), moderate (2 checks), or little (1 check) weight when considering the strength of evidence (see Table 4). Any experimental research that is reported in the manuscript was performed with the approval of an appropriate ethics committee.
No attempt was made to quantitatively synthesize the results as the data were too heterogeneous to support pooling.
The multi-method search strategy yielded 27 publications that were eligible to be included in the review – a significant increase compared with most of the recent reviews. The detailed flowchart of the search strategies and its relative results was presented (Figure 3). The results and conclusions drawn were based on the 27 studies included in the analyses despite the large number of full-text articles extracted.
Of the 27 publications, 16 were identified as randomised controlled trials, 2 as before-after studies and 9 observational studies with 11 studies published before 2009. The characteristics and quality of the studies are presented in Table 5 with their impact on outcome indicators presented in Table 3. As Trowbridge et al. (1997) was the only article in the 1990s included in two of the previous reviews, it was listed in the summation tables for the purpose of comparison.
Overview of study quality
There has been a marked increase in the volume and quality of the studies published recently in this area. Of the 16 randomised controlled trials included in this review, 7 were published between 2010 and 2011. The quality of studies published since 2010 is also demonstrably improved with much larger sample sizes, including 3 trials [54, 57, 61] with a sample size greater than 200 and 2 trials with a sample size over 580 [51, 56].
However, despite the increased volume and improved quality of the studies, there remains a lack of large cluster randomised controlled design studies, as recommended by Fayers  who argues that cluster RCTs are well suited to overcome the limitations of simple RCTs. It is well-known that system intervention trials such as routine collection of PROs, and feedback to the clinicians and systems, are prone to cross-contamination and to introducing investigator and participant biases. Two recently published studies [54, 57] were the continuation of an earlier study published by Velikova et al. (2004) . Most studies reviewed did not systematically examine outcomes and mechanisms, and placed more emphasis on processes rather than outcome measures . All studies were conducted in a limited setting (often in a single centre) thus restricting the generalisation of the findings.
No studies have adopted a comprehensive theoretical model and framework, despite the repeated demand from leading researchers in the area [25, 63–65]. All studies focused on the patient and health professional level within the clinic setting. No study to date has examined the impact of collecting PROs on health care organisations, health system improvement, quality improvement or population health at a system or societal level.
Overview of study findings
Impact on patient-provider communication
Across the 27 studies included in this review, 4 studies [39, 47, 51, 53] did not examine or report the effect of a routinely collected PRO on patient-provider communication. Among the 23 studies that did report such an impact, 21 studies (91.3%) reported a positive effect which included well-designed and conducted large RCTs [33, 36, 37, 54, 56, 57]. One study reported no significant improvement of patient-provider communication possibly due to a lower severity level of cancer patients (only 37% of patients received anticancer therapy, hence the reduced need for communication for the treatment) . Another study reporting a negative effect had an already high communication level at baseline (hence a ceiling effect leaving little room for further improvement) .
Impact on monitoring treatment response
Despite most of included 27 studies did not explicitly state their study objectives as to examine the impact on monitoring treatment response, 11 of the 27 studies did report an outcome (Table 3) [16, 20, 36, 41, 45–51]. All 11 studies found a strong or modest effect of implementing PROs on the increased monitoring activities of treatment response. The strongest effect occurred in the studies that focused on the monitoring of patient symptoms, side effects and toxicity during and after chemotherapy for the outpatients. In particular, the real-time, patient reported symptoms and toxicity (through innovative mobile phone-based, web-based or IVR systems) significantly improved the monitoring of treatment response.
Impact on detecting unrecognised problems
Although the idea of routinely collected PROs may provide better opportunities for services providers (as well as patients) to detect unrecognised problems through growing awareness, improved communication and monitoring seems intuitively plausible, only 16 out of 27 studies reported some results related to the detection of unrecognised problems (Table 3). Amongst the 16 studies, 15 studies [16, 20, 33, 36, 37, 39, 41, 43],[45–50, 53] reported either a strong or moderate positive impact on detecting unrecognised problems. However, a study by McLachlan and colleagues  did not find any difference between the intervention arm and control arm.
Impact on changes to patient health behaviour
No study to date has provided a systematic evaluation on the impact of collecting PROs on changes to patient health behaviour. It is unknown whether and how patient health behaviours have been changed.
Impact on changes to patient management
Amongst 17 studies that provided some results of changes to patient management, 13 studies [11, 20, 33, 37, 39, 45, 46, 48–50],[52, 53, 55] reported either a strong or modest positive effect on the changes to patient management whilst 4 studies [34–36, 54] found no such effect. However, it is worth noting that 10 studies did not provide any information about the changes to patient management and there were often less complete descriptions of the results on patient management when reported.
Impact on patient satisfaction
Among the 16 studies that reported results related to the impact on patient satisfaction, 13 studies [11, 16, 20, 37, 41, 43–46, 48, 49],[52, 54] reported a very strong to moderate positive effect on improved patient satisfaction. For the three studies [33, 34, 38] that did not find such a positive effect, one study  reported a possible ceiling effect meaning that both the intervention group and control group had a very high baseline patient satisfaction level potentially impeding any demonstration of a significant difference between two arms during the follow-up period.
Impact on health outcomes
Amongst the 15 studies that reported some results related to the impact on health outcomes, 13 studies [20, 35–37, 39, 41–43, 45, 47, 50, 51], reported some positive improvement, ranging from modest to strong, while two studies [34, 38] failed to find any such effect. It appears that symptoms, side effects and toxicity are most likely to be improved, followed by emotional wellbeing. There is little evidence on the improvement of both overall HRQOLs as well as social wellbeing.
Impact on quality improvement, transparency, accountability and public reporting, and on better system performances (monitoring, planning, financing, evaluating, responding)
No study to date has provided a meaningful, explicit framework nor relevant evidence on these endpoints.
Overall strength and direction of evidence
Overall, there is strong evidence supporting the notion that routinely collected PROs, with feedback, improves patient-provider communication and increases patient satisfaction (Table 6). There is some evidence to support the notion that it improves the monitoring of treatment responses and detection of unrecognised problems, and there is weak but positive evidence that, over time, it leads to changes in patient management. Despite some encouraging results, there is still a great degree of uncertainty regarding the impact of routinely collected PROs, with feedback, on patient health outcomes. There is little or no evidence that it has led to significant positive improvements in quality improvement, transparency, accountability, and public reporting, or in system performance at a population health or societal level. Apart from clinical trials and clinical practice, its impact on health services research and population health is largely unknown.
Potential moderating factors and links between routine PRO collection (with feedback) and patient outcomes
Although the evidence is limited, it appears that routine collected PROs with sufficient intensity of feedback (multiple times over a sustained period of time) [13, 39, 44, 54], targeting multiple stakeholders (doctors, nurses, allied health workers, as well as patients) [35, 52] with simple, clear, graphical and longitudinal meaningful interpretation of the results, and providing sufficient training for both health professionals and patients [20, 57], are critical links between an intervention and the intended outcomes. There is also evidence to suggest that for some complex issues such as depression and low social functioning, routine screening and feedback may need to be integrated with other strategies such as decision-making aids, education, clear management plans and clinical pathways including referrals, in order to change patient outcomes [43, 49, 51]. There is preliminary evidence that some of the impacts of PROs may be more pronounced amongst subgroups with more severe problems at baseline (e.g. depression, symptoms) [38, 42, 65]. More studies are needed to fully understanding these mediating and moderating effects.
There is very strong evidence in supporting the notion that routine collected PROs with timely feedback enhances patient-provider communication. This current study finding of a positive effect on patient-provider communication is consistent with previous reviews conducted in both cancer  and non-cancer settings [25, 27, 28]. There is also strong evidence to support the notation that routine collected PROs significantly improved the monitoring of treatment response.
There is reasonably strong evidence in supporting the notation that routine collected PROs are helpful in identifying unrecognised problems in a large variety of settings. Within studies that reported, to some extent, the results related to unrecognised problems, there is a need for the development of more comprehensive and valid measures. Such a change in the measures would contribute towards understanding specifically the PROs’ impact on identifying the underreported and unrecognised problems for different cancer patients at different settings.
Overall, there is reasonable evidence in favouring the hypothesis that implementing a routine collected PROs system brings positive changes to patient management in the settings where a patient management plan is integrated with a routine collection of PROs. It appears that the simple routine feedback of PROs may not be sufficient to improve patient management and outcomes . Other necessary resources may be needed such as education, referral services and a detailed patient management plan following the PROs . There is also a need to develop better measures of change to patient management as it is often complex and difficult to quantify .
There is strong evidence to support the notation that routine collected PROs with timely feedback significantly enhance patient’s experience and satisfaction. There may be other improved experience and satisfaction in other stakeholders such as patients’ family members, caregivers, as well as health professionals that were not measured or unreported. Future research into furthering the understanding of stakeholder experience after implementing routine collected PROs would be desirable.
Although positive evidence in supporting the notion that routine collected PROs may improve health outcomes is weak, this finding needs to be confirmed by better designed studies covering a large set of well-developed outcome measures. There is also a need to understand the impact on long-term health outcomes such as survival rate. Most of the studies included in this review did not focus on health outcomes and some of the positive improvement on the outcomes only occurred on selective measures. It is not clear how these positive improvements can be generalised across different settings.
There is a variety of models on how to routinely collect PROs and how to feed back the data to different stakeholders. Given that cancer patients are vastly different given their background, type and stage of cancer, prognosis, treatment, and the positions at the life course continuum, precaution should be exercised when attempting to apply the general observation above to each and every different setting. For example, recent studies demonstrated a positive impact of routine collected PROs on symptom control through either web-based or mobile phone based approach. However, such positive impacts were less pronounced on HRQOL.
Limitations of the current review
Our review has several limitations. First, there was no attempt made to contact the authors to ask for potential unpublished data on the topic. Thus, there may be chance of missing some grey literature or the studies that under preparation for publication. Second, given the multitude of endpoints included, and different types of studies involved, the assessment of eligibility for inclusion of potential studies required some degree of subjective judgement. Third, our application of GRADE system was rather simplistic restricted by large number of endpoints and variability of studies included. These limitations may give rise to some uncertainty in terms of synthesis of the results. Fourth, our study follows a systematic review approach with inclusion of both experimental trials and quantitative observational studies. However, we did not include qualitative studies in our review which may provide additional insight into the questions raised. This is particularly relevant with respect to questions 2 and 3 as there was very little quantitative evidence from the included studies. It is important to note that despite efforts to formulate the review endpoints based on solid and well-established causal and theoretical frameworks (providing insight into not only if but also how the introduction of PROs affects patient outcomes), the causal mechanisms and process endpoints included in the current review are by no means exhaustive. There may be other important causal mechanisms that could be benefited from a realist review approach .
There is growing evidence supporting the routine collection of PRO to enable better and patient-centred care, especially in cancer settings. Despite the strong evidence in supporting the notion that the well-implemented routine collection of PROs enhances patient-provider communication and improves patient satisfaction, and growing evidence supporting ideas that it also improves the monitoring of treatment response and the detection of the unrecognised problems, the evidence-base was weak for its impact on changes to patient management and improved health outcomes and non-existent for changes to patient health behaviour, strong and effective quality improvement, increased transparency, accountability, public reporting and better health care system performance. These evidence gaps require further committed and well-planned research in addition to the well-accepted PROs. Decision-making agencies have been well positioned for leverage on the rapid advancement of different PRO models, the application of the item response theory and computer adapted test in developing PROs, and on the acceptance of such technology by patients and health professionals over the last decade. The real-time and routinely collected PROs will enable the development of a rapid learning health system with the potential to advance our knowledge of drug development, unsurpassed models of cancer patient care and a more patient-centred health care system.
Jones JB, Snyder CF, Wu AW: Issues in the design of Internet-based systems for collecting patient-reported outcomes. Qual Life Res. 2007, 16 (8): 1407-1417. 10.1007/s11136-007-9235-z.
Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. 2009, Silver Spring, Maryland: U.S. Department of Health and Human Services Food and Drug Administration
Ousey K, Cook L: Understanding patient reported outcome measures (PROMs). Br J Community Nurs. 2011, 16 (2): 80-82.
Henly SJ: The promise of PROMIS. Nurs Res. 2010, 59 (2): 77-10.1097/NNR.0b013e3181d7d1b1.
Fromme EK, Kenworthy-Heinige T, Hribar M: Developing an easy-to-use tablet computer application for assessing patient-reported outcomes in patients with cancer. Support Care Cancer. 2011, 19 (6): 815-822. 10.1007/s00520-010-0905-y.
Fiscella K, Ransom S, Jean-Pierre P, Cella D, Stein K, Bauer JE, Crane-Okada R, Gentry S, Canosa R, Smith T, et al: Patient-reported outcome measures suitable to assessment of patient navigation. Cancer. 2011, 117 (SUPPL. 15): 3603-3617.
Burgess DJ, Powell AA, Griffin JM, Partin MR: Race and the validity of self-reported cancer screening behaviors: Development of a conceptual model. Prev Med. 2009, 48 (2): 99-107. 10.1016/j.ypmed.2008.11.014.
Jayadevappa R, Johnson JC, Chhatre S, Wein AJ, Malkowicz SB: Ethnic variation in return to baseline values of patient-reported outcomes in older prostate cancer patients. Cancer. 2007, 109 (11): 2229-2238. 10.1002/cncr.22675.
Sloan JA, Frost MH, Berzon R, Dueck A, Guyatt G, Moinpour C, Sprangers M, Ferrans C, Cella D, Aaronson N, et al: The clinical significance of quality of life assessments in oncology: A summary for clinicians. Support Care Cancer. 2006, 14 (10): 988-998. 10.1007/s00520-006-0085-y.
Henly SJ: Editorial: The promise of PROMIS. Nurs Res. 2010, 59 (2): 77-10.1097/NNR.0b013e3181d7d1b1.
Halkett G, Jiwa M, Tanner P, Fournier C, Katris P: Trialling computer touch-screen technology to assess psychological distress in patients with gynaecological cancer. Australasian Medical Journal. 2010, 3 (12): 781-785.
Clark K, Bardwell WA, Arsenault T, DeTeresa R, Loscalzo M: Implementing touch-screen technology to enhance recognition of distress. Psychooncology. 2009, 18 (8): 822-830. 10.1002/pon.1509.
Velikova G: Use of electronic quality of life applications in cancer research and clinical practice. Expert Rev Pharmacoecon Outcomes Res. 2004, 4 (4): 403-411. 10.1586/14737184.108.40.2063.
Kilbourn KM, Bargai N, Durning PE, Deroche K, Madore S, Zabora J: Validity of the psycho-oncology screening tool (POST). J Psychosoc Oncol. 2011, 29 (5): 475-498. 10.1080/07347332.2011.599359.
Macvean ML, White VM, Pratt S, Grogan S, Sanson-Fisher R: Reducing the unmet needs of patients with colorectal cancer: A feasibility study of the Pathfinder Volunteer Program. Support Care Cancer. 2007, 15 (3): 293-299. 10.1007/s00520-006-0128-4.
Mooney KH, Beck SL, Friedman RH, Farzanfar R: Telephone-linked care for cancer symptom monitoring: A pilot study. Cancer Pract. 2002, 10 (3): 147-154. 10.1046/j.1523-5394.2002.103006.x.
Forbat L, Maguire R, McCann L, Illingworth N, Kearney N: The use of technology in cancer care: Applying Foucault's ideas to explore the changing dynamics of power in health care. J Adv Nurs. 2009, 65 (2): 306-315. 10.1111/j.1365-2648.2008.04870.x.
Matthew AG, Currie KL, Irvine J, Ritvo P, Santa Mina D, Jamnicky L, Nam R, Trachtenberg J: Serial personal digital assistant data capture of health-related quality of life: A randomized controlled trial in a prostate cancer clinic. Health Qual Life Outcomes. 2007, 5: 38-10.1186/1477-7525-5-38. http://dx.doi.org/10.1186/1477-7525-5-38.
Williams PD, Williams K, Lafaver-Roling S, Johnson R, Williams AR: An intervention to manage patient-reported symptoms during cancer treatment. Clin J Oncol Nurs. 2011, 15 (3): 253-258. 10.1188/11.CJON.253-258.
Cleeland CS, Wang XS, Shi Q, Mendoza TR, Wright SL, Berry MD, Malveaux D, Shah PK, Gning I, Hofstetter WL, et al: Automated symptom alerts reduce postoperative symptom severity after cancer surgery: A randomized controlled clinical trial. J Clin Oncol. 2011, 29 (8): 994-1000. 10.1200/JCO.2010.29.8315.
Jenner AV, Mukherjee S, Roberts R, Howe D, Mort D: Patient-reported outcome measures for chemotherapy-related acute toxicity - a single centre experience. Clin Oncol. 2010, 22 (4): 321-322. 10.1016/j.clon.2010.02.008.
Yost KJ, Eton DT, Garcia SF, Cella D: Minimally important differences were estimated for six Patient-Reported Outcomes Measurement Information System-Cancer scales in advanced-stage cancer patients. J Clin Epidemiol. 2011, 64 (5): 507-516. 10.1016/j.jclinepi.2010.11.018.
Halyard MY: The use of real-time patient-reported outcomes and quality-of-life data in oncology clinical practice. Expert Rev Pharmacoecon Outcomes Res. 2011, 11 (5): 561-570. 10.1586/erp.11.62.
Fries J, Rose M, Krishnan E: The PROMIS of better outcome assessment: Responsiveness, floor and ceiling effects, and internet administration. J Rheumatol. 2011, 38 (8): 1759-1764. 10.3899/jrheum.110402.
Greenhalgh J, Long AF, Flynn R: The use of patient reported outcome measures in routine clinical practice: Lack of impact or lack of theory?. Soc Sci Med. 2005, 60 (4): 833-843. 10.1016/j.socscimed.2004.06.022.
Luckett T, Butow PN, King MT: Improving patient outcomes through the routine use of patient-reported data in cancer clinics: Future directions. Psychooncology. 2009, 18 (11): 1129-1138. 10.1002/pon.1545.
Marshall S, Haywood K, Fitzpatrick R: Impact of patient-reported outcome measures on routine practice: A structured review. J Eval Clin Pract. 2006, 12 (5): 559-568. 10.1111/j.1365-2753.2006.00650.x.
Valderas JM, Kotzeva A, Espallargues M, Guyatt G, Ferrans CE, Halyard MY, Revicki DA, Symonds T, Parada A, Alonso J: The impact of measuring patient-reported outcomes in clinical practice: A systematic review of the literature. Qual Life Res. 2008, 17 (2): 179-193. 10.1007/s11136-007-9295-0.
Espallargues M, Valderas JM, Alonso J: Provision of feedback on perceived health status to health care professionals: A systematic review of its impact. Medical Care. 2000, 38 (2): 175-186. 10.1097/00005650-200002000-00007.
Greenhalgh J, Meadows K: The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: A literature review. J Eval Clin Pract. 1999, 5 (4): 401-416. 10.1046/j.1365-2753.1999.00209.x.
Gilbody SM, House AO, Sheldon TA: Outcome measures and needs assessment tools for schizophrenia and related disorders. Cochrane Database Syst Rev. 2003, 1: CD003081
Drury M, Yudkin P, Harcourt J, Fitzpatrick R, Jones L, Alcock C, Minton M: Patients with cancer holding their own records: A randomised controlled trial. Br J Gen Pract. 2000, 50 (451): 105-110.
Taenzer P, Bultz BD, Carlson LE, Speca M, DeGagne T, Olson K, Doll R, Rosberger Z: Impact of computerized quality of life screening on physician behaviour and patient satisfaction in lung cancer outpatients. Psychooncology. 2000, 9 (3): 203-213. 10.1002/1099-1611(200005/06)9:3<203::AID-PON453>3.0.CO;2-Y.
Rosenbloom SK, Victorson DE, Hahn EA, Peterman AH, Cella D: Assessment is not enough: A randomized controlled trial of the effects of HRQL assessment on quality of life and satisfaction in oncology clinical practice. Psychooncology. 2007, 16 (12): 1069-1079. 10.1002/pon.1184.
Boyes A, Newell S, Girgis A, McElduff P, Sanson-Fisher R: Does routine assessment and real-time feedback improve cancer patients' psychosocial well-being?. Eur J Cancer Care. 2006, 15 (2): 163-171. 10.1111/j.1365-2354.2005.00633.x.
Velikova G, Booth L, Smith AB, Brown PM, Lynch P, Brown JM, Selby PJ: Measuring quality of life in routine oncology practice improves communication and patient well-being: A randomized controlled trial. J Clin Oncol. 2004, 22 (4): 714-724. 10.1200/JCO.2004.06.078.
Detmar SB, Muller MJ, Schornagel JH, Wever LDV, Aaronson NK: Health-related quality-of-life assessments and patient-physician communication: A randomized controlled trial. J Am Med Assoc. 2002, 288 (23): 3027-3034. 10.1001/jama.288.23.3027.
McLachlan SA, Allenby A, Matthews J, Wirth A, Kissane D, Bishop M, Beresford J, Zalcberg J: Randomized trial of coordinated psychosocial interventions based on patient self-assessments versus standard care to improve the psychosocial functioning of patients with cancer. J Clin Oncol. 2001, 19 (21): 4117-4125.
Trowbridge R, Dugan W, Jay SJ, Littrell D, Casebeer LL, Edgerton S, Anderson J, O'Toole JB: Determining the effectiveness of a clinical-practice intervention in improving the control of pain in outpatients with cancer. Acad Med. 1997, 72 (9): 798-800. 10.1097/00001888-199709000-00016.
Abernethy AP, Ahmad A, Zafar SY, Wheeler JL, Reese JB, Lyerly HK: Electronic patient-reported data capture as a foundation of rapid learning cancer care. Med Care. 2010, 48 (6 Suppl): S32-S38.
Basch E, Artz D, Dulko D, Scher K, Sabbatini P, Hensley M, Mitra N, Speakman J, McCabe M, Schrag D: Patient online self-reporting of toxicity symptoms during chemotherapy. J Clin Oncol. 2005, 23 (15): 3552-3561. 10.1200/JCO.2005.04.275.
Hoekstra J, De Vos R, Van Duijn NP, Schadé E, Bindels PJE: Using the symptom monitor in a randomized controlled trial: The effect on symptom prevalence and severity. J Pain Symptom Manage. 2006, 31 (1): 22-30. 10.1016/j.jpainsymman.2005.06.014.
Kornblith AB, Dowell JM, Herndon Ii JE, Engelman BJ, Bauer-Wu S, Small EJ, Morrison VA, Atkins J, Cohen HJ, Holland JC: Telephone monitoring of distress in patients aged 65 years or older with advanced stage cancer: A Cancer and Leukemia Group B study. Cancer. 2006, 107 (11): 2706-2714. 10.1002/cncr.22296.
Basch E, Iasonos A, Barz A, Culkin A, Kris MG, Artz D, Fearn P, Speakman J, Farquhar R, Scher HI, et al: Long-term toxicity monitoring via electronic patient-reported outcomes in patients receiving chemotherapy. J Clin Oncol. 2007, 25 (34): 5374-5380. 10.1200/JCO.2007.11.2243.
Weaver A, Young AM, Rowntree J, Townsend N, Pearson S, Smith J, Gibson O, Cobern W, Larsen M, Tarassenko L: Application of mobile phone technology for managing chemotherapy-associated side-effects. Ann Oncol. 2007, 18 (11): 1887-1892. 10.1093/annonc/mdm354.
Butt Z, Wagner LI, Beaumont JL, Paice JA, Straus JL, Peterman AH, Carro G, Von Roenn JH, Shevrin D, Cella D: Longitudinal screening and management of fatigue, pain, and emotional distress associated with cancer therapy. Support Care Cancer. 2008, 16 (2): 151-159. 10.1007/s00520-007-0291-2.
Given CW, Sikorskii A, Tamkus D, Given B, You M, McCorkle R, Champion V, Decker D: Managing symptoms among patients with breast cancer during chemotherapy: Results of a two-arm behavioral trial. J Clin Oncol. 2008, 26 (36): 5855-5862. 10.1200/JCO.2008.16.8872.
Hilarius DL, Kloeg PH, Gundy CM, Aaronson NK: Use of health-related quality-of-life assessments in daily clinical oncology nursing practice: A community hospital-based intervention study. Cancer. 2008, 113 (3): 628-637. 10.1002/cncr.23623.
Mark TL, Fortner B, Johnson G: Evaluation of a tablet PC technology to screen and educate oncology patients. Support Care Cancer. 2008, 16 (4): 371-378. 10.1007/s00520-007-0312-1.
Kearney N, McCann L, Norrie J, Taylor L, Gray P, McGee-Lennon M, Sage M, Miller M, Maguire R: Evaluation of a mobile phone-based, advanced symptom management system (ASyMS©) in the management of chemotherapy-related toxicity. Support Care Cancer. 2009, 17 (4): 437-444. 10.1007/s00520-008-0515-0.
Carlson LE, Groff SL, Maciejewski O, Bultz BD: Screening for distress in lung and breast cancer outpatients: A randomized controlled trial. J Clin Oncol. 2010, 28 (33): 4884-4891. 10.1200/JCO.2009.27.3698.
Dinkel A, Berg P, Pirker C, Geinitz H, Sehlen S, Emrich M, Marten-Mittag B, Henrich G, Book K, Herschbach P: Routine psychosocial distress screening in radiotherapy: Implementation and evaluation of a computerised procedure. Br J Cancer. 2010, 103 (10): 1489-1495. 10.1038/sj.bjc.6605930.
Ruland CM, Holte HH, Røislien J, Heaven C, Hamilton GA, Kristiansen J, Sandbæk H, Kvaløy SO, Hasund L, Ellison MC: Effects of a computer-supported interactive tailored patient assessment tool on patient care, symptom distress, and patients' need for symptom management support: A randomized clinical trial. J Am Med Inform Assoc. 2010, 17 (4): 403-410. 10.1136/jamia.2010.005660.
Velikova G, Keding A, Harley C, Cocks K, Booth L, Smith AB, Wright P, Selby PJ, Brown JM: Patients report improvements in continuity of care when quality of life assessments are used routinely in oncology practice: Secondary outcomes of a randomised controlled trial. Eur J Cancer. 2010, 46 (13): 2381-2388. 10.1016/j.ejca.2010.04.030.
Bainbridge BD, Seow H, Sussman J, Pond G, Martelli-Reid L, Herbert C, Evans W: Multidisciplinary health care professionals' perceptions of the use and utility of a symptom assessment system for oncology patients. Journal of Oncology Practice. 2011, 7 (1): 19-23. 10.1200/JOP.2010.000015.
Berry DL, Blumenstein BA, Halpenny B, Wolpin S, Fann JR, Austin-Seymour M, Bush N, Karras BT, Lober WB, McCorkle R: Enhancing patient-provider communication with the electronic self-report assessment for cancer: A randomized trial. J Clin Oncol. 2011, 29 (8): 1029-1035. 10.1200/JCO.2010.30.3909.
Takeuchi EE, Keding A, Awad N, Hofmann U, Campbell LJ, Selby PJ, Brown JM, Velikova G: Impact of patient-reported outcomes in oncology: A longitudinal analysis of patient-physician communication. J Clin Oncol. 2011, 29 (21): 2910-2917. 10.1200/JCO.2010.32.2453.
Grimshaw J, McAuley LM, Bero LA, Grilli R, Oxman AD, Ramsay C, Vale L, Zwarenstein M: Systematic reviews of the effectiveness of quality improvement strategies and programmes. Qual Saf Health Care. 2003, 12 (4): 298-303. 10.1136/qhc.12.4.298.
Methodological standards for the conduct of Cochrane Intervention Reviews: Cochrane Effective Practice and Organisation of Care Review Group. 2002, http://epoc.cochrane.org/epoc-resources; Accessd 2012
Guyatt GHO, Andrew D, Gunn E, Kunz R, Falck-Ytter Y, Alonso-Coello P: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. Br Med J. 2008, 336: 924-10.1136/bmj.39489.470347.AD.
Santana MJ, Feeny D, Johnson JA, McAlister FA, Kim D, Weinkauf J, Lien DC: Assessing the use of health-related quality of life measures in the routine clinical care of lung-transplant patients. Qual Life Res. 2010, 19 (3): 371-379. 10.1007/s11136-010-9599-3.
Fayers PM: Evaluating the effectiveness of using PROs in clinical practice: A role for cluster-randomised trials. Qual Life Res. 2008, 17 (10): 1315-1321. 10.1007/s11136-008-9391-9.
Basch E, Abernethy AP: Supporting clinical practice decisions with real-time patient-reported outcomes. J Clin Oncol. 2011, 29 (8): 954-956. 10.1200/JCO.2010.33.2668.
Abernethy AP, Wheeler JL, Zafar SY: Management of gastrointestinal symptoms in advanced cancer patients: The rapid learning cancer clinic model. Curr Opin Support Palliat Care. 2010, 4 (1): 36-45. 10.1097/SPC.0b013e32833575fd.
Berry DL: Patient-reported symptoms and quality of life integrated into clinical cancer care. Semin Oncol Nurs. 2011, 27 (3): 203-210. 10.1016/j.soncn.2011.04.005.
Pawson R, Manzano-Santaella A: A realist diagnostic workshop. Evaluation. 2012, 18 (2): 176-191. 10.1177/1356389012440912.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/13/211/prepub
This article is based on An Evidence Check rapid review brokered by the Sax Institute (http://www.saxinstitute.org.au) for the Cancer Institute NSW, 2012. The views expressed in this article are entirely those of the authors and should not be interpreted as that of the Sax Institute or the Cancer Institute NSW.
The authors declare that they have no competing interests.
JC contributed to the conceptualisation and design of the study. JC, LO, SH conducted the review. JC prepared the first draft of the paper and all other authors contributed to and approved the final manuscript.
Electronic supplementary material
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
Rights and permissions
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
About this article
Cite this article
Chen, J., Ou, L. & Hollis, S.J. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 13, 211 (2013). https://doi.org/10.1186/1472-6963-13-211
- Quality of Life
- Patient Reported Outcomes
- Health Services Research