Patient-relevant outcomes: what are we talking about? A scoping review to improve conceptual clarity

Background With respect to patient-centered care, measuring care effects based on patient-relevant outcomes is becoming increasingly important. There is some uncertainty about what outcomes are particularly relevant to patients and who determines their relevance. To determine this, we conducted a scoping review of the international literature with the aim to improve the conceptual clarity regarding (1) the terminology used for supposedly patient-relevant outcomes, (2) the variety of outcomes considered patient-relevant, and (3) justifications for the choice of these specific outcomes. Methods We conducted a systematic search in Embase, PubMed (including Medline), Cochrane Central, Scopus, and Google Scholar with a special focus on article titles. Search terms included patient-relevant, patient-important, patient-preferred, and outcome(s), endpoint(s), parameter(s), indicator(s). We limited the search period from January 2000 to July 2019. Full-text articles reporting outcomes that were described as patient-relevant met the inclusion criteria. Two researchers independently analyzed all eligible articles applying quantitative and structuring content analysis. Results We identified 155 articles, 44 of which met the inclusion criteria. A content analysis revealed 35 different terms used with regard to patient-relevant outcomes. However, authors predominantly referred to patient-important outcomes (23 articles, 52.3%) and patient-relevant outcomes (17 articles, 38.6%). A structuring content analysis of all extracted outcomes revealed a total of 281 codes, pooled in 32 inductive categories. Among these, the following categories dominated: symptoms, adverse events/complications, survival/mortality, pain. In just 16 of the articles (36.4%), authors provided justifications for the choice of the outcome being based either on patient and/or expert opinions. In another 13 articles (29.5%), no justification was provided. Conclusion This scoping review on patient-relevant outcomes was driven by the questions (1) what outcomes are particularly relevant to patients, and (2) who determines their relevance. We found a wide range of supposedly patient-relevant outcomes, with only one third of articles involving patients in the justification of the outcome selection. In view of this conceptual uncertainty it appears difficult to determine or even to compare a particular patient benefit of interventions. A set of generic outcomes relevant to patients would be helpful to contribute to a consistent understanding of patient relevance.


Background
Patient-centered care aims to place individuals, their values, preferences, life and health goals at the heart of the care process and to actively involve patients in care decisions [1]. In the sense of shared decision-making, involving patients in decisions implies that patients are adequately informed about existing care options and their potential effects, understand these options, and are given the opportunity to explore what is most relevant to them in order to make a choice based on their personal preferences [2]. In our understanding, this means that for shared decision-making, effect measurements based on parameters that matter to patients are urgently needed. This assumes that studies examine the effects of care that is based on outcomes which are relevant to patients, but so far systematic reviews conclude that such patientrelevant outcomes are underrepresented in recent clinical trials [3][4][5]. Indeed, this raises the question as to what outcomes are relevant to patients and who determines their relevance.
In Germany, the Institute for Quality and Efficiency in Health Care (IQEHC) officially examines the benefits and harms of medical interventions for patients. To this end, the IQEHC considers parameters to be relevant to patients when they represent how a patient feels, functions or survives; notably mortality, morbidity, and quality of life [6]. However, this understanding of patient relevance is based on a definition of the Biomarkers Definition Working Group on clinical endpoints, which does not contain any information about patient relevance [7]. Similarly to the IQEHC's understanding of patient-relevant outcomes, the working group on quality of care and patient safety research of the German Network for Health Services Research (DNVF) mentions survival and quality of life as factors that are relevant to patients, but also includes social aspects, such as social reintegration, in its interpretation [8]. Generally, these outcomes are based on the understanding that patientrelevant outcomes reflect the effects of changes in the individual patient's health status [8].
However, neither the definition provided by the IQEHC nor that provided by the working group of the DNVF offered us a comprehensive explanation as to why the outcomes mentioned are considered to be relevant to patients. Shifting the view from the German to the international context, we aimed to examine which understanding of patient relevance and which outcomes are common to international research and thereby hoped to improve the conceptual clarity of patient relevance. To this end, we conducted a scoping review of the international literature with regard to the following research questions: (1) What terminology is used for supposedly patient-relevant outcomes? (2) What outcomes are considered to be relevant to patients? (3) What explanations are provided to justify the relevance of these specific outcomes for patients?

Methods
We report our scoping review in accordance with the PRISMA extension for scoping reviews (PRISMA-ScR) [9].

Data sources and search strategy
We conducted a systematic literature search in Embase, PubMed (including Medline), Cochrane Central, Scopus, and Google Scholar. Our search strategy considered German and English references published between January 1st, 2000 and July 31st, 2019. Restricting the search period to the past 20 years was considered reasonable to concentrate on more recent research. In order to identify references that clearly focus on the topic of interest, we additionally restricted the search to the article titles. Titles had to include at least one of the following terms: (patient-relevant OR patient-important OR patientpreferred) AND (outcome(s) OR endpoint(s) OR parameter(s) OR indicator(s)). The full search strategies for each database are outlined in Additional file 1.

Study selection
After deleting all duplicates, two of the researchers (Christine Kersting (CK), Malte Kneer (MK)) reviewed all records retrieved from the database search: In the first step, we independently screened the records to limit the search results to English-and German-language articles with full-text availability published in journals. This was reasonable as we aimed to examine whether there is, in fact, a common and consistent understanding of patient relevance in international research. Accordingly, we excluded doctoral theses if not officially published, conference abstracts, commentaries on previously published articles, opinions, debates, or editorials, records on other topics, and those without full-text availability. In the second step, we independently reviewed the full texts of the remaining articles to check whether they met the inclusion criteria for our review. We defined articles as eligible when they reported outcomes that were described by the authors as relevant to patients. In case of disagreements we discussed these until a consensus was achieved.

Data extraction and analysis
In line with the three research questions of this scoping review, we analyzed all eligible articles regarding (1) the terminology used, (2) the outcomes described as patientrelevant, and (3) the justifications for why these outcomes were considered to be of relevance to patients. For the analysis we used both quantitative content analysis and qualitative structuring content analysis [10].

Terminology
All terms that were used to describe patient-relevant outcomes were assessed and extracted in original spelling to collect synonyms; however, paraphrases such as "outcomes that matter to patients" were not considered synonyms. Finally, we applied frequency calculations to quantify the number of different terms used per article and the overall number of different terms identified.

Outcomes
In a second step all outcomes described as relevant to patients were extracted from the articles and structured for the underlying issues (thematically) using inductive categories. The code set was extended continuously as new categories emerged during the full-text analyses. We additionally examined whether specific outcome categories were used more often than others by calculating frequencies and percentages.

Justification
We analyzed whether the articles provided a justification as to why the outcomes described were considered to be relevant to patients. These justifications were categorized inductively. Based on this, we distributed the outcome categories that were identified in the previous step to the justifications provided.
In a subgroup analysis we focused on articles that actively involved patients or experts in the definition of patient-relevant outcomes. We examined whether the outcomes defined in these articles were valid for specific patient groups only and whether these outcomes were different from those described as relevant to patients in articles that did not involve patients or experts. Additionally, we stratified for studies involving patients, studies involving experts, and those involving both, and compared the outcome categories between these groups.

Literature search
Excluding duplicates, the literature search yielded 155 records (Fig. 1). During the first screening, we excluded 87 records because the full-text papers were unavailable, not written in English or German, focused on another topic, or were conference abstracts, commentaries on previously published articles, opinions, debates, or editorials. Full-text screening of the remaining 68 articles resulted in 44 articles fulfilling the inclusion criteria for this review. Details on all studies are provided in Tables 1, 2, 3 and 4.

Terminology
Based on the 44 articles included in this review, we identified 35 different terms for patient-relevant outcomes ( Table 5). Of these 35 terms, the two most frequently used terms were patient-important outcome (identified in 23 articles, 52.3%) and patient-relevant outcome (identified in 17 articles, 38.6%).
In 21 articles (47.7%) one term was used consistently for supposedly patient-relevant outcomes; in 14 articles (31.8%) two terms were used, and in the remaining nine articles (20.5%) up to six different terms were used to describe the outcomes. We were not able to identify different patterns in terminology across countries. The terms identified per article are demonstrated in Tables 1, 2, 3 and 4.

Justification
In about one third (n = 13, 29.5%) of the 44 articles analyzed, no reason was provided why the outcomes reported were considered to be relevant to patients (Table 4). However, in other studies we found different explanations justifying the selection of patient-relevant outcomes that refer mainly to three core issues:   [13, 15-17, 20, 22-25] or to experts only (n = 2) [14,26] or to both patients' and experts' opinion (n = 5) [11,12,18,19,21]. One study on early childhood disease included caregivers instead of patients [21], another one on critical disease included caregivers in addition to patients [13]. Sample sizes differed largely from smaller qualitative studies including 10 to 30 patients recruited in the health care setting [21,22,24] to a cross-sectional study in a random sample of about 2000 individuals invited by mail [20]. 2. Referral to other studies [4,5,[27][28][29][30][31][32][33][34] or special classification [35,36] was used as another source of supposedly patient-relevant outcomes ( Table 2). These articles (n = 12, 27.3%) referred their choice of outcomes mainly to reviews [4,5,27,28,34] or to a specific classification such as the International Classification of Functioning, Disability and Health [36]. 3. A few articles considered patient-reported outcomes equivalent to patient-reported outcomes (n = 3, 6.8%; Table 3). In one of these articles authors argued that patient-relevant outcomes rely exclusively on the information provided by patients themselves [39].
The different shading of the bars in Fig. 2 illustrates the allocation of the outcomes extracted from the studies considering justifications of patient relevance provided by the authors. It shows that more popular outcome categories were justified on the basis of different explanations, whereas more seldom categories and those based on social aspects were commonly traced back to patients and/or experts.

Subgroup analysis of articles involving patients and/or experts
All articles that had actively involved patients and/or experts focused on a specific disease (Table 1), but outcomes considered patient-relevant showed a widespread distribution with regard to the 32 inductive categories identified, referring to as many as 29 different categories. However, these studies did not include outcomes related to satisfaction, self-efficacy, or disease-specific quality of life/health status (Fig. 2). Interestingly, we found that 12 of the 32 inductive categories were only used in articles with patient and/or expert involvement such as physiological functioning, ability to fulfil social functions, independency, reliability of diagnosis and therapy, and confidence in therapy. However, outcomes most commonly described as patient-relevant, such as symptoms, adverse events/complications, survival/mortality, and (co)morbidity/secondary disease were identical to those most commonly described as patient-relevant in the other 28 articles not involving patients and/or experts (Fig. 2). Stratifying articles involving patients and/or experts (n = 16) by the manner of involvement showed that survival/mortality was mentioned especially in studies involving experts only. On the other hand, outcomes identified only in studies involving patients and/or experts (i.e. physiological functioning, reliability of diagnosis and therapy, or confidence in therapy) were traced back to those studies involving either patients only or both patients and experts.

Discussion
In this scoping review, we analyzed the international literature of the last 20 years with regard to patient relevance. We found a large variety of terms used as well as numerous supposedly patient-relevant outcomes. No more than one third of the articles referred to the patients', the caregivers' or the experts' opinions to explain the relevance of the outcomes reported. All in all, we were not able to identify a consistent concept or understanding of patient-relevant outcomes. Table 6 summarizes the main findings and implications for future research.
According to our analysis, the reasons for the identified ambiguity appear to be heterogeneous. Some articles did not differentiate between patient-reported and patient-relevant outcomes [37][38][39]. Other articles emphasized a potential overlap of the terms, but noted that not every outcome which can be reported by patients necessarily has to be relevant for them [16,18,23,24,41]. Two articles on chronically ill children expanded the terminology from patients to the family to underline the social context and impact [21,36]. In both studies, the target group was involved in the definition of relevant outcomes. Indeed, such patient-or target groupdriven approaches are not common practice yet: two thirds of the articles we analyzed referred to previous  articles [4,5,[27][28][29][30][31][32][33][34] or did not even provide a justification as to why the reported outcomes were considered to be relevant to patients [3,[40][41][42][43][44][45][46][47][48][49][50][51]. With respect to both of these justifications, Murad et al. (2011) pointed out that neither outcomes considered patient-relevant in other studies nor those described as relevant to patients from a researchers' perspective necessarily represent what really matters to patients [20]. In addition, our subgroup analysis on studies involving patients and/or experts suggested that outcomes defined solely by experts in sense of health care professionals might not necessarily represent the patients' perspectives as well. Transferring this to practice implies that patient-driven approaches are required: Patients or representatives need to be asked when defining what is relevant for them. Referring to the patients' orin the case of young children or critical carethe caregivers' opinions, such patientdriven approaches were identified in 14 articles included in our scoping review, but were limited to specific diseases such as cardiovascular disorders [11][12][13][15][16][17][18][19][20][21][22][23][24][25]. For these particular patients, adverse events like myocardial infarction or stroke might be important outcomes but may not necessarily be applicable to other patient groups as specific diseases are commonly associated with very specific needs and preferences that cannot be generalized.
The number of outcomes identified in the context of patient relevance with 32 inductive categories certainly exceeded our expectations. However, outcomes representing social dimensions including the ability to perform daily activities or to fulfil social functions were apparently underrepresented, especially in studies not involving patients and experts. The variety of outcome measures as well as the lack of a consistent concept with regard to outcomes representing patient relevance limit the comparability of study results and make it difficult to determine the particular patient benefit of different interventions. This problem is also known from patientcentered care: Based on a systematic review and concept analysis of 417 articles, Scholl et al. (2014) found that models on patient-centeredness lack conceptual clarity, resulting in heterogeneous terminology and inconsistent outcome measures [52]. Even in the context of patientrelevant outcomes, this criticism was already voiced years ago, for example by Cleveringa et al. (2010) [42].
Meanwhile, promising initiatives like the European Qualitative research project on Patient-preferred outcomes in Early Rheumatoid Arthritis [53] or the  [11,12]. While such initiatives usually focus on individual diseases, the ICHOM has also started to develop outcome sets across diseases (e.g., overall adult health, older persons, overall pediatric health) [55]. Providing validated instruments for each outcome set, the ICHOM may have the potential to standardize outcome measurement in (clinical) trials, even though the initiative originally aimed to standardize the assessment of routine care data across countries to improve the quality of healthcare for common diseases worldwide.
Based on this scoping review we aimed to improve the conceptual clarity on patient relevance. Since the results do not allow us to derive a clear concept, in a next step we plan to close this gap by conducting a study among patients, health care professionals, and researchers. Unlike the approaches described above, we not only aim to identify a core set of generic rather than disease-specific patient-relevant outcomes, but also to contribute to a clear definition and understanding thereof. Based on the findings of this review, patients and health care professionals will be involved to define outcomes that are relevant to patients across diseases and important components of an adequate definition. Researchers will additionally ensure that the concept is feasible for research purposes. Our overarching long-term objective is to increase the comparability of study results with regard to patient relevance.

Limitations
A key strength of this scoping review is the systematic approach: The terms and outcomes were systematically and independently extracted by two researchers. Also, the literature search was not limited to fixed term sequences, but considered combinations of terms, e.g. "patient-relevant" and "outcome" instead of "patient-relevant outcome". Due to this approach, articles with titles including a statement or question like "Are outcomes reported patient-relevant?" were also covered. Nevertheless, the search strategy focused only on the titles of articles. Thus, it cannot be excluded that we missed relevant articles. This is further aggravated by the fact that the terminology used in the literature lacks consistency. For example, we did not consider search terms like "patient-centered outcomes" as we aimed to focus on outcomes that are relevant to patients, but these terms might have been used as synonyms in some articles as well. Finally, the restriction to full-text availability might represent another limitation. However, the abstracts alone did not provide sufficient information regarding supposedly patient-relevant outcomes and outcome justification.

Conclusions
Returning to the initial question what outcomes are particularly relevant to patients and who determines their relevance, we conclude that recent studies use a variety of outcomes without asking patients what really matters to them. We were unable to identify either a sound definition or a consistent outcome set of patient-relevant outcomes, not even in the few studies that actively involved patients. In our opinion, consensus on a consistent terminology and set of generic patient-relevant outcomes is needed to adequately operationalize patient-centered care, increase the comparability of study results, and thereby enable patients to make choices regarding therapy in the context of shared decision-making.  -large variety of terms found in the international literature -inconsistency of terms across and within papers -possible confusion between patientrelevant and patient-reported -consistent concept of patient relevance is needed -standard set of patient-relevant outcomes is needed -patients must be involved when defining what is relevant to them -consistency regarding patient relevance will improve the comparability of study results and enable patients to make informed choices What outcomes are considered to be relevant to patients? -wide range of outcomes found -commonly disease-specific outcomes -social dimensions underrepresented What explanations are provided to justify the relevance of these specific outcomes for patients? -one third of studies without any explanation -another third refers to the opinion of patients, experts, or caregivers as patient representatives -outcomes exclusively defined by experts do not necessarily represent patients' preferences