Assessing and reporting patient participation by means of patient preferences and experiences
BMC Health Services Research volume 20, Article number: 702 (2020)
Although patient participation is strongly associated with high quality of healthcare, valid means to measure and report a comprehensive notion of patient participation are scarce. The Patient Preferences for Patient Participation (4Ps) is a new healthcare practice and research tool, comprising patients’ preferences as well as experiences. The 4Ps employs 12 items for the patient to conceptualise patient participation. The aim of this paper is to describe how the two perspectives of patient participation, namely preferences and experiences, can be combined to visualise and report preference-based patient participation.
With four response alternatives in each section, the 4Ps offers sixteen possible combinations of degree of match per item. Theoretical and clinical principles fostered a tentative order of six ranks and three levels of preference-based patient participation. To test the standard, statistical analyses for ordinal data were performed, using data from a randomised controlled trial evaluating an intervention aiming to improve patient participation. Further, structures for visualising the preference-based patient participation of individuals and groups were suggested.
Data from the 4Ps demonstrated the individuals’ preference-based patient participation, indicating either a match or a mismatch for each item. Mismatches represented either the experience of participation surpassing the patient’s preferences, or the patient’s preferences for patient participation not being established. At group level, the suggested approach for visualising and reporting the 4Ps demonstrated that the intervention group had a significantly higher proportion of sufficient preference-based patient participation for certain items than the control group. These results had not been identified earlier, when using the preferences and experiences of patient participation as separate measures.
Ways to easily acquaint stakeholders with patients’ preferences for patient participation are needed, in order for healthcare staff to better use resources to match the basic requirements of individuals and groups. While the 4Ps can guide professionals to patient participation as framed in legislations, concept analyses and by patients, a visualisation of the results is needed to capture preference-based patient participation. The proposed route to representing degree of match in preferences and experiences may also be relevant to other dimensions of quality of healthcare.
Modern healthcare allies with patient participation , although there is no single, common understanding of the concept [2,3,4]. Healthcare professionals often recount patient participation as related mainly to being engaged in decision-making , while patients depict it in a broader sense, including sharing experiences and information, shared decision-making, and self-management . Furthermore, patients’ priorities for participation have been found to vary: the way in which and to what extent one prefers to partake can alter with, for example, type of healthcare contact and the reason for it [6, 7].
Just as patient participation is agreed to be essential for just and safe healthcare , valid means to capture variations and thus measure and report a comprehensive notion of patient participation are scarce . With different attributes for patient participation at hand, instruments should incorporate all dimensions available, offering opportunities for patients to illustrate their experience in juxtaposition to their preferences. This is vital for assessing the quality of care and improvement opportunities .
For many quality aspects of healthcare, measures are fairly straightforward, targeting a ‘good’ experience (like ‘good access’ or ‘good services’). Yet, for a thick concept like patient participation, the complexity is more pronounced, and ‘good’ in terms of patient participation can fluctuate. For example, being engaged in healthcare decision-making can represent ‘good’ participation to a patient at a certain point – while for another individual, or the same person in a different situation, being expected to engage in a healthcare decision can represent ‘bad’ participation .
Thus, to explicitly measure and report healthcare quality in terms of patient participation, patient preferences for participation need to be considered alongside experiences. The 4Ps tool – standing for Patient Preferences for Patient Participation – is such a tool, developed by the first author based on a series of qualitative and quantitative studies on patients’ preferences for and experiences of patient participation [6, 11,12,13,14,15]. In former validation studies, the 4Ps has been found easy to comprehend and of conceptual clarity, whilst promising for measuring and evaluating patient participation in clinical practice and research [16, 17].
The 4Ps applies 12 attributes that have been found to illustrate and exhaust patient participation as a concept . These attributes are framed as items, echoed in the tool’s two sections: a section for depicting one’s preferences, i.e. the importance of each item for satisfactory participation, and one for experiences, that is, the extent to which one has experienced the attributes of patient participation. The two sections of the 4Ps employ four response options each:
For patient preferences, the response options for each item are that it is: 1) unimportant; 2) somewhat important; 3) very important, or; 4) crucial (for patient participation).
For patient experiences, response options are that the attribute (of patient participation) presented in the item has been experienced: 1) not at all; 2) to some extent; 3) to a large extent, or; 4) entirely .
Used within a clinical context, the 4Ps is proposed to stimulate mutual understanding between staff and patient: the patient completes the preferences section at the onset of a healthcare interaction and the experiences section at a later, agreed point. This allows for the healthcare professional(s) and patient to jointly assess the individual’s preferences and experiences, respectively, and, most importantly, the match (or mismatch) between the patient’s experiences in relation to his or her preferences – hence evaluating ‘preference-based patient participation’. From a clinical perspective, this serves to appraise the conditions provided for patient participation; such comparisons can also be gauged at a group level, providing an illustration of, for example, how a team or unit performs in terms of preference-based patient participation. For research purposes, the 4Ps’ sections are to be used concurrently; this permits discrete appraisals of patients’ preferences and experiences, respectively, along with evaluations of the level of match between their experiences in relation to their preferences. Thus, the 4Ps can illustrate differences between, for example, groups of patients or settings, or the effects of a clinical intervention on preference-based patient participation.
However, this require further means for how to analyse, visualise, and thus represent measures of participation, recognising ordinal data vis-à-vis experiences and preferences, including the match between the two. To date, there are few (if any) examples of such enterprises. In a previous study, the 4Ps was employed to evaluate the effects on patient participation of a self-management support programme in primary care for patients with chronic heart failure (CHF) or chronic obstructive pulmonary disease (COPD) , in addition to the primary outcome self-efficacy. The data on patient participation was analysed by summarising each participant’s preferences and experiences, respectively, and the results of the two measures were presented separately at group level. While a Rasch analysis served as a means for summarising person measures from the data for each section , this procedure did not permit the detection of the level of match between preferences and experiences. Further, the above methods failed to depict any of the enhanced conditions for patient participation transpiring in a qualitative follow-up of the same study .
To capture patients’ experiences of participation in accord with patient preferences, tools such as the 4Ps are needed. Yet, in order to fully represent patients’ experiences and preferences, that is juxtaposing exclusive response options which yet communicate one and the same concept, guidelines on how to create a combined measure are commended. The aim of this paper is to describe how the two perspectives of patient participation, namely preferences and experiences, can be combined to visualise and report preference-based patient participation.
Setting and sample
This investigation employs the dataset on patient participation of the above study, where the 4Ps tool was applied for evaluative purposes :
The data was collected between 2013 and 2015 in nine primary care centres across three regions in Sweden ; 79 patients were included in the intervention group and 83 in the control group. The intervention group had been block randomised for a self-management intervention programme: six 90-min group sessions every second week, led by a trained district nurse and a physiotherapist, addressing individual action plans and goals, and support for behavioural changes. Standard primary care was provided for both intervention and control group patients, the latter receiving no additional intervention during the study.
Patient preferences for and experiences of participation were measured using the 4Ps before and after the three-month intervention period . The 4Ps was administered and collated along with assorted outcome measures, by an assigned nurse or physiotherapist separate from the intervention and blinded to the patients’ allocation to intervention or control group. The analysis is based on the preferences before the intervention and the experiences after.
Development of a match-rank for preference-based patient participation
Given the 4Ps’ four response alternatives in two sections, there are 16 possible combinations in terms of the degree of match between a patient’s preference and experience for each item. To demarcate the different levels of preference-based patient participation, a ranking for the degree of match was established through a deliberate process guided by two empirical principles:
The closer the match, the better.
It is better to get more than you preferred, than less.
The principles descend from both previous studies on patient participation [6, 7, 11,12,13,14,15,16,17, 21] and healthcare norms, literally transpiring through F. Nightingale’s momentum to provide healthcare services of high quality yet in accordance with the desires of the patient , to today’s mutual engagement of the patient and the healthcare professionals on the patient’s team by means of for example person-centred care .
The principles signified that the 16 combinations of degree of match (between patient preferences and experiences for patient participation) could be categorised into six different ranks, ranging from zero to five. To illustrate, according to the principles the four combinations that represent a complete match all represent the top rank (rank five). Consequently, for example, the rank of an item with preference Crucial and experience Entirely is equivalent to the rank of an item with preference Unimportant and experience Not at all. These both represent rank five, that is a top match while an item with preference Crucial and experience Not at all represents the lowest possible rank, zero.
Furthermore, in order to provide further guidance with reference to patient participation in relation to quality of care, the ranks of preference-based patient participation were classified into three levels:
Insufficient provision of preference-based patient participation, calling for significant progress (ranks 0–1),
Fair provision, indicating room and need for improvement (ranks 2–3), and
Sufficient provision, signifying complete or near match of patient preferences and experiences (ranks 4–5).
These levels were derived by an empirical approach, suggested to guide clinicians as well as scholars in further deliberating what is ‘good’ (or less appropriate) quality of care in terms of preference-based patient participation. For an overview of the 16 combinations of degree of match, the six ranks and the three levels of preference-based patient participation, see Fig. 1.
Visualising and reporting of data
The data from the above study  were employed to visualise the outcomes in accordance with the match-rank between the preferences before and the experiences after the intervention. To start, individual profiles were formed, electing two individuals from the data set to illustrate the procedure and results. For the reporting of individual profiles, symbols were proposed to visualise each participant’s preference-based patient participation in relation to insufficient, fair or sufficient provision of preference-based patient participation. While this route is primarily proposed for clinical purposes, it may provide additional value to exemplify research results.
Furthermore, an analysis was performed to report whether, and to what extent, the outcomes indicate a difference between intervention and control groups in terms of patient participation in the trial, by means of the match-ranking. This was completed using patient group profiles. In the patient group profiling, the percentage of group members who fell into each of the three levels of preference-based patient participation is reported, using a figure at item level. This includes comparisons of proportions at group level, with chi-square tests between intervention and control groups. This is essentially proposed for research purposes but can serve a more comprehensive evaluation of quality of care.
As noted, although these profiles predominantly apply to clinical and research employment of the 4Ps tool, respectively, they can be used to serve either purpose. Rather, they are supposedly imperative for visualising and reporting the outcomes of the 4Ps.
Preference-based patient participation – individual profiles
Preference-based patient participation for individuals is represented as a chart, including the actual ratings of preferences (P) and experiences (E), respectively. These are placed in four corresponding columns in the table, where symbols indicate the classified level of the rank. This suffices to provide an image demonstrating whether a mismatch signified that the experiences were rated as lower or higher than the individual’s preferences. Examples of the reporting of two individuals are shown in Table 1, illustrating that patient 18’s data showed a pattern where only the conditions for reciprocal communication were deemed sufficient, with a complete match between preference and experience. All other items showed less participation than preferred; in four items there was an almost complete divergence, with the preference for patient participation being “crucial” and the experience only “to some extent”. Patient 66 data, on the other hand, illustrated sufficient preference-based patient participation for all but one aspect. However, a complete match occurred only for one item (“managing treatment”). For “learning of plans”, the one item assigned as crucial for patient participation, the experienced participation was less than the individual’s preference.
Preference-based patient participation – group profiles
Analysing the data from the above study from a preference-based patient participation perspective at group level, the intervention group was found to have more preference-based patient participation in certain items (see Fig. 2). For example, the match-rank indicated that the experiences of the intervention group to a larger extent matched their preferences in terms of “taking part in planning” (69 versus 43% of the participants having a sufficient match, p = 0.01) and “phrasing personal goals” (74 versus 55% with a sufficient match, p = 0.04), when compared to the control group. In addition, for three items, in the control group there were at least 10 percentage points more of insufficient match between preferences for and experiences of patient participation. For example, in item 4 “Sharing one’s symptoms”, 12% of patients in the control group had an insufficient match compared to 2% of patients in the intervention group (p = 0.033).
When investigating a concept like patient participation, which is considered significant and often laden with positive connotations even if not thoroughly seized, insight into patients’ preferences is crucial; in order to provide for experiences that are significant to patients, the experiences need to match the notion that applies in a particular healthcare setting and/or at a particular time point. We propose this represents preference-based patient participation. The method of examining, reporting and visualising preference-based patient participation proposed in this paper offers a novel approach; while there are particular quality of care assessments including both to what extent patients have perceived certain aspects of care and how important they consider these to be [24, 25], the 4Ps offers opportunities both to prospect patients’ preferences for patient participation and to evaluate to what level patient experiences match these preferences – indicating to what extent there are conditions for patient participation that align with standards . The match between preferences and experiences represents whether participation has been good or not – and reveals any need for improvements to procure sufficient quality of care. While the structure and content of the 4Ps can suffice conditions for patient participation, based on an improved understanding of the patient’s voice and choice, as both individuals and groups, further studies are suggested to identify the ideal actions, actors, and points of care and time for employing the 4Ps .
For now, as the 4Ps tool does not provide a common route whereby a higher degree of all types of participation is consequentially better, a match-rank between preferences and experiences is important in order to visualise what constitutes ‘good’ and ‘bad’ patient participation, respectively [26, 27]. By using a match-rank, the fit of the participation experienced by the patient relative to the preference profile is measured, making it possible to illustrate quality of care from the perspective of both individual and group-level data . A summarised, Rasch-based person measure for preferences and experiences respectively was employed in earlier reporting of the 4Ps as an outcome measure . This summarised score, although clear and easy to use in statistical analyses, comes with a disadvantage: single scores are associated with an assumption that more is better, although it did not take the individuals’ preferences into account when evaluating their experiences. Rather, the alternative route applied here convey harmony with the qualitative follow-up of the same study, indicating that the intervention group experienced enhanced opportunities for taking responsibility for one’s own goals and assimilating the knowledge shared . Thus, the measures applied in this secondary analysis can serve both scholars and clinicians to comprehend the 4Ps outcomes in terms of preference-based participation, both at an individual patient level and at a group level. Presumably, reporting and visualising the 4Ps results will also aid the identification of needs for improvement with regard to patient participation.
Opportunities to easily acquaint healthcare professionals with patients’ primary preferences are scarce, although this would reveal how healthcare staff can better use resources to match the basic requirements of individuals and groups . A match between an individual’s preferences for participation and experienced participation supposedly indicates perceptions of a high quality of care; consequently, supporting patients in attaining their preferred level of participation might be of greater importance to them than increasing their desire to enact participation by means of activities and decisions . A tool like the 4Ps can also aid patients who are less prone to voice their preferences , providing them with an opportunity to obtain the kind and level of participation they prefer, aiding them in conceptualising and sharing preferences, guided by what is semantically and judicially framed as participation.
The visualisation of patterns in what is expected in terms of opportunities for patient participation and what works to support matching experiences can inform quality-improvement initiatives. With opportunities to repeat both sections of the 4Ps, this may well show that, over time, both preferences and experiences alter, potentially with an improved match as dialogues and awareness increase . Furthermore, with the 4Ps and the visualisations and reporting standards suggested, the results of particular clinical interventions can be compared between subgroups, units, departments, hospitals or regions. Findings can also be evaluated over time. However, with significant differences noted at the item level, rather than in sum scores, the data from the 4Ps necessitates an item level . This is essential to recognise, because differences between small groups need to be detected for example, when evaluating preference-based patient participation within departments or patient groups. In addition, further studies are needed to understand if certain preferences for patient participation are easier to meet than others, and whether this account for particular or all attributes.
In the study using summarised person measures for one section at a time, which used the same data set, no significant differences between intervention and control groups were identified . When using preference-based ranks, a more differentiated result emerged, showing that there were some items where preference-based participation was better in the intervention group than the control group. This indicates the possibility of the intervention having an effect on some aspects of patient participation, that was concealed when using summarized measures and not relating experiences to preferences on an individual basis.
While patient participation is still habitually associated with engagement in decision-making regarding one’s own health and healthcare, the voices of patients signify a wider conceptualisation, incorporating attributes depicted in the 4Ps relating to shared decision-making, patient-centred care, person-centred care and self-management [18, 34]. Thus, the primary notion of participation as in ‘sharing of’ (usually experiences, knowledge and information) and ‘sharing in’ (actions related to healthcare and self-care) seems vital for good care . Clinical employment of the 4Ps requires balancing individuals’ recognition of preferences with experiences at group and service levels, although prospects for a shared understanding of expectations and conditions are likely valuable for both patients and professionals. Still, introducing opportunities to share a combined measure of preferences for and experiences of patient participation in healthcare dialogues is novel, and like most new procedures, applications of the 4Ps in clinical settings are likely to entail some degree of effort . Consequently, there is a need to investigate which strategies enable implementation of an innovation like the 4Ps, and whether and how this facilitates preference-based patient participation in different contexts [37,38,39].
In this study, we have visualised preference-based patient participation, at both individual and group levels, with a focus on presenting results that are easily interpreted and meaningful to both patients and healthcare staff, as well as scientists. The route to presenting and visualising a match in preferences and experiences may also be relevant to other dimensions of quality in healthcare, suggesting means additional to for example the Rasch person measure in cases where the concept investigated requires a combination of preferences and experiences to indicate whether the outcome is good, or any level of less fitting.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
WHO. Exploring patient participation in reducing health-care-related safety risks. Copenhagen: WHO Regional Office for Europe; 2013.
Cahill J. Patient participation: a concept analysis. J Adv Nurs. 1996;24:561–71.
Sahlsten M, Larsson I, Sjöström B, Plos K. An analysis of the concept of patient participation. Nurs Forum. 2008;43(1):2–11.
Thorarinsdottir K, Kristjansson K. Patients’ perspectives on person-centred participation in healthcare: a framework analysis. Nurs Ethics. 2014;21(2):129–47.
Castro EM, Van Regenmortel T, Vanhaecht K, Sermeus W, Van Hecke A. Patient empowerment, patient participation and patient-centeredness in hospital care: a concept analysis based on a literature review. Patient Educ Couns. 2016;99(12):1923–39.
Eldh AC, Ekman I, Ehnfors M. A comparison of the concept of patient participation and patients’ descriptions as related to health care definitions. Int J Nurs Class Term. 2010;21(1):21–32.
Årestedt L, Martinsson C, Hjelm C, Uhlin F, Eldh AC. Patient participation in dialysis care—a qualitative study of patients’ and health professionals’ perspectives. Health Expect. 2019;22(6):1285–93.
Anell A, Glenngard AH, Merkur S. Sweden: health system review. 2012. Report No.: 1817–6119 Contract No.: 5.
Phillips NM, Street M, Haesler E. A systematic review of reliable and valid tools for the measurement of patient participation in healthcare. BMJ Qual Saf. 2016;25(2):110–7.
Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care. 2003;41(1 Suppl):I30–8.
Eldh AC, Ekman I, Ehnfors M. Considering patient non-participation in health care. Health Expect. 2008;11(3):263–71.
Eldh AC, Ehnfors M, Ekman I. The phenomena of participation and non-participation in health care: experiences of patients attending a nurse-led clinic for chronic heart failure. Eur J Cardiovasc Nurs. 2004;3:239–46.
Eldh AC, Ehnfors M, Ekman I. The meaning of patient participation for patients and nurses at a nurse-led outpatient clinic for chronic heart failure. Eur J Cardiovasc Nurs. 2006;5:45–53.
Eldh AC, Ekman I, Ehnfors M. Conditions for patient participation and non-participation in health care. Nurs Ethics. 2006;13(5):503–14.
Eldh AC. Patient participation - what it is and what it is not. [doctoral thesis]. Örebro: Örebro university; 2006.
Eldh AC, Luhr K, Ehnfors M. The development and initial validation of a clinical tool for patients' preferences on patient participation - the 4Ps. Health Expect. 2015;18(6):2522–35.
Luhr K, Eldh AC, Nilsson U, Holmefur M. Patient preferences for patient participation – psychometric evaluation of the 4Ps tool in patients with chronic heart or lung disorders. Nord J Nurs Res. 2018;38(2):68–76.
Eldh AC. Facilitating patient participation by embracing patients’ preferences – a discussion. J Eval Clin Pract. 2019;25(6):1070–3.
Zakrisson AB, Arne M, Hasselgren M, Lisspers K, Ställberg B, Theander K. A complex intervention of self-management for patients with COPD or CHF in primary care improved performance and satisfaction with regard to own selected activities; a longitudinal follow-up. J Adv Nurs. 2019;75(1):175–86.
Luhr K, Eldh AC, Theander K, Holmefur M. Effects of a self-management program on patient participation in patients with congestive heart failure or chronic obstructive pulmonary disease: a randomised controlled trial. Eur J Cardiovasc Nurs. 2018;18(3):185–93.
Luhr K, Holmefur M, Theander K, Eldh AC. Patient participation during and after a self-management programme in primary health care - the experience of patients with chronic obstructive pulmonary disease or chronic heart failure. Patient Educ Couns. 2018;101(6):1137–42.
Nightingale F. Notes on nursing: what it is and what it is not. New York: Dover Publications; 1969.
Summer Meranius M, Holmström I, Håkansson J, Breitholtz A, Moniri F, Skogevall S, et al. Paradoxes of person-centred care: a discussion paper. Nurs Open. 2020;00:1–9.
Wilde Larsson B, Larsson G. Development of a short form of the quality from the Patient's perspective (QPP) questionnaire. J Clin Nurs. 2002;11(5):681–7.
Degner LF, Sloan JA, Venkatesh P. The control preferences scale. Can J Nurs Res. 1997;29(3):21–43.
Sandsdalen T, Rystedt I, Grøndahl VA, Hov R, Høye S, Wilde-Larsson B. Patients' perceptions of palliative care: adaptation of the quality from the Patient's perspective instrument for use in palliative care, and description of patients' perceptions of care received. BMC Palliat Care. 2015;14:54.
Mansfield E, Bryant J, Carey M, Turon H, Henskens F, Grady A. Getting the right fit: convergence between preferred and perceived involvement in treatment decision making among medical oncology outpatients. Health Sci Rep. 2018;2(1):e101.
Bleich SN, Ozaltin E, Murray CK. How does satisfaction with the health-care system relate to patient experience? Bull World Health Organ. 2009;87(4):271–8.
Santana MJ, Manalili K, Jolley RJ, Zelinsky S, Quan H, Lu M. How to practice person-centred care: a conceptual framework. Health Expect. 2018;21(2):429–40.
Lin C, Cohen E, Livingston PM, Mohebbi M, Botti M. Achievement of patients' preferences for participation in oncological symptom management and its association with perceived quality of care. Patient Prefer Adherence. 2018;13:83–90.
Foss C. Elders and patient participation revisited – a discourse analytic approach to older persons’ reflections on patient participation. J Clin Nurs. 2011;20(13–14):2014–22.
Feo R, Frensham LJ, Conroy T, Kitson A. "It's just common sense": preconceptions and myths regarding fundamental care. Nurse Educ Pract. 2019;36:82–4.
Wasserstein RL, Schirm AL, Lazar NA. Moving to a world beyond "p < 0.05". Am Stat. 2019;73(1):1–19.
Håkansson Eklund J, Holmström IK, Kumlin T, Kaminsky E, Skoglund K, Höglander J, et al. “Same same or different?” a review of reviews of person-centered and patient-centered care. Patient Educ Couns. 2019;102(1):3–11.
Mako T, Svanäng P, Bjerså K. Patients' perceptions of the meaning of good care in surgical care: a grounded theory study. BMC Nurs. 2016;15:47.
Kvæl LAH, Debesay J, Langaas A, Bye A, Bergland A. A concept analysis of patient participation in intermediate care. Patient Educ Couns. 2018;101(8):1337–50.
Wensing M, Wollersheim H, Grol R. Organizational interventions to implement improvements in patient care: a structured review of reviews. Implement Sci. 2006;1:2.
Eldh AC, Almost J, DeCorby-Watson K, Gifford W, Harvey G, Hasson H, et al. Clinical interventions, implementation interventions, and the potential greyness in between - a discussion paper. BMC Health Serv Res. 2017;17:16.
Castro EM, Malfait S, Van Regenmortel T, Van Hecke A, Sermeus W, Vanhaecht K. Co-design for implementing patient participation in hospital services: a discussion paper. Patient Educ Couns. 2018;101:1302–5.
Open access funding provided by Linköping University.
Ethics approval and consent to participate
The primary study, from which the data was retrieved for the current analyses, was ethically approved by The Regional Ethical Review Board of Uppsala, Sweden, (number 2012/189). Participants provided written informed consent in the chief study, including the data collection, analyses and reporting of the 4Ps.
Consent for publication
The authors declare no conflict of interest. The 4Ps is available free of charge from the first author, licensed as non-commercial by means of Creative Commons.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Eldh, A.C., Holmefur, M., Luhr, K. et al. Assessing and reporting patient participation by means of patient preferences and experiences. BMC Health Serv Res 20, 702 (2020). https://doi.org/10.1186/s12913-020-05574-y