We critically analysed 26 tools that aimed to support decision making or patient understanding in the treatment of acute stroke with thrombolysis, and the majority demonstrated considerable deficiencies. Only nine tools included information on a full range of both benefits and risks of treatment with thrombolysis. The great majority of tools used frequencies to convey probabilistic information on acute stroke outcomes. However, a majority of tools also used verbal descriptors and percentages, which can cause difficulties with understanding and interpretation [60, 61]. Furthermore, many tools only presented frequencies for a good outcome, and used verbal descriptors or percentages to convey information on adverse outcomes. Whilst the great majority of tools specified the patient group to which the outcome probabilities applied, they did not compare outcomes with and without thrombolysis, did not use identical denominators and time horizons, and failed to adhere to other good practice guidance on presentation of outcome probabilities. Outcome probabilities were also frequently underpinned by aggregate-level evidence (where stated) on the benefit to harm ratios derived from randomised controlled trials, which may not necessarily reflect outcomes for individual patients within routine practice . Graphical methods were used infrequently to convey probabilistic information, despite evidence that they can enhance risk perception by exploiting rapid (automatic) perceptual capabilities of users . Patient information tools would be understood by the majority of patients/relatives; but not those with low health literacy. It is recommended that health information materials should be written at no higher than 5th to 6th grade reading levels [64, 65], in order to maximise universal understanding in the adult population [equating to Fog Index scores of < 7]. Deficiencies in information content of patient information tools were also identified. The great majority of tools lacked development informed by theory or involvement of clinicians, patients and relatives, and were not pilot tested in actual acute stroke settings.
Our findings are not unique to tools designed to support decision-making and patient understanding in the treatment acute stroke with thrombolysis. They reflect those of previous studies in other clinical settings/domains that have identified deficiencies in information content, methods of conveying probabilistic information, and lack of comprehensive development processes [6, 66–70].
Strengths and limitations
The strengths of this review include a comprehensive search strategy and critical appraisal of tools across multiple relevant domains. The authors also have experience of developing and evaluating tools to support decision making about preference-sensitive treatment options, thus adding further context expertise to the review. In terms of weaknesses, we may have missed tools that are unpublished or unavailable via the Internet, although we also surveyed key clinical networks. We may have also missed unpublished information on development processes for tools.
Implications for research
Our findings have implications that go beyond tools for this clinical setting to other contexts where decision support and risk presentation is being developed. In order to address the shortcomings of tools identified in this review, including analogous tools to support patient understanding and decision making about other preference-sensitive medical or surgical treatment options, tool development should adhere to a structured development process, recommendations of previous research [66–69] and published guidance [6, 10–13] in order to: (i) identify the views and perspectives of clinicians and patients/relatives about treatment decision-making about the available options (for example, in-depth interviews or focus groups); (ii) ascertain the complexities of the target clinical setting which may shape decision-making about the available options (for example ethnographic methods); and (iii) understand how tools work in practice (usability testing of alpha prototypes outwith the target clinical setting, and subsequent feasibility testing in the actual clinical setting), and therefore how they may be improved/adapted to the specific context of the clinical setting.
Patient decision aids utilising a structured development process have been described previously; for example to support decision-making about stroke prevention in atrial fibrillation ; management of chest pain in the emergency department ; and surgical options for breast cancer .
Development of tools should utilise mixed methods and appropriate strategies to meaningfully involve clinicians, patients and their relatives in an iterative design process to establish the optimal mode, form and content of tools in the specific context of the clinical setting. Evidence-based methods should also be utilised to augment interpretability of probabilistic information. In particular, presenting a balanced synopsis of the benefits/risks of treatment with and without the available options (and absolute difference in terms of likely benefit or harm) using natural frequencies [61, 74, 75] with identical denominators and time horizons that relate to a specific reference class . For example, out of 100 patients with acute ischaemic stroke who receive thrombolytic treatment, it is likely that 60, 30 and 10 will be functionally independent, dependent and dead respectively after three months [and there is a risk of SICH in 2 out of every 100 patients treated with thrombolysis]; whereas out of 100 patients not treated with thrombolysis it is likely that 42, 48 and 10 will be functionally independent, dependent and dead respectively after three months – the absolute benefit of treatment with thrombolysis is 18 more patients out of 100 will be independent three months after a stroke).
Graphical displays provide a further mechanism for supporting accurate interpretation of probabilistic information , and there is evidence that graphical options such as pictograms/graphs are acceptable to patients irrespective of differences in health literacy . However, the optimal form of graphical display is contentious  and we recommend that researchers elicit clinicians' and patients' views and preferences on a range of characteristics of graphical risk presentations during the development process (e.g., format such as bar graphs [clustered or stacked] and pictograms/graphs to convey natural frequencies [with and without specific treatments, including absolute differences], use of colour [being mindful of red-green colour blindness], and order that information on outcomes is presented). Nonetheless, no single method of presenting probabilistic information is likely to be consistent with preferences of all potential users; therefore, multiple methods of presenting probabilistic information is recommended [11–13].
We further advocate that researchers should investigate the potential value of tailoring outcome probabilities to individual patients  for enhancing clinical decision making based on individual differential effectiveness in order to aid patients/relatives to understand and consider trade-offs between the benefits, risks and consequences associated with available options and hence make a decision about treatment that is consistent with their preferences and values . However, a prerequisite for tailoring outcomes to individual patients is the availability of robust predictive models that adequately reflect outcomes in routine practice settings. In the absence of suitable predictive models, extensive statistical modelling work is warranted using techniques such as decision analytic modelling .
Tools to support patient/carer understanding and decision-making in emergency care contexts are likely to differ from those designed for use in contexts where there is time for protracted discussions, and where patients/carers can access support and resources outside the consultation. In emergency care settings (e.g., treatment decision making about thrombolysis in acute stroke care) usability testing should be undertaken to establish the optimal mode, form and information content of tools for a specific emergency context to enable rapid use by clinicians and patients/carers. Such tools could be used in parallel to other procedures across an emergency care pathway so as to negate/minimise any negative impact on time for decision making and receipt of treatment. In emergency settings, structured decision aids have been shown to be feasible for eliciting patients’ preferences and values about management and treatment options .
The cultural sensitivity of tools to support patient understanding and decision making is a further area in need of research. Simply translating tools into multiple languages is unlikely to address the influence of cultural variation on patient preferences and values, or the applicability of tools to support involvement of patients in decision making with clinicians across cultural groups .
The critical appraisal of tools to support decision making or patient understanding about thrombolytic treatment highlights several important implications for implementation of analogous tools in other clinical contexts. Tools with limitations across the multiple domains described in this review are unlikely to be sensitive to the needs of target users and the complexity of the clinical context. Tools with low scores on the IPDASi probabilities items checklist will limit their ability to support patients/carers to make informed values-based decisions. For example, tools that present unbalanced synopses of probabilistic information on outcome states, without appropriate methods of supporting health literacy, may cause clinicians and patients to over- or under-estimate benefits or adverse effects associated with the available options; for example as a result of using percentages to convey single event probabilities  (e.g., 60% chance of being independent after a treatment with thrombolysis for acute stroke), relative risk reduction , and framing effects .
Consequently, tools with significant limitations will diminish their ability to support patient choice, facilitate self-management of illness, improve patient adherence to treatment, and enhance communication processes between clinicians and patients. In the case of decision aids, such limitations will restrict opportunities for patients to clarify their personal preferences and values with regards to the available options - prerequisites for informed values-based decisions .
Additional issues that are important for effective implementation of quality tools include training of clinicians to support patient involvement in healthcare and skills development in risk communication, including strategies to support dissemination and social marketing of quality tools [10, 81, 82]. Relevance and quality of information content of tools may diminish rapidly over time (due to availability of new data on effectiveness of the options for a specific health condition, including changes to clinical practice and information systems that can support delivery of decision support/risk communication), with a concomitant negative impact on uptake rates. Resources and appropriate systems should therefore be in place to periodically assess the relevance of tools and where necessary, update the mode of delivery, form and information content .
Undertaking a definitive randomised controlled trial (following a positive exploratory trial) as part of a structured development process has important benefits ; however, upon trial completion the relevance and quality of information content for clinical and patient/relative benefit could have diminished. This underpinned our decision to omit an item ‘was the tool subjected to a definitive randomised controlled trial?’ from the development process checklist, and is consistent with the proposal that tools of sufficient quality may be more appropriately tested and further refined within the context of 'real time' service evaluations . There are trade-offs to consider between the pros and cons of these options for implementation of tools following a positive exploratory trial in routine practice settings. A key issue in need of further debate is whether an equivalent of level of evidence required to license or implement psychological, pharmacological or surgical interventions is warranted for implementation of tools of sufficient quality (underpinned by a structured development process and pilot tested in clinical settings, with no apparent risks to safety/adverse effects) to support patient understanding and decision making about these types of interventions.
New tools to support patient understanding and decision making about preference-sensitive healthcare options/decisions are warranted when none currently exist. Tool development is a time- and resource-intensive process, and in cases where numerous tools already exist for a health condition of interest, an optimal strategy would be to adapt currently available tools so they fulfil quality standards , informed by a critical evaluation across the multiple domains described in this review, with reference to published guidance [6, 10–13].