Comparison of audio vs. audio + video for the rating of shared decision making in oncology using the observer OPTION⁵ instrument: an exploratory analysis

Shared decision making (SDM) is an approach to clinical deliberation in which patients and clinicians think, talk, and feel their way through a troubling and uncertain situation. Together, they settle on a course of action that is sensitive to and respectful of both the research evidence and the patient’s values and context [1]. SDM has been called the “pinnacle of patient centered care” [2] and has begun to be incorporated into clinical practice guidelines [3, 4].

As SDM is incorporated into clinical practice guidelines the need to understand and measure whether and to what extent it is occurring becomes essential. Several different approaches to measuring SDM have been proposed [5–8]. These approaches differ based on which viewpoint is being assessed. Some measures, such as CollaboRATE [9, 10], the decisional conflict scale [11], and the 9-item SDM questionnaire (SDM-Q-9) are self-reported from the patient’s perspective [12], while others are self-reported from the clinician perspective such as the SDM-Q-Doc [13]. Another approach is to have a third party not involved in the encounter assess the occurrence of SDM. One such measure is the 5-item Observer OPTION⁵ instrument [14, 15]. The five items are designed to capture plausible mediating behaviors associated with SDM: drawing attention to the existence of options, supporting the patient through the process of information sharing and deliberation, sharing information, eliciting preferences, and integrating preferences [15]. As it is a third party measure, the data to which this measure is applied (i.e. the clinical encounter) is often captured either through audio or video recording [16]. Visual data may affect ratings of communication by showing non-verbal behavior that may affect observer ratings of clinician communication behavior [17]. Existing literature, however, is mixed on the effects of different modes of data collection on the subsequent analysis of data [18–23]. Understanding the effect of different modes of data collection on subsequent ratings of SDM is important because it may explain both within and across study differences in ratings of SDM. It may also aid in identifying best methodological practices for researchers.

This exploratory pilot study aimed to determine the effects of data mode, examining audio+video vs. audio only, on OPTION⁵ scores.

Forty-one unique decisions were identified within the 15 encounters. Ten of the encounters included more than one decision with a maximum of six decisions occurring in two encounters (Additional file 2: Table S1). These ranged from specific decisions such as choosing between radiation or chemotherapy, to less specific discussions such as “what to do next” ((Additional file 3: Table S2).

The overall mean OPTION⁵ score for the audio only recordings was 21.8 (95% CI 17.2, 26.4) out of 100. For the audio+video recordings, the mean score was lower (17.5, 95% CI 13.5, 21.6). Thus, the average adjusted OPTION⁵ score for audio only recordings was 4.3 points higher than audio+video with a 95% CI of 0.36, 8.21. This overall difference in mean OPTION⁵ score was reflected in the mean scores for four of the five constituent items of the OPTION⁵ instrument (Table 2). For all items, except for item 2 (i.e. clinician supports the patient through the decision making process), audio only was associated with higher scores to varying degrees (i.e. on average, audio is 0.04 points higher for item 4 – eliciting preferences, to 0.31 points higher for item 3 – sharing information). Visual inspection of a plot of total OPTION⁵ scores for the audio+video mode (y-axis) versus scores for the audio only mode (x-axis) (Fig. 2) suggests that at the higher ranges of OPTION⁵ scoring, there appears to be greater heterogeneity between audio+video and audio only scores, than on the lower end of OPTION⁵ scoring. The concordance between reviewers within mode (i.e. audio+video or audio) was approximately 60% (audio: 58.3 95% CI (37.7, 78.9); audio+video: 60 95% CI (40.4, 79.6)).

	Mean (95% CI)	Audio+Video: Mean (95% CI)	Mean Difference (95% CI)	P-Value
OPTION5 Total: (0–100 scale)	21.8 (17.2, 26.4)	17.5 (13.5, 21.6)	4.28 (0.36, 8.21)	0.032
OPTION5 Question 1: (0–4 scale) Draw attention to options	1.22 (0.96, 1.49)	0.95 (0.68, 1.22)	0.27 (0.03, 0.52)	0.028
OPTION5 Question 2: Support the patient	0.35 (0.16, 0.55)	0.38 (0.19, 0.57)	−0.03 (− 0.17, 0.12)	0.74
OPTION5 Question 3: Share information	1.22 (0.94, 1.5)	0.91 (0.63, 1.18)	0.31 (0.08, 0.55)	0.009
OPTION5 Question 4: Elicit preferences	0.70 (0.46, 0.95)	0.66 (0.42, 0.91)	0.04 (−0.22, 0.30)	0.76
OPTION5 Question 5: Integrate preferences	0.83 (0.61, 1.06)	0.66 (0.43, 0.89)	0.17 (−0.09, 0.44)	0.20

Predictive margins calculated and adjusted by the fixed effect of reviewer and the random effect of clinician and encounter

This small methodological experimental sub-study embedded in a larger observational study demonstrates that mode of recorded data, audio+video vs. audio only, influences ratings of a prominent and commonly used third-party measure of SDM, the OPTION⁵ instrument.

This finding is similar to those reported for other measures of communication in healthcare [18–22]. Riddle et al. [22] found that when clinical encounters were coded using the Moffitt Accrual Analysis System (MAAS) and the Roter Interaction Analysis System (RIAS) there were significant differences (p < 0.028 for MAAS and p < 0.01 for RIAS), between audio and video coding. In contrast to this study however, other studies have failed to show a difference between audio only and audio+video modes when rating aspects of health communication. Nicolai et al. [19] measured empathic communication during clinical encounters using the Rating Scales for the Assessment of Empathic Communication in Medical Interviews; they found no difference between audio and video formats for rating empathy but both modes resulted in higher empathy scores over transcripts. Williams et al. examined video vs. audio only clips of nursing care and rated them using the Emotional Tone Rating Scale and found that the modes were highly correlated and only one item (patronizing) after adjustment for multiple comparisons, was significantly different between modes (audio scored higher). Weingarten et al. [20]. assessed clinical encounters for patient-centeredness using the Henbest and Stewart scale of patient-centeredness and found no difference (both mean scores 1.94 out of 3) between audio and video. Dent et al. [18] assessed communication in simulated cancer consultations using the Cancode interaction analysis system and found substantial agreement [28] between audio and video as indicated by kappas of 0.72 and 0.77 for the function and content dimensions of Cancode, respectively [18]. The discrepancies in the literature on the effect of mode could reflect the different constructs being measured, with some more affected by the additional information contained in audio+video (e.g. non-verbal behaviors). In a review of the impact of clinicians’ personality and interpersonal behaviors on the quality of care, Boerebach et al. found mixed results for the impact of non-verbal behaviors on patient reported ratings of quality of care [17]. It is therefore plausible that clinicians’ non-verbal behavior affected third party’s ratings and for example, may explain the non-statistically significantly higher score for item 2. However, the difference we found was driven by items which coded for sharing information and drawing attention to the existence of options; items which arguably are more objective as compared to supporting the patient through the process of information sharing and deliberation. These findings, if confirmed in larger samples, could more definitively establish the relationship between data collection mode and OPTION⁵ scoring. Ideally, confirmatory studies will be conducted using mixed methods including interviews with raters so that differences could be further explored and explained.

This study has several limitations. Our sample was drawn from cancer consultations with no uniform focus of the discussion. This is in contrast to the typical use of the OPTION⁵ instrument, which is usually used to score a specific, focused decision (e.g. decisions about medication treatments for diabetes or management strategies for heavy menstrual bleeding). As a result, many discussions with low scores could have created an artificial floor effect, thus limiting our ability to detect a difference. We detected a difference and therefore, the effect of an artificial floor was limited. Our analytic choice to pool all decisions may have differentially affected our mode comparison (i.e. audio+video vs. audio only), thus potentially creating a difference between modes which may be instead due to aspects of the decisions or discussions.

Whether the effects observed in our exploratory study constitute a bias is an important question. Arguably, what matters is how the actors in the encounter experienced it. Third-party measures seek to ascertain second-hand how that process went. It offers an important degree of detached objectivity helpful for documenting verifiable behaviors that may not be easily recalled by participants. Video offers the advantage of non-verbal inputs, yet the meaning of those inputs, largely constituted by gesture, posture, and eye contact and other non-verbal behaviors are as influenced by experience and culture as much or more as the spoken word. Thus, we cannot say whether these differences are good or bad. However, at a minimum, they clearly have implications for aggregating outcome data across trials where a variety of SDM measures and observer-administered ratings are used. They also may influence effect sizes estimation for design of future trials.

While our results merit further investigation on methodologic grounds, investigators studying SDM and patient-clinician communication need to consider the pragmatic implications of choosing audio or audio+video to record encounters. For example, some patients may be more comfortable with audio only recording as it is less intrusive and offers a greater degree of privacy. Yet, this choice limits the richness of the available data for the investigator as audio+video provides additional data such as non-verbal communication which may provide important contextual and non-verbal relational information not available with audio only recordings. Investigators planning to record encounters need to consider these trade-offs and the extent they matter for a given set of study objectives.

Shared decision making is an important component of patient centered care [2, 29]. Research, quality assurance, and quality improvement should be supported by accurate and reliable measures. Different ways of measuring SDM are available including third party measures such as the OPTION⁵ instrument. Due to patient preference, available resources, or clinician comfort level discussions may be recorded as video or audio only. This study suggests that, for the OPTION⁵ instrument, mode of recording affects the rating of SDM in the clinical encounter. Additional research is needed to confirm and further understand the effects of mode of recording on ratings of SDM across of variety of conditions and decisions.