Physician clinical management strategies and reasoning: a cross-sectional survey using clinical vignettes of eight common medical admissions

Abstract

Background

Physicians often select clinical management strategies not strongly supported by evidence or guidelines. Our objective was to examine the likelihood of selecting, and rationale for pursuing, clinical management strategies with more or less guideline support among physicians using clinical vignettes of eight common medical admissions.

Methods

We conducted a cross-sectional survey using clinical vignettes of attending physicians and housestaff at one internal medicine program in New York City. Each clinical vignette included a brief clinical scenario and a varying number of clinical management strategies: diagnostic tests, consultations, and treatments, some of which had strong evidence or guideline support (Level 1 strategies) while others had limited evidence or guideline support (Level 3 strategies). Likelihood of selecting a given management strategy was assessed using Likert scales and multiple response options were used to indicate rationale(s) for selections.

Results

Our sample included 79 physicians; 68 (86%) were younger than 40 years of age, 34 (43%) were female. There were 31 attending physicians (39%) and 48 housestaff (61%) and 39 (49%) had or planned to have primarily primary care internal medicine clinical responsibilities. Overall, physicians were more likely to select Level 1 strategies “always” or “most of the time” when compared with Level 3 strategies (82% vs. 43%; p < 0.001), with wide variation across the eight medical admissions. There were no differences between attending and housestaff physician likelihood of selecting Level 3 strategies (47% vs. 45%, p = 0.36). Supportive evidence and local practice patterns were the two most common rationales behind selections; supportive evidence was cited as the most common rationale for selecting Level 1 when compared with Level 3 strategies (63% versus 30%; p < 0.001), whereas ruling out other severe conditions was cited most often for Level 3 strategies.

Conclusions

For eight common medical admissions, physicians selected more than 80% of management strategies with strong evidence or guideline support, but also selected more than 40% of strategies for which there was limited evidence or guideline support. The promotion of evidence-based care, including the avoidance of care that is not strongly supported by evidence or guidelines, may require better evidence dissemination and educational outreach to physicians.

Background

The American health care system suffers from uneven quality [1] and wide variation in utilization [2]; both of which contribute to an impending crisis of increased health care spending [3]. Clinical practice guidelines are formulated in effort to improve the quality of care and to decrease inappropriate care and wasteful healthcare cost [4–6]. However, more than two decades following the introduction of the first guideline, many physicians and other clinicians fail to abide by these professional recommendations [7, 8]. Partly this may be a consequence of varying satisfaction with guidelines among physicians [9, 10], although explanations for poor physician adherence to guideline-recommended care has also included lack of awareness [11], difficulty changing entrenched practice habits [12], disagreement with guideline recommendations [13], system or environmental constraints [14], and shortcomings of using guidelines to inform the care of patients with multiple comorbidities [15, 16].

While there is a rich literature on general guideline attitudes and adherence, the differences in likelihood and reasoning for selecting between clinical management strategies that have different levels of guideline support has been sparingly investigated. Within most guidelines there is a gradation of recommendations from those with high quality evidence to those with little or no evidence of efficacy. Moreover, these selections may be different for management strategies that are focused on treatments, as opposed to imaging, laboratory testing, or consultations.

Understanding whether physicians use different justifications or explanations for selecting evidence-based, highly supported medical care versus poorly supported, or even contraindicated, care could have substantial implications for reducing the provision of wasteful and potentially dangerous medical care. To investigate differences in likelihood of selecting, and rationale for pursuing, clinical management strategies with more or less guideline support among physicians, we conducted a cross-sectional study using clinical vignettes of common medical admissions among attending and housestaff physicians at one academic medical center.

Methods

Design and setting

We developed a cross-sectional, self-administered survey for physicians and housestaff working within the Department of Internal Medicine at the Mount Sinai Hospital, a large tertiary care academic medical center in New York City, New York, USA. The Department has 400 full-time faculty, 140 categorical, preliminary, and research track housestaff, and more than 10,000 annual hospital discharges. At the time the study was conducted, all investigators (with the exception of JA) were either on the faculty or trainees at the institution.

Study subjects

Attending physicians in the Department of Internal Medicine who worked at least 4-weeks per year on the general medicine or geriatrics teaching services and housestaff in the Department of Internal Medicine at the post-graduate year two level and above were eligible. Participation was voluntary and participants received a $5 appreciation gift card. This study was approved by the Mount Sinai School of Medicine Institutional Review Board and all study subjects underwent informed consent.

Clinical vignette development and other survey information

We designed a survey to examine physician clinical decision making for eight common medical admissions. All chosen medical admissions were conditions for which published evidence-based guidelines were available that rated the strength of existing evidence supporting the clinical recommendations. The eight chosen conditions were congestive heart failure [17, 18], chronic obstructive pulmonary disease [19], transient ischemic attack [20–22], community acquired pneumonia [23], asthma [24], acute coronary syndrome [25], syncope [26, 27], and end-stage lung cancer [28, 29]. Clinical vignettes were developed that provided a straightforward presentation of the medical condition. The intent was for the diagnosis to be clear to both attending physicians and housestaff (see Additional file 1 for complete vignettes).

Each vignette was constructed through an iterative process by three of the authors (KS, MM, JSR), which included solicited feedback from a second general internist and an appropriate subspecialist as to whether the clinical scenario and decisions in the vignettes were ambiguous. We then piloted the survey on three housestaff and two attending physicians and feedback was incorporated into the final vignettes. Our eight vignettes each had two parts. Part one described the initial patient presentation to the hospital and a series of possible clinical management strategies from all four core areas of patient care, including treatments, imaging, laboratory tests, and consultations. Part two presented results of the diagnostic tests included in part one and additional clinical management strategies. All responses for part one were collected before participants received the second part of the vignettes, so part one responses could not be changed after receiving additional information during part two.

The likelihood of selecting a clinical management strategy was asked using a 5-point ordinal scale (always/almost always, most of the time, some of the time, rarely, never). For each selection, respondents were asked to indicate one or more rationales, including: local practice patterns, supervisor expectations, supporting data or guidelines (evidence), malpractice concerns, academic purposes (i.e., as part of a learning exercise), or to rule out more serious conditions. These rationales were derived from several group sessions with attending and housestaff physicians where clinical management strategies were discussed. Each vignette listed clinical strategies for which there was a range of evidence. We included management strategies that were either recommended or not recommended by the guidelines, based on clinical evidence or expert opinion. Recommended strategies included both class I and II recommendations (including evidence levels A and B and C and D, respectively) and are henceforth referred to as Level 1 strategies; non-recommended strategies include class III recommendations (including evidence levels A, B, C and D) and are henceforth referred to as Level 3 strategies. In particular, class I and II guidelines recommend management strategies for which the benefits are greater than or equal to the risk, whereas class III guidelines suggest that the risk outweighs the benefits. In all, the 8 clinical vignettes included 64 Level 1 strategies and 78 Level 3 strategies.

We also collected information about demographic and professional characteristics of respondents such as: age, gender, intended or current subspecialty, and intended or current clinical activities. Respondents were also asked to rate factors that influence their practice patterns using a 5-point ordinal scale and sources of information that inform their clinical practice on a 7-point ordinal scale. Housestaff and attending physicians completed the surveys during regularly scheduled noon-conferences and administrative time respectively.

Statistical analysis

Responses to the likelihood of making a Level 1 or Level 3 clinical strategies were collapsed into two groups; “Always” and “Most of the time” versus “Some of the time”, “Rarely” and “Never.” The proportion of respondents selecting Level 1 and Level 3 strategies ‘always’ and ‘most of the time’ was calculated for each clinical scenario. Physicians’ decision selection within each of the four core areas of care; laboratory testing, imaging, treatment and consult decisions, and the rationales behind selected clinical decisions were determined. We compared the proportions of respondents that selected “All” or “Most” of the time for Level 1 and Level 3 decisions by demographic and clinical practice characteristics using Chi Squared-test and Mann–Whitney U-test as appropriate. As the clinical scenarios varied in disease and severity of presentation, to inform future work, we also compared how often Level 1 and Level 3 management strategies were selected within each vignette and in comparison across clinical scenarios, using community acquired pneumonia as the reference scenario. Finally, the rationales for were compared for selecting Level 1 and Level 3 clinical management strategies. We used a type 1 error level of 0.01 to account for multiple comparisons. All analyses were completed using SPSS® version 18.0 (PASW Statistics, Chicago, IL).

Results

Of the 129 physicians eligible for the study, 79 (61%) completed the survey; made up of 48 (61%) housestaff and 31 (39%) attending physicians. Sixty eight (86%) physicians were younger than 40 years of age and 34 (43%) were female (Table 1). Gender distribution was similar among attending physicians and housestaff that responded to the survey. Sixty eight (94%) of the physicians identified their current or former training as categorical or primary care internal medicine and half (51%) of them had or intended to have primary care clinical responsibilities.

Table 1 Respondent physician characteristics (N = 79)

Almost all (94%) of the respondents indicated their preferred practice setting as metropolitan, with none of the respondents having any intentions of practicing in a rural setting. When asked to describe factors that influence their clinical practice, the most common reasons cited were accumulated research evidence (n = 76; 96%), hospital culture (n = 71; 90%) and individual practice experience (n = 56; 71%). Only a third cited malpractice concerns (n = 29; 37%). When asked about the main sources of information used to inform clinical practice, the most common sources cited were peer-reviewed journals (n = 64; 81%) and practice guidelines (n = 60; 76%).

Clinical management strategies

Overall, physicians were more likely to select Level 1 management strategies “always” or “most of the time” when compared with Level 3 management strategies (82% vs. 43%; p < 0.001) (Table 2). However, physicians were less likely to select Level 1 strategies for vignettes focused on care for end-stage lung cancer, transient ischemic attack, and chronic obstructive pulmonary disease when compared with the vignette for community acquired pneumonia (Table 2). Similarly, physicians were more likely to select Level 3 strategies for vignettes focused on care for acute coronary syndrome, asthma, end-stage lung cancer, transient ischemic attack, and chronic obstructive pulmonary disease when compared with the vignette for community acquired pneumonia (Table 2). Housestaff and attending physicians were equally likely to choose Level 1 strategies (83% vs. 81%, p = 0.22) and Level 3 strategies (47% vs. 45%, p = 0.36). There was no difference in selection of Level 1 or Level 3 strategies by physicians’ gender, age, training, or intended clinical responsibilities (p values > 0.10; data not shown).

Table 2 Proportion of respondents selecting Level 1 and Level 3 clinical management strategies ‘always’ or ‘most of the time’, overall and stratified by each of the 8 clinical vignettes (N = 79)

Rationale for selection clinical management strategies

The two most common rationales for clinical management strategies within the four core areas of care were supportive evidence and local practice patterns (Figure 1). Evidence was cited as the most common rationale for selecting Level 1 strategies when compared with Level 3 strategies (63% versus 30%; p < 0.001). Although ruling out other severe conditions was cited more often for Level 3 strategies, there was no significant difference in the selection of either Level 1 or 3 clinical strategies for the other queried rationales (Table 3). Rationales cited by physicians for their clinical management strategies were similar by physicians’ gender and training (data not shown). Physicians who had or planned to have clinical practices that included primary care were less likely to cite supervisor expectations as the rationale for selecting either Level 1 strategies (9% vs. 27%; p < 0.001) or Level 3 strategies (7% vs. 23%; p < 0.001) (Table 4), as were physicians whose clinical practice was self-reported to be influenced by malpractice concerns (1% vs. 21% and 1% vs. 17% for Level 1 and Level 3 decisions, respectively; p values ≤ 0.005).

Figure 1

Proportion of respondents citing selected rationale for selecting Level 1 and Level 3 clinical management strategies, stratified by core areas of care: treatments, imaging, laboratory tests, and consults (N = 79).

Table 3 Proportion of respondents citing selected rationale for Level 1 and Level 3 clinical management strategies (N = 79)

Table 4 Rationale for selecting Level 1 or Level 3 clinical management strategies, stratified by physician characteristics and self-reported factors that influence clinical practice (N = 79)

Discussion

In our cross sectional study of physician clinical management strategies for eight common medical admissions at a single hospital, physicians chose strategies with strong evidence or guideline support 82% of the time, but also selected strategies for which there was limited evidence more than 40% of the time, with wide variation across the eight medical conditions. In our study these rates did not vary between attending and housestaff physicians. Although these attending and housestaff physicians trained at a variety of institutions, their similar self-reported practice styles may speak to the impact of local practice culture on clinical reasoning, particularly when selecting clinical strategies without evidence or that are unsupported by expert guidelines.

When asked to indicate a reason for selecting evidence-supported clinical management strategies, almost 75% of physicians cited that evidence supported their selection. On the other hand, when asked to indicate a reason for selecting non-evidence-supported clinical management strategies, supportive evidence was cited less frequently and local practice culture was indicated as a rationale more often.

These discretionary clinical management selections may have substantial cost, including potential for patient harm, and likely do not provide patients much benefit. One possibility is that physicians either misunderstand the evidence or are simply following the lead of the community within which they practice. Better dissemination of evidence and educational outreach to physicians, as well as attention to local practice culture, will be essential to the promotion of evidence-based care, including the avoidance of care that is not strongly supported by evidence or guidelines. As medicine grapples with the imperative of reigning in excess utilization of care and the associated costs, the Choosing Wisely campaign has helped medical professional societies identify commonly used medical tests and procedures that are not supported by clinical evidence and may be unnecessary, and in some instances can cause harm. The goal of the campaign is to encourage physicians, patients and other healthcare stakeholders to think and talk about how to make wise decisions about the most appropriate care based on a patients’ individual situation [30]. As this campaign continues to grow, our study demonstrates the challenge of influencing clinical decision making, as selection of non-evidence-based, Level 3 strategies was common in our clinical vignettes and is often justified for reasons beyond supportive evidence, such as ruling out other severe conditions.

Interestingly, neither housestaff nor attending physicians felt that their decisions were driven by medical malpractice concerns. Even when physicians chose an unsupported option, less than 5% of the time did they indicate that their thinking was driven by legal considerations. These data suggest that while medical malpractice has become a popular target for new cost containment policy, physician behavior may be driven more strongly by other factors. If this is true, the impact of tort reform on utilization may be more modest than anticipated.

While this study had a high response rate and gathered data across a wide-selection of common conditions, there are important limitations to consider. Our study employed a survey based on hypothetical clinical patients as a means to investigate clinical reasoning. While the cases were designed to be unambiguous, straight-forward, and based on available guidelines, respondents may not have found them to be a realistic proxy for clinical practice or may simply have exhibited a Hawthorne effect and would act differently in real life. In this case, our study likely represents conservative estimates of practice and our finding that more than 80% of Level 1 strategies were selected, and more than 40% of Level 3 strategies were selected, is a best case scenario. Second, the survey was not a validated tool to measure clinical management.

However, the survey was developed by several clinicians with expertise in clinical management and decision making and was piloted to test ease of interpretation and feasibility. Third, many patients admitted to a hospital are often more complex than the presented clinical vignettes, with multiple possible diagnoses, making identifying the clearly correct strategy and subsequent treatment plan difficult. Future work could attempt to identify actual strategies selected for patients with these common medical conditions and then survey the reason for provider selections. Finally, because guidelines are not uniform, the highest ranked clinical strategies for one clinical condition may not be as strongly supported as the highest ranked clinical strategies for another. This may explain the wide variation in decision making across clinical conditions that we observed. As guidelines become more uniform and become less embedded in specialty societies, the different categories of evidence will hopefully be more easily compared across guidelines.

Conclusions

For eight common medical admissions, physicians selected more than 80% of clinical management strategies for which there was strong evidence or guideline support. However, they also selected more than 40% of management options for which there was limited evidence or guideline support. As evidence was the most common reason for selecting guideline supported strategies, the promotion of evidence-based care, including the avoidance of care that is not strongly supported by evidence or guidelines, may require better evidence dissemination and educational outreach to physicians.