Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Low agreement for assessing the risk of postoperative deep venous thrombosis when deciding prophylaxis strategies: a study using clinical vignettes

  • Martin O'Flaherty1Email author,
  • Kaja Lerum2,
  • Paula Martin2 and
  • Daniel Grassi1
BMC Health Services ResearchBMC series ¿ open, inclusive and trusted20022:16

DOI: 10.1186/1472-6963-2-16

Received: 19 April 2002

Accepted: 16 August 2002

Published: 16 August 2002

Abstract

Background

Several clinical practice guidelines (CPG) on antithrombotic prophylaxis in surgical patients help to decide about the prophylaxis strategy based on the patient risk of deep venous thrombosis (DVT). However, the physician risk estimates of DVT could have little inter-observer reproducibility, which could lead to different individual prophylaxis practices.

Methods

Physicians were asked to evaluate DVT risk in eight clinical vignettes, describing actual patients cared for in our hospital. The vignettes included all possible levels of DVT risk.

Results

The degree of prophylaxis strategies accuracy was 63% (95% CI 523–75%). Overall agreement was 0.32 (z = 7.61, p < 0.001) and for each level of risk kappa was 0.38 (z = 6.50, p < 0.001); 0.1 (z = 1.65, p < 0.049) and 0.5 (z = 8.45, p < 0.001) for small, moderate and high risk group respectively

Conclusions

Our results showed that there is poor agreement when physicians have to evaluate the risk for postoperative DVT, and in the cases of low and moderate risks of DVT there is the smallest agreement. In addition, the data also showed that the overall accuracy of DVT prophylaxis strategy was only moderate and the risk evaluation did not correlate to the selection of the strategy. The issue of inter-observers variability should be taken into account when CPG performance are analysed, especially when considering the risk-evaluation to choose the appropriate actions.

Background

Deep venous thrombosis (DVT) prophylaxis is an effective intervention in the postoperative period [1]. Several clinical practice guidelines (CPG) on the topic have been published, with the aim to increase the utilisation and appropriateness of this practice. Usually, these guidelines established the choice of the prophylaxis strategies based on the type of surgery, as well as on the presence of specific patient's risk factors [1, 2]

To establish an individual patient risk level is a complex task that could lead to substantial variations between observers. The integration of several cognitive inputs and patient data under real work pressures is a task where the physicians' performance is sub-optimal [3].

In order to determine if there are variations in the postoperative DVT risk perception among physicians, we performed a study using clinical vignettes.

Methods

Study design

Clinical practice guideline

Our guideline on DVT prophylaxis is mostly based in the 5th ACCP consensus conference recommendation [1], where risk-assessment approaches are used to decide the prophylaxis strategy.

In our guideline, the postoperative DVT risk is evaluated basing on the items that follow: the age, the presence of venous thrombosis risk factors and the duration of surgery. With this information we could classify the patients in low, moderate and high-risk groups. Early mobilisation or graded compression stockings are prescribed in the case of low risk group patients. Moderate risk patients are prescribed fixed subcutaneous calcium heparin and graded compression stockings could replace heparin if high haemorrhagic risk exists. High-risk patients received nadroparin but when it was deemed inappropriate due to a high haemorrhagic risk, intermittent pneumatic calf compression (IPCC) was indicated. Both nadroparin and IPCC may be used in selected high-risk patients. The guideline has been widely publicized throughout the institution and it is available in the Hospital Intranet.

Clinical Vignettes

We have written eight comprehensive clinical vignettes including patients that cover the entire risk spectrum in order to evaluate the inter-observer variability about the risk assessment,. The information was mostly taken from actual patients cared for at our hospital. Specific details were changed in order to respect patient confidentiality and to avoid clueing physicians about the appropriate decisions. The vignettes are summarised in the table. No consent is required in our hospital to use clinical record data if no breach to patient confidentiality is possible. One of us (DG) independently reviewed the clinical vignettes for accuracy and validation regarding the study objectives.
Table 1

Clinical Vignettes summary with risk level and proposed prophylaxis strategyin the CPG.

Clinical Vignettes Summary

Post-operative DVT risk

Prophylaxis

Comment

Case 1: 31- years female post rhinoplasty, no past medical history, uneventful operation.

Small

ED/GCS

 

Case 2: 25 years male insulin dependent diabetes after a correction of hallux valgus. Developed postoperative hyperglycemia.

Small

EM/GCS*

 

Case 3: 63- years old female post hysterectomy with non-insulin dependent diabetes. After returning from the operating room a blood pressure of 225/130 was noted responding to analgesia.

High

Nadroparin

 

Case 4: 42- years old obese male with chronic stable angina history. Resection of melanoma in the malar region.

Moderate

Calcium Heparin

 

Case 5: 50- years old male with a 2-month history of macroscopic hematuria and pain in the right lumbar fossa. CT scan showed a right renal mass. He was neprhectomised

High

Nadroparin

IPCC** not indicated since hemorrhage source was controlled

Case 6: 88- years old female with a history of duodenal ulcer in her youth admitted due to hip fracture and operated 24 hs later. She was stable in the immediate postoperative period.

High

Nadroparin

 

Case 7: 38- year old male with an upper gastrointestinal haemorrhage gastrectomised after failure of medical therapy. An ulcerated leiomioma was diagnosed.

Moderate

Calcium heparin

IPCC could be acceptable

Case 8: 46 years-old obese male with hip septic arthritis and previous pulmonary embolism diagnosis. A resection of the femoral head was performed.

High

Nadroparin +IPCC

 

*ED/GCS: Early mobilization or graded compression stockings. **IPCC: Intermittent pneumatic calf compression.

We randomly chose 10 physicians from the Internal Medicine and the Intensive Care Unit to read the vignettes. Then we asked them: 1) to estimate the inhospital complications risks (nosocomial infection, cardio-respiratory events, DVT risk and hospital mortality) and to classify them in three levels: low, moderate and high risk; and 2) to order a complete diagnostic and therapeutic plan for each vignette.

Statistical analysis

Percentages, along with 95% confidence intervals, were calculated for the accuracy of the strategy selection compared to the intended level of risk for the vignette, as decided by the investigators.

The inter-observer agreement was evaluated using the kappa statistic for multiple raters and for multiple categories. Statistical calculations were performed with STATA 6.0.

Results

The degree of appropriateness of prophylaxis strategies used were 63% (95% CI 52–75%).

There was no correlation found between the accuracy of the prophylaxis strategies and the DVT risk evaluated by the physician (χ2 for trend 0.54, p = 0.46). Although no statistically significant, the low risk strata showed a higher rate of strategy inappropriate selection compared to the moderate and high risk ones (72%, 61% and 61.5 respectively, x2 = 0.7, p = 0.7). Overall agreement was 0.32 (z = 7.61, p < 0.001) and kappa by risk category was 0.38(z = 6.50, p < 0.001); 0.1(z = 1.65, p < 0.049) and 0.5 (z = 8.45, p < 0.001) for the low, moderate and high risk group respectively.

All the DVT risk wrong estimates in the low risk group were rated as "moderate" and in the moderate risk group as "low risk". In the high-risk group, 70% of the wrong estimates were "low risk" and the rest were classified as "moderate risk".

Discussion

Our results showed a poor agreement for assessing the risk of postoperative DVT, especially in the low and moderate risk groups. Wrongly classified patients who belong to moderate risk groups were classified in lower risk category. In addition, the data also showed that overall DVT prophylaxis strategy was not good enough and the risk evaluation did not correlate to the strategy selection.

As stated by Weed et al [3], our example on DVT risk assessment showed that integration of cognitive inputs and patient data must be systematically explored for different clinical settings since considerable variation may occur, especially when the physician behaviour is assessed.

The evaluation of the physician performance is a difficult task, as several factors from physicians themselves, healthcare structure and case complexity could modify the evaluation. Clinical vignettes are a validated measurement of the physician performance [4], as compared both to utilisation review and simulated patient-doctor encounters. This method permits controlling case-mix, an important confusing variable when the physician performance is evaluated.

A standardised way of risk evaluation could improve the quality of a key point in the decision process of the CPG, such as risk assessment. Computerized decision support systems have shown benefits in increasing usage and accuracy of DVT prophylaxis [5].

Conclusions

The issue of inter-observers variability should be taken into account to analyse variations in CPG performance, especially when considering risk-evaluation to select the appropriate actions.

Declarations

Acknowledgements

We thank Dr. M. Pellizzari, Dr G. Amestoy and Dr. R. Martin for their helpful revision of the manuscript. We also want to thank our colleagues who kindly accepted to participate in this study and Dr P Casais for her work with the clinical practice guideline.

Authors’ Affiliations

(1)
Departamento de Medicina, Hospital Universitario Austral
(2)
Clinical Internship, Facultad de Ciencias Biomédicas, Universidad Austral

References

  1. Dalen JE, Hirsh J, et al: Fifth ACCP Consensus Conference on Antithrombotic Therapy. Chest. 1998, 114 (suppl): 439s-769s.View Article
  2. Ravaud P, Durieux P, Fourcade A, and the Comité Scientifique Thrombose: Prophylaxie des thromboses veineuses post-operatories: recommendations de la Ássitance Publique-Hopitaux de Paris. Sang Thromb Vaisseaux. 1995, 7: 119-129.
  3. Weed LL, Weed L: Opening the black box of clinical judgement-an overview. BMJ. 1999, 319: 1279.PubMed CentralView ArticlePubMed
  4. Peabody JW, Luck J, Glassman P, Dresselhaus TR, et al: Comparison of vignettes, standardised patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000, 283: 1715-1722. 10.1001/jama.283.13.1715.View ArticlePubMed
  5. Durieux P, Rémy N, Philippe R, et al: A Clinical Decision Support System for Prevention of Venous Thromboembolism: Effect on Physician Behaviour. JAMA. 2000, 283: 2816-2821. 10.1001/jama.283.21.2816.View ArticlePubMed
  6. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/2/16/prepub

Copyright

© O'Flaherty et al; licensee BioMed Central Ltd. 2002

This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.

Advertisement