Personalized versus non-personalized computerized decision support system to increase therapeutic quality control of oral anticoagulant therapy: an alternating time series analysis
BMC Health Services Research volume 4, Article number: 27 (2004)
The quality control of oral anticoagulant therapy (OAT) during the initiation and maintenance treatment is generally poor. Physicians' ordering of OAT (especially fluindione and warfarin) can be improved by dose adjustment algorithms, taking into account the results of International Normalized Ratio (INR). Reminders at the point of care, computerized or not, have been demonstrated to be effective in changing physicians prescription behavior.
However, few studies have addressed the benefit of personalized reminders versus non personalized reminders, whereas the personalized reminders require more development to access patient record data and integrate with the computerized physician order entry system.
The Hospital Information System of George Pompidou European Hospital integrates an electronic medical record, lab test and drugs order entry system. This system allows to evaluate such reminders and to consider their implementation for routine use as well as the continuous evaluation of their impact on medical practice quality indicators.
The objective of this study is to evaluate the impact of two types of reminders on overtreatment by oral anticoagulant: a simple reminder of text formatted dose adjustment table and a personalized recommendation for oral anticoagulant dose and next date of INR control, adapted to patient data. Both types of reminders appear to the physician at the moment of drug ordering.
The study is an alternating time series experiment with three 6 months periods, each one including every 2 months according to a Latin square scheme: a control period without any reminder, a period with the simple non personalized reminder, a period with personalized reminder. All patients hospitalized in departments using the computerized physician order entry system and ordered fluindione or warfarin, will be included in the study between November 2004 and May 2006.
Main outcome will be the proportion of overcoagulation, as expressed by the proportion of observation time with INR over 4.5, assuming INR change linearly. Secondary outcome is the incidence of major haemorrhagic events. Data will be collected thanks to Hospital Information Systems databases.
Data will be analyzed taking into account patient and physician clustering effect.
Iatrogenic effects of oral anticoagulants therapy
According to a study carried out by French pharmacovigilance centres, haemorrhage subsequent to oral anticoagulant treatment (OAT) is the most common drug-related side effect resulting in hospitalisation in public hospitals in France (13% of such admissions, and 0.41% of the 3,137 admissions analysed). On the basis of these findings, the AFSSAPS (French Agency for Health Product Safety) has made the prevention of iatrogenic effects related to OAT one of its priorities. Many of these events are consequences of interactions between different drugs, resulting in inappropriate doses [1–3].
Implementation of a system of support when prescriptions are made out is likely to improve prescription practices and to decrease the frequency of side effects. It should be possible to integrate a support tool into the drug prescription system, by using nomograms to adjust OAT doses.
Decision-making tools and appropriate practice
The efficiency of reminders issued at the time of prescription has been demonstrated by a various studies [4–8]. These reminders can come in a paper, telephone  or computer form. Hunt et al.  reviewed all studies (randomised and quasi-randomised) evaluating the effect of computer-based clinical decision support systems on medical practice and treatment outcome. They showed that these systems are effective for drug prescription purposes and for the implementation of various medical strategies . We have demonstrated the efficiency of reminders in the French situation, both in the form of "paper" reminders  and in the form of computer-generated reminders . Computer-generated reminders appear the most promising given the development of computer programs including prescription aids for hospital usage.
Several types of reminders can be issued at the time of prescription :
simple, general information concerning the recommendations that should be taken into account (non-specific or non-personalised reminders),
"check list": includes questions or a precise list of practices that the doctor must tick to show that it has been done,
reminders including clinical data concerning a specific patient that must be taken into account for a given procedure (personalised reminders).
The advantage of personalised reminders over non-personalised reminders has not been demonstrated in the literature. However, the production of personalised reminders necessitates better integration of existing information and is thus more expensive to develop. It is important to determine whether this personalised tool results in a better quality of care than non-personalised tools.
Several randomised clinical trials have tried to evaluate decision support systems for the prescription of OAT, but failed to draw any conclusions about their efficiency for several reasons: heterogeneity and complexity of the systems evaluated, experimental designs difficult to apply and not necessarily adapted, and too few patients included [14–17].
Reminders issued at the time of prescription have been shown to be effective by experimental studies, but the difficulties of maintaining the effectiveness of interventions designed to improve clinical practices remains a major problem. We evaluated the effect of an active decision support system for the prescription of low molecular weight heparin as prophylaxis for venal thrombosis in an orthopaedic surgery department. In this study, the system was and was not used during alternate periods. It showed that such programs affect practices without affecting learning . Other authors have looked at the problem of the routine use of computerised systems [18, 19]. The long-term use of a computer program to modify medical practices necessitates a hospital information system that ties in prescriptions and the results of medical tests, and the continuous collection of indicators making it possible to evaluate use and effect.
Georges Pompidou European Hospital (GPEH)
Organisation of the information system at GPEH
The hospital information system currently collates prescriptions and results of biological tests and imaging procedures. Eight hundred computers, both laptops and fixed posts, are used to in care procedures (in care departments and medical offices).
The Dx-Care® program is at the centre of care delivery. It is used by doctors and nurses:
to prescribe laboratory examinations and imaging tests for a patient,
to visualise the results of laboratory tests,
to establish and to consult nursing schedules,
to archive a structured observation,
to prescribe drugs.
Dx-Care® is integrated with other applications to allow circulation of information between departments, laboratories and the pharmacy. Prescriptions for laboratory tests are transmitted to the Netlab® program which manages such tests (this program is used by all biochemistry, haemostasis and immunology laboratories, etc.) The laboratories return the results using this same program, which retransmits them to Dx-Care®. Furthermore, prescriptions of drugs are transmitted to the Phedra program, which is used by the pharmacy to manage prescriptions. The prescription is validated by the pharmacy and this validation is then transferred to Dx-Care®. The lab test prescription facility has been available in the hospital information system since 2000 and is used by all departments of the hospital. The drug prescription facility has been implemented later on, since January 2003, and its use is still increasing (currently by half of the hospital departments).
The hospital information system thus allows to install decision support systems that are activated whenever a prescription is issued and routinely to collect evaluation criteria of prescription practices. If possible and validated, the use of the hospital information system to evaluate care procedures will make it possible to collect data regularly, and routinely to assess methods for the improvement of care practices.
Organisation of quality assurance and system for recording undesirable events
With the aim of quality of care and preventing risks, the hospital is developing a system, based on the Intranet network, of declaration of undesirable events. This system must be able to record all undesirable events and incidents linked to the use of health products (drugs, medical equipment, blood products, etc.) and care (e.g. falls, lost files, bedsores, long waiting times) as well as those due to the patient environment (building, security, malevolence, etc.) and the job of health care professionals (accidental exposure to blood, chemicals, radiation, etc.).
When a health care professional decides to report an undesirable event, he or she must complete a dedicated form available on the Intranet with all relevant information. This incident form includes an item entitled "complications associated with anti-coagulants". When the doctor clicks on this item, a form specific to haemorrhagic accidents following anti-coagulant treatment appears (see form in appendix).
Prevention of thrombosis at GPEH
Among departments which already started to use the computerised drug order entry system, several (cardiology and vascular medicine departments) are heavy "consumers" of anti-thrombosis drugs (see Table 1).
In 2001, a working group representative of all the hospital departments described a group of procedures. This was done by critically reading articles published in the literature. The procedures concerned the preventive and curative indications of anticoagulants (OAT and heparins) in arterial thrombosis and venous thrombosis, and the way in which patients receiving anticoagulants are monitored and handled in cases of overdose. The procedure concerning curative OAT included a nomogram for adjusting doses of fluindione and warfarin based on previous doses and the INR (see Tables 2 and 3). These nomograms are currently available on the hospital's intranet, although they are not widely used. Their inclusion in a computerised prescription aid could increase their impact.
To evaluate the effect on the frequency of overanticoagulation of the implementation in the computerised physician order entry system of two types of tool to adjust OAT doses (one personalised and one non-personalised).
To assess any advantages of using the personalised tool rather than the non-personalised tool.
To evaluate the frequency of haemorrhagic accidents in the context of the study.
To evaluate the feasibility of long-term implementation of the intervention.
The study is an alternate time series experiment which consists of three successive six-month periods (timed to change with the changing of medical residents even though they will not be the only prescribers). Each phase will consist of:
a two-month period without active support during which evaluation criteria will be collected (period A),
a two-month period with non-personalised active support (period B),
a two-month period with personalised active support (period C).
To limit the impact of a learning effect on appropriate OAT management practice within the department over time (possible for medical residents), the order of these three periods was determined by using a Latin square plan (see Figure). This experimental design can be considered valid for an impact study in this context whereas a randomised controlled design is difficult to apply with just one hospital .
This study will include all the patients who are prescribed OAT for any indication and are hospitalised in clinical care departments where physicians are using the hospital information system to prescribe drugs.
The following table shows the number of INR examinations prescribed by these departments in a three-month period. It provides an estimation of the approximate proportion of overdoses among the INR that exceeded 2, which is supposed to be found in patients treated with OAT in these units.
All doctors authorised to prescribe drugs in the participating departments will be included in the study: residents and fellows, registered and non registered university hospital doctors. Each six-month period will coincide with an internship semester.
To prescribe a drug using Dx-Care®, the doctor selects the required drug from an exhaustive list. This opens up a dialogue box in which the doctor types the dose, the frequency of intake and the mode of administration. From this window, it is possible to add a text comment or to consult particular protocols that have been defined by the departments.
It is planned to integrate two types of decision support systems into the computerised prescription program:
personalised active system: when the drug is selected a window automatically opens suggesting a dose recommended according to the nomogram (taking into account the doses previously received by the patient and the patient's INR), together with a date for next INR control and an explication.
Definition of endpoints
Proportion of patient observation time with INR results > 4.5, assuming linear change of INRs.
Major haemorrhagic accidents
Intra-cranial haemorrhage or spontaneous haemorrhage necessitating surgery or a transfusion or decreasing haemoglobin concentration by more than 2 g/dl.
Assessment of evaluation criteria
The Netlab® application allows biological laboratories to receive prescriptions and to return results. All of the INR results can be extracted from the Netlab® database accompanied by information making it possible to identify the patient, the treatment and dose received, the prescribing doctor, the hospitalisation unit, the date the test was prescribed. Data about overdoses can therefore be collected systematically by regular database searches.
Furthermore, the storage of the information in a computerised tool will make it possible to determine previous doses and INR results each time a drug is prescribed.
When a health care professional decides to declare an undesirable event, he or she fills in a specific, pre-formatted form available on the Intranet. This form includes a list of events that must be declared at the GPEH.
The declaration form includes an item entitled "complication of haemorrhagic accidents". When the doctor clicks on this item, a specific form for the declaration of a haemorrhagic accident associated with anti-coagulant treatment appears (see form in appendix).
Determination of sample size
The determination of the number of participants necessary requires the definition of the statistical unit of interest, information about the incidence of the evaluation criteria in the study population and a hypothesis about the efficiency of the intervention.
In this study, the main aims are to guide each prescription and to reduce the number of anti-coagulant overdoses: the simplest statistical unit to study is therefore the INR result. This unit will be used to calculate the sample size.
This choice is not, however, perfect and the efficacy results will be presented using other indicators of the quality control of anti-coagulant treatments:
Given the low incidence of major haemorrhagic accidents (not currently measured at the GPEH but probably below 1%), it is not possible in this study to estimate the number of subjects necessary to demonstrate an effect of intervention on the "haemorrhagic accident" endpoint. Recording haemorrhagic accidents will give the frequency of such accidents, which will then be used for realistic estimates of power and sample size if further studies are carried out.
In previous studies evaluating the efficacy of tools to aid the prescription of OAT, the unit considered was not always the same, taking into account the number of INR per patient and the time between INR measurements to greater or lesser extents. The most recent studies considered the number of patient-days according to the method described by Rosendaal [15, 16, 21]. This method can also be used to calculate the rate of haemorrhagic events as a function of the number of patient-days for a given range of INR values.
We may also carry out an analysis for each prescribing doctor given that the intervention targets doctors directly. This will involve adjusting the effect of the intervention to the fact that intra-physician variability is a priori lower than inter-physician variability.
Number of INR measurements and predicted frequency of overdoses
During a six-month period (January to June 2004), 4 920 INRs were requested by the six departments which already routinely use the computerized physician order entry system. The frequency of overtreatment can approximately be estimated from the percentage of INR > 4.5 among INR >2. Among the 2620 INR > 2, 330 (12%) were higher than 4.5 (see Table 4). This frequency has been stable during these six months but differed considerably between departments (10% to 23%).
Hypothesis about the efficacy of the intervention
The number of INR tests necessary for a six-month period, with an α risk of 5% and a power of 80%, for the comparison of two percentages by classical methods (untreated group half the size of the treated group), for a basal incidence of the judgement criterion of 12% are and for the following hypotheses on relative reduction of the risk (RRR) of overdose, are:
RRR 30%: 2500
RRR 40%: 1300
RRR 50%: 800
RRR 60%: 500
Carrying out approximately 5000 tests over six months will make it possible to detect an intervention effect of less than 30% in this period. The experimental design includes three six-month periods and should thus ensure adequate power.
Statistical analyses will be performed with the STATA statistical software (release 8, STATA corp, College station, Tex, USA)
Standard statistical tests will be used to compare the baseline characteristics of the departments and patients.
The main analysis concerns the effect of the intervention on the number of dangerously high INRs. The analysis will be carried out using a mixed effect analysis of variance model, in which the effect linked to the period will be considered fixed and that linked to the prescription tool will be considered random .
Rosendaal's method will be used to analyse the number of patient-days with INR over the target .
According to French policy, this study was exempt from medical ethics committee approval. The anti-coagulants being evaluated are prescribed as recommended by clinical studies validated within the GPEH. These recommendations are available on the hospital's Intranet and are thus accessible to all doctors. They conform to standard practices. Neither the patients nor the doctors will be randomised. The interventions are simply different means of giving valid information to physicians. Using funding from the PHRC (Hospital Clinical Research Program), we carried out two research studies related to this project. In the first (PHRC 95), an intervention aimed at modifying the way in which emergency department doctors handle ankle injuries, the study design was a randomised controlled study and the randomisation unit was the hospital . In the second (PHRC 98), a computer-based decision-support system for the prevention of venous thromboses in orthopaedics, the experimental design was identical to that of our present study . In both cases, the Ile de France branch of the Clinical Research Delegation considered that the project was exempt from medical ethics committee approval.
Oral Anticoagulant Therapy
International Normalized Ratio
Georges Pompidou European Hospital
Hospital Clinical Research Program
Azar AJ, Deckers JW, Rosendaal FR, van Bergen PF, van der Meer FJ, Jonker JJ, Briet E: Assessment of therapeutic quality control in a long-term anticoagulant trial in post-myocardial infarction patients. Thromb Haemost. 1994, 72 (3): 347-51.
Hylek EM, Heiman H, Skates SJ, Sheehan MA, Singer DE: Acetaminophen and other risk factors for excessive warfarin anticoagulation. JAMA. 1998, 279 (9): 657-62. 10.1001/jama.279.9.657.
Landefeld CS, Beyth RJ: Anticoagulant-related bleeding: clinical epidemiology, prediction, and prevention. Am J Med. 1993, 95 (3): 315-28. 10.1016/0002-9343(93)90285-W.
Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, Grilli R, Harvey E, Oxman A, O'Brien MA: Changing provider behavior: an overview of systematic reviews of interventions. Med Care . 2001, 39 (8 Suppl 2): II2-45.
Grimshaw JM, Russell IT: Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993, 342: 1317-22. 10.1016/0140-6736(93)92244-N.
Oxman AD, Thomson MA, Davis DA, Haynes RB: No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995, 153 (10): 1423-31.
NHS Centre for Reviews and Dissemination: Effective Health Care. Getting evidence into practice. 1999, The University of York
Balas EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD, Brown GD: The clinical value of computerized information services. A review of 98 randomized clinical trials. Arch Fam Med. 1996, 5 (5): 271-8. 10.1001/archfami.5.5.271.
Weingarten SR, Riedinger MS, Conner L, Lee TH, Hoffman I, Johnson B, Ellrodt AG: Practice guidelines and reminders to reduce duration of hospital stay for patients with chest pain. An interventional trial. Ann Intern Med. 1994, 120 (4): 257-63.
Hunt DL, Haynes RB, Hanna SE, Smith K: Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998, 280 (15): 1339-46. 10.1001/jama.280.15.1339.
Auleley GR, Ravaud P, Giraudeau B, Kerboull L, Nizard R, Massin P, Garreau de Loubresse C, Vallee C, Durieux P: Implementation of the Ottawa ankle rules in France. A multicenter randomized controlled trial. JAMA. 1997, 277 (24): 1935-9. 10.1001/jama.277.24.1935.
Durieux P, Nizard R, Ravaud P, Mounier N, Lepage E: A clinical decision support system for prevention of venous thromboembolism: effect on physician behavior. JAMA. 2000, 283 (21): 2816-21. 10.1001/jama.283.21.2816.
Durieux P, Ravaud P, Dosquet P, Durocher A: [Effectiveness of clinical guideline implementation strategies: systematic review of systematic reviews]. Gastroenterol Clin Biol. 2000, 24 (11): 1018-25.
Chatellier G, Colombet I, Degoulet P: An overview of the effect of computer-assisted management of anticoagulant therapy on the quality of anticoagulation. Int J Med Inf. 1998, 49 (3): 311-20. 10.1016/S1386-5056(98)00087-2.
Poller L, Shiach CR, MacCallum PK, Johansen AM, Munster AM, Magalhaes A, Jespersen J: Multicentre randomised study of computerised anticoagulant dosage. European Concerted Action on Anticoagulation. Lancet. 1998, 352 (9139): 1505-9. 10.1016/S0140-6736(98)04147-6.
Vadher BD, Patterson DL, Leaning M: Comparison of oral anticoagulant control by a nurse-practitioner using a computer decision-support system with that by clinicians. Clin Lab Haematol. 1997, 19 (3): 203-7. 10.1046/j.1365-2257.1997.00073.x.
Walton R, Dovey S, Harvey E, Freemantle N: Computer support for determining drug dose: systematic review and meta-analysis. BMJ. 1999, 318 (7189): 984-90.
Kaplan B: Evaluating informatics applications–clinical decision support systems literature review. Int J Med Inf. 2001, 64 (1): 15-37. 10.1016/S1386-5056(01)00183-6.
Kaplan B: Evaluating informatics applications–some alternative approaches: theory, social interactionism, and call for methodological pluralism. Int J Med Inf. 2001, 64 (1): 39-56. 10.1016/S1386-5056(01)00184-8.
The AGREE Collaboration: Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument. (last accessed on July 22,2004), [http://www.agreecollaboration.org]
Rosendaal FR, Cannegieter SC, van der Meer FJ, Briet E: A method to determine the optimal intensity of oral anticoagulant therapy. Thromb Haemost. 1993, 69 (3): 236-9.
Wolfinger R, O'Connell M: Generalized linear mixed models: a pseudo-likelihood approach. J Stat Comput Simulation. 1993, 48: 233-43.
Fiessinger JN: Traitement anticoagulant. Maladie thrombo-embolique veineuse.In: Elias A FJe, editor. Masson ed. Paris ed. 1995, 122-123.
Roberts GW, Druskeit T, Jorgensen LE, Wing LM, Gallus AS, Miller C, Cosh D, Eaton VS: Comparison of an age adjusted warfarin loading protocol with empirical dosing and Fennerty's protocol. Aust N Z J Med. 1999, 29 (5): 731-6.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/4/27/prepub
We thank the PHRC-OAT Study group for their comments and advice: Lise Marin (Medical Informatics Department), Martine Alhenc-Gelas (Haemostasis Laboratory), Eric Durand (Cardiology Department), François Ledru (Cardiology Department), Agnès Lillo-Le-Louët (Pharmacovigilance), Brigitte Sabatier (Pharmacy)
IC and PD, conceived, wrote the protocol and prepared the manuscript.
GC is the statistical expert and performed the power calculations
GC and ABR revised the protocol and the manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
About this article
Cite this article
Colombet, I., Bura-Rivière, A., Chatila, R. et al. Personalized versus non-personalized computerized decision support system to increase therapeutic quality control of oral anticoagulant therapy: an alternating time series analysis. BMC Health Serv Res 4, 27 (2004). https://doi.org/10.1186/1472-6963-4-27