Skip to main content

Advertisement

We’d like to understand how you use our websites in order to improve them. Register your interest.
Download PDF

Does a joint development and dissemination of multidisciplinary guidelines improve prescribing behaviour: a pre/post study with concurrent control group and a randomised trial

BMC Health Services Research volume 6, Article number: 145 (2006) Cite this article

Abstract

Background

It is difficult to keep control over prescribing behaviour in general practices. The purpose of this study was to assess the effects of a dissemination strategy of multidisciplinary guidelines on the volume of drug prescribing.

Methods

The study included two designs, a quasi-experimental pre/post study with concurrent control group and a random sample of GPs within the intervention group. The intervention area with 53 GPs was compared with a control group of 54 randomly selected GPs in the south and centre of the Netherlands. Additionally, a randomisation was executed in the intervention group to create two arms with 27 GPs who were more intensively involved in the development of the guideline and 26 GPs in the control group.

A multidisciplinary committee developed prescription guidelines. Subsequently these guidelines were disseminated to all GPs in the intervention region. Additional effects were studied in the subgroup trial in which GPs were invited to be more intensively involved in the guideline development procedure. The guidelines contained 14 recommendations on antibiotics, asthma/COPD drugs and cholesterol drugs

The main outcome measures were prescription data of a three-year period (one year before and 2 years after guideline dissemination) and proportion of change according to recommendations.

Results

Significant short-term improvements were seen for one recommendation: mupirocin. Long-term changes were found for cholesterol drug prescriptions. No additional changes were seen for the randomised controlled study in the subgroup. GPs did not take up the invitation for involvement.

Conclusion

Disseminating multidisciplinary guidelines that were developed within a region, has no clear effect on prescribing behaviour even though GPs and specialists were involved more intensively in their development. Apparently, more effort is needed to bring about change.

Peer Review reports

Background

Health care expenditure in the Netherlands increases each year, as it does in other European countries. Drug prescribing is an important contributor to this increase in costs, and these costs are expected to continue to increase relevantly in the coming years [1–3]. Medication is not always prescribed effectively: drugs may be prescribed unnecessarily and lower-cost alternatives are not always taken into consideration. The problem can be attributed to demographic factors (growth and ageing of the population), the trend towards new but usually more expensive drugs, patients' increased awareness, pressure from the industry, and last but not least the differences in prescribing practices between primary and secondary care oriented physicians [4, 5]. Physicians are also known to prescribe more easily as a result of today's high level of work-related stress and routines [6].

Clinical guidelines may induce small improvements, both in processes and in the outcomes of care [7]. Simple top-down dissemination of mono-disciplinary guidelines alone is not effective [8–12]. To bring about change, a more powerful strategy could be multidisciplinary guidelines and probably even more so if key regional representatives from primary and secondary care are involved in the development. Implementation experts indicate that multistage involvement in the development of a guideline can be a positive contributor to effective implementation of guidelines [13, 14]. If national guidelines are involved, adjusting them to the local situation and to specified needs experienced in the target group increases commitment [15, 16]. We have therefore adopted an intensive guideline development procedure, involving local representatives of the relevant medical disciplines [17, 18].

The intensity is related to the time and energy that was put into the strategy of guideline development to create support among GPs. We set up a study in which we assessed the effects of our intensive multistage guideline development. We hypothesised that this strategy could lead to a modest but relevant change of volumes of prescriptions in the desired direction. As we used two comparisons, we were able to study the effects of dissemination separately from the effects of involving the target group in the preparation and development of the guidelines.

Methods

Design and population

A quasi-experimental study (a pre/post study with a concurrent control group) was executed (fig. 1). Since we had an enormous amount of data from the insurance companies at our disposal (about 700 GPs in the centre and south of the Netherlands), it was practically impossible to include them all. The Maastricht region (intervention region, n = 53 GPs) was compared with a random sample of 54 GPs drawn from the remaining GPs (control group). First, however, an randomised controlled trial (RCT) was carried out within the intervention group; 27 GPs were invited for a more intense role in the development procedure and were compared with 26 GPs in a corresponding control group. The inclusion criteria were: completeness of the GPs' data (no missing data per GP for more than one year) and at least 500 patients in the GPs' practice. As the two groups in the RCT turned out to be comparable, we executed the second comparison: we compared the entire Maastricht region with an external control group, selected at random from all the available data. A pre trial power-calculation was not performed; the study concerned a pragmatic trial, where evaluation was planned alongside an intervention being executed anyway. For this study, only GPs who were not already participating in another regional intervention of similar character, were sent the guideline by post and invited to give comments (n = 53 GPs). The GPs in both designs were not aware of the fact that they were in an evaluation study, because only anonymous volume data were collected from an existing database. There was therefore no need for GPs or patients to give informed consent. The asthma/COPD and cholesterol guidelines were disseminated in March 2002. Baseline data were gathered one year before, and follow-up measurements were made one and two years after the dissemination of each guideline.

Figure 1
figure1

Flowchart of research population and randomised trial in subgroup.

Full size image

Strategy

The guideline topics were chosen by the steering committee, existing of decision-making representatives (pharmacists, GPs, hospital staff and regional insurance company). Clinical importance in primary care (common problems) and expected health gains were the main criteria in prioritisation. The first multidisciplinary guideline was developed for antibiotics, because non-rational attitudes towards prescriptions and an increase of resistance in micro-organisms have been observed. Over 50% of the antibiotics prescriptions for respiratory tract problems are not used appropriately [19, 20]. Guidelines for asthma, COPD and cholesterol have also been developed, due to the high prevalence of these diseases in the population and the corresponding high costs [21]. The guidelines were actually developed by independent multidisciplinary expert teams that included community pharmacists, specialists, GPs and a hospital pharmacist, and were based on Dutch national guidelines, practical experience and consensus. The participating GPs were not involved in the trial.

After the development, we used different methods to validate the conceptual guidelines. For the guideline on antibiotics we started with visiting PTAMs in the region to get comments on the conceptual guidelines and create involvement. For the guidelines on asthma, COPD and cholesterol, the conceptual guidelines were sent by post to a random sample of GPs within the intervention group, and those GPs were asked to comment and encouraged to do so. The guideline committee stressed the importance of knowing the GPs' comments, and that the comments would be taken very seriously in finalising the guideline. Due to the low response rate and based on the expertise of the expert teams, a number of key individual GPs were also contacted in a more active way to generate comments. The guidelines were finalised by presenting the GPs comments to the multidisciplinary expert teams. The Integrated Care Unit of the University Hospital of Maastricht provided both the expertise and the facilitating conditions to disseminate the guidelines within the region.

All the GPs in the intervention group received the finalised paper guidelines in a ring binder by post. The antibiotics guideline was sent out in 2001 and the asthma, COPD and cholesterol guidelines were forwarded in 2002.

Data collection and analysis

Data were supplied by the two largest insurance companies in the region, which implied that about 70 percent of the total population in the region was covered.

Prescription data were gathered retrospectively per GP per month during the period 2001–2004. Expected directions of change have been defined based on the detailed recommendations contained in the guidelines, in combination with estimates based on the expertise of the initially involved key regional representatives. The drugs that were selected and the desired directions of change are listed in tables 1 and 2, in the first and second columns.

Table 1 Mean total number of drug prescriptions per GP per year, standardised per 1000 enlisted patients for the total intervention region versus a matched control group
Full size table
Table 2 Mean total number of drug prescriptions per GP per year, standardised per 1000 enlisted patients for the randomised more intensively involved group in the intervention region versus a matched control group
Full size table

The effect of the strategy and the differences in the pre/post changes between groups were tested with unpaired Student's t-tests and Mann Whitney tests using SPSS 12.0.1 software. Short-term effects were calculated one year after the dissemination of the guidelines, and long-term effects two years after. We set our alpha conservatively at 0.01 because of the multiple testing.

Results

As far as we can ascertain, the GPs in the two groups did not differ on the main characteristics. The mean age of male GPs in the intervention group was 50.0 years and 53.3 in the control group. The mean age of the female GPs in the intervention group was 48.8 years and 45.5 in the control group. Overall, the percentage of male GPs was only slightly different in both groups: in the intervention group, 85%(45) of the GPs were male, as compared with 76%(41) in the control group. Table 1 presents the mean total number of drug prescriptions per GP per year, standardised per 1000 enlisted patients over a baseline measurement (2001 or 2002), the short-term effects (one year) and long-term effects (two years). Table 1 also presents the absolute changes in the short term and long term per 1000 enlisted patients per GP. Table 2 presents the same data for the random sample of GPs who were invited for a more intense role in the development procedure.

Less than 10% of the GPs that were invited to play a greater role actually commented on the conceptual guidelines. The exact number varied according to the specific guideline. Most of the comments did not relate to content, but rather concerned questions about the development procedure.

For the guideline dissemination (quasi experiment), results show that a significant change in the desired direction was found for one specific antibiotic (mupirocin): a reduction of one prescription per 1000 patients compared to an increase of one (p = 0.0014) in the short term. No significant effect was seen for other antibiotics, either in the short term or the long term. For cholesterol, significant long-term effects were found; an increase of 112 prescriptions per 1000 patients in the intervention group compared to an increase of 54 in the control group (p < 0.0001). No significant effect was seen in the short term or long term for any asthma and COPD drugs.

For antibiotics and cholesterol, no additional effects were found in the randomised study of GPs with a greater involvement (table 2). For asthma and COPD drugs, significant changes were found in the control group instead of the intervention group. Instead of the desired increase, we found a decrease of 9 prescriptions per 1000 patients in the intervention group compared with a decrease of one prescription in the control group for inhaled corticosteroids for asthma (p = 0.0099).

Discussion

We found no clear impact of disseminating the rigorously developed multidisciplinary guidelines on the prescribing performance of GPs. A significant effect was found for two drugs: mupirocin and cholesterol lowering drugs. The change in mupirocin however does not seem very relevant, because it is rarely prescribed and by and large prescribed for minor ailments. The long-term effect for cholesterol lowering drugs is interesting. The standard deviation of prescribing drugs per GP turned out to be unexpectedly large. Although no significant effects were seen for the other drugs, only small changes can be seen on other indicators. The invitation for GPs to be involved in the development procedure caused no desirable effects in the subgroup.

There are some issues for consideration. First, although unlikely the quasi-experimental design cannot guarantee that the observed changes are the result of the strategy. It is possible that the effects found were caused by coincidence or by an unknown non-specific factor. Although the intervention group in both designs seems to be higher in volumes of prescription at baseline, both groups seem comparable in the exposure to quality improvement strategies that have been performed otherwise in the region. A second issue for consideration is the pragmatic base of the trial in which two regions were compared. This has caused a restriction in the number of GPs in the trial. We were not in the position to improve this. Together with the relatively high variation among the primary variable (volumes of prescription per GP per 1000 patients per year) the power of the study was negatively influenced. Another power calculation executed after ending the study, based on the large standard deviations and the number of GPs, showed that this resulted in a power of 40%. In this study only large differences could be detected. We found a few significant results, although the significant effect of mupirocin is not so interesting because mupirocin is rarely prescribed and the disease is a minor ailment. Given the fact that this study was no selected sample and the participating GPs were a fine representation for the Dutch GPs other results are not expected in larger trials. Besides, when evaluating this intervention in an economic study we wonder if the changes in prescribing behaviour are in balance to such huge efforts made in investment and implementation costs.

Only volume data of prescriptions were used in this initial stage of the study. These figures are therefore not linked to any diagnosis. However, some drugs are prescribed for several different medical problems, while the guideline may contain opposing prescription recommendations. For example, amoxicillin is recommended for the treatment of bronchitis and otitis media acuta, whereas amoxicillin is discouraged in cases of acute cystitis. On the one hand, for statin use, there is a situation of overuse for patients with low risk on cardiovascular disease; on the other hand, there is a situation of underuse for high-risk patients like diabetes patients [21]. It is therefore possible that the results are somewhat underestimated in the sense that possible results were kept hidden. Additional information about diagnosis and cholesterol rates would be needed to make a more precise analysis.

A strong aspect of our study is that long-term effects were measured. For statin-prescription, the initial non-significant effect appeared to be significant at the long run. A positive factor that may have contributed to an increased willingness to follow the cholesterol medication guideline is that our strategy concerns guidelines that have been formulated by representatives from all disciplines and was intended to improve prescription behaviour in both primary care physicians ánd medical specialists.

Overall, the implementation strategy only generated limited results. Implementation experts indicate that if guidelines are developed by the clinicians who are to use them this will be a positive contributor to the effect of guidelines. Achieving consensus on draft guidelines and creating a sense of 'ownership' through a process of development on several levels (central, local and individual) is essential for successful implementation of guidelines [22, 23]. We have paid specific attention to this aspect, firstly by involving representatives from all disciplines in the development of the guidelines. The term intensity is therefore related to the time and energy that was put into the guideline development strategy to create support among GPs. Secondly, we also focused specifically on the doctors' acceptance of the guidelines by including them in the validation procedure. It is not clear why the GPs did not respond to the invitation for comment. Possibly, we were successful in choosing opinion leaders to such extent that GPs did not see the need to comment on guidelines that were prepared by GPs they trusted. A second explanation lies in the assumption that the invitation by mail did not work: only a few GPs responded to the invitation to comment on the guideline. GPs might have had comments but did not think this would have any impact in the final guideline. In short, we did not succeed in getting GPs involved locally in the validation procedure. Apparently this way of getting GPs involved is not appropriate and possibly explains why the effect was only limited.

To create involvement in the validation procedure we searched for a successful method to achieve this. During the validation of the antibiotics guideline we started with visiting PTAMs in the region to get comments on the conceptual guidelines and create involvement. This turned out to be very time consuming and caused much delay in the dissemination of the final guideline. Therefore, during the validation of the asthma and the COPD guideline we chose for a less labour-intensive method by asking GPs to comment by mail. After finishing the analyses for this study, we subsequently started (not included in the data of the manuscript) with inviting GPs to feedback meetings. The results of this method were more convincing. We highly recommend the feedback meetings to be an effective method to create involvement among GPs during the validation procedure.

To obtain a more solid or extensive effect, an additional implementation strategy is desired. Feedback and reminders could be an necessary strategy to change GPs behaviour [24–26]. For example, a computerised decision support system could give feedback to a doctor as soon as the prescription behaviour differs from the behaviour recommended in the guideline. As long as the doctor does not deviate from the guideline, the system will not interfere with the primary care physicians' information system, and the doctor giving the prescription will not be interrupted by the feedback system.

We did not perform an economic evaluation by relating the (change in) costs of prescription to the costs of the interventions (guideline development and dissemination) [27, 28]. If we were to choose to compare the total expenses of the intensive development procedure of all guidelines with the cost savings, we would expect all the findings to be negative, because the strategy was time and energy consuming and its impact was only moderate. Before aiming to intensify the development procedure of guidelines, the costs and savings of such a strategy need to be investigated thoroughly and executed when the proceeds of the intervention outweighs the costs of it [29, 30].

Conclusion

Disseminating multidisciplinary guidelines that were developed within a region, has no clear effect on prescribing behaviour even though GPs and specialists were involved more intensively in their development. Apparently, more effort is needed to bring about change.

References

  1. 1.

    Centraal Bureau voor de Statistiek: Statistisch jaarboek 2003 (Statistical yearbook 2003). 2003, Heerlen: Centraal Bureau voor de Statistiek, 96.

  2. 2.

    Stichting Farmaceutische Kengetallen: Data en Feiten (Foundation for Pharmaceutical Statistics; Data and facts). The Hague/the Netherlands;. 2000, –2004

  3. 3.

    Breekveldt-Postma NS, Zwart-van Rijkom JE, Egberts AC, Leufkens HG, Herings RM: Rising costs of drugs in hospitals in the period 1996–2000 and over the next few years (Article in Dutch). Ned Tijdschr Geneeskd. 146 (52): 2547-51. 2002 Dec 28; Erratum in: Ned Tijdschr Geneeskd. 2003 Dec 27;247(52):2612. Ned Tijdschr Geneeskd. 2003 Jan 25;147(4):184

  4. 4.

    Dobson R: Pharmaceutical industry is main influence in GP prescribing. BMJ. 2003, 326: 301-10.1136/bmj.326.7384.301.

  5. 5.

    Prosser H, Almond S, Walley T: Influences on GPs' decision to prescribe new drugs; the importance of who says what. Family Practice. 2003, 20 (1): 61-8. 10.1093/fampra/20.1.61.

  6. 6.

    Evans J, Lambert T, Goldacre M: GP recruitment and retention: a qualitative analysis of doctors' comments about training for and working in general practice. Occas Pap R Coll Gen Pract. 2002, iii-vi. 1–33, 83

  7. 7.

    Grimshaw J, McAuley LM, Bero LA, Grilli R, Oxman AD, Ramsay C, Vale L, Zwarenstein : Systematic reviews of the effectiveness of quality improvement strategies and programmes. Qual Saf Health Care. 2003, 12 (4): 298-303. 10.1136/qhc.12.4.298.

  8. 8.

    Kosecoff J, Kanouse DE, Rogers WH, McCloskey L, Winslow CM, Brook RH: Effects of the national Institutes of Health Consensus Development Program on physician practice. JAMA. 1987, 258: 2708-13. 10.1001/jama.258.19.2708.

  9. 9.

    Grimshaw J, Eccles M, Tetroe J: Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004, 24 (Suppl 1): S31-7. 10.1002/chp.1340240506.

  10. 10.

    Grol R: Implementation of evidence and guidelines in clinical practice: a new field of research?. Int J Qual Health Care. 2000, 12 (6): 455-456. 10.1093/intqhc/12.6.455.

  11. 11.

    Jamtvedt G, Young JM, Kristoffersen DT, Thomson O'Brien MA, Oxman AD: Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2003, CD000259-3

  12. 12.

    Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA: Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ. 317 (7156): 465-468. 1998 Aug 15

  13. 13.

    Grol R: Development of guidelines for general practice care. Br J Gen Pract. 1993, 43 (369): 146-51.

  14. 14.

    Grol R, Wensing M, Eccles M, (red): Improving Patient Care; The implementation of change in clinical practice. 2005, London: Elsevier Limited

  15. 15.

    Pagliari C, Grimshaw J, Eccles M: The potential influence of small group processes on guideline development. J Eval Clin Pract. 2001, 7 (2): 165-73. 10.1046/j.1365-2753.2001.00272.x.

  16. 16.

    Hutchinson A, McIntosh A, Cox S, Gilbert C: Towards efficient guidelines: how to monitor guideline use in primary care. Health Technol Assess. 2003, 7 (18): iii-1–97

  17. 17.

    NHS Centre for Reviews and Dissemination: Getting evidence into practice. Eff Health Care. 1999, 5 (1): 1-16. [http://www.york.ac.uk/inst/crd/ehc51.pdf]

  18. 18.

    Grol R: Personal paper. Beliefs and evidence in changing clinical practice. BMJ. 315 (7105): 418-421. 1997 Aug 16

  19. 19.

    Wise R, Haut T, Cars O: Antimicrobial resistance: is a major threat to public health. BMJ. 1998, 317: 609-10.

  20. 20.

    Worrall G, Young B, Knight V: Inappropriate use of antibiotics for acute respiratory tract infections in a rural emergency department. Can J Rural Med. 2005, 10 (2): 86-8.

  21. 21.

    Abookire SA, Karson AS, Fiskio J, Bates DW: Use and monitoring of "statin" lipid-lowering drugs compared with guidelines. Arch Intern Med. 161 (1): 53-8. 10.1001/archinte.161.1.53. 2001 Jan 8

  22. 22.

    Grimshaw JM, Russell IT: Achieving health gain through clinical guidelines II: Ensuring guidelines change medical practice. Qual Health Care. 1994, 3 (1): 45-52.

  23. 23.

    Grol R: Development of guidelines for general practice care. Br J Gen Pract. 1993, 43 (369): 146-51.

  24. 24.

    Winkens RA, Pop P, Bugter-Maessen AM, Grol RP, Kester AD, Beusmans GH, Knottnerus JA: Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet. 345 (8948): 498-502. 10.1016/S0140-6736(95)90588-X. 1995 Feb 25

  25. 25.

    Weijden van der, Wensing M, Giffel M, Winkens R, Grol R: Interventions aimed at influencing the use of diagnostic tests. Report. 2000, Maastricht University: centre for Quality of Care Research (WOK)

  26. 26.

    Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, Wilsdon J, Matowe L, Needham G, Gilbert F, Bond S: Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet. 357 (9266): 1406-9. 10.1016/S0140-6736(00)04564-5. 2001 May 5

  27. 27.

    Mason J, Freemantle N, Nazareth I, Eccles M, Haines A, Drummond M: When is it cost-effective to change the behavior of health professionals?. JAMA. 286 (23): 2988-92. 10.1001/jama.286.23.2988. 2001 Dec 19

  28. 28.

    Sculpher M: Evaluating the cost-effectiveness of interventions designed to increase the utilization of evidence-based guidelines. Fam Pract. 2000, 17 (Suppl 1): S26-31. 10.1093/fampra/17.suppl_1.S26.

  29. 29.

    Severens JL: Value for money of changing healthcare services? Economic evaluation of quality improvement. Qual Saf Health Care. 2003, 12: 366-371. 10.1136/qhc.12.5.366.

  30. 30.

    Eccles M, Mason J: How to develop cost-conscious guidelines. Health Technol Assess. 2001, 5 (16): 1-69.

Pre-publication history

  1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/6/145/prepub

Download references

Acknowledgements

We would like to thank the two local insurance companies for providing the data for this study.

The health care insurance companies CZ and VGZ were the funders for this study, and for two of the authors (JM and RW). The other authors were funded by Academic Hospital Maastricht. The funding bodies did not have any role in or influence on the study design, data collection, interpretation of data, the writing of the manuscripts and the decision to submit.

Author information

Author notes

    Affiliations

    1. Integrated Care Unit, University Hospital Maastricht, The Netherlands
      • Jody D Martens
      • , Ron AG Winkens
      •  & Daisy de Bruyn
    2. Clinical Epidemiology and Medical Technology Assessment, University Hospital Maastricht, The Netherlands
      • Jody D Martens
      •  & Johan L Severens
    3. Department of General Practice, Maastricht University, The Netherlands
      • Ron AG Winkens
      •  & Trudy van der Weijden
    4. Centre for Quality of Care Research (WOK), University of Nijmegen and Maastricht University, The Netherlands
      • Trudy van der Weijden
    5. Department of Health Organisation, Policy, and Economics (BEOZ), Maastricht University, The Netherlands
      • Johan L Severens
    Authors
    1. Jody D Martens
      View author publications
      You can also search for this author in
    2. Ron AG Winkens
      View author publications
      You can also search for this author in
    3. Trudy van der Weijden
      View author publications
      You can also search for this author in
    4. Daisy de Bruyn
      View author publications
      You can also search for this author in
    5. Johan L Severens
      View author publications
      You can also search for this author in

    Corresponding author

    Correspondence to Jody D Martens.

    Additional information

    Competing interests

    The author(s) declare that they have no competing interests.

    Authors' contributions

    JM participated in the design and execution of the study, performed the statistical analysis, did the preparation and writing of the manuscript. RW did the concept development and coordination of the study and critical review of the manuscript. TW participated in the data analysis and writing of the manuscript. DB did the development of the databases, performed data-analyses and did critical review of the manuscript. JS did the coordination of the study, drafted the manuscript and did the critical reviewing of the manuscript. All authors read and approved the final manuscript.

    Ron AG Winkens, Trudy van der Weijden, Daisy de Bruyn and Johan L Severens contributed equally to this work.

    Authors’ original submitted files for images

    Below are the links to the authors’ original submitted files for images.

    Authors’ original file for figure 1

    Rights and permissions

    Reprints and Permissions

    About this article

    Cite this article

    Martens, J.D., Winkens, R.A., van der Weijden, T. et al. Does a joint development and dissemination of multidisciplinary guidelines improve prescribing behaviour: a pre/post study with concurrent control group and a randomised trial. BMC Health Serv Res 6, 145 (2006). https://doi.org/10.1186/1472-6963-6-145

    Download citation

    Keywords

    • Guideline Development
    • Mupirocin
    • Development Procedure
    • Minor Ailment
    • Computerise Decision Support System

    BMC Health Services Research

    ISSN: 1472-6963

    Contact us