The study began 4 years ago, so the search for guidelines is just from 1997 to 2007. After that, much time has been used to write and edit this article. And the data has not been updated, which can not reflect the status of guidelines in China of the last 4 years. This is a limitation of the study.
Progress of the amount of Chinese guidelines
There has been an exponential increase in the number of Chinese guidelines during the last decade. Much progress had been made. Because the Chinese Society of Rheumatic Disease developed many guidelines for rheumatic disease in 2003 (t = 7), and the Chinese Society of Osteoporosis, Bone and Mineral Disease also developed many in 2006 (t = 10), a large increase in the number of guidelines was obtained in those two years.
The impact on medical practice of Chinese guidelines was acceptable but not the same in all disciplines. The adoption speed of guidelines was not rapid
The Average Citation Rate is the number of times that all the guidelines having been cited divided by the number of all guidelines. This can reflect a journal’s influence. It is generally said that a journal has high academic impact when the average citation rate is high. The Immediacy Index is the number of times that all the guidelines were cited divided by the total number of guidelines published in the year when the guidelines were published for the first time. The index introduced by Garfield is used to measure the adoption speed. Good journals and good papers will be read and adopted quickly by many persons
. These two indexes were adopted in this study to reflect Chinese guidelines’ impact to some extent (not equate), as well as dissemination and utilization.
The results showed that the average citation rate of Chinese clinical guidelines was rather high (>30). That means the guidelines impact was acceptable. But the Immediacy Index of every year was low what indicated their adoption speed was not rapid. More attention should therefore be given to the dissemination. We can also see the impact difference of the guidelines as well as their quality from the uneven of these cited numbers.
It must be admitted that the citation indexes could not completely reflect the guidelines influence as being cited does not equal importance. And as most Chinese guidelines did not include references, the number of citations of other papers used for their development has not been calculated. These are all the limitations of this study.
Appraisals to guidelines development with the AGREE instrument; merits and shortcomings
There are accepted guideline evaluation instruments developed by different countries. Examples include the IOM’s “Provisional Instrument for Assessing Clinical Practice Guidelines” (IOM instrument), the “Method for Evaluating Research and Guidelines Evidence” (MERGE instrument), Cluzeau et al’s “Appraisal Instrument for Clinical Guidelines” (Cluzeau instrument), and Shaneyfelt et al’s methodological appraisal instrument (Shaneyfelt instrument). Of all these instruments, a study showed that The Cluzeau instrument was the most well developed and had been tested and described as a reliable and valid method of guideline evaluation
Based on the Cluzeau instrument, AGREE instrument was developed
. Up to now, it is the only guidelines instrument to have undergone extensive international validity assessment
. But there is a limitation that it does not evaluate the quality of evidence, what is better covered by GRADE, an approach to develop and present recommendations for management of patients through rating quality of evidence and grading strength of recommendation
Many other studies using the AGREE instrument for guideline evaluation reported the lowest scores in the applicability domain, and the highest in the scope and purpose domain
. Contrasting to that, the result of this study show that the lowest score is in the Editorial independence, and the highest is in the clarity and presentation in China. This result helps recognizing the defects in the development process of Chinese guidelines. It should pay more attention to the domains of editorial independence in the future, so as to stakeholder involvement, and applicability. As for items that the external review, updating, evidence selecting criteria and evidence evaluation should also receive more attention.
The method of developing guidelines in China is less scientific and lags behind the international level
Evidence-based guidelines apply the principles of evidence-based medicine to the process of guideline development. The first step is defining the clinical question. This is followed by defining the eligibility criteria for the studies. A systematic search of the literature is then conducted and the evidence is evaluated. In developing recommendations, the likely benefits, risks, inconvenience and costs associated with each treatment must be considered in addition to addressing patients’ underlying values and preferences. The quality of the data supporting the recommendations is evaluated and is reflected in a grading system that describes the strength of the recommendation and the quality of the supporting evidence. This process ultimately results in the systematic development of recommendations that incorporate evidence with patients’ preferences and values and indicates the quality of the evidence
However, in China, there have not been any criteria on how guidelines should be developed. The method of developing Chinese guidelines is less scientific and lags behind the international level. From the results it can be seen that most Chinese guidelines not with the evidence-based method. This makes it difficult to ensure quality. In addition, most Chinese ones have not listed references of important evidence. The user is not therefore able to examine the validity of the recommendations.
The interval and timeliness of Chinese guidelines’ updating
A ‘valid guideline should be up-to-date. Possible consequences of using out-of-date guidelines include a clinician’s use of diagnostic studies or treatments that do not provide the best-known outcomes
. So a good guideline should have specific updating timetable. R.E. Burton calculated that the half-life of a document in biology and medicine is three years based on Burton-Kebler’s aging equation for science and technology documents. Another study also suggested that as a general rule, guidelines should be reassessed for validity every three years
Of all Chinese guidelines developed between 1997–2007, only a few have been updated. And of them the average updating interval is more than three years, falling short of international standards. Because the search was done in 2008, some guidelines developed after 2004 have less time to be updated. So the number of updated guidelines would be expected to be less. This is also a limitation of the study.
Scope of Chinese guideline’s objective and target population are not wide enough
There is shortage of Chinese guidelines aimed at general practitioners and referral to specialist care. There is also a shortage of aiming at symptoms. This is behind international levels. For instance, in the case of ‘cough’, there is only one guidelines named “Guidelines for Cough Diagnosis and Treatment (Draft) in China”, while there are 24 in America. While general practice is developing in China, more and more general practitioners require guidelines for them, especially of referral and symptom center. Those guidelines should be assigned a priority.