Can surveying practitioners about their practices help identify priority clinical practice guideline topics?

Abstract

Background

Clinical practice guidelines are systematically developed statements designed to assist in patient and physician clinical decision making for specific clinical circumstances. In order to establish which guideline topics are priorities, practitioners were surveyed regarding their current practice.

Methods

One hundred ninety-seven practitioners in Ontario, Canada were mailed a survey exploring their current practice or opinion regarding the prophylactic use of anticonvulsant drugs in patients with malignant glioma who had never had a seizure. The survey consisted of seven questions regarding the relevance of a guideline on the subject to the practitioner's practice, the proportion of clinical cases involving anticonvulsant use, knowledge of existing guidelines on this topic, interest in reviewing a completed practice guideline and three clinical scenarios.

Results

There were 122 respondents who returned the survey (62% rate of return). Eighty percent of the practitioners who responded indicated that less than 25% of their clinical cases involved the use of anticonvulsants; however, only 16% of respondents indicated that a practice guideline would be irrelevant to their practice. Eighty percent of respondents volunteered to review a draft version of a practice guideline on the use of anticonvulsants. The survey presented the practitioners with three scenarios where anticonvulsants in patients with brain tumours may be appropriate: peri-operatively in patients without seizures, postoperatively in patients currently using anticonvulsants, and thirdly in patients not currently using anticonvulsants or undergoing surgery. In contrast to the third situation, the first two situations yielded considerable variation in practitioner response.

Conclusion

The survey established that there is some variation present in the current practice of anticonvulsant use in the patients with brain tumours. Whether there is an optimal treatment practice has yet to be determined. Practitioners do seem to feel that a guideline on anticonvulsant use in warranted, and most practitioners would be interested in being part of the guideline development process.

Background

Clinical practice guidelines are systematically developed statements designed to assist in patient and physician clinical decision making for specific clinical circumstances [1]. They can also be used to inform policy decisions [2]. The steps involved in developing evidence-based clinical practice guidelines include the identification of clinical problems, synthesis of the evidence, formulation of recommendations, independent review of the guideline, dissemination of the guideline, and maintenance of the guideline [3]. Choosing a guideline topic entails careful consideration of various issues. The Agency for Healthcare Research and Quality (AHRQ) has developed a list of six factors that should be considered when selecting a guideline topic: the potential of a guideline to reduce unwanted variation in practice, the prevalence of a specific illness, the adequacy of the evidence that informs recommendations, the amenability of a condition to prevention or treatment, the needs of a special population group, and costs [4]. Others have also identified likelihood to influence change in practice, relevance, burden of illness, and the potential for significant benefit and harm as factors to consider when selecting a guideline topic [3].

Prior to the development of a clinical practice guideline, an environmental scan or preliminary practice survey of practitioners who are the intended audience can serve many purposes. Data can be collected on clinicians' reported practice in a given field, their opinions on the relevance and need for a guideline on a particular topic, and their knowledge of existing evidence sources or guidelines that could be incorporated into the guideline development process. The survey can help prioritize potential guideline topics and aid in choosing a topic that could potentially have the greatest impact. In addition, evidence indicates that participation by practitioners in the process is linked to positive attitudes towards the guidelines and potentially influences the extent to which they integrate the final recommendations into their practice [5].

The purpose of this study was to conduct a pre-guideline practice survey of relevant clinicians on the use of prophylactic anticonvulsant drugs in patients with brain tumours. Our intent was to assess self-reports of clinicians' current use of anticonvulsant drugs for this patient population, their perceptions of the need for clinical practice guidelines on the use of anticonvulsants, feedback regarding their knowledge of existing guidelines, and their interest in participating in the guideline process as an external reviewer.

Methods

Context

The Practice Guidelines Initiative of Cancer Care Ontario's Program in Evidence-based Care provides context for this study. The Initiative's core activity is the development of cancer practice guidelines by multidisciplinary guideline panels, disease site groups (DSGs), using the methodology of the Practice Guidelines Development Cycle [3]. Recently, the Neuro-oncology DSG identified the use of prophylactic anticonvulsant drugs in patients with brain tumours as a potential guideline topic. During discussions on whether to pursue this topic, members of the DSG reflected on their perceptions of the clinical use of these drugs. Practitioners sometimes administer prophylactic anticonvulsant drugs to patients with brain tumours to prevent an initial seizure, because these drugs are used to manage patients who have a history of seizures. Practitioners also commonly use prophylactic anticonvulsant drugs for similar clinical situations: for example, in the peri-operative period following craniotomy or after closed head injury. Neurosurgeons are, therefore, familiar with the use of prophylactic anticonvulsant drugs, and some extend this use to patients with newly diagnosed brain tumours, despite the lack of evidence favouring such a practice. The concern with prescribing prophylactic anticonvulsant drugs is that there are several adverse effects associated with these drugs, including cognitive impairment, liver damage, and skin rashes.

Before the Neuro-oncology DSG developed the guideline, members decided to conduct a practice survey of their intended audience, Ontario clinicians, to determine if there was support in the practicing community for this topic and if their perceptions of use of this drug were supported.

Participants

Practitioners who managed patients with neurological diagnoses were eligible to participate in the study. One hundred ninety-seven practitioners were identified through a database of the Practice Guideline Initiative that provides information on clinicians and details of their clinical practice. Practitioners are eligible to be included on the database if they practice in Ontario, Canada and treat patients with cancer. The database includes information regarding the specific occupation of the practitioners (medical oncologist, radiation oncologist, surgeon, etc.), the specialty of the practitioner (neurology, hematology, etc.) and the location of practice (regional cancer centre, community or private medical office).

Procedure

The practitioners selected from the database were sent a letter describing the purpose of the survey, the survey, and a self-addressed postage-paid envelope in which to return the survey. The practitioners were given the option of either faxing or mailing the survey back to the researchers. To maximize response rate, a modified Dillman technique [6] was used. Two weeks after the first survey package was sent, practitioners who had not responded were sent a postcard to remind them of the survey. Two weeks after the postcards were sent, the survey, another letter describing the purpose of the survey, and a self-addressed postage-paid envelope were sent to the practitioners who still had not responded.

Survey

The survey consisted of seven questions. Three questions focused on the relevance of the potential guideline topics to the respondents' clinical practice (5-point scale: very relevant to very irrelevant); the proportion of their practice related to the targeted clinical conditions (<25%, 26–50%, 51–75% and 76%+); and knowledge of existing guidelines on these topics (open question). Respondents were then presented with three clinical scenarios:

• 1. In patients who have never had seizures, when are peri-operative anticonvulsant medications indicated? There was a 5-point likert scale to measure response for this question, ranging from "always" to "never".

• 2. An otherwise healthy 58 year old woman is found to have an enhancing right frontal lobe mass on a CT scan ordered for headaches. There is no history seizures. A craniotomy for excision of a suspected malignant glioma is planned. After successful surgery the pathology returns as a malignant glioma (GBM). The patient is planned for and starts external beam radiotherapy. Assume the patient was treated with anticonvulsant medications pre-operatively and is free of anticonvulsant drug side effects and is seizure-free. There were five choices for this question: "I recommend that patients remain on anticonvulsants"; "I recommend that patients taper and discontinue the anticonvulsants"; "I discuss the benefits and risks of anticonvulsants with the patient and let him or her decide what to do"; "other"; or "I do not encounter this clinical situation".

• 3. A 58 year old woman with known lung cancer develops multiple supratentorial brain metastases. No surgery is planned. There is no history of seizures. There were five choices for this question: "I recommend prophylactic anticonvulsants in this situation"; "I recommend that prophylactic anticonvulsants not be started in this situation, but would monitor and recommend them if a seizure occurred"; "I discuss the benefits and risks of anticonvulsants with the patient and let him or her decide what to do"; "other"; or "I do not encounter this clinical situation".

The final question asked practitioners if they would be interested in participating in the process of reviewing the guideline.

Statistical analysis

The data were analysed using the Statistical Package for the Social Sciences, version 11.5 (SPSS). Frequencies of response were tabulated for each question according to the practitioners' disciplines. Missing responses were excluded from the analysis. Crosstabulations were calculated to establish the characteristics of the practitioners who did not want to participate in reviewing a guideline.

Results

Description of participant sample

Of the 197 surveys sent to practitioners, 122 were returned (62%; 7 medical oncologists, 72 neurologists, 14 radiation oncologists, 27 surgeons, and 2 other). Return rates ranged from 54% (medical oncologists) to 93% (radiation oncologists). Practitioners who worked at regional cancer centres had a slightly higher return rate than practitioners who worked in community or private medical offices (65% versus 60%).

Only nineteen of the practitioners (16%) indicated that the development of a guideline on the use of prophylactic anticonvulsants would not be relevant to their practice. However, the majority of practitioners (80%) indicated that less than one quarter of their clinical cases involved decisions regarding prophylactic anticonvulsant drug use. Neurologists and surgeons had a greater proportion of their caseloads involving these decisions than did other disciplines.

Published guidelines

Eleven (9%) of 116 practitioners indicated that they were aware of existing guidelines. Eight of these cited the practice parameter on anticonvulsant prophylaxis published in Neurology in 2000 by Glantz et al [7]. One respondent referred to a 1993 review published by Agbi & Bernstein in Canadian Family Physician that described seizure prophylaxis for brain tumour patients [8].

Clinical scenarios

Use of peri-operative anticonvulsants

Of the 114 practitioners who responded to this item, 25 (22%) thought that peri-operative anticonvulsant medications were "always" indicated, 55 (48%) felt that peri-operative anticonvulsant medications were "sometimes" indicated, and the remaining 33 (30%) felt that peri-operative medications were "never" indicated.

Patient currently using anticonvulsants, undergoing surgery

As was the case above, there was variation in response to this scenario: 33% of the practitioners indicated that they would recommend this patient stay on anticonvulsants, 31% of the practitioners indicated that they would taper and discontinue the anticonvulsants, and 20% of the practitioners indicated that they would discuss the benefits and risks of the anticonvulsants and let the patient decide what to do.

Patient not currently using anticonvulsants, not undergoing surgery

The most common response to the scenario describing a patient not currently using prophylactic anticonvulsant drugs was to recommend that prophylactic anticonvulsants not be started unless the patient had a seizure (74%). Twelve percent of the respondents indicated that they would recommend anticonvulsants in this situation. The remaining 13% of respondents did not encounter the situation (5%), would recommend another treatment not listed (4%), or would discuss the benefits and risks and let the patient decide (4%).

Practitioners' choice to review a draft guideline

Of the 122 practitioners who responded, 88 responded that they would be interested in reviewing a draft version of a guideline, and 25 practitioners indicated they would not want to review a draft of the guideline (80% versus 20%). In comparison to those who indicated a willingness to review a draft of the document, non-reviewers were more apt to say that the topic was irrelevant (9% versus 40%, p < 0.001), were somewhat more likely to have fewer than 25% of their clinical cases involving anticonvulsant drug use decisions (77% versus 84%, p=not significant), and were more likely to indicate that they did not encounter patients represented in the clinical scenarios described (for treatment of patients currently using anticonvulsants, 8% versus 16%; for treatment of patients not currently using anticonvulsants, 3% versus 12%. Note that p-values are not reported because there are too few respondents (<5) to calculate chi square in a 'do not encounter' category). The greatest variation in response between practitioners who wanted to review and those who did not want to review was observed for the question on when peri-operative medications were indicated. Twenty-eight percent of the practitioners who wanted to review responded that peri-operative anticonvulsants were "always" necessary; none of the non-reviewers indicated that peri-operative anticonvulsants were "always" indicated (p < 0.001).

Discussion

The survey presented the practitioners with three situations where anticonvulsants in patients with brain tumours may be appropriate: peri-operatively in patients without seizures; postoperatively in patients currently using anticonvulsants, and in patients not currently using anticonvulsants and not undergoing surgery. In contrast to the third scenario, the first two situations yielded considerable variation in practitioner response. One limitation of the findings presented is that they are based on self-reported data rather than on an objective analysis of clinical practice. Self-reporting may bias response tendencies: practitioners may respond to questions in the way that they think the researchers want them to answer or they may have poor recall of their own practice [9, 10]. Another possible limitation is the potential response bias. The return rate was 62%, and it is possible that that the non-responders have no variation in practice. Nonetheless, the potential limitations of the self-reported data and response bias are unlikely to account for the considerable variation reported.

By identifying the practice variations, the guideline can address these variations so that the guideline will be applicable to more practitioners, not just a subgroup of practitioners performing what is assumed to be the most common practice. It is important to recognize that variations in practice do not necessarily imply that there is an explicitly wrong and right practice. Variations in practice can be based on patients' needs, morbidity rates and variations in consumer preferences for different outcomes [11]. The evidence, vis a vis, a guideline, will help determine if there is a "correct answer" or whether the variation is appropriate.

The data gathered from the survey may serve as an important baseline for comparing future self reports in order to measure the degree of change that occurs as a guideline is developed, documented, and implemented. The primary purpose for conducting a practice survey such as this is to gauge the level of interest in the clinical community. In this case, the majority of clinicians (84%) supported the development of a clinical practice guideline on the use of anticonvulsants in patients with brain tumours because it was relevant to their practice, and 80% volunteered to serve as external reviewers for the guideline. This early 'buy-in' not only promotes the guideline process and development but also provides a voice for clinicians and may facilitate acceptance and implementation of the eventual guideline recommendations. It will be interesting to observe whether the practitioners who volunteered to review the guideline, review the guideline when it is complete (projected completion date is winter 2004).

The 25 practitioners who said they did not want to review a guideline were more apt to report that the topic was not relevant to their practice than were those who volunteered to review a guideline (40% versus 9%). Nonetheless, the topic was a priority for the majority of respondents. Unfortunately, the survey did not ask practitioners why they did not want to participate any further in the development of the guideline. Collecting data on why practitioners do not want to participate in the guideline process, even in circumstances where the topic may be relevant to a component of their clinical practice, could help guideline developers tailor the process in the future to encourage the participation of more practitioners. Another possible question for future surveys could involve asking the practitioners if there are any topics that they feel are priorities.

Conclusions

The survey was designed to answer three questions: the reported current practice in Ontario for the prophylactic use of anticonvulsants as part of the treatment for brain tumour patients, the need for a guideline on such a use of anticonvulsants, and the involvement of practitioners in the development of this guideline. The survey established that there is some variation in the reported current practice of prophylactic anticonvulsant therapy for patients with brain tumours. Whether there is an optimal practice has yet to be determined. Practitioners do seem to feel that a guideline on prophylactic anticonvulsant use is warranted, and most practitioners would be interested in participating in the guideline development process.