- Research article
- Open Access
- Open Peer Review
Impact of picture archiving communication systems on rates of duplicate imaging: a before-after study
BMC Health Services Research volume 8, Article number: 234 (2008)
Electronic health information systems, such as picture archiving communication systems (PACS), are commonly believed to reduce the need for duplicate testing. However, empirical data to support this belief are not available.
Before-after study using administrative claims data from the Ontario Health Insurance Plan to determine whether the introduction of PACS at 10 hospitals in the Thames Valley region of southwestern Ontario, Canada between June 2004 and December 2005 reduced the frequency of duplicate imaging examinations. The imaging modalities studied were: chest and abdominal X-ray; computed tomography of the abdomen/pelvis, head, and chest. The frequency of duplicate testing was examined at 3 different time frames: 7 days, 30 days, and 60 days after a given index test.
Overall frequencies of duplicate imaging were: 2.7% within 7 days of an index imaging test, 6.7% within 30 days, and 9.8% within 60 days. Comparing the 12 months before and 12 months after PACS, absolute reductions in the frequency of duplicate X-rays using 7-day, 30-day, and 60-day time frames were: 0.2% (P = 0.01), 0.6% (P < 0.001), and 0.9% (P < 0.001), respectively. In contrast, there were absolute increases in the frequency of duplicate CT scans after PACS of 0.0% (P = 0.92), 0.5% (P = 0.01), and 0.5% (P = 0.01), respectively.
The frequency of duplicate imaging is relatively low and we did not find large reductions in duplicate imaging after the introduction of PACS. Independent evaluation of electronic medical systems should be conducted to confirm widely held beliefs of their potential benefits.
Picture archiving and communication systems (PACS) are computer networks dedicated to the storage, retrieval, and presentation of images produced by medical imaging devices. PACS replaces film archives and provides new capabilities such as the ability to access imaging information from various physical locations simultaneously and off-site viewing by clinicians and radiologists. PACS is commonly believed to reduce the number of unnecessary duplicate imaging tests ordered because original images are lost, misfiled, or stored at a remote location. If true, this would not only have important implications for decision-makers contemplating PACS implementation, but also for population health, given recent concerns about the potential risks of cancer from exposure to radiation during computed tomography scanning.
Very few studies have evaluated the reduction in redundant imaging associated with PACS implementation. A survey of clinicians conducted at Hammersmith Hospital, United Kingdom revealed that 78% of respondents had a problem with unavailability of inpatient images before PACS versus only 14% after PACS. Forty percent of respondents indicated that they ordered one or more repeat examinations per month because of unavailable images before PACS versus only 12% after PACS.[2, 3] More recently, in a survey of British Thoracic Society members, 71% of respondents agreed that there were fewer lost images when using PACS. However, the survey did not ask directly about the impact of PACS on the ordering of redundant imaging tests. Studies evaluating other aspects of PACS have again almost exclusively used survey- or interview-based methods to determine users' perceptions about PACS and have generally found very high levels of user satisfaction with many different aspects of PACS.[2, 3, 5–11] While these studies provide valuable insight into user perceptions, they rely on users' subjective and potentially biased interpretations of their experience with PACS. To our knowledge, few empirical data are available to support these perceptions. Accordingly, we conducted a before-after study using administrative health insurance claims data to determine whether the introduction of PACS was associated with a reduction in the frequency of duplicate imaging examinations.
The Thames Valley Digital Imaging Network project was an $18 million (Canadian dollars) [CAD]) initiative that implemented PACS at 10 hospitals in southwestern Ontario between June 2004 and December 2005. For this study, we considered PACS to have been fully implemented when a hospital officially began "filmless" operations. These dates, along with the annual imaging volumes of each hospital, are listed in Table 1.
We conducted a before-after study to examine changes in the frequency of repeat diagnostic imaging associated with the introduction of PACS in Thames Valley hospitals. We studied the following imaging modalities: chest X-ray, abdominal X-ray, computed tomography (CT) of the abdomen/pelvis, CT of the head, and CT of the chest. Spine X-ray, abdominal ultrasound, and magnetic resonance imaging were also considered for inclusion in this study, but were excluded since preliminary analyses revealed extremely low rates of repeat testing for these modalities (data not shown). This study received full approval from the Sunnybrook Health Sciences Centre Research Ethics Board.
Diagnostic imaging services in Ontario are provided without patient user fees. Ontario radiologists receive fee-for-service payment from the Ministry's Ontario Health Insurance Plan (OHIP) for outpatient X-ray examinations, and for both inpatient and outpatient CT examinations. Imaging examinations performed at Hospital A (Table 1) were not billed to OHIP during the observation period of this study; therefore, this hospital was excluded. From the OHIP database, we identified claims paid for imaging examinations using the following fee codes: chest X-ray, X090, X091, X092; abdominal X-ray, X100, X101; CT abdomen/pelvis, X126, X409, X410, X231, X232, X233; CT head, X188, X400, X401, X402, X405, X408; and, CT chest, X125, X406, X407. We used the institution number associated with each claim to identify imaging exams performed at Thames Valley institutions.
We used the change in the frequency of duplicate imaging examinations after the introduction of PACS as our primary outcome measure. For the purposes of this study, a duplicate examination was defined as the occurrence of the same imaging examination (i.e., identical OHIP fee code) within a specified time frame after the index test. Since the OHIP database does not contain information about the clinical indications for diagnostic imaging tests, duplicate examinations–as defined in this study–include both unnecessary and clinically warranted duplicate examinations. We assumed that the frequency of clinically warranted duplicate examinations was stable before and after the introduction of PACS and that, as a result, any observed change in the frequency of duplicate testing after PACS was attributable to a change in the frequency of unnecessary duplicate examinations (e.g., those ordered because original images were lost, misfiled, or at a remote location).
For each institution and each imaging modality of interest, we identified the number of index imaging tests that occurred at that institution during the 12 months before and 12 months after the introduction of PACS. Next, we determined the number of index tests associated with a duplicate test at that institution within a specified time after the index test (7 days, 30 days, or 60 days). Duplicate tests were not eligible to be counted as index tests. The frequency of duplicate testing was expressed as the proportion of index tests associated with a duplicate test. For each imaging modality, we compared the frequency of duplicate testing in the 12 months before PACS (combined across all institutions) to the frequency of duplicate testing in the 12 months after PACS (combined across all institutions) using a chi-squared test. P values less than 0.05 were considered statistically significant.
The frequencies of duplicate imaging are presented in Table 2. Overall, duplicate imaging, for X-ray and CT combined, was a relatively infrequent phenomenon with frequencies of 2.7% within 7 days of index imaging, 6.7% within 30 days, and 9.8% within 60 days.
For X-ray examinations, we observed statistically significant reductions in the frequency of duplicate testing after PACS using 7 day, 30 day, and 60 day time frames to identify duplicate tests. Although relative reductions in duplicate testing were 9.1%, 8.7%, and 8.5%, respectively, these translated into numerically small absolute reductions in duplicate testing of 0.2%, 0.6%, and 0.9%, respectively.
For CT, we did not observe any reduction in duplicate testing after PACS using a 7 day time frame and in fact observed statistically significant increases in the frequency of duplicate CT examinations after PACS using 30 day and 60 day time frames. Once again, although relative increases were 7.6% and 5.7%, respectively, the absolute increases were numerically small (absolute increase of 0.5% in each case).
There is a common belief that the introduction of electronic health information systems, such as PACS, will reduce the need for duplicate testing.[2, 3] For instance, the majority (58%) of PACS users, including clinicians and radiologists in Thames Valley, believe that the introduction of PACS at their institution was associated with a reduction in the number of unnecessary duplicate imaging tests (unpublished Canada Health Infoway survey data). However, empirical data supporting these beliefs have not been previously available. To our knowledge, this study is the first to report objective data regarding the impact of PACS on the frequency of duplicate testing.
Overall, we found that duplicate testing was a relatively infrequent phenomenon, with a maximum frequency of about 10% within 60 days of an index imaging examination. Since many of these duplicates would be expected to have been clinically indicated and since PACS is only hypothesized to reduce unnecessary duplicate examinations (e.g., due to unavailability of original images), it is not surprising that we only observed small absolute reductions in the frequency of duplicate X-ray examinations after the introduction of PACS. To put our findings into context, using the largest reduction observed in our study (0.9% absolute reduction in the frequency of duplicate X-ray examinations after PACS) and based on annual X-ray volumes in the Thames Valley institutions (approximately 70,000 OHIP claims for chest X-rays and abdominal X-rays in the 2005/2006 fiscal year) and an average fee per X-ray examination of approximately $30 (CAD), PACS might be expected to result in 630 fewer duplicate chest or abdominal X-ray examinations in the year after the introduction of PACS or a cost-savings to OHIP of $18,900 (CAD).
We observed an unexpected increase in the frequency of CT examinations after PACS using 30 day and 60 day time frames to identify duplicates. It is possible that improved image availability after PACS may have encouraged physicians to more frequently order repeat imaging tests to monitor response to treatment or to follow-up previously detected imaging abnormalities. It is also possible that, independent of PACS, the frequency of duplicate CT imaging was increasing during the observation period of this study, thus masking a true effect of PACS in reducing the frequency of redundant examinations. Indeed, previous analyses of OHIP data suggest that this may have occurred, with much more marked increases in the prevalence of repeat CT imaging over the past decade in Ontario (58% increase between 1996 and 2004) compared to plain chest x-ray examinations (6% increase between 1996 and 2004).
Reducing duplicate testing, however, is only one of many potential benefits of PACS. Previous survey- and interview-based evaluations of PACS have examined users' perceptions about PACS and have generally found high levels of user satisfaction and have reported perceptions of more efficient time utilization[6, 8, 10], reduced turn-around time from image acquisition to reporting[6, 8], and good value for money. These potential benefits are important from a health services perspective and warrant further evaluation. Indeed, the mismatch between perceived and measured benefits found in our study underscores the need for rigorous, independent evaluation of objective outcome measures to determine the benefits of new health information technologies.
This study has some limitations. First, a strength of our before-after study design was that each institution served as its own control. However, this design also assumes that the frequency of clinically warranted duplicate imaging was stable before and after PACS and that any observed change in the frequency of duplicate imaging was due to a reduction in unnecessary duplicates after PACS. It is possible that contextual increases in the frequency of duplicate testing over time may have masked true reductions in duplicate imaging due to PACS, as may have been the case for increases in duplicate CT imaging that we observed after PACS. Second, the OHIP database does not contain the reason why a given imaging test was ordered. Therefore, we could not discriminate between imaging tests repeated for clinical reasons versus those repeated because of image unavailability. More resource intensive evaluations using chart review or a prospective survey of physicians at the time of ordering could be used to address this limitation. However, since we found that baseline rates of duplicate testing are relatively low, it is unlikely that large absolute reductions in redundant imaging would be found using any methodology.
In conclusion, we found that the overall frequency of duplicate imaging examinations is relatively low and we did not find large absolute reductions in the frequency of duplicate imaging examinations after the introduction of PACS. Independent evaluation of electronic medical systems should be conducted to confirm widely held beliefs of their potential benefits.
Brenner DJ, Hall EJ: Computed tomography–an increasing source of radiation exposure. N Engl J Med. 2007, 357: 2277-2284. 10.1056/NEJMra072149.
Bryan S, Weatherburn GC, Watkins JR, Buxton MJ: The benefits of hospital-wide picture archiving and communication systems: a survey of clinical users of radiology services. Br J Radiol. 1999, 72: 469-478.
Bryan S, Weatherburn G, Buxton M, Watkins J, Keen J, Muris N: Evaluation of a hospital picture archiving and communication system. J Health Serv Res Policy. 1999, 4: 204-209.
Singh S, Gulati A, Harrison BD, Curtin JJ, Seaton D: Picture archiving and communications system (PACS): the benefits and problems of digital imaging in the NHS. Clin Med. 2007, 7: 202-203.
Bauman RA, Gell G: The reality of picture archiving and communication systems (PACS): a survey. J Digit Imaging. 2000, 13: 157-169.
Hayt DB, Alexander S: The pros and cons of implementing PACS and speech recognition systems. J Digit Imaging. 2001, 14: 149-157. 10.1007/s10278-001-0014-z.
Pilling JR: Picture archiving and communication systems: the users' view. Br J Radiol. 2003, 76: 519-524. 10.1259/bjr/67551353.
Reiner BI, Siegel EL, Hooper F, Protopapas Z: Impact of filmless imaging on the frequency of clinician review of radiology images. J Digit Imaging. 1998, 11: 149-150.
Watkins J: A hospital-wide picture archiving and communication system (PACS): the views of users and providers of the radiology service at Hammersmith Hospital. Eur J Radiol. 1999, 32: 106-112. 10.1016/S0720-048X(99)00134-5.
Chan L, Trambert M, Kywi A, Hartzman S: PACS in private practice–effect on profits and productivity. J Digit Imaging. 2002, 15: 131-136. 10.1007/s10278-002-5019-8.
Siegel EL, Reiner BI: Filmless radiology at the Baltimore VA Medical Center: a 9 year retrospective. Comput Med Imaging Graph. 2003, 27: 101-109. 10.1016/S0895-6111(02)00083-6.
Ontario Health Insurance Plan (OHIP) Schedule of Benefits and Fees. [http://www.health.gov.on.ca/english/providers/program/ohip/sob/physserv/d_radiol.pdf]
You JJ, Alter DA, Iron K, Slaughter PM, Kopp A, Przybysz R, Thiruchelvam D, Devore L, Laupacis A: Diagnostic Services in Ontario: Descriptive Analysis and Jurisdictional Review. 2007, Toronto: Institute for Clinical Evaluative Sciences
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/234/prepub
This study was funded by Canada Health Infoway and the Ontario Ministry of Health and Long-Term Care (MOHLTC). Dr. You is supported by an Ontario MOHLTC Career Scientist Award. Dr. Tu is supported by a Canada Research Chair in Health Services Research and a Career Investigator Award from the Heart and Stroke Foundation of Ontario. The Institute for Clinical Evaluative Sciences receives core operating funding from the Ontario MOHLTC. The Thames Valley Digital Imaging Network project was funded by Canada Health Infoway and the Ontario MOHLTC.
Canada Health Infoway and the Ontario MOHLTC worked with the authors in formulating the questions to be answered by the study and were provided with a draft copy of the manuscript for comments. They did not participate in the conduct of the study; in the collection, management, analysis, or interpretation of the data; or in the preparation or approval of the manuscript.
The authors declare that they have no competing interests.
JJY designed the study, interpreted the data and drafted the manuscript. LY performed the analyses. JVT participated in the design of the study, helped to interpret the data and revised the manuscript. All authors read and approved the final manuscript.