Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Impact of picture archiving communication systems on rates of duplicate imaging: a before-after study

BMC Health Services Research20088:234

DOI: 10.1186/1472-6963-8-234

Received: 22 May 2008

Accepted: 12 November 2008

Published: 12 November 2008

Abstract

Background

Electronic health information systems, such as picture archiving communication systems (PACS), are commonly believed to reduce the need for duplicate testing. However, empirical data to support this belief are not available.

Methods

Before-after study using administrative claims data from the Ontario Health Insurance Plan to determine whether the introduction of PACS at 10 hospitals in the Thames Valley region of southwestern Ontario, Canada between June 2004 and December 2005 reduced the frequency of duplicate imaging examinations. The imaging modalities studied were: chest and abdominal X-ray; computed tomography of the abdomen/pelvis, head, and chest. The frequency of duplicate testing was examined at 3 different time frames: 7 days, 30 days, and 60 days after a given index test.

Results

Overall frequencies of duplicate imaging were: 2.7% within 7 days of an index imaging test, 6.7% within 30 days, and 9.8% within 60 days. Comparing the 12 months before and 12 months after PACS, absolute reductions in the frequency of duplicate X-rays using 7-day, 30-day, and 60-day time frames were: 0.2% (P = 0.01), 0.6% (P < 0.001), and 0.9% (P < 0.001), respectively. In contrast, there were absolute increases in the frequency of duplicate CT scans after PACS of 0.0% (P = 0.92), 0.5% (P = 0.01), and 0.5% (P = 0.01), respectively.

Conclusion

The frequency of duplicate imaging is relatively low and we did not find large reductions in duplicate imaging after the introduction of PACS. Independent evaluation of electronic medical systems should be conducted to confirm widely held beliefs of their potential benefits.

Background

Picture archiving and communication systems (PACS) are computer networks dedicated to the storage, retrieval, and presentation of images produced by medical imaging devices. PACS replaces film archives and provides new capabilities such as the ability to access imaging information from various physical locations simultaneously and off-site viewing by clinicians and radiologists. PACS is commonly believed to reduce the number of unnecessary duplicate imaging tests ordered because original images are lost, misfiled, or stored at a remote location. If true, this would not only have important implications for decision-makers contemplating PACS implementation, but also for population health, given recent concerns about the potential risks of cancer from exposure to radiation during computed tomography scanning.[1]

Very few studies have evaluated the reduction in redundant imaging associated with PACS implementation. A survey of clinicians conducted at Hammersmith Hospital, United Kingdom revealed that 78% of respondents had a problem with unavailability of inpatient images before PACS versus only 14% after PACS. Forty percent of respondents indicated that they ordered one or more repeat examinations per month because of unavailable images before PACS versus only 12% after PACS.[2, 3] More recently, in a survey of British Thoracic Society members, 71% of respondents agreed that there were fewer lost images when using PACS. However, the survey did not ask directly about the impact of PACS on the ordering of redundant imaging tests.[4] Studies evaluating other aspects of PACS have again almost exclusively used survey- or interview-based methods to determine users' perceptions about PACS and have generally found very high levels of user satisfaction with many different aspects of PACS.[2, 3, 511] While these studies provide valuable insight into user perceptions, they rely on users' subjective and potentially biased interpretations of their experience with PACS. To our knowledge, few empirical data are available to support these perceptions. Accordingly, we conducted a before-after study using administrative health insurance claims data to determine whether the introduction of PACS was associated with a reduction in the frequency of duplicate imaging examinations.

Methods

Setting

The Thames Valley Digital Imaging Network project was an $18 million (Canadian dollars) [CAD]) initiative that implemented PACS at 10 hospitals in southwestern Ontario between June 2004 and December 2005. For this study, we considered PACS to have been fully implemented when a hospital officially began "filmless" operations. These dates, along with the annual imaging volumes of each hospital, are listed in Table 1.
Table 1

Thames Valley Imaging Network

Hospital

Annual number of X-ray examinations

Annual number of CT examinations*

PACS introduced (filmless date)

A†

-

-

Jun 1, 2004

B‡

  

Feb 1, 2005

C‡

43609

36016

Mar 1, 2005

D‡

  

Mar 1, 2005

E

5071

-

Jun 1, 2005

F

1297

-

Aug 1, 2005

G

4007

-

Aug 1, 2005

H

2919

-

Sep 1, 2005

I

8501

5773

Oct 1, 2005

J

6924

3599

Dec 1, 2005

Abbreviations: PACS, picture archiving communication systems. X-ray counts are for chest and abdomen; computed tomography counts are for abdomen/pelvis, head, and chest, and were obtained using Ontario Health Insurance Plan (OHIP) data from the 2005/2006 fiscal year (April 1, 2005 to March 31, 2006).

*Computed tomography services were not provided by all Thames Valley hospitals during the observation period of this study.

†Diagnostic imaging exams were not billed to OHIP during the observation period of this study. Therefore, this institution was excluded.

‡These three hospital sites are part of the same corporation and are identified with a single institution number in the OHIP database. For the purposes of this study, hospitals B-D were treated as a single institution with a filmless date of March 1, 2005.

Study design

We conducted a before-after study to examine changes in the frequency of repeat diagnostic imaging associated with the introduction of PACS in Thames Valley hospitals. We studied the following imaging modalities: chest X-ray, abdominal X-ray, computed tomography (CT) of the abdomen/pelvis, CT of the head, and CT of the chest. Spine X-ray, abdominal ultrasound, and magnetic resonance imaging were also considered for inclusion in this study, but were excluded since preliminary analyses revealed extremely low rates of repeat testing for these modalities (data not shown). This study received full approval from the Sunnybrook Health Sciences Centre Research Ethics Board.

Data sources

Diagnostic imaging services in Ontario are provided without patient user fees. Ontario radiologists receive fee-for-service payment from the Ministry's Ontario Health Insurance Plan (OHIP) for outpatient X-ray examinations, and for both inpatient and outpatient CT examinations. Imaging examinations performed at Hospital A (Table 1) were not billed to OHIP during the observation period of this study; therefore, this hospital was excluded. From the OHIP database, we identified claims paid for imaging examinations using the following fee codes: chest X-ray, X090, X091, X092; abdominal X-ray, X100, X101; CT abdomen/pelvis, X126, X409, X410, X231, X232, X233; CT head, X188, X400, X401, X402, X405, X408; and, CT chest, X125, X406, X407.[12] We used the institution number associated with each claim to identify imaging exams performed at Thames Valley institutions.

Outcome measure

We used the change in the frequency of duplicate imaging examinations after the introduction of PACS as our primary outcome measure. For the purposes of this study, a duplicate examination was defined as the occurrence of the same imaging examination (i.e., identical OHIP fee code) within a specified time frame after the index test. Since the OHIP database does not contain information about the clinical indications for diagnostic imaging tests, duplicate examinations–as defined in this study–include both unnecessary and clinically warranted duplicate examinations. We assumed that the frequency of clinically warranted duplicate examinations was stable before and after the introduction of PACS and that, as a result, any observed change in the frequency of duplicate testing after PACS was attributable to a change in the frequency of unnecessary duplicate examinations (e.g., those ordered because original images were lost, misfiled, or at a remote location).

Statistical analysis

For each institution and each imaging modality of interest, we identified the number of index imaging tests that occurred at that institution during the 12 months before and 12 months after the introduction of PACS. Next, we determined the number of index tests associated with a duplicate test at that institution within a specified time after the index test (7 days, 30 days, or 60 days). Duplicate tests were not eligible to be counted as index tests. The frequency of duplicate testing was expressed as the proportion of index tests associated with a duplicate test. For each imaging modality, we compared the frequency of duplicate testing in the 12 months before PACS (combined across all institutions) to the frequency of duplicate testing in the 12 months after PACS (combined across all institutions) using a chi-squared test. P values less than 0.05 were considered statistically significant.

Results

The frequencies of duplicate imaging are presented in Table 2. Overall, duplicate imaging, for X-ray and CT combined, was a relatively infrequent phenomenon with frequencies of 2.7% within 7 days of index imaging, 6.7% within 30 days, and 9.8% within 60 days.
Table 2

Frequency of within-institution duplicate imaging 12 months before and 12 months after introduction of PACS

 

Frequency of duplicate imaging

 
 

Before PACS

After PACS

P

Duplicate test within less than 7 days

   

   X-ray

   

Chest

1146/60598 (1.9)

1070/60366 (1.8)

0.12

Abdomen

339/8512(4.0)

273/8355 (3.3)

0.01

Total

1485/69110 (2.2)

1343/68721 (2.0)

0.01

   Computed tomography

   

Abdomen/Pelvis

275/14213 (1.9)

351/16060 (2.2)

0.13

Head

1099/16523 (6.7)

1180/18102 (6.5)

0.62

Chest

78/6627 (1.2)

120/8173 (1.5)

0.13

Total

1452/37363 (3.9)

1651/42335 (3.9)

0.92

Duplicate test within less than 30 days

   

   X-ray

   

Chest

3948/58212 (6.8)

3592/58204 (6.2)

<0.001

Abdomen

667/8248 (8.1)

595/8123 (7.3)

0.07

Total

4615/66460 (6.9)

4187/66327 (6.3)

<0.001

   Computed tomography

   

Abdomen/Pelvis

596/13969 (4.3)

786/15764 (5.0)

0.003

Head

1640/16364 (10.0)

1856/17868 (10.4)

0.26

Chest

200/6525 (3.1)

309/8019 (3.9)

0.01

Total

2436/36858 (6.6)

2951/41651 (7.1)

0.01

Duplicate test within less than 60 days

   

   X-ray

   

Chest

6032/56841 (10.6)

5519/56902 (9.7)

<0.001

Abdomen

857/8143 (10.5)

789/8012 (9.9)

0.16

Total

6889/64984 (10.6)

6308/64914 (9.7)

<0.001

   Computed tomography

   

Abdomen/Pelvis

968/13679 (7.1)

1174/15461 (7.6)

0.09

Head

1883/16257 (11.6)

2157/17736 (12.2)

0.10

Chest

341/6405 (5.3)

498/7857 (6.3)

0.01

Total

3192/36341 (8.8)

3829/41054 (9.3)

0.01

Abbreviations: PACS, picture archiving communication systems.

All numbers reported as no. of index tests associated with a duplicate test/total no. of index tests (%).

All P values are for difference in frequency of duplicate imaging before versus after PACS.

For X-ray examinations, we observed statistically significant reductions in the frequency of duplicate testing after PACS using 7 day, 30 day, and 60 day time frames to identify duplicate tests. Although relative reductions in duplicate testing were 9.1%, 8.7%, and 8.5%, respectively, these translated into numerically small absolute reductions in duplicate testing of 0.2%, 0.6%, and 0.9%, respectively.

For CT, we did not observe any reduction in duplicate testing after PACS using a 7 day time frame and in fact observed statistically significant increases in the frequency of duplicate CT examinations after PACS using 30 day and 60 day time frames. Once again, although relative increases were 7.6% and 5.7%, respectively, the absolute increases were numerically small (absolute increase of 0.5% in each case).

Discussion

There is a common belief that the introduction of electronic health information systems, such as PACS, will reduce the need for duplicate testing.[2, 3] For instance, the majority (58%) of PACS users, including clinicians and radiologists in Thames Valley, believe that the introduction of PACS at their institution was associated with a reduction in the number of unnecessary duplicate imaging tests (unpublished Canada Health Infoway survey data). However, empirical data supporting these beliefs have not been previously available. To our knowledge, this study is the first to report objective data regarding the impact of PACS on the frequency of duplicate testing.

Overall, we found that duplicate testing was a relatively infrequent phenomenon, with a maximum frequency of about 10% within 60 days of an index imaging examination. Since many of these duplicates would be expected to have been clinically indicated and since PACS is only hypothesized to reduce unnecessary duplicate examinations (e.g., due to unavailability of original images), it is not surprising that we only observed small absolute reductions in the frequency of duplicate X-ray examinations after the introduction of PACS. To put our findings into context, using the largest reduction observed in our study (0.9% absolute reduction in the frequency of duplicate X-ray examinations after PACS) and based on annual X-ray volumes in the Thames Valley institutions (approximately 70,000 OHIP claims for chest X-rays and abdominal X-rays in the 2005/2006 fiscal year) and an average fee per X-ray examination of approximately $30 (CAD),[12] PACS might be expected to result in 630 fewer duplicate chest or abdominal X-ray examinations in the year after the introduction of PACS or a cost-savings to OHIP of $18,900 (CAD).

We observed an unexpected increase in the frequency of CT examinations after PACS using 30 day and 60 day time frames to identify duplicates. It is possible that improved image availability after PACS may have encouraged physicians to more frequently order repeat imaging tests to monitor response to treatment or to follow-up previously detected imaging abnormalities. It is also possible that, independent of PACS, the frequency of duplicate CT imaging was increasing during the observation period of this study, thus masking a true effect of PACS in reducing the frequency of redundant examinations. Indeed, previous analyses of OHIP data suggest that this may have occurred, with much more marked increases in the prevalence of repeat CT imaging over the past decade in Ontario (58% increase between 1996 and 2004) compared to plain chest x-ray examinations (6% increase between 1996 and 2004).[13]

Reducing duplicate testing, however, is only one of many potential benefits of PACS. Previous survey- and interview-based evaluations of PACS have examined users' perceptions about PACS and have generally found high levels of user satisfaction and have reported perceptions of more efficient time utilization[6, 8, 10], reduced turn-around time from image acquisition to reporting[6, 8], and good value for money[5]. These potential benefits are important from a health services perspective and warrant further evaluation. Indeed, the mismatch between perceived and measured benefits found in our study underscores the need for rigorous, independent evaluation of objective outcome measures to determine the benefits of new health information technologies.

This study has some limitations. First, a strength of our before-after study design was that each institution served as its own control. However, this design also assumes that the frequency of clinically warranted duplicate imaging was stable before and after PACS and that any observed change in the frequency of duplicate imaging was due to a reduction in unnecessary duplicates after PACS. It is possible that contextual increases in the frequency of duplicate testing over time may have masked true reductions in duplicate imaging due to PACS, as may have been the case for increases in duplicate CT imaging that we observed after PACS. Second, the OHIP database does not contain the reason why a given imaging test was ordered. Therefore, we could not discriminate between imaging tests repeated for clinical reasons versus those repeated because of image unavailability. More resource intensive evaluations using chart review or a prospective survey of physicians at the time of ordering could be used to address this limitation. However, since we found that baseline rates of duplicate testing are relatively low, it is unlikely that large absolute reductions in redundant imaging would be found using any methodology.

Conclusion

In conclusion, we found that the overall frequency of duplicate imaging examinations is relatively low and we did not find large absolute reductions in the frequency of duplicate imaging examinations after the introduction of PACS. Independent evaluation of electronic medical systems should be conducted to confirm widely held beliefs of their potential benefits.

Declarations

Acknowledgements

This study was funded by Canada Health Infoway and the Ontario Ministry of Health and Long-Term Care (MOHLTC). Dr. You is supported by an Ontario MOHLTC Career Scientist Award. Dr. Tu is supported by a Canada Research Chair in Health Services Research and a Career Investigator Award from the Heart and Stroke Foundation of Ontario. The Institute for Clinical Evaluative Sciences receives core operating funding from the Ontario MOHLTC. The Thames Valley Digital Imaging Network project was funded by Canada Health Infoway and the Ontario MOHLTC.

Canada Health Infoway and the Ontario MOHLTC worked with the authors in formulating the questions to be answered by the study and were provided with a draft copy of the manuscript for comments. They did not participate in the conduct of the study; in the collection, management, analysis, or interpretation of the data; or in the preparation or approval of the manuscript.

Authors’ Affiliations

(1)
Department of Medicine, McMaster University
(2)
Institute for Clinical Evaluative Sciences

References

  1. Brenner DJ, Hall EJ: Computed tomography–an increasing source of radiation exposure. N Engl J Med. 2007, 357: 2277-2284. 10.1056/NEJMra072149.View ArticlePubMedGoogle Scholar
  2. Bryan S, Weatherburn GC, Watkins JR, Buxton MJ: The benefits of hospital-wide picture archiving and communication systems: a survey of clinical users of radiology services. Br J Radiol. 1999, 72: 469-478.View ArticlePubMedGoogle Scholar
  3. Bryan S, Weatherburn G, Buxton M, Watkins J, Keen J, Muris N: Evaluation of a hospital picture archiving and communication system. J Health Serv Res Policy. 1999, 4: 204-209.PubMedGoogle Scholar
  4. Singh S, Gulati A, Harrison BD, Curtin JJ, Seaton D: Picture archiving and communications system (PACS): the benefits and problems of digital imaging in the NHS. Clin Med. 2007, 7: 202-203.View ArticleGoogle Scholar
  5. Bauman RA, Gell G: The reality of picture archiving and communication systems (PACS): a survey. J Digit Imaging. 2000, 13: 157-169.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Hayt DB, Alexander S: The pros and cons of implementing PACS and speech recognition systems. J Digit Imaging. 2001, 14: 149-157. 10.1007/s10278-001-0014-z.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Pilling JR: Picture archiving and communication systems: the users' view. Br J Radiol. 2003, 76: 519-524. 10.1259/bjr/67551353.View ArticlePubMedGoogle Scholar
  8. Reiner BI, Siegel EL, Hooper F, Protopapas Z: Impact of filmless imaging on the frequency of clinician review of radiology images. J Digit Imaging. 1998, 11: 149-150.View ArticlePubMedPubMed CentralGoogle Scholar
  9. Watkins J: A hospital-wide picture archiving and communication system (PACS): the views of users and providers of the radiology service at Hammersmith Hospital. Eur J Radiol. 1999, 32: 106-112. 10.1016/S0720-048X(99)00134-5.View ArticlePubMedGoogle Scholar
  10. Chan L, Trambert M, Kywi A, Hartzman S: PACS in private practice–effect on profits and productivity. J Digit Imaging. 2002, 15: 131-136. 10.1007/s10278-002-5019-8.View ArticlePubMedGoogle Scholar
  11. Siegel EL, Reiner BI: Filmless radiology at the Baltimore VA Medical Center: a 9 year retrospective. Comput Med Imaging Graph. 2003, 27: 101-109. 10.1016/S0895-6111(02)00083-6.View ArticlePubMedGoogle Scholar
  12. Ontario Health Insurance Plan (OHIP) Schedule of Benefits and Fees. [http://www.health.gov.on.ca/english/providers/program/ohip/sob/physserv/d_radiol.pdf]
  13. You JJ, Alter DA, Iron K, Slaughter PM, Kopp A, Przybysz R, Thiruchelvam D, Devore L, Laupacis A: Diagnostic Services in Ontario: Descriptive Analysis and Jurisdictional Review. 2007, Toronto: Institute for Clinical Evaluative SciencesGoogle Scholar
  14. Pre-publication history

    1. The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/8/234/prepub

Copyright

© You et al; licensee BioMed Central Ltd. 2008

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Advertisement