Clinical decision support must be useful, functional is not enough: a qualitative study of computer-based clinical decision support in primary care
© Kortteisto et al.; licensee BioMed Central Ltd. 2012
Received: 13 December 2011
Accepted: 5 October 2012
Published: 8 October 2012
Health information technology, particularly electronic decision support systems, can reduce the existing gap between evidence-based knowledge and health care practice but professionals have to accept and use this information. Evidence is scant on which features influence the use of computer-based clinical decision support (eCDS) in primary care and how different professional groups experience it. Our aim was to describe specific reasons for using or not using eCDS among primary care professionals.
The setting was a Finnish primary health care organization with 48 professionals receiving patient-specific guidance at the point of care. Multiple data (focus groups, questionnaire and spontaneous feedback) were analyzed using deductive content analysis and descriptive statistics.
The content of the guidance is a significant feature of the primary care professional’s intention to use eCDS. The decisive reason for using or not using the eCDS is its perceived usefulness. Functional characteristics such as speed and ease of use are important but alone these are not enough. Specific information technology, professional, patient and environment features can help or hinder the use.
Primary care professionals have to perceive eCDS guidance useful for their work before they use it.
Health information technology (HIT), particularly electronic decision support systems [1, 2], can reduce the existing gap between evidence-based knowledge and health care practice [3, 4]. Benefits of computer-based clinical decision support (eCDS) to the quality of care and patient safety depend on the health care professionals’ acceptance and use of the eCDS in their decision making . Automatic provision of eCDS in the workflow was found to be an independent predictor of improved clinical practice . This means that eCDS guidance is provided automatically via electronic patient records (EPR) without extra effort by the professional . Speed, real time delivery, and fitting in the workflow are important features for effective eCDS . Too much guidance, e.g. drug interaction alerts, can be intrusive [9, 10]. There is little evidence on eCDS in primary care or in settings where a variety of clinical areas need to be covered [11, 12].
Health care professionals prefer patient-specific and relevant guidance, provided in a way that does not interfere with care or require inordinate effort and time [13, 14]. In practice, plenty of factors influence the use of guidance . Characteristics of the patient and the environment may be the most important factors for the acceptance of eCDS guidance . Many barriers have been identified [17, 18], particularly, too frequent or false alarms, lack of co-ordination between nurses and physicians, poor interface usability, time pressures, and inadequate training [19–22]. Facilitators include limiting the number of reminders, their ease of use and utility, documenting system problems and receiving feedback [21, 23].
In Finland, electronic patient records and national guideline databases have long been used, which makes piloting a new HIT in primary care environment feasible . Despite this, our implementation study of 18 months indicated that actual use of eCDS in the primary care context was only modest . Therefore, we examine in this paper the professionals’ feedback more thoroughly. The main purpose was to increase our understanding for improving the application of the eCDS in the workflow of physicians, nurses, and other professionals . The aim was to assess and describe in depth the specific reasons for using or not using the eCDS in primary care.
Computer-based clinical decision support (eCDS) functions (drug alerts, reminders, guideline links, and virtual health check = VHC) and procedures: examples
In the prescription procedure, the eCDS system checks against the patient medication list whether the drug selected has an interaction with other medication and, if so, displays a pop-up with an interaction alert and a suggestion of drug change.
Warfarin, in the patient’s medication list, has an interaction with the medicine chosen (ibuprofen).
Do you want to choose another medicine?
Patient-specific reminders are elicited by three trigger events in the EPR: 1) opening a patient record, 2) recording a new diagnosis, and 3) prescribing new medication.
Short version: Hypertension – check blood pressure?
The short version appears automatically, and the long version is shown when the mouse cursor hovers over the reminder.
Long version: This patient has hypertension. The latest BP measurement is more than a year old. Regular follow-up has been shown to improve the control of hypertension.
Guideline links based on patient diagnosis-list and ICD-10 codes are shown when the user clicks the diagnosis.
DG Hypertensio essentialis (primaria) I10
DG Fibrillatio atriorum I48
DG Angina pectoris I20
Virtual health check
The user can run a VHC on her/his patients in the appointment schedule by clicking on the decision support file in her/his folder. Patient-specific decision support messages appear on the screen when the user clicks on a specific link for a patient.
Name: Matti Maa (PICa): Control visit
Name: Anne Aamu (PICa): Acute visit
Name: Sanni Salo (PICa): Diabetes control
We targeted the 48 professionals of the health centre; 15 physicians, 24 nurses (ward nurses, general practice nurses, and public health nurses) and nine other professionals (physiotherapists, head nurses, and a psychologist). All signed an informed consent form. During the study, four physicians left and four physicians entered the health centre.
Data collection took place between July 2009 and February 2011, by means of focus group interviews, a questionnaire, and spontaneous feedback.
In January 2010, six months after introduction of the eCDS to the primary care professionals, we convened three focus groups with the help of the chief officers. We aimed at involving as many physicians as possible, with at least one representative from preventive care, nurse practice, physiotherapy, and the two inpatient wards. Six physicians, five nurses, and one physiotherapist participated in one of two profession-specific groups or in one multidisciplinary group. The agenda was planned in advance by the moderator (TK) and facilitator (MK) and tailored to each group . The aim was an open, informal discussion among participants about their experiences of the eCDS . Broad themes such as general ideas about the eCDS, experiences of the use, practical problems, advantages /disadvantages for work, barriers to use and facilitators, and development issues guided the discussion. Each group discussion was audio recorded, transcribed, and the facilitator took notes on the general atmosphere and various pertinent issues .
In February 2011, at the end of the implementation period, a fourth focus group was convened for physicians. At this point, we wanted to gather opinions and experiences on the eCDS functionalities that were only available to physicians; 13 of the 15 physicians participated. The agenda was again planned ahead, using the previous broad themes, adding content and thresholds of reminders, by the moderator (JK) and facilitator (TK). JK acted as moderator, since his medical background was expected to facilitate discussion . The aim was to obtain authentic feedback on the eCDS in physicians’ work. The group discussion was audio recorded and notes were made in a memo by a secretary.
A questionnaire was sent to professionals in September 2010. After one reminder, 28 of the 45 professionals (62%) had responded; nine physicians, twelve nurses, two public health nurses, and five others. The six structured questions reported in this paper dealt with the system’s capacity and quality, as well as its perceived usefulness and ease of use, according to the technology acceptance model (TAM) . It hypothesizes that perceived usefulness and ease of use are the main determinants of behavioural intention to use IT, which is expected to lead to actual use . In this study, the perceived usefulness indicates the professional’s subjective probability for increasing her or his work performance by using the eCDS system. The perceived ease of use indicates the professional’s feelings for new system to be free of effort. In addition, reasons for using or not using the eCDS functions were queried in an open question: twelve out of 28 respondents (43%) elaborated their reasons.
The professionals were encouraged from the beginning to provide feedback via personal email, whenever they recognized specific issues or questions for system developers or researchers. The vendor added a feedback channel within the EPR-system from January 2010. Twelve spontaneous feedback messages were submitted by physicians; two via the personal email and ten via the EPR.
A theory-based framework of analysis with emerging sub-themes – helping (+) or hindering (−), quotations are described in text
Content of computer-based decision support guidance
Usefulness for professional’s work (+)
Non-helpfulness for professional’s work (−)
Ease of use (+)
Too much, too small text (−)
General attitude (+)
Poor competence (−)
Reasons for visit (−)
Low thresholds for reminders (−)
Busy practice (−)
Swine-flu epidemic (−)
Development issues (+)
In the end, the qualitative and quantitative analyses were combined. The main categories were the content of the eCDS guidance, the functionality of the eCDS system, the features related to the professions involved, the features of the patient groups, and environmental factors. A unique category of development issues emerged.
The content of the eCDS guidance
Perceived functionality and usefulness of the computer-based clinical decision support – number of responses in the questionnaire
It is easy to use
It is rapid enough
It is reliable
It is of high quality
It supports my work
It influences my decisions
The physicians’ consensus was that the guideline links and VHC were not useful; the guideline texts were too long, and use of the VHC function would have necessitated more time for paper work. Only physiotherapists considered the guideline links useful.
" ‘Good system; it does not disturb, some reminders are useful.’ "
" ‘Interaction alerts with a prescription are useful.’ "
" ‘Drug alerts motivate to clear up medication lists, but it takes time.’ "
" ‘I think that there is too much text … too many links.’ "
" ‘Use of virtual health checks requires more time for paperwork.’ "
From the other professionals:
" ‘Guideline links are useful for the physiotherapist.’ "
" ‘For our work [health promotion] and customer relationships, there aren’t necessarily relevant things.’ "
" ‘Mainly drug interaction alerts, and I [nurse] do not prescribe.’ "
" ‘Okay, good information but it does not influence my work much in practice, how you use it depends on the group of professionals you belong to.’ "
" ‘Most of the drug alerts are not relevant for physiotherapists’ work.’ "
" ‘Reminders do not support psychologist’s work’ "
The functionality of the eCDS system
The majority of the physicians and other professionals found the ease of use and speed good, while most of the nurses expressed no opinion, stating that they could not answer these questions (Table 3). The physicians reported that specific features of the system (EPR, eCDS, or their integration) were hindering the use. These included known problems with the EPR system, reminders’ position on the screen, irritating drug interaction and contraindication alerts, and a lack of structured recording of smoking status (causing inappropriate reminders). The nurses experienced the eCDS as impractical for work at a busy call centre.
" ‘There was generally a problem with the functioning of the electronic patient record system … yes, big problems with the computer.’ "
" ‘Reminders’ position on the left side of the screen.’ "
" ‘If a patient has 20 reminders, I just go past them quickly.’ "
" ‘Reminders’ texts are sometimes too strict in the short version. If you don’t move the cursor over the text and see the whole reminder, the wording doesn’t work.’ "
" ‘Too low triggering threshold with drug interaction alerts. I never bother to read them.’ "
" ‘Irritating alerts.’ "
" ‘Excess alerts – e.g., asthma and opiate, warfarin and paracetamol.’ "
" ‘The reminder “great cardiovascular risk [SCORE], clarify the patient’s smoking status…”. It is not possible to record smoking status in a structured manner in the EPR.’ "
From the nurses:
" ‘We [general-practice nurses] receive such a staggering number of phone calls that we really don’t look for anything, just do exactly what they [patients] call about.’ "
The features related to professionals
The health care professionals’ motivation to learn to use a new IT system in practice seemed to vary. Some nurses discussed inadequate skills or interest in using IT in general. Primary care professionals seemed to have generally positive attitudes toward future use of eCDS, but more training and time to learn to use a new system were described as necessary. In addition, it was suggested that a local opinion leader could be useful to teach others.
" ‘I think that we should use it more.’ "
" ‘Do not need. I use Terveysportti [national health portal for professionals] .’ "
" ‘I get an exact search when I use Terveysportti.’ "
" ‘Should there be an enthusiast or someone who uses it and thinks about it, and then tells us what the tricks are, what is good?’ "
From the nurses:
" ‘We have certainly more potential than we have been used to … Yes, it is absolutely a good thing … Yeah, I would say the same, learn to use it and intensify its use then’ "
" ‘All experienced this training as necessary.’ "
" ‘The individual differences, where you are used to searching for information… e.g., poor IT skills or no interest in the computer.’ "
Features of the patient groups
" ‘The patients usually have a reason for their visit. This has to be the primary focus for the physician. If there are ten VHC reminders even before the patient arrives, there is in no way enough time.’ "
" ‘My patient had a fasting plasma glucose value of 6.0 once, previously normal. Decision support suggests diabetes. To my mind, a single FPG value of 6.0 does not justify diagnosing diabetes.’ "
" ‘The swine-flu epidemic has consumed an awful lot [of time] , we didn’t even have any general-nurse practice appointments available for a long time.’ "
" ‘When I am busy, I don’t look for anything really.’ "
" ‘Nothing more than simply doing what I have to do.’ "
" ‘In that situation, I could imagine that once you get acquainted with one [specific disease] , it will be more useful than it is now.’ "
" ‘Make the system more visual; nice and desirable like an iPad… to give the user the feeling “Oh, what else could I look for? I want more of this.”’ "
" ‘If it were possible, a visual sign like when Microsoft™ e-mail shows pop-ups for new mail.’ "
" ‘A transient box, in the middle enough, for a short enough time, and automatically [closing]. If there would be a movement, then it would be noticed. […] it should be neutral on the screen and definitely should not disturb.’ "
" ‘Request for a new interaction alert: enoxaparin or dalteparin prescription when a patient has warfarin medication. This alert reminds physician to check the appropriateness of continued simultaneous use.’ "
" ‘Patient-tailored threshold values, different from guideline-based’ "
" ‘In the virtual health check retrospective testing, one slim patient got an obesity warning, because of a flawed weight or height recording. It was related to a missing digit, a recording of the height as 52 cm instead of the correct 152 cm. These types of typos should be guarded against by the system.’ "
The main findings of our study are twofold: perceived usefulness results in professionals using eCDS guidance while perceived non-helpfulness leads to non-use in primary health care. The guidance has to be designed for each profession and tailored, so that also e.g. nurses and physiotherapists can find it relevant for their practice. The profession groups have their own duties and specific practices; therefore their information needs vary greatly [37–40]. Our study indicates that even within one profession there are differences in the perceived usefulness of eCDS guidance. Lugtenberg et al.  reported similarly that perceived barriers for Dutch general practitioners varied greatly with the set of guideline recommendations involved.
Our results are in accordance with Davis’s technology acceptance theory , wherein perceived usefulness is considered a determinant with the main determinants of attitude and behavioural intention to use IT. Our results are strengthened by our use of multiple study methods: both open and structured questions, as well as focus group discussions with participants describing their specific reasons for using or not using the eCDS system or guidance. According to McDermott , perceived usefulness is a key characteristic in acceptance of a computer-based intervention among physicians. We discovered in addition that, while the majority of the physicians perceived the eCDS guidance to be useful for their work, the opposite was true for nurses and others, who sometimes appeared unable even to answer the question.
It is interesting that perceived usefulness showed in the data more consistently than perceived ease of use. It may be that the eCDS system basically was easy to use, since it had been integrated to the EPR system and the guidance popped on screen without specific effort, indicating perceived ease of use in TAM model . Seven of nine physicians replied ‘yes’ to the question of easy to use, and the one negative answer came from a nurse (Table 3). It is conceivable that perceived ease of use, together with the main determinants of attitude and behavioural intention to use, was influencing in the background to the professionals’ perceived usefulness of the eCDS as TAM suggests .
The functionality of the eCDS system seemed good. Most responded positively to the questions dealing with this, with only one physician and nurse answering in the negative. However, the majority of the nurses found themselves unable to answer these functionality questions, which seemingly confirms the failure of the implementation among nurses . In general, the professionals did not report any eCDS-system-based problem encumbering their work. By contrast, they seemed satisfied, because the eCDS system did not disturb them or force them to do anything. Indeed, this has been cited elsewhere as a key feature of a successful eCDS system . According to the physicians, a more visually oriented system might be more desirable to them.
The results reaffirm that plenty of barriers exist to the use of reminders, even automatic ones, in primary care [17, 19–22]. These were related to the studied eCDS system, inaccurate data in the EPR or code mismatch between the EPR and the eCDS system, the usability of the EPR itself, the groups of professionals, and patient preferences. The list bears a remarkable resemblance to the list of hurdles for use of clinical information systems as well as clinical guidelines [15, 42]. One exception was seen: lack of agreement with the recommendations was evaluated as the most common perceived barrier to use of guideline recommendations among Dutch physicians , while this was not brought up as a barrier in our study. On the contrary, all respondents reported reliance on the reliability of the eCDS guidance.
The swine-flu epidemic and full-scale vaccination effort caused clear and measurable changes, especially in the nurses’ practice, for several months. It probably had a negative influence on the eCDS implementation and use, as busy practice was a common environmental reason for non-use of the eCDS. A recent usability evaluation study of a CDS tool for osteoporosis disease management  also indicated that physicians’ lack of time was a major challenge to point-of-care use of the tool.
A facilitating factor seems to be a generally positive attitude toward eCDS among primary care professionals. They did report needing more training and more time to learn to use the eCDS system. In particular, nurses expected some changes to their general nursing practice before actual implementation of the eCDS. Indeed, good training has been seen to be associated with favourable assessment of IT among nurses .
Strengths and limitations
Generalization of qualitative results can be problematic by the standards of the positivist approaches typical of quantitative research, in view of issues such as the representativeness of the sample . Generalization can, however, be approached from a theoretical standpoint that takes into account the nature of the study question and the framework. So we can postulate that our results strengthen existing knowledge of the meaningful elements behind eCDS use among health care professionals. By using multiple methods and a long time period, we were able to extend our understanding of primary care realities during the implementation phase . We believe that these results have validity for application in other primary care settings where a new automatic eCDS system integrated with EPR is being introduced.
Malterud  raises another issue, surrounding the researchers’ commitment to reflexivity. Our preconceptions stemming from previous studies [47, 48] directed our initial choices and influenced the study process. We therefore chose to use deductive methods in the qualitative analyses and to describe the process as explicitly as possible. Four of us have a background in medicine and one in physiotherapy, which might aid us in understanding the different perspectives of the professionals. Two of us (JK and IK) also participated in development of the target eCDS system, which could have affected the study. From the beginning of the eCDS development, a study project was in progress to describe and evaluate what was going on in practice when the eCDS was piloted [49, 50] and implemented . The same choice has been made in many previous studies that involve IT developers as researchers [5, 51].
A limitation is our use of a non-standardized questionnaire. However, the questions reported here were planned based on the theory  and the evidence of applied research [52, 53]. Limited sample size and potential selection bias in responses due to participated versus non-participated professionals decrease a generalization of the results.
Perceived usefulness seems to be decisive for the use of eCDS guidance in primary care practice, and therefore the content of the eCDS is a critical issue. Information needs of profession groups in various environments (e.g., preventive care) have to be determined, and the eCDS guidance tailored according to those needs. Functionality-related characteristics such as speed and ease of use of the eCDS system and reliability of guidance are important but not sufficient in relation to uptake by primary care professionals.
Although this study targeted multi-profession groups in primary care, the studied eCDS system had been essentially based on medical evidence and to aid in physicians’ work. In this respect, there are several issues for future studies, e.g. targeting to patient’s reason for visit and tailoring with patient-specific threshold values. Unless future research considers in depth all professions’ needs, experiences and opinions the full potential of eCDS services may not become a reality.
This study was funded by the Finnish Funding Agency for Technology and Innovation (Tekes), the National Institute for Health and Welfare (THL), Duodecim Medical Publications Ltd, ProWellness Ltd., and Doctoral Programs in Public Health (DPPH). We are grateful to the staff at the Sipoo health centre who gave their time to participate in the study process. The authors would like to thank other members of the EBMeDS study group for their help.
- Car J, Black A, Anandan C, Cresswell K, Pagliari C, McKinstry B, Procter R, Majeed A, Sheikh A: The impact of eHealth on the quality & safety of health care:a systematic overview & synthesis of the literature. 2008, Report for the NHS Connecting for Health Evaluation Programme. London: Imperial College LondonGoogle Scholar
- Lau F, Kuziemsky C, Price M, Gardner J: A review on systematic reviews of health information system studies. J Am Med Inform Assoc. 2010, 17 (6): 637-645.View ArticlePubMedPubMed CentralGoogle Scholar
- Grol R, Grimshaw J: From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003, 362 (9391): 1225-1230.View ArticlePubMedGoogle Scholar
- McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348 (26): 2635-2645.View ArticlePubMedGoogle Scholar
- Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG: Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006, 144 (10): 742-752.View ArticlePubMedGoogle Scholar
- Kawamoto K, Houlihan CA, Balas EA, Lobach DF: Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005, 330 (7494): 765.View ArticlePubMedPubMed CentralGoogle Scholar
- Varonen H, Kaila M, Kunnamo I, Komulainen J, Mantyranta T: [Computerised decision support: the way towards advisory patient record]. Duodecim. 2006, 122 (10): 1174-1181.PubMedGoogle Scholar
- Bates DW, Gawande AA: Improving safety with information technology. N Engl J Med. 2003, 348 (25): 2526-2534.View ArticlePubMedGoogle Scholar
- Ahearn MD, Kerr SJ: General practitioners’ perceptions of the pharmaceutical decision-support tools in their prescribing software. Med J Aust. 2003, 179 (1): 34-37.PubMedGoogle Scholar
- Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS: Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003, 163 (21): 2625-2631.View ArticlePubMedGoogle Scholar
- Bryan C, Boren SA: The use and effectiveness of electronic clinical decision support tools in the ambulatory/primary care setting: a systematic review of the literature. Inform Prim Care. 2008, 16 (2): 79-91.PubMedGoogle Scholar
- Gosling AS, Westbrook JI, Spencer R: Nurses’ use of online clinical evidence. J Adv Nurs. 2004, 47 (2): 201-211.View ArticlePubMedGoogle Scholar
- Greenes RA: Clinical decision support: The Road Ahead. 2007, Boston, MA: ElsevierGoogle Scholar
- McDermott L, Yardley L, Little P, Ashworth M, Gulliford M: Developing a computer delivered, theory based intervention for guideline implementation in general practice. BMC Fam Pract. 2010, 11: 90.View ArticlePubMedPubMed CentralGoogle Scholar
- Francke AL, Smit MC, de Veer AJ, Mistiaen P: Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Med Inform Decis Mak. 2008, 8: 38.View ArticlePubMedPubMed CentralGoogle Scholar
- Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL: A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Inform Decis Mak. 2006, 6: 6.View ArticlePubMedPubMed CentralGoogle Scholar
- Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M: Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003, 326 (7384): 314.View ArticlePubMedPubMed CentralGoogle Scholar
- Zheng K, Padman R, Johnson MP, Diamond HS: Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system. Int J Med Inform. 2005, 74 (7–8): 535-543.View ArticlePubMedGoogle Scholar
- Patterson ES, Doebbeling BN, Fung CH, Militello L, Anders S, Asch SM: Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods. J Biomed Inform. 2005, 38 (3): 189-199.View ArticlePubMedGoogle Scholar
- Patterson ES, Nguyen AD, Halloran JP, Asch SM: Human factors barriers to the effective use of ten HIV clinical reminders. J Am Med Inform Assoc. 2004, 11 (1): 50-59.View ArticlePubMedPubMed CentralGoogle Scholar
- Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM: Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc. 2005, 12 (4): 438-447.View ArticlePubMedPubMed CentralGoogle Scholar
- Short D, Frischer M, Bashford J: Barriers to the adoption of computerised decision support systems in general practice consultations: a qualitative study of GPs’ perspectives. Int J Med Inform. 2004, 73 (4): 357-362.View ArticlePubMedGoogle Scholar
- Fung CH, Woods JN, Asch SM, Glassman P, Doebbeling BN: Variation in implementation and use of computerized clinical reminders in an integrated healthcare system. Am J Manag Care. 2004, 10 (11 Pt 2): 878-885.PubMedGoogle Scholar
- Hamalainen P, Reponen J, Winblad I: eHealth of Finland: check point 2006. Stakes reports. 2007, Helsinki, Finland: Valopaino OyGoogle Scholar
- Kortteisto T, Komulainen J, Kunnamo I, Mäkelä M, Kaila M: Implementing clinical decision support for primary care professionals – the process. Finnish Journal of eHealth and eWelfare. 2012, 4 (3): 153-164.Google Scholar
- Fung CH, Tsai JS, Lulejian A, Glassman P, Patterson E, Doebbeling BN, Asch SM: An evaluation of the Veterans Health Administration’s clinical reminders system: a national survey of generalists. J Gen Intern Med. 2008, 23 (4): 392-398.View ArticlePubMedPubMed CentralGoogle Scholar
- EBMeDS context-sensitive guidance at the point of care. http://www.ebmeds.org/ebmeds/ebmeds_home.asp.
- Steward DW, Shamdasani PN: Focus groups: Theory and Practice. 1990, London: Sage Publications, Volume 20Google Scholar
- Kitzinger J: Qualitative research: introducing focus groups. BMJ. 1995, 311 (7000): 299-302.View ArticlePubMedPubMed CentralGoogle Scholar
- Barbour RS: Doing Focus Groups. 2007, London: SageView ArticleGoogle Scholar
- Sim J: Collecting and analysing qualitative data: issues raised by the focus group. J Adv Nurs. 1998, 28 (2): 345-352.View ArticlePubMedGoogle Scholar
- Davis FD: Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Q. 1989, 13 (3): 319-340.View ArticleGoogle Scholar
- Lapointe L, Rivard S: Clinical information systems: understanding and preventing their premature demise. Healthc Q. 2005, 8 (2): 92-100.PubMedGoogle Scholar
- Elo S, Kyngas H: The qualitative content analysis process. J Adv Nurs. 2008, 62 (1): 107-115.View ArticlePubMedGoogle Scholar
- Fleuren M, Wiefferink K, Paulussen T: Determinants of innovation within health care organizations: literature review and Delphi study. Int J Qual Health Care. 2004, 16 (2): 107-123.View ArticlePubMedGoogle Scholar
- Silverman D: Doing Qualitative Research: A Practical Handbook. 2005, London: Sage Publications, 2Google Scholar
- Goossens A, Bossuyt PM, de Haan RJ: Physicians and nurses focus on different aspects of guidelines when deciding whether to adopt them: an application of conjoint analysis. Med Decis Making. 2008, 28 (1): 138-145.View ArticlePubMedGoogle Scholar
- Hayrinen K, Saranto K, Nykanen P: Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform. 2008, 77 (5): 291-304.View ArticlePubMedGoogle Scholar
- Kloda LA, Bartlett JC: Clinical information behavior of rehabilitation therapists: a review of the research on occupational therapists, physical therapists, and speech-language pathologists. J Med Libr Assoc. 2009, 97 (3): 194-202.View ArticlePubMedPubMed CentralGoogle Scholar
- Randell R, Dowding D: Organisational influences on nurses’ use of clinical decision support systems. Int J Med Inform. 2010, 79 (6): 412-421.View ArticlePubMedGoogle Scholar
- Lugtenberg M, Burgers JS, Besters CF, Han D, Westert GP: Perceived barriers to guideline adherence: A survey among general practitioners. BMC Fam Pract. 2011, 12: 98.View ArticlePubMedPubMed CentralGoogle Scholar
- Van Der Meijden MJ, Tange HJ, Troost J, Hasman A: Determinants of success of inpatient clinical information systems: a literature review. J Am Med Inform Assoc. 2003, 10 (3): 235-243.View ArticlePubMedPubMed CentralGoogle Scholar
- Lugtenberg M, Zegers-van Schaick JM, Westert GP, Burgers JS: Why don’t physicians adhere to guideline recommendations in practice? An analysis of barriers among Dutch general practitioners. Implement Sci. 2009, 4: 54.View ArticlePubMedPubMed CentralGoogle Scholar
- Kastner M, Lottridge D, Marquez C, Newton D, Straus SE: Usability evaluation of a clinical decision support tool for osteoporosis disease management. Implement Sci. 2010, 5: 96.View ArticlePubMedPubMed CentralGoogle Scholar
- Alpay L, Needham G, Murray P: The potential of information technology for nurses in primary care: a review of issues and trends. Prim Health Care Res Dev. 2000, 1: 5-13.View ArticleGoogle Scholar
- Malterud K: Qualitative research: standards, challenges, and guidelines. Lancet. 2001, 358 (9280): 483-488.View ArticlePubMedGoogle Scholar
- Kortteisto T, Kaila M, Komulainen J, Mantyranta T, Rissanen P: Healthcare professionals’ intentions to use clinical guidelines: a survey using the theory of planned behaviour. Implement Sci. 2010, 5: 51.View ArticlePubMedPubMed CentralGoogle Scholar
- Varonen H, Kortteisto T, Kaila M: What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians. Fam Pract. 2008, 25 (3): 162-167.View ArticlePubMedGoogle Scholar
- Korhonen M, Miettinen M: Information quality as data content and coherence in a regional information system for diabetes care. IADIS: 2008. 2008Google Scholar
- Miettinen M, Korhonen M: Information quality in healthcare: coherence of data compared between organization’s electronic patient records. 21st IEEE International Symposium on Computer-Based Medical Systems: 2008. 2008, Jyväskylä, Finland: IEEE Computer SocietyGoogle Scholar
- Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005, 293 (10): 1223-1238.View ArticlePubMedGoogle Scholar
- Gagnon MP, Godin G, Gagne C, Fortin JP, Lamothe L, Reinharz D, Cloutier A: An adaptation of the theory of interpersonal behaviour to the study of telemedicine adoption by physicians. Int J Med Inform. 2003, 71 (2–3): 103-115.View ArticlePubMedGoogle Scholar
- Raitoharju R: Information technology acceptance in the Finnish social and healthcare sector. Exploring the effects of cultural factors. 2007, Turku, Finland: School of Economics, A-4Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/12/349/prepub