Using computer decision support systems in NHS emergency and urgent care: ethnographic study using normalisation process theory
© Pope et al.; licensee BioMed Central Ltd. 2013
Received: 21 November 2012
Accepted: 11 March 2013
Published: 23 March 2013
Information and communication technologies (ICTs) are often proposed as ‘technological fixes’ for problems facing healthcare. They promise to deliver services more quickly and cheaply. Yet research on the implementation of ICTs reveals a litany of delays, compromises and failures. Case studies have established that these technologies are difficult to embed in everyday healthcare.
We undertook an ethnographic comparative analysis of a single computer decision support system in three different settings to understand the implementation and everyday use of this technology which is designed to deal with calls to emergency and urgent care services. We examined the deployment of this technology in an established 999 ambulance call-handling service, a new single point of access for urgent care and an established general practice out-of-hours service. We used Normalization Process Theory as a framework to enable systematic cross-case analysis.
Our data comprise nearly 500 hours of observation, interviews with 64 call-handlers, and stakeholders and documents about the technology and settings. The technology has been implemented and is used distinctively in each setting reflecting important differences between work and contexts. Using Normalisation Process Theory we show how the work (collective action) of implementing the system and maintaining its routine use was enabled by a range of actors who established coherence for the technology, secured buy-in (cognitive participation) and engaged in on-going appraisal and adjustment (reflexive monitoring).
Huge effort was expended and continues to be required to implement and keep this technology in use. This innovation must be understood both as a computer technology and as a set of practices related to that technology, kept in place by a network of actors in particular contexts. While technologies can be ‘made to work’ in different settings, successful implementation has been achieved, and will only be maintained, through the efforts of those involved in the specific settings and if the wider context continues to support the coherence, cognitive participation, and reflective monitoring processes that surround this collective action. Implementation is more than simply putting technologies in place – it requires new resources and considerable effort, perhaps on an on-going basis.
KeywordsComputer technology CDSS Urgent care Emergency care Normalisation process theory
Governments and corporations – both private and public – have placed great hope and expectation in the capacity of technological innovation in general, and information and communication technologies (ICTs) in particular, to play a significant role in overcoming the structural tension of rising costs and the need for fiscal restraint in the delivery of public healthcare services [1–4]. Digital ICTs have been enrolled in telemedicine to enable remote consultations [5, 6] and telemonitoring to allow patients to undertake routine testing and data transmission and enabling remote surveillance by clinicians or clerical workers [7, 8]. For all this enthusiasm, the significant gap between the high hopes and expectations of policy-makers and managers and the actual practice of implementation of these technologies has been highlighted by research which has consistently revealed an extensive catalogue of delays, compromise and failure [9–14]. It appears that technological interventions do not slip seamlessly into established practice but instead they meet with resistance and rejection because they threaten established work and organisational routines or clash with established professional identities and power relations [15–20] We also know that interventions that ‘work’ in one setting or speciality may not work elsewhere [21, 22].
Much research about health technologies has sought to understand why interventions do, or more frequently do not, get embedded in practice [10, 23, 24]. A great deal of research in this area has been characterised by an assumption that there is an ‘it’ (i.e. a technology) that is implemented – rather than seeing the ensemble of work practices and social relations that shape each other to bring a technology into use. This focus has helped to identify a variety of barriers and facilitators to successful implementation but less attention has been paid to wider strategic, political or workforce issues. In terms of research design, with a few notable exceptions [25, 26] this work has relied on case studies focussing on specific occupational groups (e.g. doctors or nurses) and their response to particular technological interventions. While this has been excellent at revealing the complex and particular processes that shape particular outcomes, it has not enabled systematic analyses of wider issues, especially the tricky problems associated with transferring a technology from one setting to another.
So, while policy pushes the case for new technologies to deliver healthcare improvement, the empirical literature continues to highlight an implementation gap. We ‘know’ that there are myriad barriers to success. This paper reports a project which explored how an innovative ICT was successfully brought into use by a range of different staff in different settings. Specifically we try to understand how a particular computer technology becomes normalised; how a technology becomes embedded in work practices and in different healthcare settings.
Computer decision support systems in healthcare
The focus of our study was a single technology, a computer decision support system (CDSS) deployed in different emergency and urgent healthcare settings. CDSS are computer programmes designed to assist with decision-making. CDSS typically combine an expert knowledge base with an algorithmic or inference based set of rules which guide or inform decisions. CDSS have become increasingly popular in healthcare, and have diffused rapidly across the NHS, for example to support GP prescribing e.g. ‘PRODIGY’,  and diagnosis and treatment decisions by doctors and nurses e.g. Odyssey ‘TAS’, . In clinical consultations these technologies are used to structure questioning, gather patient information and to synthesise these with clinical knowledge to support or inform diagnosis and treatment decisions. CDSS have been developed for use by non-clinical staff, notably in emergency medical care where they have been successfully used to help manage and prioritise the dispatch of ambulances. CDSS are promoted as helping to standardise and structure work, allowing staff to work more efficiently, safely, or faster, or enabling substitution or reallocation of tasks .
The CDSS of interest here is used to support the provision of a telephone-based service in which call-handlers assess and prioritise calls from patients (or their carers) to assign them to appropriate care provision. This CDSS is an expert system built on an extensive clinical evidence base subject to a continuous process of evidence review and update. A series of logical algorithms (pathways) underpin questions which a call-handler uses to ask the caller/patient to determine the clinical skills required to provide care, and the timeframe in which they must be accessed. The call-handlers work in a large open plan office (call centre) and sit at a desk facing two computer screens. Each has a headset to receive incoming telephone calls and works through questions on the screens, clicking on appropriate ‘answers’ and typing free text information as necessary. The particular CDSS studied offers some flexibility in how the questions are phrased, and prompts are suggested which allow the call-handlers to probe for details and work through the linear algorithm or ‘pathway’ and reach a ‘disposition’ - a recommended course of action, which can include immediate 999 ambulance request, referral to a range of different primary care services, or suggestions for self-care. Call-handlers typically work 10–12 hour shifts and cover the whole 24 hour period. In addition to the call handlers a number of other staff work in or near the call centre area (supervisors, clinical support staff and trainers). Each of the call centres in this study was located with or in a larger healthcare organisation (NHS Ambulance Trust or Primary Care service) and this meant that that senior managers and other health service professionals and staff worked nearby.
The literature on this type of healthcare call-handling work is necessarily limited as this is a relatively new mode of healthcare provision. To date published work in this field has mainly looked at nurse-led call-handling [30, 31] in part because the extension of this telephone-based service to a non-clinical workforce is so new. To our knowledge, the research reported here is the first comparative case study of a CDSS used by clinical and non-clinical call-handlers in different healthcare settings.
Of course, computer assisted call-handling has a longer history in settings outside healthcare, notably the finance and insurance sectors. Contemporary sociological critiques of this type of work have pressed the case for seeing call-handling in the context of globalisation of capitalism and labour substitution , Taylorised mass production and control  and in terms of power and resistance . While these analyses are powerful, and have highlighted aspects of expertise, performativity and normative constraint, we nonetheless felt that it was important to explore how digital ICTs were being enrolled in the everyday practice of healthcare work. In particular we sought an approach that would focus on the collective social action of call-handling and technology use and show how workforce relations were located within the ‘special’ context of health services and the particular social norms that frame and constrain this sphere of action.
The CDSS we studied was designed to allow non-clinical staff (as well as clinical staff), to make comprehensive clinical assessments of telephone callers to emergency and urgent care services. We carried out a cross-case comparative study this new technology in three different healthcare settings: ‘999’ emergency care, a GP out of hours (‘OOH’) urgent care service, and a new ‘single point of access’ (‘SPA’) service for urgent and unscheduled care.
We drew on Normalization Process Theory, or NPT [35, 36] to provide a theoretical lens for our empirical and methodological analyses. NPT is concerned with ‘how and why things become, or don’t become, routine and normal components of everyday work’ (36 p 535) and it defines four core mechanisms that shape the social processes of implementation, embedding and integration of ensembles of social practices. These dynamic and interrelated domains are ‘coherence’ – the extent to which an intervention is understood as meaningful, achievable and desirable; ‘cognitive participation’ – the enrolment of those actors necessary to deliver the intervention; collective action – the work that brings the intervention into use; and ‘reflexive monitoring’ – the on-going process of adjusting the intervention to keep it in place. We used these domains to provide a framework for our findings in each of the three case studies and as a way of integrating our findings across the three cases.
The choice of NPT was a deliberate one. Our research team included social scientists from sociological, psychological and health services research backgrounds and we sought a robust theoretical framework for our analysis which could be practically and appropriately applied to the comparative study of the CDSS in the chosen settings. We were familiar with a number of theoretical approaches which might be loosely collected under the heading ‘Science and Technology Studies’ (STS) which had grown out of sociological research and parallel Sociotechnical network approaches which drew on social informatics and computer science. Both of these perspectives attempted to go beyond a naive technological determinism and argued instead that technologies such as ICTs should be understood as socially shaped and comprised of dynamic interrelationships between people, artefacts, practices and structures. Reviewing these fields we considered using Actor Network Theory [37, 38] but discounted this because we felt that while this helpfully alerted us to the importance of networks of human and non-human actors it focussed more on the performative aspects of technology in use and had less to say about the normative constraints and the environment (with perhaps the notable exceptions of Callon’s  study of markets and MacKenzie’s  classic examination of the nuclear missile industry). Similarly while sociotechnical interaction networks  helpfully capture the social co-construction of technologies and wider ecology of the networks surrounding them we were minded to concur with Meyer’s  assertion that these provided a research strategy rather than theory.
We also considered stances that apply psychology and economics to the problem of understanding technologies in use. These include the Theory of Planned Behaviour  and the Theory of Rational Action/Rational Choice Theory  both of which assure the centrality of action. We were concerned that these tended to focus on human behaviour (and neglect non-human actants which we felt were important) and over-emphasise instrumentality. While these approaches had a considerable track record in experimental research they appeared to perform less well when applied to ethnographic research of the type we were proposing. Likewise the Technology Acceptance Model  appears to offer a hybrid of rational action approach and diffusion theories [46, 47] which can explain adoption, appears less well suited to examining embedding and normalisation.
NPT provided a middle range theory designed explicitly for examining the types of phenomena we were interested in, namely technologies in everyday use, the work of normalisation and the practices of embedding . As the theory is relatively new, we were also keen to explore the use of the constructs of this theory as a structured approach to data interpretation to add to the early corpus of empirical applications of this approach to demonstrate how it might align with the other theoretical perspectives we have noted above.
Ethical approval for this study was given by the Wiltshire Research Ethics Committee (REC 08/H0104/56) in August 2008.
This paper focuses on ethnographic data from our study. This method was chosen to capture the complexities of social interaction in a naturalistic rather than experimental way. Greenhalgh and Swinglehurst  provide a particularly helpful discussion of the merits of this approach for studying ICTs and organisational practices. We used non-participant observation conducted over 20 months, at different times and days of the week across the three sites. This observation focused on the call-handlers using the CDSS but we also observed other staff, including clinical supervisors (999 and SPA site), ambulance dispatchers (999), GPs (OOH) and their interactions with each other and the CDSS. The observation was purposively structured to capture activity at different times of day/days of the week and covered all or part of a shift depending on the setting (between 4 and 8 hours). The observational fieldwork included informal conversations with staff and in addition we interviewed staff and stakeholders (policy makers, commissioners, system developers, managers) using a semi-structured topic guide developed from our knowledge of the relevant literature and initial observations. Detailed notes of each observation period were overtly taken and transcribed soon afterwards and these included verbatim or near verbatim statements. All details about the nature of calls and the triage process were anonymised.
Call-handler interviewees were purposively sampled and interviews were conducted during working hours at points in the day/evening when the service was less busy (with the agreement of shift managers). They were given the study participant information leaflet and time to consider if they would like to participate. Stakeholders and managers (including policy-makers, commissioners, system developers, corporate and operational managers) were also approached and provided with a participant information sheet and a time for the interview was arranged if they agreed. Call-handler interviews typically took 30–45 minutes because of the constraints of their work patterns, other interviews were between 60–90 minutes. Where possible, interviews took place in a private office or meeting room but in OOH some interviews took place in the call centre room. All the interviews were recorded. The interview topic guide was adjusted to reflect the experience and seniority of each participant. Alongside the observation and interviews we collected documents concerning the CDSS design, training materials and reports and this was used to inform our analysis.
The data were analysed, initially via systematic reading and open coding, undertaken independently by five members of the research team and then in collaborative meetings  which allowed discussion of emerging themes. A sixth member of our team (CM) had been closely involved in the development of NPT so as a way of enhancing the validity of the coding we deliberately excluded him from this analytical process to reduce the possibility that we would simply ‘fit’ the data into the domains offered by NPT. Our coding discussions formed the basis of an agreed coding frame, and where there were disagreements about codes we attempted to resolve these amongst the coding team calling on CM for advice and clarification about the theory to inform this consensus as necessary (whist there were a number of early disagreements in the coding process we found that were able to resolve these satisfactorily amongst the team). We used NPT as a framework for the coding and analysis combined with a thematic approach, using constant comparison  to examine codes and refine emerging themes to augment the understanding offered by the NPT domains. To support the analysis we wrote narrative data summaries and used matrix/charting [52, 53] techniques to facilitate comparison. As the study progressed, the analysis was structured to examine all the data within each setting, and then across the settings using our research questions and the framework provided by Normalization Process Theory.
Results and discussion
Observational data - time spent in each setting
Number of hours
Number of days
Nov 2008 – Jul 2010
Oct 2009 – July 2010
Sept 2009 – Aug 2010
Nov 2008 – Aug 2010
Interviewees by role and location
Staff /stakeholder group
Call supervisor/ manager
Other call centre staff
Emergency call-handing for ‘999’
The benefits for me were not taking some people to hospital and only responding vehicles to people who really needed it. (Control room manager)
It wasn’t that long ago a good ambulance service took everyone to hospital but now it’s all changed. Now it’s about how many patients we can keep out of hospital safely. (Senior Manager)
Call-handler explains that the CDSS has been designed by ‘a huge group of people…non-clinicians, clinicians - lots of input…it’s a very safe system to use, used correctly’. (Observation notes)
We’ve always been in the forefront of new things, new initiatives … projects. …you can guarantee if something’s going to happen, we will be at the forefront of that and we’ve got a reputation for that, of wanting to try new things and wanting to push the boundaries a little bit further, so, it didn’t surprise us when we won the pilot for [the CDSS]. (Senior Manager)
For the call-handlers, there was less choice about participation – their use of the CDSS was imposed on them by their managers. However, we know from previous studies that this is not always sufficient to make an intervention work e.g. . The active buy-in of the call-handlers was important to successful implementation. In practice, this was achieved because of a strong commitment amongst the call-handlers to disciplining ‘time-wasters’ (the inebriated and those with minor ailments who should not be calling 999), although alongside this the call-handlers had some concerns that the CDSS might ‘de-skill’ them (something that we return to shortly). In addition to the human actors enrolled at 999 the CDSS had to link to other technological devices systems and geographical and demographic databases – for example maps showing incident and ambulance locations - in order to ‘perform’ the work of 999 call-handling. To summarise in NPT terms, we can see how cognitive participation – the enrolment of the actors necessary to make the intervention work – took place in this case. Notably, attention to cognitive participation shows that diverse motivations for participation were contained within the broader discursive formation of coherence.
[The CDSS] is harder, and it’s easier, if you know what I mean. It’s harder because you’re having to get a lot more information and you seem to, sort of, everything’s on top of you … (Duty Manager)
… “that was naughty…sometimes you let your heart rule your head”. […] There were two main reasons why she overrode the disposition 1) she was an elderly lady who was anxious, 2) there were difficulties in communicating - both her speech and her hearing - was making it a lengthy procedure. The patient had already spoken to [another service] and to the ambulance service. If the call was referred on the patient would be required to speak to a third agency. She felt that this call was going to take a long time to resolve, potentially leading to further distress for the patient. She [the call-handler] justified her actions by saying “I haven’t done an override recently…” (Observation notes)
The development of experiential expertise and the use of tacit knowledge and discretion has of course been highlighted in other organisational research such as the call centre studies noted earlier, and in classic studies like Suchman’s analysis  of how creative improvisational reasoning underpins ‘plans and situated actions’ . What is novel here is that this CDSS offered these non-clinical staff an opportunity to perform clinical identity and gain some additional status and authority. Despite the introduction of an extensive auditing system, these discretionary practices and outcomes were tolerated. In NPT terms, we see the importance of the wider institutional setting within which the collective action is taking place: how implementation takes place at the point of interaction between established and emergent rules, practices and identities. The call-handlers effectively operationalized the innovation and built shared understandings of how this could be done, but this was not exactly as expected. Indeed, tolerance for unanticipated outcomes and willingness to moderate expectations and processes was key to embedding the CDSS in this site.
Throughout the implementation process, the Department of Health, the developers and the Ambulance Trust were attentive to the need for incremental adjustments, a process referred to as reflexive monitoring within NPT. For example, whilst extensive call-audit procedures were introduced alongside the CDSS, which could have been used to ‘police’ the call-handlers very closely, in practice these procedures operated with a tolerance that allowed call-handlers to adapt phrasing and use some of their ‘own’ knowledge in managing calls. This call-handler discretion helped make the system work and possibly also reduced staff turnover by making the job more satisfying. The managers also found it necessary to introduce new ‘clinical supervisors’ (typically a nurse or paramedic) to support the call-handlers in dealing with complex or non-standard cases. Whilst the developers tried to accommodate these kinds of cases within the software, this proved ineffective, and the presence and actions of clinical supervisors effectively legitimised ‘fuzzy logic’ in the day-to-day operation of the CDSS. The Trust also provided additional clinical training for the call-handlers to help them understand symptoms (e.g. explaining what happened when someone had a stroke). This, combined with the presence of clinicians reinforced the sense that call-handling work has a strong clinical component, making their work both possible and rewarding. The interactions between these human actors and the technology and interpretive flexibility in everyday use of the CDSS led to continual evolution of the system. We know from Science and Technology Studies [37, 38] that technologies are never fixed or finished and this was true at 999 where the CDSS developers continued, some 2–3 years into deployment, to upgrade and adapt the system.
Single point of access to urgent and unscheduled care
Single Point of Access (SPA) was a new service, using the CDSS to triage callers and direct them to the relevant primary care services. The call-handler was able to book appointments with appropriate care professionals (commonly a GP) and could also book patient transport for this. The call-handler could also offer self-care advice, prescribed within the CDSS, if indicated. The service was operated by the same ambulance trust as 999 above, as a new venture – both for them (this was the first time they had delivered non-emergency services) and more generally (this was the first attempt to link primary care services in this way).
…we’d already realised that, … the whole health care model’s becoming so complicated it’s difficult to navigate your way, way through it; it’s not the patient’s fault that they don’t know how to, how to access and who to access and that the NHS… should get its act together and make this as easy as possible. (Senior manager)
I would like to think that they would all …be skilled to take all levels of calls. I think if we start to carve out different types of calls coming to different, then you sort of lose that, the capacity. If they are all trained to take the 999 calls, then when they’re not doing an urgent call, they can pick up a 999 call and vice versa. So I think it’s, you know, you have better use of resources if they’re all trained up to take all calls. (Senior manager)
The big prize will be to apply it to those callers who would previously have phoned their GP in-hours because they had an unscheduled care need or they would have phoned their doctor out of hours…or they would have phoned NHS Direct… And my belief is that [the CDSS’s] true worth will then be… truly exhibited. (Senior manager)
Again, framing this using the domains of NPT, the intervention made sense to the key actors in SPA because of commitment to integrated care. But the need for coherence and buy-in was rather different in this case, compared with 999. First, because successful implementation required buy in from primary care and community services and, as part of this, integration of the CDSS with a range of external technological systems (e.g. appointment booking). And second, because the service recruited new call-handlers, who – by default – did not share any of the 999 call-handlers’ reservations about the CDSS and had no pre-existing practices to be adjusted or accommodated.
… the final disposition was a 19 minute ambulance. The call-handler explained that an ambulance was going to be on its way but the patient did not want an ambulance: she refused it. So the call-handler called the clinical supervisor and clicked on early exit. He then arranged an appointment at one of the primary care centres at 6.50pm, by selecting a PCC from the list and clicking on the option book appointment. He confirmed the appointment to the caller and then filled in a box on care details with some brief comments on the case. (Observation notes)
A conversation between a developer, clinical supervisor and researcher comparing 999 and SPA. The clinical supervisor mentions that she thinks that the work requires a “different skill set”. The developer thinks that, in some ways, the out-of-hours work is more difficult, and describes it as having “more complexity”. She says that some call-takers might be suited to one type of work rather than the other. She characterises the difficulties of the different types of work broadly as: on the one hand emergency work is stressful/requires thinking on your feet/acting quickly/confidently, on the other hand, out-of-hours work is complex, less straightforward, and more time consuming – requires patience. She notes that “there are some fantastic  call-takers” but she says that she think some of them might find out-of-hours work “a bit boring” in contrast to the “adrenaline” involved in 999 work. (Observation notes)
The everyday work that it takes to make the intervention work is different in these two different cases. In NPT terms, the collective action is shaped by contextual factors, including the skills required for specific forms of work, and the teamwork required to link aspects of the work (particularly important in SPA).
if you are uncertain, … you can go to a supervisor and be reassured that you are doing the correct thing. Whereas, I think, if you didn’t have a clinical supervisor you would always err on the side of caution… perhaps, sending an ambulance when you wouldn’t necessarily need one. [ …] So it's really good to have somebody who has the medical knowledge in the room, should you need them. (Call-handler)
Whilst at 999 the introduction of new practices to enable reflexive monitoring occurred alongside the introduction of the CDSS, many of the audit and monitoring practices were firmly in place when the SPA service commenced and were easily accepted by call-handlers. They reported deriving a sense of competency, of being ‘good at what they do’, from passing audits, although it is worth noting that the SPA call-handlers did not have target rates for dispositions, a particularly unpopular aspect of auditing in 999.
Thus we see that coherence about the technology had to be established at SPA, in part built on the foundational sense-making achieved in 999, but also built around new service integration and delivery. New staff had to be enrolled into bringing the CDSS into everyday use, and this entailed longer and more complex negotiations with patients, a broader range of dispositions than found at 999 and some subtly different ways of auditing and monitoring practice. Turning to our final case study, our theoretical framework allowed us to systematically examine similarities and differences once more.
General practitioner out of hours service
we were very happy with it; it worked very well. Like I said, we won an award … because it was so effective. The PCTs locally were very happy and they’ve audited it. And we have a clinical governance system that reviews calls and the outcomes of the calls, and the performance of the operators, the GPs, nurses, etc. (General Practitioner)
The positive, with using that system, was that the calls used to take on average 2 minutes …. with the old system, they didn’t need as many operators, so it was more cost effective. (Call-handler)
The main advantage is that it’s a nationally recognised system. It’s obviously gone through all the Royal Colleges, it’s been agreed, and so from a litigation perspective, a medico-legal perspective, it’s very safe and secure. So that gives us real security, the fact that it’s good. [ … ] But, basically, [it] is very, very similar. It works in a similar way to the way we do our [old] protocols. (Senior Manager)
There's nothing that I can say that I think it is better than the way we were doing it, not at the moment. It is, first line triage done by non-clinical staff, for half the price that you're going to have to pay a nurse. If that’s good, if that’s saving people money, that’s fine, but they're asking people who have 60 hours training and not a medical background at all, to do what they're doing, and sometimes I think, oh dear [laughs]. (Call-handler)
Quite often there’ll be a heated discussion going on somewhere of people really that are just running it down, and probably people that have never used it. I mean, the doctors obviously haven’t… they don’t have to use it because we’re using it, but there’s a lot of negativity about it. I don’t know what the answer is to that one, but it’s the old saying: you need to get the people in charge or the right… the people at the top need to be in, trained and confident with it before we start pushing it up at the bottom. (Call-handler)
You’ve got to drive it and you’ve got to have the knowledge and the background to work with that because you’ve got to understand the inference of what it is that you’re asking. If you don’t understand what you’re asking, you’re not going to be able to use it effectively. (Call-handler)
Call-handler: If it's children’s coughs, colds and flu and things, then fine, I will […] let it go where it wants to go. And if it goes ‘to pharmacy within three days’, then that’s fine, but that’s not what the patient wants to hear. The patient has rung us up because they want to see a doctor or a nurse, not because they can see a pharmacy in three days. But I will let it take me and then they will say well no, I want to see somebody. So that’s wrong.
Q: You have an early exit and you will…
Call-handler: Yeah, give the patient an appointment. But then you know it's taking you where it's supposed to take you but the patients don’t want that, they have expectations other than seeing the pharmacist in Tesco’s in three days. (Call-handler)
…for the patient who’s at end of life, [where it’s] not possible to answer questions on the telephone, and you’re asking for a specific type of pain or, and they’re obviously, they can’t answer you. It’s not appropriate to put them all the way through questioning when you wouldn’t be leaving them for two hours or six hours anyway. You would want them to be seen as quickly as possible. So there’s no point in putting them through the torment of taking a load of questions, for them to be asked all over again by the doctor. So in areas like that, it’s not really appropriate to assess them. (Manager)
As a coda to this case, towards the end of our research the OOH service took on reception work at the local Urgent Care Centre located in an acute hospital, using the CDSS to triage face-to-face encounters. Initially it was assumed that the work that would be required to implement the system in the new setting was well understood from the original OOH experience, but this was not the case. Face to face work required additional skills, including negotiating with more than one person simultaneously - often a patient came to the desk with a carer or family member, or using visual and non-verbal cues (e.g. establishing the amount of blood loss without asking the patient) to support assessment; in effect coherence and participation had to be established and reprised once again in this new environment.
Our analysis shows how the ‘same technology’ was used distinctively in different settings reflecting important differences in the nature of the service and care provided and the social interactions between workers, stakeholders and service users. Previous research has alerted us to the fact that the implementation of technologies depends on the interplay of technology and the workforce in everyday practice [55–57]. Our study has extended this understanding by using a theoretical framework to enable cross-case comparison and identification of the mechanisms underpinning successful implementation. We have shown, using NPT how coherence was achieved around the CDSS despite local context variation. Across all three sites there was agreement that the CDSS was suitable for the (varied) tasks and that appropriate resources were in place to enable effective implementation, although these varied between settings. There were differences between settings where the CDSS replaced an established system with existing staff and where the service and/or the staff were new and the work of establishing coherence had to be altered to reflect this. It was clear that knowledge, experience and work identities built through doing call-handling work influenced the coherence of the CDSS for staff in the different settings. What is especially interesting in the wider policy context – where this same CDSS is now being used to support a national ‘111’ urgent care service  is that coherence was not just a local ‘problem’, it was necessarily underpinned by wider understandings and discourses for example about the necessity of rationing and the need to modify caller/patient behaviour and beyond that the very legitimacy of evidence based medicine and the kinds of expert knowledge which underpinned the CDSS.
In all three settings key players were successful in enrolling a diverse network of people and technologies to bring the CDSS into use. We have shown that managers in 999 and OOH had to work harder to enrol call-handlers because of their prior use of a different technology, and the CDSS developers had higher engagement with the 999 setting which built trust and fostered enrolment and legitimation of the CDSS. Considerable effort was expended in enrolling staff although not all were in the same position regarding the CDSS – for example call-handlers had little power to resist its introduction if they wanted to retain their employment. What is important however is that the technology was implemented despite, or even because of, the emergent processes by which the call-handers and stakeholders understood, engaged with and reinterpreted implementation – a finding that is supported by a previous study of a similar healthcare decision support system .
Again, referring to the framework provided by NPT, we have seen how implementing and using the CDSS required collective purposive action. In 999 call-handlers used the CDSS in the management, categorisation and prioritisation of emergency calls and it was viewed positively despite the apparent intensification of their work. In SPA the CDSS facilitated the management of urgent care calls, sorting by urgency and enabling referral to services and/or the giving of health advice but it also extended this work beyond that done in 999; calls were longer and required more probing questions. In OOH the CDSS managed calls to out-of-hours care and face-to-face attendees at an Urgent Care Centre and here their work was extended and had become more scripted. Earlier studies of call centre operators – largely in non-healthcare settings - identified similar features and noted the demanding nature of call centre work, but this research has tended to focus on issues of management and surveillance (see for example [60, 61]) rather than implementation. For the purposes of understanding implementation what is important is that deployment of this CDSS and embedding it in practice, changed the work and the workers in each setting. Call-handling used expertise based on discretion, negotiation and translation skills and required emotional labour. The skills created and sustained by introducing the CDSS included experiential, embodied and clinical expertise, and in turn this work offered an identity as ‘health workers’ and not as generic call centre operatives. This identity appears key to the recruitment and retention of these staff . Deployment of the CDSS also disrupted some existing divisions of labour and hierarchies, for example, at 999 and SPA a new role – clinical supervisor – was introduced.
Although similar monitoring, appraisal and adaptation mechanisms keep the CDSS in place across the three settings, there were once again, differences in how these mechanisms were operationalised. For example the formal audit processes were undertaken differently in OOH, where the processes were more covert and focussed on identifying poor practice, compared with 999 and SPA where audit was seen more as a positive learning tool. Successful deployment of the CDSS entailed significant and long-term involvement from the developers including the need to adapt the system for each setting. One of the questions for further roll out of the CDSS will be how much this input can be reduced to implement the system at scale. It is worth noting that all three sites devoted considerable additional staff resources to support call-handlers, including clinical supervision (999 and SPA only) and audit and training staff.
We have argued that implementing the CDSS and maintaining its everyday use was enabled by the collective action of a range of actors who established coherence and secured participation and engaged in on-going appraisal and adjustment to keep the technology in place. This effort was necessary to bring the CDSS into use and continues to be required to keep it working. As well as detailing how the same technology was implemented in three different settings this paper has demonstrated how NPT can be harnessed to provide a theoretical framework to structure cross-case analyses. Not everything that is technically possible will work in practice, but we can save time and money – and fulfil more hopes – if we can develop robust ways of understanding where effort invested is most likely to be successful. By using NPT we have been able to examine how sites differ in key respects – but also discern patterns and features which can help identify generative mechanisms that might, in turn, inform the implementation of these kinds of technologies in the future.
Our analysis has shown that the CDSS must be understood both as a computer technology and as a set of practices related to that technology, kept in place by a network of actors in particular contexts. The four domains of NPT - collective action, coherence, cognitive participation and reflexive monitoring play out differently in each setting even where the same technology is employed. The three settings are characterised by different ‘work’ and different workforce characteristics and while there might be a common core of training or skills, the content and format of this varies across the three settings. The ability and divisions of labour created and sustained by introducing the CDSS are not just those required to operate a computer system ‘by rote’ but are also about individual experiential, embodied expertise and team sharing of knowledge within a particular context or environment.
To our knowledge this is the first ethnographic study of a single CDSS used in three different emergency and urgent care settings. Our ethnography offers rich detail but like other case study research can be criticised as having limited generalizability. We have attempted to address this by deliberately employing a robust middle range theory (NPT) to provide an analytical framework that allows for comparison and explanation of how this technology was successfully brought into everyday, routine, use. Nonetheless we concede that the three settings studied may not be typical of similar services, and it remains for further research to test the reach of our interpretations. In addition we note that our research was located in particular moment in the development of the CDSS technology and of the life of each of the organisations studied. Some of what we observed was temporally contingent; for example, we suspect that the coherence and cognitive participation of future deployments of the CDSS will be strongly influenced by the sense-making and enrolment of the cases we studied as healthcare organisations learn from earlier adopters. A final limitation concerns the use of NPT. As noted earlier this theory is a relatively young middle range theory and our study is one of an emerging body of empirical applications of NPT. Other theoretical approaches might offer additional and no doubt different insights about the use of this particular CDSS. Our use of NPT means that we have paid less attention to the structures of networks keeping the technology in use (as Actor Network Theory might) or to organisational psychology (as the Theory of Planned Behaviour would). However we have demonstrated that this ICT has established some legitimacy amongst a range of users, who have been effectively enrolled in its deployment, who have collectively acted to bring it into use and who undertake the necessary reflexive tasks to keep it in play: in short we have explained how – unlike so many digital technologies in healthcare – the CDSS has been normalised.
Our research in these three different settings suggests that the CDSS has a strong chance of embedding in the long term and becoming routine in these services. However, we need to recognise that although single technologies can be implemented in different settings, this takes more effort than simply slotting a technology into place. Successful implementation has been achieved, and will only continue to be maintained, through the efforts of those involved in the specific settings and if the wider context continues to support the coherence, cognitive participation, and reflective monitoring processes that surround this collective action. Not least, technological interventions may require new resources to support their effective use, for example, requiring new roles, new organizational functions and considerable management time, all – perhaps – on an on-going basis.
Emergency /Ambulance service
Computer Decision Support Systems
Normalisation Process Theory
Out of hours service
Single point of access to urgent care.
The CDSS project was funded by the National Institute for Health Research Health Services and Delivery Research Programme (project number 08/1819/217). We thank all the anonymous participants for taking part in this research.
This paper presents independent research commissioned by the National Institute for Health Research HS&DR programme. The views and opinions expressed here are those of the authors and do not necessarily reflect those of the HS&DR programme NIHR NHS, or the Department of Health.
- Bates D: The quality case for information technology in healthcare. BMC Medical Informatics and Decision Making. 2002, 2: 7-10.1186/1472-6947-2-7.PubMedPubMed CentralView Article
- Taylor R, Bower A, Girosi F, Bigelow J, Fonkych K, Hillestad R: Promoting Health Information Technology: Is There A Case For More-Aggressive Government Action?. Heal Aff. 2005, 24 (5): 1234-1245. 10.1377/hlthaff.24.5.1234.View Article
- Young C, Ayling M: Digital First: the delivery choice for England’s population. 2011, London: NHS Transform, http://digital.innovation.nhs.uk/dl/cv_content/32200 [accessed 12/3/13]
- Darzi A: High Quality Care For All NHS Next Stage Review Final Report.Command Paper. CM7432. 2008, London: Department of Health, http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_085825 (accessed 16/1/12)
- MacFarlane A, Murphy A: Clerkin P. Telemedicine services in the republic of Ireland: an evolving policy context. Health Policy. 2006, 76: 246-258.View Article
- Dyb K, Halford S: Placing globalising technologies: telemedicine and the making of place. Sociology. 2009, 43 (2): 232-249.View Article
- Barlow J, Singh D, Bayer S, Curry R: A systematic review of the benefits of home telecare for frail elderly people and those with long term conditions. Journal of Telemedicine and Telecare. 2007, 13: 172-179. 10.1258/135763307780908058.PubMedView Article
- Webster A: Information and communications technologies and health care: user centered devices and patient work. Digital Welfare for the Third Age: health and social care informatics for older people. Edited by: Loader B, Hardey M, Keeble L. 2009, London: Routledge
- Westbrook JI, Braithwaite J: Will information and communication technology disrupt the health system and deliver on its promise?. Medical J Aust. 2010, 193 (7): 399.
- Finch TL, Mair FS, May C: Teledermatology in the U.K.: lessons in service innovation. Br J Dermatol. 2007, 156 (3): 521-532. 10.1111/j.1365-2133.2006.07608.x.PubMedView Article
- Haux R: Health information systems: past, present, future. Int J Med Inform. 2006, 75: 268-281. 10.1016/j.ijmedinf.2005.08.002.PubMedView Article
- Doolin B: Power and resistance in the implementation of a medical management information system. Inf Syst J. 2004, 14 (4): 343-362. 10.1111/j.1365-2575.2004.00176.x.View Article
- Timmons S: Nurses resisting information technology. Nurs Inq. 2003, 10 (4): 257-269. 10.1046/j.1440-1800.2003.00177.x.PubMedView Article
- Dent M: Organisation and change in renal work: a study of the impact of a computer system within two hospitals. Sociology of Health and Illness. 1990, 12: 413-431. 10.1111/1467-9566.ep11340410.View Article
- Li J, Westbrook J, Callen J, Georgiou A: The role of ICT in supporting disruptive innovation: a multi-site qualitative study of nurse practitioners in emergency departments. BMC Medical Informatics and Decision Making. 2012, 12: 27-10.1186/1472-6947-12-27.PubMedPubMed CentralView Article
- Halford S, Lotherington AT, Obstfelder A, Dyb K: Getting the Whole Picture? New information and communication technologies in health care work and organization. Information, Communication and Society. 2010, 13: 442-465. 10.1080/13691180903095856.View Article
- de Bont A, Bal R: Telemedicine in interdisciplinary work practices: On an IT system that met the criteria for success set out by its sponsors, yet failed to become part of every-day clinical routines. BMC Med Inform Decis Mak. 2008, 8 (1): 47-10.1186/1472-6947-8-47.PubMedPubMed CentralView Article
- Heath C, Luff P, Svensson M: Technology and Medical Practice. Sociology of Health and Illness. 2003, 25: 75-97. 10.1111/1467-9566.00341.PubMedView Article
- Mclaughlin J, Webster A: Rationalising Knowledge: IT systems, professional identities and power. Sociol Rev. 1998, 46 (4): 781-802. 10.1111/1467-954X.00140.View Article
- Lehoux P, Sicotte C, Denis J-L, Berg M, Lacroix A: The theory of use behind telemedicine: how compatible with physiciansÂ´ clinical routine?. Social Science & Medicine. 2002, 54 (6): 889-904. 10.1016/S0277-9536(01)00063-6.View Article
- Hendy J, Fulop N, Reeves BC, Hutchings A, Collin S: Implementing the NHS information technology programme: qualitative study of progress in acute trusts. BMJ. 2007, 334: 1360-10.1136/bmj.39195.598461.551.PubMedPubMed CentralView Article
- Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG: Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006, 144: 742-752.PubMedView Article
- Gagnon M-P, Duplantie J, Fortin J-P, Landry R: Implementing telehealth to support medical practice in rural/remote regions: what are the conditions for success?. Implement Sci. 2006, 1: 18-10.1186/1748-5908-1-18.PubMedPubMed CentralView Article
- Obstefelder A, Engeseth K, Wynn R: Characteristics of successfully implemented telemedical applications (systematic review). Implement Sci. 2007, 2: 25-10.1186/1748-5908-2-25.View Article
- Nicolini D: The work to make telemedicine work: A social and articulative view. Social Science & Medicine. 2006, 62 (11): 2754-2767. 10.1016/j.socscimed.2005.11.001.View Article
- May C, Harrison R, MacFarlane A, Williams T, Mair F, Wallace P: Why do telemedicine systems fail to normalize as stable models of service delivery?. Journal of Telemedicine and Telecare. 2003, 9 (1): 25-26. 10.1258/135763303322196222.View Article
- Purves I: PRODIGY: implementing clinical guidance using computers. Br J Gen Pract. 1998, 48: 1552-1553.PubMedPubMed Central
- Odyssey ‘TAS’. http://www.plain.co.uk/ (accessed 3/7/12)
- Berner E: Clinical Decision Support Systems: State of the Art. Report for Agency for Healthcare Research and Quality,. 2009, U.S: Department of Health and Human Services Publication No. 09-0069-EF, http://healthit.ahrq.gov/images/jun09cdsreview/09_0069_ef.html (accessed 3/7/12)
- Hanlon G, Strangleman T, Goode J, Luff D, O’Cathain A, Greatbatch D: Knowledge, technology and nursing: the case of NHS direct. Human Relations. 2005, 58 (2): 147-171. 10.1177/0018726705052179.View Article
- Greatbatch D, Hanlon G, Goode J, O’Caithain A, Strangleman T, Luff D: Telephone triage, expert systems and clinical expertise. Sociology of Health & Illness. 2005, 27 (6): 802-830. 10.1111/j.1467-9566.2005.00475.x.View Article
- Taylor P, Bain P: ‘India calling to the far away towns’: the call centre labour process and globalization. Work, Employment and Society. 2005, 19 (2): 261-282. 10.1177/0950017005053170.View Article
- Bain P, Watson A, Mulvey G, Gall G: Taylorism, Targets and the Pursuit of Quantity and Quality by Call Centre Management. NewTechnology, Work and Employment. 2002, 17 (3): 154-169. 10.1111/1468-005X.00102.View Article
- Taylor P, Bain P: Subterranean Worksick Blues: Humour as Subversion in Two Call Centres’. Organ Stud. 2003, 24 (9): 1487-1509. 10.1177/0170840603249008.View Article
- May C: Innovation and Implementation in Health Technology: Normalizing Telemedicine. The New Sociology of the Health Service. Edited by: Gabe J, Calnan M. 2009, London: Routledge
- May C, Finch T: Implementation, embedding, and integration: an outline of Normalization Process Theory. Sociology. 2009, 43 (3): 535-554.View Article
- Latour B: Reassembling the social: an introduction to Actor–network theory. 2005, Oxford: University Press
- Callon M: Some Elements of a Sociology of Translation: Domestication of the Scallops and the Fishermen of Saint Brieuc Bay. Power, Action and Belief: a new Sociology of Knowledge? Sociological Review Monograph. Edited by: Law J. 1986, London: Routledge and Kegan Paul, 196-233.
- Callon M: An Essay on Framing and Overflowing: Economic Externalities Revisited by Sociology. The Laws of the Markets. Edited by: Callon M. 1998, Oxford and Keele: Blackwell/Sociological Review, 244-269.
- Mackenzie D: Inventing accuracy: a historical sociology of nuclear missile guidance. 1990, Cambridge: MIT press
- Kling R, McKim G, King A: A Bit More to IT: Scholarly Communication Forums as Socio-Technical Interaction Networks. J Am Soc Inf Sci Technol. 2003, 54 (1): 46-67.View Article
- Meyer E: Social informatics: an information society for all? In remembrance of Rob Kling. Edited by: Berleur J, Numinen MI, Impagliazzo J. 2006, 37-48. Socio-technical interaction networks: a discussion of the strengths, weaknesses and future of Kling’s STIN model, International Federation for Information processing.View Article
- Ajzen I: The theory of planned behaviour. Organ Behav Hum Decis Process. 1991, 50: 179-211. 10.1016/0749-5978(91)90020-T.View Article
- Hedstrom P: Rational Choice and Social Structure: On Rational-Choice Theorizing in Sociology. Social. Edited by: Wittrock B. 1996, London: Sage, Theory and Human Agency
- Venkatesh V, Bala H: Technology Acceptance Model 3 and a Research Agenda on Interventions. Decis Sci. 2008, 39 (2): 273-315. 10.1111/j.1540-5915.2008.00192.x.View Article
- Rogers E: Diffusion of innovations. 1962, NY: Free Press
- Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion Of Innovations In Service Organizations: Systematic Review And Recommendations. Milbank Quarterly. 2004, 82 (4): 581-629. 10.1111/j.0887-378X.2004.00325.x.PubMedPubMed CentralView Article
- May CR, Mair F, Finch T, Macfarlane A, Dowrick C, Treweek S, Rapley T, Ballini L, Ong BN, Rogers A, Murray E, Elwyn G, Légaré F, Gunn J, Montori VM: Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009, 21 (4): 29.View Article
- Greenhalgh T, Swinglehurst D: Studying technology use as social practice: the untapped potential of ethnography. BMC Medicine. 2011, 9: 45-10.1186/1741-7015-9-45.PubMedPubMed CentralView Article
- Smith AF, Goodwin D, Mort M, Pope C: Expertise in practice: an ethnographic study exploring acquisition and use of knowledge in anaesthesia. Br J Anaesth. 2003, 91: 319-328. 10.1093/bja/aeg180.PubMedView Article
- Glaser B: The constant comparative method of qualitative analysis. Soc Probl. 1965, 12 (4): 436-445. 10.2307/798843.View Article
- Miles M, Huberman A: Qualitative data analysis. 1983, Sage, London, 93-98.
- Ritchie J, Spencer L, O’Connor W: Carrying out qualitative analysis. Qualitative Research Practice; a Guide for Social Science Students and Researchers. Edited by: Ritchie J, Lewis J. 2003, London: Sage, 219-262.
- Hartswood M, Procter R, Rouncefield M, Slack R: Making a Case in Medical Work: Implications for the ElectronicMedical Record’. Computer Supported CooperativeWork. 2003, 12: 241-266. 10.1023/A:1025055829026.View Article
- Suchman LA: Plans and situated actions: the problem of human machine communication. 1987, Cambridge: Cambridge University Press
- Berg M: Rationalizing Medical Work: decision support techniques and medical practices. 1997, Cambridge, MA: MIT Press
- Timmermans S, Berg M: The practice of medical technology. Sociology of Health and Illness. 2003, 25: 97-114. 10.1111/1467-9566.00342.PubMedView Article
- Mathieson SA: Andrew Lansley looks to telehealth and 111 to support NHS change. 2012, The Guardian, http://www.guardian.co.uk/government-computing-network/2012/jun/20/lansley-telehealth-nhs-1119accessed 12
- Collin-Jacques C, Smith C: Nursing on the Line: Experiences from England and Quebec (Canada). Human Relations. 2005, 58 (1): 5-32. 10.1177/0018726705050933.View Article
- Taylor P, Bain P: ‘An assembly line in the head’: work and employee relations in the call centre. Ind Relat J. 1999, 30: 101-117. 10.1111/1468-2338.00113.
- Callaghan G, Thompson P: Edwards revisited: technical control and call centres. Econ Ind Democracy. 2001, 22: 13-37. 10.1177/0143831X01221002.View Article
- Turnbull J, Prichard J, Pope C, Halford S, Salisbury C: Reconfiguring the emergency and urgent care workforce: mixed methods study of skills and the everyday work of non-clinical call-handlers in the NHS. J Health Serv Res Policy. 2012, 17 (4): 233-240. 10.1258/jhsrp.2012.011141.PubMedView Article
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6963/13/111/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.