Skip to main content
Download PDF

Is real world evidence influencing practice? A systematic review of CPRD research in NICE guidances

BMC Health Services Research volume 16, Article number: 299 (2016) Cite this article

Abstract

Background

There is currently limited evidence regarding the extent Real World Evidence (RWE) has directly impacted the health and social care systems. The aim of this review is to identify national guidelines or guidances published in England from 2000 onwards which have referenced studies using the governmental primary care data provider the Clinical Practice Research Datalink (CPRD).

Methods

The methodology recommended by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) was followed. Four databases were searched and documents of interest were identified through a search algorithm containing keywords relevant to CPRD. A search diary was maintained with the inclusion/exclusion decisions which were performed by two independent reviewers.

Results

Twenty-five guidance documents were included in the final review (following screening and assessment for eligibility), referencing 43 different CPRD/GPRD studies, all published since 2007. The documents covered 12 disease areas, with the majority (N =7) relevant to diseases of the Central Nervous system (CNS). The 43 studies provided evidence of disease epidemiology, incidence/prevalence, pharmacoepidemiology, pharmacovigilance and health utilisation.

Conclusions

A slow uptake of RWE in clinical and therapeutic guidelines (as provided by UK governmental structures) was noticed. However, there seems to be an increasing trend in the use of healthcare system data to inform clinical practice, especially as the real world validity of clinical trials is being questioned. In order to accommodate this increasing demand and meet the paradigm shift expected, organisations need to work together to enable or improve data access, undertake translational and relevant research and establish sources of reliable evidence.

Peer Review reports

Background

It is generally agreed that the provision of healthcare should be based on evidence, principally so that a patient receives the best advice or treatment for their condition [1]. As medical evidence is vast and at times contradictory, it is important to have a standard format which presents the evidence for a specific disease or treatment in a way that will help healthcare professionals to grasp and apply such evidence in everyday practice [2]. Standardised evidence also helps to address issues such as inappropriate variability among healthcare professionals in the provision of care [3]. Examples of this are guidelines and guidances.

Guidelines and guidances are documents which incorporate current evidence from reviewed sources in order to develop clear and comprehensive recommendations on the prevention, treatment and care of patients with specific diseases and conditions [4]. These documents can then be used by health and social care professionals to support their decision-making on the care of a patient. The benefits of using guidelines/guidances include the rapid dissemination of updates and changes in clinical practice and the ability to tailor treatment to different clinical situations [5]. Within the UK, the main body responsible for the generation and publication of guidelines and guidances is the National Institute for Health and Care Excellence (NICE) whose primary objective is to advise professionals working in the National Health Service (NHS) on how to provide the highest achievable standard of care [6].

Although the process for developing these documents has moved from being primarily expert knowledge based to being primarily evidence based, there is still concern regarding the sources of evidence. There is heavy reliance on Randomised Controlled Trials (RCTs) for generating evidence for clinical guidelines as (according to many ‘hierarchies’ of evidence) they are thought of as the ‘gold standard’ [7]. However, there are several disadvantages which make evidence from RCTs appear less practical in terms of application to patient care, a key one being the fact that RCTs are generally conducted under controlled conditions on a small number of patients over a fairly short period of time. Even if treatment proves effective in the trial, this does not mean the same effect will translate into the general population as patients in the ‘real world’ can often be more diverse in terms of age, ethnicity, gender and tend to have more comorbidities which may have an impact on the efficacy of a treatment [8]. Therefore there is a limit to the type of evidence that can be generated from RCTs to address key clinical questions which clinicians face on a daily basis [9]. Additionally, there is also the cost of running clinical trials [10] and the increased interest in obtaining return on investment in healthcare [1]. One possible solution is the use of routinely collected data or clinical databases, research outputs of which are often collectively called Real World Evidence (RWE).

Since the transition of paper healthcare records to Electronic Health records (EHRs) it has been possible to create large datasets containing important information such as clinical events, laboratory results, treatment history, etc. [11] These are often referred to as big data or Real World Data (RWD) and present several advantages to health care:

  • they help to strengthen current understanding of healthcare delivery and the outcomes of patients [12],

  • they greatly increase the potential of generating new knowledge as researchers can work to answer important clinical questions (which may otherwise not have been possible) [12] and

  • they can support the development of evidence-based personalised medicine through the linking of EHRs to genomic datasets [13]. EHRs may also enable patients to take a more active role in their healthcare by presenting their health records to other healthcare professionals, if and when necessary [14].

Lastly (and in this case more importantly), RWD could help with the dissemination of key information by bridging the knowledge gap for clinicians and by improving the quantity and quality of evidence used in guidelines and guidances. Best evidence can only be generated when starting with the best data [4]. An example of such a database is the Clinical Practice Research Datalink (CPRD).

CPRD (previously the General Practice Research Database) is one of the largest longitudinal databases in the world containing anonymised EHR data (e.g. demographics, symptoms, behavioural factors, tests, etc.) for 11.3 million patients in the UK [15]. CPRD has been used in over 1500 observational research studies covering a variety of disease and therapeutic areas [16]. However, it is currently not known to what extent CPRD studies have been used to inform clinical practice.

In this context, this study aimed at systematically reviewing the literature to identify guidelines or guidances published from 2000 onwards in England which have referenced studies using RWD from the CPRD. The review has focused particularly on governmental organisations (in terms of data providers and guideline developers) as the UK healthcare system is one of the best integrated systems globally.

Methods

Operational definitions

Guidances and guidelines were categorised according to the definitions provided by the NICE.

The categories were as follows:

  • NICE Clinical Knowledge Summaries: “A readily accessible summary of the current evidence base and practical guidance on best practice in respect of over 330 common and/or significant primary care presentations” [17].

  • Technology appraisals guidance: “Recommendations on the use of new and existing medicines and treatments within the NHS” [18].

  • Clinical guidelines: “Recommend how healthcare professionals should care for people with specific conditions” [19].

Results which did not fit these categories but focused on the delivery of medications were defined as ‘prescribing’ guidelines.

Search strategy

This systematic review adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [20] and was in line with the protocol agreed by all authors.

Guidelines and guidances of interest were identified by a systematic search of four databases from 1st January 2000 to 21st March 2016 (last day of search update): NICE Evidence Search, Medline PubMed, Embase and the National Clinical Guideline Centre. All four databases were searched for guideline/guidance documents referencing studies using data from CPRD using combinations of the following keywords: “CPRD”, “Clinical Practice Research Datalink”, “GPRD”, and “General Practice Research Database”. Indicatively for Medline the following algorithm was used: (((CPRD OR “Clinical Practice Research Datalink” OR GPRD OR “General Practice Research Database”)) AND “guideline”[Publication Type]) AND (“2000/01/01”[Date - Publication]: “3000”[Date - Publication]).

A search diary recording the search results for each database and the meeting of the inclusion/ exclusion criteria for each document was maintained. The specific inclusion criteria were as follows: 1) a UK guideline/ guidance and 2) references research using data from CPRD or GPRD. Documents were excluded if they met one or more of the following criteria: 1) Irrelevant, 2) Not written in English, 3) Not a guidance or guideline (any other primary or secondary research paper), 4) Only mentions CPRD/GPRD (e.g. as a potential source for future studies), 5) Not available (as being updated) and 6) Draft documents or in consultation. Reference lists of all studies previously identified as having met the inclusion criteria were also manually reviewed for additional relevant documents.

The search and assessment of eligibility for included studies were performed by two reviewers working independently. Any duplicate documents were consolidated. All decisions were reached by consensus, with the addition of a third reviewer where required. A relevant PRISMA flow chart was constructed to detail the number of papers retrieved and the steps undertaken.

Data extraction

All data were extracted by two independent investigators and consensus was reached after the involvement of a third investigator where required. Full text was available for all documents. The following information was extracted for each identified document meeting the inclusion criteria: title, year of publication, disease area, the CPRD studies cited, the exact sentences referencing and the type of guideline/ guidance and the references. Each guidance document was categorised by disease area following the categories of the British National Formulary (BNF). The information was then summarised and tabulated in a standard form. In the assessment of the results, studies referenced in more than one guideline document, were treated separately as providing different evidence in each.

A purely descriptive approach was adopted for data synthesis. Sums and means were derived where appropriate. No further statistical analysis was undertaken. As this review did not include any primary research, no form of quality assessment was necessary.

Results

Search results

The PRISMA search strategy yielded 297 documents in total from four bibliographic databases, of which 293 were not duplicates. Following screening, 178 documents were excluded as they were not a guidance or guideline. A further 90 records were excluded based on the other exclusion criteria. In total, 25 documents were included in the final review (Fig. 1), referencing a total of 43 CPRD/GPRD studies.

Fig. 1
figure 1

Flowchart for the systematic review following the PRISMA methodology. 297 documents in total were identified from four bibliographic databases, of which 293 were not duplicates. Following screening, 178 documents were excluded as they were not a guidance or guideline. A further 90 records were excluded based on the other exclusion criteria. 25 documents were included in the final review

Full size image

Study characteristics

Baseline characteristics and detailed information of the included studies, including how the evidence from these studies was used to inform the documents, are listed in Table 1. Of the 25 documents included in the review, 12 were guidelines while the remainder (N =13) were guidances. The guidelines/guidances were published between 2007 and 2015 with 2008 being the only year a guidance/guideline referencing CPRD had not been published. No relevant documents were identified before 2007. The majority of documents were published in 2012 and 2015 (N = 7 for each year). Approximately 16 of the documents were published in the last 3 years. Of the guidelines identified, ten were clinical and the remainder were prescribing. Of the guidances identified, seven were Clinical Knowledge Summaries, two were prescribing, three were Technology Appraisals and the remainder were clinical.

Table 1 An overview of the studies included in the review
Full size table

The guidelines and guidances covered 12 topics (grouped according to the BNF). The majority of the documents (N =7) were focused on diseases of the Central Nervous system (CNS). The guidances covered eight topics with the majority again focusing on diseases of the CNS. The Guidelines covered seven topics with the majority focusing on Infections, the Central nervous system and Ear, nose and oropharynx (N =4 for each topic). The top seven disease areas are listed in Table 2.

Table 2 Top seven disease areas identified in the review
Full size table

Forty-three studies using CPRD data were referenced in the 25 documents with three studies receiving the most citations [21, 22, 24]. The guideline which referenced the most CPRD studies (N =15) was ‘Suspected Cancer: recognition and referral’ (2015). Only three guidances referenced more than one CPRD study (N =2 in all cases),’Osteoporosis: assessing the risk of fragility fracture’ (2012), ‘Rivaroxaban for the prevention of stroke and systemic embolism in people with atrial fibrillation’ (2012), and ‘Sore throat – acute’ (2012).

The evidence used from CPRD studies can be grouped into five categories. Almost three quarters provided information on disease epidemiology (N = 32) while the remainder were in relation to pharmacovigilance (N =12), pharmacoepidemiological evidence (N = 5), incidence or prevalence rates (N = 3) and health utilisation (N = 1).

Discussion

Main findings

The results of this study show RWD from CPRD have not been used to provide input for guidelines and guidances too often. The identified numbers of guidelines and guidances referencing CPRD studies seem significantly small considering that over 900 documents providing guidance have been published by NICE since its inception [64] and that CPRD has been used in over 2000 publications since it began its existence in 1987. These findings correlate to a similar study by Tricoci et al. in the USA who conducted a review of the American College of Cardiology (ACC) and the American Heart Association (AHA) guidelines in 2009. The study identified 16 relevant guidelines for their review with only 11 % of the recommendations being based on evidence from multiple sources instead of expert opinion or evidence from a single study [65]. However, the results of this review do show an increase in the frequency of RWD studies being used in guidelines/guidances in recent years. Despite the fact that clinical datasets and large longitudinal databases have been in existence for well over three decades, interest in RWD and large clinical databases for health research to influence guidelines and guidances is still in its infancy. Improvements in data processing, innovation in bioinformatics, the increased uptake of EHR systems, the increased demand for better and more efficient health care and the need for rapid generation of evidence are all contributing towards an increasing trend in its use by researchers, clinicians and policy makers [66].

Disease areas

Identified documents covered a variety of disease areas demonstrating the breadth of research using RWD generated from EHRs. This is partly due to the range of data available through large healthcare databases and datasets and the depth of the data being enhanced by linking to other datasets.

The majority of the documents focused on diseases or treatments to do with the CNS. It appears, in this area of health, the benefits of RWE are being more utilised. For example, two of the documents refer to the treatment of patients with mental illness which correlates with research trends as several studies have investigated the patterns, drug effects and outcomes of patients used RWD [67]. The document which cited the most CPRD studies was the guideline on Suspected Cancer which highlighted the need for better methods of diagnosis and early detection and gave precise and more up-to-date information on how to detect over 200 cancers [16]. This reflects the increased interest in this disease area where several studies have looked into using EHRs to ‘flag’ recognised diagnostic clues in a timely manner [68]. This will have a substantial benefit to cancer patients as delays in diagnosis have been linked to poorer prognosis [69].

Evidence from studies using CPRD is significantly under-represented in conditions which are primarily treated in primary care (e.g. Diabetes, Obesity, Asthma, etc.). This is not because there is a lack of studies on the subject. The possible reasons for this have already been discussed above and as a result, there needs to be a review of the kind of evidence used in guidelines and guidances, for if the data is not for the purpose it was created then it is a waste of a potentially health-changing resource.

Studies using CPRD data were not referenced in guidelines in other disease areas at all, such as nutrition and blood, eye, and anaesthesia. For most of these disease areas, the reason is quite clear. CPRD currently has no or limited data on diseases and treatments that are mainly administered in secondary care (e.g. drugs administered through the eye) and there are no efficient centralised databases containing such information. However, the linking of datasets from varied health settings can provide a fuller picture of disease and health outcomes in the general population and therefore provide even more robust evidence [70]. A good example of this is the cardiovascular disease research using linked bespoke studies and electronic health records (CALIBER) dataset comprising of CPRD GOLD data and linked data from Hospital Episodes Statistics (HES), deprivation data, the Office for National Statistics (ONS) mortality information and the Myocardial Ischaemia National Audit Project (MINAP) [71]. This was a bespoke linked dataset to perform studies to improve the health of patients suffering from cardiovascular diseases and has been used in a number of useful studies, for example, to identify new associations for a range of risk factors in cardiovascular disease [72].

This shows that the ability to link datasets from a variety of sources provides immense opportunity to not only get a fuller picture of a patient’s medical history, but also investigate the interactions and associations between different treatments/diseases in different clinical settings and possibly developing predictors of health outcomes [73].

The future

Looking towards how clinical evidence can be improved, one would directly look into the organisations providing the data. As CPRD and other data providers continue to expand their linkages to other data sources and the benefits of linked data sources increase in recognition, funds should be invested in creating datasets in all sectors of health. This will enable healthcare professionals to make sound decisions based on RWD, regardless of their line of work. The Health and Social Care Information Centre (HSCIC) who manage and maintain the balance between the sharing of information for community benefit and respecting the confidentiality and wishes of patients have been key in enabling research through linked data in the UK [74].

Strengths and limitations

There are several strengths to this study. Firstly, it was conducted using the gold-standard method for conducting systematic reviews [75]. The PRISMA methodology has been found to improve the completeness of systematic review and meta-analysis reporting [76]. Furthermore, an exhaustive search of multiple bibliographic databases was followed. NICE and the National Clinical Guideline Centre are the main databases for UK clinical guidelines and guidances with the majority of health-related organisations referencing these sites for further information or access. The study focused on identifying research which has used data from the most widely used source of RWE (CPRD) and in a country that uses medical informatics research extensively. Lastly, the authors have the relevant experience and knowledge of CPRD, the provision of healthcare in the UK and the process for conducting systematic reviews.

However, limitations of the review need to be acknowledged. Firstly, this review focused only on guidelines and guidances which used evidence from studies using CPRD data. There are other longitudinal databases in the UK such as The Health Improvement Network (THIN) and QResearch. It also did not look into what type of data was used in each study (e.g. use of linked data). For future studies, it would be interesting to investigate whether evidence from other longitudinal databases have been used to inform guidelines/guidances. This study also focused on guidelines and guidances published for health and social care in England. Future studies could compare how RWD is used not only nationally but also internationally and identify the trends and differences that may exist. Lastly, guidelines and guidances which were currently ‘under review’ were not included in the review as they were liable to change once published. Therefore it would be worth conducting the review again at such a time when these guidelines/guidances become available to see how they affect the current results.

Conclusions

In this systematic review, we confirmed that Real World Evidence from the Clinical Practice Research Datalink has been used inconsistently but increasingly in the last decade, to inform guidelines and guidances published in the United Kingdom. The increased uptake in recent years, noted in our results, shows that this area of healthcare is changing and this review captures a phase in this transition. To capitalise on the potential value of using Real World Evidence, researchers need to ensure they undertake research of translational value to the healthcare community. Organisations which develop guidelines should also work to identify Real World Evidence sources which will give a more realistic view of how an intervention works in actual healthcare settings. Finally, key points extrapolated from our review include increasing the quality of available Real World Evidence (which will require investment on capacity, skills and accessibility) and maintaining public trust (which will be key for wider uptake).

Abbreviations

ACC, American college of cardiology; AHA, American heart association; BNF, British national formulary; CALIBER, cardiovascular disease research using linked bespoke studies and electronic health records; CNS, central nervous system; CPRD, Clinical Practice Research Datalink; EHR, electronic health record; HES, Hospital Episodes Statistics; HSCIC, Health and Social Care Information Centre; MINAP, myocardial ischaemia national audit project; NHS, National Health Service; NICE, National Institute for health and Care Excellence; ONS, Office for National Statistics; PRISMA, preferred reporting items for systematic reviews and meta-analyses; RCT, randomised controlled trial; RWD, real world data; RWE, real world evidence; THIN, the health improvement network

References

  1. Clancy CM, Cronin K. Evidence-based decision making: global evidence, local decisions. Health Aff (Millwood). 2005;24:151–62.

    Article  Google Scholar 

  2. Sniderman AD, Furberg CD. Why guideline-making requires reform. JAMA. 2009;301:429–31.

    Article  CAS  PubMed  Google Scholar 

  3. Shaneyfelt TM, Centor RM. Reassessment of clinical practice guidelines: go gently into that good night. JAMA. 2009;301:868–9.

    Article  CAS  PubMed  Google Scholar 

  4. Wilson M. Big data in health care decisions: where Are We Now and what does the future hold? Values Outcomes Spotlight. 2015;1:7–9.

    Google Scholar 

  5. Isern D, Moreno A. Computer-based execution of clinical guidelines: a review. Int J Med Inform. 2008;77:787–808.

    Article  PubMed  Google Scholar 

  6. Rawlins MD. NICE: moving onward. N Engl J Med. 2013;369:3–5.

    Article  CAS  PubMed  Google Scholar 

  7. Zhang Z. Big data and clinical research: perspective from a clinician. J Thorac Dis. 2014;6:1659–64.

    PubMed  PubMed Central  Google Scholar 

  8. Rawlins M. De Testimonio: on the evidence for decisions about the use of therapeutic interventions. Clin Med. 2008;8:579–88.

    Article  Google Scholar 

  9. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58:323–37.

    Article  PubMed  Google Scholar 

  10. Garrison LP, Neumann PJ, Erickson P, et al. Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value Health. 2007;10:326–35.

    Article  PubMed  Google Scholar 

  11. Wang W, Krishnan E. Big data and clinicians: a review on the state of the science. JMIR Med Inform. 2014;2:1–11.

    Article  Google Scholar 

  12. Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013;309:1351–2.

    Article  CAS  PubMed  Google Scholar 

  13. Roden DM, Pulley JM, Basford MA, et al. Development of a large-scale de-identified DNA biobank to enable personalized medicine. Clin Pharmacol Ther. 2008;84:362–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Cáceres SB. Electronic health records: beyond the digitization of medical files. Clinics (Sao Paulo). 2013;68:1077–8.

    Article  Google Scholar 

  15. Herrett E, Gallagher AM, Bhaskaran K, et al. Data resource profile: clinical practice research datalink (CPRD). Int J Epidemiol. 2015;44:827–36.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Kousoulis AA, Rafi I, de Lusignan S. The CPRD and the RCGP: building on research success by enhancing benefits for patients and practices. Br J Gen Pract. 2015;65:54–5.

    Article  PubMed  PubMed Central  Google Scholar 

  17. NICE. Clinical knowledge summaries. NICE. 2016. http://cks.nice.org.uk/#?char=A. Accessed 21 Mar 2016.

  18. NICE. NICE Technology appraisal guidance. NICE. 2016. https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-technology-appraisal-guidance. Accessed 21 Mar 2016

  19. NICE. Types of guideline. NICE. 2016. www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-guidelines/types-of-guideline. Accessed 21 Mar 2016.

  20. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Van Staa T-P, Cooper C, Brusse LS, et al. Inflammatory bowel disease and the risk of fracture. Gastroenterology. 2003;125:1591–7.

    Article  PubMed  Google Scholar 

  22. Card T, West J, Hubbard R, et al. Hip fractures in patients with inflammatory bowel disease and their relationship to corticosteroid use: a population based cohort study. Gut. 2004;53:251–5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. West J, Logan RFA, Card TR, et al. Fracture risk in people with celiac disease: a population-based cohort study. Gastroenterology. 2003;125:429–36.

    Article  PubMed  Google Scholar 

  24. Mikuls TR, Farrar JT, Bilker WB, et al. Gout epidemiology: results from the UK general practice research database, 1990–1999. Ann Rheum Dis. 2005;64:267–72.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Osborn DPJ, Levy G, Nazareth I, et al. Relative risk of cardiovascular and cancer mortality in people with severe mental illness from the united Kingdom’s general practice research database. Arch Gen Psychiatry. 2007;64:242–9.

    Article  PubMed  Google Scholar 

  26. Schneider-Lindner V, Delaney JA, Dial S, et al. Antimicrobial drugs and community-acquired methicillin-resistant staphylococcus aureus, United Kingdom. Emerg Infect Dis. 2007;13:994–1000.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Gunnell D, Irvine D, Wise L, et al. Varenicline and suicidal behaviour: a cohort study based on data from the general practice research database. BMJ. 2009;339:b3805.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Vasilakis C, Jick H, del Mar Melero-Montes M. Risk of idiopathic venous thromboembolism in users of progestagens alone. Lancet. 1999;354:1610–1.

    Article  CAS  PubMed  Google Scholar 

  29. Smitten AL, Choi HK, Hochberg MC, et al. The risk of herpes zoster in patients with rheumatoid arthritis in the United States and the United Kingdom. Arthritis Rheumatol. 2007;57:1431–8.

    Article  Google Scholar 

  30. De Giuli F, Khaw K-T, Cowie MR, et al. Incidence and outcome of persons with a clinical diagnosis of heart failure in a general practice population of 696,884 in the United Kingdom. Eur J Heart Fail. 2005;7:295–302.

    Article  PubMed  Google Scholar 

  31. Mehta NN, Azfar RS, Shin DB, et al. Patients with severe psoriasis are at increased risk of cardiovascular mortality: cohort study using the general practice research database. Eur Heart J. 2010;31:1000–6.

    Article  PubMed  Google Scholar 

  32. Abuabara K, Azfar RS, Shin DB, et al. Cause-specific mortality in patients with severe psoriasis: a population-based cohort study in the U.K. Br J Dermatol. 2010;163:586–92.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Mehta NN, Yu Y, Pinnelas R, et al. Attributable risk estimate of severe psoriasis on major cardiovascular events. Am J Med. 2011;124:775.e1–6.

    Article  PubMed  Google Scholar 

  34. Petersen I, Johnson AM, Islam A, et al. Protective effect of antibiotics against serious complications of common respiratory tract infections: retrospective cohort study with the UK general practice research database. BMJ. 2007;335:982.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Dunn N, Lane D, Everitt H, et al. Use of antibiotics for sore throat and incidence of quinsy. Br J Gen Pract. 2007;57:45–9.

    PubMed  PubMed Central  Google Scholar 

  36. Majeed A, Moser K, Carroll K. Trends in the prevalence and management of atrial fibrillation in general practice in England and Wales, 1994–1998: analysis of data from the general practice research database. Heart. 2001;86:284–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Gallagher AM, Rietbrock S, Plumb J, et al. Initiation and persistence of warfarin or aspirin in patients with chronic atrial fibrillation in general practice: do the appropriate patients receive stroke prophylaxis? J Thromb Haemost. 2008;6:1500–6.

    Article  CAS  PubMed  Google Scholar 

  38. Cornish R, Macleod J, Strang J, et al. Risk of death during and after opiate substitution treatment in primary care: prospective observational study in UK general practice research database. BMJ. 2010;341:c5475.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Van Staa T-P, Geusens P, Pols HAP, et al. A simple score for estimating the long-term risk of fracture in patients using oral glucocorticoids. QJM. 2005;98:191–8.

    Article  PubMed  Google Scholar 

  40. Van Staa TP, Leufkens HG, Abenhaim L, et al. Oral corticosteroids and fracture risk: relationship to daily and cumulative doses. Rheumatology (Oxford). 2000;39:1383–9.

    Article  Google Scholar 

  41. Lawrenson RA, Logie JW. Antibiotic failure in the treatment of urinary tract infections in young women. J Antimicrob Chemother. 2001;48:895–901.

    Article  CAS  PubMed  Google Scholar 

  42. De Vries F, Setakis E, Zhang B, et al. Long-acting {beta}2-agonists in adult asthma and the pattern of risk of death and severe asthma outcomes: a study using the GPRD. Eur Respir J. 2010;36:494–502.

    Article  PubMed  Google Scholar 

  43. Parkin L, Sharples K, Hernandez RK, et al. Risk of venous thromboembolism in users of oral contraceptives containing drospirenone or levonorgestrel: nested case–control study based on UK general practice research database. BMJ. 2011;342:d2139.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Stowe J, Andrews N, Wise L, et al. Investigation of the temporal association of Guillain-barre syndrome with influenza vaccine and influenzalike illness using the United Kingdom general practice research database. Am J Epidemiol. 2009;169:382–8.

    Article  PubMed  Google Scholar 

  45. Rowlands S, Devalia H, Smith C, et al. Otitis externa in UK general practice: a survey using the UK general practice research database. Br J Gen Pract. 2001;51:533–8.

    CAS  PubMed  PubMed Central  Google Scholar 

  46. Byford S, Barrett B, Despiégel N, et al. Impact of treatment success on health service use and cost in depression: longitudinal database analysis. Pharmacoeconomics. 2011;29:157–70.

    Article  PubMed  Google Scholar 

  47. Mackenzie IS, Morant SV, Bloomfield GA, et al. Incidence and prevalence of multiple sclerosis in the UK 1990–2010: a descriptive study in the general practice research database. J Neurol Neurosurg Psychiatry. 2014;85:76–84.

    Article  CAS  PubMed  Google Scholar 

  48. Mangtani P, Cumberland P, Hodgson CR, et al. A cohort study of the effectiveness of influenza vaccine in older people, performed using the United Kingdom general practice research database. J Infect Dis. 2004;190:1–10.

    Article  PubMed  Google Scholar 

  49. Jones R, Latinovic R, Charlton J, et al. Alarm symptoms in early diagnosis of cancer in primary care: cohort study using general practice research database. BMJ. 2007;334:1040.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Stapley S, Peters TJ, Neal RD, et al. The risk of pancreatic cancer in symptomatic patients in primary care: a large case–control study using electronic records. Br J Cancer. 2012;106:1940–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Stapley S, Peters TJ, Neal RD, et al. The risk of oesophago-gastric cancer in symptomatic patients in primary care: a large case–control study using electronic records. Br J Cancer. 2013;108:25–31.

    Article  CAS  PubMed  Google Scholar 

  52. Walker S, Hyde C, Hamilton W. Risk of breast cancer in symptomatic women in primary care: a case–control study using electronic records. Br J Gen Pract. 2014;64:e788–93.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Walker S, Hyde C, Hamilton W. Risk of uterine cancer in symptomatic women in primary care: case–control study using electronic records. Br J Gen Pract. 2013;63:e643–8.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Shephard EA, Stapley S, Neal RD, et al. Clinical features of bladder cancer in primary care. Br J Gen Pract. 2012;62:e598–604.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Shephard E, Neal R, Rose P, et al. Clinical features of kidney cancer in primary care: a case–control study using primary care records. Br J Gen Pract. 2013;63:e250–5.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Shephard EA, Neal RD, Rose P, et al. Quantifying the risk of multiple myeloma from symptoms reported in primary care patients: a large case–control study using electronic records. Br J Gen Pract. 2015;65:e106–13.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Price SJ, Shephard EA, Stapley SA, et al. Non-visible versus visible haematuria and bladder cancer risk: a study of electronic records in primary care. Br J Gen Pract. 2014;64:e584–9.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Dommett RM, Redaniel MT, Stevens MCG, et al. Features of childhood cancer in primary care: a population-based nested case–control study. Br J Cancer. 2012;106:982–7.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Dommett RM, Redaniel T, Stevens MCG, et al. Risk of childhood cancer with symptoms in primary care: a population-based case–control study. Br J Gen Pract. 2013;63:e22–9.

    Article  PubMed  Google Scholar 

  60. Dommett RM, Redaniel MT, Stevens MCG, et al. Features of cancer in teenagers and young adults in primary care: a population-based nested case–control study. Br J Cancer. 2013;108:2329–33.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Kernick D, Stapley S, Goadsby PJ, et al. What happens to new-onset headache presented to primary care? A case-cohort study using electronic primary care records. Cephalalgia. 2008;28:1188–95.

    Article  CAS  PubMed  Google Scholar 

  62. Kernick D, Stapley S, Campbell J, et al. What happens to new-onset headache in children that present to primary care? A case-cohort study using electronic primary care records. Cephalalgia. 2009;29:1311–6.

    Article  CAS  PubMed  Google Scholar 

  63. Hamilton W, Kernick D. Clinical features of primary brain tumours: a case–control study using electronic primary care records. Br J Gen Pract. 2007;57:695–9.

    PubMed  PubMed Central  Google Scholar 

  64. Rawlins M. NICE times: a valedictory dispatch. Lancet. 2013;381:2225–7.

    Article  PubMed  Google Scholar 

  65. Tricoci P, Allen JM, Kramer JM, et al. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA. 2009;301:831–41.

    Article  CAS  PubMed  Google Scholar 

  66. Raghupathi W, Raghupathi V. Big data analytics in healthcare: promise and potential. Heal Inf Sci Syst. 2014;2:1–10.

    Article  Google Scholar 

  67. Howard L, Kirkwood G, Leese M. Risk of hip fracture in patients with a history of schizophrenia. Br J Psychiatry. 2007;190:129–34.

    Article  PubMed  Google Scholar 

  68. Singh H, Hirani K, Kadiyala H, et al. Characteristics and predictors of missed opportunities in lung cancer diagnosis: an electronic health record-based study. J Clin Oncol. 2010;28:3307–15.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Murphy DR, Laxmisan A, Reis BA, et al. Electronic health record-based triggers to detect potential delays in cancer diagnosis. BMJ Qual Saf. 2014;23:8–16.

    Article  PubMed  Google Scholar 

  70. Bradley CJ, Penberthy L, Devers KJ, et al. Health services research and data linkages: issues, methods, and directions for the future. Health Serv Res. 2010;45:1468–88.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Denaxas SC, George J, Herrett E, et al. Data resource profile: cardiovascular disease research using linked bespoke studies and electronic health records (CALIBER). Int J Epidemiol. 2012;41:1625–38.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Asaria M, Grasic K, Walker S. Using linked electronic health records to estimate healthcare costs: Key challenges and opportunities. Pharmacoeconomics. 2016;34:155–60.

    Article  PubMed  Google Scholar 

  73. Fleurence RL, Naci H, Jansen JP. The critical role of observational evidence in comparative effectiveness research. Health Aff. 2010;29:1826–33.

    Article  Google Scholar 

  74. Health & Social Care Information Centre. A guide to confidentiality in health and social care. Health & Social Care Information Centre. 2013. www.hscic.gov.uk/media/12822/Guide-to-confidentiality-in-health-and-social-care/pdf/HSCIC-guide-to-confidentiality.pdf. Accessed 18 Oct 2015.

  75. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

    Article  PubMed  PubMed Central  Google Scholar 

  76. Tunis AS, McInnes MDF, Hanna R, et al. Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement? Radiology. 2013;269:413–26.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The contributions of Dr Tim Williams and Tarita Murray-Thomas are acknowledged in advising and reviewing for this study.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Availability of data and materials

The results of the study are included in the manuscript. The search algorithms used for the four databases was as follows:

NICE Evidence Search: “CPRD” or “Clinical Practice Research Datalink” OR “GPRD” OR “General Practice Research Database”.

Medline PubMed: (“CPRD” OR “Clinical Practice Research Datalink” OR “GPRD” OR “General Practice Research Database”) AND guid*.

Embase: (“CPRD” OR “Clinical Practice Research Datalink” OR “GPRD” OR “General Practice Research Database”) AND guid*.

The National Clinical Guideline Centre: “CPRD” or “Clinical Practice Research Datalink” OR “GPRD” OR “General Practice Research Database”.

Authors’ contributions

JO contributed to the design of the study, performed the search and selected the eligible documents. JO also extracted the data and drafted the manuscript. JCa contributed to the design of the study, agreement on study inclusion and drafted parts of and edited the manuscript. AK contributed to the design of the study, performed the search and selected the eligible documents, drafted parts of and edited the manuscript. All authors have read and approved the final manuscript.

Competing interests

The authors were employees of the Medicines and Healthcare products Regulatory Agency at the time when this research was carried out. They declare no other competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

This study is a systematic review of published guidelines and guidances. It was therefore not necessary to apply for ethical approval.

Author information

Authors and Affiliations

  1. Clinical Practice Research Datalink (CPRD), Medicines and Healthcare products Regulatory Agency, 151 Buckingham Palace Road, Victoria London, SW1W 9SZ, UK

    Jessie O. Oyinlola, Jennifer Campbell & Antonis A. Kousoulis

Authors
  1. Jessie O. Oyinlola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jennifer Campbell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonis A. Kousoulis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jessie O. Oyinlola.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oyinlola, J.O., Campbell, J. & Kousoulis, A.A. Is real world evidence influencing practice? A systematic review of CPRD research in NICE guidances. BMC Health Serv Res 16, 299 (2016). https://doi.org/10.1186/s12913-016-1562-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-016-1562-8

Keywords

  • Guidance
  • Real world evidence
  • Clinical practice research datalink

BMC Health Services Research

ISSN: 1472-6963

Contact us