Is real world evidence influencing practice? A systematic review of CPRD research in NICE guidances

Operational definitions

Guidances and guidelines were categorised according to the definitions provided by the NICE.

Results which did not fit these categories but focused on the delivery of medications were defined as ‘prescribing’ guidelines.

Search strategy

This systematic review adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [20] and was in line with the protocol agreed by all authors.

Guidelines and guidances of interest were identified by a systematic search of four databases from 1st January 2000 to 21st March 2016 (last day of search update): NICE Evidence Search, Medline PubMed, Embase and the National Clinical Guideline Centre. All four databases were searched for guideline/guidance documents referencing studies using data from CPRD using combinations of the following keywords: “CPRD”, “Clinical Practice Research Datalink”, “GPRD”, and “General Practice Research Database”. Indicatively for Medline the following algorithm was used: (((CPRD OR “Clinical Practice Research Datalink” OR GPRD OR “General Practice Research Database”)) AND “guideline”[Publication Type]) AND (“2000/01/01”[Date - Publication]: “3000”[Date - Publication]).

A search diary recording the search results for each database and the meeting of the inclusion/ exclusion criteria for each document was maintained. The specific inclusion criteria were as follows: 1) a UK guideline/ guidance and 2) references research using data from CPRD or GPRD. Documents were excluded if they met one or more of the following criteria: 1) Irrelevant, 2) Not written in English, 3) Not a guidance or guideline (any other primary or secondary research paper), 4) Only mentions CPRD/GPRD (e.g. as a potential source for future studies), 5) Not available (as being updated) and 6) Draft documents or in consultation. Reference lists of all studies previously identified as having met the inclusion criteria were also manually reviewed for additional relevant documents.

The search and assessment of eligibility for included studies were performed by two reviewers working independently. Any duplicate documents were consolidated. All decisions were reached by consensus, with the addition of a third reviewer where required. A relevant PRISMA flow chart was constructed to detail the number of papers retrieved and the steps undertaken.

Data extraction

All data were extracted by two independent investigators and consensus was reached after the involvement of a third investigator where required. Full text was available for all documents. The following information was extracted for each identified document meeting the inclusion criteria: title, year of publication, disease area, the CPRD studies cited, the exact sentences referencing and the type of guideline/ guidance and the references. Each guidance document was categorised by disease area following the categories of the British National Formulary (BNF). The information was then summarised and tabulated in a standard form. In the assessment of the results, studies referenced in more than one guideline document, were treated separately as providing different evidence in each.

A purely descriptive approach was adopted for data synthesis. Sums and means were derived where appropriate. No further statistical analysis was undertaken. As this review did not include any primary research, no form of quality assessment was necessary.

Search results

The PRISMA search strategy yielded 297 documents in total from four bibliographic databases, of which 293 were not duplicates. Following screening, 178 documents were excluded as they were not a guidance or guideline. A further 90 records were excluded based on the other exclusion criteria. In total, 25 documents were included in the final review (Fig. 1), referencing a total of 43 CPRD/GPRD studies.

Study characteristics

Forty-three studies using CPRD data were referenced in the 25 documents with three studies receiving the most citations [21, 22, 24]. The guideline which referenced the most CPRD studies (N =15) was ‘Suspected Cancer: recognition and referral’ (2015). Only three guidances referenced more than one CPRD study (N =2 in all cases),’Osteoporosis: assessing the risk of fragility fracture’ (2012), ‘Rivaroxaban for the prevention of stroke and systemic embolism in people with atrial fibrillation’ (2012), and ‘Sore throat – acute’ (2012).

The evidence used from CPRD studies can be grouped into five categories. Almost three quarters provided information on disease epidemiology (N = 32) while the remainder were in relation to pharmacovigilance (N =12), pharmacoepidemiological evidence (N = 5), incidence or prevalence rates (N = 3) and health utilisation (N = 1).

Main findings

The results of this study show RWD from CPRD have not been used to provide input for guidelines and guidances too often. The identified numbers of guidelines and guidances referencing CPRD studies seem significantly small considering that over 900 documents providing guidance have been published by NICE since its inception [64] and that CPRD has been used in over 2000 publications since it began its existence in 1987. These findings correlate to a similar study by Tricoci et al. in the USA who conducted a review of the American College of Cardiology (ACC) and the American Heart Association (AHA) guidelines in 2009. The study identified 16 relevant guidelines for their review with only 11 % of the recommendations being based on evidence from multiple sources instead of expert opinion or evidence from a single study [65]. However, the results of this review do show an increase in the frequency of RWD studies being used in guidelines/guidances in recent years. Despite the fact that clinical datasets and large longitudinal databases have been in existence for well over three decades, interest in RWD and large clinical databases for health research to influence guidelines and guidances is still in its infancy. Improvements in data processing, innovation in bioinformatics, the increased uptake of EHR systems, the increased demand for better and more efficient health care and the need for rapid generation of evidence are all contributing towards an increasing trend in its use by researchers, clinicians and policy makers [66].

Disease areas

Identified documents covered a variety of disease areas demonstrating the breadth of research using RWD generated from EHRs. This is partly due to the range of data available through large healthcare databases and datasets and the depth of the data being enhanced by linking to other datasets.

The majority of the documents focused on diseases or treatments to do with the CNS. It appears, in this area of health, the benefits of RWE are being more utilised. For example, two of the documents refer to the treatment of patients with mental illness which correlates with research trends as several studies have investigated the patterns, drug effects and outcomes of patients used RWD [67]. The document which cited the most CPRD studies was the guideline on Suspected Cancer which highlighted the need for better methods of diagnosis and early detection and gave precise and more up-to-date information on how to detect over 200 cancers [16]. This reflects the increased interest in this disease area where several studies have looked into using EHRs to ‘flag’ recognised diagnostic clues in a timely manner [68]. This will have a substantial benefit to cancer patients as delays in diagnosis have been linked to poorer prognosis [69].

Evidence from studies using CPRD is significantly under-represented in conditions which are primarily treated in primary care (e.g. Diabetes, Obesity, Asthma, etc.). This is not because there is a lack of studies on the subject. The possible reasons for this have already been discussed above and as a result, there needs to be a review of the kind of evidence used in guidelines and guidances, for if the data is not for the purpose it was created then it is a waste of a potentially health-changing resource.

Studies using CPRD data were not referenced in guidelines in other disease areas at all, such as nutrition and blood, eye, and anaesthesia. For most of these disease areas, the reason is quite clear. CPRD currently has no or limited data on diseases and treatments that are mainly administered in secondary care (e.g. drugs administered through the eye) and there are no efficient centralised databases containing such information. However, the linking of datasets from varied health settings can provide a fuller picture of disease and health outcomes in the general population and therefore provide even more robust evidence [70]. A good example of this is the cardiovascular disease research using linked bespoke studies and electronic health records (CALIBER) dataset comprising of CPRD GOLD data and linked data from Hospital Episodes Statistics (HES), deprivation data, the Office for National Statistics (ONS) mortality information and the Myocardial Ischaemia National Audit Project (MINAP) [71]. This was a bespoke linked dataset to perform studies to improve the health of patients suffering from cardiovascular diseases and has been used in a number of useful studies, for example, to identify new associations for a range of risk factors in cardiovascular disease [72].

This shows that the ability to link datasets from a variety of sources provides immense opportunity to not only get a fuller picture of a patient’s medical history, but also investigate the interactions and associations between different treatments/diseases in different clinical settings and possibly developing predictors of health outcomes [73].

The future

Looking towards how clinical evidence can be improved, one would directly look into the organisations providing the data. As CPRD and other data providers continue to expand their linkages to other data sources and the benefits of linked data sources increase in recognition, funds should be invested in creating datasets in all sectors of health. This will enable healthcare professionals to make sound decisions based on RWD, regardless of their line of work. The Health and Social Care Information Centre (HSCIC) who manage and maintain the balance between the sharing of information for community benefit and respecting the confidentiality and wishes of patients have been key in enabling research through linked data in the UK [74].

Strengths and limitations

There are several strengths to this study. Firstly, it was conducted using the gold-standard method for conducting systematic reviews [75]. The PRISMA methodology has been found to improve the completeness of systematic review and meta-analysis reporting [76]. Furthermore, an exhaustive search of multiple bibliographic databases was followed. NICE and the National Clinical Guideline Centre are the main databases for UK clinical guidelines and guidances with the majority of health-related organisations referencing these sites for further information or access. The study focused on identifying research which has used data from the most widely used source of RWE (CPRD) and in a country that uses medical informatics research extensively. Lastly, the authors have the relevant experience and knowledge of CPRD, the provision of healthcare in the UK and the process for conducting systematic reviews.

However, limitations of the review need to be acknowledged. Firstly, this review focused only on guidelines and guidances which used evidence from studies using CPRD data. There are other longitudinal databases in the UK such as The Health Improvement Network (THIN) and QResearch. It also did not look into what type of data was used in each study (e.g. use of linked data). For future studies, it would be interesting to investigate whether evidence from other longitudinal databases have been used to inform guidelines/guidances. This study also focused on guidelines and guidances published for health and social care in England. Future studies could compare how RWD is used not only nationally but also internationally and identify the trends and differences that may exist. Lastly, guidelines and guidances which were currently ‘under review’ were not included in the review as they were liable to change once published. Therefore it would be worth conducting the review again at such a time when these guidelines/guidances become available to see how they affect the current results.