Characteristics of included studies

For a full description of the included studies, see Additional file 2. Studies were undertaken in a wide range of countries (145 studies in over 59 countries,). A significant proportion (n = 33, 23%) were from low- and middle-income countries in regions such as sub-Saharan Africa and Central America (n = 32), and several were conducted in Middle-Eastern states (n = 4).

Eleven studies used observational (ethnographic) methods to collect data, and 37 used documentary analysis. However, these represent less than a quarter of included studies, the majority of which were or included semi-structured interviews (n = 79), or included a survey (n = 44). Twelve studies were longitudinal while the rest were cross-sectional. Thirteen systematic reviews and fifty-three case studies were included.

Most studies reported perceptions or experiences of respondents (n = 109; n = 64 respectively), rather than documentary proof or observational results about the use of evidence in policy (n = 14; n = 11 respectively). Evidence’ was defined in 121 studies. Where it was possible to identify what kinds of evidence were being discussed, most focused on the use of research evidence (n = 90) with 33 focusing specifically on systematic reviews. However, 59 studies looked at the use of non-formal evidence, which included local data, surveillance data, personal experience, clinical expertise, or other informal knowledge.

The context of the study was usually non-specific, referring to general policy (n = 84) or practice (n = 37). Changes to specific policy areas or policies were explored in 22 and 14 studies respectively, and information/evidence diffusion in 13. Some studies explicitly set out to look at uptake or adoption of research (n = 41), and others described interventions aiming to increase uptake [14], or the context after a specific piece of research or policy (such as after the introduction of the 1999 White Paper “Saving Lives: our healthier nation” [15, 16]. The vast majority of studies were conducted in health or health-related fields. Most new evidence in the area focused on the health sector, but research was also conducted in areas including traffic [17, 18], criminal justice [19–23], drugs policy [22, 24–37], and environmental conservation [20, 22, 38–42] (see Figure 2).

Who are these studies written by and for?

137 study reports were written by researchers or people with academic affiliations, with clinical researchers co-authoring a proportion of these (n = 57). Policymakers were credited as authors in 3 studies, [25, 39, 43] and one of those was a governmental report.

The population samples themselves were predominantly policymakers or advisors (n = 86, n = 32 respectively), health care managers (n = 49), or researchers (49), although many other groups were also included (see Figure 3). Where researchers were included in the study population (n = 49), they often outnumbered the policy and practice participants. Other participants included commissioners, health economists, third sector workers, patients, industry and business representatives, and justice and criminal workers. Because it was not always clear who had been involved and what their roles were, it was not possible to give numbers for all these groups. Also included in this other’ category (n = 62) were all documents analysed.

What factors affect use of evidence?

All studies reported either barriers, facilitators, or both, of the use of evidence. Studies also described processes of research use (n = 50), strategies and interventions to increase research use (n = 24), and assessments of the uptake of research (n = 33) (see Figure 4).

Contact and relationships

Contact, collaboration and relationships are a major facilitator of evidence use, reported in over two thirds of all studies. Timing and opportunity was the most prominent barrier (n = 42) within this theme. Many studies also discussed the role of relationships, trust, and mutual respect. The serendipitous nature of the policy process was emphasised in some studies, which discussed the role of informal, unplanned contact in policy development and in finding evidence.

Organisations and resources

Organisational factors such as lack of access to research, poor dissemination and costs were highly reported factors affecting the use of research. Other barriers were lack of managerial support, professional bodies, material and personnel resources, managerial will and staff turnover. Professional bodies were seen as barriers where useful guidelines were not available, or where they were perceived to be political or biased. In the case of the WHO, it was seen as unreliable, unsupportive, and with dubious claims to be evidence-based’ [31, 44]. Other factors mentioned in connection with organisational and resource barriers included poor long term policy planning [45], inflexible and non-transparent policy processes [46, 47] and in developing countries, lack of effective health care systems [24]. Leadership and authority were reported as facilitators, with emphasis on community leadership [48] and policy entrepreneurialism of policy champions [43, 49].

Among the facilitators under the theme Organisation and Resources, availability, access and dissemination were considered important facilitators, as was managerial support (n = 22).

Research and researcher characteristics

Characteristics of research evidence were widely reported as factors affecting uptake of research, with clarity, relevance and reliability of research findings reported as important factors. The format of research output was also an important factor in uptake. The importance of the research findings themselves was discussed in 19 studies, usually studies describing the uptake of health inequalities research. The quality and authoritativeness of research was clearly a factor in uptake, particularly where other evidence in the area was poor quality [50].

Emerging as a new stream of research, eleven studies evaluated or described knowledge broker roles or related concepts [6, 35, 37, 51–57] with dedicated dissemination strategies evaluated in 7 studies and mentioned as a facilitator in 43. Incentives to use evidence and client demand for research evidence were described as facilitators in one study each [10, 58].

Researchers themselves were described as factors affecting uptake of their research. Having a good understanding of the policy process and the context surrounding policy priorities was supportive of research uptake [17, 18, 59–61]. A barrier to uptake was identified where researchers were described as having different priorities from policymakers, with pressure to publish in peer-reviewed journals [27, 62, 63]. Researchers were valued more when it was clear they were non-partisan and producing unbiased results [40, 57, 64], and provision of expert advice was also reported as helpful.

Policymaker characteristics

Policymakers’ characteristics were also reported to play a role in evidence uptake, with their research skills and awareness (or lack of) reported as a barrier in 34 studies. Some studies reported that policymakers’ beliefs about the utility of evidence-use was a major factor in evidence use (barrier: n = 2, facilitator: n = 3), and, in general, personal experiences, judgments, and values were reported as important factors in whether evidence was used. However, these findings were nearly all (91%) based on studies of perceptions, of which half were perceptions of researchers.

Some studies reported that left-leaning, younger and/or female policymakers were more likely to use research evidence [65, 66]. Being more highly educated was reported as a barrier [67], but there was no consensus about the effect of being clinically trained [61, 68].

Policy characteristics

Perhaps surprisingly, legal support and the existence of guidelines for the use of evidence were scarcely reported as factors affecting uptake of evidence. The importance and complexity of the policy area was also discussed, especially in comparison with the relative simplicity of clinical problems.

However, competing pressures (economic, political, social, and cultural factors) were seen to impact on the policy process and hinder the development of evidence-based policy. Political pressures, finances, and competing priorities were all discussed (n = 12), with the media (n = 3) vested interest and pressure/lobby groups (n = 3) and unclear decision-making practices (n = 2) also reported as barriers.

Other factors

One study which studied use of evidence in prisons reported potential security breaches from data loss as a potential barrier to evidence use [19]. Other studies reported consumer-related barriers (such as issues around privacy and choice [49, 55, 69]), differences between types of policymaker (such as civil servants vs. managers) [29, 70] and public opinion. External events were reported as a facilitator in one study. The role of local context, contingency, and serendipity in influencing policy processes and outcomes overall emerged as a theme throughout the results.

Comparing with Innvaer (2002): focus of new evidence in the area

Strengths and weaknesses of the study

The review is exhaustive, and we followed a pre-published protocol and rigorous review methods, including the advice of an advisory group (details available from the corresponding author) (see Additional file 3 for a PRISMA checklist report). However, this paper has only counted the frequencies with which factors are mentioned without any weighting. Without more research, it is difficult to say what impact different factors might have.

Most studies still employ relatively superficial methods such as surveys or short interviews. These were all based on self-reports, however, so given the contentious nature of the topic combined with understandable fear of audit/performance monitoring these results may not be reliable. However, there is some evidence that researchers are employing impact assessment, intervention, or observational studies as well to explore how evidence and policy are related. We were unable to double-screen and double-code all studies due to lack of resources. However, all studies were data-extracted at least twice (once at the beginning, and again with the finalized list of factors which was developed iteratively) so we have confidence in the consistency of approach. No methodological assessment of included studies was undertaken, as this was primarily a descriptive exercise. In addition, the heterogeneity of study designs and the difficulty of comparing quality across these domains limited the usefulness of such an exercise. Quality appraisal would be a valuable step in any in-depth review of a subset of these studies.

Strengths and weaknesses in relation to other studies

Recent systematic reviews in the area have focused on the use of research evidence, [74] or on the impact of research evidence on policy [5]. Orton [75], included in this review, does not include any evaluations of evidence use, ethnographies, or case studies, relying only on self-report questionnaires and interviews to provide the results. Without empirical data exploring access to information and perceived impact [74], and without investigating the policy process, or testing current theories about knowledge utilization, it is hard to draw useful conclusions. Few studies have systematically appraised the use of evidence in this wider sense.

The reviews all found similar findings with regard to barriers and facilitators of the use of evidence. There still appears to be a need for high-quality, simple, clear and relevant research summaries, to be delivered by known and trusted researchers.

Possible change in future research practice and policymaking

Most studies in this review are descriptive. Because most studies do not go into the content of the facilitators and barriers they identified, we know little about when, why and how the identified barriers and facilitators come into play in the use of evidence in policymaking. Based on this review, future research can use Table 2 to identify themes and factors relevant for their field of research, be it organisations, collaboration, research, researchers, policymakers or policy. Identifying the content and relative importance of these factors and new undiscovered factors in different contexts, at different levels, or in different countries, may contribute to our understanding of evidence use in policy.

One future objective for researchers can be drawn from the results found in Table 1, namely that four of the five top barriers to the use of evidence is a lack of relevance and importance. If research becomes available, the possibility of increased use improves. If policymakers’ research skills improve, calculations of costs will become more accurate. The natural question is to explore why policymakers do not prioritize overcoming barriers relating to themselves. The barrier called lack of clarity, relevance and reliability of research calls for change in researchers’ objectives and methods, but we need to know what policymakers define as clear, relevant and reliable research, and why and when policymakers will use such research. Of special relevance to this question, is the research on knowledge translation done in the past five to ten years, which formed a new strand of research. This body of work draws on the theory that interpersonal relations are important for knowledge exchange, through employing knowledge brokers or similar. There has also been a growth in resources aimed at helping decision-makers to navigate research evidence, such as Cochrane-produced evidence summaries. These are not only aimed at practitioners within the health field, and the knowledge translation field will hopefully soon make efforts at addressing the broader issues around evidence use in policy more widely to identify underlying mechanisms behind knowledge use.

“What this paper adds” box