Skip to main content
  • Research article
  • Open access
  • Published:

Creating performance intelligence for primary health care strengthening in Europe



Primary health care and its strengthening through performance measurement is essential for sustainably working towards universal health coverage. Existing performance frameworks and indicators to measure primary health care capture system functions like governance, financing and resourcing but to a lesser extent the function of services delivery and its heterogeneous nature. Moreover, most frameworks have weak links with routine information systems and national health priorities, especially in the context of high- and middle-income countries. This paper presents the development of a tool that responds to this context with the aim to create primary health care performance intelligence for the 53 countries of the WHO European Region.


The work builds-off of an existing systematic review on primary care and draws on priorities of current European health policies and available (inter)national information systems. Its development included: (i) reviewing and classifying features of primary care; (ii) constructing a set of tracer conditions; and (iii) mapping existing indicators in the framework resulting from (i). The analysis was validated through a series of reviews: in-person meetings with country-nominated focal points and primary care experts; at-distance expert reviews; and, preliminary testing with country informants.


The resulting framework applies a performance continuum in the classical approach of structures-processes-outcomes spanning 6 domains – primary care structures, model of primary care, care contact, primary care outputs, health system outcomes, and health outcomes – that are further classified by 26 subdomains and 63 features of primary care. A care continuum was developed using a set of 12 tracer conditions. A total of 139 indicators were mapped to the classification, each with an identified data source to safeguard measurability. Individual indicator passports and a glossary of terms were developed to support the standardization of the findings.


The resulting framework and suite of indicators, coined the Primary Health Care Impact, Performance and Capacity Tool (PHC-IMPACT), has the potential to be applied in Europe, closing the gap on existing data collection, analysis and use of performance intelligence for decision-making towards primary health care strengthening.

Peer Review reports


Four decades following the adoption of the Declaration of Alma-Ata, the evidence base for primary health care (PHC) as the most inclusive, effective and efficient approach to make progress towards universal health coverage (UHC) and enhance population health [1,2,3,4] has solidified a PHC approach as the ambition of countries worldwide [5]. In spite of progress made to strengthen PHC, in 2019 the global health community is confronted with the work still to be done. From a European perspective this includes widening inequities and gender differences for noncommunicable disease (NCD) outcomes, the substantial burden of mental illness, rapid population ageing and the global threat of antimicrobial resistance [6,7,8]. The region also faces persisting quality deficiencies, increasing vulnerable groups, and impoverishing levels of out-of-pocket payments [7, 9, 10].

With the 2030 Sustainable Development Goals (SDGs) on the horizon there is renewed impetus and political commitment for PHC strengthening as an accelerator towards UHC [5]. For this, measuring the performance of primary care services has a fundamental role.

A PHC approach from a services delivery perspective can be characterized as primary care: the continuum of first contact promotive, preventive, diagnostic, curative, rehabilitative and palliative care services delivered across the life-course [5]. How countries select, design, organize, manage and improve their primary care services is heterogeneous. As a result, there is considerable diversity in the scope of primary care services, types and profiles of health professionals like nurses, general practitioners, social workers, public health professionals, narrow specialists, paediatricians or occupational therapists, and settings of care like single offices of general practitioners, group practices, multi-profile teams or polyclinics.

Without primary care performance measurement sensitive to these differences, countries often lack the necessary information to monitor and evaluate their options for improvement. Despite the numerous initiatives to strengthen primary care measurement [11,12,13,14,15], the lack of comparable data on primary care in Europe continues to limit performance intelligence for decision-making. Of the factors contributing to this we highlight three.

First, there is no single approach to provide basic, up-to-date information on the organization and delivery of primary care. This is in contrast to internationally comparable information on financial resources (e.g. System of Health Accounts) or professional classifications (e.g. International Labour Standards) that define a more standardized approach. Importantly, even these standards face constraints to capture primary care, such as for making international comparisons on its costs and workforce. The challenge of comparability is especially relevant in the context of the WHO European Region as member countries range from western, eastern and southern Europe, the Baltic countries, central Asia and the Caucasus.

Second, frameworks defined for global use are strained to measure variations for health outcomes that matter most to European countries. The World Health Organization’s (WHO) global UHC service coverage index illustrates this point. According to 2017 reporting, nearly 40% of European countries had an average coverage score of 77 or more on the index; the highest globally [16]. However, the inclusion of tracer conditions and services like malaria and sanitation limits the sensitivity of the index to high- and middle-income countries. Global frameworks are also strained to capture European policy priorities, like the importance the region’s member countries have weighted to people-centred services [17] and measurement of patient experiences [18, 19].

Third, most primary care frameworks and performance assessments have weak links with routine information systems and national health priorities [20]. This is despite the wide uptake of electronic information systems and health records in primary care across Europe [21, 22]. It means primary care monitoring efforts have yet to fully leverage and integrate existing data infrastructure to best support evidence-informed decision-making [23, 24].

This paper describes the development of a new tool for monitoring PHC performance across the 53 member countries of the WHO European Region. Our research was guided by the aim to create robust performance intelligence in Europe that captures the ability of primary care to respond to population health needs. Specifically, this work responds to the policy commitment of the WHO European Region member countries enacted in 2016, calling for intensified regional monitoring on health services delivery [25], and is supplementary to global monitoring efforts, like monitoring UHC [16] and foreseen monitoring framework for the implementation of the Astana Declaration [5].


The following details the processes we undertook between mid-2016 and 2018 in a three-staged approach: first, reviewing and classifying features of primary care, second, constructing a set of tracer conditions and third, mapping existing indicators in the framework devised in the first stage.

Targeted literature review of features of primary care

As a starting point, we reviewed the literature for characteristics of primary care in existing frameworks, tools and surveys. We took as a basis a study by Kringos et al. (2010) being the most comprehensive review on the core dimensions of primary care to-date [26]. To update the review, we extracted priority areas and strategies of a contemporary European policy: the WHO European Framework for Action on Integrated Health Services Delivery [25]. The policy was developed and endorsed by countries and aligns with current European policies [18, 27] making it highly attuned to policy priorities in the region.

A literature review search strategy was developed to target recently published scientific and grey literature on frameworks and tools for health services delivery in general, and primary care in particular [28]. We conducted initial searches between October 2016 and May 2017 using PubMed to identify scientific literature published since 2010. We brought the existing systematic review up-to-date by searching new key terms including: avoidable hospitalization; chronic disease management; community-based care; drugs and medical devices; financial incentives; information systems; integrated care; job satisfaction; patient-centredness; patients with complex needs; population health management; responsible use of medicines; role of nurses; shared care plans; task-shifting; technology assisted care; unmet need; waste and appropriateness of care; and workload.

We hand-searched websites and databases of key international actors active in monitoring PHC, namely WHO using WHOLIS, the World Bank, Organisation for Economic Co-operation and Development (OECD) and European Commission and collated sources already known to the authors. Reference lists of relevant work identified were also reviewed and titles were searched in a snowballing approach. With the exception of global frameworks and a recent PHC initiative for low- and middle-income countries [13], work from the European context was prioritized.

Two authors (EB,DK) completed the initial document review. For each framework identified, the features of primary care, their respective classification and key definitions were extracted and logged in an Excel spreadsheet. The authors jointly carried out an analysis of the review findings to identify crosscutting themes.

Review of tracer conditions

To tailor the framework to the European context, we used the method of tracer conditions [29], like applied to monitoring UHC globally [16, 30]. Tracer conditions have been used in health services research on the premise that a carefully selected set of health problems makes it possible to profile the strengths and weakness of services delivery and health systems [31]. On this basis, we sought to construct a set of tracer conditions to inform the selection of indicators that–when analysed together– could serve to gauge the ability of primary care to respond to a range of health needs individually and concurrently as multimorbidities, while also measuring across population groups and life stages.

The selection of tracer conditions prioritized the following: relevance to the burden of disease in Europe; responsiveness to the strength of primary care; and representativeness of primary care’s functions. The final selection also gave consideration to the measurability of conditions and the parsimony of the set, weighing together the balanced representation of different target populations and life stages, gender importance and types of services. The overall manageability of the set was also prioritized for the selection of a core group of tracers that could serve the purpose of scoping the tool to high-priority health improvement areas.

Current global and European health policies were reviewed by two authors (EB,IK) as a proxy for priority health improvement areas. To prioritize conditions amendable to primary care, European lists of ambulatory care sensitive conditions were consulted [32], together with priority conditions included in an earlier study on the strength of primary care in Europe [12]. To achieve a comprehensive and holistic view to primary care, the type of condition (e.g. acute, chronic), relation to the life-course, gender importance and function of primary care (e.g. prevention, detection, treatment, management) were also considered. For each possible tracer condition, findings for the criteria considered were recorded.

Identifying existing indicators

To identify existing, internationally standardized indicators, we searched by features of the framework resulting from the first stage, in databases of international organizations (WHO, OECD, European Commission) and topic-specific databases of research consortiums e.g. cancer, medicines and tuberculosis. We reviewed existing global and European surveys related to health services delivery, patient-reported experience and outcome measures or conditions amendable to the strength of primary care. Standardized country reports by international organizations were also reviewed through searches on their respective websites. Indicators from the initial literature review and health strategies used to select tracer conditions were also extracted.

To select indicators for the framework, we prioritized the following:

  • measurability through an existing or feasible data source;

  • available internationally standardized indicators and survey questions;

  • relevance to the European context;

  • balanced coverage across the framework and its classification; and,

  • balanced representation of perspectives e.g. patients, practitioners, policy-makers.

We prioritized the first criterion on data availability in our selection to align to information systems and make use of the vast amount of available data, also minimizing the burden of new data collection. We undertook a detailed process of identifying sources by drawing on an existing study on health information systems [22]. We further expanded upon this to scan the availability of health services delivery data across Europe. The findings from this scan have been published elsewhere [21] and were used to ensure existing data sources are drawn from. Alternative sources where applicable, such as national documents (e.g. health policies, directives or prikaz, guidelines) or key informants, were also considered as part of asserting the measurability of indicators to merit their selection.

Indicators were mapped to the classification resulting from stage one. For the purposes of this mapping, indicator passports were developed to clearly define the scope and measurement of each individual indicator. The indicator passports draw from the literature reviewed and existing international standards where available to detail the following: alignment to the framework (domain, subdomain, feature), indicator/question title, indicator/question definition, numerator/denominator or answer choices, unit of measurement, rationale, relevant definitions, disaggregation, known limitations and possible data sources. A glossary of terms accompanies the indicator passports with emphasis put to capturing different terminology used in the European countries. The terms and definitions draw from existing international classifications including the International Classification for Health Accounts, International Standard Classification of Occupations and International Standard Classification of Education.

Validation of the framework, tracers and indicators

Country-nominated focal points representing ministries of health, health insurance funds, centres on health services or similar from 30 countries assessed face and content validity of the framework and selection of tracer conditions at a meeting in June 2017 [33]. The set of tracer conditions was also presented and validated with country representatives. They were asked to consider the relevance and comprehensiveness of the framework and tracer conditions in relation to their systems. In the same period, the framework was presented to members of WHO’s European Primary Health Care Advisory Group – a group of appointed experts to support the continued advancement of PHC [34]. All comments and discussion points were recorded and adjustments to features were made for a revised version of the framework that was then applied for the mapping and selection of indicators.

To review the indicators identified, we engaged more than 40 experts between November 2017 and June 2018. Reviewers spanned three profiles: (i) staff of WHO; (ii) experts in relevant fields from academia, think tanks and international organizations; and (iii) representatives of professional and patient associations, as well as practising clinicians.

Attention was put to ensure that reviewers were representative of countries across the region and included a range of language skills – with approximately one quarter (23%) being native Russian-speakers – and gender balance (49% females, 51% males). Nearly half of reviewers (42%) were trained medical doctors. Reviewers also included information specialists on European and central Asian countries.

Each reviewer received a written request for their feedback on a subset of indicators. Reviewers were provided the indicator passport, background on the framework and selection criteria. They were asked to score the indicator’s overall quality, provide comments or amendments and/or suggest an alternative indicator or source. One author (IK) consolidated the feedback from all reviewers. Indicators that were rated of poor quality, too vague or not meaningful for analysis were removed. For others, comments were used to revise the indicator passports. Comments included important feedback for updating indicators to current international standards as well as making explicit the limitations of indicators where known and for identifying alternative sources of data if available.

Further to technical reviews, we also conducted a preliminary test of the framework and indicators as an initial validation of its use in practice. The indicators were translated to Russian and applied in Kazakhstan on the basis of testing their applicability in the context of a Russian-speaking country and the characteristics and cultural nuances of health systems in central Asia and Caucasus countries. This process extended from December 2017 to June 2018 and included a series of workshops with national centres responsible for data collection. Further revisions were incorporated into the indicator passports and the glossary of terms.


The findings across the three stages of our research process are summarized in Fig. 1 and described to follow. Supplementary files of the literature reviewed, selection of tracers, individual indicator passports and glossary of key terms is available electronically [35].

Fig. 1
figure 1

Summary of findings

Classifying features of primary care

We found a total of 50 different frameworks, assessment tools and survey instruments, recorded in Additional file 1 [35]. From the literature reviewed, we identified approximately 50 domains and 100 features of primary care. We further analysed these findings to identify crosscutting themes and to cluster related features. We noted several areas of clear consensus in the literature. For example, the contribution of primary care to first contact access, comprehensive services and continuity and coordination of care is recognized across frameworks [12, 14, 33, 36].

We developed a hierarchy for the classification of findings, adopting the classical framework of Donabedian’s structure-process-outcome logic model [37]. These components were classified and sequenced as the capacity, performance, and impact of primary care. We defined a taxonomy to describe these components from broad to specific as domains, subdomains, and features [35].

Under primary care capacity, we put attention to disaggregate the unique yet often over-looked resource needs of primary care, including the primary care workforce, medicines, diagnostics, facility infrastructure, and information system. We were also attentive to differentiate the system’s capacity, as the enabling system structures underpinning primary care, from the function of services delivery. We classified this as the model of care: the result of deliberate decisions taken that determine the contents, design, organization, management and quality improvement elements of services delivery.

This classification is a point of departure from existing frameworks. It captures the less prevalent ‘software’ of primary care’s capacity [13], such as managerial autonomy for determining staffing, budgeting and strategic planning in primary care facilities, prescribing authority of general practitioners to initiate or refill prescriptions and the existence and scale of quality improvement mechanisms, like practice audits, patient complaint systems or peer review teams. The taxonomy around the selection of services spans from identifying needs, to the selection of preventive care, diagnostic procedures, treatment and disease management services as well as services for supporting self-management. The model of care also bridges between the system on one side, and the provision of services and perspective of patients, on the other.

This domain, at the intersection with the performance of primary care, is captured as care contact. It is novel in distinguishing the ‘structural’ from its implications on performance. For example, we find skill-mix and multidisciplinary teams an agreed upon feature of primary care [12, 14, 33, 38, 39]. However, it has typically been classified as a feature of coordination. We argue the setup and structure of teams captures how providers are organized and their resulting level of interaction and joint-work rather as a measure of coordination.

Importantly, while in the reviewed literature there was clear consensus on the policy importance of capturing the perspective of patients, to a lesser extent had this priority area been translated into monitoring frameworks. The domain of care contact collocates patient experience as a core feature of overall primary care performance and is found an important distinction from earlier classifications.

In line with existing system frameworks, outputs via services delivery (access, responsiveness, safety, effectiveness) and health system outcomes (quality, equity, efficiency) were classified from a services delivery perspective, linking to the final component of impact on health outcomes (health status and well-being). Importantly, in distinguishing between outputs from outcomes, we recognize that suboptimal outcomes can be attributed to features of the health system beyond the scope of primary care services and have labelled it as such. Figure 2 illustrates the resulting framework. Like other frameworks, we acknowledge that primary care performance lies within larger socio-political contexts, though these factors are outside the scope of health systems [12, 19, 36].

Fig. 2
figure 2

Monitoring framework

Adopting the approach of people-centred systems, the framework begins with health outcomes. In this way, health outcomes are the lens through which capacity and performance are monitored. By cascading the framework’s domains, features and subdomains as shown, the taxonomy applies system thinking principles to associate connections between capacity, performance and impact. Importantly, as the field of systems thinking has revealed, changes to systems can have unpredictable and multi-direction results [40].

Selecting tracer conditions

Fourteen disease-specific strategies actively being implemented in Europe and globally were identified and reviewed (Additional file 2). We screened 32 conditions, identified first and foremost as priority health improvement areas in Europe and selected a set of 12 grouped in 7 clusters based on our selection criteria. These conditions span: reproductive, maternal, neonatal and child health; communicable diseases; cardiovascular diseases; diabetes; respiratory diseases; cancer; and mental health (Table 1).

Table 1 Selected tracer conditions

The selection process included a review within and across clustered to assess the conditions in combination in order to gauge the balance across the criteria applied and the manageability of the set. Guided by our primary aim to select a core set of tracers for profiling primary care, we prioritized one or two conditions per cluster, with one exception (cancer). In a last stage, we also considered the measurability of conditions in primary care [21], resulting in the exclusion of those specific to ageing, like dementia. Nonetheless, ensuring the representativeness of varied population groups and life stages was among the core selection criteria applied. For example, a range of primary care services for older adults are included such as, influenza vaccines (prevention), colorectal cancer (detection, management) and cardiovascular diseases (prevention, detection, treatment, management).

Mapping primary care indicators

More than 20 international databases were reviewed to extract existing and reported on indicators. Fifteen global and European surveys were also reviewed on topics including medicines and e-health, patient experience, primary care doctors, and on tracer conditions like influenza and NCDs. An additional 8 standardized country reports were identified e.g. Health Systems in Transition series, OECD country reviews and WHO country assessments.

From this, we consolidated a total 243 indicators/questions. We plotted the indicators in our classification of subdomains/features. When mapped, we found the largest number of indicators for the structure domain (health workforce, financing) and health impact (morbidity, mortality), consistent with their wide use in health system performance frameworks [20]. The model of care domain was the least populated from this initial mapping and further targeted reviews were conducted to identify indicators for its features. In a final stage of development, we reassessed the initial classification with the indicators identified for refinements at the feature-level. Following this, three features remained without assigned indicators: acceptability, equity, and responsiveness.

We applied the tracer conditions to scope the selection and explicitly link indicators across the framework using theory of change principles as described [41]. In doing so, health outcomes are preceded by related performance and capacity indicators. For example, impact indicators on diabetes link to preceding diabetes-related performance indicators e.g. hospitalizations, managed insulin-levels, and capacity indicators e.g. prevention services for diabetes, existence of patient registries. Indicators that measured conditions outside the scope of the selected tracer conditions were excluded. A balance in the number of indicators for each tracer condition was also sought. We retained relevant disaggregations, such as public-private mix and rural-urban status for analysis purposes. The full set of indicator passports is available electronically in Additional file 3 and related glossary of terms in Additional file 4 [35].

Final selection and new tool

We consolidated our findings in a tool we refer to as the Primary Health Care Impact, Performance and Capacity Tool (PHC-IMPACT). The tool includes a total of 139 indicators mapped to a detailed framework in a hierarchy of domains (6), subdomains (26), and features (63) as summarized in Table 2 and expanded in Table 3.

Table 2 Overview of final framework and suite of indicators
Table 3 Overview of framework for monitoring the impact, performance and capacity of PHC, indicators and data sources

The measurability of each indicator has been safeguarded through an extensive process of identifying a possible source for each. More than half (61%) of the indicators can be sourced from more than one type of data source, increasing the potential measurability across countries. The range of possible data sources include: national and international databases (43 indicators); existing surveys (11 indicators); policies (27 indicators) and reports (10 indicators); and key informants (62 indicators) (Table 3).


Creating performance intelligence with PHC-IMPACT

This research set out with the aim to create robust performance intelligence for PHC strengthening in Europe. Through the three-staged research process described, the tool has been tailored for a classification, set of tracer conditions and selection of indicators that are sensitive to primary care, policy priorities and information systems in Europe.

The broad suite of indicators is intentional in order to allow the possible tailoring of the indicators, functioning as a menu of options to be selected on the basis of a country’s policy priorities. The customization of the tool is among its core advantages and an important feature to increase the tool’s responsiveness within and transferability across countries. Other unique features are found to include the following: diversified data sources accommodate a range of perspectives for a more holistic view to primary care; the translation and means testing of the tool in Russian attempts to redress context-specific classification challenges unique to the membership of the WHO European Region; the cascading of the taxonomy developed facilitates varied entry points to analysis and a uniquely detailed approach to capture the model of primary care; and, the indicator passports and glossary of key terms developed are a practical resources for data collection.

Prior to the use of the tool, a review of the indicators for further tailoring to the context of a specific country should be conducted. The selection of tracer conditions should also be reviewed, with the possibility to adjust this list given a country’s health priorities. This process of customizing the tool’s suite of indicators should rely on the meaningful engagement of key stakeholders for the full benefits of co-designing with the intended users.

Addressing classification limitations

The relevance of existing indicators for use in monitoring primary care was a significant limitation. For example, nurses in primary care are not defined in existing databases, surveys or reporting, ultimately limiting the extent to which the number, profile, role and performance of nurses in primary care can be assessed. We have attempted to redress these limitations with adjustments to standard indicators using existing definitions and classifications to improve their sensitivity to primary care. In particular, we have harmonized the varied terminology for the primary care workforce, types of facilities and levels of education that have previously limited the relevance of frameworks and tools across Europe. The glossary of terms developed supports the classification defined [35].

We also found metrics for equity and responsiveness to be limited. A similar finding was reported in a recent review on health system performance assessment frameworks [20]. We recognize the need to operationalize both in a more comprehensive manner, suggesting the use of composite indicators and highlight acceptability as a feature in need of further development. The disaggregation of indicators existing in international databases was retained where possible and includes gender, age, socioeconomic status and rural–urban classifications. These variables are found especially relevant for the analysis of equity. An approach that links the relevant indicators and disaggregations is suggested for a more holistic appraisal of equity considerations.

The tool is found novel in its attempt to capture the capacity of primary care beyond system structures, with indicators like prescribing authority to assess the ability of general medical practitioners to issue initial prescriptions and/or refills for treatment and the autonomy of managers on planning, staffing and budgeting. These features of the model of care, together with the domain on care contact, are found important specifications for depicting primary care that allow for further analysis and comparisons across countries.

To further improve the sensitivity of the tool, answer choices where applicable were disaggregated to capture responses as country-wide, in some regions or pilot status.

Aligning to information systems and other sources of data for applying PHC-IMPACT

Collecting data should rely as much as possible on existing international databases, surveys and country reporting. The development of an electronic data processing system is already being explored to build linkages to existing databases and create a common living platform for interactive analysis. For other indicators, there is untapped potential to uptake data from national information systems, in particular on hospitalizations for ambulatory sensitive conditions where reporting for many non-OECD, non-EU countries of Europe is out-dated. For these countries, strengthening linkages with national systems should be prioritized.

An electronic questionnaire is being built to serve as a data collection tool for qualitative indicators. Where key informants are a data source, we suggest diversified informants to capture varied profiles (policy, managerial, clinical) for accurate responses on the tool’s range of topics. By soliciting a range of informants with complementary knowledge, this approach also facilitates multi-stakeholder engagement while ultimately yielding more objective and reliable results.

A modular approach could also be taken to data collection for a focus on topics like mental health, maternal and child health, and out-of-hours services based on a country’s priorities. While the tool has prioritized available data, the extent to which indicators are measurable across countries varies making some features more ambitious and future-oriented in some contexts. In instances where preferred databases or survey sources are not available, and the indicator cannot be answered by one informant, we suggest a pragmatic approach is taken using the method of expert consensus. This method is applicable for one quarter (27%) of the total indicator set. Drawing from established group-based methods, a Delphi technique followed by a consensus workshop could be used to generate estimates. The highly structured method preserves anonymity while capturing a range of perspectives. This data could support meaningful analytics as countries aspire to and work in parallel on advancing the use and alignment of data in their national information systems. Lessons on expert consensus methods in health services research could be explored for use here [42,43,44,45].

Policy relevance and research implications

PHC-IMPACT has been designed with view to the information needs of decision-makers in the WHO European Region both for international monitoring and country-specific priority-setting. The merit of a regional approach to monitoring has already been advocated for SDG 3 in the scope of financial protection [46]. We have excluded conditions or services considered a basic expectation or fundamental to primary care, such as family planning and childhood vaccination. The exclusion of these services is a prioritization of the tool’s sensitivity to disaggregate performance in high- and middle-income countries rather than a statement of their importance. Moreover, disease- or service-specific monitoring tools and instruments are in use for this purpose.

The comprehensiveness of the tool’s taxonomy brings depth to the analysis for country-specific use. It is seen as a vehicle for identifying priority areas for improvement, while also shedding light on the information landscape and overall availability of data and actors involved. For analysis purposes, the tool facilitates linkages along different continuums: a clinical continuum, linking prevention, diagnosis, treatment and disease management for tracer conditions; a performance continuum in the classical structure-process-outcome sequence; and a continuum of stakeholders cascading the delivery of services from the micro-level (health professionals, patients), to meso- (managers, regional health authorities) and macro-level (policy-makers, health insurers). These relations between indicators have strong analytical potential to improve the coherence of PHC strengthening and signalling of policy opportunities to accelerate performance gains.

For example, the availability and provision of services in primary care can be assessed across to the care continuum to gauge the extent to which the full continuum of services are available and if not, where gaps in services provision lie (i.e. lack of risk assessment services for cardiovascular diseases or mental health; limited role of primary care in follow-up for tuberculosis. This analysis draws on the indicators related to the selection of services. In a similar approach, the role of different primary care practitioners can be profiled, clustering the provision of services by general medical practitioners, nurses, or specialists working in primary care, among others. This analysis has the potential to shed light on the different roles and scope or practice of primary care practitioners, including important insights into the potential for general medical practitioners to confirm an initial diagnosis or role nurses in risk assessment or follow-up services in primary care. Other thematic clusters of indicators could aggregate indicators by policy-relevant themes such as patient engagement, out-of-hours primary care, integrated health and social care, prescribing practices in primary care, among others still.

The tool should be treated as a living resource to be adjusted and improved upon as new types and sources of data become available. This includes the continuous improvement and development of indicators that are defined according to global standards such as total primary health care expenditure and the UHC services coverage index. It requires piloting beyond the initial country-validation described to test its approach and robustness for country-specific, cross-country comparisons and use overtime. Ultimately, despite its attempt to be comprehensive, not all complexities can be accounted for.

Further to piloting PHC-IMPACT, areas for continued development include: developing composite indicators for features like equity and responsiveness and priority policy areas like scope of practice; selecting a core set of indicators for use in dashboards for the purposes of international benchmarking; improving metrics for hard-to-measure topics, like medicines, primary care workforce and acceptability; exploring methods of expert consensus; and intensifying data collection from existing sources and for newer indicators like patient reported experience measures in primary care. Patient experiences measures are found an important area for further research and development to allow cross-country comparisons with the necessary adjustments for their wider use in eastern European and central Asian countries. Tools developed in the scope of the Patient-Reported Indicators Survey (PaRIS) Initiative of the OECD are one platform that could be coordinated with and tools adapted from for use in the context of countries across the European Region [47].


Performance intelligence on the ability of primary care to respond to health needs is vital for systems to evolve rather than react to health needs and make sustainable progress towards UHC targets. This paper presents a tool for creating performance intelligence sensitive to the WHO European Region. The tool addresses limitations of existing classifications to better capture primary care, improve linkages with (inter)national information systems, and ensure specificity to high- and middle-income countries. The framework and suite of indicators consolidated have promising analytical power that merits further development through its application.

Availability of data and materials

All data generated or analysed during this study are included in this published article and its supplementary information files [35].



noncommunicable diseases


Organisation for Economic Co-operation and Development


primary health care


Primary Health Care Impact, Performance and Capacity Tool


Sustainable Development Goals


universal health coverage


World Health Organization


  1. Ghebreyesus TA, et al. Primary health care for the 21st century, universal health coverage, and the sustainable development goals. Lancet. 2018;392(10156):1371–2.

    Article  Google Scholar 

  2. Balabanova D, et al. Good health at low cost 25 years on: lessons for the future of health systems strengthening. Lancet. 2013;381(9883):2118–33.

    Article  Google Scholar 

  3. Kringos D, et al. The strength of primary care in Europe: an international comparative study. Br J Gen Pract. 2013;63(616):e742–50.

    Article  Google Scholar 

  4. Pavlic DR, et al. Strength of primary care service delivery: a comparative study of European countries, Australia, New Zealand, and Canada. Prim Health Care Res Dev. 2018;19(3):277–87.

    Article  Google Scholar 

  5. Declaration of Astana. in The Global Conference on Primary Health Care. 2018. Astana, Kazakhstan.

  6. OECD, Health at a Glance: Europe 2018. 2018, OECD: Paris.

  7. WHO Regional Office for Europe. European Health Report 2018 – more than numbers – evidence for all. Copenhagen: WHO Regional Office for Europe; 2018.

    Google Scholar 

  8. European Centre for Disease Prevention and Control, Surveillance of antimicrobial resistance in Europe - Annual report of the European Antimicrobial Resistance Surveillance Network (EARS-NET). 2018, ECDC: Stockholm.

  9. National Academies of Sciences, E.a.M., Crossing the global quality chasm: improving health care worldwide 2018, National Academies of sciences, Endineering and Medicine: Washington, D.C. .

  10. Cylus J, Thomson S, Evetovits T. Catastrophic health spending in Europe: equity and policy implications of different calculation methods. Bull World Health Organ. 2018;96(4):599–609.

    Article  Google Scholar 

  11. Schafer WL, et al. QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care. BMC Fam Pract. 2011;12:115.

    Article  Google Scholar 

  12. Kringos DS, et al. The European primary care monitor: structure, process and outcome indicators. BMC Fam Pract. 2010;11:81.

    Article  Google Scholar 

  13. Veillard J, et al. Better measurement for performance improvement in low- and middle-income countries: the primary health care performance Initiative (PHCPI) experience of conceptual framework development and Indicator selection. Milbank Q. 2017;95(4):836–83.

    Article  Google Scholar 

  14. WHO Regional Office for Europe, Primary care evaluation tool. 2010, WHO Regional Office for Europe: Copenhagen.

  15. The Commonwealth Fund, 2015 Commonwleath Fund International Survey of Primary Care Physicians in 10 Nations. 2015, The Commonwealth Fund: Washington, DC.

  16. WHO and World Bank, Tracking universal health coverage. 2017 global monitoring report. Geneva: World Health Organization and the World Bank; 2017.

    Google Scholar 

  17. Santana M-J, et al. Measuring patient-centred system performance: a scoping review of patient-centred care quality indicators. BMJ Open. 2019;9(1):e023596.

    Article  Google Scholar 

  18. WHO Regional Office for Europe, Health 2020. A European policy framework supporting action across government and society for health and well-being. Copenhagen: WHO Regional Office for Europe; 2013.

    Google Scholar 

  19. WHO Regional Office for Europe, Priorities for health systems strengthening in the European Region 2015–2020. walking the talk on people centredness. Copenhagen: WHO Regional Office for Europe; 2015.

    Google Scholar 

  20. Kekri O, Macarayan E, Klazinga N, Health system performance assessment in the WHO European Region. which domains and indicators have been used by Member States for its measurement?, in WHO Health Evidence Network Synthessi Report 55. Copenhagen: WHO Regional Office for Europe; 2018.

    Google Scholar 

  21. WHO Regional Office for Europe, Availability of national health services delivery data across the WHO European Region. scanning survey results. Copenhagen: WHO Regional Office for Europe; 2018.

    Google Scholar 

  22. OECD, Strengthening health information infrastructure for health care quality governance in OECD. Health Policy Studies. Paris: OECD; 2013.

    Google Scholar 

  23. World Health Assembly. Digital health - draft resolution proposed by Algeria, Australia, Brazil, Estonia, Ethiopia, Germany, India, Indonesia, Israel, Italy, Luxembourg, Mauritius, Morocco, Panama, Philippines and South Africa. Geneva: World Health Organization; 2018.

  24. WHO Regional Office for Europe. Towards a roadmap for the digitalization of national health systems in Europe. Copenhagen: WHO Regional Office for Europe; 2018.

    Google Scholar 

  25. WHO Regional Office for Europe. Strengthening people-centred health systems in the WHO European Region: framework for action on integrated health services delivery in Regional Committee for Europe 66th Session. Copenhagen: WHO Regional Office for Europe; 2016.

    Google Scholar 

  26. Kringos DS, et al. The breadth of primary care: a systematic literature review of its core dimensions. BMC Health Serv Res. 2010;10:65.

    Article  Google Scholar 

  27. United Nations. Transforming our world: the 2030 Agenda for Sustainable Development in Seventieth session Agenda items 15 and 116. New York: United Nations; 2015.

    Google Scholar 

  28. Grant M, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26:91–108.

    Article  Google Scholar 

  29. Kessner D, Kalk C, Singer J. Assessing health quality: the case for tracers. N Engl J Med. 1973;288.

    Article  CAS  Google Scholar 

  30. Boerma T, et al. Monitoring intervention coverage in the context of universal health coverage. PLoS Med. 2014;11(9).

    Article  Google Scholar 

  31. Nolte E, Bain C, McKee M. Population health in Peformance measurement for health system improvement. Copenhagen: WHO Regional Office for Europe; 2009.

    Google Scholar 

  32. WHO Regional Office for Europe. Assessing health services delivery performance with hospitalizations for ambulatory care sensitive conditions. Copenhagen: WHO Regional Office for Europe; 2016.

    Google Scholar 

  33. Shi L, Starfield B, Xu J. Validating the adult primary care assessment tool. J Fam Pract. 2001;50(2):161–4.

    Google Scholar 

  34. WHO Regional Office for Europe. Primary Health Care Advisory Group First Meeting Report. Copenhagen: WHO Regional Office for Europe; 2017.

    Google Scholar 

  35. Barbazza, E., et al, Supplementary files: creating performance intelligence for primary health care strengthening in Europe. 2019:

  36. Primary Health Care Performance Initiative, Primary Health Care Performance Initiative: Methodology Note. 2015, Primary health care performance Initative: Washington, DC.

  37. Donabedian, A., The definition of quality and approaches to its assessment. Explorations in quality assessment and monitoring, 1980.

  38. Pan American Health Organization, Integrated Health Service Delivery Networks: Concepts, Policy Options and a Road Map for Implementation in the Americas. 2011, PAHO: Washington, D.C.

  39. World Health Organization. Framework on integrated, people-cenred health services. Geneva: World Health Organization; 2016.

    Google Scholar 

  40. Adam T, de Savigny D. Systems thinking for strengthening health systems in LMICs: need for a paradigm shift. Health Policy Plan. 2012;27(suppl_4):iv1–3.

    PubMed  Google Scholar 

  41. Seidman G. Does SDG 3 have an adequate theory of change for improving health systems performance? J Glob Health. 2017;7(1):–010302.

  42. Aljamal MS, Ashcroft D, Tully MP. Development of indicators to assess the quality of medicines reconciliation at hospital admission: an e-Delphi study. Int J Pharm Pract. 2016;24(3):209–16.

    Article  Google Scholar 

  43. Aronson BD, Janke KK, Traynor AP. Investigating student pharmacist perceptions of professional engagement using a modified Delphi process. Am J Pharm Educ. 2012;76(7):125.

    Article  Google Scholar 

  44. Campbell SM, Cantrill JA. Consensus methods in prescribing research. J Clin Pharm Ther. 2001;26(1):5–14.

    Article  CAS  Google Scholar 

  45. Black N, et al. Consensus development methods: a review of best practice in creating clinical guidelines. J Health Serv Res Policy. 1999;4(4):236–48.

    Article  CAS  Google Scholar 

  46. Yerramilli P, Fernandez O, Thomson S. Financial protection in Europe: a systematic review of the literature and mapping of data availability. Health Policy. 2018;122(5):493–508.

    Article  Google Scholar 

  47. OECD. Patient-reported indicators survey. Paris: OECD; 2017.

    Google Scholar 

  48. Allemani C, et al. Global surveillance of trends in cancer survival 2000–14 (CONCORD-3): analysis of individual records for 37 513 025 patients diagnosed with one of 18 cancers from 322 population-based registries in 71 countries. Lancet. 2018;391(10125):1023–75.

    Article  Google Scholar 

Download references


We thank Enrique Loyola, Ivo Rakovic, Michael van den Berg, Zhamin Yelgezekova, the Ministry of Health of Kazakhstan, technical reviewers, staff of the WHO European Centre for Primary Health Care, WHO European network of Integrated Health Services Delivery Focal Points, members of the Primary Health Care Advisory Group.


This work was carried out with the financial support of the Government of Kazakhstan through the WHO European Centre for Primary Health Care in Almaty, Kazakhstan and the Marie Skłodowska-Curie Innovative Training Network (HealthPros – Healthcare Performance Intelligence Professionals) that has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement Nr. 765141. The content reflects only the authors’ views as the funding bodies were not involved in the design of the study, the collection, analysis and interpretation of data and in writing the manuscript.

Author information

Authors and Affiliations



(I) Conception and design: EB, DK, NSK, JET [48]; Collection and assembly of data: EB, DK, IK; (III) Data analysis and interpretation: all authors; (IV) Manuscript writing: all authors; (V) Final approval of manuscript: all authors.

Corresponding author

Correspondence to Erica Barbazza.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Literature reviewed

Additional file 2.

Tracer conditions

Additional file 3.

Indicator passports

Additional file 4.

Glossary of terms

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Barbazza, E., Kringos, D., Kruse, I. et al. Creating performance intelligence for primary health care strengthening in Europe. BMC Health Serv Res 19, 1006 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: