Skip to main content
  • Research article
  • Open access
  • Published:

Development and pilot of an internationally standardized measure of cardiovascular risk management in European primary care



Primary care can play an important role in providing cardiovascular risk management in patients with established Cardiovascular Diseases (CVD), patients with a known high risk of developing CVD, and potentially for individuals with a low risk of developing CVD, but who have unhealthy lifestyles. To describe and compare cardiovascular risk management, internationally valid quality indicators and standardized measures are needed. As part of a large project in 9 European countries (EPA-Cardio), we have developed and tested a set of standardized measures, linked to previously developed quality indicators.


A structured stepwise procedure was followed to develop measures. First, the research team allocated 106 validated quality indicators to one of the three target populations (established CVD, at high risk, at low risk) and to different data-collection methods (data abstraction from the medical records, a patient survey, an interview with lead practice GP/a practice survey). Secondly, we selected a number of other validated measures to enrich the assessment. A pilot study was performed to test the feasibility. Finally, we revised the measures based on the findings.


The EPA-Cardio measures consisted of abstraction forms from the medical-records data of established Coronary Heart Disease (CHD)-patients - and high-risk groups, a patient questionnaire for each of the 3 groups, an interview questionnaire for the lead GP and a questionnaire for practice teams. The measures were feasible and accepted by general practices from different countries.


An internationally standardized measure of cardiovascular risk management, linked to validated quality indicators and tested for feasibility in general practice, is now available. Careful development and pilot testing of the measures are crucial in international studies of quality of healthcare.

Peer Review reports


Cardiovascular disease (CVD) is a major cause of premature death in Europe and also an important cause of morbidity, contributing substantially to escalating healthcare costs [1]. Cardiovascular risk management (CVRM) is targeted at three broad populations: those with established and diagnosed disease (e.g. angina, stroke), those at high risk of developing CVD due to known risk factors (e.g. hypertension, smoking), and individuals at low risk for developing CVD, but with unhealthy lifestyles. CVRM involves risk assessment and communicating to patients the risk of developing CVD [2, 3], providing relevant guideline based treatments, such as counselling on lifestyle as well as cardio-protective medication and follow-up where appropriate [4, 5]. Primary care has an important role in the delivery of CVRM and quality improvement programs have been developed to further strengthen this role [6].

Previous research has emphasised the importance of addressing global CVD risk assessment rather than focusing on individual risk factors such as smoking [5]. Various global CVD risk assessment and communication instruments for use in primary care have been developed across Europe [712], but there are differential and inconsistent use in routine clinical practice between and within countries [1315]. The EUROASPIRE studies have advocated the need for more effective lifestyle management of patients at high risk and medication management of patients with coronary heart disease (CHD) [16]. These findings were derived from patients receiving treatment from medical specialists, whereas most patients received treatment in primary care, particularly in countries with a strong primary care system [17, 18].

Quality indicators (QI) are used widely across Europe and internationally as part of quality improvement and pay-for-performance schemes [1921], but research suggests that not all patients received the necessary evidence-based care that underpin these indicators [22, 23].

We started a large project (EPA-Cardio) to assess CVRM in primary care across Europe. The participating countries were recruited from the TOPAS Europe Association, which is a collaboration between researchers of quality improvement in healthcare, founded in January 2005 In the first stage of the EPA-Cardio-project, we developed quality indicators for the prevention and management of cardiovascular disease in primary care [24]. Careful development and testing of measures linked to these quality indicators were deemed crucial for valid international comparisons. This paper describes the methods used to develop the set of measures, the results of a pilot in different countries of these measures, and the resulting final standardized EPA-Cardio instrument.


The development of the data collection instruments and piloting took place between March 2007 and January 2008. A core group of researchers (SC, MW, JvL, and SL) developed the prototypes of the instruments, and discussed and refined it in meetings with the project partners. Pilot tests were done with GPs volunteering to participate.

Development of the EPA-Cardio instrument

The EPA-Cardio instrument was developed by the project partners of the EPA-Cardio project, from 9 European countries, Austria, Belgium, Finland, France, Germany, Slovenia, Switzerland, the Netherlands and the UK. The measures were linked to the previously selected quality indicators (QI), using a modified Delphi Technique with expert panels in each participating country by a team, which was part of the first project [22, 23]. The expert panels rated a core set of 44 indicators valid across 4 conceptual quality domains (lifestyle, clinical processes and outcomes, organisation and patient perspective). Furthermore, a broader set of additional 62 indicators had been rated "restricted valid" with a lower level of agreement. We decided to include these indicators in our study, because the core set mainly reflected the secondary prevention in patients with diabetes or established CVD, neglecting the primary prevention and patient perspective, which were represented in the wider set (broad set). (Table 1)

Table 1 Allocation of the quality indicators (QI) - Data collection methods and patient groups

The core group of researchers reviewed each of the 106 indicators and made 2 decisions per indicator (Figure 1): First, to which patient group was the indicator relevant; and second to the recommended data collection instruments. Indicators that were not applicable to a certain patient group, representing rather organisational aspects in general, were allocated to "general organisation" (table 1). As some quality of care-domains, such as patient perspective, and the "primary-prevention group" were under-represented in the quality indicator set, a number of additional measures (not linked to the quality indicators) were selected for subsequent research. The instruments were developed and agreed in a series of meetings between March and August 2007. All project partners discussed their disagreements, which led to several revisions of the instruments. The resulting EPA-Cardio instrument was developed in English and translated into relevant languages using forward and backward procedures [25].

Figure 1
figure 1

The development of the EPA-Cardio instrument - a multi-stage process.

Patient populations (table 2)

Table 2 Inclusion- and exclusion criteria for different patient groups

Patients with established/diagnosed CHD

Some indicators related to CVD overall, and others in particularly to coronary heart disease (CHD). To facilitate data collection and homogenise our study group, we decided to focus on the patient group to whom all these indicators apply, i.e. patients with CHD. We excluded patients with diabetes because our primary focus was on patients with CVD.

Patients 'at high risk'

While we used widely accepted risk-calculation models to define high risk, for pragmatic reasons we defined 'high risk' for the EPA-Cardio pilot as a 'male smoker over 60 years of age. For these individuals (without other risk factors), the risk of fatal or nonfatal CVD events within 10 years was estimated at 14.4%, using a German electronic risk calculation instrument [10], meeting the definition-criteria of high risk. We expected to facilitate patient sampling by focusing on one risk factor other than age and gender.

Adults with unhealthy life styles

Only a few indicators related to this population. In EPA-Cardio, we focused on all individuals between 18 and 45 years. As risk and risk factors for CVD increases with age, we used this age limit with the intention of separating this group from patients at high risk of CVD.

Patient samples

Data collection procedures differed between the participating countries, due to ethical and legal regulations, and the number of available practice team members. The research team provided guidance for patient sampling and data collection as part of the EPA-Cardio instrument.

Established CHD and 'at high risk' groups

In Germany, Slovenia, Switzerland, and the Netherlands, patient selection and data extraction of the medical record audit was conducted by a practice nurse (guided by the GP). Support by telephone and email was provided by a researcher in the relevant country. In Belgium, the GP took over this task, and, in the UK, a researcher visited the practice to conduct the searches and the audit in collaboration with practice staff.

Where there identified more than 30 patients in a practice, we used random numbers to select patients for inclusion. We included all identified patients where there were less than 30 patients. We used consecutive sampling if the practice conducted manual searches. All patients selected for the medical audit (established CHD and 'at high risk') per practice were included in the patient survey (maximum of 30 patients per group per practice). As in most countries (except the UK), informed consent was requested for auditing medical records, questionnaires with informed consent sheets were sent after selecting eligible patients, and data were abstracted only for those who had sent back written consents. We intended to include at least 15 patients of each group per practice.

Primary prevention group (aged 18-45)

It was planned to select the primary prevention group by electronic searches (considering age group and exclusion criteria) followed by random sampling of 40 individuals from the search list. Where an electronic search was not feasible, the practice approached regular visitors who met the inclusion criteria consecutively. As we aimed to include 15 individuals per practice, we planned to send 40 questionnaires per practice in cases of electronic selection, or to hand out 15 questionnaires to practice visitors ready to participate in the study.


We allocated each indicator to the most appropriate instrument: data abstraction forms for the medical record for patients with CHD and patients at high risk respectively, patient questionnaires for all 3 groups as appropriate, an interview guide for the lead GP, and a questionnaire for practice members. The designing of the abstraction forms for the medical record followed that of Banks [26], in order to facilitate data encoding and analysis.

Questionnaire for patient survey

In all cases, selected patients were sent a questionnaire and letter explaining the aims of the study along with an informed consent sheet, a questionnaire (relevant to the appropriate group), and a self-addressed prepaid envelope to return the questionnaire to the research group and the written consent sheet to the practice team. The questionnaires included questions derived from the quality indicators, questions about sociodemographic issues and other conditions. Validated instruments to evaluate the quality of primary care from the patient perspective were also included for all 3 patient groups [27]. Other validated instruments were included to evaluate the quality of chronic care for patients with CHD [2830] and instruments to assess lifestyle behaviour for patients at high risk and the primary prevention-group) [3134].

Physician interview

A researcher conducted the interview with the lead GP or practice manager, either by telephone or face to face and recorded the answers in the interview questionnaire. The interview questionnaire contained questions representing indicators on organisational issues and risk assessment methods. In addition, practice engagement in cardiovascular quality improvement projects and public health projects on cardiovascular health were assessed, and also GP's views and activities on primary prevention.

Practice questionnaire

Practice members completed a practice questionnaire sent by the researchers. This questionnaire contained questions to identify the practice according to size, location, number and function of practice staff and number of listed or yearly attending patients. Other items were derived from the EPA practice management [35] and represented the EPA dimensions "information process and technology", "organisation of chronic care and prevention", and "quality improvement". These items were included to obtain an insight into the quality management activities of included practices considered to have an influence on study results.

Pilot study

The aim of the pilot study was to assess the acceptability of the developed data collection instruments and feasibility of the methods, irrespective of the difference of nationally adapted methods. We conducted the pilot study at the end of 2007 in 2 practices in 6 of the participating countries Belgium, Germany, the Netherlands, Slovenia, Switzerland and the United Kingdom. Not all countries had the materials translated and ready in time for the pilot study. We focused only on patients with CHD and patients at high risk in the pilot, because most data-collection instruments and procedures were related to these groups and the primary-prevention group was easier to select. Researchers chose the practices out of national GP-networks, approaching them by phone or mail. In each of the participating countries, pilot practices received between 100€ and 400€ as an incentive to take part.

All practices received a package with the study materials prior to the researcher visit (UK), or conducting the data collection in house (all other countries). This consisted of instruction sheets, the flow chart (Figure 2), questionnaires for patients, patient information, informed consent sheets and questionnaires for the practice teams.

Figure 2
figure 2

EPA-Cardio pilot study - flow chart.

Data collection procedures included patient selection, sending of patient questionnaires, data abstraction from the medical records, interview with the lead GP and completing of the practice questionnaire.

At the end of the pilot study, researchers interviewed the lead nurse by phone (face-to-face in the UK at the time of the visit), using an interview guide, and asked questions about the acceptability and feasibility of the instruments as well as the data-collection procedures. The pilot results were discussed at a meeting of all EPA-Cardio country co-ordinators, common standards were agreed, as also national adaptations to revise the instruments and procedures for subsequent use in the project. Ethical approval was sought as appropriate in the different participating countries.


The instrument was piloted in 12 European general practices, 2 general practices per country, in Belgium, Germany, the Netherlands, Slovenia, Switzerland and the UK. These practices selected a total of 268 patients with established CHD and 127 patients at high risk.

The practice teams rated study materials and instructions as easily understandable. Despite differences in data-collection procedures between the countries, all were capable of completing the assessment. It was possible to complete all procedures within 4-8 weeks. In the UK, an external reviewer completed the assessment within one working day per practice (without waiting for the replies from the patient's questionnaires).

Data-collection procedures

Selection of patient groups, data abstraction

The group of patients with CHD was easier to select than the group of patients at high risk, because smoking status was not well documented in the medical records, and small practices could not select enough patients in this group. Therefore, it was possible to identify 30 patients with CHD in most practices, whereas the number of patients selected at high risk was between 8 and 30 (in larger practices with patient lists of >10000 patients).

Patient Questionnaires

The patients had some difficulties in completing the questionnaires, especially the PACIC-instrument, often asking nurses for explanations. Patient lists were used as check lists to follow the audit procedures, control the questionnaire responses, and send a reminder if necessary. Most of the countries sent reminders by mail. In Germany, the practice nurses reminded patients by telephone. Sending reminders was not permissible in the UK. The response rates were between 50% (UK) and 100% (Slovenia).

Interview with the lead GP

The researcher carried out the interview with the lead GP either in a face-to-face meeting (UK) or, in most countries, by telephone. The interview lasted 15-20 min.

Questionnaire for the practice team

Practice teams had no problems with completing the questionnaire for practice members. They needed 20-30 minutes to answer all questions.

Factors that contributed to the success or failure of data collection

There was a good level of practice co-operation with the research team of each country. Practices considered the instructions for patient selection and data-collection procedures as helpful, as well as the possibility to contact the research team if problems arose with study procedures. The practices reported that the selection of the eligible study patients from the medical records raised the major problem and consumed the most time of the audit. In most countries, nurses needed the help of the GP because they were not familiar with ICD or ICPC codes and searches run with the practice software.

The final EPA-Cardio instrument

Finally, researchers of 9 European countries developed and agreed the final EPA-Cardio instrument (table 3). The instrument consisted of 4 types of paper-based instruments: abstraction forms to collect the data from the medical records, an interview questionnaire for the lead GP, patient questionnaires for 3 different patient groups, and a questionnaire for practice teams. The instrument included 74 quality indicators (37 core set and 37 broad set) that were previously agreed in the Delphi procedures. The main revisions in comparison to the pilot instruments were changes to the definition of the high-risk group to facilitate patient selection. We finally defined this group by risk calculation with recommended tools according to national guidelines, e.g. 10% fatal CVD-risk as calculated by the Dutch risk tables or by using a proxy measure: Patients with three out of the following four risk factors hypertension, hypercholesterolaemia, smoking and men over 60 years. The final EPA-Cardio instrument included a questionnaire for the primary-prevention group additionally. We also decided to apply to all the countries the PACIC-instrument that had been used in only 4 countries (Austria, Germany, the Netherlands and Switzerland) during the pilot. The changes to the instruments had to be translated again.

Table 3 The final EPA-Cardio instrument


This study provided an internationally standardized measure of cardiovascular risk management in primary care, linked to previously selected quality indicators and pilot tested for feasibility.

Strengths and limitations

The EPA-Cardio project was conducted by the same team that developed the EPA-practice management accreditation tool (European Practice Assessment). Lessons learned in the process were applied to EPA-Cardio [35, 36]. The basis of the EPA-Cardio instrument were quality indicators developed using a validated Delphi procedure. However, this methodology is did not cover all aspects of cardiovascular prevention and care as described and recommended in guidelines [20]. The assessment of important activities of primary care, such as communication, empathy, teamwork, consultation time, counselling and continuity of care, require other methodologies, such as in-depth interviews e.g. [37, 38]. To complete the assessment of cardiovascular prevention and management, we supplemented the EPA-Cardio instrument with patient evaluation instruments. We therefore chose the EUROPEP-instrument because it is validated, used in many surveys and exists in the languages of all participants [27]. As patients' perspectives contribute considerably to the improvement in the quality of care [39], we also included the PACIC instrument to reflect the view of patients with established CHD [28]. This project aims to provide better cover of primary prevention, and we therefore included validated lifestyle-assessment instruments [31] and self-developed questions on attitudes towards primary prevention for individuals and GPs.

We included a convenience sample of countries out of the TOPAS Europe Association in the study for feasibility reasons and good experiences with collaboration in past projects [35, 36]. This begs the question of whether EPA-Cardio be different if, for example, Greece and Denmark and Portugal had taken part in the study.

In the pilot, we used a small convenience sample of 2 practices in 6 of the 9 participating countries, as some countries were not able to have the study materials ready in time. The different methods used to select patient groups may lead to different populations (with limited generalisability) and different data-collection procedures, and therefore may restrict the comparability. The results of the subsequent observational study should therefore be interpreted carefully.

Methods of data collection in general practices

The systematic collection of healthcare data across different countries in primary practice is difficult because of differences in levels of computerisation and different practice software within and between European countries [40]. The use of computers to record clinical data, and also the content and quality of electronic patient records, varies significantly [41, 42]. In order not to exclude countries or practices with lower levels of computerisation, it is important to apply methodologies that prevent research only from being conducted selectively in technologically highly developed practices [22, 43]. To make data collection feasible for practices with different technological levels in all countries, we developed paper-based abstraction tools. However, electronic data abstraction in general practice reduces the associated workload and is a pre-condition for the implementation of quality assessment in routine care [44].

The challenge of cross-national assessment

The aim of the study was to develop standardized data collection procedures in all countries. However, it is also necessary to adapt to differences in national procedures to conduct cross-country collaborations. Perfect standardisation of data collection methods in cross-national studies is not possible due to differences in legislation and practice resources. For example, whether national data protection acts allow anyone not employed by the practice to look at patient medical records [45]. Another difficulty is the different standards of the ethics committees between, and even within, countries [46, 47]; for example, in some countries practice staff are allowed to collect patient data for quality improvement purposes without the permission of patients (e.g. UK, the Netherlands), whereas in others (e.g. Germany, Belgium) the informed consent of the patient is required.

The future of the EPA-Cardio instrument

We used the final EPA-Cardio instrument in the EPA-Cardio project within 9 participating countries between May 2008 and August 2009 [23]. The results of this large observational study may be used to compare countries with different quality improvement strategies, such as disease management programmes in Germany [48], indicator-based incentive contracts in the UK [49], or practice-based support with outreach visits in the Netherlands [50]. This may identify best practice examples and derive quality improvement recommendations for the prevention and management of cardiovascular diseases in Europe.


An internationally standardized measure of CVRM in primary care is now available, linked to previously selected quality indicators. It has been pilot tested for feasibility. Careful development and pilot testing were deemed crucial for international comparisons. Detailed assessment, analysis and feedback can be the starting point for developing quality improvement activities and for deriving recommendations at a national and pan-European level.


  1. Leal J, Luengo-Fernandez R, Gray A, Petersen S, Rayner M: Economic burden of cardiovascular diseases in the enlarged European Union. Eur Heart J. 2006, 27 (13): 1610-1619. 10.1093/eurheartj/ehi733.

    Article  PubMed  Google Scholar 

  2. Sheridan SL, Viera AJ, Krantz MJ, Ice CL, Steinman LE, Peters KE, et al: The effect of giving global coronary risk information to adults: a systematic review. Arch Intern Med. 2010, 170 (3): 230-239. 10.1001/archinternmed.2009.516.

    Article  PubMed  Google Scholar 

  3. Waldron CA, van der WT, Ludt S, Gallacher J, Elwyn G: What are effective strategies to communicate cardiovascular risk information to patients? A systematic review. Patient Educ Couns. 2010.

    Google Scholar 

  4. Graham I, Atar D, Borch-Johnsen K, Boysen G, Burell G, Cifkova R, et al: European guidelines on cardiovascular disease prevention in clinical practice. Fourth Joint Task Force of the European Society of Cardiology and other Societies on Cardiovascular Disease Prevention in Clinical Practice. Eur J Cardiovasc Prev Rehabil. 2007, 14 (S2): S1-S113. 10.1097/01.hjr.0000277983.23934.c9.

    Article  PubMed  Google Scholar 

  5. Ferket BS, Colkesen EB, Visser JJ, Spronk S, Kraaijenhagen RA, Steyerberg EW, et al: Systematic review of guidelines on cardiovascular risk assessment: Which recommendations should clinicians follow for a cardiovascular health check?. Arch Intern Med. 2010, 170 (1): 27-40. 10.1001/archinternmed.2009.434.

    Article  PubMed  Google Scholar 

  6. Van Lieshout J, Wensing M, Campbell SM, Grol R: Primary care strength linked to prevention programs for cardiovascular disease. Am J Manag Care. 2009, 15 (4): 255-262.

    PubMed  Google Scholar 

  7. D'Agostino RB, Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM, et al: General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study. Circulation. 2008, 117 (6): 743-753.

    Article  PubMed  Google Scholar 

  8. Conroy RM, Pyorala K, Fitzgerald AP, Sans S, Menotti A, De Backer G, et al: Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J. 2003, 24 (11): 987-1003. 10.1016/S0195-668X(03)00114-3.

    Article  CAS  PubMed  Google Scholar 

  9. Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, May M, Brindle P: Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ. 2007, 335 (7611): 136-10.1136/bmj.39261.471806.55.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Krones T, Keller H, Sonnichsen A, Sadowski EM, Baum E, Wegscheider K, et al: Absolute Cardiovascular Disease Risk and Shared Decision Making in Primary Care: A Randomized Controlled Trial. Ann Fam Med. 2008, 6 (3): 218-227. 10.1370/afm.854.

    Article  PubMed  PubMed Central  Google Scholar 

  11. van Steenkiste B, van der WT, Stoffers HE, Kester AD, Timmermans DR, Grol R: Improving cardiovascular risk management: a randomized, controlled trial on the effect of a decision support tool for patients and physicians. Eur J Cardiovasc Prev Rehabil. 2007, 14 (1): 44-50. 10.1097/01.hjr.0000239475.71805.1e.

    Article  PubMed  Google Scholar 

  12. Bonnevie L, Thomsen T, Jorgensen T: The use of computerized decision support systems in preventive cardiology--principal results from the national PRECARD survey in Denmark. Eur J Cardiovasc Prev Rehabil. 2005, 12 (1): 52-55. 10.1097/00149831-200502000-00008.

    PubMed  Google Scholar 

  13. Getz L, Kirkengen AL, Hetlevik I, Romundstad S, Sigurdsson JA: Ethical dilemmas arising from implementation of the European guidelines on cardiovascular disease prevention in clinical practice. Scandinavian Journal of Primary Health Care. 2004, 22 (4): 202-208. 10.1080/02813430410006693.

    Article  PubMed  Google Scholar 

  14. Mosca L, Linfante AH, Benjamin EJ, Berra K, Hayes SN, Walsh BW, et al: National Study of Physician Awareness and Adherence to Cardiovascular Disease Prevention Guidelines. Circulation. 2005, 111 (4): 499-510. 10.1161/01.CIR.0000154568.43333.82.

    Article  PubMed  Google Scholar 

  15. Chauhan U: Cardiovascular disease prevention in primary care. Br Med Bull. 2007, ldm002

    Google Scholar 

  16. Kotseva K, Wood D, De Backer G, De Bacquer D, Pyorala K, Keil U: Cardiovascular prevention guidelines in daily practice: a comparison of EUROASPIRE I, II, and III surveys in eight European countries. The Lancet. 2009, 373 (9667): 929-940. 10.1016/S0140-6736(09)60330-5.

    Article  Google Scholar 

  17. WHO: World Health Report 2008. 2008, WHO, Geneva

    Google Scholar 

  18. Starfield B, Shi L, Macinko J: Contribution of primary care to health systems and health. Milbank Q. 2005, 83 (3): 457-502. 10.1111/j.1468-0009.2005.00409.x.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Burge FI, Bower K, Putnam W, Cox JL: Quality indicators for cardiovascular primary care. Can J Cardiol. 2007, 23 (5): 383-388.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Perera R, Dowell T, Crampton P, Kearns R: Panning for gold: an evidence-based tool for assessment of performance indicators in primary health care. Health Policy. 2007, 80 (2): 314-327. 10.1016/j.healthpol.2006.03.011.

    Article  PubMed  Google Scholar 

  21. Campbell SM, Reeves D, Kontopantelis E, Sibbald B, Roland M: Effects of pay for performance on the quality of primary care in England. N Engl J Med. 2009, 361 (4): 368-378. 10.1056/NEJMsa0807651.

    Article  CAS  PubMed  Google Scholar 

  22. Kirk SA, Campbell SM, Kennell-Webb S, Reeves D, Roland MO, Marshall MN: Assessing the quality of care of multiple conditions in general practice: practical and methodological problems. Qual Saf Health Care. 2003, 12 (6): 421-427. 10.1136/qhc.12.6.421.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, et al: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348 (26): 2635-2645. 10.1056/NEJMsa022615.

    Article  PubMed  Google Scholar 

  24. Campbell SM, Ludt S, Van Lieshout J, Boffin N, Wensing M, Petek D, et al: Quality indicators for the prevention and management of cardiovascular disease in primary care in nine European countries. Eur J Cardiovasc Prev Rehabil. 2008, 15 (5): 509-515. 10.1097/HJR.0b013e328302f44d.

    Article  PubMed  Google Scholar 

  25. Guillemin F, Bombardier C, Beaton D: Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993, 46 (12): 1417-1432. 10.1016/0895-4356(93)90142-N.

    Article  CAS  PubMed  Google Scholar 

  26. Banks NJ: Designing medical record abstraction forms. Int J Qual Health Care. 1998, 10 (2): 163-167. 10.1093/intqhc/10.2.163.

    Article  CAS  PubMed  Google Scholar 

  27. Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, et al: Patients in Europe evaluate general practice care: an international comparison. Br J Gen Pract. 2000, 50 (460): 882-887.

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM: Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Med Care. 2005, 43 (5): 436-444. 10.1097/01.mlr.0000160375.47920.8c.

    Article  PubMed  Google Scholar 

  29. Glasgow RE, Emont S, Miller DC: Assessing delivery of the five 'As' for patient-centered counseling. Health Promot Int. 2006, 21 (3): 245-255. 10.1093/heapro/dal017.

    Article  PubMed  Google Scholar 

  30. Morisky DE, Green LW, Levine DM: Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care. 1986, 24 (1): 67-74. 10.1097/00005650-198601000-00007.

    Article  CAS  PubMed  Google Scholar 

  31. Glasgow RE, Ory MG, Klesges LM, Cifuentes M, Fernald DH, Green LA: Practical and relevant self-report measures of patient health behaviors for primary care research. Ann Fam Med. 2005, 3 (1): 73-81. 10.1370/afm.261.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Topolski TD, LoGerfo J, Patrick DL, Williams B, Walwick J, Patrick MB: The Rapid Assessment of Physical Activity (RAPA) among older adults. Prev Chronic Dis. 2006, 3 (4): A118..

    PubMed  PubMed Central  Google Scholar 

  33. Segal-Isaacson CJ, Wylie-Rosett J, Gans KM: Validation of a Short Dietary Assessment Questionnaire: The Rapid Eating and Activity Assessment for Participants Short Version (REAP-S). The Diabetes Educator. 2004, 30 (5): 774-781. 10.1177/014572170403000512.

    Article  CAS  PubMed  Google Scholar 

  34. Yaroch AL, Nebeling L, Thompson FE, Hurley TG, Hebert JR, Toobert DJ, et al: Baseline Design Elements and Sample Characteristics for Seven Sites Participating in the Nutrition Working Group of the Behavior Change Consortium. J Nutr. 2008, 138 (1): 185S-192.

    CAS  PubMed  Google Scholar 

  35. Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, et al: Testing a European set of indicators for the evaluation of the management of primary care practices. Fam Pract. 2006, 23 (1): 137-147. 10.1093/fampra/cmi091.

    Article  PubMed  Google Scholar 

  36. Engels Y, Campbell S, Dautzenberg M, Van den HP, Brinkmann H, Szecsenyi J, et al: Developing a framework of, and quality indicators for, general practice management in Europe. Fam Pract. 2005, 22 (2): 215-222. 10.1093/fampra/cmi002.

    Article  PubMed  Google Scholar 

  37. Campbell SM, Hann M, Hacker J, Burns C, Oliver D, Thapar A, et al: Identifying predictors of high quality care in English general practice: observational study. BMJ. 2001, 323 (7316): 784-10.1136/bmj.323.7316.784.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Carpiano RM, Flocke SA, Frank SH, Stange KC: Tools, teamwork, and tenacity: an examination of family practice office system influences on preventive service delivery. Prev Med. 2003, 36 (2): 131-140. 10.1016/S0091-7435(02)00024-5.

    Article  PubMed  Google Scholar 

  39. Rao JK, Weinberger M, Kroenke K: Visit-specific expectations and patient-centered outcomes: a literature review. Arch Fam Med. 2000, 9 (10): 1148-1155. 10.1001/archfami.9.10.1148.

    Article  CAS  PubMed  Google Scholar 

  40. Green LA, Fryer GE, Froom P, Culpepper L, Froom J: Opportunities, challenges, and lessons of international research in practice-based research networks: the case of an international study of acute otitis media. Ann Fam Med. 2004, 2 (5): 429-433. 10.1370/afm.224.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Thiru K, Hassey A, Sullivan F: Systematic review of scope and quality of electronic patient record data in primary care. BMJ. 2003, 326 (7398): 1070-10.1136/bmj.326.7398.1070.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Heidenreich R, Himmel W, Böckmann H, et al: Documentation of electronic patient records in German general practices: a telephone survey [article in German]. German Journal for Quality in Health Care. 2005, 99 (9): 573-580.

    Google Scholar 

  43. Chan SC, Lee TW, Teoh LC, Abdullah ZC, Xavier G, Sim CK, et al: Audit on cardiovascular disease preventive care in general practice. Singapore Med J. 2008, 49 (4): 311-315.

    CAS  PubMed  Google Scholar 

  44. de Lusignan S: An educational intervention, involving feedback of routinely collected computer data, to improve cardiovascular disease management in UK primary care. Methods Inf Med. 2007, 46 (1): 57-62.

    CAS  PubMed  Google Scholar 

  45. Iversen A, Liddell K, Fear N, Hotopf M, Wessely S: Consent, confidentiality, and the Data Protection Act. BMJ. 2006, 332 (7534): 165-169. 10.1136/bmj.332.7534.165.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Hearnshaw H: Comparison of requirements of research ethics committees in 11 European countries for a non-invasive interventional study. BMJ. 2004, 328 (7432): 140-141. 10.1136/bmj.328.7432.140.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Byrne G, Morgan J, Kendall S, Offredy M: A critical account of obtaining research governance approval. Nurse Res. 2005, 13 (2): 7-18.

    Article  PubMed  Google Scholar 

  48. Miksch A, Laux G, Ose D, Joos S, Campbell S, Riens B, et al: Is there a survival benefit within a German primary care-based disease management program?. Am J Manag Care. 2010, 16 (1): 49-54.

    PubMed  Google Scholar 

  49. Ashworth M, Jones RH: Pay for performance systems in general practice: experience in the United Kingdom. Med J Aust. 2008, 189 (2): 60-61.

    PubMed  Google Scholar 

  50. Hulscher ME, van Drenth BB, van der Wouden JC, Mokkink HG, van Weel C, Grol RP: Changing preventive practice: a controlled trial on the effects of outreach visits to organise prevention of cardiovascular disease. Qual Health Care. 1997, 6 (1): 19-24. 10.1136/qshc.6.1.19.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Rabin R, de Charro F: EQ-5D: a measure of health status from the EuroQol Group. Ann Med. 2001, 33 (5): 337-343. 10.3109/07853890109002087.

    Article  CAS  PubMed  Google Scholar 

Pre-publication history

Download references


The Bertelsmann Foundation (Gütersloh, Germany) initiated and co-ordinated the EPA-Cardio project, in collaboration with IQ Healthcare of the Radboud University Nijmegen (Nijmegen, The Netherlands). The Bertelsmann Foundation also supported the project financially (without restrictions on the scientific work), and each of the project participants invested additional resources in the project. The following individuals were part of the EPA-Cardio group, in addition to the authors of this study protocol: Ester Cornelis (Belgium), Eva Frigola (Spain), Glyn Elwyn (UK), Reinhold Glehr (Austria), Margalit Goldfracht (Israel), Hector Falcoff (France), Lilian Michlig (Switzerland), Esko Kumpusalo (Finland), Janko Kersnik (Slovenia), Kati Kettunen (Finland), Beat Künzi (Switzerland), Solene Martin (France), Davorina Petek (Slovenia), Ingrid Pichler (Austria), Martin Roland (UK), Marianne Samuelson (France), Veerle van der Stighelen (Belgium), Trudy van der Weijden (Netherlands).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Sabine Ludt.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

MW developed the overall outline of the EPA-Cardio project. SC co-ordinated the international selection of performance indicators, on which the EPA-Cardio audit instrument is based. SL co-ordinated the development and pilot testing of the measures and wrote the draft and the final version of this paper. JvL contributed to the development and selection of measurements. RG was the project leader of EPA-Cardio and JS was supervisor of the research reported in this paper. All authors critically assessed and approved this paper.

Sabine Ludt, Stephen M Campbell, Jan van Lieshout contributed equally to this work.

Authors’ original submitted files for images

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Ludt, S., Campbell, S.M., van Lieshout, J. et al. Development and pilot of an internationally standardized measure of cardiovascular risk management in European primary care. BMC Health Serv Res 11, 70 (2011).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: